Roland> https://fedorahosted.org/elfutils/wiki/DwarfLint
Roland> Please follow up on elfutils-devel@lists.fedorahosted.org about Roland> details.
Roland> If you have any more suggestions for checks, please post them to Roland> elfutils-devel and we'll get them onto the TODO list.
Roland> Especially, if you have come across bad data emitted by GCC that is not Roland> listed on SuspiciousDebuginfoCases, then let us know--and no less so if Roland> GCC has already been fixed.
Here, I recommend simply searching for "GCC" in src/gdb/dwarf2read.c. This will yield a number of PRs (some open, some closed) and other oddities.
Searching through the "debug" bugs in GCC bugzilla would also yield some stuff.
Some other things:
check_debug_aranges
We saw cases in the wild where an arange entry was terminated early. That is, the 0,0 entry occurred before the end of the particular arange buffer. There is a bug, maybe closed, in Red Hat bugzilla for this.
Consider checking that every CU has an arange entry. This is not strictly required, but it is nicer for consumers. ISTR filing a bug about this in GCC bugzilla.
check_debug_pubnames / pubtypes
FWIW, GCC doesn't emit these any more. So, IMO anyway, they are not very important to check. They were never all that usable in the first place.
But, if you want to check these, then check that they are complete and correct.
In the past we noticed that names were not always fully qualified (I forget if it was namespace- or class- qualification that GCC didn't do properly). I'm sure I filed a GCC bug for this.
Tom
Here, I recommend simply searching for "GCC" in src/gdb/dwarf2read.c. This will yield a number of PRs (some open, some closed) and other oddities.
Searching through the "debug" bugs in GCC bugzilla would also yield some stuff.
Ok. Petr can do that in some spare time. But please do make us aware explicitly of anything that you come across.
check_debug_aranges
We saw cases in the wild where an arange entry was terminated early. That is, the 0,0 entry occurred before the end of the particular arange buffer. There is a bug, maybe closed, in Red Hat bugzilla for this.
Consider checking that every CU has an arange entry. This is not strictly required, but it is nicer for consumers. ISTR filing a bug about this in GCC bugzilla.
I think this comes under what we call "connectivity" checks, and might be done already. .debug_aranges should be deriveable from scratch from the DIE trees (and our writer/compressor will always do that and ignore the original contents). So I think we already intend to compare the data completely to what we would build from scratch. Petr should know if any parts of that checking are still missing.
check_debug_pubnames / pubtypes
FWIW, GCC doesn't emit these any more. So, IMO anyway, they are not very important to check. They were never all that usable in the first place.
But, if you want to check these, then check that they are complete and correct.
We have multiple levels of checks. For these, we may only ever bother with the low-level checks, i.e. that the section formats themselves are valid and that their DIE pointers are valid pointers. The high-level "correct" predicate is not a DWARF thing, but an issue of language-specific semantics. So far, all of our stuff is only on pure language-agnostic and producer-blind DWARF specification requirements.
In the past we noticed that names were not always fully qualified (I forget if it was namespace- or class- qualification that GCC didn't do properly). I'm sure I filed a GCC bug for this.
We are far off from having anything like the language-specific knowledge modules we'd need to detect that.
Thanks, Roland
11.09.2010 00:54, Roland McGrath wrote:
check_debug_aranges
We saw cases in the wild where an arange entry was terminated early. That is, the 0,0 entry occurred before the end of the particular arange buffer. There is a bug, maybe closed, in Red Hat bugzilla for this.
That would get a citation for unnecessary padding (with zero or non-zero bytes, depending on where it is) and probably also for addresses covered with CUs, but not with aranges. I've added a test case for that now, just to be sure.
Consider checking that every CU has an arange entry. This is not strictly required, but it is nicer for consumers. ISTR filing a bug about this in GCC bugzilla.
I think this comes under what we call "connectivity" checks, and might be done already. .debug_aranges should be deriveable from scratch from the
It wasn't but is now. It emits a warning in --strict.
DIE trees (and our writer/compressor will always do that and ignore the original contents). So I think we already intend to compare the data completely to what we would build from scratch. Petr should know if any parts of that checking are still missing.
Up until now we checked that aranges cover everything that's claimed by CUs. Now I've added the reciprocal check, that looks whether CU claims match what aranges cover.
PM
We saw cases in the wild where an arange entry was terminated early. That is, the 0,0 entry occurred before the end of the particular arange buffer. There is a bug, maybe closed, in Red Hat bugzilla for this.
That would get a citation for unnecessary padding (with zero or non-zero bytes, depending on where it is) and probably also for addresses covered with CUs, but not with aranges. I've added a test case for that now, just to be sure.
Yes, I thought those would be caught already. Good that they are. For paranoia, it would be good to dig up a real-world case like Tom saw and verify that it's caught and diagnosed correctly. But if Tom can't find the case easily and you already have a synthetic test case, that's fine.
Consider checking that every CU has an arange entry. This is not strictly required, but it is nicer for consumers. ISTR filing a bug about this in GCC bugzilla.
I think this comes under what we call "connectivity" checks, and might be done already. .debug_aranges should be deriveable from scratch from the
It wasn't but is now. It emits a warning in --strict.
Every CU that has any PCs (i.e. compile_unit has low_pc, high_pc, or ranges attributes at all), that is. In today's GCC, I'm not sure there is ever a CU with no PCs at all, except for a type_unit (-gdwarf-4). With compression, we will certainly generate partial_unit entries with no PCs eventually.
This should be a default warning, because libdw and things using it will fail completely to notice the CU for by-address lookups if it has no .debug_aranges entry.
DIE trees (and our writer/compressor will always do that and ignore the original contents). So I think we already intend to compare the data completely to what we would build from scratch. Petr should know if any parts of that checking are still missing.
Up until now we checked that aranges cover everything that's claimed by CUs. Now I've added the reciprocal check, that looks whether CU claims match what aranges cover.
Right, any mismatch there is highly suspect.
Good stuff!
Thanks, Roland
Tom> Consider checking that every CU has an arange entry. Tom> This is not strictly required, but it is nicer for consumers. Tom> ISTR filing a bug about this in GCC bugzilla.
[...]
Roland> Every CU that has any PCs (i.e. compile_unit has low_pc, Roland> high_pc, or ranges attributes at all), that is. In today's GCC, Roland> I'm not sure there is ever a CU with no PCs at all, except for a Roland> type_unit (-gdwarf-4).
Make a file that just has "int x;" in it, then compile with -g. GCC won't emit a .debug_aranges.
I think it ought to emit an empty one, see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42288 for the rationale.
Tom
Make a file that just has "int x;" in it, then compile with -g. GCC won't emit a .debug_aranges.
This was exactly the test I used (I even called it 'x' instead of 'foo' for some reason!). It emits low_pc..high_pc that is an empty range in .text (but a relocated one, so after link it will wind up some specific 0x123,0x123 pair rather than zeros).
It didn't occur to me last week to look for the .debug_aranges section. But I concur that it should be consistent: .debug_aranges should be exactly the set of ranges (whatever that is) emitted for any compile_unit DIE. That should likewise resolve to whatever specific empty range it is, not absolute zeros. (I say so not because the relocated empty range seems especially useful--it tells you nothing except what the link order of the .o files happened to be--but just to stick to the aforementioned consistency in a thorough way.)
I think it ought to emit an empty one, see http://gcc.gnu.org/bugzilla/show_bug.cgi?id=42288 for the rationale.
I guess I concur with the rationale that each and every CU appearing in some rhs in the .debug_aranges section helps identify the section as complete. Before considering that issue, I would have said that it was questionable for a compile_unit to have the PC attributes at all when it does not actually cover any PCs. I'm still not 100% convinced.
Thanks, Roland
11.10.2010 23:14, Roland McGrath wrote:
It didn't occur to me last week to look for the .debug_aranges section. But I concur that it should be consistent: .debug_aranges should be exactly the set of ranges (whatever that is) emitted for any compile_unit DIE. That should likewise resolve to whatever specific empty range it is, not absolute zeros. (I say so not because the relocated empty range seems especially useful--it tells you nothing except what the link order of the .o files happened to be--but just to stick to the aforementioned consistency in a thorough way.)
So let's have main.c with just empty main function, and then two files x.c and y.c, containing just int x and y respectively. We link them all together to get:
[ b] compile_unit [...] name (strp) "main.c" low_pc (addr) 0x0000000000400474 <main> high_pc (addr) 0x000000000040047a [ 5f] compile_unit [...] name (string) "x.c" low_pc (addr) 0x000000000040047c high_pc (addr) 0x000000000040047c [ a8] compile_unit [...] name (string) "y.c" low_pc (addr) 0x000000000040047c high_pc (addr) 0x000000000040047c
Now is that legal? If yes, then the debug_aranges can't be accurate for that situation. If not, I'll have to add a check for that. (Which at this point means probably just making XFAIL test case out of it.)
Handling empty ranges in dwarflint doesn't work, they are diagnosed as error. If they should be accepted, then I guess only as a reciprocal value to empty range in .debug_info. That kind of tracking is not there either.
PM
[ 5f] compile_unit [...] name (string) "x.c" low_pc (addr) 0x000000000040047c high_pc (addr) 0x000000000040047c [ a8] compile_unit [...] name (string) "y.c" low_pc (addr) 0x000000000040047c high_pc (addr) 0x000000000040047c
Now is that legal? If yes, then the debug_aranges can't be accurate for that situation. If not, I'll have to add a check for that. (Which at this point means probably just making XFAIL test case out of it.)
Right. I hadn't thought through the example to articulate it. But this is an example of why I was so dubious about empty ranges. I don't see any abstract rationale for it. These two CUs cover no PCs at all, so I always would have expected just no *_pc/ranges attributes at all there.
Handling empty ranges in dwarflint doesn't work, they are diagnosed as error. If they should be accepted, then I guess only as a reciprocal value to empty range in .debug_info. That kind of tracking is not there either.
Well, we have to decide whether we think GCC is wrong today and should be fixed, or if its existing practice is kosher enough and then dwarflint needs to accept it somehow.
The only rationale I've heard for empty ranges is Tom's idea that every CU should be the rhs of some .debug_aranges entry just so you can be sure that .debug_aranges is really complete (vs. a buggy old GCC, or some different letter-of-DWARF producer where some CUs with actual PCs had no .debug_aranges data emitted, is what I assume Tom must have in mind). As things stand in GCC, indeed these two CUs will both have entries there in the linked output--conflicting matches for the same (empty) PC range with different rhs in each. That meets Tom's test. But it's obviously not actually useful in its own right.
The only possible rationale that I can think of that would really justify a no-code CU producing any .debug_aranges entry as inherently useful is a different one, that isn't satisfied today at all. That is, if the "PC" set that contributes to the *_pc/ranges attributes in the tree, and thus matching in .debug_aranges, were to include not only PC addresses but data addresses too. Then e.g. the x.c CU above would have *_pc/ranges covering the data address range indicated by the DW_AT_location DW_OP_addr &x. This is useful in that it makes it possible to do a by-address lookup of a data address and resolve to the CU that contains some DIE with a fixed-address location description, so you could quickly go from "tell me about 0x123" to "that is 'x' and I know its type and decl coordinates et al". But a) there isn't actually any GDB feature that does a lookup in that direction (even by slow exhaustive search means) today (p/a will just tell you about the ELF symbol matches) and b) no producer has ever emitted .debug_aranges or *_pc/ranges attributes taking data addresses into account, so you could not really have a consumer depend on it in the foreseeable future. So, I tend to think that if this sort of use were any motivation at all, then it should only be a consideration for some new kind of by-address quick index scheme rather than shoehorning it into .debug_aranges.
In short, Petr has helped me become even more dubious about Tom's idea that .debug_aranges getting anything at all for CUs without PCs is any kind of useful.
Thanks, Roland
"Roland" == Roland McGrath roland@redhat.com writes:
Roland> The only rationale I've heard for empty ranges is Tom's idea Roland> that every CU should be the rhs of some .debug_aranges entry Roland> just so you can be sure that .debug_aranges is really complete Roland> (vs. a buggy old GCC, or some different letter-of-DWARF producer Roland> where some CUs with actual PCs had no .debug_aranges data Roland> emitted, is what I assume Tom must have in mind).
I see it as following the wording of the standard. .debug_aranges is optional. Therefore it may or may not exist for a given CU.
I think this, like several other things in DWARF, is pointless and bad -- but it is what we've got.
I don't know if there is any plausible scenario where this could really happen. Maybe hand-written assembly?
It would be very interesting to see a .debug_aranges report from dwarflint run on everything in F13 or F14.
Roland> In short, Petr has helped me become even more dubious about Tom's idea Roland> that .debug_aranges getting anything at all for CUs without PCs is any Roland> kind of useful.
I ran across this while trying to code defensively against the standard. Perhaps this is me being too pedantic. OTOH, the lax route has not worked out very well for gdb historically -- DWARF changes causing crashes, etc.
Right now, GDB isn't using aranges at all. I guess it could; we could drop the range info from the index.
If we do that, I will make it follow whatever you decide here. I don't actually care all that much. Actually, I tend to think we should fork DWARF a little to fix its most egregious problems.
Tom
"Roland" == Roland McGrath roland@redhat.com writes:
Roland> The only rationale I've heard for empty ranges is Tom's idea Roland> that every CU should be the rhs of some .debug_aranges entry Roland> just so you can be sure that .debug_aranges is really complete Roland> (vs. a buggy old GCC, or some different letter-of-DWARF producer Roland> where some CUs with actual PCs had no .debug_aranges data Roland> emitted, is what I assume Tom must have in mind).
I see it as following the wording of the standard. .debug_aranges is optional. Therefore it may or may not exist for a given CU.
That's exactly what I meant by "letter-of-DWARF" above. But, DWARF just talks about "producers" in a unitary, or rather, vague, way. It's not at all clear about compilers vs linkers and so forth. For the quick-lookup sections to be much use at all--especially a by-address one--you pretty much need to have all the CUs that get linked together to have been compiled consistently.
I think this, like several other things in DWARF, is pointless and bad -- but it is what we've got.
I don't know if there is any plausible scenario where this could really happen. Maybe hand-written assembly?
Do you mean hand-assembled DWARF, or gas -g? Hmm. My only guess was either buggy compilers, or some non-GCC compiler that built some library then linked together with GCC code or something.
But I don't actually know off hand about gas -g (or gas assembling explicit .loc directives), I'd forgotten that's another producer we have. Assembling the trivial file "foo: nop" with -g on f13/x86-64, I do see it generates a .debug_aranges. Assembling a trivial data-only .s file (e.g. just ".comm x,4") with -g, it emits no DWARF sections at all.
It would be very interesting to see a .debug_aranges report from dwarflint run on everything in F13 or F14.
I think Petr may be set up to do that easily. (If you weren't already aware, you can look at debuginfo-test-scripts, it's in the Google hits for that token. It might have bit-rotted.)
I ran across this while trying to code defensively against the standard. Perhaps this is me being too pedantic. OTOH, the lax route has not worked out very well for gdb historically -- DWARF changes causing crashes, etc.
Right now, GDB isn't using aranges at all. I guess it could; we could drop the range info from the index.
FWIW, libdw has always depended on .debug_aranges and its by-address lookups will fail completely to notice any CUs not represented there. We haven't seen any problems in the wild. But, stap does little if anything that really relies on by-address lookups, and nothing based on libdw has probably ever been used anywhere near as much as GDB in the kinds of ways where those are important.
If we do that, I will make it follow whatever you decide here. I don't actually care all that much. Actually, I tend to think we should fork DWARF a little to fix its most egregious problems.
I'm not sure I'd call this a fork, but maybe I don't know exactly what you mean. DWARF is such a vague standard for the corners anyway. IMHO for all the optimized lookup sort of stuff what's going to be useful is fresh new extensions/sections rather than tweaking the stuff already specified. For all those things, anything you'd really choose as optimal is stuff you need to do at link time or after, rather than stuff the compiler/assembler can emit in fragments to have generic linker features stitch together. Good choices for consumers to use efficiently don't mesh with things you can emit with standard relocations to come out correctly, you'd like tables sorted on final-linked address for direct binary search of the data mapped untouched from the files, etc.
This (as a vague set of ideas) is one of the subjects on which I mean to write up notes for discussion in person on the GCC Summit trip and still haven't yet. (I don't have any even slightly-specific ideas about any name-based lookup stuff, just by-address stuff.)
Thanks, Roland
elfutils-devel@lists.fedorahosted.org