Hi,
While hunting down some issues I had with dwarf_output on the examples in the gdb testsuite I came across some "that doesn't look like valid dwarf" things. Most of these were already discussed off list with Petr to make sure dwarflint knows about them. But it seems better to have a public archive of these issues, so others can help us interpret whether or not they are really "invalid dwarf", just confusing, or something else.
Silly usage note. If you are seeing lots of: error: .rela.debug_info: offset 0x6: invalid relocation 10 (<INVALID RELOC>). you are probably running dwarflint against old elfutils libraries. Make sure LD_LIBRARY_PATH is explicitly set to the freshly build libraries.
dwarflint would sometimes detect some low-level dwarf datastructure corruption but then would run some of the high-level tests anyway. That didn't end well, because dwarflint would crash on the bad data. Petr already fixed this on the dwarf branch.
The gdb testsuite contains a lot of "minimal dwarf" examples (mostly written by hand. There are some assumptions that might or might not be valid. You can get some of these also with some minimal C "program".
$ echo > empty.c $ gcc -g -c empty.c $ dwarflint empty.o error: .debug_abbrev: couldn't load CU headers; assuming CUs are of latest DWARF flavor. error: .debug_abbrev: no abbreviations. error: .debug_info: data not found.
$ echo "static int empty;" > empty.c $ gcc -g -c empty.c $ eu-dwarflint empty.o error: .debug_line: table 0: sequence of opcodes not terminated with DW_LNE_end_sequence.
These examples really don't have any .debug_line info, just an short debug_line section header. Maybe gcc shouldn't output any header at all, or maybe it should make the empty table by having only the DW_LNE_end_sequence in it instead? Or should dwarflint ignore the table if the header says the table is empty?
Attached are also some examples that show "bad child" marking in a dwarf file. These can happen easily when you are writing dwarf examples by hand (as is done in the gdb testsuite a lot). The diff of the .s files will show that "accidentally" some DIEs were marked as not having children but the "end of child list" markers are still there (it also tweaks the siblings to be some other reference type, because dwarflint does see the discrepancy in that case - go dwarflint! - except my original didn't have sibling markers of course...).
The result is that something that relies on dwarf_siblingof will just not see the last few DIEs. Since we are under the impression the CU is done. This caused havoc for dwarf_output which was suddenly missing some referenced DIEs. But you can also see that eu-readelf will just stop early (interestingly binutils readelf will show them (but with an indentation of zero, which is rather odd for non-CU dies).
I am hoping that since unit headers have a length and the "top-level" CU die needs to have children that cover the whole unit dwarflint somehow can detect that there is something odd going on with the children relationship even if there are no sibling_AT markers. A zero or negative "indentation level" for a DIE that isn't the top-level CU DIE should probably cause at least a warning.
Cheers,
Mark
Mark Wielaard mjw@redhat.com writes:
Attached are also some examples that show "bad child" marking in a dwarf file. These can happen easily when you are writing dwarf examples by hand (as is done in the gdb testsuite a lot). The diff of the .s files will show that "accidentally" some DIEs were marked as not having children but the "end of child list" markers are still there (it also tweaks the siblings to be some other reference type, because dwarflint does see the discrepancy in that case - go dwarflint! - except my original didn't have sibling markers of course...).
The result is that something that relies on dwarf_siblingof will just not see the last few DIEs. Since we are under the impression the CU is done. This caused havoc for dwarf_output which was suddenly missing some referenced DIEs. But you can also see that eu-readelf will just stop early (interestingly binutils readelf will show them (but with an indentation of zero, which is rather odd for non-CU dies).
I am hoping that since unit headers have a length and the "top-level" CU die needs to have children that cover the whole unit dwarflint somehow can detect that there is something odd going on with the children relationship even if there are no sibling_AT markers. A zero or negative "indentation level" for a DIE that isn't the top-level CU DIE should probably cause at least a warning.
In your case, the DIEs that would normally be children became siblings of the CU DIE. But the dwarf standard prescribes that each CU contain one CU DIE and its children, so this setup is invalid. dwarflint should have diagnosed this case, and this is now fixed on the dwarf branch.
Thanks, PM
$ echo > empty.c $ gcc -g -c empty.c $ dwarflint empty.o error: .debug_abbrev: couldn't load CU headers; assuming CUs are of latest DWARF flavor. error: .debug_abbrev: no abbreviations. error: .debug_info: data not found.
This produces zero-length .debug_info and .debug_line sections. There is nothing wrong with that, though it would of course also be fine if it didn't emit the ".section" directives at all.
It produces a .debug_abbrev section that is nothing but a single 0 byte. That is superfluous and it would be better if it omitted it entirely, so I would recommend reporting that as a GCC bug. (Many such files will add up to lots of wasted zero bytes in the linked output.)
However, for dwarflint, this should not be reported in this way. .debug_abbrev is not a "first class" section (only .debug_info, .debug_types, and .debug_frame are). That is, pieces of it only matter when they are pointed to by some CU's header. So, what it ought to be reporting here is that there is an "unused hole" at offset [0,1) in .debug_abbrev.
That first error message makes it sound like dwarflint is reading .debug_abbrev as a whole independent of individual CU header pointers. It shouldn't do that, except to note the unreferenced portions of the section. Each portion of .debug_abbrev is only meaningful as being pointed to by a particular CU (or by more than one CU, as will be the case after compression). Each CU in a file could have a different DWARF version number in its header. The meaning of each portion of .debug_abbrev only has to fit the format version of the referring CU. Of course, if multiple CUs refer to the same .debug_abbrev, then it is suspicious if they are different versions. But even that is potentially valid, since .debug_abbrev format itself hasn't actually changed between 2, 3, and 4. The only thing that is really invalid is if a form code is used in a .debug_abbrev portion that is not valid in the DWARF version of its referring CU.
$ echo "static int empty;" > empty.c $ gcc -g -c empty.c $ eu-dwarflint empty.o error: .debug_line: table 0: sequence of opcodes not terminated with DW_LNE_end_sequence.
This produces a zero-length .debug_line section but a CU with a DW_AT_stmt_list attribute pointing to its start. That is invalid DWARF and hence a GCC bug. The place where DW_AT_stmt_list points must have a valid line information header, even if that says it's an empty table. I think what would be most correct here is to omit the DW_AT_stmt_list attribute entirely from that CU DIE.
Attached are also some examples that show "bad child" marking in a dwarf file. These can happen easily when you are writing dwarf examples by hand (as is done in the gdb testsuite a lot). The diff of the .s files will show that "accidentally" some DIEs were marked as not having children but the "end of child list" markers are still there (it also tweaks the siblings to be some other reference type, because dwarflint does see the discrepancy in that case - go dwarflint! - except my original didn't have sibling markers of course...).
This is valid DWARF because of the 7.5.2 mention of "null entries". That is, the extra zero byte (pedantically, ULEB128 amounting to zero) is the abbreviation code for the next entry and this is a null entry. A null entry is valid even as a sibling to the top-level compile_unit.
So, for the presence of an extra zero byte like that dwarflint should only mention it as being unnecessary alignment padding--if it's not needed to make the next entry fit that .debug_info section's sh_addralign. Regardless of sh_addralign, there is never an actual reason for such padding inside a single CU (since a single CU comes from a single input section). So it should be flagged as suspicious in any place but between CUs without looking at any sh_addralign calculations.
Separately, dwarflint should be checking all DW_AT_sibling values for matching the sibling layout as indicated by the basic DIE header format.
Thanks, Roland
On Mon, 2011-03-07 at 09:56 -0800, Roland McGrath wrote:
$ echo > empty.c $ gcc -g -c empty.c $ dwarflint empty.o error: .debug_abbrev: couldn't load CU headers; assuming CUs are of latest DWARF flavor. error: .debug_abbrev: no abbreviations. error: .debug_info: data not found.
This produces zero-length .debug_info and .debug_line sections. There is nothing wrong with that, though it would of course also be fine if it didn't emit the ".section" directives at all.
It produces a .debug_abbrev section that is nothing but a single 0 byte. That is superfluous and it would be better if it omitted it entirely, so I would recommend reporting that as a GCC bug. (Many such files will add up to lots of wasted zero bytes in the linked output.)
Bug and proposed patch: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48041
$ echo "static int empty;" > empty.c $ gcc -g -c empty.c $ eu-dwarflint empty.o error: .debug_line: table 0: sequence of opcodes not terminated with DW_LNE_end_sequence.
This produces a zero-length .debug_line section but a CU with a DW_AT_stmt_list attribute pointing to its start. That is invalid DWARF and hence a GCC bug. The place where DW_AT_stmt_list points must have a valid line information header, even if that says it's an empty table. I think what would be most correct here is to omit the DW_AT_stmt_list attribute entirely from that CU DIE.
Actually, the header is there, but there are no tables. Reported with possible patch: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48045
But since the debug_line table header is actually there, dwarflint is still not really happy. empty.o as produced by my patched gcc attached.
$ eu-dwarflint empty.o warning: .debug_line: table 0: the file #1 `empty.c' is not used. error: .debug_line: table 0: sequence of opcodes not terminated with DW_LNE_end_sequence.
So maybe a better solution would be to just add a dummy end_sequence.
Cheers,
Mark
Mark Wielaard mjw@redhat.com writes:
On Mon, 2011-03-07 at 09:56 -0800, Roland McGrath wrote:
This produces a zero-length .debug_line section but a CU with a DW_AT_stmt_list attribute pointing to its start. That is invalid DWARF and hence a GCC bug. The place where DW_AT_stmt_list points must have a valid line information header, even if that says it's an empty table. I think what would be most correct here is to omit the DW_AT_stmt_list attribute entirely from that CU DIE.
Actually, the header is there, but there are no tables. Reported with possible patch: http://gcc.gnu.org/bugzilla/show_bug.cgi?id=48045
But since the debug_line table header is actually there, dwarflint is still not really happy. empty.o as produced by my patched gcc attached.
$ eu-dwarflint empty.o warning: .debug_line: table 0: the file #1 `empty.c' is not used. error: .debug_line: table 0: sequence of opcodes not terminated with DW_LNE_end_sequence.
So maybe a better solution would be to just add a dummy end_sequence.
Well, missing DW_LNE_end_sequence is quite common, as I recall. Chances are they are all cases of empty table with no opcodes whatsoever. Perhaps I might supress the error for these cases.
PM
This produces a zero-length .debug_line section but a CU with a DW_AT_stmt_list attribute pointing to its start. That is invalid DWARF and hence a GCC bug. The place where DW_AT_stmt_list points must have a valid line information header, even if that says it's an empty table. I think what would be most correct here is to omit the DW_AT_stmt_list attribute entirely from that CU DIE.
Actually, the header is there, but there are no tables.
That's not what I saw.
$ gcc -g -S -xc <(echo 'static int x;') -o - | grep debug_line .section .debug_line,"",@progbits .Ldebug_line0: .long .Ldebug_line0
The only entrance into the section is that one:
.section .debug_line,"",@progbits .Ldebug_line0: .text
So the section is indeed empty.
But since the debug_line table header is actually there, dwarflint is still not really happy. empty.o as produced by my patched gcc attached.
$ eu-dwarflint empty.o warning: .debug_line: table 0: the file #1 `empty.c' is not used. error: .debug_line: table 0: sequence of opcodes not terminated with DW_LNE_end_sequence.
So maybe a better solution would be to just add a dummy end_sequence.
I'm not sure the specification is clear this is invalid. It says, "Every line number program sequence must end with a DW_LNE_end_sequence instruction..." But I don't see any wording that says there actually has to be any line number program sequence at all. Note that a .debug_line table with no line number program sequences still has a valid meaning: it defines the include directory path for the CU, and it defines the file entries that the CU's DW_AT_decl_file et al refer to.
Thanks, Roland
On Wed, 2011-03-09 at 09:51 -0800, Roland McGrath wrote:
This produces a zero-length .debug_line section but a CU with a DW_AT_stmt_list attribute pointing to its start. That is invalid DWARF and hence a GCC bug. The place where DW_AT_stmt_list points must have a valid line information header, even if that says it's an empty table. I think what would be most correct here is to omit the DW_AT_stmt_list attribute entirely from that CU DIE.
Actually, the header is there, but there are no tables.
That's not what I saw.
$ gcc -g -S -xc <(echo 'static int x;') -o - | grep debug_line .section .debug_line,"",@progbits .Ldebug_line0: .long .Ldebug_line0
The only entrance into the section is that one:
.section .debug_line,"",@progbits .Ldebug_line0: .text
So the section is indeed empty.
Ah, yes. That is what I thought first too. But gcc is sneaky :) If gcc detects gas has support for the .loc directive, it will not output the section itself, but let gas generate it (because the assembler has exact info on instruction sizes). See gcc/dwarf2out.c DWARF2_ASM_LINE_DEBUG_INFO as described in http://gcc.gnu.org/onlinedocs/gccint/SDB-and-DWARF.html and the gas manual http://sourceware.org/binutils/docs/as/Loc.html
Cheers,
Mark
Ah, yes. That is what I thought first too. But gcc is sneaky :) If gcc detects gas has support for the .loc directive, it will not output the section itself, but let gas generate it (because the assembler has exact info on instruction sizes).
Sure, but there is no .loc in there! In fact, it's gas that is quite sneaky here.
/* If there is no line information and no non-empty .debug_info section, or if there is both a non-empty .debug_info and a non-empty .debug_line, then we do nothing. */
So its the presence of some .section .debug_info content (even without any .file, let alone .loc) that makes it emit the .debug_line header.
If there is a bug here, it's a gas bug.
Thanks, Roland
On Wed, 2011-03-09 at 10:34 -0800, Roland McGrath wrote:
Ah, yes. That is what I thought first too. But gcc is sneaky :) If gcc detects gas has support for the .loc directive, it will not output the section itself, but let gas generate it (because the assembler has exact info on instruction sizes).
Sure, but there is no .loc in there! In fact, it's gas that is quite sneaky here.
/* If there is no line information and no non-empty .debug_info section, or if there is both a non-empty .debug_info and a non-empty .debug_line, then we do nothing. */
So its the presence of some .section .debug_info content (even without any .file, let alone .loc) that makes it emit the .debug_line header.
If there is a bug here, it's a gas bug.
Indeed. So you can "trick" gas into creating a needless .debug_line section by just doing:
$ echo ".section .debug_info" > empty.s; echo ".byte 0x0" >> empty.s $ as -g empty.s $ eu-readelf -w a.out
DWARF section [ 4] '.debug_info' at offset 0x40: [Offset]
DWARF section [ 5] '.debug_line' at offset 0x41:
Table at offset 0:
Length: 25 DWARF version: 2 Prologue length: 19 Minimum instruction length: 1 Maximum operations per instruction: 1 Initial value if 'is_stmt': 1 Line base: -5 Line range: 14 Opcode base: 13
Opcodes: [ 1] 0 arguments [ 2] 1 argument [ 3] 1 argument [ 4] 1 argument [ 5] 1 argument [ 6] 0 arguments [ 7] 0 arguments [ 8] 0 arguments [ 9] 1 argument [10] 0 arguments [11] 0 arguments [12] 1 argument
Directory table:
File name table: Entry Dir Time Size Name
Line number statements:
That of course is an invalid .debug_info section, but there really is no reason for gas to output the .debug_line section since there was no explicit .debug_line sections, nor where there any .loc or .line directives.
Current dwarflint does error on the above, but doesn't mention that nothing refers to the "empty" .debug_line sections: $ eu-dwarflint a.out error: .debug_abbrev: data not found.
So, the bug is in gas. I cannot immediately think of how to trick gcc itself into producing this bug. But if this gas bug is fixed we might need to fix gcc to not unconditionally produce a DW_AT_line_stmt_list for the CU.
Cheers,
Mark
That of course is an invalid .debug_info section, but there really is no reason for gas to output the .debug_line section since there was no explicit .debug_line sections, nor where there any .loc or .line directives.
You mean any .loc or two-operand .file directives.
So, the bug is in gas. I cannot immediately think of how to trick gcc itself into producing this bug. But if this gas bug is fixed we might need to fix gcc to not unconditionally produce a DW_AT_line_stmt_list for the CU.
Yes, it could emit DW_AT_stmt_list only if it ever emitted a two-operand .file directive. I think the only situation in which it would ever fail to emit one of those is a file that does not define anything at all (i.e. empty or containing only type and extern declarations). That is sufficiently rare and meaningless that it doesn't seem really worth giving gcc a special case for it. If anything, I'd almost consider it a feature if gcc *did* emit .file 1 "empty.c" for an empty file--then you get DWARF data that lists each and every source file that was compiled, so that things like the rpmbuild procedure that copies sources into /usr/src/debug would include some file of #include's and macros and whatnot that led to defining nothing. (Though I'm not really convinced that would actually be a feature, it just came to mind.)
Thanks, Roland
Hi Roland,
That of course is an invalid .debug_info section, but there really is no reason for gas to output the .debug_line section since there was no explicit .debug_line sections, nor where there any .loc or .line directives.
You mean any .loc or two-operand .file directives.
Yes, typo. Although I admit I hadn't noticed .file with one argument had different semantics.
But if this gas bug is fixed we might need to fix gcc to not unconditionally produce a DW_AT_line_stmt_list for the CU.
Yes, it could emit DW_AT_stmt_list only if it ever emitted a two-operand .file directive. I think the only situation in which it would ever fail to emit one of those is a file that does not define anything at all (i.e. empty or containing only type and extern declarations). That is sufficiently rare and meaningless that it doesn't seem really worth giving gcc a special case for it.
I assume it is rare, but indeed, a simple file with just #include <stdio.h> seems a testcase.
If anything, I'd almost consider it a feature if gcc *did* emit .file 1 "empty.c" for an empty file --then you get DWARF data that lists each and every source file that was compiled, so that things like the rpmbuild procedure that copies sources into /usr/src/debug would include some file of #include's and macros and whatnot that led to defining nothing.
Doesn't/Shouldn't it also look at the CUs name and comp_dir? BTW. Where is the code that does the splitting/copying hosted?
Thanks,
Mark
Doesn't/Shouldn't it also look at the CUs name and comp_dir?
Oh yeah, the CU DW_AT_name is what says that. So never mind what I said.
BTW. Where is the code that does the splitting/copying hosted?
It's part of rpm (git://rpm.org/rpm.git), in tools/debugedit.c (this is installed as /usr/lib/rpm/debugedit and that's used by /usr/lib/rpm/find-debuginfo.sh).
Thanks, Roland
Roland McGrath roland@redhat.com writes:
But since the debug_line table header is actually there, dwarflint is still not really happy. empty.o as produced by my patched gcc attached.
$ eu-dwarflint empty.o warning: .debug_line: table 0: the file #1 `empty.c' is not used. error: .debug_line: table 0: sequence of opcodes not terminated with DW_LNE_end_sequence.
So maybe a better solution would be to just add a dummy end_sequence.
I'm not sure the specification is clear this is invalid. It says, "Every line number program sequence must end with a DW_LNE_end_sequence instruction..." But I don't see any wording that says there actually has to be any line number program sequence at all. Note that a .debug_line table with no line number program sequences still has a valid meaning: it defines the include directory path for the CU, and it defines the file entries that the CU's DW_AT_decl_file et al refer to.
I suppressed that message in dwarflint to only appear if there are actual line number program opcodes in the table.
PM
elfutils-devel@lists.fedorahosted.org