2009-09-29 Status
by Petr Machata
Last week + today
-----------------
Time spent on elfutils: ~50%
writer:
* Emit .debug_ranges
* Emit .debug_loc
* dwarfcmp ./a.out ./a.out.out now works
It's on pmachata/dwarf-writer.
Next in plan
------------
* Merge in dwarf branch.
* Make dwarfcmp work for bigger files.
* We can't optimize certain references right away. For these we now
pick the biggest form available. We could shrink these fields
afterwards. But the codebase is not ready for this, e.g. gaps have to
be able to move, size of ElfData has to be patched, etc. So work on
preparing these bits, they will be useful later.
PM
14 years, 3 months
Adding a NOTE section after a copy ..
by Jay Vaughan
Hi there elfutils-devel'ers!
I am writing an application which is being used internally to flag ELF objects in our team
.. the idea is that we want to be able to add a custom NOTE to our internal binaries to
assist with tracking our builds and so on. We do not wish to change our toolchain, thus
using the 'ld --build-id' feature isn't really feasible at the moment, so I have been
given the responsibility of using libelf to accomplish the same thing ..
In order to make this 'easy', I have decided that my tool will take two args - INFILE and
OUTFILE - and simply process the ELF objects of INFILE through libelf, writing them all
out to OUTFILE, and then .. when done .. appending a new NOTE Section to the OUTFILE. I
have seen this technique working just fine .. e.g. in the elfutils test suite .. so I
believe this is an acceptable approach for me so far.
All works fine - I am able to copy the files smoothly, as well as locate my own NOTE
section that will be copied into the new binary .. this NOTE section is stored in my tool
using __attribute__((section(OURSECTION_NAME)) and I retrieve this section (for 'cloning')
before I do the write-out to the new OUTFILE..
However.. I am getting this error when I attempt to elf_update() the OUTFILE:
ecpnoter: elf32_updatefile.c:622: __elf32_updatefile: Assertion `last_offset < scn_start +
dl->data.d.d_off' failed.
(ecpnoter is the name of my tool) ..
Does this mean that the last_offset of my newly added section is greater than the length
of data in my section?
Am I correct in assuming that the Section Header for OUTFILE needs to be adjusted - or
flag'ed in some way - in order to make elf_update re-adjust all the sections to accomodate
the newly added section and that I do *not* have to manually adjust the section header
itself, somehow?
Please note that I am copying the old EHDR and PHDR's from INFILE to OUTFILE at the
beginning of the process .. how do I get ehdr->e_shnum incremented "automagically" to
accomodate the newly added NOTE section? I assume (correct me please) that the elf_*
functions can be told to re-compute these headers appropriately, no?
Right now I'm doing this :
/* Flag our ELF objects for update */
elf_flagelf(out_elf, ELF_C_SET, ELF_F_LAYOUT);
elf_flagphdr(out_elf, ELF_C_SET, ELF_F_DIRTY);
elf_flagshdr(new_scn_p, ELF_C_SET, ELF_F_DIRTY);
.. but I'm not sure this is correct, as I am really struggling to understand fully the
libelf functionality with the given *ahem* documentation ..
Any help provided is greatly appreciated, especially if my_cluefulness++; // happens .. :)
--
;
--
Jay Vaughan
jay.vaughan(a)thalesgroup.com
14 years, 5 months
eu-ld buglets
by Roland McGrath
I had occasion to try running eu-ld and it crashed with any --rpath-link
switch. It looks like the rest of the code is not expecting the change
that gen_rxxpath_data makes (normalize_dirlist crashes, but AFAICT
open_along_path2 is not expecting it either and would crash if that were
changed). Also by eyeball I noticed some wrong-looking free calls while
grokking the surrounding code, though I didn't test them empirically.
Maybe you want to fix these up with appropriate comment changes et al.
Thanks,
Roland
diff --git a/src/ld.c b/src/ld.c
index 989bfab..0000000 100644
--- a/src/ld.c
+++ b/src/ld.c
@@ -1354,22 +1354,22 @@ gen_rxxpath_data (void)
used (or ever be invented for that matter). */
if (ld_state.rpath != NULL)
{
- struct pathelement *endp = ld_state.rpath;
+ //struct pathelement *endp = ld_state.rpath;
ld_state.rpath = ld_state.rpath->next;
- endp->next = NULL;
+ //endp->next = NULL;
}
if (ld_state.rpath_link != NULL)
{
- struct pathelement *endp = ld_state.rpath_link;
+ //struct pathelement *endp = ld_state.rpath_link;
ld_state.rpath_link = ld_state.rpath_link->next;
- endp->next = NULL;
+ //endp->next = NULL;
}
if (ld_state.runpath != NULL)
{
- struct pathelement *endp = ld_state.runpath;
+ //struct pathelement *endp = ld_state.runpath;
ld_state.runpath = ld_state.runpath->next;
- endp->next = NULL;
+ //endp->next = NULL;
/* If rpath information is also available discard it.
XXX Should there be a possibility to avoid this? */
@@ -1377,7 +1377,7 @@ gen_rxxpath_data (void)
{
struct pathelement *old = ld_state.rpath;
ld_state.rpath = ld_state.rpath->next;
- free (old);
+ free (old); /* XXX isn't it on the obstack?? */
}
}
if (ld_state.runpath_link != NULL)
@@ -1392,7 +1392,7 @@ gen_rxxpath_data (void)
{
struct pathelement *old = ld_state.rpath_link;
ld_state.rpath_link = ld_state.rpath_link->next;
- free (old);
+ free (old); /* XXX isn't it on the obstack?? */
}
/* The information in the strings in the list can actually be
14 years, 5 months
dwarflint: ranges in .debug_loc vs. PCc claimed for containing DIE
by Petr Machata
I've implemented the test for $SUBJ, and am running into lots of
violations that look genuine. Current criteria are: for each attribute
with loclist value (i.e. VS_location, .is_list holds) check that the
address ranges of that location list form a subset of PC ranges of
containing PC-ful DIE.
One violator is src/size form elfutils:
error: .debug_info: DIE 0x13c1: attribute `location': range [0x4014a6,
0x401561) runs off DIE scope
error: .debug_info: DIE 0x13bc: DIE scope is: [4013d4, 40155a), [4018c0,
4018cf), [401db6, 401e03)
DWARF section [29] '.debug_info' at offset 0x34c5:
[Offset]
Compilation unit at offset 0:
Version: 2, Abbreviation section offset: 0, Address size: 8, Offset
size: 4
[ b] compile_unit
producer "GNU C 4.4.1 20090725 (Red Hat 4.4.1-2)"
language ISO C99 (12)
name "../../elfutils/src/size.c"
comp_dir
"/home/petr/proj/elfutils/dwarflint/build/src"
low_pc 0x0000000000401050 <print_version>
high_pc 0x0000000000402429
stmt_list 0
[ 125b] subprogram
name "handle_elf"
decl_file 1
decl_line 667
prototyped
low_pc 0x0000000000401320 <handle_elf>
high_pc 0x0000000000401ed0 <handle_ar>
frame_base location list [ 320]
sibling [ 16ae]
[ 1390] inlined_subroutine
abstract_origin [ 11a9]
entry_pc 0x00000000004013d4 <handle_elf+0xb4>
ranges range list [ 280]
call_file 1
call_line 692
sibling [ 1439]
[ 13bc] lexical_block
ranges range list [ 2c0]
[ 13c1] variable
abstract_origin [ 11e7]
location location list [ 74c]
(In following, the values on the right match those on the left after
adding CU's low_pc.)
DWARF section [34] '.debug_ranges' at offset 0x7771:
[ 2c0] 0x0000000000000384..0x000000000000050a 0x4013d4 0x40155a
0x0000000000000d66..0x0000000000000db3 0x401db6 0x401e03
0x0000000000000870..0x000000000000087f 0x4018c0 0x4018cf
DWARF section [33] '.debug_loc' at offset 0x6209:
[ 74c] 0x00000003d9..0x0000000456 [ 0] reg13 0x401429 0x4014a6
0x0000000456..0x0000000511 [ 0] reg12 0x4014a6 0x401561
0x0000000870..0x000000087f [ 0] reg12 0x4018c0 0x4018cf
0x0000000d66..0x0000000db3 [ 0] reg13 0x401db6 0x401e03
So that all looks genuine--the ranges cited by dwarflint are all there
and they really do run off of the PC space of DIE 13bc. I don't see how
to explain this other than by error of toolchain--if a variable is
inside a lexical_block, it's range of validity should not leak out of
that scope.
There are more violations like that. I only examined a couple, total
output has dozens of cases.
I guess I'll just document this on the wiki, dunno what more to do at
this point. Obviously, anyone is welcome to point out errors in my
reasoning above.
How to handle these in the writer? Should we correct this e.g. by
trimming location ranges to their claimed PC scope?
Thanks,
PM
14 years, 5 months
dwarflint: ranges of DIEs in hierarchy
by Petr Machata
Hi,
I'm implementing a check that the range of .debug_loc addresses doesn't
run off range claimed by DW_AT_low_pc/high_pc pair or DW_AT_ranges. It
just so happens that it's also easy to check that ranges of a DIE are
sub-range of its parent, since I need that range-tracking logic anyway.
Such a check sounded plausible to me, but it wasn't really that hard
to found a case that gives me second thoughts (from readelf):
[ 5333] subprogram
name "parse_opt"
decl_file 1
decl_line 283
prototyped
type [ 435]
low_pc 0x0000000000403510 <parse_opt>
high_pc 0x0000000000403ab0 <print_pubnames>
frame_base location list [ 2b1]
sibling [ 566d]
[ 5358] formal_parameter [...]
[ 5368] formal_parameter [...]
[ 5378] subprogram
name "add_dump_section"
decl_file 1
decl_line 286
prototyped
low_pc 0x004034b0 <add_dump_section.9838>
high_pc 0x0040350e
frame_base location list [ 4f7]
static_link 1 byte block: 56
[ 0] reg6
sibling [ 53d5]
... so that's a case of internal function add_dump_section inside other
function parse_opt. The internal function wasn't inlined, and so
occupies different part of address space.
I can't decide whether that case is kosher, I can argue both sides.
Shouldn't the parent either claim addresses of its child, or not have
that DIE as a child in a first place?
PM
14 years, 5 months
dwarfcmp for individual entries
by Roland McGrath
I took a break from tearing my hair out in dwarf_output::copier hell to
hack on dwarfcmp a bit. I'm glad I did.
I fixed some bugs in the tracker and comparator logic along the way. The
main thing I've done is add a new mode of operation in dwarfcmp: with four
or more arguments instead of two, you get to choose particular DIE pairs
(by hex offset) and compare just those. This is especially useful for
comparing two entries you'd think should be consolidated as duplicates and
seeing exactly why they won't be.
I've also souped it up to give more information at the end for all the
"reference mismatch" cases, not just the "context mismatch" subset.
That's key so that when you have a hairy chain of structs pointing to
other structs, you see the eventual non-reference difference that breaks
the matching of the whole chain. e.g.:
41417 vs 749b7: type=[0x3b27e] vs type=[0x6f485] (<structure_type name="Dwarf_Die"> with reference mismatches)
reference 3b27e vs 6f485: 3b298 vs 6f49f: type=[0x3b24f] vs type=[0x6f456] (<pointer_type/> with reference mismatches)
reference 3b24f vs 6f456: 3b24f vs 6f456: type=[0x3b18d] vs type=[0x6f450] (<structure_type name="Dwarf_CU"> != <structure_type name="Dwarf_CU"/>)
reference 3b18d vs 6f450: 3b18d vs 6f450: byte_size=0x60 vs declaration=1
That shows how everything is the same about 'struct Dwarf_Die' itself,
but its member of type 'struct Dwarf_CU *' leads to "struct Dwarf_CU;" in
one case and the real "struct Dwarf_CU { ... };" in the other.
I've just pushed all those fixes and changes so you can use that feature.
There is a crash in double-free at final destructor time in some cases,
which must be some sort of ref-count braino in sharing_stack though I
have failed to figure it out at all. The error (valgrind catches it as
use-after-free, otherwise a ref-count assert hits) goes away with:
- typedef subr::sharing_stack<die> die_path;
+ typedef subr::stackish<die> die_path;
in dwarf_tracker. But sharing_stack exists because it vastly reduces the
memory consumption of the tracker, so we don't really want that change.
We should figure out this crash, but I'm leaving it there for the moment
(the crash, not the workaround).
The new feature helped me a lot. Using the dwarflint binary as a test
case, I was demoralized about my dwarf_output::copier failing to
consolidate duplicate 'class dwarf' entries as the first big example.
But it turns out that it was actually correct about all those, after I
used the new dwarfcmp feature to investigate them. (I'm still fairly
demoralized about how shockingly slow it is, but I haven't really looked
into that much yet.)
Many failures to match are due to one of a couple of GCC bugs I
discovered. One only applies when using ccache (really it's using
-fpreprocessed vs internal cpp in cc1{,plus}), so it gets out of the way
in practice if you just don't use ccache.
The wrong decl coordinates bug (523810) bites many more, and exists all
over in existing DWARF data. I'm still seeing it because the F-11
compiler still has the bug and that's what I'm using, but Jakub has
already fixed it in the F-12 compiler. It's also something we could
probably work around fairly easily with some fuzziness options in for
comparator and copier/collector.
I've created this tracker bug in Fedora to list all the GCC issues I've
come across that will foul up our attempts to do maximal compression:
https://bugzilla.redhat.com/show_bug.cgi?id=526977
If anything else like that comes up, be sure to add its Fedora gcc bug #
to 526977's blockers list, and also post here about the details.
Thanks,
Roland
14 years, 5 months
dwarflint high-level check: suspicious presence of name
by Roland McGrath
I've been noticing some instances of const_type and the like having a name
attribute. (This comes up in the elfutils build on F-11/x86_64.) At least
in C and C++, this is suspicious compiler behavior. The various "modifier"
and "qualifier" sorts of DW_TAG_*_type never get their own names in C/C++
syntax.
Only struct/class/union (DW_TAG_{structure,class,union}_type) have their
own names, or DW_TAG_base_type. When you write "const foo" in C, it's
whatever DIE describes "foo" (sans const) that has a name, and then the
"const foo" DIE is just a <const_type type="#foo"/> with no other
attributes. In these languages, the only way you associate a name with one
of these types is with typedef, but that is a named DW_TAG_typedef pointing
at the DIE that is const_type or whatever.
This applies to:
array_type
pointer_type
reference_type
ptr_to_member_type
const_type
volatile_type
restrict_type
rvalue_reference_type
and perhaps other *_type but I think the ones I omitted never appear in
C/C++. I don't know the rules for other languages. For these sorts of
high-level tests, dwarflint should be set up to categorize them by the
DW_AT_language code of the CU. For now it is probably wisest to default
all languages to the C/C++ rules and we can hone it later after
encountering actual data for other languages. But I'm suggesting that you
structure it as language-sensitive now, rather than just removing name from
the "expected optional" attributes for these tags generically. I'd leave
the generic checks to match the DWARF spec reference tables, and then layer
on the more-specific per-language checks as well.
At least with gcc-4.4.1-2.fc11.x86_64 you should get a few hits for these
checks just on the elfutils build itself.
Thanks,
Roland
14 years, 5 months