elfutils status 2009-1-5
by Roland McGrath
Last three weeks (one working week before end-of-2008 holidays):
Actual time on elfutils: 110%
:-) I don't remember the proportion for the week of 2008-12-15.
I worked a fair bit through the holiday break, and all on elfutils.
* 0.138 released, including elfutils/version.h and bug compat
* 0.138 libelf regression fix (ed9d3bc1, post coming)
* much more wiki content: DwarfProducer et al
* reviewed dwarflint code
* C++ interface prototype working
* skeleton dwarfcmp working
* debuginfo-test-scripts hacks
-> now set up for mass testing against all fedora debuginfo ELF files
-> sent some debuginfo audit detail posts to fedora-devel-list
This week:
Expected time on elfutils this week: 60%
* post brain dump wrt 0.138 libelf regression
* merge fixes-for-c++ to trunk?
* clean up & comment C++ header, dwarfcmp code
* direct next dwarflint steps
* move roland/dwarf-c++ to be a shared branch, merge in dwarflint
* ping archer group wrt imported_unit/partial_unit support
* F11 feature page update, move to "wrangler" step
* write DwarfValues wiki page wrt attr value design
* try to hit DwarfTasks 2.5: populate dwarf writer data from input copy
* start fleshing out C++ attr value interfaces
15 years, 3 months
C++ and dwarfcmp
by Roland McGrath
Hi! Happy New Year!
I got started a bit over xmas and now I've snuck up on the first milestone
in DwarfTasks. I'll post a formal status report in the morning.
But I wanted to mention the technical stuff.
First is the new branch:
roland/fixes-for-c++
This has changes to the existing C code that I think can be merged in soon.
This is so far just one little thing I noticed while doing the C++ stuff.
It's either a bug or a change in the error case behavior, depending how you
look at it. I think it's right enough for C libdw too, and without it I'd
have to do some bad workaround in the C++ wrapper. I'll put other nits of
the same ilk here as I come across them.
Forking from that is the new branch:
roland/dwarf-c++
This contains new C++ code and dwarfcmp.
This is highly prototypey, unfinished, unclean stuff.
The placement of files and code will change before it's real.
The C++ header is thoroughly unreadable. It will get cleaner and commented
before it's firm, but it will still be unreadable (welcome to C++).
The hope is that C++ code that just uses the interfaces will be relatively
easy to read though.
The code works enough for dwarfcmp to run, and it succeeded on three small
cases (both comparing files to themselves and comparing different files).
It is roughly up to the 1.2 milestone on DwarfTasks. This week I'll do the
exhaustive tests for "dwarfcmp -q file file" and fix bugs.
It's not finished or used yet, but I also did the basic sketch of the code
for the 2.5 milestone (skeleton writable interface). I hope to get that
prototype finished this week too.
When any of this is slightly cleaned up, I'll turn this into a shared
branch for the new DWARF work in general, and we can merge dwarflint there.
My next other task will be writing up the missing DwarfValues page about
the thinking for attr values, the missing heart of the C++ interface.
I should get some words down on that this week, if not any actual work.
Thanks,
Roland
15 years, 3 months
mass testing scripts
by Roland McGrath
I've refined the stuff I posted before and cobbled together some basic
scripts for doing mass testing on .debug files from the distros.
I've put the scripts at:
git://git.fedorapeople.org/~roland/debuginfo-test-scripts.git
http://fedorapeople.org/gitweb?p=roland/public_git/debuginfo-test-scripts...
and here's the README from there:
These are some scripts for unpacking lots of -debuginfo rpms and then
running tests against all the ELF files therein.
Please send all questions to <elfutils-devel(a)lists.fedorahosted.org>.
Start with a mostly empty directory where you have a ton of free disk
space. I just use the checkout itself, but you can also cd into a
different directory to hold the data and call the scripts by names
not starting with ./ like I show here.
$ git clone git://git.fedorapeople.org/~roland/debuginfo-test-scripts.git test
$ cd test
Now extract a giant mound of data:
$ ./extract-distro.sh /mirror/fedora/development
The arguments are directories where it will find "*/debug" subdirectories
containing many .rpm files. The example is a mirror of rawhide; other
examples that work are .../releases/10/Everything and .../updates{,/testing}.
This will chug and chug for some hours, depending how much is in the
directories you gave it. It fills the current directory with one
subdirectory for each rpm it unpacked, with names from the rpm file names,
e.g. foo-debuginfo-1.0-1.i386/.
You can run this again later with more directories or the same ones.
It will silently skip any rpm files it has already unpacked.
For all of rawhide (four arch's) this was around 35G when done, I think.
It chugs in parallel doing unpacking and compressing, with each new
subdirectory grows and then shrinks substantially. So be sure you have
plenty of breathing room on the disk.
This is probably overly parallel and chugs along swamping the machine.
Run it on a spare (fastish) machine you are not using, or while you sleep.
Once unpacked and repacked, each rpm subdir contains:
src.tar.bz2 of all usr/src/debug files
usr/lib/debug/*.bz2 compressed in place
summary one line, counts
exec list of file names of each type
dyn
rel
archive
The summary file looks like:
src 40M->5.6M exec 18 dyn 272 rel 3 archive 17 bz2 60M->39M
That says 40M of usr/src/debug files (du -sh when unpacked) turned into
5.6M of src.tar.bz2 on disk; the rpm had 18 ET_EXEC files, 272 ET_DYN
files, 3 ET_REL files, 17 .a files; compressing each of those ELF files
took 60M of usr/lib/debug space down to 39M.
The summary file is for edification mainly, but machine parseable too. A
summary will include "src missing", "debug missing", and/or a "bogon" count
if something unusual happened during unpacking. You should look into any
of those.
The lists of files are used by the other scripts:
$ ./list-files.sh exec [rpm...]
or dyn/rel/archive or "norel" or "all" will spit the names of all the
appropriate files in the given rpms, or in the whole set if no rpm... args.
Now, to actually do some testing! So far we have:
$ ./single-file-test.sh [all|norel|etc] workdir testcmd ...
This runs the "testcmd ..." in isolation, once for each ELF file of the
given type(s). Each {} in the "testcmd ..." line is replaced with an
individual file name when it's run. Each test runs inside a subdirectory
created in workdir (can be any path, gets created as needed).
When a test run completes normally, its subdirectory is removed.
At the end of the run, the only things left in workdir/ will be
the subdirectories for cases that failed.
The testcmd is considered a success only if it exits with 0 status, sends
no output to its stdout or stderr, and leaves no files behind in its
current directory. Also note that the command arguments do not preserve
proper quoting. So if you need to get fancy just write a wrapper script
that needs only the file name argument.
The script's exit status is 0 for an orderly run regardless of test
failures. It only fails for some problem setting up the test runs, and
only such errors should show up on stderr. For each failure, the script
writes one line to stdout:
FAIL (status) foo-debuginfo-1.2-3.ppc/.../name dirname
Where (status) is (1) or (0) or whatever the exit status was, or is (files)
when it was 0 but extra files were left behind. A line with (0) means
there was something sent to stdout/stderr, which shows up in dirname/log.
The rpm/file name shows the file that broke the test. workdir/dirname/
contains whatever it left behind (e.g. core dumps), plus:
cmd exact command we ran
log stdout/stderr content
status 0/1/etc
name uncompressed ELF file (from foo-debuginfo.../name.bz2)
Concrete examples:
$ ./single-file-test.sh all elflint-test eu-elflint -d -q --gnu-ld {}
$ ./single-file-test.sh all elfcmp-test eu-elfcmp -q {} {}
---
Enjoy!
Please send all questions to <elfutils-devel(a)lists.fedorahosted.org>.
15 years, 3 months