Hi,
The last release had lots of new features. I don't have any grand plans for the next release. I was mostly going to spend time on bug fixing and cleanups. The following items is just what I personally wanted to spend some time on. I don't think everything below makes it for the next release (0.159 lets say somewhere in March). And I might have forgotten stuff. But it is probably what I will spend some time on next.
- Bugs, there are more, but these two were what I wanted to at least look at:
https://bugzilla.redhat.com/show_bug.cgi?id=1037051 elfutils FTBFS if "-Werror=format-security" flag is used Currently we just remove that flag, but if we can rewrite the code to enable it by default for all files that would be nice. Dunno if that is really possible in all cases though.
https://bugzilla.redhat.com/show_bug.cgi?id=1020842
libelf: segment fault on x86-64 while file's bss offset have a large number This slipped through the cracks, there has even been a patch proposed on the list: http://thread.gmane.org/gmane.comp.sysutils.elfutils.devel/3181
And lets see what else people report. https://bugzilla.redhat.com/buglist.cgi?bug_status=NEW&component=elfutil...
- Merge robustify commits to master if they make sense. And look at the portable branch patches suggested by Mike Frysinger https://lists.fedorahosted.org/pipermail/elfutils-devel/2014-January/003698....
- Move .gnu_debugaltlink handling from libdw to libdwfl and make ENABLE_DWZ the default. As discussed before: https://lists.fedorahosted.org/pipermail/elfutils-devel/2013-December/003626...
- string functions (this would be a new feature). I abandoned this since it was a lot of work for such a small feature and we were right before a release. But I keep writing code that would be much easier if we had string functions for the common DWARF constants. This would resurrect: https://git.fedorahosted.org/cgit/elfutils.git/log/?h=mjw/dwarfstrings
- speedup/cache __libdwfl_addrsym. This is the main reason eu-readelf (without -N) and to a lesser extend eu-stack (without -q) is so slow. They do address lookups without any caching, so they potentially need to search through all of them each time. It has been discussed before. https://lists.fedorahosted.org/pipermail/elfutils-devel/2013-July/003126.htm... Needs lots of testing to make sure the results returned are equal.
- Jan's idea to use /proc/PID/map_files when available seems very attractive: https://lists.fedorahosted.org/pipermail/elfutils-devel/2013-December/003637... As an alternative fallback we can try to reuse the technique we are already using in dwfl_linux_proc_find_elf for extracting the vdso through getting the in-memory ELF image from the target process.
I got this working through some ugly hacks. elf_from_remote_memory does work for the vdso, but not always for other images. That code needs to be cleaned up and we need to find a way to pass information about the target pid from the dwfl_linux_proc_report callback to the dwfl_linux_proc_find_elf callback (or maybe introduce new report and find_elf callbacks for this functionality).
- unwinder backends for aarch64 and arm7hl - aarch64 is mostly there on my branch https://git.fedorahosted.org/cgit/elfutils.git/log/?h=mjw/aarch64-unwind But needs actual testing and last time I played with it glibc didn't seem to properly mark _start and _clone causing the link register to point to the current pc and gcc didn't always seem to produce .eh_frame for all functions. So might need some interaction with the rest of the toolchain to make sure everything is correct. - arm32 might be a bit tricky. When there is a .debug_frame it should be simple. If not, then we might just give up for now. Or add a EXIDX to CFI translator fallback to the backend.
- Maybe tweak the defaults for eu-stack based on feedback. Maybe use more debug information than just the line numbers if available (e.g. use the subprogram or inlined_subroutine names for the addresses found).
Not all the above might make it before 0.159, and maybe some other stuff that I forgot about pops up, but it is a start and should keep me busy till March. And of course we can push out a new release earlier if some really nasty regressions/bugs are found and fixed before that.
Cheers,
Mark
On Wednesday 15 January 2014 10:16:34 Mark Wielaard wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1037051 elfutils FTBFS if "-Werror=format-security" flag is used Currently we just remove that flag, but if we can rewrite the code to enable it by default for all files that would be nice. Dunno if that is really possible in all cases though.
i'd argue that this is good justification for my --disable-werror patch. there are plenty of compiler versions & flags that people can enable on platforms that'll trigger warnings elfutils doesn't expect. users are left in the crappy scenario of either they don't get to build anything at all, or they have to manually edit the Makefiles to delete the -Werror flag. i don't really understand the desire to force -Werror on everyone all the time.
then bugs like this turn into a wish list item rather than FTB and can be evaluated as such. using crazy flags ? then pass in --disable-werror and we don't care. using reasonable flags ? then we'll look at fixing your report, but you can work around it in the mean time with --disable-werror. -mike
On Wed, 2014-01-15 at 10:33 -0500, Mike Frysinger wrote:
On Wednesday 15 January 2014 10:16:34 Mark Wielaard wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1037051 elfutils FTBFS if "-Werror=format-security" flag is used Currently we just remove that flag, but if we can rewrite the code to enable it by default for all files that would be nice. Dunno if that is really possible in all cases though.
i'd argue that this is good justification for my --disable-werror patch. there are plenty of compiler versions & flags that people can enable on platforms that'll trigger warnings elfutils doesn't expect. users are left in the crappy scenario of either they don't get to build anything at all, or they have to manually edit the Makefiles to delete the -Werror flag. i don't really understand the desire to force -Werror on everyone all the time.
then bugs like this turn into a wish list item rather than FTB and can be evaluated as such. using crazy flags ? then pass in --disable-werror and we don't care. using reasonable flags ? then we'll look at fixing your report, but you can work around it in the mean time with --disable-werror.
I do see your point, although I am unsure how to distinguish "crazy" from "non-crazy" compile flags. Also if people really insist on using crazy compile flags can't they just also add -Wno-error to CLFAGS themselves?
I do like to fix any non-crazy issues though. I just posted a patch to handle -Wformat=2 (which includes -Wformat-security) by default for all files now to solve the above bug. It is not the prettiest solution, but not terribly ugly either (nothing we didn't do before in other places, but we will see if someone yells and screams about the change).
Do you have any other examples (or just crazy examples to know what people out there really do)?
Thanks,
Mark
On Thursday, January 23, 2014 14:32:29 Mark Wielaard wrote:
On Wed, 2014-01-15 at 10:33 -0500, Mike Frysinger wrote:
On Wednesday 15 January 2014 10:16:34 Mark Wielaard wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1037051 elfutils FTBFS if "-Werror=format-security" flag is used Currently we just remove that flag, but if we can rewrite the code to enable it by default for all files that would be nice. Dunno if that is really possible in all cases though.
i'd argue that this is good justification for my --disable-werror patch. there are plenty of compiler versions & flags that people can enable on platforms that'll trigger warnings elfutils doesn't expect. users are left in the crappy scenario of either they don't get to build anything at all, or they have to manually edit the Makefiles to delete the -Werror flag. i don't really understand the desire to force -Werror on everyone all the time.
then bugs like this turn into a wish list item rather than FTB and can be evaluated as such. using crazy flags ? then pass in --disable-werror and we don't care. using reasonable flags ? then we'll look at fixing your report, but you can work around it in the mean time with --disable-werror.
I do see your point, although I am unsure how to distinguish "crazy" from "non-crazy" compile flags. Also if people really insist on using crazy compile flags can't they just also add -Wno-error to CLFAGS themselves?
browse Gentoo bugs and you'll find a wide variety of flags :)
you're right that adding -Wno-error themselves for exceptional cases should work since the build prepends the flag to the user's flags. but all the other major projects out there have standardized on --disable-werror which makes it a no brainer to utilize. elfutils does the right thing with CFLAGS, but there are lots of projects that do not, so you end up being hesitant to dig down into the build system with things like this. the configure flag takes all that away.
I do like to fix any non-crazy issues though. I just posted a patch to handle -Wformat=2 (which includes -Wformat-security) by default for all files now to solve the above bug. It is not the prettiest solution, but not terribly ugly either (nothing we didn't do before in other places, but we will see if someone yells and screams about the change).
i'm glad elfutils supports the extra -W flags. but it's a bit of a losing battle. gcc will always be introducing new -W flags, or changing how the existing ones behave, or adding their own false positives. it's admirable that the project tries to stay ahead of the curve (and it is certainly much more open than it was in the past which is great), but i don't think it's a battle that can be reasonable won all the time.
i have to support elfutils on every Gentoo arch with a variety of binutils (usually the last ~2), gcc (usually the last ~3), and glibc (usually the last ~3) versions. and elfutils itself is not always the latest release (since it has to undergo testing before it can be moved into the stable area). backporting warning fixes for every entry in that matrix is not worth anyone's time. Gentoo might have a bit more entries than most distros, but i doubt it is the only one to run into this problem.
Do you have any other examples (or just crazy examples to know what people out there really do)?
i see people building with -O0 -g and -flto and omit frame pointer and various -finline flags. and whatever the latest hotness is that gcc has released (like the new graphite stuff). those have a way of tickling warnings that aren't useful. -mike
On Wed, 15 Jan 2014 16:16:34 +0100, Mark Wielaard wrote:
- Maybe tweak the defaults for eu-stack based on feedback. Maybe use more debug information than just the line numbers if available (e.g. use the subprogram or inlined_subroutine names for the addresses found).
There is also libbacktrace in GCC. But it has its own DWARF parser/handling. libbacktrace needs just the raw PC unwound stack, it figures out DWARF and inlined frames on its own. At least Fedora does not package libbacktrace yet.
Jan
On Wed, 2014-01-15 at 21:47 +0100, Jan Kratochvil wrote:
On Wed, 15 Jan 2014 16:16:34 +0100, Mark Wielaard wrote:
- Maybe tweak the defaults for eu-stack based on feedback. Maybe use more debug information than just the line numbers if available (e.g. use the subprogram or inlined_subroutine names for the addresses found).
There is also libbacktrace in GCC. But it has its own DWARF parser/handling. libbacktrace needs just the raw PC unwound stack, it figures out DWARF and inlined frames on its own. At least Fedora does not package libbacktrace yet.
I forgot about that one. It is completely different from what we provide since it was designed to be used in-process and doesn't actually do unwinding (it just uses the system/libgcc _Unwind_Backtrace to get the raw call stack). Since it is in-process and could be called from a signal handler it also needs to do everything in a signal safe way. But that just means that whatever they do, we can do but for different situations. I'll take a look to see if they use/provide the DWARF information in an interesting way that we could steal.
Cheers,
Mark
On Wed, 15 Jan 2014 22:55:27 +0100, Mark Wielaard wrote:
I forgot about that one. It is completely different from what we provide since it was designed to be used in-process and doesn't actually do unwinding (it just uses the system/libgcc _Unwind_Backtrace to get the raw call stack). Since it is in-process and could be called from a signal handler it also needs to do everything in a signal safe way.
While GCC uses it for self-unwind via backtrace_full()->_Unwind_Backtrace() libbacktrace is not bound to such use.
libbacktrace provides also backtrace_pcinfo() where one can feed arbitrary sequence of PC addresses. Also backtrace_create_state() can specify arbitrary ELF filename (and not just the default "/proc/self/exe").
I do not see now how to specify different filenames for different PCs during the backtrace but that could be at least easily patched/improved.
I do not push for libbacktrace but I still see it as a valid option.
Jan
On Sat, 2014-01-18 at 21:18 +0100, Jan Kratochvil wrote:
On Wed, 15 Jan 2014 22:55:27 +0100, Mark Wielaard wrote:
I forgot about that one. It is completely different from what we provide since it was designed to be used in-process and doesn't actually do unwinding (it just uses the system/libgcc _Unwind_Backtrace to get the raw call stack). Since it is in-process and could be called from a signal handler it also needs to do everything in a signal safe way.
While GCC uses it for self-unwind via backtrace_full()->_Unwind_Backtrace() libbacktrace is not bound to such use.
It certainly looks like it is.
libbacktrace provides also backtrace_pcinfo() where one can feed arbitrary sequence of PC addresses.
Yes, backtrace_pcinfo () is like elfutils dwarf_getscopes[_die] for the current executable. But it caches all scopes and it has a more familiar callback interface to iterate the DIEs making memory management slightly easier. I'll add something like that to eu-stack to show inlined frames. It is a nice idea.
Also backtrace_create_state() can specify arbitrary ELF filename (and not just the default "/proc/self/exe").
The given filename should be the name of the current executable, normally argv[0], if NULL it will use getexecname () or /proc/self/exe. It is used for printing error messages and when dl_phdr_info doesn't provide the module name.
I do not see now how to specify different filenames for different PCs during the backtrace but that could be at least easily patched/improved.
There is none, it uses dl_iterate_phdr and only works for the current executable. I am sure that if you really wanted you could add support for other dwfl_report like mechanisms, support 32-on-64 bit, separate debuginfo through debug_link or build-id elf lookups, core file support, etc. It might even make sense to use libdwfl for that. But I don't understand why you want to rearchitecturing libbacktrace like that.
IMHO it seems easier to steal any missing functionality (DWARF debuginfo function name lookups and inline frames) and add it to eu-stack directly using libdw. Which seem easy, we just need to decide whether or not we also want to cache the results somewhere.
I do not push for libbacktrace but I still see it as a valid option.
It is a valid option if you need to generate backtraces with names in a language runtime or your current executable that needs to be (mostly) signal safe and don't need the flexibility that elfutils provides. I would certainly recommend it for that.
Cheers,
Mark
elfutils-devel@lists.fedorahosted.org