Hi,
While working on pahole and the perf tools that now ship in the Linux kernel sources I noticed this:
[acme@doppio pahole]$ size object_samples/zweinberg@mozilla.com/libgklayout.so text data bss dec hex filename 12135799 1088572 29152 13253523 ca3b93 object_samples/zweinberg@mozilla.com/libgklayout.so [acme@doppio pahole]$ readelf -SW object_samples/zweinberg@mozilla.com/libgklayout.so | grep debug [26] .debug_aranges PROGBITS 0000000000000000 ca2d50 1d1f80 00 0 0 16 [27] .debug_pubnames PROGBITS 0000000000000000 e74cd0 5d29ed 00 0 0 1 [28] .debug_info PROGBITS 0000000000000000 14476bd 431ca1e 00 0 0 1 [29] .debug_abbrev PROGBITS 0000000000000000 57640db 1999c5 00 0 0 1 [30] .debug_line PROGBITS 0000000000000000 58fdaa0 58281c 00 0 0 1 [31] .debug_frame PROGBITS 0000000000000000 5e802c0 4454f0 00 0 0 8 [32] .debug_str PROGBITS 0000000000000000 62c57b0 571e13 01 MS 0 0 1 [33] .debug_loc PROGBITS 0000000000000000 68375c3 131493a 00 0 0 1 [34] .debug_ranges PROGBITS 0000000000000000 7b4bf00 2e7130 00 0 0 16 [acme@doppio pahole]$ [acme@doppio pahole]$ perf record -f -- pahole object_samples/zweinberg@mozilla.com/libgklayout.so > /dev/null [ perf record: Captured and wrote 10.779 MB perf.data (~470950 samples) ] [acme@doppio pahole]$ perf report --sort comm,dso,symbol | head -25
# # (470890 samples) # # Overhead Command Shared Object Symbol # ........ ................ ......................... ...... # 13.51% pahole /usr/lib64/libdw-0.141.so [.] relocate.6672 10.61% pahole /lib64/libc-2.10.1.so [.] __GI_strcmp 8.84% pahole /lib64/libc-2.10.1.so [.] _int_malloc 8.07% pahole /usr/lib64/libdw-0.141.so [.] __dwfl_linux_kernel_report_modules_internal 7.84% pahole /lib64/libc-2.10.1.so [.] __tsearch 5.51% pahole ./build/libdwarves.so.1.0.0 [.] tag__recode_dwarf_type 5.19% pahole /lib64/libc-2.10.1.so [.] malloc_consolidate 4.41% pahole ./build/libdwarves.so.1.0.0 [.] namespace__recode_dwarf_types 3.61% pahole /usr/lib64/libdw-0.141.so [.] __libdwfl_report_elf 3.47% pahole ./build/libdwarves.so.1.0.0 [.] list__for_all_tags 2.49% pahole /lib64/libc-2.10.1.so [.] _int_free 2.07% pahole /lib64/libc-2.10.1.so [.] __tfind 1.89% pahole /lib64/libc-2.10.1.so [.] __GI___libc_free 1.74% pahole ./build/libdwarves.so.1.0.0 [.] cu__table_add_tag 1.60% pahole /usr/lib64/libdw-0.141.so [.] check_notes 1.35% pahole /lib64/libc-2.10.1.so [.] __GI_vfprintf 0.96% pahole /lib64/libc-2.10.1.so [.] __malloc 0.88% pahole ./build/libdwarves.so.1.0.0 [.] __die__process_tag [acme@doppio pahole]$
Relocation is being always performed, but I don't need this, is there a way for me to ask libdwfl to not perform relocations?
- Arnaldo
Relocation is being always performed, but I don't need this, is there a way for me to ask libdwfl to not perform relocations?
There are many layers of answer to give here. So I'll just spew out some facts, and you can either give more context about your usage or ask more specific questions from there.
Relocation is only relevant to ET_REL files, of which in normal practice you'll only encounter .ko files for Linux kernel modules.
From what you cited, I don't know why you are looking at any Linux kernel
stuff at all in that use case. If you are just examining some normal DSO file offline, then I can't imagine why you ever called dwfl_linux_kernel* at all. Certainly you shouldn't be doing that on the same Dwfl object to which you report that DSO file as a module, they are not sharing an address/symbol space.
Your trace indicates you called dwfl_linux_kernel_report_modules. This does not cause any modules' DWARF files to be loaded at all, by itself. It just reads /proc/modules and registers each module in the Dwfl. You don't load any ELF/DWARF files at all until you ask for one.
If you were instead using dwfl_linux_kernel_report_offline, then it might make (a little) more sense. That does the equivalent of: find /lib/modules/`uname -r` -name '*.ko' and adds each of those as an offline file with dwfl_report_offline. That is indeed a little slow (opens Elf handles on all those files), but still it does not look at any DWARF. (To address just the file opening overhead there you can use the PREDICATE parameter to avoid looking at what you don't care about.)
So, clearly you are actually getting a Dwarf handle for each module from libdwfl. I have no idea why you do that. This is what instigates the relocation work. There is no way to get a usable DWARF handle for any ET_REL file and avoid that relocation.
Finally, I'll mention that a future version of libdwfl will handle ET_REL files differently. Instead of today's eager relocation of whole sections before getting a Dwarf handle, libdw will do individual relocation adjustments as needed while reading the DWARF data. This will reduce the startup time to just get the Dwarf handle to similar to that for non-ET_REL files, at the cost of higher overheads at fine grain when you actually use individual pieces of DWARF data from an ET_REL file.
But it sure sounds like you are just asking explicitly for useless work that you never need.
Thanks, Roland
Em Wed, Jun 24, 2009 at 03:28:57PM -0700, Roland McGrath escreveu:
Relocation is being always performed, but I don't need this, is there a way for me to ask libdwfl to not perform relocations?
There are many layers of answer to give here. So I'll just spew out some facts, and you can either give more context about your usage or ask more specific questions from there.
Relocation is only relevant to ET_REL files, of which in normal practice you'll only encounter .ko files for Linux kernel modules.
For me it is not relevant at all, I just need the type definitions and function parameter list, variable name/type, etc. Only in very specific cases I want addresses, and even then no need for relocations in most cases.
From what you cited, I don't know why you are looking at any Linux kernel
stuff at all in that use case. If you are just examining some normal DSO file offline, then I can't imagine why you ever called dwfl_linux_kernel* at all. Certainly you shouldn't be doing that on the same Dwfl object to which you report that DSO file as a module, they are not sharing an address/symbol space.
Well, I'm not using those functions at all:
[acme@doppio pahole]$ grep dwfl_linux_kernel * [acme@doppio pahole]$
Your trace indicates you called dwfl_linux_kernel_report_modules. This does not cause any modules' DWARF files to be loaded at all, by itself. It just reads /proc/modules and registers each module in the Dwfl. You don't load any ELF/DWARF files at all until you ask for one.
Probably thru one of the many callbacks and defaults in elfutils I may well end up using something I haven't asked explicitely.
What I'm using that has a dwfl_ prefix is:
[acme@doppio pahole]$ grep dwfl_ *.[ch] dwarf_loader.c: int build_id_len = dwfl_module_build_id(mod, &build_id, &vaddr); dwarf_loader.c: Elf *elf = dwfl_module_getelf(dwflmod, &dwflbias); dwarf_loader.c: Dwarf *dw = dwfl_module_getdwarf(dwflmod, &dwbias); dwarf_loader.c: * __func__, dwfl_errmsg(-1)); dwarf_loader.c: /* Duplicate an fd for dwfl_report_offline to swallow. */ dwarf_loader.c: int dwfl_fd = dup(fd); dwarf_loader.c: if (dwfl_fd < 0) dwarf_loader.c: .section_address = dwfl_offline_section_address, dwarf_loader.c: .find_debuginfo = dwfl_standard_find_debuginfo, dwarf_loader.c: .find_elf = dwfl_build_id_find_elf, dwarf_loader.c: Dwfl *dwfl = dwfl_begin(&callbacks); dwarf_loader.c: if (dwfl_report_offline(dwfl, filename, filename, dwfl_fd) == NULL) dwarf_loader.c: dwfl_report_end(dwfl, NULL, NULL); dwarf_loader.c: dwfl_getmodules(dwfl, cus__process_dwflmod, &parms, 0); dwarf_loader.c: dwfl_end(dwfl); dwarves.h: self->cached_symtab_nr_entries = dwfl_module_getsymtab(self->dwfl); dwarves.h: name = dwfl_module_getsym(cu->dwfl, id, &sym, NULL); \ dwarves.h: ++id, name = dwfl_module_getsym(cu->dwfl, id, &sym, NULL)) [acme@doppio pahole]$
I'll look at the dwfl internals to figure out where is that it ends up using dwfl_linux_kernel_ prefixed routines...
If you were instead using dwfl_linux_kernel_report_offline, then it might make (a little) more sense. That does the equivalent of: find /lib/modules/`uname -r` -name '*.ko' and adds each of those as an offline file with dwfl_report_offline. That is indeed a little slow (opens Elf handles on all those files), but still it does not look at any DWARF. (To address just the file opening overhead there you can use the PREDICATE parameter to avoid looking at what you don't care about.)
So, clearly you are actually getting a Dwarf handle for each module from libdwfl. I have no idea why you do that. This is what instigates the relocation work. There is no way to get a usable DWARF handle for any ET_REL file and avoid that relocation.
Finally, I'll mention that a future version of libdwfl will handle ET_REL files differently. Instead of today's eager relocation of whole sections before getting a Dwarf handle, libdw will do individual relocation adjustments as needed while reading the DWARF data. This will reduce the startup time to just get the Dwarf handle to similar to that for non-ET_REL files, at the cost of higher overheads at fine grain when you actually use individual pieces of DWARF data from an ET_REL file.
But it sure sounds like you are just asking explicitly for useless work that you never need.
Well, as you can see above, I'm not. But it may well be that something like dwfl_standard_find_debuginfo ends up pointing to kernel specific routines, I'll check that in the elfutils sources.
- Arnaldo
For me it is not relevant at all, I just need the type definitions and function parameter list, variable name/type, etc. Only in very specific cases I want addresses, and even then no need for relocations in most cases.
If you are dealing with an ET_REL file on an SHT_RELA architecture, then you need relocation of the pointers inside and between the different DWARF sections. In short, you can't handle an ET_REL file at all without doing some kind of relocation.
I'll look at the dwfl internals to figure out where is that it ends up using dwfl_linux_kernel_ prefixed routines...
Well, just set a breakpoint and look at the backtrace from your code. The only implicit calls to any such things in libdwfl come from the argp callbacks for -k/-K options.
Thanks, Roland
Em Sun, Jun 28, 2009 at 02:07:35PM -0700, Roland McGrath escreveu:
For me it is not relevant at all, I just need the type definitions and function parameter list, variable name/type, etc. Only in very specific cases I want addresses, and even then no need for relocations in most cases.
If you are dealing with an ET_REL file on an SHT_RELA architecture, then you need relocation of the pointers inside and between the different DWARF sections. In short, you can't handle an ET_REL file at all without doing some kind of relocation.
I'll look at the dwfl internals to figure out where is that it ends up using dwfl_linux_kernel_ prefixed routines...
Well, just set a breakpoint and look at the backtrace from your code. The only implicit calls to any such things in libdwfl come from the argp callbacks for -k/-K options.
Nevermind, it was a bug in the kernel perf tools, getting confused with prelinked libraries :-\
I fixed it and now it makes more sense and its in line with what oprofile reports:
[acme@doppio pahole]$ perf report --sort comm,dso,symbol --dsos /usr/lib64/libdw-0.141.so | head -20
# # (135909 samples) # # Overhead Command Shared Object Symbol # ........ ................ ......................... ...... # 43.39% pahole /usr/lib64/libdw-0.141.so [.] __libdw_find_attr 27.16% pahole /usr/lib64/libdw-0.141.so [.] __libdw_form_val_len 5.79% pahole /usr/lib64/libdw-0.141.so [.] lookup 4.99% pahole /usr/lib64/libdw-0.141.so [.] findcu_cb 4.14% pahole /usr/lib64/libdw-0.141.so [.] __dwarf_attr_internal 2.14% pahole /usr/lib64/libdw-0.141.so [.] __dwarf_siblingof_internal 1.13% pahole /usr/lib64/libdw-0.141.so [.] __libdw_findcu 0.98% pahole /usr/lib64/libdw-0.141.so [.] __libdw_get_uleb128 0.97% pahole /usr/lib64/libdw-0.141.so [.] dwarf_tag 0.94% pahole /usr/lib64/libdw-0.141.so [.] __libdw_formref 0.87% pahole /usr/lib64/libdw-0.141.so [.] __dwarf_formsdata_internal 0.83% pahole /usr/lib64/libdw-0.141.so [.] dwarf_decl_file 0.79% pahole /usr/lib64/libdw-0.141.so [.] dwarf_getsrclines [acme@doppio pahole]$
Sorry for the noise,
- Arnaldo
elfutils-devel@lists.fedorahosted.org