On 10/6/20 3:59 PM, Mark Wielaard wrote:
Hi,
Changing subject because this has nothing to do with that Change
Proposal anymore.
On Tue, Oct 06, 2020 at 01:49:13PM -0600, Jeff Law wrote:
> However, I think it's perfectly valid to discuss zstd if folks wanted to
> change the compression scheme for debug sections. In fact, I'd claim
> sticking with zlib, gzip, bzip2, xz, 7z, etc is unwise. The world has
> moved and zstd seems like the place we should be. In fact, we use it
> for various things within GCC already.
Personally I must admit that I am not really a fan of using ELF
section/file compression. It makes it impossible to simply mmap the
data in or to quickly read just a tiny bit because you first have to
decompress (and allocate new memory) for it. IMHO .debug files are no
different from other ELF files for which we would also not do this. We
can just use rpm package compression to reduce the distro distribution
size, but we should not (re)compress the install/on-disk files. That
will just mean programs will create an extra cache of uncompressed
files they need to consult frequently.
I'm not taking a position on whether or not we compress sections. My
position is that if we're compressing them, then zstd seems like a
better solution than the others mentioned. I certainly understand the
desire to just mmap in the stuff and move on.
[ ... ]
Secondly you can use ELF section compression. The ELF spec leaves room
for adding new compression algorithms. The Chdr struct(s) contain a
ch_type which describes the algorithm. Currently only one is
specified, but there is a lot of room for expansion:
/* Legal values for ch_type (compression algorithm). */
#define ELFCOMPRESS_ZLIB 1 /* ZLIB/DEFLATE algorithm. */
#define ELFCOMPRESS_LOOS 0x60000000 /* Start of OS-specific. */
#define ELFCOMPRESS_HIOS 0x6fffffff /* End of OS-specific. */
#define ELFCOMPRESS_LOPROC 0x70000000 /* Start of processor-specific. */
#define ELFCOMPRESS_HIPROC 0x7fffffff /* End of processor-specific. */
So you could propose something on gnu-gabi(a)sourceware.org for a GNU
extension or at generic-abi(a)googlegroups.com for a generic ELF one.
And then get the ELF processing tools to adopt the new compression
type.
ohhh, I didn't know it was baked in at this level. Yea, if we're going
to do section compression with zstd, then it's clearly best to get it
officially supported at the ABI level.
jeff