Hi list,
it has been recently proposed to switch openSUSE to run ldconfig via a %transfiletriggerin/-un scriptlet instead of manually in %post & %postun the same way as Fedora does it at the moment.
However, an interesting issue has been raised: what happens if package A gets upgraded in the same transaction as package B, but B needs A during the upgrade. A will install a new shared library, but ldconfig will run after B has already tried to upgrade.
Do we have some mechanism to prevent this? Or does this essentially never happen in practice?
The only "workaround" that came up was to use ordinary file triggers, but that is not a great idea either these would fire way too frequently (as mentioned by Florian Weimer on this list in 2018: [1]).
Cheers,
Dan
Footnotes: [1] https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/...
* Dan Čermák:
it has been recently proposed to switch openSUSE to run ldconfig via a %transfiletriggerin/-un scriptlet instead of manually in %post & %postun the same way as Fedora does it at the moment.
However, an interesting issue has been raised: what happens if package A gets upgraded in the same transaction as package B, but B needs A during the upgrade. A will install a new shared library, but ldconfig will run after B has already tried to upgrade.
Fedora ships the soname links in the package, addressing this issue. For example, openssl-libs contains:
/usr/lib64/libcrypto.so.1.1 /usr/lib64/libcrypto.so.1.1.1k /usr/lib64/libssl.so.1.1 /usr/lib64/libssl.so.1.1.1k
As a result, running ldconfig is purely an optimization.
There are some packages that install fails into /etc/ld.so.conf.d, and such packages still have to run ldconfig explicitly. (We should perhaps move those shared objects to /usr/lib64, too.)
Thanks, Florian
On Wed, Sep 01, 2021 at 11:01:01AM +0200, Florian Weimer wrote:
- Dan Čermák:
it has been recently proposed to switch openSUSE to run ldconfig via a %transfiletriggerin/-un scriptlet instead of manually in %post & %postun the same way as Fedora does it at the moment.
However, an interesting issue has been raised: what happens if package A gets upgraded in the same transaction as package B, but B needs A during the upgrade. A will install a new shared library, but ldconfig will run after B has already tried to upgrade.
Fedora ships the soname links in the package, addressing this issue. For example, openssl-libs contains:
/usr/lib64/libcrypto.so.1.1 /usr/lib64/libcrypto.so.1.1.1k /usr/lib64/libssl.so.1.1 /usr/lib64/libssl.so.1.1.1k
As a result, running ldconfig is purely an optimization.
There are some packages that install fails into /etc/ld.so.conf.d, and such packages still have to run ldconfig explicitly. (We should perhaps move those shared objects to /usr/lib64, too.)
Did anyone measure if ldconfig caching is actually useful on modern machines?
Rich.
* Richard W. M. Jones:
On Wed, Sep 01, 2021 at 11:01:01AM +0200, Florian Weimer wrote:
There are some packages that install f[ile]s into /etc/ld.so.conf.d, and such packages still have to run ldconfig explicitly. (We should perhaps move those shared objects to /usr/lib64, too.)
Did anyone measure if ldconfig caching is actually useful on modern machines?
It's not actually a cache in the strictest sense. Shared objects installed into directories listed in /etc/ld.so.conf, /etc/ld.so.conf.d can only be loaded after running ldconfig. The dynamic loader does not parse these configuration files.
On some architectures (s390x in particular), hundreds of additional system calls are needed just for loading just libc.so.6. These are all failing open calls. The kernel does not have a negative dentry cache, so I expect them to be quite slow. There are glibc plans to reduce this to a more manageable count (perhaps ~20 system calls) by removing the legacy HWCAPS mechanism, but this hasn't happened yet (the feature was deprecated in 2.33 only).
Thanks, Florian
On Wed, Sep 01, 2021 at 11:01:01AM +0200, Florian Weimer wrote:
- Dan Čermák:
it has been recently proposed to switch openSUSE to run ldconfig via a %transfiletriggerin/-un scriptlet instead of manually in %post & %postun the same way as Fedora does it at the moment.
However, an interesting issue has been raised: what happens if package A gets upgraded in the same transaction as package B, but B needs A during the upgrade. A will install a new shared library, but ldconfig will run after B has already tried to upgrade.
Fedora ships the soname links in the package, addressing this issue. For example, openssl-libs contains:
/usr/lib64/libcrypto.so.1.1 /usr/lib64/libcrypto.so.1.1.1k /usr/lib64/libssl.so.1.1 /usr/lib64/libssl.so.1.1.1k
As a result, running ldconfig is purely an optimization.
There are some packages that install fails into /etc/ld.so.conf.d, and such packages still have to run ldconfig explicitly. (We should perhaps move those shared objects to /usr/lib64, too.)
It has always struck me that ldconfig's impl isn't a great match for RPMs use case. IIUC ldconfig scans all files in all registered library directory paths, since it doesn't know what files might have changed since its last run (or does it check mtime at all to optimize work it does?) RPM meanwhile definitely knows exactly what files it has just created/deleted/modified and thus need ldconfig processing.
Is there scope for optimizing ldconfig, or provide an alternative impl, that is able to process /only/ the libraries that RPM has just touched and thus not scan any other files or directories ? ie make it a non-issue if ldconfig is invoked many times, by ensuring each invokation would only do a tiny bit of work.
Regards, Daniel
* Daniel P. Berrangé:
It has always struck me that ldconfig's impl isn't a great match for RPMs use case. IIUC ldconfig scans all files in all registered library directory paths, since it doesn't know what files might have changed since its last run (or does it check mtime at all to optimize work it does?)
It has a real auxiliary cache in /var/cache/ldconfig and uses the mtime to check if it is up to date.
RPM meanwhile definitely knows exactly what files it has just created/deleted/modified and thus need ldconfig processing.
There's also the matter of load path shadowing, so it's easier to re-scan everything to deal with removals that reveal other files.
Is there scope for optimizing ldconfig, or provide an alternative impl, that is able to process /only/ the libraries that RPM has just touched and thus not scan any other files or directories ?
I don't think ldconfig processing time is currently an issue for Fedora.
Thanks, Florian
Hi Florian,
Florian Weimer fweimer@redhat.com writes:
- Dan Čermák:
it has been recently proposed to switch openSUSE to run ldconfig via a %transfiletriggerin/-un scriptlet instead of manually in %post & %postun the same way as Fedora does it at the moment.
However, an interesting issue has been raised: what happens if package A gets upgraded in the same transaction as package B, but B needs A during the upgrade. A will install a new shared library, but ldconfig will run after B has already tried to upgrade.
Fedora ships the soname links in the package, addressing this issue. For example, openssl-libs contains:
/usr/lib64/libcrypto.so.1.1 /usr/lib64/libcrypto.so.1.1.1k /usr/lib64/libssl.so.1.1 /usr/lib64/libssl.so.1.1.1k
As a result, running ldconfig is purely an optimization.
Would that still be the case when the SONAME is bumped? I would assume that the old symlink is no longer valid. Or is that also not an issue because we rebuild the dependency on a SONAME bump?
Thanks,
Dan
On Wed, Sep 1, 2021 at 10:45 AM Dan Čermák dan.cermak@cgc-instruments.com wrote:
Hi Florian,
Florian Weimer fweimer@redhat.com writes:
- Dan Čermák:
it has been recently proposed to switch openSUSE to run ldconfig via a %transfiletriggerin/-un scriptlet instead of manually in %post & %postun the same way as Fedora does it at the moment.
However, an interesting issue has been raised: what happens if package A gets upgraded in the same transaction as package B, but B needs A during the upgrade. A will install a new shared library, but ldconfig will run after B has already tried to upgrade.
Fedora ships the soname links in the package, addressing this issue. For example, openssl-libs contains:
/usr/lib64/libcrypto.so.1.1 /usr/lib64/libcrypto.so.1.1.1k /usr/lib64/libssl.so.1.1 /usr/lib64/libssl.so.1.1.1k
As a result, running ldconfig is purely an optimization.
Would that still be the case when the SONAME is bumped? I would assume that the old symlink is no longer valid. Or is that also not an issue because we rebuild the dependency on a SONAME bump?
Since this happens at package build time via a buildroot policy script[1], this is seamlessly handled.
If we want to add this to rpm-config-SUSE, I could add it...
[1]: https://src.fedoraproject.org/rpms/redhat-rpm-config/blob/rawhide/f/brp-ldco...
* Dan Čermák:
Hi Florian,
Florian Weimer fweimer@redhat.com writes:
- Dan Čermák:
it has been recently proposed to switch openSUSE to run ldconfig via a %transfiletriggerin/-un scriptlet instead of manually in %post & %postun the same way as Fedora does it at the moment.
However, an interesting issue has been raised: what happens if package A gets upgraded in the same transaction as package B, but B needs A during the upgrade. A will install a new shared library, but ldconfig will run after B has already tried to upgrade.
Fedora ships the soname links in the package, addressing this issue. For example, openssl-libs contains:
/usr/lib64/libcrypto.so.1.1 /usr/lib64/libcrypto.so.1.1.1k /usr/lib64/libssl.so.1.1 /usr/lib64/libssl.so.1.1.1k
As a result, running ldconfig is purely an optimization.
Would that still be the case when the SONAME is bumped? I would assume that the old symlink is no longer valid. Or is that also not an issue because we rebuild the dependency on a SONAME bump?
The symbolic link will be updated once the package is unpacked, along with the real file. For /usr/lib64, the glibc dynamic loader will use files there even if they are not in the cache, as long the soname is available there.
Once symbolic links are involved, soname changes are more problematic with ldconfig, actually. RPM does not delete old files immediately. If ldconfig runs with the old files still around, it may adjust the soname symbolic links to point to the wrong version of the library. That is why we eliminated the symbolic link indirection in glibc 2.34.
Thanks, Florian