https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Bug ID: 2019329 Summary: nosync.so is linked against GLIBC_2.34 symbols, it breaks mock Product: Fedora Version: 35 Status: NEW Component: nosync Assignee: mizdebsk@redhat.com Reporter: praiskup@redhat.com QA Contact: extras-qa@fedoraproject.org CC: java-sig-commits@lists.fedoraproject.org, mizdebsk@redhat.com Target Milestone: --- Classification: Fedora
I'm not sure what changed, but the new dependency is breaking Mock builds for older distributions (e.g. even Fedora 34 doesn't provide 2.34 symbols).
$ cat ~/.config/mock.cfg | grep nosync config_opts['nosync'] = True $ mock -r fedora-34-x86_64 --shell ... Running scriptlet: ca-certificates-2021.2.50-1.0.fc34.noarch
83/134 /bin/sh: /lib64/libc.so.6: version `GLIBC_2.34' not found (required by /var/tmp/tmp.mock.khvcffv5/lib64/nosync.so) error: %prein(ca-certificates-2021.2.50-1.0.fc34.noarch) scriptlet failed, exit status 1
Error in PREIN scriptlet in rpm package ca-certificates Installing : krb5-libs-1.19.2-2.fc34.x86_64
84/134 error: ca-certificates-2021.2.50-1.0.fc34.noarch: install failed ... Failed: ca-certificates-2021.2.50-1.0.fc34.noarch tpm2-tss-3.1.0-1.fc34.x86_64
Error: Transaction failed
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #1 from Tom Hughes tom@compton.nu --- There are two GLIBC_2.34 symbols that nosync.so is requiring:
0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34 pthread_testcancel 0000000000000000 DF *UND* 0000000000000000 GLIBC_2.34 dlsym
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Mikolaj Izdebski mizdebsk@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Keywords| |Triaged
--- Comment #2 from Mikolaj Izdebski mizdebsk@redhat.com --- I can reproduce this issue in mock with F34 chroot on F35 x86_64 host.
Package versions (on the F35 host): mock-2.12-2.fc35.noarch nosync-1.1-11.fc35.x86_64
Reproducer command: mock -r rep.cfg --shell
Mock config: config_opts['basedir'] = '/tmp/nosync-rep' config_opts['cache_topdir'] = '/tmp/nosync-rep' config_opts["rpmbuild_networking"] = False config_opts["use_host_resolv"] = False config_opts["isolation"] = "simple" config_opts["use_bootstrap"] = False config_opts["plugin_conf"]["yum_cache_enable"] = False config_opts["plugin_conf"]["root_cache_enable"] = False config_opts["plugin_conf"]["package_state_enable"] = False config_opts["nosync"] = True config_opts["nosync_force"] = True config_opts["root"] = "nosync-rep" config_opts["target_arch"] = "x86_64" config_opts["chroot_setup_cmd"] = "install shadow-utils"
config_opts["yum.conf"] = """ [main] keepcache=0 debuglevel=2 reposdir=/dev/null gpgcheck=0 assumeyes=1 install_weak_deps=0 metadata_expire=-1
[f34] name=f34 baseurl=https://ftp.icm.edu.pl/pub/Linux/dist/fedora/linux/releases/34/Everything/x8... enabled=1 """
Actual result - several scriplet failures during mock chroot setup - mock fails with the following error: ERROR: Command failed: # /usr/sbin/groupadd -g 135 mock
Disabling nosync in mock fixes the issue.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #3 from Pavel Raiskup praiskup@redhat.com ---
Disabling nosync in mock fixes the issue.
Yes. Well: s/fixes the issue/makes the build succeed/ because we don't want to disable nosync in mock :-)
It seems that nosync.so+mock always worked this way (nosync from host used inside the target-chroot). It's weird that this problem is happening for the first time right now.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #4 from Tom Hughes tom@compton.nu --- Well mock.so uses very few symbols from libc so the risk is low but glibc 2.34 has merged libdl into libc which means all dl* symbols have new versions as they are now in libc and dlsym is the one function that mock.so does have to use...
There's also a pthread symbol being referenced somehow which suffers from the same problem because libpthread has been merged into libc.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Mikolaj Izdebski mizdebsk@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Mikolaj Izdebski mizdebsk@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Severity|unspecified |high Priority|unspecified |high
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #5 from Pavel Raiskup praiskup@redhat.com --- Is there some plan for this issue? What would happen if we built the nosync module statically?
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #6 from Tom Hughes tom@compton.nu --- Is this bug even in the right place? I mean nosync hasn't really done anything wrong - would it be possible for mock to add nosync to the buildroot (so that it comes from the right distribution) instead of injecting it from outside?
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #7 from Pavel Raiskup praiskup@redhat.com ---
Is this bug even in the right place?
I believe it is. That's how it was designed.
would it be possible for mock to add nosync to the buildroot
Mock has been using nocache.so for _all_ chroots (when turned on). But it would be hard (or even impossible) to make the plugin available inside all the supported chroots (consider also third-party configs people are using daily on their own).
Technically it is possible. Not a 5-minute work, but possible.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #8 from Mikolaj Izdebski mizdebsk@redhat.com --- The bug is correctly reported against nosync. I need to find time to work on the issue. Thanks for bumping the bug.
My current plan is to try to link nosync against oldest versioned glibc symbols. I think that on older systems unversioned symbols from libdl/libpthread should be able to satisfy versioned requirements, but I'll need to confirm that. In general, I'll need to explore how exactly symbol versioning in glibc works.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #9 from Tom Hughes tom@compton.nu --- I don't think that is possible sadly - as far as I know the only way to do that is to have an older version of glibc-devel that has the old versions as the primary names.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Kevin Fenzi kevin@scrye.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |kevin@scrye.com
--- Comment #10 from Kevin Fenzi kevin@scrye.com --- Just upgrading builders to f35 and hit this. ;(
I guess I will disable nosync for now until we sort it out...
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #11 from Pavel Raiskup praiskup@redhat.com --- This patch could help to link against 2.2.5 symbols: https://copr-dist-git.fedorainfracloud.org/cgit/praiskup/nosync-libc-2.2.5/n...
Though this fails in i386, and this library needs to be built multilib: https://copr.fedorainfracloud.org/coprs/praiskup/nosync-libc-2.2.5/build/304...
.... cc -shared -fPIC -ldl -lpthread -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1 -fstack-protector-strong -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m32 -march=i686 -mtune=generic -msse2 -mfpmath=sse -mstackrealign -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -o nosync.so fsync.o open.o /usr/bin/ld: nosync.so: no symbol version section for versioned symbol `dlsym@GLIBC_2.2.5' collect2: error: ld returned 1 exit status
Even if the glibc versioning on i386 was fixed (or whatever is causing this), this doesn't seem to be a long-term solution?
It would be nice if nosync was a glibc subpackage with statically linked dlsym() and pthread_testcancel(). But this is probably unrealistic.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Pavel Raiskup praiskup@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Blocks| |2036722
Referenced Bugs:
https://bugzilla.redhat.com/show_bug.cgi?id=2036722 [Bug 2036722] nosync.so is linked against GLIBC_2.34 symbols, breaking mock for older chroots
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Miroslav Suchý msuchy@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |msuchy@redhat.com
--- Comment #12 from Miroslav Suchý msuchy@redhat.com --- Ping. Any progress here?
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #13 from Nicolas Chauvet (kwizart) kwizart@gmail.com --- @pavel, Have you reproduced this build error only on copr or also in koji scratch build ? (is it related to how koji provide the glibc32 package ?)
Can we have this fix conditionally applied on x86_64 at least ? EPEL doesn't build i686 packages, so we might not hit the issue there.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #14 from Nicolas Chauvet (kwizart) kwizart@gmail.com --- Side note:
readelf -a /usr/lib/libc.so.6 |grep dlsym 3259: 00087340 192 FUNC GLOBAL DEFAULT 15 dlsym@GLIBC_2.0 3261: 00087340 192 FUNC GLOBAL DEFAULT 15 dlsym@@GLIBC_2.34 706: 00087310 33 FUNC LOCAL DEFAULT 15 dlsym_doit 2421: 0015b9e0 75 FUNC LOCAL DEFAULT 15 do_dlsym 2880: 0015bb30 169 FUNC LOCAL DEFAULT 15 __libc_dlsym 4241: 00087340 192 FUNC LOCAL DEFAULT 15 ___dlsym 7007: 00087340 192 FUNC GLOBAL DEFAULT 15 dlsym@GLIBC_2.0 8429: 00087340 192 FUNC GLOBAL DEFAULT 15 dlsym@@GLIBC_2.34
So for x86, the symbol version is 2.0, not 2.2.5 (it might be different for others architectures).
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #15 from Ben Cotton bcotton@redhat.com --- This message is a reminder that Fedora Linux 35 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 35 on 2022-12-13. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '35'.
Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version.
Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 35 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #16 from Nicolas Chauvet (kwizart) kwizart@gmail.com --- I wonder if it would be possible to solve this problem by building nosync using an el8 (or even el7) target and later tag it for newer fedora releases ? (that would be easier than messing with hardcoded symbols version that could be different for each architecture.)
Of course this break assumption for all fedora build, I'm not sure FESCO could allow such. It will open it's can of worms for sure.
Alternative would be to provide it into the infra-tag koji repo for infra (and outside infra) consumption...
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Miroslav Suchý msuchy@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Version|35 |37
--- Comment #17 from Miroslav Suchý msuchy@redhat.com --- bump bump. Please.
BTW one way how to fix this is to build it statically https://docs.fedoraproject.org/en-US/packaging-guidelines/#packaging-static-...
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Nicolas Chauvet (kwizart) kwizart@gmail.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Flags| |needinfo?(praiskup@redhat.c | |om) | |needinfo?(msuchy@redhat.com | |)
--- Comment #18 from Nicolas Chauvet (kwizart) kwizart@gmail.com --- @praiskup@redhat.com Can you fix the patch to pick the 2.0 dlsym symbol for i386 ? that seems the easier way forward... See also https://bugzilla.redhat.com/show_bug.cgi?id=2019329#c14
BTW one way how to fix this is to build it statically https://docs.fedoraproject.org/en-US/packaging-guidelines/#packaging-static-...
@msuchy@redhat.com as #c14 it doesn't sym that easy as the needed symbols aren't part of the static libc, as I understand.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Miroslav Suchý msuchy@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(msuchy@redhat.com | |) |
--- Comment #19 from Miroslav Suchý msuchy@redhat.com ---
Can you fix the patch
We cannot. I and Pavel - we are reporters. Mikolaj is the one who can fix this.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Pavel Raiskup praiskup@redhat.com changed:
What |Removed |Added ---------------------------------------------------------------------------- Flags|needinfo?(praiskup@redhat.c | |om) |
--- Comment #20 from Pavel Raiskup praiskup@redhat.com --- (In reply to Nicolas Chauvet (kwizart) from comment #18)
@praiskup@redhat.com Can you fix the patch to pick the 2.0 dlsym symbol for i386 ? that seems the easier way forward...
I tried, but I realized that the nosync.so doesn't work correctly with EL7 chroots (while with nosync.so built on F34 everything works fine):
Installing : gawk-4.0.2-4.el7_3.1.x86_64 46/172 /sbin/install-info: relocation error: /var/tmp/tmp.mock.2qns6qkr/lib64/nosync.so: symbol dlsym, version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference Installing : diffutils-3.3-5.el7.x86_64 47/172 /sbin/install-info: relocation error: /var/tmp/tmp.mock.2qns6qkr/lib64/nosync.so: symbol dlsym, version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference Installing : 1:findutils-4.5.11-6.el7.x86_64 48/172 /sbin/install-info: relocation error: /var/tmp/tmp.mock.2qns6qkr/lib64/nosync.so: symbol dlsym, version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference Installing : cpio-2.11-28.el7.x86_64 49/172 /sbin/install-info: relocation error: /var/tmp/tmp.mock.2qns6qkr/lib64/nosync.so: symbol dlsym, version GLIBC_2.2.5 not defined in file libc.so.6 with link time reference
Feel free to experiment with nosync-1.1-13.1.copr5146043 packages (x86_64 and i686) from `dnf copr enable praiskup/nosync-libc-2.2.5`.
The updated patch is here: https://copr-dist-git.fedorainfracloud.org/cgit/praiskup/nosync-libc-2.2.5/n...
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
Jan Kratochvil jan@jankratochvil.net changed:
What |Removed |Added ---------------------------------------------------------------------------- CC| |jan@jankratochvil.net
--- Comment #21 from Jan Kratochvil jan@jankratochvil.net --- Still a problem for: nosync-1.1-14.fc38.x86_64
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #22 from Aoife Moloney amoloney@redhat.com --- This message is a reminder that Fedora Linux 37 is nearing its end of life. Fedora will stop maintaining and issuing updates for Fedora Linux 37 on 2023-12-05. It is Fedora's policy to close all bug reports from releases that are no longer maintained. At that time this bug will be closed as EOL if it remains open with a 'version' of '37'.
Package Maintainer: If you wish for this bug to remain open because you plan to fix it in a currently maintained version, change the 'version' to a later Fedora Linux version. Note that the version field may be hidden. Click the "Show advanced fields" button if you do not see it.
Thank you for reporting this issue and we are sorry that we were not able to fix it before Fedora Linux 37 is end of life. If you would still like to see this bug fixed and are able to reproduce it against a later version of Fedora Linux, you are encouraged to change the 'version' to a later version prior to this bug being closed.
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #24 from Pavel Raiskup praiskup@redhat.com --- *** Bug 2254328 has been marked as a duplicate of this bug. ***
https://bugzilla.redhat.com/show_bug.cgi?id=2019329
--- Comment #25 from Nicolas Chauvet (kwizart) kwizart@gmail.com --- The current workaround for this bug is to build nosync with epel7 (aka minimal build target) and use it on newer fedora builders.
java-sig-commits@lists.fedoraproject.org