Okay, this one has me stumped. Any chromium package I build through rawhide refuses to render most of the strings.
At first, I thought this was gcc 11, but then I noticed that the first build with this problem was built before GCC 11 landed in rawhide (the compiler was the same n-v-r as the one in Fedora 33). Next, I thought it must be due to a newer system component that Chromium uses dynamically, but I was able to disprove that by installing the Fedora 33 build (same version-release) into a Rawhide VM, and it works fine. Google Chrome also works fine in rawhide.
It seems that this must be something that is contained within chromium, that when built in rawhide, builds broken.
Here's a screenshot of what it looks like: https://twitter.com/spotfoss/status/1338918235719299072/photo/1
Note that some text strings are rendering (in the UI, in the "search box"), but most of them are not (the "HTML5Test" text below the icon, all of the strings in the developer console).
Chromium has a lot of bundled components, so it is usually fairly resistant to system changes. There are no differences between how Chromium builds (within the RPM spec) on Fedora 33 and Rawhide. It also doesn't use %{optflags}, so the compiler flags are equivalent.
Could this be due to some quirk of binutils in the way chromium gets linked in rawhide? Is there something else unique to how packages are built in rawhide right now? Are any other rawhide packages having similar string issues?
Here are some builds for comparison:
Fedora 34: Chromium 87.0.4280.88 (built against GCC 11): https://koji.fedoraproject.org/koji/taskinfo?taskID=57595738 Fedora 34: Chromium 87.0.4280.88 (built against GCC 10, same as Fedora 33): https://koji.fedoraproject.org/koji/buildinfo?buildID=1655036 Fedora 33: Chromium 87.0.4280.88 (build against GCC 10, but actually works in rawhide): https://koji.fedoraproject.org/koji/buildinfo?buildID=1655036 Fedora 34: Chromium 88.0.4324.27 (beta version of chromium, also broken) https://koji.fedoraproject.org/koji/taskinfo?taskID=56977476
Thanks in advance, ~spot
On Thu, Dec 17, 2020 at 5:12 PM Tom Callaway spotrh@gmail.com wrote:
Okay, this one has me stumped. Any chromium package I build through rawhide refuses to render most of the strings.
At first, I thought this was gcc 11, but then I noticed that the first build with this problem was built before GCC 11 landed in rawhide (the compiler was the same n-v-r as the one in Fedora 33). Next, I thought it must be due to a newer system component that Chromium uses dynamically, but I was able to disprove that by installing the Fedora 33 build (same version-release) into a Rawhide VM, and it works fine. Google Chrome also works fine in rawhide.
It seems that this must be something that is contained within chromium, that when built in rawhide, builds broken.
Here's a screenshot of what it looks like: https://twitter.com/spotfoss/status/1338918235719299072/photo/1
Note that some text strings are rendering (in the UI, in the "search box"), but most of them are not (the "HTML5Test" text below the icon, all of the strings in the developer console).
Can you try downgrading cairo to the f33 version (1.16.0), just to rule that out? We updated it to the 1.17.4 development version in rawhide and I think we're the first distro to ship it, so it could possibly have issues.
I downgraded cairo to 1.16.0-9.fc33 and it had no effect, the bug remained.
Thanks, ~spot
On Thu, Dec 17, 2020 at 11:19 AM Kalev Lember kalevlember@gmail.com wrote:
On Thu, Dec 17, 2020 at 5:12 PM Tom Callaway spotrh@gmail.com wrote:
Okay, this one has me stumped. Any chromium package I build through rawhide refuses to render most of the strings.
At first, I thought this was gcc 11, but then I noticed that the first build with this problem was built before GCC 11 landed in rawhide (the compiler was the same n-v-r as the one in Fedora 33). Next, I thought it must be due to a newer system component that Chromium uses dynamically, but I was able to disprove that by installing the Fedora 33 build (same version-release) into a Rawhide VM, and it works fine. Google Chrome also works fine in rawhide.
It seems that this must be something that is contained within chromium, that when built in rawhide, builds broken.
Here's a screenshot of what it looks like: https://twitter.com/spotfoss/status/1338918235719299072/photo/1
Note that some text strings are rendering (in the UI, in the "search box"), but most of them are not (the "HTML5Test" text below the icon, all of the strings in the developer console).
Can you try downgrading cairo to the f33 version (1.16.0), just to rule that out? We updated it to the 1.17.4 development version in rawhide and I think we're the first distro to ship it, so it could possibly have issues.
-- Kalev _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Tom Callaway spotrh@gmail.com writes:
Okay, this one has me stumped. Any chromium package I build through rawhide refuses to render most of the strings.
At first, I thought this was gcc 11, but then I noticed that the first build with this problem was built before GCC 11 landed in rawhide (the compiler was the same n-v-r as the one in Fedora 33). Next, I thought it must be due to a newer system component that Chromium uses dynamically, but I was able to disprove that by installing the Fedora 33 build (same version-release) into a Rawhide VM, and it works fine. Google Chrome also works fine in rawhide.
Chromium has a lot of bundled components, so it is usually fairly resistant to system changes. There are no differences between how Chromium builds (within the RPM spec) on Fedora 33 and Rawhide. It also doesn't use %{optflags}, so the compiler flags are equivalent.
Could this be due to some quirk of binutils in the way chromium gets linked in rawhide? Is there something else unique to how packages are built in rawhide right now? Are any other rawhide packages having similar string issues?
Have you tried the reverse? rawhide-built chromium into fc33?
Thanks, --Robbie
I cannot install the rawhide-built chromium into F33 without bringing glibc from rawhide with me:
[spot@localhost ~]$ sudo rpm -Uvh chromium-common-87.0.4280.88-1.fc34.x86_64.rpm chromium-87.0.4280.88-1.fc34.x86_64.rpm error: Failed dependencies: libc.so.6(GLIBC_2.33)(64bit) is needed by chromium-common-87.0.4280.88-1.fc34.x86_64 libc.so.6(GLIBC_2.33)(64bit) is needed by chromium-87.0.4280.88-1.fc34.x86_64
When I _just_ update glibc from rawhide on an otherwise Fedora 33 instance (glibc, glibc-all-langpacks, glibc-common, glibc-devel, glibc-headers-x86, glibc-langpack-en), then install the rawhide built chromium, it exhibits the same missing strings bug.
~spot
On Thu, Dec 17, 2020 at 11:29 AM Robbie Harwood rharwood@redhat.com wrote:
Tom Callaway spotrh@gmail.com writes:
Okay, this one has me stumped. Any chromium package I build through
rawhide
refuses to render most of the strings.
At first, I thought this was gcc 11, but then I noticed that the first build with this problem was built before GCC 11 landed in rawhide (the compiler was the same n-v-r as the one in Fedora 33). Next, I thought it must be due to a newer system component that Chromium uses dynamically,
but
I was able to disprove that by installing the Fedora 33 build (same version-release) into a Rawhide VM, and it works fine. Google Chrome also works fine in rawhide.
Chromium has a lot of bundled components, so it is usually fairly
resistant
to system changes. There are no differences between how Chromium builds (within the RPM spec) on Fedora 33 and Rawhide. It also doesn't use %{optflags}, so the compiler flags are equivalent.
Could this be due to some quirk of binutils in the way chromium gets
linked
in rawhide? Is there something else unique to how packages are built in rawhide right now? Are any other rawhide packages having similar string issues?
Have you tried the reverse? rawhide-built chromium into fc33?
Thanks, --Robbie
Tom Callaway spotrh@gmail.com writes:
I cannot install the rawhide-built chromium into F33 without bringing glibc from rawhide with me:
[spot@localhost ~]$ sudo rpm -Uvh chromium-common-87.0.4280.88-1.fc34.x86_64.rpm chromium-87.0.4280.88-1.fc34.x86_64.rpm error: Failed dependencies: libc.so.6(GLIBC_2.33)(64bit) is needed by chromium-common-87.0.4280.88-1.fc34.x86_64 libc.so.6(GLIBC_2.33)(64bit) is needed by chromium-87.0.4280.88-1.fc34.x86_64
When I _just_ update glibc from rawhide on an otherwise Fedora 33 instance (glibc, glibc-all-langpacks, glibc-common, glibc-devel, glibc-headers-x86, glibc-langpack-en), then install the rawhide built chromium, it exhibits the same missing strings bug.
Interesting. Seems like that points at the problem being glibc-adjacent, no? That's still really wide though...
Thanks, --Robbie
Certainly not ruling out glibc as the problem here, but if it was glibc, I would think the problem would arise when I install the Fedora 33 build in rawhide, and it does not...
~spot
On Thu, Dec 17, 2020 at 12:10 PM Robbie Harwood rharwood@redhat.com wrote:
Tom Callaway spotrh@gmail.com writes:
I cannot install the rawhide-built chromium into F33 without bringing
glibc
from rawhide with me:
[spot@localhost ~]$ sudo rpm -Uvh chromium-common-87.0.4280.88-1.fc34.x86_64.rpm chromium-87.0.4280.88-1.fc34.x86_64.rpm error: Failed dependencies: libc.so.6(GLIBC_2.33)(64bit) is needed by chromium-common-87.0.4280.88-1.fc34.x86_64 libc.so.6(GLIBC_2.33)(64bit) is needed by chromium-87.0.4280.88-1.fc34.x86_64
When I _just_ update glibc from rawhide on an otherwise Fedora 33
instance
(glibc, glibc-all-langpacks, glibc-common, glibc-devel,
glibc-headers-x86,
glibc-langpack-en), then install the rawhide built chromium, it exhibits the same missing strings bug.
Interesting. Seems like that points at the problem being glibc-adjacent, no? That's still really wide though...
Thanks, --Robbie
Il 17/12/20 17:12, Tom Callaway ha scritto:
Okay, this one has me stumped. Any chromium package I build through rawhide refuses to render most of the strings.
Seems similar to what I reported for Falkon:
https://bugzilla.redhat.com/show_bug.cgi?id=1904652
Does Chromium uses qt5-qtwebengine?
Mattia
On Thu, Dec 17, 2020 at 12:02 PM Mattia Verga via devel devel@lists.fedoraproject.org wrote:
Il 17/12/20 17:12, Tom Callaway ha scritto:
Okay, this one has me stumped. Any chromium package I build through rawhide refuses to render most of the strings.
Seems similar to what I reported for Falkon:
https://bugzilla.redhat.com/show_bug.cgi?id=1904652
Does Chromium uses qt5-qtwebengine?
QtWebEngine *is* Chromium, so it would make sense that it exhibits the same problem.
Neal Gompa wrote:
QtWebEngine *is* Chromium, so it would make sense that it exhibits the same problem.
To be clear (and I know you know this, but your readers might not know), QtWebEngine (qt5-qtwebengine) and Chromium (chromium) are distinct packages (none of them depends on each other), each containing a slightly different copy of essentially the same source code (plus some higher-layer code that is entirely different: Qt wrapper libraries vs. a browser application). But since the source code is largely the same, things such as miscompilations by the compiler are likely to affect both the same way. And the symptoms in the 2 screenshots definitely look identical.
Kevin Kofler
On 12/20/20 7:45 PM, Kevin Kofler via devel wrote:
Neal Gompa wrote:
QtWebEngine *is* Chromium, so it would make sense that it exhibits the same problem.
To be clear (and I know you know this, but your readers might not know), QtWebEngine (qt5-qtwebengine) and Chromium (chromium) are distinct packages (none of them depends on each other), each containing a slightly different copy of essentially the same source code (plus some higher-layer code that is entirely different: Qt wrapper libraries vs. a browser application). But since the source code is largely the same, things such as miscompilations by the compiler are likely to affect both the same way. And the symptoms in the 2 screenshots definitely look identical.
I think this has been fixed with the latest gcc drop into rawhide.
jeff
Am 30.12.20 um 04:53 schrieb Jeff Law:
To be clear (and I know you know this, but your readers might not know), QtWebEngine (qt5-qtwebengine) and Chromium (chromium) are distinct packages (none of them depends on each other), each containing a slightly different copy of essentially the same source code (plus some higher-layer code that is entirely different: Qt wrapper libraries vs. a browser application). But since the source code is largely the same, things such as miscompilations by the compiler are likely to affect both the same way. And the symptoms in the 2 screenshots definitely look identical. I think this has been fixed with the latest gcc drop into rawhide.
jeff
Don't you need to recompile stuff first to have an effect? :)
Best regards, Marius
Am 30.12.20 um 14:07 schrieb Mattia Verga via devel:
Il 30/12/20 10:14, Marius Schwarz ha scritto:
Don't you need to recompile stuff first to have an effect? :)
I've just pushed a rebuild for qt5-qtwebengine, let's see if that's enough.
I had a chromium 85 build running instead of the 87er, that had no problems rendering texts. My guerss is, that chromium needs a rebuild too.
Best regards, Marius
I rebuilt chromium, but it did not resolve the issue.
~spot
On Wed, Dec 30, 2020 at 5:35 PM Marius Schwarz fedoradev@cloud-foo.de wrote:
Am 30.12.20 um 14:07 schrieb Mattia Verga via devel:
Il 30/12/20 10:14, Marius Schwarz ha scritto:
Don't you need to recompile stuff first to have an effect? :)
I've just pushed a rebuild for qt5-qtwebengine, let's see if that's
enough.
I had a chromium 85 build running instead of the 87er, that had no problems rendering texts. My guerss is, that chromium needs a rebuild too.
Best regards, Marius _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
Il 02/01/21 22:57, Kevin Kofler via devel ha scritto:
Tom Callaway wrote:
I rebuilt chromium, but it did not resolve the issue.
So what can we do to resolve the issue? Surely we cannot leave both Chromium and Falkon unable to render text forever.
Kevin Kofler
The problem seems to be qt5-qtwebengine is unable to use system fonts. It doesn't render text using default fonts (i.e. 'serif' or 'sans-serif' css styles), but it does render text using custom css fonts. In fact, it renders most of qt.io homepage (using 'Titillium Web' custom css font) and also some text on Bodhi homepage (using 'Open Sans' custom css font).
However, opening, for example, Falkon settings, I can get the fonts under the Character settings and the preview works.
Mattia
Okay, to try to narrow this down a bit, I setup a very large AWS Fedora 33 instance and built chromium packages, then installed them into a second Fedora Rawhide instance. As before, these packages work fine. (This is our baseline)
Next, I updated glibc (and JUST glibc, glibc-common, glibc-devel, glibc-headers, glibc-langpack-en) from Rawhide on the Fedora 33 instance and built the chromium package again. This build, when installed into Fedora Rawhide, exhibits the text rendering issue.
This makes me very suspicious of something in glibc between 2.32 (Fedora 33) and 2.32.9000 (rawhide), but I'm not sure where to look from here. Any ideas?
~spot
On Sun, Jan 3, 2021 at 4:49 AM Mattia Verga via devel < devel@lists.fedoraproject.org> wrote:
Il 02/01/21 22:57, Kevin Kofler via devel ha scritto:
Tom Callaway wrote:
I rebuilt chromium, but it did not resolve the issue.
So what can we do to resolve the issue? Surely we cannot leave both
Chromium
and Falkon unable to render text forever.
Kevin Kofler
The problem seems to be qt5-qtwebengine is unable to use system fonts. It doesn't render text using default fonts (i.e. 'serif' or 'sans-serif' css styles), but it does render text using custom css fonts. In fact, it renders most of qt.io homepage (using 'Titillium Web' custom css font) and also some text on Bodhi homepage (using 'Open Sans' custom css font).
However, opening, for example, Falkon settings, I can get the fonts under the Character settings and the preview works.
Mattia
devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org
* Tom Callaway:
This makes me very suspicious of something in glibc between 2.32 (Fedora 33) and 2.32.9000 (rawhide), but I'm not sure where to look from here. Any ideas?
Does the issue go away if you disable the Chromium sandbox?
One difference is that the stat functions are called in a different way, and perhaps the sandbox isn't ready for that.
Thanks, Florian
On Fri, Jan 08, 2021 at 02:58:02PM +0100, Florian Weimer wrote:
- Tom Callaway:
This makes me very suspicious of something in glibc between 2.32 (Fedora 33) and 2.32.9000 (rawhide), but I'm not sure where to look from here. Any ideas?
Does the issue go away if you disable the Chromium sandbox?
One difference is that the stat functions are called in a different way, and perhaps the sandbox isn't ready for that.
Or another case of the faccessat -> faccessat2 problem ?
Regards, Daniel
* Daniel P. Berrangé:
On Fri, Jan 08, 2021 at 02:58:02PM +0100, Florian Weimer wrote:
- Tom Callaway:
This makes me very suspicious of something in glibc between 2.32 (Fedora 33) and 2.32.9000 (rawhide), but I'm not sure where to look from here. Any ideas?
Does the issue go away if you disable the Chromium sandbox?
One difference is that the stat functions are called in a different way, and perhaps the sandbox isn't ready for that.
Or another case of the faccessat -> faccessat2 problem ?
That issue would be visible with just a glibc upgrade, without a rebuild. The initial test (Fedora 33 on rawhide) seems to show that this is not happening.
Thanks, Florian
Looks like this might be it. Running with --no-sandbox brings back the strings. Is there a reference to how the stat calls should now be done?
Thanks, ~spot
On Fri, Jan 8, 2021 at 8:58 AM Florian Weimer fweimer@redhat.com wrote:
- Tom Callaway:
This makes me very suspicious of something in glibc between 2.32 (Fedora 33) and 2.32.9000 (rawhide), but I'm not sure where to look from here. Any ideas?
Does the issue go away if you disable the Chromium sandbox?
One difference is that the stat functions are called in a different way, and perhaps the sandbox isn't ready for that.
Thanks, Florian -- Red Hat GmbH, https://de.redhat.com/ , Registered seat: Grasbrunn, Commercial register: Amtsgericht Muenchen, HRB 153243, Managing Directors: Charles Cachera, Brian Klemm, Laurie Krebs, Michael O'Neill
* Tom Callaway:
Looks like this might be it. Running with --no-sandbox brings back the strings. Is there a reference to how the stat calls should now be done?
I suspect some of the preprocessor conditionals in SyscallSets::IsFileSystem in
need fixing.
I expect more fstatat/fstatat64 usage in current glibc.
Thanks, Florian
Based on my (admittedly extremely limited) understanding of things, this seems correct as is:
#if defined(__x86_64__) || defined(__aarch64__) case __NR_newfstatat: // fstatat(). EPERM not a valid errno. #elif defined(__i386__) || defined(__arm__) || \ (defined(ARCH_CPU_MIPS_FAMILY) && defined(ARCH_CPU_32_BITS)) case __NR_fstatat64: #endif
Is fstatat64 actually implemented on x86_64 now?
Alternately, if you'd prefer to simply open an upstream bug with Google, just let me know. :) I want to be helpful here, but not waste your time.
Thanks, ~spot
* Tom Callaway:
Based on my (admittedly extremely limited) understanding of things, this seems correct as is:
#if defined(__x86_64__) || defined(__aarch64__) case __NR_newfstatat: // fstatat(). EPERM not a valid errno. #elif defined(__i386__) || defined(__arm__) || \ (defined(ARCH_CPU_MIPS_FAMILY) && defined(ARCH_CPU_32_BITS)) case __NR_fstatat64: #endif
Is fstatat64 actually implemented on x86_64 now?
It's not. It may be a completely different system call.
Is there a way to get logging of filtered system calls from the sandbox?
Alternately, if you'd prefer to simply open an upstream bug with Google, just let me know. :) I want to be helpful here, but not waste your time.
It makes sense to do that anyway, to share information across distributions.
Thanks, Florian
Florian Weimer wrote:
It's not. It may be a completely different system call.
Is there a way to get logging of filtered system calls from the sandbox?
What I'd suggest doing is strace-ing the rendering process in the version built against glibc 2.32 vs. the version built against glibc 2.33/2.32.9x and compare the syscalls it uses, to see what changed.
I see a few syscall use changes in glibc, but they are in .c files, not in .h files, so I am surprised that the version of glibc used at build time apparently matters.
Kevin Kofler
Florian Weimer wrote:
It's not. It may be a completely different system call.
So now I had a new idea how to figure out what difference the version of glibc we are compiling against can make: track down the symbol version: nm -D --with-symbol-versions Downloads/libQt5WebEngineCore.so.5.15.2 | grep '@GLIBC_2.33' U fstat64@GLIBC_2.33 U fstatat64@GLIBC_2.33 U lstat64@GLIBC_2.33 U stat64@GLIBC_2.33
So we are getting new symbol versions of the above 4 functions. So now we only need to know what is different between the above and the syscalls presumably used previously: nm -D --with-symbol-versions Downloads/libQt5WebEngineCore.so.5.15.1 | grep 'stat(at)?64' U __fxstat64@GLIBC_2.2.5 U __fxstatat64@GLIBC_2.4 U __lxstat64@GLIBC_2.2.5 U __xstat64@GLIBC_2.2.5 (That's the version from F33 GA, definitely built against an older glibc.)
Kevin Kofler
Kevin Kofler via devel wrote:
So now I had a new idea how to figure out what difference the version of glibc we are compiling against can make: track down the symbol version: nm -D --with-symbol-versions Downloads/libQt5WebEngineCore.so.5.15.2 | grep '@GLIBC_2.33' U fstat64@GLIBC_2.33 U fstatat64@GLIBC_2.33 U lstat64@GLIBC_2.33 U stat64@GLIBC_2.33
So we are getting new symbol versions of the above 4 functions. So now we only need to know what is different between the above and the syscalls presumably used previously: nm -D --with-symbol-versions Downloads/libQt5WebEngineCore.so.5.15.1 | grep 'stat(at)?64' U __fxstat64@GLIBC_2.2.5 U __fxstatat64@GLIBC_2.4 U __lxstat64@GLIBC_2.2.5 U __xstat64@GLIBC_2.2.5 (That's the version from F33 GA, definitely built against an older glibc.)
PS: This commit: https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=8ed005daf0ab03e142500... is why the version of glibc used at compile time makes a difference.
Until glibc 2.32, stat64 etc. were redirected to __xstat64 etc. by inline functions or macros; glibc 2.33 changed them to true functions. As a result, code compiled against glibc 2.32 or older will get the old __xstat64 etc. implementation that is still present, whereas code compiled against glibc 2.33 will get the new stat64 etc. implementation.
Kevin Kofler
* Kevin Kofler via devel:
Florian Weimer wrote:
It's not. It may be a completely different system call.
So now I had a new idea how to figure out what difference the version of glibc we are compiling against can make: track down the symbol version: nm -D --with-symbol-versions Downloads/libQt5WebEngineCore.so.5.15.2 | grep '@GLIBC_2.33' U fstat64@GLIBC_2.33 U fstatat64@GLIBC_2.33 U lstat64@GLIBC_2.33 U stat64@GLIBC_2.33
So we are getting new symbol versions of the above 4 functions. So now we only need to know what is different between the above and the syscalls presumably used previously: nm -D --with-symbol-versions Downloads/libQt5WebEngineCore.so.5.15.1 | grep 'stat(at)?64' U __fxstat64@GLIBC_2.2.5 U __fxstatat64@GLIBC_2.4 U __lxstat64@GLIBC_2.2.5 U __xstat64@GLIBC_2.2.5 (That's the version from F33 GA, definitely built against an older glibc.)
Right, and the glibc 2.33 versions all call fstatat64 in the end (system call number 0x106). The older x versions call the earlier system calls (numbers 4, 5, 6).
Thanks, Florian
Florian Weimer wrote:
I suspect some of the preprocessor conditionals in SyscallSets::IsFileSystem in
need fixing.
Unfortunately, the fix is more complicated than that. It is really not helpful behavior from glibc to use the fstatat syscall to implement fstat. (What is the benefit of doing that?) It is particularly hard to detect that an fstatat call is really an fstat in a seccomp sandbox because BPF does not support validating string arguments, but the path being an empty string is a necessary condition to check.
I have come up with this fix: https://src.fedoraproject.org/rpms/qt5-qtwebengine/blob/master/f/qtwebengine... that works for me (and it actually ends up calling the fstat syscall as the old glibc did, because that is the only safe way to prevent retriggering another SIGSYS from the SIGSYS handler).
So far, it has not yet been applied to the chromium package, only to qt5-qtwebengine.
Kevin Kofler
* Kevin Kofler via devel:
Unfortunately, the fix is more complicated than that. It is really not helpful behavior from glibc to use the fstatat syscall to implement fstat. (What is the benefit of doing that?)
The fstat/fstat64 system call does not exist on arc and riscv/rv32. glibc tries to consolidate the implementations as far as possible, so if the kernel deprecates a system call and the replacement is already supported by all architectures, glibc uses that, rather than maintaing architecture-specific code to call the earliest possible implemented system call.
Thanks, Florian
Florian Weimer wrote:
The fstat/fstat64 system call does not exist on arc and riscv/rv32. glibc tries to consolidate the implementations as far as possible, so if the kernel deprecates a system call and the replacement is already supported by all architectures, glibc uses that, rather than maintaing architecture-specific code to call the earliest possible implemented system call.
That explains it. So my patch calling the fstat64 syscall (fstat on 64-bit platforms) in the fstatat handler should be fine on those platforms that support the fstat(64) syscall.
But the thing is, fstatat is not really a newer version of fstat, it unfortunately has very different security properties. fstat allows retrieving the stat metadata only of already open files (if you know or guess the fd). On the other hand, fstatat allows retrieving the stat metadata of ANY file on the file system. It even accepts an absolute path as the relative pathspec, in which case the fd is ignored entirely. (And I guess it also allows directory traversal using "..", but that does not matter anyway since it also accepts absolute paths to begin with.) And the only way to distinguish the fstat case ("" pathspec) from the stat case (absolute pathspec) is to actually look at the string, which cannot be done in BPF.
Kevin Kofler
Kevin Kofler via devel wrote:
But the thing is, fstatat is not really a newer version of fstat, it unfortunately has very different security properties. fstat allows retrieving the stat metadata only of already open files (if you know or guess the fd). On the other hand, fstatat allows retrieving the stat metadata of ANY file on the file system. It even accepts an absolute path as the relative pathspec, in which case the fd is ignored entirely. (And I guess it also allows directory traversal using "..", but that does not matter anyway since it also accepts absolute paths to begin with.) And the only way to distinguish the fstat case ("" pathspec) from the stat case (absolute pathspec) is to actually look at the string, which cannot be done in BPF.
Actually, there is one way the SIGSYS handler could call back into fstatat without triggering another SIGSYS: instead of passing the original empty string, it could pass a global empty string constant, which has a known address that could be whitelisted in BPF.
So the BPF would look like: if (args[3] == AT_EMPTY_PATH) { if (args[1] == global_empty_string) { // pointer comparison allow(); } else { trap(); // to the SIGSYS handler checking that the string is empty } } else { error(EACCES); }
But all this is just papering over the fact that the fstatat syscall is just too flexible for sandboxing.
Kevin Kofler
* Kevin Kofler via devel:
But the thing is, fstatat is not really a newer version of fstat, it unfortunately has very different security properties. fstat allows retrieving the stat metadata only of already open files (if you know or guess the fd). On the other hand, fstatat allows retrieving the stat metadata of ANY file on the file system. It even accepts an absolute path as the relative pathspec, in which case the fd is ignored entirely. (And I guess it also allows directory traversal using "..", but that does not matter anyway since it also accepts absolute paths to begin with.) And the only way to distinguish the fstat case ("" pathspec) from the stat case (absolute pathspec) is to actually look at the string, which cannot be done in BPF.
This is currently not a major consideration for system call design. We can't add this downstream from the kernel if support just isn't there. You have to solve these issues for porting to other architectures anyway.
Thanks, Florian
Florian Weimer wrote:
This is currently not a major consideration for system call design. We can't add this downstream from the kernel if support just isn't there. You have to solve these issues for porting to other architectures anyway.
So the upstream Linux kernel does not care about security? Sad!
Kevin Kofler
* Kevin Kofler via devel:
Florian Weimer wrote:
This is currently not a major consideration for system call design. We can't add this downstream from the kernel if support just isn't there. You have to solve these issues for porting to other architectures anyway.
So the upstream Linux kernel does not care about security? Sad!
I don't think that's a correct characterization of the situation. Unfortunately, seccomp filters also block system calls that are necessary to avoid bugs (see faccessat2). And developers that usually subscribe to the Move Fast, Break Things motto need many months to fix broken seccomp filters.
Thanks, Florian
Am 17.12.20 um 17:12 schrieb Tom Callaway:
Okay, this one has me stumped. Any chromium package I build through rawhide refuses to render most of the strings.
Afaik chromium can't access libva anymore. On the pinephone, where i noticed this bug, it said so itself.
Best regards, Marius