F37 proposal: Build all JDKs in Fedora against in-tree libraries and
with static stdc++lib (System-Wide Change proposal)
by Ben Cotton
https://fedoraproject.org/wiki/Changes/JdkInTreeLibsAndStdclibStatic
This document represents a proposed Change. As part of the Changes
process, proposals are publicly announced in order to receive
community feedback. This proposal will only be implemented if approved
by the Fedora Engineering Steering Committee.
== Summary ==
This is initial step to move JDKs to be more like other JDKs, to build
proper transferable images, and to lower certification burden of each
binary. Long storyshort, first step in:
https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
This first step will move, one by one, individual JDKs in F37 to be
built `--with-stdc++lib=static` and against in-tree (bundeld)
libraries: `--with-zlib="bundled" --with-freetype="bundled"
--with-libjpeg="bundled" --with-giflib="bundled"
--withlibpng="bundled" --with-lcms="bundled"
--with-harfbuzz="bundled" `
We already made a heavy testing of the behavior, and user should not
face negative experience. I'm not sure if this is
== Owner ==
* Name: [[User:jvanek| Jiri Vanek]]
* Email: jvanek(a)redhat.com
== Detailed Description ==
Please see https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
for whole picture
Please see https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs#Move_JD...
for this particular step. I would rather keep the details in the main
page then here.
== Feedback ==
According to short investigations, there are already precedents, where
certification is a reason to build once, certificate, and repack.
According to developers, the non-portbale JDK is causing upredicted
behavior different from other JDK vendors
According to JDK packagers and testers, there is to much JDKs now, and
the https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs#Move_Fe...
is the only way out
== Benefit to Fedora ==
Please see https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs#Motivation
for whole picture.
This particular proposal's main benefit will be that Fedora's JDKs as
packed in RPMs will again start to resemble upstream JDKs and other
vendors build, and some platform specific issues disappear, while JDKs
remain same in view of system integration and user experience
== Scope ==
* Proposal owners: push improved version of
https://src.fedoraproject.org/rpms/java-latest-openjdk/pull-request/98#re...
to all JDKs - one by one from latest, over 17 to 11 and 8. Once
settled down in F37 the backport to F36 is expected.
* Other developers: really, nothing. If there will be unexpected
impact to other developers, the
https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs may
need rework
* Release engineering: N/A [https://pagure.io/releng/issues #Releng
issue number]
* Policies and guidelines: N/A (not needed for this Change)
* Trademark approval: N/A (not needed for this Change)
* Alignment with Objectives:
== Upgrade/compatibility impact ==
The compatibility and upgrade path should remain completely smooth.
== How To Test ==
Install system JDK (java-17-openjdk) and ru your favorite application
or development. No regression should be noted.
== User Experience ==
Because of in-tree libraries, minimal image or font rendering
differences canbe spotted after very detailed investigations -
https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs#Side_ef...
- still, no of th e
https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs#Known_i...
should be hit by this proposal.
== Dependencies ==
No dependent packages should notice the change.
== Contingency Plan ==
* Contingency mechanism: Revert the patches and rework
https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
* Contingency deadline: before f37 release
* Blocks release? Unless the java-stack will become completely borked then no.
== Documentation ==
https://fedoraproject.org/wiki/MoveFedoraJDKsToBecomePortableJDKs
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
6 hours, 50 minutes
How much free space in /var is required for upgrades?
by Jason L Tibbitts III
So I went to do a dnf system-upgrade from F35 to F36 on a test machine,
as part of my usual testing. In the middle of the process, it appears
that /var filled up and that left the system in an unfortunate state.
Surprisingly (to me) it did boot with a random mix of F35 and F36
packages and even though it's a throwaway test box, I wanted to play
around with fixing it a bit and trying to understand why it ran out of
space instead of just reinstalling.
Turns out that "dnf --releasever 36 --nogpgcheck remove --duplicates"
was able to effectively everything in the system, and while running this
/var filled up again. When that happened, dnf couldn't even be aborted;
I had to kill -9. The culprit is the write-ahead log,
/var/lib/rpm/rpmdb.sqlite-wal. I resized /var and reran, and by the end
of the process had grown to over 9GB:
-rw-r--r--. 1 root root 9124576392 May 13 13:11 rpmdb.sqlite-wal
Of course it immediately went to 0 once the transaction completed,
though rpmdb.sqlite went from:
-rw-r--r--. 1 root root 281739264 May 11 14:24 rpmdb.sqlite
to
-rw-r--r--. 1 root root 730648576 May 13 13:15 rpmdb.sqlite
which seems... odd for what's effectively just reinstalling the existing
package set.
Anyway, obviously the solution is to make sure that /var is "big enough"
before you do a system upgrade. And we do have warnings about
filesystems being too small, but nothing about needing an extra 10GB for
this. Certainly my case might be somewhat pathological and it was good
that in the end I was able to get the system back into a useful state
without wiping it. But in the end I wonder:
1) Is it really expected that the wal file will grow to that size?
2) Is there anything to be done to reduce the size of the log?
3) Is there any better way to handle a lack of space in /var during an
RPM transaction?
4) Can we estimate how large the file will grow, and refuse to start a
system upgrade if there is not enough space? Certainly we already do
this to some degree, but it seems that the estimate of the required
space is a bit too small.
- J<
11 hours, 4 minutes
F36 release retrospective
by Ben Cotton
If you missed the announcement at the release party, the F36 release
retrospective survey is now open:
https://fedoraproject.limequery.com/231354
It should only take a few moments of your time. No matter how you
participated in the development and release of F36, I'd like your
input. (Remember, this is about the process of producing F36, not the
end result.) If you have any questions, please let me know. If you
have suggestions for the next time around, there's a field for that in
the survey!
The survey is open through 27 May. As I did for F35[1], I'll share
results on the Community Blog in June..
[1] https://communityblog.fedoraproject.org/f35-retrospective-results/
--
Ben Cotton
He / Him / His
Fedora Program Manager
Red Hat
TZ=America/Indiana/Indianapolis
16 hours, 52 minutes
Uninitialized variables and F37
by Steve Grubb
Hello,
This is a continuation of the discussion from F36 Change: GNU Toolchain
Update.
Uninitialized variables are a big problem. They can be sources of information
exposure if parts of a buffer are not initialized. They can also cause
unexpected execution paths if the attacker can groom the memory to a value of
their choosing. If the variable is a pointer to heap, this can cause free to
corrupt memory under certain circumstances. If the uninitialized memory is
part of user input, this can lead to improper input validation. This is not
hypothetical. All of these come from a paper doing an emprical study of
android flaws. [1] The data used in the paper is here. [2]
Part of the problem is that compilers and static analysis tools can't always
find them. I created a test program that has 8 uses of unintialized variables.
Gcc 11 misses all of them. Gcc 12 finds 2. Clang 13 finds 1. cppcheck finds 2 or
3 - but does so much complaining you'd think it found all. Valgrind finds 2.
Flexelint, a commercial linter, finds 1.
Since tools can't always find them, the only option we have right now is force
initialization to something the attacker cannot control. Kees Cook started a
discussion on the llvm developers mail list a while back. He makes a very
clear argument. I would be repeating his points, so please read the original
discussion here (also read the replies):
https://lists.llvm.org/pipermail/cfe-dev/2020-April/065221.html
He talks about -ftrivial-auto-var-init=zero being used for production builds
and -ftrivial-auto-var-init=<pattern> being used for debug builds. The use
is not just the kernel. Consider a server that returns data across the
network to a client. It could possibly leak crypto keys or passwords if the
returned data structure has uninitialized memory.
For more background, the creator of this technology for LLVM presented a talk
about this feature at a past LLVM developer conference:
https://www.youtube.com/watch?v=I-XUHPimq3o
He said this would have prevented over 900 fixed CVE's in Chrome and 12% of
all Android CVE's.
From deep inside the LLVM thread above, comes this nugget:
---
To add in, we (Microsoft) currently use zero initialization technology in
Visual Studio in a large amount of production code we ship to customers (all
kernel components, a number of user-mode components). This code is both C and
C++.
We already have had multiple vulnerabilities killed because we shipped this
technology in production. We received bug reports with repros that worked on
older versions of Windows without the mitigation and new versions of Windows
that do have it. The new versions don't repro, the old ones do.
---
Microsoft is also digging in to uninitialized variables. They have a lengthy
blog post that talks about extending this to heap memory. [3]
I think this would be an important step forward to turn this on across all
compilations. We could wipe out an entire class of bugs in one fell swoop.
But then, what about heap allocations? Calloc has existed for a long time. It
might be worthwhile to have a CFLAG that can tell glibc (or other allocators)
to substitute something like calloc for malloc.
Cheers,
-Steve
[1] - https://picture.iczhiku.com/resource/paper/shkeTWJEaFUuWCMc.pdf
[2] - http://ml-papers.gitlab.io/android.vulnerabilities-2017/appendix/
MSR2017/vulnerabilitiesList.html
[3] - https://msrc-blog.microsoft.com/2020/07/02/solving-uninitialized-kernel-p...
18 hours, 10 minutes