devel-announce December 2023

devel-announce@lists.fedoraproject.org

5 participants
24 discussions

F40 Change Proposal: GNU Toolchain for F40 (System-Wide)
by Aoife Moloney 28 Dec '23

28 Dec '23

Wiki -> https://fedoraproject.org/wiki/Changes/GNUToolchainF40 This is a proposed Change for Fedora Linux. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Update the Fedora 40 GNU Toolchain to gcc 14.0, binutils 2.41, glibc 2.39 and gdb 13.2. The set of core GNU Toolchain packages for Fedora 3XX are as follows: * GNU C Compiler 14.0 ** Associated runtimes for C++ (libstdc++), Go (gccgo), OpenMP (gomp), Fortran (gfortran), D (phobos), Objective C/C++. * GNU Binary Utilities 2.41 * GNU C Library 2.39 * GNU Debugger 13.2 (immediately available in Fedora 39) The gcc 14.0 change will be tracked in this top-level GNU Toolchain system-wide update. The "Porting to Modern C" system-wide change requested is tracked here: https://fedoraproject.org/wiki/Changes/PortingToModernC The binutils 2.41 change will be tracked in this top-level GNU Toolchain system-wide update. The glibc 2.39 change will be tracked in this top-level GNU Toolchain system-wide update. The gdb 13.2 update was rolled out across all Fedora releases and the move to 13+ will be handled in the same fashion. The debugger is capable of debugging the binary artifacts produced by the rest of the system toolchain. == Owner == * Name: [[User:codonell|Carlos O'Donell]] * Email: carlos(a)redhat.com == Detailed Description == The GNU Compiler Collection, GNU Binary Utilities, GNU C Library, and the GNU Debugger make up the core part of the GNU Toolchain and it is useful for our users to transition these components as a complete implementation when making a new release of Fedora.  The GNU Compiler Collection is expected to release version 14.0, before the Fedora 40 release. It will contain many new features, documented here: https://gcc.gnu.org/gcc-14/changes.html. The latest point release for gcc 14 will be included in Fedora 40, this will most probably be 14.0. Enabling warnings and other features related to porting to modern C will be tracked as part of the [https://fedoraproject.org/wiki/Changes/PortingToModernC Port to Modern C] system-wide change request.  The GNU Binutils version 2.41 was released before Fedora 40; and we have already been using this version of binutils in Fedora Rawhide successfully to build the distribution. Given the present schedule for Fedora 40 we will continue to use Binutils 2.41.  The GNU C Library version 2.39 is expected to be release before Fedora 40; we have started closely tracking the glibc 2.39 development code in Fedora Rawhide and are addressing any issues as they arise. Given the present schedule Fedora 40 will branch after the release of glibc 2.39. However, the mass rebuild schedule means Fedora 40 will mass rebuild (if required) before the final release of glibc 2.39, but after the ABI is frozen.  The GNU Debugger version 13.2 has already been rolled out across all Fedora releases at the same time. === Enable GNU TLS Descriptors === This update will enable the use of GNU TLS Descriptors for x86_64 to match what has already been in place in AArch64. This allows x86_64 to use faster static TLS access modes if static TLS space remains available. This is an internal implementation detail of the way in which thread local storage is implemented. Recent advances in the toolchain allow this to be used along with a limited set of libraries that use static TLS access modes and are loaded dynamically. The dynamic loader has additional tunables to start applications with more reserved static TLS space if required for better performance or for dynamically loaded libraries. == Benefit to Fedora == Stays up to date with latest features, improvements, security and bug fixes from gcc, glibc, binutils, and gdb upstream. The goal is to track and transition to the latest components of the GNU Toolchain. == Scope == * Proposal owners: Fedora Toolchain Team (gcc, glibc, binutils, gdb, ...) developers need to ensure that gcc, glibc, binutils, and gdb in rawhide are stable and ready for the Fedora 40 branch. * Other developers: Given that glibc is backwards compatible and we have been testing the new glibc in rawhide it should make very little impact when updated, except for the occasional deprecation warnings and removal of legacy interfaces from public header files. * Release engineering: A mass rebuild is strongly encouraged; [https://pagure.io/releng/issue/11865 #11865] * Policies and guidelines: N/A (not needed for this Change) * Trademark approval: N/A (not needed for this Change) * Alignment with Objectives: N/A == Upgrade/compatibility impact == The compiler, the static linker and the the library are backwards compatible with the previous version of Fedora.  Some source changes may be required for gcc 14 rebase: https://gcc.gnu.org/gcc-14/changes.html Any source level changes required for glibc 2.39 will be noted here: https://sourceware.org/glibc/wiki/Release/2.39#Packaging_Changes == How To Test ==  The GNU Compiler Collection has its own test suite which is run during the package build and examined by the gcc developers before being uploaded.  The GNU C Library has its own test suite which is run during the package build and examined by the glibc developers before being uploaded. This test suite has over 6200 tests that run to verify the correct operation of the library. In the future we may also run the microbenchmark to look for performance regressions.  The GNU Binutils has its own test suite which is run during the package build and examined by binutils developers before being uploaded. The regression test suite is run to verify the correct operation of the static linker and attendant utilities.  The GNU Debugger has its own test suite which is run during the package build and examined by gdb developers before being uploaded. The regression test suite is run to verify the correct operation of the debugger. == User Experience ==  Upgrading the 4 main GNU Toolchain components (gcc, binutils, glibc, and gdb) ensures that users have an up to date working system compiler, assembler, static linker, core language runtimes (C, C++, etc), dyanmic linker, and debugger. All of these components are being updated to provide support for newer language features, and hardware features; enabling users to make use of these features for their applications. In some cases the components are updated in a synchronized fashion if a feature requires support across the components that constitute the implementation e.g. compiler feature that requires language library support. == Dependencies == All packages do not need to be rebuilt due to backwards compatibility. However, it is advantageous if a mass rebuild is performed during the Fedora 40 cycle. The mass rebuild would ensure all packages can be built with the newer compiler and core runtime. == Contingency Plan == * Contingency mechanism glibc: If glibc 2.39 proves too disruptive to compiling the distribution we could revert to 2.38, but given that Rawhide has started tracking glibc 2.39, no show-stopper problems are expected. At this point we can still revert to upstream version 2.38 if insurmountable problems appear, but to do so may require a mass rebuild to remove new symbols from the ABI/API. * Contingency mechanism binutils: If binutils 2.41 proves too distruptive to assembling and linking the distribution we could revert to 2.40, but given that Rawhide is using 2.41, no show-stopper problems are expected. At this point we can still revert if insurmountable problems appear, but to do so may require a mass rebuild if the defects involve generated binaries.   * Contingency mechanism for gcc: If gcc 14 proves too disruptive to compiling the distribution we could revert to gcc 13.2.  * Contingency deadline: Fedora mass rebuild on 2024-01-17. * Blocks release? ** Yes, upgrading to gcc 14.0 does block the release. ** Yes, upgrading to binutils 2.41 does block the release. ** Yes, upgrading to glibc 2.39 does block the release. ** No, upgrading to gdb 13+ does not block the release. == Documentation == The gcc manual contains the documentation for the release and doesn't need any more additional work. The binutils manual contains the documentation for the release and doesn't need any more additional work. The glibc manual contains the documentation for the release and doesn't need any more additional work. The gdb manual contains the documentation for the release and doesn't need any more additional work. == Release Notes ==  See https://gcc.gnu.org/gcc-14/changes.html for the GNU Compiler Collection version 14 release notes.  The GNU C Library version 2.39 will be released at the beginning of February 2024. The current NEWS notes can be seen here as they are added: https://sourceware.org/git/?p=glibc.git;a=blob;f=NEWS;hb=HEAD The GNU Binary Utilities version 2.41 was released August 2023. The current release notes: https://sourceware.org/pipermail/binutils/2023-July/128719.html -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 0

F40 Change Proposal: Optimized Binaries for the AMD64 Architecture (System-Wide)
by Aoife Moloney 28 Dec '23

28 Dec '23

Wiki -> https://fedoraproject.org/wiki/Changes/Optimized_Binaries_for_the_AMD64_Arc… This is a proposed Change for Fedora Linux. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Additional paths will be inserted into the search path used for executables on systems which have a compatible CPU. Those additional paths will mirror the AMD64 "microarchitecture levels" supported by the glibc-hwcaps mechanism: `x86-64-v2`, `x86-64-v3`, `x86_64-v4`. Systemd will be modified to insert the additional directories into the `$PATH` environment variable (affecting all programs on the system) and the equivalent internal mechanism in `systemd` (affecting what executables are used by services). Individual packages can provide optimized libraries via the glibc-hwcaps mechanism and optimized executables via the extended search path. This optimized code will be used if the CPU supports it. ''Which'' packages provide the optimized code and at which level will be made by individual package maintainers based on benchmark results. == Owner == * Name: [[User:Zbyszek| Zbigniew Jędrzejewski-Szmek]] NOTE: I'm writing and filling this proposal on the last day allowed for system-wide proposals. It is too large for one person. If you are interested, please let me know or even add yourself to the list of Owners. I would love to have more people working on this. == Detailed Description == Fedora binaries for the AMD64 architecture are compiled with code-generation flags that support almost all CPU variants. But newer generations of processors gained additional instructions that may be used to generate faster code. A vendor-independent x86-64 psABI supplement defines four "microachitecture levels": `x86-64-v1` (the baseline, our code targets this), `x86-64-v2` (+`SSE3`, CentoOS targets this), `x86-64-v3` (+`AVX`), `x86-64-v4` (+`AVX512`) [1]. When code is compiled for a higher microarchitecture level it will crash (with `SIGILL`, "illegal instruction") on CPUs which do not support it. Benchmark results show small differences in performance: usually in the range from -5% to 10%, with no discernible difference for most code, but '''some''' applications benefit, with gains of 120% in some benchmarks [e.g. 2, 4]. Over the years, various people have expressed interest in raising the required microarchitecture levels. But we have been very conservative in making changes, because support is missing in many older CPUs that are still in use, and in fact, even in some CPUs produced and sold today. By raising the required level we would make Fedora completely unusable on many machines. It also seems that recompiling ''all'' packages with the changed options would largely be a waste of resources, because for most code it makes no difference. But for some of the numerical or cryptographic code there are noticeable gains and it seems to be worth the effort to provide optimized code. This also makes Fedora more attractive to people interested in optimization. The dynamic linker already has the `glibc-hwcaps` mechanism to load optimized implementations of ''shared objects'' [3]. This means that packages can provide optimized libraries and they linker will be automatically load them from separate directories if appropriate. (For AMD64, this is `/usr/lib64/glibc-hwcaps/x86-64-v{2,3,4}/`.) To extend the glibc-hwcaps mechanism to ''executables'', `systemd` will be modified to extend the search path with appropriate directories. When started, it will check the CPU capabilities and modify the executable search path it has internally and which is also used to set `$PATH` for services. (For AMD64, `/usr/bin/glibc-hwcaps/x86-64-v{2,3,4}/`.) Note: the ELF format provides the IFUNC mechanism to dynamically select a variant of a function (symbol) when an executable is loaded [5]. This is in particular used to load code using specific CPU instructions when those are supported. This mechanism is both more general (because it allows arbitrary selection criteria), more fine-grained (because there can be other variants than just a few fixed microarchitecture levels), and more efficient (because only the parts of the code that benefit from this need to be provided in multiple variants). In particular, glibc already makes extensive use of this to provide optimized code, which is then widely used by other libraries and programs. This means that even though we compile code in a way where the lowest baseline is supported, modern CPU instructions are already widely used. This is one of the reasons why compiling for a higher baseline often doesn't make any difference in benchmarks. The IFUNC mechanism or an equivalent mechanism should generally be preferred. Nevertheless, that needs to be implented in the program or library itself, which is not trivial. The two mechanisms in this Proposal are intended for the packages which do not support IFUNCs or some other equivalent mechanism. [1] https://hackweek.opensuse.org/all/projects/support-glibc-hwcaps-and-micro-a… [2] https://gitlab.archlinux.org/archlinux/rfcs/-/blob/master/rfcs/0002-march.r… [3] https://sourceware.org/pipermail/libc-alpha/2021-February/122207.html [4] https://blog.centos.org/2023/08/centos-isa-sig-performance-investigation/ [5] https://jasoncc.github.io/gnu_gcc_glibc/gnu-ifunc.html Glibc-hwcaps together with the new feature in systemd provide a generic mechanism. It will be up to individual packages to actually provide code which makes use of it. Individual package maintainers are encouraged to benchmark their packages after recompilation, and provide the optimized variants if useful. (I.e. the code in question is measureably faster '''and''' the program is ran often enough for this to make a difference.) The Change Owners will implement the packaging changes for a few packages while developing the general mechanism and will submit those as pull requests. Other maintainers are asked to do the same for their packages. Optimized variants of programs and libraries MAY be packaged in a separate subpackage. The general packaging rules should be applied, i.e. a separate package or packages SHOULD be created if it is files are large enough. Available benchmark results [2,4] are narrow and not very convincing. We should plan an evaluation of results after one release. If it turns out that the real gains are too small, we can scrap the effort. On the other hand, we should also consider other architectures. For example, microarchitecture levels `z{14,15}` for `s390x` or `power{9,10}` for `ppc64le`. Other architectures are not included in this Change Proposal to reduce its scope. == Feedback == == Benefit to Fedora == The developers who are interested in this kind of optimization work can perform it within Fedora, without having to build separate repositories. The users who have the appropriate hardware will gain performance benefits. Faster code is also more energy-efficient. The change will be automatic and transparent to users. Note that other distributions use higher microarchitecture levels. For example RHEL 9 uses x86-64-v2 as the baseline, RHEL 10 will use x86-64-v3, and other distros provide optimized variants (OpenSUSE, Arch Linux). We implement the same change in Fedora in a way that is scoped more narrowly, but should provide the same performance and energy benefits. == Scope == * Proposal owners: ** Extend systemd to set the executable search path using the same criteria as the dynamic linker. ** Implement packaging changes for at least one package with a library and at least one package with executables and submit this as pull requests. ** Provide a pull request for the Packaging Guidelines to describe the changes listed in Description above. * Other developers: ** Do benchmarking and implement packaging changes for other packages if beneficial. * Release engineering: [https://pagure.io/releng/issue/11864 #11864] * Policies and guidelines: TBD. * Trademark approval: N/A (not needed for this Change) * Alignment with Community Initiatives: == Upgrade/compatibility impact == No impact. == How To Test == * Use `/usr/bin/ld.so --help` to check which hwcaps are supported by the system. * Install one or more packages which provide optimized code. * Restart the system or re-login to reinitialize `$PATH`. * Check that appropriate directories are present in `$PATH`. * Run some benchmarks and check that the optimized code is indeed faster. == User Experience == There should be no impact for users. If the optimized code is available and installed for their hardware, various tasks may finish faster and use less energy. == Dependencies == == Contingency Plan == * Contingency mechanism: Undo the changes in packages which introduced them and recompile. * Contingency deadline: Any time. * Blocks release? No. == Documentation == == Release Notes == Packages which benefit from being compiled for higher AMD64 microarchitecture levels (`x86-64-v2`, `x86-64-v3`, `x86_64-v4`) are now provided with optimized variants which will be used automatically on appropriate CPUs. This includes: TBD1, TBD2, TBD3. -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 0

F40 Change Proposal: Move /var/run selinux-policy entries to /run (Self-Contained)
by Aoife Moloney 24 Dec '23

24 Dec '23

wiki -> https://fedoraproject.org/wiki/Changes/Move_var_run_selinux_policy_entries_… This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Actual path for system runtime files moved from /var/run to /run some 10 years ago [1], but the policy has been managed since then in a way that keeps the old entries and have updates still with the incorrect path while the real path is handled by file equivalency feature. This can confuse sysadmins not to be sure which path should be actually used and can also effect in userspace tools not working properly [2]. [1] https://fedoraproject.org/wiki/Features/UsrMove [2] https://bugzilla.redhat.com/show_bug.cgi?id=2241366 == Owner == * Name: Zdenek Pytela * Email: zpytela(a)redhat.com == Detailed Description == The change actually means just replacing "/run = /var/run" file-context equivalency rules with "/var/run = /run". While the change as such is quite simple, it can have effect on other components using their own selinux policy with file-context entries. == Feedback == == Benefit to Fedora == Removing technical debt which originated 10 years ago. More straightforward handling of file-context entries in the /run filesystem. == Scope == * Proposal owners: ** Add all relevant patches to upstream repository ** Ensure the system boots with the targeted policy ** Ensure the system boots with the mls policy ** Ensure updates from older releases work, more specifically with custom selinux packages installed. * Other developers: ** Developers of custom selinux policies need to confirm system updates work. * Release engineering: [https://pagure.io/releng/issues #Releng issue number] (a check of an impact with Release Engineering is needed) * Policies and guidelines: No update required. * Trademark approval: N/A (not needed for this Change) * Alignment with Objectives: == Upgrade/compatibility impact == Users can be affected by this change if they use a local policy with file-context entries in /run which occurs quite rarely, but is possible. == How To Test == * Install a new system and check for error messages and audit records. * Update an existing system and check if all updates completed without an error. * Optionally, install and boot the selinux-policy-mls package. * Check for errors reported by dnf or rpm. == User Experience == There should be no visible change for end users. The change should be transparent, without any further action needed on the system. System admins may need to take an action based on compatibility with the changes. == Dependencies == Components with a custom selinux policy: container-selinux pcp cockpit == Contingency Plan == * Contingency mechanism: Revert all changes in case of serious problems with updates. * Contingency deadline: 2024-02-06 (Branch Fedora Linux 40 from Rawhide) * Blocks release? No * Blocks product? No == Documentation == To be added later. == Release Notes == -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 0

F40 Change Proposal: SPDC Licence Phase 3 (System-Wide)
by Aoife Moloney 24 Dec '23

24 Dec '23

wiki -> https://fedoraproject.org/wiki/Changes/RemovePythonMockUsage This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == The third phase of transition from using Fedora's short names for licenses to [https://spdx.org/licenses/ SPDX identifiers] in the License: field of Fedora package spec files. This phase focuses on finishing migrating packages from ELN set. We still do not expect that all packages from Fedora Linux will be migrated in this phase. == Owner == * Name: [[User:msuchy| Miroslav Suchý]], [[User:jlovejoy| Jilayne Lovejoy]], [[User:dcantrell| David Cantrell]], [[User:ref| Richard Fontana]] * Email: msuchy(a)redhat.com, dcantrell(a)redhat.com, jlovejoy(a)redhat.com, rfontana(a)redhat.com == Detailed Description == This is follow-up of [[Changes/SPDX_Licenses_Phase_2|Phase 2]]. During this phase, all remaining packages should be migrated to use SPDX license identifiers in the License: field of the package spec file. So far, package maintainers have been updating their packages in accordance with the guidance provided at https://docs.fedoraproject.org/en-US/legal/update-existing-packages/ and filing issues in the [https://gitlab.com/fedora/legal/fedora-license-data fedora-license-data repo]. Miroslav has been tracking how many packages that have been updated. Given the large number of packages in Fedora, this progress is good, but slow. The intake of newly discovered licenses is still more than we are able to process. We want to focus on adding the new license to both fedora-license-data and SPDX.org list. At the same time, we want to focus on the ELN subset of Fedora and cooperate with maintainers of these packages to finish the migration of these packages. This Change will be followed by [[Changes/SPDX_Licenses_Phase_4|Phase 4]], where we want to finish the migration of the remaining Fedora packages. == Feedback == See [[Changes/SPDX_Licenses_Phase_1#Feedback|feedback section of Phase 1]] Discussions on the mailing list: * [https://lists.fedoraproject.org/archives/search?q=SPDX+statistics&page=1&ml… regular SPDX Statistics] * [https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org… SPDX - How to handle MIT and BSD] Challenges: * license-fedora2spdx tool uses mapping of legacy Fedora short names to SPDX identifiers using the [https://gitlab.com/fedora/legal/fedora-license-data/-/tree/main fedora-license-data] to suggest SPDX identifiers. Where there is an apparent one-to-one mapping, the package maintainer could simply update the License field: and move on. * for many packages, particularly packages that use "umbrella" legacy short names that may refer to a large set of unrelated or loosely related licenses, further inspection will be needed. Currently, Fedora documentation provides sparse advice on how to do this inspection and thus, a range of methods are used. Progress to date: [https://docs.google.com/spreadsheets/d/1QVMEzXWML-6_Mrlln02axFAaRKCQ8zE807r… Burndow chart] == Benefit to Fedora == The use of standardized identifiers for licenses will align Fedora with other distributions and facilitates efficient and reliable identification of licenses. Depending on the level of diligence done in this transition, Fedora could be positioned as a leader and contributor to better license information upstream (of Fedora). == Scope == * Change Owners: ** Continue adding newly found licenses to fedora-license-data and to SPDX.org list. ** Continue to report progress ** Focus on the ELN subset of Fedora. * Other developers: ** All packages (during the package review) should use the SPDX expression. - this is redundant as this has already been approved since Phase 1, but it should be reminded. ** Migrate the existing License tag from a short name to an SPDX expression. * Release engineering: nothing * Policies and guidelines: all done in previous phases * Trademark approval: N/A (not needed for this Change) * Alignment with Objectives: == Upgrade/compatibility impact == License strings are not used anything in run time. This change will not affect the upgrade or runtime of Fedora. During the transition period, developer tools like rpminspect, licensecheck, etc. may produce false negatives. And we have to define a date where we flip these tools from old Fedora's short names to the SPDX formula. == How To Test == See [[Changes/SPDX_Licenses_Phase_1#How_To_Test|How to test section of Phase 1]] == User Experience == Users should be able to use standard software tools that audit licenses. E.g. for Software Bills of Materials. == Dependencies == No other dependencies. == Contingency Plan == * Contingency mechanism: There will be no way back. We are already beyond of point to return. We are heading to explore strange new worlds... to boldly go where no man has gone before. * Contingency deadline: Beta freeze. But it is expected that not all packages will be converted by that time and the change will continue in the next release. * Blocks release? No. This change has no impact on runtime of any package. == Documentation == [https://docs.fedoraproject.org/en-US/legal/allowed-licenses/ Allowed Licenses] [https://docs.fedoraproject.org/en-US/legal/update-existing-packages/#_proce… Update existing packages] == Release Notes == In Fedora 40, all core RPM packages use SPDX identifiers as a standard. In total XX percent of packages have been migrated to SPDX identifiers. The remaining packages are estimated to be migrated in upcoming releases of Fedora. -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 1

F39 Election Results
by Aoife Moloney 23 Dec '23

23 Dec '23

Hi folks, The elections for the Fedora Linux 39 cycle have completed. ## Fedora Council Akashdeep Dhar is elected to the Fedora Council ## Fedora Engineering Steering Committee (FESCo) The following candidates are elected to FESCo: * Kevin Fenzi * Zbigniew Jędrzejewski-Szmek * David Cantrell * Tomáš Hrčka * Josh Stone ## Fedora Mindshare Committee Luis Bazan is elected to the Fedora Mindshare Committee. Congratulations to all those elected and thank you to the candidates and voters. For more details, see the Community Blog post: https://communityblog.fedoraproject.org/f39-election-results/ Kind regards, Aoife -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 0

F40 Change Proposal: Remove Python Mock Useage (System-Wide)
by Aoife Moloney 23 Dec '23

23 Dec '23

Wiki -> https://fedoraproject.org/wiki/Changes/RemovePythonMockUsage This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == `python-mock` has been [[Changes/DeprecatePythonMock|deprecated since Fedora 34]] - 6 releases ago, but is still in use in many packages. We plan to go through the remaining usages and clean them up, with the goal of retiring `python-mock` from Fedora. == Owner == * Name: [[User:Salimma|Michel Lind]] * Email: michel @ michel-slm.name * Name: [[ User:Gotmax23|Maxwell G]] * Email: maxwell @ gtmx.me == Detailed Description == `python-mock` has been marked as deprecated since Fedora 34, but many packages are still using it Binary packages from 16 source packages require `python3-mock` <nowiki> $ fedrq whatrequires python3-mock -b rawhide -F source --notsrc | wc -l 16 $ repoquery -q --repo=rawhide{,-source} --whatrequires python3-mock | grep -v src$ | wc -l 16 $ repoquery -q --repo=rawhide --whatrequires python3-mock --source | wc -l 16 </nowiki> 124 source packages require `python3-mock` <nowiki> $ fedrq whatrequires python3-mock -b rawhide -F source --src | wc -l 124 $ repoquery -q --repo=rawhide{,-source} --whatrequires python3-mock | grep src$ | wc -l 124 </nowiki> For a combined 134 packages <nowiki> $ fedrq whatrequires python3-mock -b rawhide -F source | wc -l 134 $ (repoquery -q --repo=rawhide{,-source} --whatrequires python3-mock | grep src$; repoquery -q --repo=rawhide --whatrequires python3-mock --source) | pkgname | sort | uniq | wc -l 134 </nowiki> Some of these are false positives - packages not using dynamic build requirements might still be marked as requiring `python-mock` after upstream no longer does so. The rest should be easily portable as described in the previous proposal. Some packages could be dead upstream and not used by anything else in Fedora, in which case we can consider removing them. == Feedback == == Benefit to Fedora == Eventually, we might be able to no longer maintain a standard library backport in a separate package. This will also make branching packages for EPEL easier - there have been several instances where the Fedora package needs to be modified to not use `python-mock` because of a policy decision to not bring `python-mock` into newer EPEL releases (where the default Python already contains `unittest.mock`). == Scope == * Proposal owners: ** Set up a COPR with a stub package providing `python3-mock` and rebuild all packages that list python3-mock as a BR. The ones that successfully build don't actually need it ** Put up PRs for the affected packages. In case the changes are minimal and no response is heard from the package maintainers the PRs will be merged after a sufficient time. ** Set up a second COPR to rebuild packages with the PRs applied to remove `python3-mock`. This will allow us to track progress in one place * Other developers: Package maintainers should review and merge any PRs filed against their packages, and if necessary forward the patches upstream (e.g. by filing upstream PRs). * Release engineering: N/A (not needed for this Change) * Policies and guidelines: N/A (not needed for this Change) * Trademark approval: N/A (not needed for this Change) * Alignment with Community Initiatives: Not specifically, but this is in line with Fedora's "First" Foundation == Upgrade/compatibility impact == N/A == How To Test == Package owners who receive a PR should be able to verify that the CI passes on the PR, and alternatively they can check the provided COPRs == User Experience == No changes == Dependencies == N/A == Contingency Plan == * Contingency mechanism: defer retiring `python-mock` for another cycle * Contingency deadline: Final Freeze * Blocks release? No == Documentation == The previous [[Changes/DeprecatePythonMock#How_to_migrate_to_unittest.mock|howto]] on migrating to unittest.mock still applies. Upstream [https://docs.python.org/3/library/unittest.mock.html unittest.mock documentation] == Release Notes == -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 0

F40 Change Proposal: LLVM 18 (System-Wide)
by Aoife Moloney 23 Dec '23

23 Dec '23

Wiki -> https://fedoraproject.org/wiki/Changes/LLVM-18 This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Update all llvm sub-projects in Fedora Linux to version 18. == Owner == * Name: [[User:tstellar| Tom Stellard]] * Email: <tstellar(a)redhat.com> == Detailed Description == All llvm sub-projects in Fedora will be updated to version 18, and there will be a soname version change for the llvm libraries. Compatibility packages clang17, llvm17, and lld17 will be added to ensure that packages that currently depend on clang and llvm version 17 libraries will continue to work. We may add other compatibility packages too if they're determined to be necessary to maintain functionality in other RPMS that use llvm/clang. We also plan to retire these older compatibility packages (that we own): * llvm14 * llvm15 * llvm16 * clang14 * clang15 * clang16 * lld14 * lld15 * lld16 We will also be asking the maintainers of the following packages to retire them if possible: * llvm7.0 * llvm8.0 * llvm11 * llvm12 * llvm13 Other notable changes: * clang will emit DWARF-5 by default instead of DWARF-4. This matches the upstream default. We have been using DWARF-4 as the default for the last few releases due to: https://bugzilla.redhat.com/show_bug.cgi?id=2064052 * The compatibility packages will now include the same content as the main package. In previous releases, the compatibility packages contained only libraries and headers, and the binaries and other content was stripped out. These packages will be supported for use as dependencies for other RPM packages, but not for general purpose usage by end users. Fedora users should use Clang/LLVM 18. * The compatibility packages added for Fedora 40 will be retired prior to the Fedora 41 branch. * We will be enabling Fat LTO in redhat-rpm-config if this feature is complete in time for the upstream LLVM 18 release. Fat LTO is a feature that allows the compiler to produce libraries that contain LTO bitcode along side the traditional ELF binary code so that the libraries can be linked in both LTO mode and non-LTO mode. gcc also supports this feature and has it enabled in Fedora. In Fedora 39 and older, with LTO enabled, clang produces binaries with only LTO bitcode, so we need to run a post-processing script (brp-llvm-compile-to-elf) on the libraries to convert them to ELF code so they can be used by other packages. Enabling Fat LTO will allow us to remove this script and simplify the build process. ===LLVM Build Schedule=== ====Important Dates==== * Jan 26: Upstream: 18.0.0-rc1 Release * Feb 6: Fedora: f40 branch created * Feb 6: Upstream: 18.0.0-rc2 Release * Feb 20: Fedora: f40 Beta Freeze * Feb 20: Upstream: 18.0.0-rc3 Release * Mar 5: Upstream: 18.0.0 Release * Apr 2: Fedora: f40 Final Freeze ====Plan==== # Build nightly trunk (LLVM 18) snapshots in [https://copr.fedorainfracloud.org/coprs/g/fedora-llvm-team/llvm-snapshots/p… copr]. # Build LLVM 18.0.0-rc1 in COPR. # Build LLVM 18.0.0-rc1 into a rawhide side-tag in Koji. # Build LLVM 18.0.0-rc1 into a f39 side-tag in Koji. # Build LLVM 18.0.0-rc2 into a rawhide side-tag in Koji. # Build LLVM 18.0.0-rc2 into a f39 side-tag in Koji. # Build LLVM 18.0.0-rc3 into a rawhide side-tag in Koji # Build LLVM 18.0.0-rc3 into a f39 side-tag in Koji # Push F39 Bodhi Update with 18.0.0-rc3 (or 18.0.0-rc2 if -rc3 is not ready) as a Beta Freeze exception. # Continue building new release candidates and pushing them to stable until the Final Freeze. We are not planning to push 18.0.0-rc1 into rawhide because the library ABI is not stabilized at that point. Typically, the ABI stabilizes after -rc3, but there are no guarantees from upstream about this. Given the history of minimal ABI changes after -rc3, we feel like it's safe to push -rc3 into rawhide. The worst case scenario would be an ABI change -rc4 or the final release that we force us to patch LLVM to maintain compatibility with the -rc3 ABI. This scenario would not require rebuilding LLVM library users in Fedora, so this would not require much extra work from our team. Updates after 18.0.0-rc3 will generally be very small and can be done after the Final Freeze is over. If we are late packaging release candidates after -rc3 or the final release, we will not ask for a Final Freeze exception, unless they contain a fix for a critical release blocking bug. == Feedback == == Benefit to Fedora == New features and bug fixes provided by the latest version of LLVM. == Scope == * Proposal owners: ** Review existing llvm and clang compatibility packages and orphan any packages that are no longer used. ** Build and test early release candidates of LLVM 18 in COPR. * Other developers: ** Fix build issues found with LLVM-18 or switch their package to use the llvm17 compat libs. The LLVM team will not block Bodhi updates on dependent packages that fail to build or run with LLVM-18. There should be around 6-8 weeks between when -rc1 lands in the koji side-tag and the Final Freeze for package maintainers to fix issues uncovered with the LLVM-18 update. * Release engineering: [https://pagure.io/releng/issue/11845 #11845] * Policies and guidelines: N/A (not needed for this Change) * Trademark approval: N/A (not needed for this Change) * Alignment with Community Initiatives: == Upgrade/compatibility impact == This change should not impact upgrades. == How To Test == The CI tests for the llvm sub-packages in Fedora will be used to catch regressions that might be potentially introduced by the update to LLVM 18. == User Experience == == Dependencies == Packages that depend on one of the llvm packages will need to be updated to work with LLVM18 or will need to switch to using one of the llvm17 compat packages. == Contingency Plan == * Contingency mechanism: (What to do? Who will do it?) N/A (not a System Wide Change) If there are major problems with LLVM 18, the compatibility package provide a way for other packages to continue using LLVM 17. * Contingency deadline: Final Freeze == Documentation == LLVM sub-projects in Fedora have been updated to version 18: *llvm *clang *lld *lldb *compiler-rt *libomp *llvm-test-suite *libcxx *python-lit *flang *mlir *polly *libclc *llvm-bolt == Release Notes == -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 0

F40 Change Proposal: Replace Kubernetes 1.28 in Rawhide (F40) with v1.29 (Self-Contained)
by Aoife Moloney 23 Dec '23

23 Dec '23

Wiki -> https://fedoraproject.org/wiki/Changes/Kubernetes-1.29 This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Replace Kubernetes 1.28 in rawhide (F40) with v1.29. == Owner == * Name: [[User:buckaroogeek| Brad Smith]] * Email: bradley.g.smith(a)gmail.com == Detailed Description == The upstream Kubernetes project just released Kubernetes (K8S) v1.29. The upstream k8s team maintains 3 versions concurrently. Fedora provides a target version of Kubernetes with each release. Fedora 38 has k8s v1.26; Fedora 39 has k8s v1.27, and rawhide (F40) currently has k8s v1.28. If approved this change will replace k8s v1.28 with k8s v1.29 in rawhide for the F40 release. See https://src.fedoraproject.org/rpms/kubernetes for the current k8s packages in Fedora. == Feedback == TBD == Benefit to Fedora == Fedora F40 will have the most current Kubernetes version when F40 is released to production. This will enable Fedora users to deploy and maintain current Kubernetes on Fedora Server and Fedora CoreOS using Fedora provided rpms. == Scope == * Proposal owners: Execute communication plan (community blog and mailing list posts) announcing the change. Test and deploy the update (already available in COPR). * Other developers: Coordinate with the CRI-O packager so that they release a compatible version of CRI-O. Kubernetes and CRI-O have the same release cycle and version numbering, i.e kubernetes v1.29 requires cri-0 1.29 (if cri-o is used as the container runtime interface). * Policies and guidelines: N/A (not needed for this Change) * Trademark approval: N/A (not needed for this Change) * Alignment with Community Initiatives: n/a == Upgrade/compatibility impact == Kubernetes update guidelines do not permit a version skip. For example, if a Kubernetes cluster is currently using Kubernetes v1.27, then in order to update to k8s v1.29, first v1.28 needs to be deployed. Skipping versions, as this proposal does, requires effective communications and an effective replacement. In this instance, Kubernetes v1.28 will be available from COPR. There is another change request in DRAFT form that addresses this problem by shifting to multiple versioned packages of Kubernetes for each release (i.e. the package changes from kubernetes-1.29.f40 to kubernetes1.29-1.29.f40 and so on. The Kubernetes user community is somewhat used to using COPR for past releases. Kubernetes 1.23 was also skipped in Fedora and only available in COPR. Kubernetes 1.25 was initially provided in Fedora 37 but moved to COPR when upstream adopted a version of the go language not available for F37. == How To Test == == User Experience == Users managing kubernetes clusters using Fedora packages on Fedora machines will need to be aware of this change. An unplanned version update can break a cluster, hence rawhide is the best starting point for this update. == Dependencies == None. == Contingency Plan == == Documentation == N/A (not a System Wide Change) == Release Notes == -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 0

F40 Change Proposal: Build Fedora Cloud Edition Images Using Kiwi in Koji (System-Wide)
by Aoife Moloney 23 Dec '23

23 Dec '23

Wiki -> https://fedoraproject.org/wiki/Changes/KiwiBuiltCloudImages This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. == Summary == Fedora Cloud Edition images will be built with [https://osinside.github.io/kiwi/ Kiwi], which will replace the unmaintained [https://github.com/redhat-imaging/imagefactory ImageFactory] tooling that is currently being used to build the cloud base images. We can already build Fedora Cloud Edition images outside of Koji using [https://pagure.io/fedora-kiwi-descriptions composite Kiwi definitions]. However the integration with Koji must be enabled to fulfuill our goal of building official images within the Fedora infrastructure and fully replace the current usage of ImageFactory. This transition is consistent with the direction of the [https://fedoraproject.org/wiki/Cloud/Cloud_PRD Cloud Product Requirements Description (PRD)]. Kiwi provides the Cloud Working Group with a tool that preserve previous choices to build images using composable configurations and to provide a reproducible process for building images related to the cloud edition, including Fedora Cloud Base images for Vagrant, Azure, AWS, GCP, and generic images. This also opens up the ability to run container builds and WSL2 builds using the the composable image definitions to maintain a base image and then update the specifics needed for each specialized image using a smaller configuration file. == Owner == * Name: [[User:davdunc| David Duncan]] * Email: davdunc(a)amazon.com == Detailed Description == While working on the production of cloud images for Fedora Linux 38 and Fedora Linux 39, the cloud-sig team did significant work to support transition from the current ImageFactory-based build tools that are outdated (but still functioning) to a tool that is supported by a broader community. The cloud team has successfully built and tested the creation of images with the <code>kiwi</code> application. Successful builds and tests of image builds supporting all of the previous change proposals and configuration changes to the Fedora Cloud base images has been validated and can be reproduced using the [https://pagure.io/fedora-kiwi-descriptions kiwi descriptions]. The cloud edition WG finds that kiwi provides the most consistent experience with the least number of concerns over our current deliverables today. The cloud working group continues to focus on building support for specific requirements around specialized images that are planned parts of the [https://fedoraproject.org/wiki/Cloud/Cloud_PRD cloud edition PRD] included in section 2.3. == Feedback == We have evaluated a number of existing image build tools as part of this Change. Ultimately, the Fedora Cloud WG chose to adopt kiwi because it retains ideal qualities of our current tooling in a way that benefits the cloud-sig and the community at large. We have cultivated a strong relationship with the upstream project, who has been receptive to our needs and made improvements based on our requirements. Kiwi is not a disruption, but an opportunity to decrease the complexity necessary to produce current and additional use cases immediately and to ensure that builds are execute securely. We are aware of Fedora Workstation's trial of osbuild (the upstream project for [https://console.redhat.com/insights/image-builder Red Hat Image Builder]) for their live image, and have closely evaluated it as an option for Fedora Cloud as well. Discussions with members of the image builder team have been promising, but their mission doesn't directly align with the Cloud Working Group's goals immediately. Without that alignment, we are not prioritizing the same goals today. This is not a shortcoming of the cloud working group or the osbuild tools, it is a difference in timing of feature delivery. Fedora Workstation and Fedora Cloud are two different groups. We use different tools for building images today so their changes are typically independent of those we make. Currently, Fedora Workstation uses Lorax and Fedora Cloud uses ImageFactory and Oz. The cloud working group is working aggressively to eliminate our usage of ImageFactory because it is legacy code and not easily extended. We also evaluated {{package|mkosi}} and decided not to pursue it due to the lack of flexibility to support all the image types we are aiming to offer. Its highly opinionated view of how images should be structured and limited framework for customization make it difficult to recommend as a framework for our builds. Additionally, it cannot support all of Fedora's architectures due to requiring GPT, nor can it fully support Fedora Cloud's preferred disk setup due to the aforementioned opinions of how images should be structured. Finally, when testing the generated images, the results did not line up with how we expected images to be laid out and it caused difficulties when dealing with certain classes of package upgrades (such as bootloader or kernel packages). There is also no Koji plugin at this time for running mkosi builds. == Benefit to Fedora == Most importantly, the kiwi builders eliminate a series of legacy build tools for Fedora Cloud Base images Visible to advanced users: * Allows Fedora Images to be built on many different platforms and distributions without modification to the runners * Extends the composition strategies available to users * Leaves the base image configuration that can be managed to ensure that it meets standard requirements for local virt installations * Includes the ability to leverage user-defined scripting in the image definition. * Adds a koji builder and image definitions that are simple to update and modify * Provides increased time for prioritization of features in the Fedora Images according to user feedback and user requirements * Supports multiple build types, from ISO to raw disk images, and all the way to WSL2 and containers. This also aligns with the Fedora Asahi Remix and its usage of kiwi to build its images, as this lays the groundwork for those images to eventually be built in Fedora infrastructure as support for Apple Silicon Macs gets upstreamed. == Scope == * Proposal owners: ** Build and test [https://pagure.io/fedora-kiwi-descriptions kiwi definition files]: COMPLETE ** Package {{package|kiwi}}: COMPLETE ** Add configuration to [https://pagure.io/pungi-fedora/ Fedora Pungi configuration] * Other developers: ** Enable kiwi plugin in Koji: [https://pagure.io/releng/issue/11726 releng issue #11726] ** Add support for <code>KiwiBuild</code> tasks to {{package|pungi}}: [https://pagure.io/pungi/issue/1710 pungi issue #1710] Submit image build requirements as a kiwi descriptions * Release engineering: [https://pagure.io/releng/issue/11854 #11854] * Policies and guidelines: Fedora Cloud Edition documentation should be updated to reflect the usage of the new tooling and how to use and contribute to it. * Trademark approval: N/A (not needed for this Change) * Alignment with Community Initiatives: All software and requests are consistent with the decision process and similar exceptions across other groups in Fedora. == Upgrade/compatibility impact == The previous methodologies for using Fedora Quickstarts for Fedora Cloud Edition will be retired. The kiwi descriptions will support builds. We will use Toddler and Ansible to deliver images to the various public cloud targets (GCP, AWS, Azure, OCI, etc.) == How To Test == Test by working with the various images # Import the image into a test account for the associated cloud provider(s) # start an instance from that image # login to the instance successfully. == User Experience == this provides a simplified method for creating composable image definitions and overlays. Users will find that there are additional images supporting targeted workloads and build methods. They will find that those images are more readily available. == Dependencies == This Change depends on work in {{package|pungi}} to enable the use of the <code>KiwiBuild</code> Koji task as part of composes. It also depends on release engineering to enable the kiwi plugin in Koji. == Contingency Plan == * Contingency mechanism: Revert back to ImageFactory and continue to support builds using the kickstart (.ks) files for image builds. * Contingency deadline: Beta freeze * Blocks release? Yes == Documentation == Documentation for kiwi is available from [https://osinside.github.io/kiwi the upstream site]. Once the Koji plugin is enabled, we will create accompanying documentation for SIG members on using the functionality. == Release Notes == Fedora Cloud Images are now built with the [https://osinside.github.io/kiwi kiwi] image build tool, using definitions from the [https://pagure.io/fedora-kiwi-descriptions fedora-kiwi-descriptions] repository. This has enabled Fedora Cloud to introduce 64-bit ARM cloud images for Azure and Google Cloud, as well as 64-bit ARM Vagrant images. -- Aoife Moloney Fedora Operations Architect Fedora Project Matrix: @amoloney:fedora.im IRC: amoloney

1 0

F40 proposal: Move /var/run selinux policy entries to /run (System-Wide)
by Zdenek Pytela 21 Dec '23

21 Dec '23

Details: https://fedoraproject.org/wiki/SELinux/Changes/Move_var_run_selinux_policy_… This is a proposed Change for Fedora Linux. This document represents a proposed Change. As part of the Changes process, proposals are publicly announced in order to receive community feedback. This proposal will only be implemented if approved by the Fedora Engineering Steering Committee. Move /var/run selinux-policy entries to /run Summary Actual path for system runtime files moved from /var/run to /run some 10 years ago [1], but the policy has been managed since then in a way that keeps the old entries and have updates still with the incorrect path while the real path is handled by file equivalency feature. This can confuse sysadmins not to be sure which path should be actually used and can also effect in userspace tools not working properly [2]. [1] https://fedoraproject.org/wiki/Features/UsrMove [2] https://bugzilla.redhat.com/show_bug.cgi?id=2241366 Owner Name: Zdenek Pytela Email: zpytela(a)redhat.com Current status Targeted release: Fedora 40 Last updated: 2023-12-20 FESCo issue: <will be assigned by the Wrangler> Tracker bug: <will be assigned by the Wrangler> Release notes tracker: <will be assigned by the Wrangler> Detailed Description The change actually means just replacing "/run = /var/run" file-context equivalency rules with "/var/run = /run". While the change as such is quite simple, it can have effect on other components using their own selinux policy with file-context entries. Benefit to Fedora Removing technical debt which originated 10 years ago. More straightforward handling of file-context entries in the /run filesystem. Scope Proposal owners: Add all relevant patches to upstream repository Ensure the system boots with the targeted policy Ensure the system boots with the mls policy Ensure updates from older releases work, more specifically with custom selinux packages installed. Other developers: Developers of custom selinux policies need to confirm system updates work. Release engineering: #Releng issue number (a check of an impact with Release Engineering is needed) Policies and guidelines: No update required. Trademark approval: N/A (not needed for this Change) Alignment with Objectives: Upgrade/compatibility impact Users can be affected by this change if they use a local policy with file-context entries in /run. How To Test Install a new system and check for error messages and audit records. Update an existing system and check if all updates completed without an error. Optionally, install and boot the selinux-policy-mls package. User Experience There should be no visible change for end users. The change should be transparent, without any further action needed on the system. System admins may need to take an action based on compatibility with the changes. Contingency Plan Contingency mechanism: Revert all changes in case of serious problems with updates. Contingency deadline: 2024-02-06 (Branch Fedora Linux 40 from Rawhide) Blocks release? No Blocks product? No Documentation To be added later. -- Zdenek Pytela Security SELinux team

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

devel-announce December 2023