On Mon, May 08, 2023 at 09:24:08AM -0700, Kevin Fenzi wrote:
I'm broadly in favor here, some comments in line...
...snip...
First, we want to ensure that there are up to date [https://src.fedoraproject.org/container/fedora-toolbox fedora-toolbox] OCI images published on [https://registry.fedoraproject.org/ registry.fedoraproject.org] as release-blocking deliverables at critical points in the development schedule, just like the installation ISOs for the Editions from [https://download.fedoraproject.org/pub/fedora/linux/releases/ download.fedoraproject.org]. This must at least happen when an upcoming Fedora release is branched from Rawhide, and for the Beta and Final release candidates. If possible, they should be updated more frequently as part of the nightly composes. We do not expect this to happen after a Fedora release has gone GA.
I think we need some clarity wrt. to the dependency order here. Let's say we:
In order to do this at branch point, we will need to move building this image into the compose process and mark it blocking.
OK, so we build things, but then we need to publish to registry.fp.o, which is asynchrounous (?). When we test the newly built ISOs, do we test them also with the latest image that we get from registry.fp.o? And if we find a bug anywhere in this pipeline, we respin everything?
...snip...
It will be beneficial to consider the [https://src.fedoraproject.org/container/fedora-toolbox fedora-toolbox] images as release-blocking deliverables because Fedora's [https://opencontainers.org/ OCI] infrastructure is often broken. Here are [https://pagure.io/releng/issue/11092 two] [https://pagure.io/releng/issue/11367 recent] examples of <code>fedpkg container-build</code> not working. In the second case, it was preventing the images from being rebuilt to pull in an [https://bugzilla.redhat.com/show_bug.cgi?id=2170878 important] bug-fix. The broken infrastructure prevents regular Fedora contributors from jumping in to rebuild and publish the images at critical points in the development schedule. Making them release-blocking deliverables would attract greater attention and scrutiny from release engineering and ensure that a Fedora development cycle does not proceed with broken or outdated or missing <code>fedora-toolbox</code> images.
I'd like to note that making this blocking doesn't waive any kind of magic wand that makes our infrastructure more reliable. ;) The container build pipeline is a long collection of fragile things. It may well result in us slipping more based on things not working. ;(
Hmm, quoting from https://pagure.io/releng/issue/11092:
Also the aarch64 cluster is running on Fedora 33 boxes, so we should probably try to do a full redeploy :-(
We can't upgrade it from f33 because docker is no longer in f34+ and openshift origin / 3.11 doesn't support any newer either.
Is this still true? I don't think we want to make the Fedora release process contingent on something that requires F33.
Zbyszek