I just thought I would share a short status update on koji builders with
everyone. As many of you know, we like to keep the Fedora koji builders
on the current most recent stable Fedora release. This is to make sure
we have compatible rpm, etc to build rawhide and all our stable updates.
TLDR summary: All Fedora koji builders are now on Fedora 31 and are
using only python3 (the hub is still python2 however).
Due to a bug in the linux kernel on armv7, we never upgraded to Fedora
30 when it was released, so builders stayed on Fedora 29 for the last
cycle. Happily that bug was tracked down and fixed and I have been
moving them to Fedora 31 for the past month or so.
Normally it doesn't take to long to reinstall everything (Thanks
ansible!) but this time it's been a longer road due to: New aarch64
hardware that needed racking/cabling/setup (many thanks to Smooge for
getting this done), firmware updates for most all hardware, RHEL8
reinstalls for virthosts that run buildvm's, and many other items.
We still have pending:
* Some more aarch64 hardware to bring online early in the new year.
* A bug around armv7 buildvm's with more than 24GB memory. While thats
being investigated all the buildvm-armv7 instances have just 24GB
* A bug around power9 hosts, causing our cloud/container image builds to
fail. This was thought fixed, but doesn't seem to be yet.
* Daily db dumps still cause slowness, likely in the new year we will
schedule an outage and patition the problem tables.
A few stats:
155 total enabled builders
47 x86_64 builders
32 ppc64le builders
32 aarch64 builders
27 armhfp builders
24 s390x builders
Around 10,000 builds a month.
Close to 1.1 million builds since fc6 days
Show replies by date