On Thu, May 30, 2019 at 3:18 PM Kevin Fenzi <kevin(a)scrye.com> wrote:
> As discussed in the EPEL SIG meeting yesterday, I've written up my
> thoughts on how to handle epel8 branches.
TLDR: I like it. ;)
> # Considerations
> * The process must be simple for a Fedora packager to adapt to
> * It must be possible to stage big (possibly backwards-incompatible) changes
> * Where possible, the packager experience should be the same as EPEL 7
> # Proposal
> There will be two branches created for EPEL 8 in dist-git for each component:
> ## epel8:
> Most packagers will do their work here. This branch will be set up by
> default with a package.cfg file containing:
> targets = epel8 epel8-rawhide
> Recent fedpkg supports using package.cfg files in the root of the
> dist-git repository to trigger builds for multiple releases at the
> same time.
> ## epel8-rawhide:
> This branch will be left alone until and unless the packager decides
> that they want to stage a major (possibly incompatible) change for the
> next RHEL 8.Y minor release. At that time, they will need to remove
> the package.cfg file from the epel8 branch and manually merge the
> proposed changes desired into the epel8-rawhide branch and build
Also might there be people who want to always keep something in rawhide
and never push it to the stable stream? Or do we want to encourage only
things destined for the next minor change land in epel8-rawhide?
> The package.cfg setup will mean that running `fedpkg build` in the
> epel8 branch will cause it to be built both for the
> epel8-rawhide-candidate and epel8-stable-candidate tags in Koji.
How about we just call the stable tag 'epel8-candidate' ?
Sure, that's actually a vestige of my first draft of this, which I
rewrote which had "epel8" as the combined repo and "epel-stable" as
the branched-off one for when you wanted to lock things. After
consideration, I realized that packagers acting like EPEL 7 (generally
only doing the stable, compatible release) would be the common case
and I reversed that to the version you see here. I left the tag names
specific to eliminate ambiguity, but that's entirely a cosmetic choice
and as long as they're unique and consistent, I don't care what they
> Packages built for epel8-rawhide-candidate will behave similarly
> Rawhide in Fedora and be signed and tagged into an epel8-rawhide
> Packages built for epel8-stable-candidate will behave similarly to
> Fedora stable releases and be required to go through Bodhi to get to
> an epel8 compose (and associated epel8-testing compose).
> For packages operating in the default configuration, the packager will
> need to build in the epel8 branch and then submit the built package to
> Bodhi, just as they would have done for EPEL 7. The side-effect here
> is that the build will also produce a build that goes to the
> epel8-rawhide repository without Bodhi intervention.
Or we could at some point hook in gating, just like fedora rawhide.
Sorry, just had a dream of a pleasant future. ;)
I suppose I should have phrased that differently, or just said
"processed in the same manner as Fedora Rawhide". But yes, if we can
get these properly gated, that's also great.
> When the time comes where an incompatible change needs to land, they
> must be coordinated to land on an approved schedule. The exact
> mechanism of scheduling and coordinating this is out of scope for this
> document and will be decided on by the EPEL Steering Committee.
Yeah, but we should probibly try and figure it out.
I was worried that the logistics of this would derail the conversation
about the branching, so I tried to preempt that. The timing is
something that will be a policy, the rest of this proposal is about
I guess there is:
A. Right after the new minor release comes out
B. Right after the new minor release comes out for CentOS
C. Some arbitrary time after the new minor release comes out.
No comment at this time :)
> At this time, the packager must remove the package.cfg file from the
> epel8 branch and package the new version in the epel8-rawhide branch.
> With the package.cfg file removed from the epel8 branch, builds in
> that branch will be built only for the epel8-stable-candidate tag. As
> before, composes including these builds will be managed by Bodhi
> updates. Building from the epel8 branch will therefore not be
> automatically built for epel8-rawhide any longer.
> Builds intended for the epel8-rawhide repository will need to be built
> instead from the epel8-rawhide branch, which will build against the
> epel8-rawhide-candidate target, which will then be signed and pushed
> to the epel8-rawhide repository like before.
> Once the package is approved to be promoted from the epel8-rawhide
> compose to the stable compose, the package.cfg should be recreated in
> the epel8 branch (this can be automated to make it easier) and a new
> build will be made in the epel8 branch that will produce builds in the
> epel8-stable-candidate and epel8-rawhide-candidate tags, with the
> former then being submitted to Bodhi. This new build must naturally
> have a higher ENVR to preserve the upgrade path.
> ## %dist tag
> Packages built against epel8-rawhide-candidate will be built with a
> %dist tag of .epel8_rawhide
> Packages built against epel8-rawhide-candidate will be built with a
> %dist tag of .epel8_Y where “Y” is the latest stable release of RHEL
> This dist tag structure ensures that the version of the package in the
> stable epel8 repository will win out over the one in the epel8-rawhide
> repository if all other aspects of the EVR are the same. (So one would
> only pick up a newer version from epel8-rawhide if it was indeed a
> higher version number.)
It does also mean you have to do another build of anything to get it
stable. The rawhide build never goes stable, just a rebuilt version of
it does... thats a bit more work, but not too bad.
Yes, and that was a conscious choice I was making here. It's a little
heavy-handed, but it means we are less able to move unstable content
into the stable compose without conscious effort.
> # Historical Composes
> Since major changes may occur at RHEL 8.Y releases, we want to support
> allowing our users to lock onto a repository that matches that
> release. For this, we will generate historical composes, which will
> match the stable package set of the prior minor release once the new
> minor release comes out.
> At 00:00 UTC of the day following a new RHEL 8.Y release, an updated
> epel-release package will be pushed, updating the %dist tag to the new
> .epel8_Y value. All new builds will thus have the new dist tag. A
> script will be run at this time to apply a new Koji tag (epel8-8.Y) to
> the latest build of a package with one of the following tags: [
> epel8-stable, epel8-stable-pending ]. A compose of the epel8.Y
> repository will be created at this time from all packages currently
> tagged as epel8-8.Y.
So, we will also have to unpush/obsolete/close all pending bodhi updates
right? Since things will need rebuilding with the new dist tag for the
That's one point I wasn't prepared to make a firm statement on. We're
presumably *not* going to be requiring a mass-rebuild at every
minor-release point. And if other content in EPEL still has the older
dist-tag, does it really make sense to abandon pending updates? I
think it's good enough that we have a snapshot of what was stable at
the cutover and not worry too much about the actual dist tag in the
> Historical composes are intended to be frozen and unchanging, but this
> approach leaves open the possibility of tagging other builds into
> epel8-8.Y and regenerating the compose if the need arises. It will
> need to be communicated that these repositories will not receive
> updates and are intended to be only a snapshot of the past that is
> known to work with a particular RHEL 8.Y base.
Yeah, I really hope we can avoid that since it will be a lot more work.
Thanks very much for writing this up!
Glad I could help.