On Thu, Nov 18, 2021 at 1:04 PM Mohan Boddu <mboddu@bhujji.com> wrote:
On Thu, Nov 18, 2021 at 2:32 PM Troy Dawson <tdawson@redhat.com> wrote:
>
> In our last EPEL Steering Committee meeting, Carl brought up a new proposal for our epel9 / epel9-next rollout.  Sometimes IRC isn't the best way to explain things like that, it got a little confusing.  Carl and I had a good video chat to clean up confusion and talk about some pros and cons of the various proposals.
> Here are the three proposals.
>
> * PLAN A
> Plan A is basically what we've been working towards for the past couple of months.
> - launch epel9-next now-ish (ideally aligned with c9s launch promotion)
> - After RHEL9 goes GA
> -- perform a mass branch and mass rebuild to populate epel9
> -- launch epel9 after RHEL9 GA is launched.
>
> ** Plan A Pros:
> - epel9-next and epel9 are only set up once, and not changed
> - ready to go now
>
> ** Plan A Cons:
> - complexity and added work of mass branch and mass rebuild
> - mass rebuild will have a moderate rate of failure due to buildroot differences (unshipped devel packages)
> - epel9 not available at rhel9 ga
> - confusing messaging to packagers:
> -- target epel9-next for ~6 months
> -- after epel9 exists target that instead, only use epel9-next when needed
> - confusing messaging to users:
> -- use epel9-next now for c9s and rhel9 beta
> -- use epel9-next temporarily at rhel9 launch but don’t leave it enabled
> -- use epel9 primarily once it exists
>

I like this plan as it is my brain child, it is what we have been
communicating and planning, it doesn't really diverge from what EPEL
has been. But... (see my opinion on Plan B)

>
> * PLAN B
> - epel9-next stays the way it is currently setup.
> - Setup epel9 using RHEL9 Beta for the buildroot.
> -- Pull in any errata as it comes.
> -- Use the repos you would for RHEL9 GA: AppStream, BaseOS, CRB
> - Launch epel9 and epel9-next together (In 1-2 weeks).
> - When RHEL9 GA is released, switch epel9 buildroot from RHEL9 Beta to RHEL9 GA
>
> ** Plan B Pros:
> - simple messaging to packagers:
> -- epel9 is the primary target, use epel9-next only when needed (same as epel8-next)
> - simple messaging to users:
> -- use epel9 everywhere (epel-next-release is a recommends on c9s)
> - no mass branching
> - no mass rebuild

Maybe we can run a mass rebuild to capture all the differences between
rhel9 beta and rhel9 ga. That would solve one of the con's below.


For Plan B or C, doing a mass rebuild doesn't feel right.
It should be fairly easy to tell if a package is going to install or not.
The packages only need to be rebuilt if they cannot be installed.
 
> - No confusion from using the full CentOS Stream buildroot
> -- epel9 buildroot will only have AppStream, BaseOS and CRB
>
> ** Plan B Cons:
> - potential for large divergence between rhel9 beta and ga
> - changing our messaging right before the launch
>

I didn't see one question answered here, the maintainer has to request
both epel9 and epel9-next branch or up until rhel9 ga if a maintainer
requests epel9 branch they will also get epel9-next branch (like it
was set up for epel8-playground).

If you request an epel9 branch, you do not automatically get an epel9-next branch.
We don't do that for epel8, nor do we do it for epel9.
And yes, I've tested it.
 
If we run a mass rebuild, then it
wont be much different from that of Plan A, except there might be a
few (or more, depending on delta) packages that will work on rhel9 ga.
So, I am not against this idea as well. One thing I would like to
mention here is, even if we can setup this way since rhel 9 beta is
out, we cannot do that same thing during rhel10 time as we might setup
epel10-next way in advance before the rhel10 beta. We cannot keep this
plan consistent going further.


You have a good point there about not being able to do this for epel10.
 
>
> * PLAN C
> - epel9-next stays the way it is currently setup.
> - setup up epel9 using c9s for the buildroot
> -- Use the repos you would for RHEL9: AppStream, BaseOS, CRB
> - freeze epel9 buildroot before c9s switches to 9.1 content
> - launch epel9 and epel9-next together (1-2 weeks)
> - switch epel9 buildroot from frozen c9s to rhel9 ga later
>
> ** Plan C Pros:
> - simple messaging to packagers:
> -- epel9 is the primary target, use epel9-next only when needed (same as epel8-next)
> - simple messaging to users:
> -- use epel9 everywhere (epel-next-release is a recommends on c9s)
> - no mass branching
> - no mass rebuild
> - No confusion from using the full CentOS Stream buildroot
> -- epel9 buildroot will only have AppStream, BaseOS and CRB
>
> ** Plan C Cons:
> - potential infrastructure complexity of freezing the epel9 buildroot
> - changing our messaging right before the launch
>

I can see how its advantageous, but I dont like this plan, since there
are some uncertainties, we dont really know when c9s will start
pointing to rhel 9.1, we can only guestimate currently. Even if we
know for sure, we have to ask centos stream folks to freeze their work
until we sort out things on our side. And what if the package versions
changed in rhel9 ga, because they found an issue in that build.


Why would the CentOS Stream folks need to freeze their work?
Wouldn't we just stop syncing?

Would the packages match RHEL9 GA exactly?  No.
Would they be much much closer than if we used RHEL9 Beta?  Yes
 
>
> Please let us know what you think.
> If we do choose to go with Plan B or C we will need to make the decision fairly quickly.

I am okay with either plan A or plan B.