I definitely like the idea of tracking this better and making sure
tell releng whats needed. In fact, I think that may be all we need,
especially with bodhi2 pushing updates faster than 1 did.
I see 3 parts here:
1. tracking these issues (that will be done by QA with our new trackers)
2. knowing what needs to be done (that I'm trying to figure out in this thread), and
implementing necessary tools if needed
3. making sure it's done (it needs to be part of some QA or RelEng SOP to make sure
it's not forgotten when pushing out the new release)
I am with Kevin here, we have things tightly coupled with mirrors
mirroring, making changes by a day or two throws timings way off. purely
because we have a built in sync buffer of the weekend. To slip the go/no go
decision to Monday we would need to push out the ship date from Tuesday to
Friday to give mirrors syncing time and that is making things somewhat tight.
We really need to slip a week for any slip
OK, a week slip it is.
> > Leaves less time to sync mirrors,
> > update common bugs, etc etc.
> I would say the opposite - all of that can start happening right away, it's
> not blocked on waiting for the FN-x push. So in case the announcement gets
> out on Tuesday as usual, it's the same time, but if it gets pushed back to
> Wednesday or later day, it's more time for these tasks to happen. The only
> exception is that FN-x updates repo, which will get shorter sync time
> because we want to make sure people download the fixed packages, not old
> ones. Currently that behavior is undefined.
we can not put the bits onto the mirrors until we are sure they are the bits,
otherwise we offer the mirrors lots of churn, wasted iops and bandwidth and
I think there's still some misunderstanding here, but I don't want to spend more
time on this. My core topic is to figure out what steps we need to take in order to
deliver the updated (fixed) packages in FN-1/FN-2 repos to all our users on the FN release
day. Slip duration is secondary.
> Looking into existing MM scripts, I have the same opinion, but I
> Adrian to confirm. If we want to make it even simpler, we can drop all
> alternative metadata and leave just the current hash (that script would be
> run once the push containing that critical update is performed).
I am okay with having a way to say ship only the latest metadata.
Great, so this could be the technical means to do what we're looking for. Now I need
to discuss this with MirrorManager devs and ideally convince them to implement this.
we have no way of ensuring always that people are getting the latest
that they have the latest bits installed. but people can always shoot
themselves in the foot. people can and will do a distro update without
updating the running os first.
Yes, they can. We're trying to limit the unintentional foot shooting in this thread.
I.e. once we tell them it's safe to upgrade ("Fedora N+1 is here!"), it
should really be safe to upgrade.
I would suggest not filing a rel-eng ticket
telling us what to do as that will not go over well. We should now sit down
and work out a process. then likely a ticket needs to be filed asking that
process be followed.
That's exactly what I was trying to say, in a different words. I hope that means we
agree with each other :-) This very thread is my attempt to "sit down and work out a
process", and once we have it, and we implement the tool to do the necessary tasks,
RelEng's "New Release SOP" can then include something like:
"If there are any accepted blockers for previous stable releases [link], please look
at the date when the last of them was pushed stable, and run
`./mirror-manager-drop-older-metadata REPO DATE`"
Or QA can ask you to please not forget about this step, if that's what you prefer.
Does that sound OK?