Hi, folks! So I've (finally) got ready an initial round of draft changes to various wiki pages for the purpose of implementing the 'No More Alphas' Change. You can find all the drafts in the NoMoreAlphas category:
https://fedoraproject.org/wiki/Category:NoMoreAlpha
For each page I cloned the original text and saved it as the first edit, so you can use the 'History' tab's diff feature to see the actual changes to each page (just compare the current state to the first edit).
To summarize the changes, at least as I remember them, and call out some significant choices:
* Obviously, all Alpha-related points on the schedule template on the Fedora Release Life Cycle page were removed. For schedule points that were previously derived from Alpha points that got removed, I adjusted them to derive from some other still-remaining point.
* My proposal for 'what do we do about release criteria / validation' is basically: the 'Fedora 27 Alpha Release Criteria' page gets renamed 'Basic Release Criteria' (note: not versioned, I don't think it should be), and we document that *all* composes - not just Beta and Final candidate composes, but also Rawhide and Branched nightlies - will be automatically tested for (and release of them gated on) compliance with them. Which is more or less what's proposed in the Change. All external references to the 'Alpha' criteria get changed to 'Basic' (e.g. this is what changed on the other criteria pages, and on the test matrix template pages). Practically speaking, we currently have automated testing for *most* of the existing Alpha criteria, but a few items aren't covered. We can choose to move these to the Beta criteria, or leave them on Basic and just accept that *actual* coverage doesn't quite meet what we aspire to. I do *NOT* propose to have any kind of blocker tracking bug for the Basic release criteria; it doesn't seem to fit in the process, there is no Alpha release to block, and we can't realistically block nightly composes on manual test results. So a tracker bug wouldn't really have any reason to exist. In the case where a violation of the Basic criteria makes it into composes despite the automated testing, it should be marked as a Beta blocker.
* There is a problem with what to do about the Change schedule: our two sources for it don't actually agree on what the schedule *is*. The Changes Policy page - https://fedoraproject.org/wiki/Changes/Policy - claims that the 'Completion deadline' "falls on the same day as the Alpha milestone freeze", but the Fedora Release Life Cycle page - https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle - claims it falls on the same day as the branch point, which is two weeks *before* the Alpha freeze. Given this inconsistency I didn't really feel confident proposing a draft for the Changes/Policy page yet; probably it'd be good for FESCo(?) as owners of the Change process to decide when they actually want the key dates of the Change process to be, In A World Without Alphas. If FESCo wants to figure that out and let me know, I can draft the changes.
Please do look these over and provide any kind of feedback! Thanks.
On Wed, Aug 02, 2017 at 05:16:28PM -0700, Adam Williamson wrote:
Hi, folks! So I've (finally) got ready an initial round of draft changes to various wiki pages for the purpose of implementing the 'No More Alphas' Change. You can find all the drafts in the NoMoreAlphas category: https://fedoraproject.org/wiki/Category:NoMoreAlpha
Thanks for doing this!
- There is a problem with what to do about the Change schedule: our two
sources for it don't actually agree on what the schedule *is*. The Changes Policy page - https://fedoraproject.org/wiki/Changes/Policy - claims that the 'Completion deadline' "falls on the same day as the Alpha milestone freeze", but the Fedora Release Life Cycle page - https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle - claims it falls on the same day as the branch point, which is two weeks *before* the Alpha freeze. Given this inconsistency I didn't really feel
It's even worse: on the FESCo-approved F27 schedule, the Completion deadline is neither of those, but instead at two weeks *before* the branch point. This is almost certainly my fault by oversight when I was proposing what the schedule would look like without an alpha.
On Thu, Aug 03, 2017 at 09:51:56AM -0400, Matthew Miller wrote:
claims that the 'Completion deadline' "falls on the same day as the Alpha milestone freeze", but the Fedora Release Life Cycle page - https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle - claims it falls on the same day as the branch point, which is two weeks *before* the Alpha freeze. Given this inconsistency I didn't really feel
It's even worse: on the FESCo-approved F27 schedule, the Completion deadline is neither of those, but instead at two weeks *before* the branch point. This is almost certainly my fault by oversight when I was proposing what the schedule would look like without an alpha.
Ooh. We discussed this. https://meetbot.fedoraproject.org/teams/fesco/fesco.2017-03-17-16.00.log.htm... down at about 17:21:38. I think we may have been suffering from meeting fatigue at that point, but the crucial but if the conversation was:
17:21:22 <mattdm> what about the change checkpoint? 17:21:38 <mattdm> (and if we push that back, should we also push back the self-contained changes deadline?) 17:21:42 <dgilmore> mattdm: the earlier changes are submitted the better 17:21:43 <mattdm> (sorry to reopen all of that) 17:21:53 <mattdm> ok. so, leave the deadline? 17:22:06 <dgilmore> I think so yes 17:22:11 <mattdm> I guess having the checkpoint earlier is ok too 17:22:24 <dgilmore> after a cycle or two we can evaluate if we need to make changes 17:22:37 <mattdm> definitely 17:22:37 <sgallagh> I agree with keeping the change timeline where it is 17:23:04 <mattdm> ok, so, now, previously almost-voted-on schedule with everything the same but the branch pushed back by two weeks
.... so I think the F27 schedule as it is represents the current policy. I'm, as of right now, not particularly convinced of the _realism_ of this and am thinking we should put it back to being *at* the branch point. In a magical world we would at branch point decide to pull in only changes that are complete, but we don't currently have any way to do that.
On Wed, Aug 02, 2017 at 05:16:28PM -0700, Adam Williamson wrote:
quite meet what we aspire to. I do *NOT* propose to have any kind of blocker tracking bug for the Basic release criteria; it doesn't seem to fit in the process, there is no Alpha release to block, and we can't realistically block nightly composes on manual test results. So a tracker bug wouldn't really have any reason to exist. In the case where a violation of the Basic criteria makes it into composes despite the automated testing, it should be marked as a Beta blocker.
Will the blockerbugs app start showing F28 blockers at the F27 branch point? Or sometime later? From my point of view, the sooner the better.
On Thu, 2017-08-03 at 21:18 -0400, Matthew Miller wrote:
On Wed, Aug 02, 2017 at 05:16:28PM -0700, Adam Williamson wrote:
quite meet what we aspire to. I do *NOT* propose to have any kind of blocker tracking bug for the Basic release criteria; it doesn't seem to fit in the process, there is no Alpha release to block, and we can't realistically block nightly composes on manual test results. So a tracker bug wouldn't really have any reason to exist. In the case where a violation of the Basic criteria makes it into composes despite the automated testing, it should be marked as a Beta blocker.
Will the blockerbugs app start showing F28 blockers at the F27 branch point? Or sometime later? From my point of view, the sooner the better.
It tends to go wrong in various ways when you have multiple releases 'active' at once, so we've stopped doing that lately, I'm afraid. You can quite easily deploy a local instance, though, and configure it to show the F28 blocker trackers; the bugs themselves are created two releases in advance, so the F28 blocker tracker bugs already exist.
On Thu, Aug 03, 2017 at 08:50:09PM -0700, Adam Williamson wrote:
Will the blockerbugs app start showing F28 blockers at the F27 branch point? Or sometime later? From my point of view, the sooner the better.
It tends to go wrong in various ways when you have multiple releases 'active' at once, so we've stopped doing that lately, I'm afraid. You can quite easily deploy a local instance, though, and configure it to show the F28 blocker trackers; the bugs themselves are created two releases in advance, so the F28 blocker tracker bugs already exist.
Is there any chance of running that at, say, https://qa.fedoraproject.org/blockerbugs-nextrelease/ instead of a local instance?
On Fri, 2017-08-04 at 00:38 -0400, Matthew Miller wrote:
On Thu, Aug 03, 2017 at 08:50:09PM -0700, Adam Williamson wrote:
Will the blockerbugs app start showing F28 blockers at the F27 branch point? Or sometime later? From my point of view, the sooner the better.
It tends to go wrong in various ways when you have multiple releases 'active' at once, so we've stopped doing that lately, I'm afraid. You can quite easily deploy a local instance, though, and configure it to show the F28 blocker trackers; the bugs themselves are created two releases in advance, so the F28 blocker tracker bugs already exist.
Is there any chance of running that at, say, https://qa.fedoraproject.org/blockerbugs-nextrelease/ instead of a local instance?
I mean, in *theory* we could for sure, it's just yet another maintenance task for someone, and I'm not jumping up to volunteer for it right now...if we could script the update of the instances a bit more (right now one of us has to go into the admin UI and update it manually every milestone), maybe I'd be more inclined :P
On Tue, Aug 08, 2017 at 04:24:16PM -0700, Adam Williamson wrote:
Is there any chance of running that at, say, https://qa.fedoraproject.org/blockerbugs-nextrelease/ instead of a local instance?
I mean, in *theory* we could for sure, it's just yet another maintenance task for someone, and I'm not jumping up to volunteer for it right now...if we could script the update of the instances a bit more (right now one of us has to go into the admin UI and update it manually every milestone), maybe I'd be more inclined :P
Is the maintenance headache in installing and running/updating the service, or is it in this milestone updating? If it's the latter, we could probably ask Jan to handle it as part of FPgM duties. Would that help? (Probably for both the "nextrelease" *and* current app.)
On Tue, 2017-08-08 at 23:45 -0400, Matthew Miller wrote:
On Tue, Aug 08, 2017 at 04:24:16PM -0700, Adam Williamson wrote:
Is there any chance of running that at, say, https://qa.fedoraproject.org/blockerbugs-nextrelease/ instead of a local instance?
I mean, in *theory* we could for sure, it's just yet another maintenance task for someone, and I'm not jumping up to volunteer for it right now...if we could script the update of the instances a bit more (right now one of us has to go into the admin UI and update it manually every milestone), maybe I'd be more inclined :P
Is the maintenance headache in installing and running/updating the service, or is it in this milestone updating? If it's the latter, we could probably ask Jan to handle it as part of FPgM duties. Would that help? (Probably for both the "nextrelease" *and* current app.)
Well kinda both, though adding a second server doesn't add a *huge* maintenance burden (just a minor annoyance). Rather than just farming the work off onto someone else, though, it'd be more efficient for someone (anyone...) to take a bit of time to automate the milestone and release updates, it shouldn't be *too* crazily difficult to do (though we'd need to find some kind of way to identify when a milestone release happens, I guess).
To be clear, if this isn't just an idle thought but a specific request that "yes, blockerbugs-nextrelease would be an extremely useful thing for me, please do it", we probably *can* do that. If that's the case, please file a ticket in fedora-qa pagure and we'll likely get to it.
On Wed, Aug 09, 2017 at 05:41:34PM -0700, Adam Williamson wrote:
To be clear, if this isn't just an idle thought but a specific request that "yes, blockerbugs-nextrelease would be an extremely useful thing for me, please do it", we probably *can* do that. If that's the case, please file a ticket in fedora-qa pagure and we'll likely get to it.
Sorry, yeah, I know I don't always do a great job of separating idle thoughts from things I think are really important (or from Official FPL Requests). In this case, it would be hugely helpful for me in my (and Jan's) tracking of overall Fedora planning/development/release status. I'll file the ticket. Thanks.
On Thu, Aug 3, 2017 at 2:16 AM, Adam Williamson adamwill@fedoraproject.org wrote:
Hi, folks! So I've (finally) got ready an initial round of draft changes to various wiki pages for the purpose of implementing the 'No More Alphas' Change. You can find all the drafts in the NoMoreAlphas category:
https://fedoraproject.org/wiki/Category:NoMoreAlpha
For each page I cloned the original text and saved it as the first edit, so you can use the 'History' tab's diff feature to see the actual changes to each page (just compare the current state to the first edit).
Thanks, Adam, awesome work. I'm a little late to the party, sorry. I read through the changed and they all look reasonable. Just a single question:
https://fedoraproject.org/wiki/User:Adamwill/NoMoreAlpha/BugZappers/ HouseKeeping/Trackers Shouldn't we remove the F27/F28 Alpha tracker bugs?
On 08/03/2017 05:46 AM, Adam Williamson wrote:
- My proposal for 'what do we do about release criteria / validation'
is basically: the 'Fedora 27 Alpha Release Criteria' page gets renamed 'Basic Release Criteria' (note: not versioned, I don't think it should be), and we document that *all* composes - not just Beta and Final candidate composes, but also Rawhide and Branched nightlies - will be automatically tested for (and release of them gated on) compliance with them. Which is more or less what's proposed in the Change. All external references to the 'Alpha' criteria get changed to 'Basic' (e.g. this is what changed on the other criteria pages, and on the test matrix template pages). Practically speaking, we currently have automated testing for *most* of the existing Alpha criteria, but a few items aren't covered. We can choose to move these to the Beta criteria, or leave them on Basic and just accept that *actual* coverage doesn't quite meet what we aspire to. I do *NOT* propose to have any kind of blocker tracking bug for the Basic release criteria; it doesn't seem to fit in the process, there is no Alpha release to block, and we can't realistically block nightly composes on manual test results. So a tracker bug wouldn't really have any reason to exist. In the case where a violation of the Basic criteria makes it into composes despite the automated testing, it should be marked as a Beta blocker.
On a slightly different thought, if we run all existing Alpha criteria tests in rawhide, we can then probably look at existing Alpha blocker as Branch blocker.. i.e, we don't branch unless the blockers are fixed and thereby keeping rawhide at Alpha quality all the time. We might want to call 'Basic Release Criteria' as 'Basic Branch Criteria'.
Regards, Sudhir
On Tue, Aug 22, 2017 at 07:18:04PM +0530, Sudhir D wrote:
On a slightly different thought, if we run all existing Alpha criteria tests in rawhide, we can then probably look at existing Alpha blocker as Branch blocker.. i.e, we don't branch unless the blockers are fixed and thereby keeping rawhide at Alpha quality all the time. We might want to call 'Basic Release Criteria' as 'Basic Branch Criteria'.
Hmmmm. I guess this would avoid needing to fix problems on both the branch and rawhide. But, sometimes the fix should be different (e.g. backport or quick fix on branch, update to new version or other long term solution on rawhide).
On 08/22/2017 07:34 PM, Matthew Miller wrote:
On Tue, Aug 22, 2017 at 07:18:04PM +0530, Sudhir D wrote:
On a slightly different thought, if we run all existing Alpha criteria tests in rawhide, we can then probably look at existing Alpha blocker as Branch blocker.. i.e, we don't branch unless the blockers are fixed and thereby keeping rawhide at Alpha quality all the time. We might want to call 'Basic Release Criteria' as 'Basic Branch Criteria'.
Hmmmm. I guess this would avoid needing to fix problems on both the branch and rawhide. But, sometimes the fix should be different (e.g. backport or quick fix on branch, update to new version or other long term solution on rawhide).
Such fix must then be tracked separately for branched version.
On Wed, 2017-08-02 at 17:16 -0700, Adam Williamson wrote:
Hi, folks! So I've (finally) got ready an initial round of draft changes to various wiki pages for the purpose of implementing the 'No More Alphas' Change. You can find all the drafts in the NoMoreAlphas category:
https://fedoraproject.org/wiki/Category:NoMoreAlpha
For each page I cloned the original text and saved it as the first edit, so you can use the 'History' tab's diff feature to see the actual changes to each page (just compare the current state to the first edit).
To summarize the changes, at least as I remember them, and call out some significant choices:
- Obviously, all Alpha-related points on the schedule template on the
Fedora Release Life Cycle page were removed. For schedule points that were previously derived from Alpha points that got removed, I adjusted them to derive from some other still-remaining point.
- My proposal for 'what do we do about release criteria / validation'
is basically: the 'Fedora 27 Alpha Release Criteria' page gets renamed 'Basic Release Criteria' (note: not versioned, I don't think it should be), and we document that *all* composes - not just Beta and Final candidate composes, but also Rawhide and Branched nightlies - will be automatically tested for (and release of them gated on) compliance with them. Which is more or less what's proposed in the Change. All external references to the 'Alpha' criteria get changed to 'Basic' (e.g. this is what changed on the other criteria pages, and on the test matrix template pages). Practically speaking, we currently have automated testing for *most* of the existing Alpha criteria, but a few items aren't covered. We can choose to move these to the Beta criteria, or leave them on Basic and just accept that *actual* coverage doesn't quite meet what we aspire to. I do *NOT* propose to have any kind of blocker tracking bug for the Basic release criteria; it doesn't seem to fit in the process, there is no Alpha release to block, and we can't realistically block nightly composes on manual test results. So a tracker bug wouldn't really have any reason to exist. In the case where a violation of the Basic criteria makes it into composes despite the automated testing, it should be marked as a Beta blocker.
- There is a problem with what to do about the Change schedule: our two
sources for it don't actually agree on what the schedule *is*. The Changes Policy page - https://fedoraproject.org/wiki/Changes/Policy - claims that the 'Completion deadline' "falls on the same day as the Alpha milestone freeze", but the Fedora Release Life Cycle page - https://fedoraproject.org/wiki/Fedora_Release_Life_Cycle - claims it falls on the same day as the branch point, which is two weeks *before* the Alpha freeze. Given this inconsistency I didn't really feel confident proposing a draft for the Changes/Policy page yet; probably it'd be good for FESCo(?) as owners of the Change process to decide when they actually want the key dates of the Change process to be, In A World Without Alphas. If FESCo wants to figure that out and let me know, I can draft the changes.
Please do look these over and provide any kind of feedback! Thanks.
As there was no opposition to these changes, I have gone ahead and implemented them all, save for the changes to the Release Life Cycle page, as those conflict with some changes planned by jkurik to some extent. I'll review the status of that tomorrow and make any changes to that page that I still think should be made, without unilaterally deciding the question of how to change the release cycle.
Thanks everyone!
Quick notes-more-or-less-to-self: I need to do a new wikitcms release to add the Basic milestone, and the blocker bug creation script needs updating to not create Alpha blocker bugs any more.
-
Adam Williamson -
Kamil Paral -
Matthew Miller -
Sudhir D