https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
To follow up, I think the "not release blocking sidecar" model didn't really work for Fedora 25, because we had last minute bugs there, and at least the Fedora websites team refused to link to Fedora 24 content. (I can't find the discussion in a quick search).
On 03/23/2017 11:17 AM, Colin Walters wrote:
To follow up, I think the "not release blocking sidecar" model didn't really work for Fedora 25, because we had last minute bugs there, and at least the Fedora websites team refused to link to Fedora 24 content. (I can't find the discussion in a quick search).
Yeah. I think a lot of the "problems" around f25 was on us. We need to get better at identifying problems sooner and fixing them before timing becomes critical. Guess what? We've already identified and squashed quite a few bugs for Fedora 26. I'm happy with our progress so far.
As far as the websites team, they are great and they are willing to help us do anything we want them to do as long as we communicate with them before hand what our desires are. If we want the ability to "not flip to f25" when the rest of f25 comes out we can't tell them that the week before f25 comes out. They do a great job with what resources they have.
Dusty
On Thu, Mar 23, 2017 at 3:17 PM, Colin Walters walters@verbum.org wrote:
To follow up, I think the "not release blocking sidecar" model didn't really work for Fedora 25, because we had last minute bugs there, and at least the Fedora websites team refused to link to Fedora 24 content. (I can't find the discussion in a quick search).
For the last two releases the atomic stuff has been broken and in both cases wasn't found until just before release and in most cases after the rest of the release had been declared go. While I realise this is historical it's not a good track record. For it to be release blocking there has to be constant attention and resources to ensure that things are tested constantly and breakage is detected quickly so it can be dealt with in a timely manner. It's a reasonable commitment from the team so this needs to be taken into account.
Peter
On 03/23/2017 11:55 AM, Peter Robinson wrote:
On Thu, Mar 23, 2017 at 3:17 PM, Colin Walters walters@verbum.org wrote:
To follow up, I think the "not release blocking sidecar" model didn't really work for Fedora 25, because we had last minute bugs there, and at least the Fedora websites team refused to link to Fedora 24 content. (I can't find the discussion in a quick search).
For the last two releases the atomic stuff has been broken and in both cases wasn't found until just before release and in most cases after the rest of the release had been declared go. While I realise this is historical it's not a good track record. For it to be release blocking there has to be constant attention and resources to ensure that things are tested constantly and breakage is detected quickly so it can be dealt with in a timely manner. It's a reasonable commitment from the team so this needs to be taken into account.
Peter, as I stated in my earlier reply, you are correct.
We've addressed this for f26 as we've been actively involved in getting release artifacts in good shape and bugs identified and fixed since f26 branched (we still have some work to do on rawhide, though). Please come to me if you find any quality lacking and/or neglect of responsibilities during the f26 release cycle.
Dusty
On 03/23/2017 11:14 AM, Colin Walters wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
First off, thank you for not proposing this for f26.
I think we should weigh the pros and cons. Some of which have become apparent to me during the past few weeks since f26 branched. There are some benefits to having us not be included in the alpha process and there are some negatives. I think this might best be served as a ticket in our tracker so that we can have a discussion there and ultimately make a decision.
Additionally I have a todo item for our working group to create a requirements doc that we collaborate on with Fedora Releng so that we better understand the expectations from each side. I think a document like this will help us define the process and quell confusion as time goes on.
Dusty
On Thu, Mar 23, 2017 at 11:31 AM, Dusty Mabe dusty@dustymabe.com wrote:
On 03/23/2017 11:14 AM, Colin Walters wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
First off, thank you for not proposing this for f26.
I think we should weigh the pros and cons. Some of which have become apparent to me during the past few weeks since f26 branched. There are some benefits to having us not be included in the alpha process and there are some negatives. I think this might best be served as a ticket in our tracker so that we can have a discussion there and ultimately make a decision.
For those not already aware, for F27 and forward there won't be an Alpha release. It's being removed.
https://fedoraproject.org/wiki/Changes/NoMoreAlpha
josh
On Thu, 2017-03-23 at 11:14 -0400, Colin Walters wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
I'd just like to point to my full comment in that bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310#c11
for me, it doesn't make much sense just to ask the narrow question "should Atomic be release blocking", because I don't think we have a good answer as to exactly what that would/should *mean*. I think it'd be more useful to have a slightly wider discussion about how it would make the most sense to deliver Atomic bits during the pre-release phase, and then from there, consider how the overall release process can maybe be adapted to accommodate that.
On Thu, Mar 23, 2017 at 10:14 AM, Colin Walters walters@verbum.org wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
Does it make sense for Atomic Host to be "release blocking" when we can just release every two weeks (or really whenever we want) where as traditional release is bound to the standard cycle? It's loosely coupled.
I would like be clear that I'm not saying or suggesting it either is or it isn't, but I'm not convinced either direction and I'm curious what others think about the justification of not being tightly bound to the release cycle.
-AdamM
cloud mailing list -- cloud@lists.fedoraproject.org To unsubscribe send an email to cloud-leave@lists.fedoraproject.org
On 04/11/2017 11:09 AM, Adam Miller wrote:
On Thu, Mar 23, 2017 at 10:14 AM, Colin Walters walters@verbum.org wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
Does it make sense for Atomic Host to be "release blocking" when we can just release every two weeks (or really whenever we want) where as traditional release is bound to the standard cycle? It's loosely coupled.
I would like be clear that I'm not saying or suggesting it either is or it isn't, but I'm not convinced either direction and I'm curious what others think about the justification of not being tightly bound to the release cycle.
Also, once we go into rolling releases, how would this work?
On Tue, Apr 11, 2017 at 11:12:08AM -0700, Josh Berkus wrote:
or it isn't, but I'm not convinced either direction and I'm curious what others think about the justification of not being tightly bound to the release cycle.
Also, once we go into rolling releases, how would this work?
The rolling releases will still be based on an underlying overall Fedora OS version. I think it helps from a PR perspective to have the Atomic Host based on new versions ready on "launch day".
I was talking with Dusty about this the other day, and came up with this wording:
Final:
It must be possible to build valid Fedora Atomic Host deliverables (ostree, ISO, and images) from GA or post-GA packages for this Fedora release.
* May include zero-day updates * "Valid" means passing the Fedora Atomic Host test automatic suite and manual validation
Beta:
It must be possible to build valid Fedora Atomic Host deliverables (ostree, ISO, and images) from this branch of Fedora.
* I carefully didn't say "beta release" bits here. If we're successfully building on the new branch but have some particular issue at beta release time, it's okay to present an earlier build and later update once that's fixed. If necessary, we can call this "pre-release" rather than "beta".
On Wed, Apr 12, 2017 at 08:38:42AM -0400, Matthew Miller wrote:
<...snip...>
The rolling releases will still be based on an underlying overall Fedora OS version. I think it helps from a PR perspective to have the Atomic Host based on new versions ready on "launch day".
I was talking with Dusty about this the other day, and came up with this wording:
Final:
It must be possible to build valid Fedora Atomic Host deliverables (ostree, ISO, and images) from GA or post-GA packages for this Fedora release.
- May include zero-day updates
- "Valid" means passing the Fedora Atomic Host test automatic suite and manual validation
Beta:
It must be possible to build valid Fedora Atomic Host deliverables (ostree, ISO, and images) from this branch of Fedora.
- I carefully didn't say "beta release" bits here. If we're
successfully building on the new branch but have some particular issue at beta release time, it's okay to present an earlier build and later update once that's fixed. If necessary, we can call this "pre-release" rather than "beta".
-- Matthew Miller mattdm@fedoraproject.org Fedora Project Leader
I think both of these sound fine. There's a clear line between passing and failing, and I don't *really* expect either of these to fail. I'll bring these to the QA group for discussion on Monday about accepting them as new release criteria. It would be good if people were there for the discussion. Our meetings are at 1500 UTC.
Thanks!
// Mike -- Fedora QA
On Wed, 2017-04-12 at 09:20 -0400, Mike Ruckman wrote:
On Wed, Apr 12, 2017 at 08:38:42AM -0400, Matthew Miller wrote:
<...snip...>
The rolling releases will still be based on an underlying overall Fedora OS version. I think it helps from a PR perspective to have the Atomic Host based on new versions ready on "launch day".
I was talking with Dusty about this the other day, and came up with this wording:
Final:
It must be possible to build valid Fedora Atomic Host deliverables (ostree, ISO, and images) from GA or post-GA packages for this Fedora release.
- May include zero-day updates
- "Valid" means passing the Fedora Atomic Host test automatic suite and manual validation
Beta:
It must be possible to build valid Fedora Atomic Host deliverables (ostree, ISO, and images) from this branch of Fedora.
- I carefully didn't say "beta release" bits here. If we're
successfully building on the new branch but have some particular issue at beta release time, it's okay to present an earlier build and later update once that's fixed. If necessary, we can call this "pre-release" rather than "beta".
-- Matthew Miller mattdm@fedoraproject.org Fedora Project Leader
I think both of these sound fine. There's a clear line between passing and failing, and I don't *really* expect either of these to fail. I'll bring these to the QA group for discussion on Monday about accepting them as new release criteria. It would be good if people were there for the discussion. Our meetings are at 1500 UTC.
FWIW, this looks pretty reasonable to me as well, though I *can* see the possibility of grumbling at a Final go/no-go if everything else works but there's an Atomic showstopper. I might go for s/valid/working/ in the Final criterion, as it seems like a clearer word.
El mié, 12-04-2017 a las 08:38 -0400, Matthew Miller escribió:
On Tue, Apr 11, 2017 at 11:12:08AM -0700, Josh Berkus wrote:
or it isn't, but I'm not convinced either direction and I'm curious what others think about the justification of not being tightly bound to the release cycle.
Also, once we go into rolling releases, how would this work?
The rolling releases will still be based on an underlying overall Fedora OS version. I think it helps from a PR perspective to have the Atomic Host based on new versions ready on "launch day".
I was talking with Dusty about this the other day, and came up with this wording:
Final:
It must be possible to build valid Fedora Atomic Host deliverables (ostree, ISO, and images) from GA or post-GA packages for this Fedora release.
- May include zero-day updates
- "Valid" means passing the Fedora Atomic Host test automatic suite and manual validation
Beta:
It must be possible to build valid Fedora Atomic Host deliverables (ostree, ISO, and images) from this branch of Fedora.
- I carefully didn't say "beta release" bits here. If we're
successfully building on the new branch but have some particular issue at beta release time, it's okay to present an earlier build and later update once that's fixed. If necessary, we can call this "pre-release" rather than "beta".
I am going to strongly object to the Beta criteria. its not realistic to support some random nightly. it causes a manual workload on someone and increases technical debt. Something we are working really hard to pay off. Its either part of Beta or it does not exist.
Dennis
On Mon, May 01, 2017 at 04:36:16PM -0500, Dennis Gilmore wrote:
I am going to strongly object to the Beta criteria. its not realistic to support some random nightly. it causes a manual workload on someone and increases technical debt. Something we are working really hard to pay off. Its either part of Beta or it does not exist.
What manual workload does it cause, on who specifically, and do we have resources to cover that?
What technical debt does it increase?
In any case, the proposal is not "some random nightly". It is "a selected tree/image built using the normal (two-week) release process".
El vie, 05-05-2017 a las 09:35 -0400, Matthew Miller escribió:
On Mon, May 01, 2017 at 04:36:16PM -0500, Dennis Gilmore wrote:
I am going to strongly object to the Beta criteria. its not realistic to support some random nightly. it causes a manual workload on someone and increases technical debt. Something we are working really hard to pay off. Its either part of Beta or it does not exist.
What manual workload does it cause, on who specifically, and do we have resources to cover that?
What technical debt does it increase?
In any case, the proposal is not "some random nightly". It is "a selected tree/image built using the normal (two-week) release process".
the copying would all have to be manual. additionally any pre release nightlies that are produced regardless of being done as part of branched or thier own compose will be marked as a nightly and will not be suitable for release, with a short life span that will not keep it around for very long.
Dennis
On Tue, May 09, 2017 at 10:53:04AM -0500, Dennis Gilmore wrote:
I am going to strongly object to the Beta criteria. its not realistic to support some random nightly. it causes a manual workload on someone and increases technical debt. Something we are working really hard to pay off. Its either part of Beta or it does not exist.
What manual workload does it cause, on who specifically, and do we have resources to cover that?
What technical debt does it increase?
the copying would all have to be manual. additionally any pre release nightlies that are produced regardless of being done as part of branched or thier own compose will be marked as a nightly and will not be suitable for release, with a short life span that will not keep it around for very long.
Sorry, I'm not following. We wouldn't want any copying any more than we already have for the regular two-week release process. And, for that matter, we would want _builds_ that are made as part of that same process, and so suitable for exactly the same release that the existing non-beta releases are.
I definitely don't want to treat these as extra and special. A two-week candidate should start being produced as a matter of course at branch.
On Tue, Apr 11, 2017, at 02:09 PM, Adam Miller wrote:
On Thu, Mar 23, 2017 at 10:14 AM, Colin Walters walters@verbum.org wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
Does it make sense for Atomic Host to be "release blocking" when we can just release every two weeks (or really whenever we want) where as traditional release is bound to the standard cycle? It's loosely coupled.
The concrete thing I'm trying to achieve is that bugs like https://bugzilla.redhat.com/show_bug.cgi?id=1435310 are able to be fixed during freeze time. It's exceedingly strange to me that we had to structure the request as affecting the "Cloud Base" image just to satisfy the process.
There are other possibilities - we could carry "overrides" in just FAH. That would be useful in a variety of circumstances of course, but carries its own risks/rewards.
To turn this around again - feel free to propose *alternate solutions*, but just questioning the rationale doesn't make sense to me, because that bug is an example of where (IMO) the process is broken.
On Tue, Apr 11, 2017 at 02:32:09PM -0400, Colin Walters wrote:
The concrete thing I'm trying to achieve is that bugs like https://bugzilla.redhat.com/show_bug.cgi?id=1435310 are able to be fixed during freeze time. It's exceedingly strange to me that we had to structure the request as affecting the "Cloud Base" image just to satisfy the process.
It didn't have to be "structured" as affecting the Cloud Base image to get it in. It only was accepted *because* it affected the Cloud Base image. If it hadn't impacted the Base image, it wouldn't have been accepted because of how the Release Criteria are written and that Atomic doesn't currently block release.
That said, SELinux issues tend to crop up this time during release, and it takes time for the policy to get updated. It would have only blocked Final - had the problems persisted. As it stands, if the Atomic WG files for Freeze Exceptions when we encounter bugs with our deliverables, I really don't expect to see much complaint from the QA team during blocker review (which is when we also go over Freeze Exception Requests).
There are other possibilities - we could carry "overrides" in just FAH. That would be useful in a variety of circumstances of course, but carries its own risks/rewards.
To turn this around again - feel free to propose *alternate solutions*, but just questioning the rationale doesn't make sense to me, because that bug is an example of where (IMO) the process is broken.
My "alternate solution" at this point is to file for FE status with a solid rationale on what needs fixed and why. Provided we're not looking to push in a new kernel or something crazy, I don't foresee us having any issues getting fixes in. And it also fits within the current process we're already following in the rest of Fedora.
It might still be good to have a wider discussion to clear up confusion about where and how Atomic fits into the greater Fedora release process so we have something more concrete to go off of for F27.
// Mike -- Fedora QA
On Tue, 2017-04-11 at 14:32 -0400, Colin Walters wrote:
On Tue, Apr 11, 2017, at 02:09 PM, Adam Miller wrote:
On Thu, Mar 23, 2017 at 10:14 AM, Colin Walters walters@verbum.org wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
Does it make sense for Atomic Host to be "release blocking" when we can just release every two weeks (or really whenever we want) where as traditional release is bound to the standard cycle? It's loosely coupled.
The concrete thing I'm trying to achieve is that bugs like https://bugzilla.redhat.com/show_bug.cgi?id=1435310 are able to be fixed during freeze time. It's exceedingly strange to me that we had to structure the request as affecting the "Cloud Base" image just to satisfy the process.
There are other possibilities - we could carry "overrides" in just FAH. That would be useful in a variety of circumstances of course, but carries its own risks/rewards.
To turn this around again - feel free to propose *alternate solutions*, but just questioning the rationale doesn't make sense to me, because that bug is an example of where (IMO) the process is broken.
If that's the sole motivation, then making Atomic 'release blocking' is not the only possible solution. We accept fixes for both 'release blocker' and 'freeze exception' issues during freezes. Atomic-specific issues can be nominated for a 'freeze exception' with no changes to existing policy or process needed:
https://fedoraproject.org/wiki/QA:SOP_freeze_exception_bug_process
Though we could add some explicit wording to the 'Freeze exception bug principles' on that page to make it clear that critical Atomic bugs are strong FE candidates.
On Tue, Apr 11, 2017, at 02:09 PM, Adam Miller wrote:
On Thu, Mar 23, 2017 at 10:14 AM, Colin Walters walters@verbum.org wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1435310 raised the issue that apparently, Atomic Host isn't "release blocking". I think we have plenty of resources to match whatever criteria there are for that for Fedora 27. Thoughts?
Does it make sense for Atomic Host to be "release blocking" when we can just release every two weeks (or really whenever we want) where as traditional release is bound to the standard cycle? It's loosely coupled.
The concrete thing I'm trying to achieve is that bugs like https://bugzilla.redhat.com/show_bug.cgi?id=1435310 are able to be fixed during freeze time. It's exceedingly strange to me that we had to structure the request as affecting the "Cloud Base" image just to satisfy the process.
There's no reason something like that can't be treated the same as the Alternate Architectures where it's not blocking but is done as an automatic feature enhancement, so it doesn't cause a respin of the entire compose but it's pulled in if there's a new RC done.
-
Adam Miller -
Adam Williamson -
Colin Walters -
Dennis Gilmore -
Dusty Mabe -
Josh Berkus -
Josh Boyer -
Matthew Miller -
Mike Ruckman -
Peter Robinson