On Wed, 2010-10-06 at 12:32 -0700, Adam Williamson wrote:
On Wed, 2010-10-06 at 14:58 -0400, John Poelstra wrote:
Adam Williamson said the following on 10/06/2010 01:32 PM Pacific Time:
On Thu, 2010-09-23 at 12:58 +0100, Adam Williamson wrote:
Hi, everyone. So we partly used the proposed new nice-to-have bug tracking system during the F14 Beta process, and it seemed to go well. In a quick burst of airport productivity, I've quickly written up a bunch of proposed new wiki pages and modifications to existing ones to document the nice-to-have process (and, incidentally, extend documentation of the blocker process, since we don't seem to have much of it beyond the blocker meeting SOP right now). All the pages can be found here:
https://fedoraproject.org/wiki/Category:NTH_adjustment_drafts
Thanks to James for his feedback on this. I haven't had much feedback from anyone else. However, given that in practice everyone involved in the release review process has been happy using the NTH system drafted here so far, I intend to make the draft changes final (with modifications to reflect James' feedback) by the end of the week, so if you have any feedback you've been sitting on, now would be the perfect time to send it :) Thanks everyone!
Can you be more specific as to which page we are actually giving feedback on? There are five of them there and they almost all look the same.
All of them. They're mostly modifications of existing pages. I'm not quite sure how you get that they look the same, they're very different.
General note ... There are a few broken links on this page. I didn't inspect *all* of them, but it looks like they will resolve once you've moved the documents into their proposed locations. So probably not a big deal, I can understand the desire to not change the links before and after moving the pages into their final location.
https://fedoraproject.org/wiki/User:Adamwill/Draft_desktop_results_nth_adjus... is a proposed change to https://fedoraproject.org/wiki/QA:Desktop_validation_results_template .
Good stuff.
https://fedoraproject.org/wiki/User:Adamwill/Draft_desktop_validation_nth_ad... is a proposed change to https://fedoraproject.org/wiki/QA:Desktop_validation_testing .
Same as above, equally as wholesome :)
https://fedoraproject.org/wiki/User:Adamwill/QA:SOP_blocker_bug_meeting_nth_... is a proposed change to https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting .
https://fedoraproject.org/wiki/User:Adamwill/QA:SOP_blocker_bug_process_nth_... is a proposed new page; it's not particularly specific to the nice-to-have proposal, actually, it just became apparent while I was doing this that we have no page which explains the entire blocker bug review process, and we should have one. This is it.
Funny how when you've been involved in a process for a while, you forget which parts aren't clearly documented/described for others to engage. Nice addition.
https://fedoraproject.org/wiki/User:Adamwill/QA:SOP_nth_process_nth_draft is a proposed new page which covers the whole nice-to-have review process much as the above proposed page covers the blocker review process.
My first reaction looking at the proposed blocker and nth process SOP pages was that they should be combined into a single page, since there is a lot of duplicate document structure/format. But I can't think of any good proposals to offer at the moment that make it better (not worse). It seems you've already been down this route too.
Perhaps a reflection on my visual learning habit, I've been playing around with some minor edits of the 2 previous wiki pages that make it a bit easier for me to rapidly locate/grok information without too much reading. Of course, let me know if the edits are too invasive or detract from your intended message.
I included a summary of the whole proposed NTH process in my initial review request mail. These are the wiki changes necessary to document that process.
Thanks, James
On Thu, 2010-10-07 at 11:07 -0400, James Laska wrote:
All of them. They're mostly modifications of existing pages. I'm not quite sure how you get that they look the same, they're very different.
General note ... There are a few broken links on this page. I didn't inspect *all* of them, but it looks like they will resolve once you've moved the documents into their proposed locations. So probably not a big deal, I can understand the desire to not change the links before and after moving the pages into their final location.
Yeah, they'll work after the pages are all put live.
https://fedoraproject.org/wiki/User:Adamwill/QA:SOP_nth_process_nth_draft is a proposed new page which covers the whole nice-to-have review process much as the above proposed page covers the blocker review process.
My first reaction looking at the proposed blocker and nth process SOP pages was that they should be combined into a single page, since there is a lot of duplicate document structure/format. But I can't think of any good proposals to offer at the moment that make it better (not worse). It seems you've already been down this route too.
Yeah, I went back and forth. If you make them one page some of the phrasing gets very long and clunky and possibly hard to parse, I was happier with it as two pages.
Perhaps a reflection on my visual learning habit, I've been playing around with some minor edits of the 2 previous wiki pages that make it a bit easier for me to rapidly locate/grok information without too much reading. Of course, let me know if the edits are too invasive or detract from your intended message.
Thanks, I'll revert them all later ;)
Adam Williamson said the following on 10/07/2010 01:24 PM Pacific Time:
https://fedoraproject.org/wiki/User:Adamwill/QA:SOP_nth_process_nth_draft is a proposed new page which covers the whole nice-to-have review process much as the above proposed page covers the blocker review process.
Thanks for doing all this Adam.
The "Nice-to-have bug principles" guidelines feels a little mushy and subjective to me, though after thinking about it for a while I can't propose anything better--you've done a good job putting some structure around what they might be.
And since they can't block the release they won't be able to cause that many problems. I suppose if NTH bugs were to ever become a big distraction from addressing blocker bugs we could re-evaluate.
On the other hand it has taken us a *long* time to get to the place where we are today where churn in RC has been reduced to a bare minimum. I still subscribe to the theory (realizing some in Fedora don't) that every additional change adds a level of risk of delaying the release. So my hope is that by formalizing the NTH we aren't encouraging an increased amount of churn.
I DO think this is an important section in the guidelines that will help cover this concern:
"In general, nice-to-have bugs are usually bugs for which an update is not an optimal solution, and for which the fix is reasonably small and testable (this consideration becomes progressively more important as a release nears, so bugs may be downgraded from nice-to-have status late in the release process if it transpires that the fix is complex and hard to test)."
Would it be overkill to put more explicit testing sign-off around NTH bugs?
John
On Fri, 2010-10-08 at 07:12 -0400, John Poelstra wrote:
Adam Williamson said the following on 10/07/2010 01:24 PM Pacific Time:
https://fedoraproject.org/wiki/User:Adamwill/QA:SOP_nth_process_nth_draft is a proposed new page which covers the whole nice-to-have review process much as the above proposed page covers the blocker review process.
Thanks for doing all this Adam.
The "Nice-to-have bug principles" guidelines feels a little mushy and subjective to me, though after thinking about it for a while I can't propose anything better--you've done a good job putting some structure around what they might be.
And since they can't block the release they won't be able to cause that many problems. I suppose if NTH bugs were to ever become a big distraction from addressing blocker bugs we could re-evaluate.
On the other hand it has taken us a *long* time to get to the place where we are today where churn in RC has been reduced to a bare minimum. I still subscribe to the theory (realizing some in Fedora don't) that every additional change adds a level of risk of delaying the release. So my hope is that by formalizing the NTH we aren't encouraging an increased amount of churn.
I DO think this is an important section in the guidelines that will help cover this concern:
"In general, nice-to-have bugs are usually bugs for which an update is not an optimal solution, and for which the fix is reasonably small and testable (this consideration becomes progressively more important as a release nears, so bugs may be downgraded from nice-to-have status late in the release process if it transpires that the fix is complex and hard to test)."
Would it be overkill to put more explicit testing sign-off around NTH bugs?
I don't see why not. I think this topic came up in a previous mail. I'd propose that NTH bugs must be tested and have appropriate bodhi karma for them to be included. But as noted in a previous mail, I *think* this might be something that release engineering will need to specify on their documentation regarding how to compose release candidates (jkeating or dgilmore can correct me here). Is there an SOP for that process now?
Another distinction to consider 1. Using the language you noted earlier that "[NTH] bugs are usually bugs for which an update is not an optimal solution". To me this implicitly states that NTH packages must be on the media. If it's not on the media, it's not eligible. 2. Extending the above ... I wonder if it's reasonable that NTH cannot be accepted for critical path components. For critical path, it's a blocker or not, let's not fiddle with critpath if a respin is needed to address blocker issues.
Thanks, James
On Fri, 2010-10-08 at 11:23 -0400, James Laska wrote:
Would it be overkill to put more explicit testing sign-off around NTH bugs?
I don't see why not. I think this topic came up in a previous mail. I'd propose that NTH bugs must be tested and have appropriate bodhi karma for them to be included. But as noted in a previous mail, I *think* this might be something that release engineering will need to specify on their documentation regarding how to compose release candidates (jkeating or dgilmore can correct me here). Is there an SOP for that process now?
As noted in the meeting, for final release, all fixes (blocker and NTH) have to go through testing and be pushed to stable, we do not use a side repo for the final composes. This is necessary to ensure that final images match the final published repos.
Another distinction to consider 1. Using the language you noted earlier that "[NTH] bugs are usually bugs for which an update is not an optimal solution". To me this implicitly states that NTH packages must be on the media. If it's not on the media, it's not eligible.
That's probably true.
2. Extending the above ... I wonder if it's reasonable that NTH cannot be accepted for critical path components. For critical path, it's a blocker or not, let's not fiddle with critpath if a respin is needed to address blocker issues.
I disagree with this. It just doesn't match practice. Both before implementing the formal NTH process this release and after, many - in fact, I think most - NTH bugs have been in critpath components. Especially given that many of them will be anaconda or X driver or kernel issues, I don't think this would make any sense.
On Fri, 2010-10-08 at 12:42 -0700, Adam Williamson wrote:
On Fri, 2010-10-08 at 11:23 -0400, James Laska wrote:
Would it be overkill to put more explicit testing sign-off around NTH bugs?
I don't see why not. I think this topic came up in a previous mail. I'd propose that NTH bugs must be tested and have appropriate bodhi karma for them to be included. But as noted in a previous mail, I *think* this might be something that release engineering will need to specify on their documentation regarding how to compose release candidates (jkeating or dgilmore can correct me here). Is there an SOP for that process now?
As noted in the meeting, for final release, all fixes (blocker and NTH) have to go through testing and be pushed to stable, we do not use a side repo for the final composes. This is necessary to ensure that final images match the final published repos.
Another distinction to consider 1. Using the language you noted earlier that "[NTH] bugs are usually bugs for which an update is not an optimal solution". To me this implicitly states that NTH packages must be on the media. If it's not on the media, it's not eligible.
That's probably true.
2. Extending the above ... I wonder if it's reasonable that NTH cannot be accepted for critical path components. For critical path, it's a blocker or not, let's not fiddle with critpath if a respin is needed to address blocker issues.
I disagree with this. It just doesn't match practice. Both before implementing the formal NTH process this release and after, many - in fact, I think most - NTH bugs have been in critpath components. Especially given that many of them will be anaconda or X driver or kernel issues, I don't think this would make any sense.
Oh your right. Lemme rethink if there is a better way to articulate my thoughts. I was searching for a generic way to say, potentially disruptive changes to core packages aren't a good fit for NTH. The NTH xorg bug#596557 discussed during the blocker meeting [1] being a good example.
I'll see if there's a way to rephrase my proposal to be more accurate.
Thanks, James
[1] http://meetbot.fedoraproject.org/fedora-bugzappers/2010-10-08/fedora-bugzapp...
On Fri, 2010-10-08 at 17:05 -0400, James Laska wrote:
Oh your right. Lemme rethink if there is a better way to articulate my thoughts. I was searching for a generic way to say, potentially disruptive changes to core packages aren't a good fit for NTH. The NTH xorg bug#596557 discussed during the blocker meeting [1] being a good example.
I'll see if there's a way to rephrase my proposal to be more accurate.
The principles already mention this:
"and for which the fix is reasonably small and testable (this consideration becomes progressively more important as a release nears, so bugs may be downgraded from nice-to-have status late in the release process if it transpires that the fix is complex and hard to test)"
it's always been a trade-off between how much benefit we get from taking the fix and how complex and potentially dangerous the fix is; I tried to encapsulate that in the principles.
On Fri, 2010-10-08 at 07:12 -0400, John Poelstra wrote:
On the other hand it has taken us a *long* time to get to the place where we are today where churn in RC has been reduced to a bare minimum. I still subscribe to the theory (realizing some in Fedora don't) that every additional change adds a level of risk of delaying the release. So my hope is that by formalizing the NTH we aren't encouraging an increased amount of churn.
I hope so too, that's not the aim of the proposal, just to formalize what we're doing already.
I DO think this is an important section in the guidelines that will help cover this concern:
"In general, nice-to-have bugs are usually bugs for which an update is not an optimal solution, and for which the fix is reasonably small and testable (this consideration becomes progressively more important as a release nears, so bugs may be downgraded from nice-to-have status late in the release process if it transpires that the fix is complex and hard to test)."
Would it be overkill to put more explicit testing sign-off around NTH bugs?
Hard to be sure. On the one hand, they have more attention from more people than most bugs. On the other hand, the whole point is the compressed timescale involved. So it's kinda 50/50.