On Mon, 2009-11-23 at 10:00 -0500, James Laska wrote:
On Fri, 2009-11-20 at 20:37 -0800, Adam Williamson wrote:
On Fri, 2009-11-20 at 15:43 -0800, John Poelstra wrote:
I've been been thinking for a while that it would benefit us to rework our release criteria (https://fedoraproject.org/wiki/Fedora_Release_Criteria) from being just a description of blocker bugs to more about the broader criteria that needs to be met to issue a public release.
I was talking to James Laska about this earlier today, and we both have thoughts on it too. I think it's a great idea to have revised and more detailed criteria, and I especially like the organization - a main page with an overview, and detailed criteria for each release.
Well, the above says it already, but a big +1 from me on having release specific content. From my experience, this has always been implicit as we all slug through the release ... so I appreciate having what we've already done a more explicit.
I had one major high-level suggestion. Release criteria are more or less unavoidably partially subjective. I don't think it's feasible to come up with concrete rules to cover every possible situation. Therefore, the criteria should explicitly embrace and cover this subjectivity. It should be made clear that things like 'boots successfully' are to some degree dependent on subjective, contextual judgements; do we block the Alpha for a bug that stops 0.1% of systems booting? 0.5%? 1%? 5%? I think rather than trying to define something like this, we should just explicitly acknowledge that it'll be a judgment call.
Yeah, this will be a tough nut to crack ... but I don't feel like this is new and scary for us. During F12, the group got into a good rhythm when it came to assessing the impact of blocker bugs. First, how common of a use case is it. Next, how common is the hardware environment. And last, something that Adam pointed out to me, how common is the local system configuration (e.g. are we using a custom xorg.conf to drive output to 2 HDTV's -- maybe a bad example, but you get the idea).
Just a thought, either ...
- We adjust the following 3 criteria (grabbed from the Alpha page) to
include a statement about "common hardware/configuration".
- The installed system boots and starts up properly
- The installed system is able to download updates with yum.
- Installer boots and runs on all primary architectures: i686 and x86_64
- Or we leave the above criteria as is, and add instructions to the
https://fedoraproject.org/wiki/Blocker_Bug_FAQ for how to evaluate if your uninstallable system is a common issue or not?
Looks like part#2 is partially stubbed out for us already :)
https://fedoraproject.org/wiki/Fedora_13_Alpha_Release_Criteria#Alpha_Blocke...
Thanks, James