I had a similar conversation on test cases while trying to plan for an upcoming test day:

http://lists.laptop.org/pipermail/testing/2012-February/002609.html

In general, it might be useful to know how skilled most test day participants are at testing, as well as the test approach preferred by the Fedora project as a whole when planning these events.



2012/2/20 "Jóhann B. Guðmundsson" <johannbg@gmail.com>
On 02/20/2012 04:19 AM, Jon Stanley wrote:
I had a conversation with Adam at FUDCon about things that might be
beneficial to QA that I could champion and make sure that it happened,
and provide regular status reports, the whole nine yards. What we came
up with is updating "How to debug" and "How to test" pages in the
wiki, soliciting input from maintainers, etc.

When I initiated that effort after those initial proposals of mine I came pretty fast to the conclusion that in order to be able to do that we needed to have it mandatory for packagers/maintainers to submit at least a minimum debugging information for new components which we QA ( or more me at the time )  could create proper debugging pages around that info.

I even went to the lengths to request that from Fesco/FPC which settled on making it "optional" for packagers to do so.

Needless to say now 6000+ components later we have no more how to debug or how to test pages then we did back then which makes it pretty evident that unless fesco/fpc make it mandatory you will be chasing your tail and other maintainers for a long time before it's completed.

I appraise your efforts in taking up the torch where I left it and my advice to you is start with the critical path packages then work your way up from there.

JBG