On Fri, 2011-06-24 at 12:53 -0700, Adam Williamson wrote:
Hey, folks. So, right now, as most of you know, we have two release
validation 'matrices' - sets of tests we run and document on each
candidate build of Fedora to check compliance with the release criteria.
A few of us have been working in ticket #151 -
https://fedorahosted.org/fedora-qa/ticket/151 - to ensure the tests are
in line with the criteria, particularly that we have tests to cover each
criterion, and we've introduced a few new tests to cover criteria that
weren't previously covered.
Some of these don't really fit easily into the category of
'installation' tests or 'desktop' tests, so I'm proposing we
introduce a
new, third matrix, which I'm currently thinking of as 'base'. There have
already been three tests written in this proposed new category:
https://fedoraproject.org/wiki/QA:Testcase_base_firstboot
https://fedoraproject.org/wiki/QA:Testcase_base_startup
https://fedoraproject.org/wiki/QA:Testcase_base_system_logging
there will likely be more as we go through the Beta and Final criteria.
I've thrown up a quick draft of how the matrix might look here:
https://fedoraproject.org/wiki/User:Adamwill/Draft_base_validation_matrix
Just wanted to make sure the group knows about this week, and ask for
feedback on the test cases, the matrix and the overall plan here - do
you agree a third matrix is the best way to organize things?
Good thinking, an additional matrix seems appropriate. I'm sure as we
start adding more tests, a theme or organizational scheme might surface.
Until then, just tossing them into this additional 'base' matrix seems
like a great idea!
Are any SOP updates needed to ensure this matrix is included when
setting up test runs in Fedora 16?
Any ideas
for naming, etc? Any comments on the test cases written so far? Thanks!
One thought, the 'base' matrix page looks a bit sparse. I was about to
suggest that we add a standard header to the new 'base' matrix with some
basic instructions, but I wonder if something else might be more
appropriate. When Andre and Hurry announce test runs, instead of
linking to each individual test matrix, we'd send a link to the
appropriate Category:Test_Results page (e.g.
https://fedoraproject.org/wiki/Category:Fedora_15_Final_RC_Test_Results).
What if we move the stock "instructions" (see
https://fedoraproject.org/wiki/Test_Results:Fedora_15_Final_RC1_Install#I...) out
of each individual page, and into the category page? Or at the very least, make it an
trancludable [1] wiki template {{Fedora_15_Test_Results_Instructions}}.
(also thanks to Athmane Madjouj and Rui He for helping out with this
work.)
Yes, thank you! :D
Thanks,
James
[1]
http://www.mediawiki.org/wiki/Help:Transclusion