Hey, folks!
I spent some time today fiddling around with the installation validation
matrix. I haven't applied the changes to Beta TC1 to give us some time
to review/tweak them, but they're in the template:
https://fedoraproject.org/wiki/Template:Installation_test_matrix
I like the new look!
so, I did a few things:
* Split several test groups out from the big ugly "Miscellaneous" table
into their own tables
* Moved a few test cases from "Miscellaneous" into the storage tables
* Dropped three duplicated tests
* Cleaned up the table formatting:
** We don't need the ugly nested tables to have collapsible tables with
nice header lines, you can do it within a single table like this
** Dropped all the icky hard-coded column widths, let's just let
mediawiki sort it out
** Made the tables span the full page width (more space!)
** Renamed "Release Level" to "Milestone" (it's shorter, and
it's the
term we've been standardizing on across the docs)
** Dropped all remaining "test area" columns (not needed with enough
sub-tables)
* Moved the instructions and notes we have for a few of the tables
inside the tables themselves
* Dropped the sub-sections from the matrix, instead you can wrap table
titles in <h4></h4> and they show up in the ToC (trick I found in the
Mediawiki docs)
Most of the table titles ends with "tests". I think we could remove that word,
it's obvious and it will make the titles and TOC even shorter/easier to read.
* Tried to give each of the zillion tables we now have a color,
it's not
the prettiest - anyone with a better eye than me can find the HTML color
list at
http://www.w3schools.com/tags/ref_color_tryit.asp?color=White
and go nuts.
Where I created new tables I tried to be strategic about the
'environments', because we clearly have just too many tests now to run
them all in every possible environment. So I tried to reduce the
environments where possible without hurting test coverage too much. If
anyone thinks it simplifies things too much in terms of arch coverage,
yell.
The "Miscellaneous" table have an ARM column, which is completely gray. Either
it is an error, or the column can be removed.
Thoughts: wow, the page is getting long. The separate tables make it a
bit longer, but the table formatting improvements and loss of section
heads make it a bit shorter, so the changes are kind of a wash, but
that's *really* a lot of tests on one page.
I think it would be feasible to split it into three: something like
sanity tests ("Image sanity tests", "Default boot and install",
"ARM
disk images", "Cloud images", "PXE boot tests", "USB stick
tests",
"Virtualization tests"), installation storage tests (all the storage
stuff) and installer functionality tests (the rest). I actually used
this split, more or less, to come up with the color scheme (each of
those 'areas' uses variations on one color). Does that seem like a good
idea? We'd have more results pages per compose, but each one would be
shorter. With relval the creation of the results pages is just as easy
either way (doing it by hand, it gets tedious if there are too many).
Provided that people are used to edit a single table and not the whole page (that would
lead to many save collisions), I think it's better to have a single page, it's
easier to quickly see what has been done and what hasn't. Maybe if we interlinked all
relevant pages (i.e. every TC1 validation page would contain a link to other TC1
validation pages in the footer, or something). That would help with Desktop, Server and
Base matrices as well. Installation matrices could still stay on a single page.
Thoughts #2: there's a little bit of instruction text at the top of the
page:
"Please click [show] in each table to view the tests of each media
installation, and click [edit] to post your test results using the
syntax in Key Section. "
which suggests we initially meant the collapsible tables to be
*collapsed* by default. Does anyone remember if we ever did that? Does
anyone think it might be a good idea? (I'm also thinking of collapsing
the page ToC by default, because it sure takes up a lot of vertical
space).
I'd have the tables expanded by default, but collapsing TOC is probably a good idea.
Feedback and improvements welcome! Thanks :)
Thank you.