First of all, I'd like to thank you, for this feedback, James.
On 03/22/2010 09:06 PM, James Laska wrote:
To be honest, this seems like a sound starting point. Do we need
more
than just...
* what test is? (aka a test_name and only if really needed, a
test_version)
* what are we testing? - some unique human-readable (and
human-created) identifier for a test run? The most obvious
choice being a package envra, but also a build stamp for an ISO
install test run (e.g. F-13-Alpha-TC0)?
I belive, that we can easily identify Tests just using the name/version.
Identifying test runs - we talked about this with Kamil yesterday, and
it's obvious, that we will need to store different information to
different types of tests (aka envr to package sanity test, build stamp
to install test) - as you said.
I'm currently thinking about the database schema (look at the updated
schema, if you please
<
https://fedoraproject.org/wiki/AutoQA_resultsdb_schema>), which would
allow us to do so - we'll have basic Testrun table storing common data,
and some other table(s) to store the specific values.
I'm not sure, how to make this specific-values table(s) right - the
schema actually proposed is quite flexible, but it might not be
providing en easy survey. The other possibility I can think of is using
one _huge_ sparse table, with all the possible extra values - but this
seems to have more disadvantages, than advantages, at least from the
performance point of view.
I would think that UUID's make good sense for internally
referencing
data, but I really don't want to be passing around UUID's in URLs when
directing people to test result dashboards. Does this help?
I completely agree - the UUID was intended for internal tools usage, not
for humans - i was once again thinking about the future, in which we
have a TCMS. Then we certainly will need a way to interconnect the (most
probably) standalone databases using some common key - and this UUID
seems like the easiest way.
In the current db schema, the UUID is kind of useless information (since
it can be generated directly from the information stored in the same
table), but it seemed like a good idea to put it there, at least so we
could discuss it :)
I know if says 'hard coded', but this doesn't seem bad
for now.
Good to hear, that none of us has problem with this solution used for
now, since it's IMHO the most straightforward way right now.
I think it will be common for a test to live in multiple test plans.
For example, we have the Rawhide Acceptance Test plan [1] which includes
a *small* subset of tests to validate that the repo and the install
images are sane. I envision some of those tests would be used again
(possibly a slightly different context) in the installation test plan
[2].
[1]
https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
[2]
https://fedoraproject.org/wiki/QA:Fedora_13_Install_Test_Plan
OK, thank you for this information. Made me happy :)
As long as I never have to use the UUID (or using it is an
exception).
I mean, if it's just an internal representation (like how git uses hash
strings for representing commits), that seems fine. But if we expect
users/testers to be passing around the UUID's, I don't know if that
improves things.
As I wrote, I certainly do not want users to use UUIDs themselves - i
remember the time Ubuntu transferred from device paths to UUIDs in
grub.conf and fstab - i never really knew what disc am i mounting ever
since :)
What I love about storing our test cases and plans in the wiki right
now
is that I don't need to remember some internal unique ID to reference
the test. I just call it what it is, for example
'QA:Testcase_Mediakit_ISO_Size'. What is nice is that the wiki takes an
optional version parameter in the event you are referencing something
other than HEAD
(
https://fedoraproject.org/w/index.php?title=QA:Testcase_Mediakit_ISO_Size...).
My idea of storing the Tests is quite the same - there will be a history
stored in database, while the latest (i.e. the one with the biggest
version number) is considered the HEAD. This concept seems natural and
right to me too.
Once again, thank you a lot for your feedback.
Looking forward to hear also from the other people involved
Joza