# AutoQA ResultsDB Discussion # Date: 2010-02-26 # Time: 15:00 UTC (10:00 EST, 16:00 CET) # Participants - wwoods, jlaska, kparal, jskladan
= Review definition/goals =
https://fedorahosted.org/pipermail/autoqa-devel/2010-February/000201.html
Goals * One central database to store test results (audience QA) * Multiple front-ends to present stored results (audience QA/developers)
= Recap of research =
* Overview - https://fedoraproject.org/wiki/AutoQA_resultsdb_approaches
Spikesource schema - http://dev.spikesource.com/wiki/index.php/Trpi-schema_view * No concept of WAIVE results
RHTS/Beaker - http://beaker.fedorahosted.org/ * stores log files directly * presents most important data on main results page (e.g. stdout) * Hiearchy - Job contains RecipeSet(s), which contains Recipe(s), which contains test(s), which contain phase(s)
Beakerlib * Allows specification of test phases * Provides common test writing format in bash * beakerlib xml journal http://jskladan.fedorapeople.org/beakerlib_format.xml ** test which produced the output: http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/beakerlib_inits...
RATS * Test results linked by a unique key (timestamp of tree being tested) * install.py test also uses concept of "phases" - http://git.fedorahosted.org/git/?p=autoqa.git;a=blob;f=tests/rats_install/in... * Multiple tests (2), each with multiple phases - Viewed from single location - http://jlaska.fedorapeople.org/rats.png
= Ideate =
* Define an output standard for each test type for easier inspection of results (similar to beakerlib) * TCMS - something we're gonna forget about for now (db scheme review in 'open topics')
= Terminology = * test case - one unit with a single result (but may contain several phases) * test phase - parts of a test case, their results are not stored separately in the results db, but define pass/fail result of the test case itself (internally for the test) * test plan - contains several test cases, provides meta data around test cases (responsibility, workflow, high-level criteria etc...) * mapping to the beakerlib_format.xml - depends on particular test, for initscripts checking: <log> ~ test case, <phase> ~ test phase
= Open topics = * How to handle policy (waive) in tools? ** https://fedoraproject.org/wiki/AutoQA_resultsdb_approaches#DB_schema - rpmdiff solution * Is defining a common test writing and output format the same as providing a results front-end/workflow? * Examine nitrate/testopia resultsdb format? * Test case results vs autotest job results (unhandled exception, etc)
= Tasks = * [wwoods] - Examine beakerlib output format if it is sufficient for our purposes * [jlaska] - Examine nitrate/testopia DB schema * [kparal + jskladan] - Define personas and write use cases Example: https://fedoraproject.org/wiki/No_frozen_rawhide_announce_plan#Use_Cases Wiki page: https://fedoraproject.org/wiki/AutoQA_resultsdb_use_cases * [kparal + jskladan] - Propose some initial DB schema
= Next Meeting = * 2010-03-05 - mailing list checkin (conference call if needed)