On Thu, 2010-05-06 at 13:39 +0800, Li Ming wrote:
Hi,
So far, we have almost solved all technical problems for auto
install test. And a workable test driver has been put to git. What we are doing now is to list all possible tests which can meet each deliverable. As discussed with James, we should created these test drivers to map to test deliverables which we get from release engineering:
Nicely done Liam! I like clearly defining the methods by which we initiate testing. There are too many possibilities out there, it helps to pinpoint how it really happens.
- URL install source available *url_install.py
Since a URL install source also includes a "images/boot.iso" file, we probably will need to include: * bootiso_install.py * iso_sanity.py (sanitize the boot.iso)
- DVD image(s) available *iso_sanity.py *dvd_install.py
Howabout some other ISO-based tests ... like * hdiso_install.py (using the DVD iso) * nfsiso_install.py (again, using DVD iso)
- CD image(s) available *iso_sanity.py *cd_install.py
Same as above, but with CD's instead of DVD: * hdiso_install.py (using the DVD iso) * nfsiso_install.py (again, using DVD iso)
For each test driver, we are trying to list all possible tests, I listed some in wiki, you can see at here:
https://fedoraproject.org/wiki/Is_anaconda_broken_proposal#Roadmap
On "step 3", I listed some possible tests,but I think I must have missed some, do you have some to supplement? For Possible dvd_install.py tests, I try to list all install tests, but for url_install.py and cd_install.py tests ,I only listed some default install,because cd_install.py tests are mainly test boot and swapping disc, url_install.py tests are mainly test remote boot and install. I am not sure whether list all possible tests for url_install.py and cd_install.py, what's your suggestion?
Can we list as many as possible? It'll feel a bit tedious, but hopefully this exercise helps identify the patterns. Things that each test driver will need to do, and things that only 1 (or subset) test driver will need to do.
It might be helpful to experiment with listing this in a wiki table format? Or whatever method will help identify commonalities in the different test drivers. This won't answer the question, "how should we prioritize these tests?".
Test driver Test dvd_install.py cd_install.py hdiso_install.py ... ============+===============+==============+=================+====== DVD boot YES NO NO CD boot NO YES NO boot.iso boot NO NO NO PXEimage boot NO NO YES stage2=media YES YES NO stage2=http NO NO NO stage2=hd NO NO NO stage2= ... repo=cdrom: YES YES NO repo=http NO NO NO repo=hd NO NO YES repo=nfs NO NO NO repo=nfsiso NO NO NO ks=http YES YES YES ks=file YES [1] YES [1] YES ks=hd YES YES YES ks=floppy YES YES YES ks=cdrom YES YES NO install? YES YES YES upgrade? YES YES YES [1] . . .
[1] These are technically possible, but are likely *infrequent* use cases. When we start prioritizing the scenarios we want to support, these are the type of tests I'd consider low priority.
Thanks, James