I have a proposal regarding installation matrix [1] related to environments. Adam did a
truly stellar job in his latest results table restructuring, he eliminated a lot of
duplicated testing and simplified the tables a lot (for example, there's no point in
testing additional HTTP repos in anaconda on all three major architectures, because that
functionality it architecture independent). We don't have limitless resources and must
keep focused on areas where it's likely for a bug to occur, instead of testing
everything. Here's a proposal of some last touches to installation tables:
1. Testcase_Anaconda_User_Interface_Basic_Video_Driver was changed to a generic x86 test.
In this case, it is problematic, because BIOS and UEFI fallback video driver are
completely different code paths. I have a personal experience from that past when one of
those worked and the other didn't. We should distinguish BIOS and UEFI in this case.
To fix this, I see two viable options:
a) move the test case to Miscellaneous section, so that the table in User interface
section doesn't get more complicated
b) keep the test case in User interface section, but create a separate table for it (right
under the first table). This new table would have x86 BIOS and x86 UEFI columns. Again,
the purpose is to keep the original table simple (avoid adding columns which are not
required for anything else).
2. Testcase_Install_to_Previous_KVM and Testcase_Install_to_Current_KVM differentiate BIOS
and UEFI. I always forget whether this is about the host or the guest (or both, or which
combination). But I think host differentiation doesn't make any difference - libvirt
should work the same, I assume. It makes sense to differentiate the guests. Unfortunately
I believe the UEFI mode for libvirt/kvm [2] is not available under an open source license,
therefore it's not packaged and not trivial to use. Which means not that many people
probably use it. I wonder whether we really need to keep this separated. I would gladly
merge this together to a single x86 column and split up once OVMF is available under a
better license. If we keep it separate, we should provide better instructions for the UEFI
part.
PS: I know nothing about Xen, which is also in the table, so I have no idea whether
it's a similar story or not.
3. Testcase_Anaconda_save_traceback_to_bugzilla, Testcase_Anaconda_updates.img_via_URL,
Testcase_Anaconda_user_creation, Testcase_Anaconda_updates.img_via_installation_source and
Testcase_Anaconda_traceback_debug_mode all differentiate BIOS and UEFI. I think that's
a waste of our energy. None of these testcases should IMO be affected by firmware. We
should make them generic x86 tests. We could go even further and make them completely
arch-independent (at least some of them), but I'm a bit careful around ARM, so not
completely sure about that.
What da ya think?
Thanks,
Kamil
[1]
https://fedoraproject.org/wiki/Test_Results:Fedora_21_Final_TC1_Installation
[2]
http://www.linux-kvm.org/page/OVMF