----- Original Message -----
On 2016-11-17 07:43 AM, Stephen John Smoogen wrote:
> 2. The Fedora QA group has 1 mac mini which is very old and is only
> used for total install and not dual boot. It would not have found this
> issue. The Fedora QA group also has no one using Mac hardware day to
> day.
This bit isn't quite true. We found the bug *on* that Mac Mini. I'm
worried it's not likely to find *other* bugs that people are likely to
encounter on the systems they actually want to run Fedora on (newer
laptops), but it did find this one.
Newer Mac laptops don't have working keyboards or touchpads as they're
not connected through USB internally. That's not Fedora's problem though.
The problem is if the installer doesn't work when the pre-requisite
hardware does.
The problem is that we didn't get around to running the test
until the
day before the go/no-go. There's a lot of stuff to test, and anything
which only one person is likely to test is a risk. Frankly speaking,
given how humans work, things that involve digging some piece of
hardware you never touch out of a pile and hooking it up to a keyboard
and mouse and a monitor and power and network is quite likely to get
passed over in favour of something you can run in a VM. Especially if
it's 4:30. This is why I have an Unused Arm Devices Pile Of Shame on my
desk...
So, partly this is our fault because we could've tested this earlier and
didn't. But it's also the case that we really need more redundancy in as
much of the required testing as possible.
Is there any continuous testing done on the images on the installer? Is it
on real hardware? Is it possible to mock hardware setups? Comparing
boot setups on working and non-working installations.
I think it would be possible to do testing that didn't rely quite as much
on manual testing, through regression testing on "mock" hardware (a hacked
up VM with a test disk image), comparing the partition types after installation
against a working setup, comparing the file lists in the boot partition,
etc.
I'm surprised that the Anaconda, and blivet developers aren't taking part
of this conversation. I'd certainly like them to point out all the ways in
which they're already doing what I mentioned, and showing how we could
add more test cases.