On Fri, Nov 25, 2016 at 7:26 AM, Bastien Nocera <bnocera(a)redhat.com> wrote:
But if the installer is (completely) broken, it might as well be dropped.
Alas, it's not completely broken.
Unwieldy, perhaps. It's kinda hard to argue that the installer needs
to be this complicated, that Fedora (mainly QA) has to put in so much
effort into the installer each cycle testing for bugs in new features
and also regressions.
> 2. The Fedora QA group has 1 mac mini which is very old and is only
> used for total install and not dual boot. It would not have found this
The testing should be switched to be a dual-boot test, as it's what
Mac users are more likely to be using (and also a necessity for firmware
The firmware angle is a very good point. Ostensibly the firmware could
be downloaded, extracted and placed on the FAT32 volume (that Fedora
doesn't use) in the proper location, and NVRAM pointed to that
firmware file, and the firmware will consume it and thereby update the
firmware, without needing macOS. But the path of least resistance is
having a minimal macOS installed, even if it's not otherwise used.
> The Fedora QA group also has no one using Mac hardware day to
This isn't a problem. There are people using Macs day-to-day, and they report
bugs. The problem here, and I can't emphasise this enough, this problem is
a systemic problem with the installer QA, specifically.
Automated testing is problematic because it's not really possible to
virtualize Macs. The various guides for using kvm to virtualize macOS
actually use BIOS firmware, not UEFI, and one of several hacked up
bootloaders (e.g. Chameleon) is use to fake out XNU into booting on
non-Apple hardware. On these systems, there is no EFI System partition
of either the FAT32 or HFS+ variety, so it wouldn't test the installer
behavior we need to test. I don't know what work would be needed to
get Beaker to help do baremetal installations of Fedora on Mac
hardware, and if that is sufficiently not fakey that it'd be a valid
test. So for now it's a manual test. And as the installer regressions
can come at any time, it's a lot of manual testing, and in this
particular case it's not enough to just test if the install media
boots, and the installer launches, an installation writing to the
drive must be done.
Once the machine is installed, it's usually fairly straight
update packages, downgrade them, and fix hardware specific problems as long
as the device can be booted, and a sufficient amount of hardware is working.
The installer not working, especially when it's a last minute problem,
it becomes a blocker. Do we need a different schedule for installer
I'm pretty confident this bug was in Rawhide before F25 branch. I'm
also pretty confident, looking at the Mac I use for testing, that the
small HFS+ volume Anaconda creates and uses as an EFI system partition
only on Macs, was already present for each of the tests I did, and
that's why I never ran into the problem. It looks virtually certain I
did not in fact have completely free space for the installations,
everything but the three Apple creates partitions (FAT32 ESP, macOS
main volume, macOS recovery volume), and the Fedora HFS+ ESP were
deleted, but the presence of that one Fedora HFS+ ESP was seemingly
enough for me to not hit the bug.
So it can't be proven that this is a case of insufficient testing;
it's a case of testing that was imprecise or was just not thorough.
Possibly there should be a single Mac test case that asks for a very
explicit setup, with instructions on how to get to that state using a
path of least resistance, to just test the most minimal "get from A to
B" path, instead of 2-3 or more ways to get there. So at least we have
the most common and simple path tested, and flagged earlier on in the
release cycle that it hasn't been tested.
It's better, but I don't think that the problem lies in the
although some things could be done better there.
The main problem to me seems to be that the installer sees too little
testing, or too little testing when big changes occur, or not a wide
enough breadth of testing scenarios, at the development stage.
I had no idea macefi stuff was touched. If I'd known this, I'd
probably have been more skeptical and vigilant how I was testing,
instead of becoming more trusting of the installer (and more
complacent of its, in my opinion, inevitable betrayal) - but that's
rather circular logic on my part.