On Thu, Nov 10, 2016 at 6:44 PM, Josh Boyer <jwboyer@fedoraproject.org> wrote:
> On Thu, Nov 10, 2016 at 5:09 PM, Adam Williamson
> <adamwill@fedoraproject.org> wrote:
>> On Thu, 2016-11-10 at 13:31 -0800, Chris Murphy wrote:
>>> https://meetbot.fedoraproject.org/fedora-meeting-2/2016-11-10/f25-final-gono-go-meeting.2016-11-10-17.00.log.html
>>> > > > 17:10:26 <adamw> i can't really vote -1 on this under the current criteria unless someone tries on a newer mac and it works. but given https://www.happyassassin.net/testcase_stats/25/Installation/QA_Testcase_dualboot_with_OSX_Miscellaneous.html , i am entirely open to dropping the criterion on the basis of failure to test.
>>> I think it's specious to drop the criterion on this basis. There are
>>> plenty of other things that don't get tested and their criterion don't
>>> get dropped.
>> I am actually planning to propose precisely this.
> My sincere apologies for not being in the meeting.  I could not attend
> due to a conflict with a different meeting.
> I am somewhat befuddled at the decision to block the entire release
> for this issue.  We seem to have created a criteria under the premise
> that "lots of people have Macs and want to run Linux/Fedora on them",
> yet our empirical data seems to have shown a distinct lack of testing
> that would bear that premise out.
> I agree 100% that lots of people have Macs.  I agree in part that
> people want to run Linux/Fedora on them.  I agree that a subset of
> those that want to run Linux on a Mac also want to dual-boot OS X.
> What I cannot get my head around though is how we've essentially made
> a decision based on perceived marketing value of those target users at
> the expense of every other platform we support.  Our engineering and
> testing resources are clearly not sufficient to cover this case.  If
> they are, then we have a fairly large communication problem
> illustrated here.  Or if that wasn't the reason, and it was simply
> "because we have a criteria written down" that also seems somewhat
> myopic.
> Please don't misunderstand me.  I want this to work and I think it is
> valuable.  I also appreciate everyone pitching in at the last minute
> and I'm sure it will get fixed.  However, I think we really need to
> take a strong look at what our Project can sustain, the value of the
> distribution as a whole across all Editions and platforms, and the
> resulting impacts of every possible slip.  The conversation I read in
> the IRC logs does not seem to reflect that, and despite people wishing
> for "not hero-ing" that seems like exactly what happened and will
> continue to happen, extending even into the release itself over a
> major holiday.
> We are technical people and bugs bother us and we want to fix them.
> Yet, we need to make judgement calls on blockers based on reality and
> overall benefit/cost of blocking the release, not because we have a
> known root-cause at the last minute and can probably fix it if we
> slip.  That way lies madness and we'll continue to have further
> arguments about playing catch-up in the schedule with a shorter cycle
> next release, etc.  How can we ensure that we balance that with a
> broader focus across the Project so that we don't continue to have
> problems of our own making?

My perspective of this imbalance is that Anaconda churns code, and Fedora is it's testing canary. And the consensus view is that more canaries are needed, rather than less poison gas.

17:39:34 <adamw> (i.e. it seems we have quite a lot of people who want to install os x on macs post-release, but almost no-one who wants to test pre-release)
17:39:39 <adamw> i guess this is because 'burner macs' are rare

Adam gets this right. Mac users by and large have no concept of OS testing. Only recently is there a public beta test program. Developers by and large test their application for compatibility, rather than testing OS installation - but even in pre-release testing, installer problems are rare. In a decade of testing, I can't think of a single bug I've hit in Apple's installer.

This is pretty much a case of in for a penny, in for a pound. Fedora contributes specialized QA people who can do this work correctly. And out of that you get defectors from Appleland who hopefully write some cool stuff or contribute in ways they otherwise wouldn't.

Now, in the Go, No Go meeting logs there's this idea of changing the criterion to take the form of a Hippocratic Oath, "do no harm" to macOS, rather than guarantee a successful installation of Fedora. This is further elaborated by noting even if the criterion is done away with, there's still a concern over the vary narrow path to data loss, and that alone is reason to block. And so now I can't help but think of this years old bug:


In that bug, two macOS users actually do in fact experience unrecoverable data loss. It's not hypothetical at all. QA said it was a blocker. Upstream said it's not. And therefore it's not a blocker. So there's general consensus to block on hypothetical data loss bugs when installing on Macs, but with proven data loss we can't be bothered with users who do this, it's their own fault for having trusted the installer.

This is actually the bug that has made me walk back my installer testing. I find bugs and I try not to report them. I don't think the installer can be made better anymore. I don't think I've helped users with my hundreds of hours of testing the intstaller, intstead I think I've hurt users my helping to incentivizing feature creap and code churn that makes the installer complex and buggy. That project is a classic case of having bitten off more than it can chew. It can't QA its own code sufficiently before it gets into Fedora, and then even Fedora can't QA it fast enough to avoid bugs like this one, and the one this thread is about, and many other blocker bugs that go disproportionately to the installer.

So yea sure, more canaries.

Chris Murphy