On Tue, May 03, 2022 at 05:14:53PM -0700, Adam Williamson wrote:
but these bugs weren't discovered at that time. This is likely
because
of your idea 2: "basic functionality" is a bit up for debate. You can
take an extremely minimalist approach to this (run the app, click a
couple of buttons, say it's OK if nothing explodes and no babies are
eaten) or a slightly more maximalist one (run the app, and actually try
and do something useful with it). In this case, before Final, when we
ran this test we mostly did the minimalist thing. At Final RC stage, we
went a bit more maximalist.
Yeah, the tension: we're moving towards more polish, but the available time
to fix gets smaller. And it's not like fixing the things that are at that
"more maximalist" end takes less time.
> As a barometer for "embarrasing", you can imagine
me trying to explain
> the issue to a tech reporter, and weigh how awkward I will feel saying
> "this is fine" vs. how I will feel explaining that we delayed the
whole
> release for that same issue.
I agree with the sentiment but I'm not sure about the phrasing. It's
extremely subjective, and I don't think subjective criteria work very
well. It also wouldn't necessarily "solve the problem": the bugs we
Yesssss, sorry. That was meant to be a sentiment-based way to hopefully
convey what I'm trying to say, not a suggested phrasing.
discovered this time really are pretty embarrassing, honestly.
Imagine
doing a keynote showing off the sleek default apps included in GNOME,
running them, and trying to do...well...actually anything at all useful
with them. It wouldn't go very well.
This is why I don't do live demos. :)
> 3. Problems found which are not regressions should not be
blockers. We're
> just hurting ourselves when we make this our forcing function to get
> something fixed.
This is one of those things that sounds great until it doesn't. I can't
quite recall any specifics, but I definitely think there have been
cases recently where we've had strong support for a bug that is not a
regression to be a blocker. Sometimes a bug is just really bad but we
didn't see it before; even if you can make a wonk-y argument that
there's no point making it a blocker because if we do, it just means
the previous release stays as the "current" one for longer and it has
the same bug, in practice it's hard to hold that line when now there's
a bug report that everyone can see that says how badly this thing is
broken.
I feel like I can defend that line pretty confortably. Send 'em to me. :)
In seriousness, I think we look to blocking the relase because it's such a
big signal. The train comes to a stop and everyone can see. But it's a big
signal _because it's painful_, and pain isn't the best motivator. And while
that pain does sometimes land on people who can do something, even if that
_would_ help, it's indiscriminate.
In the metaphor, people can't get to where they are going, the tracks are
blocked, etc., etc., and none of that has anything to do with the issue to
fix. (I'm tempted to keep running with the train metaphor, but reluctantly
will stop.)
This is the idea of Prioritized Bugs, and I think it's relatively
successful. It's another way to make sure we're focusing on serious
problems. Making that more visible could help.
> I don't know how to phrase this in a way that doesn't
make Adam sad with
> me, but maybe something like: Desktop application blockers discovered
> during the final freeze are automatically waived unless the relevant Spin
> or Edition team decides otherwise.
You're right that this makes me sad. I don't think it's a good
approach. I think it's an attempt to solve a problem that I would maybe
look at differently, and which we're currently discussing in a ticket:
https://pagure.io/fedora-workstation/issue/304
For me, the big question your mail never quite arrived at is, *why* did
these bugs show up in Fedora 36 Final RCs at all? They really should
not have done. They are bugs in applications that are, supposedly, core
parts of upstream GNOME. They appear in 42.0 releases of those
applications - i.e. in releases of those applications that are,
according to upstream's versioning scheme, stable releases for public
consumption.
Stable releases of core components of a major desktop should never
contain bugs like "deleting contacts sometimes doesn't work" or "you
can't add photos to an album in the Photos application because the
dialog where you're supposed to do it is completely broken and the list
entries multiply like rabbits who've been dosed up on viagra".
Distribution validation testing is not *for* finding bugs like this.
The reason we're all instinctively feeling that something is Not Right
here is that something *is* Not Right, but the big thing that's Not
Right is the upstream GNOME release process. It's not (IMHO) any part
of the Fedora process - there are things we could tighten up there, but
I see those as subsidiary problems.
It's Not Right that GNOME can ship a 42.0 release containing entirely
broken applications. That should not be happening. It's not something
we should have to design our Fedora distribution validation testing
process to fix.
So, yeah, I think there's something to do this. There were a lot of
last-minute-feeling kinds of changes not just to applications but also with,
like the dark-mode desktop background stuff.
Shouldn't it be rare that we're discovering general "this doesn't
work"
issues _that aren't Fedora Linux specific_ in Fedora QA?
Also, I think some of the time-tension (from the start of this message)
stems from an assumption that isn't working out: "this app might be rough at
beta, but will more polished by the time we're doing final validation,
because upstream is working on that".
Maybe this is something we can bring to GUADEC to work together to improve.
(Virtual at the end of July.)
The criterion and the test case were written with the unspoken, but
IMHO reasonable, assumption that in general we could trust desktops to
provide us with more-or-less-working software. They were never written
with the intent of finding this kind of bug. The scenario I was
envisioning when we wrote them was "oh, we accidentally packaged a
broken development version of app X" or "we're still including random
third-party app Y in the Workstation edition but it's not been
maintained for years and doesn't work any more, let's throw it out". I
was never envisaging having to deal with "GNOME shipped us completely
broken applications in a stable release". I don't think our goal should
be to design a release validation process that deals with that, because
*that shouldn't happen*.
This all makes sense.
I still think my suggestion might help even then, for the same reason I gave
to Kamil: it flips the default pressure, moves it away from QA, away from
the threat of stopping the whole train and motivation-through-guilt.
> 5. Okay, and... bigger: we should aim for more approaches which
let us
> decouple as much as possible from the Release. (My grand hope is that we
> can release every deliverable on its own schedule, but I also understand
> the _highly aspirational_ nature of that idea. But...) What if we could
> just easily ship GNOME Photos from GNOME 41 until a fix is found in the
> updated one?
I mean, we maybe could. I dunno if we tried that yet. There's not
necessarily anything in the current rules/policies that precludes this,
AFAIR.
For small things, it might just be a matter of shipping an old RPM. Maybe
with the dreaded Epoch. But in a lot of cases it probably means we are gonna
need a bigger [container].
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader