Adam Williamson wrote:
The 'comment' field exists to allow people to express all
these things,
but as it's just a completely free-form text field, it's intrinsically
impossible to really base any programmatic stuff or even policy on it.
In theory maintainers could submit updates without using autokarma and
then keep a careful eye on the feedback and 'tend' their updates
manually, but I think it's pretty clear that in practice, this is not
what happens: maintainers really want to be able to use the karma system
as a 'helper', they want to farm out the evaluation process to Bodhi/the
karma system. But our current system is too stupid to handle this
perfectly, so we get these breakdowns.
It's not that they WANT to be able to do that, it's that the system is
rigged to encourage that broken practice. Autokarma (ab)use was much lower
before the enactment of the Update Policies. And really, autokarma needs to
just go away entirely. Having an intelligent being interpret the free-form
text field is the only way to make sane decisions (which also implies that
we should not arbitrarily impose any restrictions that, by their nature,
cannot take the free-form text into account).
With a more flexible karma system we have a *lot* of opportunity to
do
much cleverer stuff. We can provide presets for all the above different
things that are currently commonly expressed via +1 or -1 with a
comment. This opens up possibilities at two different levels: the distro
policy level, and the packager level. We can make the distro policy much
more fine-grained, if we want to - we can require certain of the 'karma
types' to be available in all updates, and for instance, block any
update where X people pull the 'it's completely busted' or 'it
introduces a security vulnerability' cord, regardless of how much
broadly-categorized 'positive' karma it has. At the packager level, the
packager gets the freedom to define a much more fine-grained policy for
when they're happy that updates to their package are 'good to go', but
they still don't have to sit there reading the emails and manually
interpreting what people have written. You get to define the policy that
makes the most sense for your package, within the confines of the
distro-wide policy - if you have a good package-specific test suite, you
can say to the auto-karma system 'don't send this update out until at
least one person sets the "I ran the test suite and it passed" karma
property.
To me, this just screams "OVERENGINEERED!!!". :-(
You are introducing a lot of complexity, that will ultimately always only be
an approximation to reality. You just cannot reliably quantify all the
details. E.g. "this introduces a regression, but the regression is that you
sometimes have to click OK twice (instead of once) to format a 5.25" floppy
in KFloppy, whereas it fixes a critical bug in KDE Plasma Desktop where all
your data was sent to the NSA and then securely wiped from your hard disk".
:-) (Yes, of course that is an exaggerated example. ;-) I sure hope we don't
have bugs like that. ;-) ) If you only have "fixes a bug, but introduces a
regression" as a feedback type, you probably end up making the wrong
decision. If you try to get more fine-grained, then you again need numbers
to quantify the severity of one issue vs. the other, and those will
inherently be subjective. (Users always think THEIR bug is the end of the
world whereas regressions that don't affect them are entirely unimportant.)
The complexity also means there are a lot more arbitrary parameters to deal
with. The current stable/unstable thresholds are already bad enough, and
often end up set to the wrong value. A decision process tends to be the
worse the more arbitrary parameters it needs.
And of course, more complexity means less transparency. It becomes harder
and harder to understand what really needs to be satisfied for an update to
be allowed to go stable.
So I can only advocate for the KISS approach: The update is stable when the
maintainer says so, period. We do not need any karma, be it a simple ±1 or a
long (and inherently non-exhaustive) list of all the things that can
possibly happen. So let's just do away with the 3 radio buttons and use a
free-form text field only. We just need somebody able to read comments, and
that is what we have maintainers for!
Kevin Kofler