On 02/03/2014 09:15 PM, Adam Williamson wrote:
There is certainly a resource question for QA: it's generally
reasonable
to guess that the Products will want to define a minimum expected level
of quality and that the project as a whole will want the primary
products (that wording is a hedge against the possibility that we wind
up defining lots more Products - right now I'm assuming the three
currently-defined Products are 'primary' ones) to meet their minimum
requirements for a Fedora release to ship.
Right now we have fairly minimal requirements for the desktop and KDE
spins, and almost nothing for server or 'cloud' areas of the project, so
it's reasonable to assume the overall testing workload in a .next
universe will be higher than it is right now, and Johann is right to say
that, right now, 'QA' struggles to perform all the work that's
*currently* required.
So this certainly is an area of concern that will likely need to be
looked at and resolved. Johann is of the opinion that the way to do this
is for QA to test the base system and leave everything above that to the
products, but we have not decided anything like that yet: as the summary
note above says, we really can't plan much until the .next / Product
proposals take more concrete form. Especially, we need to know what the
Products think their minimum quality requirements will be, obviously.
And I should add to that on a 6 month development cycle and Adam was it
you that posted a summary of the time we spent in release blocker bug
meetings for a whole cycle on your blog or was it Tim or Kamil which
should give people a bit clearer picture how many hours QA spends on
just that alone.
Anyway the only way I can see us remotely manage to cover multiple
products is to alter Anaconda release cycle either before alpha or after
beta which is necessary to free up resources to do so.
( Anaconda is and always has been QA's nr.1 resource hog )
I should also mention that the QA community proposal I'm working on with
Mike will only be applied to base ( small group of components ) to
begin with to see for the first if it will work and if it does iron out
any issues that we find *before* we applied on a larger set of
components as well as update-testing if turns out to be success.
JBG