On Fri, 2017-11-03 at 14:00 -0700, Adam Williamson wrote:
On Fri, 2017-11-03 at 11:00 -0400, James Antill wrote:
> Hey, so we put this together for test requirements for Modularity
> server:
>
>
https://fedoraproject.org/w/index.php?title=User%3AJames%2FDraft%3A
> Fedo
> ra_27_Final_Release_Criteria%2Bmodularity&diff=501506&oldid=501503
Thanks for that!
I'm not sure I like the preamble. For a start, the criteria are not
"instructions" for testing. Beyond that, I'm not sure it serves any
function; we don't need to specifically state that the criteria
really do apply to this or that. What's the purpose of the preamble,
to you?
Mainly as a big hint to testers that just because they tested package
FOO in Fedora 27 doesn't mean it necessarily works in Fedora Modular 27
(which is kind of new/unique).
The meat of the requirements seems fine, but I'd probably choose
to
represent them a bit differently. The requirement isn't really about
'modular data' in the repositories, that's kind of an implementation
detail. The requirement is for the installer and the installed system
to be able to manipulate modules, right? So, I'd want to follow the
pattern established by existing requirements related to *packages* in
that area.
Yeh, that seems fair.
For instance, we have a Beta criterion:
"Package set selection
When installing with the generic network install image, interactively
selecting a package set other than the default must work."
So, wouldn't it make sense to add another criterion directly below
it:
"Module selection
When installing with a release-blocking
[
https://docs.pagure.org/modularity/ Modularity]-enabled image,
changing the default module selection
must work."
or something like that?
I'm not 100% sure what the status of anaconda will be for 27 GA, so
I'm not sure this will be true.
Similarly, for the installed system, I'd probably want to add a
new
criterion to "Post-install requirements" at Beta or Final rather than
have this new section. It's an interesting point that we don't
actually require package installation or removal to work in the
criteria; this is intentional, though arguable. We only require
package *update* to work in the criteria, the logic being that if
package install or removal is broken we can ship an update to fix it.
I'm not sure I'm still married to that position, though. :) So we
might want to extend the criteria to require that basic package
installation and removal with the official tools works, and add a
corresponding criterion for module interactions directly beneath it.
WDYT? Thanks!
It's still true that as long as the platform module can be updated we
can fix DNF which means we can fix anything else ... at least as well
as we could before :).
Installing a module should work though, as should changing the stream
of an installed module ... so it would be cool to test those even if
issues don't become release blockers (I kind of assume people test
installing packages ;).