Hey, so we put this together for test requirements for Modularity server:
https://fedoraproject.org/w/index.php?title=User%3AJames%2FDraft%3AFedo ra_27_Final_Release_Criteria%2Bmodularity&diff=501506&oldid=501503
On Fri, 2017-11-03 at 11:00 -0400, James Antill wrote:
Hey, so we put this together for test requirements for Modularity server:
https://fedoraproject.org/w/index.php?title=User%3AJames%2FDraft%3AFedo ra_27_Final_Release_Criteria%2Bmodularity&diff=501506&oldid=501503
Thanks for that!
I'm not sure I like the preamble. For a start, the criteria are not "instructions" for testing. Beyond that, I'm not sure it serves any function; we don't need to specifically state that the criteria really do apply to this or that. What's the purpose of the preamble, to you?
The meat of the requirements seems fine, but I'd probably choose to represent them a bit differently. The requirement isn't really about 'modular data' in the repositories, that's kind of an implementation detail. The requirement is for the installer and the installed system to be able to manipulate modules, right? So, I'd want to follow the pattern established by existing requirements related to *packages* in that area.
For instance, we have a Beta criterion:
"Package set selection
When installing with the generic network install image, interactively selecting a package set other than the default must work."
So, wouldn't it make sense to add another criterion directly below it:
"Module selection
When installing with a release-blocking [https://docs.pagure.org/modula rity/ Modularity]-enabled image, changing the default module selection must work."
or something like that? (It'd be nice to be more precise about exactly what we expect from the installer, but I can't be more precise because I don't know yet :>)
Similarly, for the installed system, I'd probably want to add a new criterion to "Post-install requirements" at Beta or Final rather than have this new section. It's an interesting point that we don't actually require package installation or removal to work in the criteria; this is intentional, though arguable. We only require package *update* to work in the criteria, the logic being that if package install or removal is broken we can ship an update to fix it. I'm not sure I'm still married to that position, though. :) So we might want to extend the criteria to require that basic package installation and removal with the official tools works, and add a corresponding criterion for module interactions directly beneath it.
WDYT? Thanks!
On Fri, 2017-11-03 at 14:00 -0700, Adam Williamson wrote:
On Fri, 2017-11-03 at 11:00 -0400, James Antill wrote:
Hey, so we put this together for test requirements for Modularity server:
https://fedoraproject.org/w/index.php?title=User%3AJames%2FDraft%3A Fedo ra_27_Final_Release_Criteria%2Bmodularity&diff=501506&oldid=501503
Thanks for that!
I'm not sure I like the preamble. For a start, the criteria are not "instructions" for testing. Beyond that, I'm not sure it serves any function; we don't need to specifically state that the criteria really do apply to this or that. What's the purpose of the preamble, to you?
Mainly as a big hint to testers that just because they tested package FOO in Fedora 27 doesn't mean it necessarily works in Fedora Modular 27 (which is kind of new/unique).
The meat of the requirements seems fine, but I'd probably choose to represent them a bit differently. The requirement isn't really about 'modular data' in the repositories, that's kind of an implementation detail. The requirement is for the installer and the installed system to be able to manipulate modules, right? So, I'd want to follow the pattern established by existing requirements related to *packages* in that area.
Yeh, that seems fair.
For instance, we have a Beta criterion:
"Package set selection
When installing with the generic network install image, interactively selecting a package set other than the default must work."
So, wouldn't it make sense to add another criterion directly below it:
"Module selection
When installing with a release-blocking [https://docs.pagure.org/modularity/ Modularity]-enabled image, changing the default module selection must work."
or something like that?
I'm not 100% sure what the status of anaconda will be for 27 GA, so I'm not sure this will be true.
Similarly, for the installed system, I'd probably want to add a new criterion to "Post-install requirements" at Beta or Final rather than have this new section. It's an interesting point that we don't actually require package installation or removal to work in the criteria; this is intentional, though arguable. We only require package *update* to work in the criteria, the logic being that if package install or removal is broken we can ship an update to fix it. I'm not sure I'm still married to that position, though. :) So we might want to extend the criteria to require that basic package installation and removal with the official tools works, and add a corresponding criterion for module interactions directly beneath it.
WDYT? Thanks!
It's still true that as long as the platform module can be updated we can fix DNF which means we can fix anything else ... at least as well as we could before :). Installing a module should work though, as should changing the stream of an installed module ... so it would be cool to test those even if issues don't become release blockers (I kind of assume people test installing packages ;).