On Wed, 2018-05-30 at 14:23 +0200, Lukas Ruzicka wrote:
Hello,
I was checking the release criteria and I spotted this one:
https://fedoraproject.org/wiki/Fedora_29_Final_Release_Criteria#Release_i...
which does not have a test case connected to it. However, there is one test
case that does pretty much the same, although uses a different attitude. It
is the
https://fedoraproject.org/wiki/QA:Testcase_base_edition_self_identification
I think that the check for installed packages could be added to the above
mentioned test case and kill two birds with one stone:
1. If using a specific release of Fedora, check that the *fedora-release*
package is installed:
- rpm -qi fedora-release
2. Check that the *generic-release* package is in the repository:
- dnf search generic-release
The added criteria to pass the test would be:
1. The fedora-release package is installed.
2. The generic-release package is in the repository.
What do you think about it?
Unfortunately it's not quite sufficient. Just checking that a fedora-
release package is installed does not enforce that criterion; there is
pretty much always going to be a fedora-release package unless we
really screwed up, but it may not contain the correct 'names,
information and repository configuration'. There's two particular
requirements we always have to remember to check: the release must be
*at least* 1 (it cannot be 0.x, as this causes the 'Timbuktu warning'
to appear in some installer modes), and the updates-testing repo *must*
be disabled by default. Both those things are usually false until quite
late in the release cycle; somewhere around Final freeze time releng
will do a fedora-release package update that bumps the release to -1
and disables updates-testing.
As for generic-release - doing a 'dnf search' from an installed system
again doesn't quite cut the mustard, as that's looking in the 'fedora'
repo, and until updates-testing is disabled, it's looking in the
'updates-testing' repo. At least conceivably, the package could be
there but not in the actual release repository. It also may not be
'appropriately versioned' (again, the release should be -1 or higher).
These have always been sort of magic things that we and releng
magically know, and it's certainly true that we should write them into
a test case to formalize the knowledge. I would not want to extend this
test case to cover that, though. Lately I've actually been trying to
make the test cases vaguely related to identification *more* specific
and *less* general, in the interests of making them easier to automate.
If we were going to extend an existing test case,
https://fedoraproject.org/wiki/QA:Testcase_base_artwork_release_identific...
might be a better choice, but really, I think it would be best to have
a new separate test case. One tricky thing is that we shouldn't mark it
as a fail early in the cycle when fedora-release has u-t enabled and a
release of < 1; this is correct at that time. The test case should
cover this somehow - it should have two - or, in fact, three, thinking
about Rawhide composes - different expected results for what the
packages should be versioned and what repositories should be enabled at
different points in the cycle.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net
http://www.happyassassin.net