On Thursday, February 22, 2018 3:50:38 PM CET Pierre-Yves Chibon wrote:
On Wed, Feb 14, 2018 at 05:28:20PM +0100, Petr Šplíchal wrote:
> When, for a particular package, it is the most efficient way to
> maintain tests in a separate repo why should we discourage from
> this approach? There are packages where it makes more sense to
> store test code directly in dist-git. And it is still an option.
> But why should we enforce this for all?
My worries are basically that this mechanism is hiding the tests from
the package maintainers. It splits the concerns between people
maintaining the artifacts and people maintaining the tests, which is
exactly what the original plan to bring CI was *not* about.
Since we'll have to "hook" the remote repo into dist-git code, I must be
missing something. Where do you see the hidden magic?
The idea has always been to bring the tests where the code lives, so
that both can be worked on as one, to make tests be a concern of the
package maintainer very much like updating the artifact (rpms, image..)
is, while getting support from QE for them (the tests).
If the tests live in separate repo, they still can "work as one".
In addition, this is what I fear most:
The tests for the package are stored elsewhere, we're not sure where (the tests
namespace, a random git repo on the internet...), the packager update package
and the tests fail.
What do you think will happen?
a) the packager will look for $random_place_of_the_internet where tests are and
spend time trying to fix them.
Yes, please don't force us to use random_place_of_the_internet, but rather
allow maintainers to use standardized place.
Then e.g. 'rpmtests/<PKGNAME>' repo can automatically grant commit access
to package maintainers of 'rpms/<PKGNAME>' to not overcomplicate things,
but still, we can grant non-packagers access to 'rpmtests/<PKGNAME>'.
What about proventesters, could we grant them access to all tests, or could we
create new role?
b) the packager will turn off the tests because they get in the way
Polemics (even below) about turning-off the tests is FUD. Since tests are
opt-in feature, why would anyone thought this way?
If the packager wants to turn off the tests, they will have to go
through
dist-git to do it, they may very well end up turning the tests off anyway, but
if the tests are right there, they may as well have a quick look at them to see
if they can fix them quickly before deciding.
In addition, if the packager turn the tests off and the people maintaining the
tests do not realize that, they will end up spending time maintaining
$somewhere_else tests that aren't being used.
Having them interact directly with the dist-git repo will increase the
chances
that they see/realize when something is turned off.
If that means we have less tests in dist-git because the maintainers do
not want them, I'd say so be it. In the long term this is their loss,
they are the ones who will get the bug reports and will have to deal
with them, they are the ones who will have to dive into old code rather
than going back into something that is still fresh in their mind.
There's no loss, no. Everything is as transparent as with in-dist-git
testing, except it gives us more flexibility
- to allow more people to contribute.
- keep the package's git smaller and concentrated on packaging
As long as, it is clear that they have been approached and that it is
their
choice to not merge pull-requests adding tests, I think the people offering to
help should not be the ones blamed.
The help here is very much welcome, I wouldn't blame anyone though (I for
one only wait for a bit more flexible form of CI).
Btw., the far more important problem is that we can not setup CI for
non-atomic host packages, and that we can not use the CI for Rawhide
gating.
Pavel