On Tue, Jan 18, 2022 at 11:04 AM Michal Srb <msrb@redhat.com> wrote:


št 13. 1. 2022 o 18:07 Miroslav Vadkerti <mvadkert@redhat.com> napísal(a):
Hi,

On Wed, Jan 12, 2022 at 9:12 PM Miro Hrončok <mhroncok@redhat.com> wrote:
On 12. 01. 22 21:00, Michal Srb wrote:
>
> I guess this is one of those early design decisions that did not age very well.
> CI is trying to install all (sub)packages for given builds automatically,
> before running tests. This means that for example lorax tests don't need to
> explicitly declare that a specific lorax subpackage is needed, because (in
> theory) all subpackages should be pre-installed on the test system
> automatically. And this works surprisingly well for the vast majority of
> packages as they don't have any conflicts on subpackage level. Maybe that's why
> this CI behavior seemed like a good idea initially. I am actually not sure when
> this behavior was introduced, but it's been the default in CI for a very long
> time now.

Relevant issue from 2019:

"RFE: Don't install all sub-packages into the testing environment"

https://pagure.io/fedora-ci/general/issue/29

Commented, with package excluding most of the packages should be unblocked.

 



However in this case it's even worse. It installs all *other* packages from
that update, which does not seem like a good idea even for vast majority of the
packages.

> I personally wouldn't be opposed to a nuclear solution when we would simply
> change this behavior (with some delay after the announcement) so that all tests
> would need to properly declare all their dependencies.

+1

Well we are fine to do that change, but I will find another couple of folks who think the default install all is good :( So maybe we just need to be able to fine-tune it. We now can exclude packages from the koji build, so feels like we are missing the "install only these (sub)packages" scenario ...

I can think of 2 (more) use cases where the default install-everything behavior can be problematic.

* opening a PR against tests/ namespace -- this simply runs the tests on the existing build, no Koji task id is explicitly passed to Testing Farm. So if the tests don't pull in the required packages, CI will fail. It could be argued that this is a bug in CI, but this is how it works now.

Right, tests should require what they need for testing, there are no objections about that. If they don't, they fail but it is not a bug in CI.


* reverse dependency testing -- I have a new build and I want to make sure that packages that depend on my component will still work when I push the build to the repository. So I can tell Testing Farm to run the tests of the dependent components, but I don't necessarily know the NVR of those components.

With all that said, I don't really see a good reason why tests shouldn't properly declare all their dependencies.

I am not advocating for going against that idea, I am just saying if they don't, we now have to install everything now. The reason is, that artifact installation is currently Testing Farm's responsibility. Once we drop this requirement and make the artifact installation part of the responsibility of tmt, we can drop that. The API now clearly states:

artifacts - Array of objects (environment.artifacts) - Additional artifacts to install in the test environment.

I guess we should file an issue somewhere ...

Best regards,
/M




Thanks,
Michal


 

Can somebody give me a package where this would be handy?

Thanks,
/M
 

--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok
_______________________________________________
CI mailing list -- ci@lists.fedoraproject.org
To unsubscribe send an email to ci-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure


--
Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE
IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252
Remote Czech Republic :: Red Hat Czech s.r.o


_______________________________________________
CI mailing list -- ci@lists.fedoraproject.org
To unsubscribe send an email to ci-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
_______________________________________________
CI mailing list -- ci@lists.fedoraproject.org
To unsubscribe send an email to ci-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure


--
Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE
IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252
Remote Czech Republic :: Red Hat Czech s.r.o