Hi Adam,

On Wed, Jan 12, 2022 at 6:14 PM Adam Williamson <adamwill@fedoraproject.org> wrote:
On Wed, 2022-01-12 at 11:47 +0100, Miroslav Vadkerti wrote:
> On Tue, Jan 11, 2022 at 10:30 PM Adam Williamson <adamwill@fedoraproject.org>
> wrote:
>
> > On Tue, 2022-01-11 at 21:15 +0100, Miroslav Vadkerti wrote:
> > > Hi Adam,
> > >
> > > More information is in Testing Farm artifacts, you can find the link in
> > > Jenkins log:
> > >
> > >
> > http://artifacts.dev.testing-farm.io/7d699360-cd4e-4307-b17a-d76424461732/
> > >
> > > Seems like some conflict:
> > >
> > > Error:
> > >  Problem: package pipewire-plugin-jack-0.3.43-1.fc35.x86_64 requires
> > > jack-audio-connection-kit, but none of the providers can be installed
> > >   - package pipewire-jack-audio-connection-kit-0.3.43-1.fc35.x86_64
> > > conflicts with jack-audio-connection-kit provided by
> > > jack-audio-connection-kit-1.9.19-1.fc35.x86_64
> > >   - package pipewire-jack-audio-connection-kit-0.3.43-1.fc35.x86_64
> > > conflicts with jack-audio-connection-kit provided by
> > > jack-audio-connection-kit-1.9.19-1.fc35.i686
> > >   - conflicting requests
> >
> > Ah, OK, thanks, I see. So that's just this test not really coping with
> > an update containing packages that conflict with default-installed
> > packages, I guess...it's trying to assert that it can install the
> > packages from the update, but in this case it's correct that it can't
> > install pipewire-j-a-c-k since regular j-a-c-k is already installed.
> >
>
> The user has to cope with conflicting packages himself,
> we are able to exclude some packages from updating using
> prepare's exclude field, something we use for example
> in curl:
>
> https://src.fedoraproject.org/rpms/curl/pull-request/11#_2__6
>
> Which has conflicting packages. Maybe that would help here also,
> maybe not, not sure ...
>
> If there is a way how CI should better handle the update here, we will
> be happy to fix it.

Well, I guess the interesting wrinkle here is exactly how the test
operated. The update in question contained two packages, pipewire and
lorax:

https://bodhi.fedoraproject.org/updates/FEDORA-2022-4cf1bb6eed

of those two packages, it's lorax that lists this test as gating in its
config file:

https://src.fedoraproject.org/rpms/lorax/blob/rawhide/f/gating.yaml#_7

pipewire does not. And the test reports itself as having run on the
lorax package, not on the update. In resultsdb the 'type' is
'koji_build' (i.e. a single Koji build, not a Bodhi update) and the
'item' is 'lorax-35.9-2.fc35'. In CI messages, the 'artifact' dict is
all about the 'koji-build' 'lorax-35.9-2.fc35':

https://apps.fedoraproject.org/datagrepper/id?id=2022-eaec10eb-2ecb-4055-8e60-0cdb1bb9b71d&is_raw=true&size=extra-large

however, it seems this specific test actually runs as a test of the
entire update. It pulls in all the packages from the update and fails
if it can't install any of them.

So...we're reporting a failure of a test of "the lorax package" which
didn't actually have anything to do with the lorax package. And we're
using the gating configuration of the lorax package to decide whether
to gate on a test which actually runs against *every package in the
update*. From pipewire's perspective, there's no way it can communicate
"this test isn't safe for my package, don't run/gate on it" that would
work if some *other* package in the update specifies that test as a
gating test...

Given the above, I'm not sure we can really say it's ever entirely safe
for a package to gate on this test. Because this test is going to
function differently depending on what *other* packages are in the same
update. Even if a package can affirm that *its own subpackages* should
always be installable in any update it's a part of, it can never safely
affirm that *all subpackages of any other package in the same update*
should always be installable.

I guess this is one of those early design decisions that did not age very well. CI is trying to install all (sub)packages for given builds automatically, before running tests. This means that for example lorax tests don't need to explicitly declare that a specific lorax subpackage is needed, because (in theory) all subpackages should be pre-installed on the test system automatically. And this works surprisingly well for the vast majority of packages as they don't have any conflicts on subpackage level. Maybe that's why this CI behavior seemed like a good idea initially. I am actually not sure when this behavior was introduced, but it's been the default in CI for a very long time now.
 
I personally wouldn't be opposed to a nuclear solution when we would simply change this behavior (with some delay after the announcement) so that all tests would need to properly declare all their dependencies.

Thanks,
Michal


I guess the smallest 'fix' for this would be to make all packages from
the update *available* to dnf during this test (in case the package
under test has interdependencies with the other packages in the update,
which is likely why they're bundled together), but only have it assert
that *subpackages of the specific package under test* are installable.
As long as the test is claimed to be a test of the *package* and not
the *update*, it shouldn't fail if a subpackage of *some other package
in the update* is not installable.

WDYT?
--
Adam Williamson
Fedora QA
IRC: adamw | Twitter: adamw_ha
https://www.happyassassin.net

_______________________________________________
CI mailing list -- ci@lists.fedoraproject.org
To unsubscribe send an email to ci-leave@lists.fedoraproject.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure