Hello,
I want to consult our current strategy regarding per-arch tasks. We currently have no such
tasks, all of them are generic (they can run on any host system and test any single or
multiple item arches). I want to convert task-rpmdeplint [1] to arch-specific, to avoid a
race condition [2]. Also, I believe per-arch tasks will be necessary for supporting
dist-git tasks (we'll need to assume that they are arch-specific by default). I found
several issues in our current implementation.
First, I submitted a diff for making --arch cmdline argument a single value instead of a
multiple value list:
https://phab.qa.fedoraproject.org/D1171
That has the benefit of ${arch} being a simple string, and that makes it easier in formula
to operate with it - for example pass it to a bash script that needs to do something. It
also makes possible to easily extend architecture list that is passed to e.g. koji
directive. That is important for tasks dealing with multilib (rpmdeplint).
My view is that we should simplify things in order to keep our sanity and reasonable
implementation, at the expense of some performance optimization (running x86_64 and i386
tasks at the same time, because hardware allows that). So --arch will be a single value,
and that will determine what architecture should the task check. So if an arch-specific
task wants to run for armhfp, i386 and x86_64, trigger will schedule 3 different jobs on
particular hosts, and they will run independently. The task will be able to determine
whether it's arch-specific or generic in its formula (or scheduling database in the
future), therefore deciding whether it's executed once or multiple times (with
different --arch arguments). It will also be able to say which arches it supports or
doesn't support (so that we don't run it on e.g. armhfp even if we have armhfp
machines, if the task doesn't support it). If we need performance optimizations in the
future, we can do that by allowing the task to not be tied to a particular arch host (so
that we can execute armhfp rpmdeplint on x86_64 minion). But those runs will be always
independent (one job for x86_64, one job for armhfp, one job for i386).
Does that sound reasonable? If agreed, I really want to push the aforementioned change
before dist-git tasks take off, since then we'd be breaking our formula API (and
workflow expectations) for a lot of people.
Second, we don't have any non-x86_64 minions yet. So even if we support the above, we
are currently able to execute just x86_64 arch-specific tasks. I assume we'll get some
arm builder with armhfp buildslaves in the future, but that will take time. i386 is a
different story. We are able to create the minions (we just don't do it yet), and we
even have a single i386 builder (staying idle all the time). However, i386 switched to an
alternate arch since F26, so I'm not really sure if we want to invest a lot of time
into it (we should obviously focus primary arches first). Also, creating minions for i386
means data storage requirements doubled, and number of issues to debug doubled. I wonder
whether it's a worthy investment of time at this very moment. Our architecture should
obviously be prepared to handle arch-dependent tasks (especially from formula/API
perspective, so that we don't force all users to amend their code in the future), but
I'm wondering whether we should actually *have* multiple arches support right now, or
rather keep it simple and tell everyone that only x86_64 is supported during the pilot.
WDYT?
If we decide to go with i386 minions right now, we'll need support in trigger. It will
either read the formula for a new field that we add, or it will have hardcoded value that
it should run task-rpmdeplint (and any dist-git task) with multiple architectures. So it
will schedule two jobs for each such task, one with --arch x86_64, and one with --arch
i386. Can somebody with trigger experience estimate whether this is a difficult change to
do or not? Also, do you see any further changes required in other tools?
[1] I mostly ignore task-depcheck, because it's going away. But the same issue would
apply also there.
[2]
https://phab.qa.fedoraproject.org/T894