On Tue, 21 Mar 2017 09:44:54 -0400 (EDT)
Kamil Paral <kparal(a)redhat.com> wrote:
> Hello,
>
> I want to consult our current strategy regarding per-arch tasks. We
> currently have no such tasks, all of them are generic (they can run
> on any host system and test any single or multiple item arches). I
> want to convert task-rpmdeplint [1] to arch-specific, to avoid a
> race condition [2]. Also, I believe per-arch tasks will be
> necessary for supporting dist-git tasks (we'll need to assume that
> they are arch-specific by default). I found several issues in our
> current implementation.
>
> First, I submitted a diff for making --arch cmdline argument a
> single value instead of a multiple value list:
>
https://phab.qa.fedoraproject.org/D1171
> That has the benefit of ${arch} being a simple string, and that
> makes it easier in formula to operate with it - for example pass it
> to a bash script that needs to do something. It also makes possible
> to easily extend architecture list that is passed to e.g. koji
> directive. That is important for tasks dealing with multilib
> (rpmdeplint).
>
> My view is that we should simplify things in order to keep our
> sanity and reasonable implementation, at the expense of some
> performance optimization (running x86_64 and i386 tasks at the same
> time, because hardware allows that). So --arch will be a single
> value, and that will determine what architecture should the task
> check. So if an arch-specific task wants to run for armhfp, i386
> and x86_64, trigger will schedule 3 different jobs on particular
> hosts, and they will run independently. The task will be able to
> determine whether it's arch-specific or generic in its formula (or
> scheduling database in the future), therefore deciding whether it's
> executed once or multiple times (with different --arch arguments).
> It will also be able to say which arches it supports or doesn't
> support (so that we don't run it on e.g. armhfp even if we have
> armhfp machines, if the task doesn't support it). If we need
> performance optimizations in the future, we can do that by allowing
> the task to not be tied to a particular arch host (so that we can
> execute armhfp rpmdeplint on x86_64 minion). But those runs will be
> always independent (one job for x86_64, one job for armhfp, one job
> for i386).
>
> Does that sound reasonable? If agreed, I really want to push the
> aforementioned change before dist-git tasks take off, since then
> we'd be breaking our formula API (and workflow expectations) for a
> lot of people.
I got a silent approval from Josef and Martin who are too scared to
respond publicly. So I pushed the patch.
> Second, we don't have any non-x86_64 minions yet. So even if we
> support the above, we are currently able to execute just x86_64
> arch-specific tasks. I assume we'll get some arm builder with
> armhfp buildslaves in the future, but that will take time. i386 is
> a different story. We are able to create the minions (we just don't
> do it yet), and we even have a single i386 builder (staying idle
> all the time). However, i386 switched to an alternate arch since
> F26, so I'm not really sure if we want to invest a lot of time into
> it (we should obviously focus primary arches first). Also, creating
> minions for i386 means data storage requirements doubled, and
> number of issues to debug doubled. I wonder whether it's a worthy
> investment of time at this very moment. Our architecture should
> obviously be prepared to handle arch-dependent tasks (especially
> from formula/API perspective, so that we don't force all users to
> amend their code in the future), but I'm wondering whether we
> should actually *have* multiple arches support right now, or rather
> keep it simple and tell everyone that only x86_64 is supported
> during the pilot.
Again, the general feel from behind the scenes seems to be to not
support i386 minions right now. So I'll go ahead with claiming to
everyone that we only support x86_64 tasks for the moment (even
though the framework should be more or less prepared to handle
multiple arches).
The generic tasks (like rpmlint, rpmgrill, abicheck) will continue to
download, test and submit results even to for other architectures
(per "supported_arches" in taskotron.yaml, which is currently set to
['x86_64', 'i386', 'armhfp']), because those tasks are not
minion-dependent.
Getting ppc64 or ppc64le machines should be only matter of filling a
ticket with fedora infra, these arches are part of the fedora cloud.
aarch64 is going to added soon.
Dan