On Wed, Jul 1, 2020 at 12:57 PM Stephen John Smoogen <smooge(a)gmail.com> wrote:
> On Wed, 1 Jul 2020 at 14:46, Miroslav Suchý <msuchy(a)redhat.com> wrote:
> > Dne 30. 06. 20 v 9:44 Pierre-Yves Chibon napsal(a):
> > > What are you talking about here? The Fedora release process? The
> > > in dist-git?
> > This. And creating new gpg keys, new mock configs, new tags in Koji, add
release to retrace.f.o, Copr, ... I have a
> > dream where you just bump up number in - let say - PDC and everything else will
happen automagically. At right time.
> I think choosing the one tool which is so end of life to do it in.. is
> a sign of why we can't do this. Every release some set of tools in
> Fedora get added by some team who have been working on their own
> schedules and their own API without any idea of any other teams
> working on.
> We then have to do a lot of integration and make it work
> before the release deadline to make it work. Then usually after 1 or 2
> releases that software team is no longer in existence and we have to
> continue with it waiting for the promised replacement which will do
> all those things you list above.. but instead get some other tool
> which has to be shoved in.
This is an excellent summary of the problem over the past couple of years.
I think one of the problems with PDC was that so many teams had to
adapt all their *giant* tools to it, and this integration effort was
unsustainable when we have natural contributor turnover. Everything
ended up tightly coupled together so that it was really difficult to
remain agile as tools and business requirements (naturally) changed.
Also we never drew the line and wrote a list of things that PDC was
*not* going to do, so the Second Syndrome effect just kept growing
until it collapsed.
Instead of having everything having to talk to PDC to determine its
configuration, I'm approaching this problem from the other end -
making Ansible talk to all the services according to each service's
API. Here are some things I like about this approach:
1. Ansible is really simple and well-documented.
2. It's easy to start small and get value incrementally.
3. The playbooks can (and do) change independently from the APIs. This
kind of agility is essential because SOPs must be able to change over
4. If we haven't completely implemented something in Ansible yet, the
service itself is not completely broken. The workaround does not
require multiple teams of developers (like with PDC). The workaround
is that the administrator simply does the thing they used to do
manually (and file tickets for the missing RFEs in the Ansible modules)
For the koji-ansible project, we're using it to configure some large
products internally. Now we have to expand this concept to the rest of
the pipeline (Bodhi, Pungi configuration, etc.)
Clement started on https://github.com/cverna/bodhi-ansible
but I think
that's abandoned at this point. I do think this is the way to go,
though. In some ways this is the point - it should be possible for
these Ansible modules to be isolated so that when contributor turnover
happens, the whole system does not fall apart.
This sort of worries me though: abusing Ansible to do this sort of
thing is not what it was made for. It also makes mentally modeling how
everything works so much harder because the sequencing (or execution
flow) of actions is non-obvious.
And Ansible's own APIs are horrifically unstable, to the point that
I've had *bar conversations* about how people have to pin to specific
Ansible releases because all the crap they build on top of it to bend
Ansible to their needs relies on the part of Ansible that's
deliberately *not* stable: the Python module extension interface.
We're potentially trading one kind of technical debt for one that's
arguably worse: debt we can't fix because we're eating our own tails.
真実はいつも一つ！/ Always, there's only one truth!