Harry G. Coin via FreeIPA-users wrote:
On 11/9/21 1:18 PM, Rob Crittenden wrote:
> Harry G. Coin via FreeIPA-users wrote:
>> I think freeipa represents something of a research opportunity.
>>
>> Among all the 'functional subsystem packages' out there, freeipa is the
>> 'tallest pyramid with the widest base' I'm aware of. By that I mean
it
>> has the largest number of dependencies which themselves are also
>> subsystems (over against libraries with little more than libc/kernel
>> dependencies); and that in order to be considered 'up and useful' all of
>> the subsystems contribute to the necessary 'running properly'
>> condition. In fact many of the subsystems freeipa relies upon are
>> themselves reliant on what you might call 'functional subsystems'.
>>
>> You might argue a 'distro' or a GUI environment would be better, but
>> that's more like a collection of shorter pyramids where the
'breakage'
>> of one doesn't materially impact the others.
>>
>> A hypothesis worth getting actual metrics on would be how often it would
>> appear 'freeipa breaks upon install' and 'freeipa breaks upon
upgrade'
>> when installed via a stream-style release, over against 'vm' or
'docker'
>> releases, and that over against 'lts/rhel/centos' vs fedora.
>>
>> Over the coming decades, a question needs resolving-- how high a 'brick
>> wall' is possible and sustainable (where each 'brick' is a
dependency)
>> and what can be done to make 'higher walls' more reliable? Because each
>> subsystem represents thousands of hours of focused labor, and that
>> packaged and made available to other developers faced with either
>> re-creating the function or using the subsystem.
>>
>> Presently we're faced with the smallest changes in a dependency
>> corrupting a database upon 'upgrade', or some seeming trifle breaking
>> the ui, etc.
>>
>> I suppose it would lead to a 'so, then, with that, now what?' question
>> -- and that would be really interesting as well.
>>
>> So, there's a Monday morning idea.
> Yes, we have a lot of dependencies, and dependencies of dependencies.
> Some of the major packages work with us and test their release
> candidates against IPA before releasing. Others not so much. We have
> automation that tends to find most of this in the wild on Fedora. We
> have no control over CentOS or the other rebuilds and have some limited
> influence in other distributions.
>
> IPA is a release blocker in Fedora so if the automation isn't passing
> during a release it can be blocked, but there is no package resolver I
> know of that detects whether an upgrade in a package will break IPA. I'm
> not sure there is the computing capacity to test for that.
>
> I have no memory of a database corruption issue on upgrade in all my
> time in IPA, either NSS or 389-ds.
My memory differs, and I have an intuition about the reason why which
may be useful.
The folks closest to freeipa necessarily stay very current and generally
move almost in sync with one another. Then, 'the rest of us' might not
upgrade 'quickly' or, particularly in replication environments, 'in the
right order' or 'with the expected upgrades to subsystem already in
place across replicas'... and most especially -- in the context of
active usage of the overall collection of masters and replicas. AFAIK,
there is no 'overall setting of maintenance mode that blocks all writes
across masters and slaves pending upgrades, waits for pending writes to
be flushed/completed, then upgrades'. Nor is there what you might call
an equivalent to 'mysqlcheck --all-databases --autorepair' for
freeipa. Freeipa depends on, what at least four programming
languages? And I'd hazard that the complexity of the ldap schema and
search expressions qualifies it as it's own language. Certainly the
interlocking dance of keys and certificates in DNSSEC is its own little
world. Given the background of readers here I think it would be no
challenge at all for others to give quite a further roster. So in some
ways it's a marvel it works at all, and even more so well on balance.
It's worth some thoughtful academic instrumentation I think.
First, I'm not arguing that this isn't a good idea. It is. I'm trying to
address some of your concerns without being too defensive.
Upgrade order shouldn't matter. We've seen cases where lots of LDAP
conflict entries are created when multiple servers are upgraded
simultaneously though I've never been able to reproduce this myself. But
with 389-ds now hiding conflict entries by default this shouldn't have
any operational impact in current versions.
During an upgrade the LDAP listeners are disabled for a good portion of
it so there are no outside writes. This also blocks replication.
You are correct that there is no easy way to tell whether the database
on server A matches the one on server B. A user contributed a script to
assist with this, checkipaconsistency[1], which compares the entry count
on a number of IPA objects between servers.
The number of programming languages goes to your original point: there
are a lot of dependencies, so where is the line? IPA itself uses mostly
javascript, C and python but also a lot of configuration file formats
and LDIF. It depends on what constitutes a "language" to be able to
define what anything uses.
The CA is mostly written in java. 389-ds is C but most of its utilities
have moved from perl to python. Kerberos is all C AFAIK.
Your overall point is correct: there are a lot of moving parts.
ipa-healthcheck was created to help minimize potential problems. It is
far from comprehensive but as we run into corner cases we try to add a
check for it.
rob
[1]
https://github.com/peterpakos/checkipaconsistency