Thanks Rob
Replies to questions interposed below.
On 10/17/23 11:53, Rob Crittenden wrote:
Harry G Coin via FreeIPA-users wrote:
> What's the 'current best practice' for what you might call a 'fully
> deployed' freeipa install (meaning one that uses DNSSEC and all the
> documented capability subsections)?
>
> From what I can tell, there are two approaches:
>
> Approach 1: Run it in a VM, then from time to time shut it down,
> snapshot the 'known good' image, then bring the VM back up'. When
> there's a problem that's not 'trivial', just revert to the 'known
good'
> snapshot and pay the price of re-entry of lost data.
>
> The reasoning here is: Freeipa is much too interwoven among all the
> sub-packages and dependency-hell, the time required to re-enter data
> necessary to bring a backup current is at least known, compared to the
> almost impossible-to-guess time and multi-package expertise required to
> diagnosis much less cure problems in anything like the time available in
> production environments.
Yes, this can help back out of failed upgrades, for example. Note that
this and any backup and restore procedure can wreak havoc on
replication. If you effectively take one server back in time then you
should re-init it from another server.
Could you flesh out 'the official' command sequence for that? For example:
1. Restore the VM image to a 'known good' image (known-good means
ipa-healthcheck reports no errors).
2. Boot the VM.
3. If there is no replication involved, re-enter data lost during the
downtime and you're done. Otherwise:
4. If there is replication involved, especially multi-master
replication: then run these commands in this order on these systems:
<Rob will fill this in? Please?>
I'm not sure why you bring up dependencies in the context of backup and
restore.
'security' and 'other' seemingly 'unrelated' 'upgrades'
to packages n
levels deep but whose previously un-noticed freeipa killing
race-condition or other bug manifests after the upgrade. I find myself
obligated to prevent any security or other change from happening until
the lowest possible usage times. For example today's 'random freeipa
bother' is:
Problem: cannot install both protobuf-3.5.0-15.el8.x86_64 and
protobuf-3.19.0-2.el8s.x86_64
- package liborc1-1.7.9-1.el8.x86_64 requires
libprotobuf.so.15()(64bit), but none of the providers can be installed
- cannot install the best update candidate for package
protobuf-3.19.0-2.el8s.x86_64
- cannot install the best update candidate for package
liborc1-1.7.5-1.el8s.x86_64
(try to add '--allowerasing' to command line to replace conflicting
packages or '--skip-broken' to skip uninstallable packages or '--nobest'
to use not only best candidate packages)
Also, there's a bit of a blurring of what comes to mind when the words
'multi-master' are adverstised, then the 'reality hair' (or at least
AFAIK) in the kasp db and perhaps other places (SOA), where one system
'is more master than others even with m-m agreements'. There is 'lore'
to do with 'manual steps' to 'move' 'special' files among what are
otherwise thought of as 'master' systems. This is not a complaint, I'm
glad there is at least some way, but it generates worry there are other
sub-systems that also require 'as yet unlearned lore' to actually
provide 'a whole system' from a backup.
> Approach 2: Use the ipa-backup, ipa-restore commands. Just keep
a
> fresh install of the unconfigured pacakges, do ipa-restore and it will
> get every single detail correct across all the available documented
> functionality, databases, including the dnssec ones and the other
> 'trust' addons, etc.
>
> The reasoning here is: It's better than approach 1 in that you don't
> have to take down the system even for a moment to get a 'known good'
> snapshot. However, you have to trust there is testing of 'everything
> the ipa-backup ipa-restore' process does actually does test the whole of
> the documented functionality.
>
> For production reasons we'd prefer approach 2, as taking the system down
> and bringing it back up as in approach 1 introduces a momentary loss of
> capability but has the benefit of 'total confidence'. But is it really
> 'the best approach' in the production world?
Backups are a good thing to have, but restoration should be due to
catastrophe. When doing a restore recommend deleting all other servers
and re-installing. You could conceivably re-init from the restored
server as well, but it really is a nuke from orbit operation.
This is really helpful, thank you. The point then of 'ipa-backup' is--
an offline file meant to be used if everything burns to the ground, not
to recover this or that failed master or replica.
In either situation there will be down time.
I agree. What I was really hoping for
was a way to not take a 'known
good' system down in order to get a 'known-everything-synced'
snapshot. While filesystems (like xfs, and others) have means to
'freeze' or 'snapshot', as far as I know freeipa doesn't have a way to
'flush write caches and pause pending writes across all subsystems'.
Some level of backup is essentially online with MMR in 389-ds. This
can't prevent undesired loss or modification of data but it otherwise
provides the uptime you seem to be looking for.
All about uptime! Thanks Rob
Harry
rob