Harry G Coin via FreeIPA-users wrote:
On 10/18/23 07:30, Florence Blanc-Renaud via FreeIPA-users wrote:
> Hi,
>
> this guide explains the possible strategies for disaster recovery:
>
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/...
>
> And that one how to recover:
>
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/...
>
> Hope this helps,
> flo
>
Thanks Flo!
Fundamentally freeipa is having a tension in that restoration from
system loss is almost never 'a committee or team effort' but a one
person task--- but freeipa now has such richness that few people have
the bandwidth to know all the necessary details for each of the distinct
complex subdomains freeipa integrates. A goal yet to be attained is a
freeipa 'best practice' advice that includes all the details necessary
to restore prior functionality ( snapshot the vm? ipa-backup? the CA
thing? Kasp.db? Subdomain account number ranges on replicas? Does
promoting a replica fix the SOA record? Is when 'a database that's too
old' a testable fact or like when your coffee temp is just right? ) .
Freeipa ought to have within itself such maps as necessary to itself
comprehend the intended role of the system being restored then maneuver
and marshal such file changes 'from what it used to be' to 'what it
needs to be' to make the system fill its intended role -- without
requiring the system-restore person to be expert in each of freeipa's
'deep dive expertise file areas'.
This was quite the dense set of questions :-). I'll try to cover everything.
As far as I'm aware, ipa-backup and ipa-restore save and restore all
things necessary to recover an IPA installation. This does hedge a bit.
If, for example, you back up a server that doesn't have optional
services installed, you could be in trouble. The CA, for example. The
user has some responsibility in identifying the most critical server(s)
to back up.
The DNSSEC keys are stored in LDAP so can be recovered, if missing,
AFAIK. DNS isn't my area. But we do back up the kasp.db in any case, if
DNS is configured.
I'm not sure what you mean regarding SOA. It is...ahem...problematic in
that all servers may have different ones, but this is due to not having
an atomic way to increment values over replication.
The rest of this post is detail on the above, and probably only
interesting for few.
In general all the above advice flo linked to includes most, but not all
required to actually accomplish restoration of the functional experience
prior to the loss. Perhaps the advice should be qualified to apply only
when the supplemental packages to do with trust and dns are not in
use. You might consider advising folks to use ipa-healthcheck for the
purpose of deciding 'how bad the current situation is' on this or that
system. Too is missing the advice relating to version management among
'masters' and 'replicas' (the order in which upgrades get installed
among systems) and whether backups made by 'the last one' remain usable
by 'the next one'.
ipa-healthcheck is not comprehensive and I don't want to leave the
impression that if it passes then your install is a-ok. It consists of
checks for common problems seen in deployments over the years but it
doesn't check every possible thing that can go wrong.
I do believe that it is documented that a restore can only be done on
the same version as the backup.
We do recommend keeping versions in sync.
The following is very valuable feedback on the documentation. The best
way to get our doc writers attention on this is to use the feedback
link. This will create internal tasks for them to tweak things that
doesn't involve an e-mail that can be missed or forgotten from the devs.
Notice that the detail in the first link provided there makes the
blanket statement affirming it is possible to replace a lost server by
creating a new replica. However, did the author take responsibility for
all of freeipa's parts when that was written? For example if the server
holding the file kasp.db fails, does that statement yet apply? Is there
a comprehensive list of 'files that are exceptions to the rule'
maintained by anyone (sub account number ranges on replicas)?
Most IPA data is stored in LDAP therefore a new replica should be able
to recover anything missing. The CA and KRA is the exception.
The next paragraph mentions VM snapshots, but leaves it to the reader to
guess when 'a too outdated database' (4.2 sec 6) is or isn't the case.
Too, does not mention that's not an answer if replication is active
(timing issues, the documentation is missing about whether and when to
walk users through the 're-init' Rob mentioned was required upstream).
Might freeipa's replication setup itself notice when 'systemctl stop
sssd;sss_cache -e;systemctl start sssd' is necessary and do that itself?
I believe the "too old" is restoring outside of the replication window
which will freak it out. Assuming I'm understanding it right, this is
what I meant by re-initing the world.
IPA should be independent from the SSSD cache.
The last paragraph mentions idM backups, detailed in section 5,
which
too I think might require a list of important files not backed up, and
might too include Rob's advise that such is only useful when 'everything
has burned to the ground and a fresh install on all systems is the next
step'.
Maybe in an appendix. It is an absolutely huge list. Much of what is
backed up consists of whole directories.
Section 3.2 might make more clear freeipa does not just have two states,
'master' and the term rhel uses 'regular replica' relevant to integrity,
but at least six major variants relevant to integrity: replica without
CA and not dnssec master, replica with ca but not dnssec master, etc.
.. and 'master with ca and dnssec db', etc. It's likely this list may
double in length in the case of KRA usage. And potentially need to be
entirely revised in light of sub-domain account number range
allocation. Also the freeipa instance mentioned in SOA records has a
status of its own that needs thought in the 'promoting replica to
master' advice.
What you're referring to is roles. It probably would be nice to add that
the backup server should consist of all roles used by the system. If it
were also the renewal master, crl generator, etc. then even better.
Does the advice of 'hidden replica' saving the day cover such files on
the same list as kasp.db? If it's a manual step required as presently
documented then ....?
As mentioned, kasp.db is backed up and I believe that ods-exporter will
recover the keys from Custodia.
Last: As freeipa is itself in the 'policy' services sector,
is not
what ansible provides something the internal maps freeipa maintains a
better place to capture and deploy that knowledge?
Perhaps. The backup/restore pre-dates Ansible. It would be an absolutely
gigantic yaml file though. I don't know that Ansible would do things any
better than the current python.
rob