Thanks Thierry,
IPA backup had failed much before, unfortunate not able to restore those logs. But I did
some progress, by trying to restore different daily backups. And I found one, that was
restored "successfully", and 389ds has started after that.
But new problem is that replica from another master permanently fails ("Error (16) :
Incremental update connection error. Backing off, will retry update later.") and my
work around is having this cron job:
ipa-replica-manage -v re-initialize --from=<another-master>
The result of this is:
1. DB can reinit with latest data, this is good
2. Any action (add user/host) fails with errors like "IPA Error 4203:
DatabaseError". Fine, at least I have that server are read-only in my infrastructure.
On the top of this in Oct 2018, certmonger had failed to renew certs, and
"back-in" time proposed resolution didn't help. This is obstacle to add new
server, I tried promoting client into server, but replica prepare fails (domain level 0),
so I am not sure what we can do!?
Maybe built new IPA (latest version), same domain name and realm, and manually migrate
from crippled infrastructure.