Hello Everyone,

Over the weekend we lost a replica during an upgrade and had to rebuild it. The OS (CentOS 7.6) was reinstalled from scratch, the host then added to the IPA domain, and then turned into a replica.

Sequence of events:
1) ns01 upgraded from FreeIPA 4.4.0-14 to 4.6.4-10
2) ns02 corrupted during upgrade process
3) on ns01, "ipa-replica-manage del ns02" ran.
4) ns02 rebuilt from scratch with latest CentOS 7.6 packages.
5) ns02 added to IPA domain
6) ns02 added as replica

The process went well, no errors during the "ipa-replica-install --setup-ca --setup-kra --setup-dns --forwarder=x.x.x.x" process.

However, on ns01, I'm getting the following message in /var/log/messages:

Apr  8 13:54:36 ns01 ns-slapd: [08/Apr/2019:13:54:36.294135188 +0000] - ERR - slapi_ldap_bind - Error: could not bind id [cn=Replication Manager cloneAgreement1-ns02.dev.example.net-pki-tomcat,ou=csusers,cn=config] authentication mechanism [SIMPLE]: error 32 (No such object)
...
Apr  8 13:59:36 ns01 ns-slapd: [08/Apr/2019:13:59:36.547881587 +0000] - ERR - slapi_ldap_bind - Error: could not bind id [cn=Replication Manager cloneAgreement1-ns02.dev.example.net-pki-tomcat,ou=csusers,cn=config] authentication mechanism [SIMPLE]: error 32 (No such object)

If I run a search in ns01's LDAP I get this result:

[root@ns01 ~]# ldapsearch -xLLL -h ns01.dev.example.net -D "cn=directory manager" -W -b "ou=csusers,cn=config"
Enter LDAP Password:
dn: ou=csusers,cn=config
objectClass: top
objectClass: organizationalUnit
ou: csusers

dn: cn=Replication Manager masterAgreement1-ns02.dev.example.net-pki-tomca
 t,ou=csusers,cn=config
cn: Replication Manager masterAgreement1-ns02.dev.example.net-pki-tomcat
objectClass: top
objectClass: person
sn: manager
userPassword:: redacted!!!

So there's a "masterAgreement1" but no "cloneAgreement1".

Is that something hanging around from the previous replica agreement? If so, how do I fix whatever is running that query every 5 minutes?

Or is it indicative of something else that is wrong? I ran the tool from https://github.com/peterpakos/checkipaconsistency and it reports everything is fine (except that ns01 has a dangling AD trust that was supposed to be removed, but that's for another post I guess).

How do I identify the process that is running that query that causes the error message in /var/log/messages?

Many, many thanks,

Anthony Clark