On 2017/7/28 13:03, Petr Vobornik wrote:
On Fri, Jul 28, 2017 at 12:15 AM, Patrick Hemmer via FreeIPA-users
<freeipa-users@lists.fedorahosted.org> wrote:
I'm trying to setup a FreeIPA replica on 4.5.2 and the ipa-replica-install
script dies with:

        [27/40]: setting up initial replication
    Starting replication, please wait until this has completed.
    Update in progress, 14 seconds elapsed
    [ldap://fll2aipa01stg.ipa-stg.chewy.net:389] reports: Update failed!
Status: [-1  - LDAP error: Can't contact LDAP server]

        [error] RuntimeError: Failed to start replication


When I look in the /var/log/dirsrv/slapd-IPA-STG-CHEWY-NET/errors of the new
replica, the last few lines contains:
        [27/Jul/2017:17:54:36.501614930 -0400] NSMMReplicationPlugin -
agmt="cn=meTofll2aipa01stg.ipa-stg.chewy.net" (fll2aipa01stg:389): Unable to
acquire replica: permission denied. The bind dn "" does not have permission
to supply replication updates to the replica. Will retry later.
        [27/Jul/2017:17:54:42.511659900 -0400] NSMMReplicationPlugin -
agmt="cn=meTofll2aipa01stg.ipa-stg.chewy.net" (fll2aipa01stg:389): Unable to
acquire replica: permission denied. The bind dn "" does not have permission
to supply replication updates to the replica. Will retry later.
        [27/Jul/2017:17:54:54.517563545 -0400] NSMMReplicationPlugin -
agmt="cn=meTofll2aipa01stg.ipa-stg.chewy.net" (fll2aipa01stg:389): Unable to
acquire replica: permission denied. The bind dn "" does not have permission
to supply replication updates to the replica. Will retry later.
        [27/Jul/2017:17:55:18.527945464 -0400] NSMMReplicationPlugin -
agmt="cn=meTofll2aipa01stg.ipa-stg.chewy.net" (fll2aipa01stg:389): Unable to
acquire replica: permission denied. The bind dn "" does not have permission
to supply replication updates to the replica. Will retry later.
        [27/Jul/2017:17:56:06.546462326 -0400] NSMMReplicationPlugin -
agmt="cn=meTofll2aipa01stg.ipa-stg.chewy.net" (fll2aipa01stg:389): The
remote replica has a different database generation ID than the local
database.  You may have to reinitialize the remote replica, or the local
replica.


In the /var/log/dirsrv/slapd-IPA-STG-CHEWY-NET/errors of the original
master, the last few lines has:
        [27/Jul/2017:17:54:33.567167570 -0400] NSMMReplicationPlugin -
Warning: unable to acquire replica for total update, error: -1, retrying in
2 seconds.
        [27/Jul/2017:17:54:35.572200957 -0400] NSMMReplicationPlugin -
Warning: unable to acquire replica for total update, error: -1, retrying in
3 seconds.
        [27/Jul/2017:17:54:36.498618557 -0400] NSMMReplicationPlugin -
conn=115 op=6 replica="dc=ipa-stg,dc=chewy,dc=net": Unable to acquire
replica: error: permission denied
        [27/Jul/2017:17:54:38.579074442 -0400] NSMMReplicationPlugin -
Warning: unable to acquire replica for total update, error: -1, retrying in
4 seconds.
        [27/Jul/2017:17:54:42.504309388 -0400] NSMMReplicationPlugin -
conn=115 op=7 replica="dc=ipa-stg,dc=chewy,dc=net": Unable to acquire
replica: error: permission denied
        [27/Jul/2017:17:54:42.586071823 -0400] NSMMReplicationPlugin -
Warning: unable to acquire replica for total update, error: -1, retrying in
5 seconds.
        [27/Jul/2017:17:54:54.514797243 -0400] NSMMReplicationPlugin -
conn=115 op=9 replica="dc=ipa-stg,dc=chewy,dc=net": Unable to acquire
replica: error: permission denied
        [27/Jul/2017:17:55:18.521047403 -0400] NSMMReplicationPlugin -
conn=115 op=11 replica="dc=ipa-stg,dc=chewy,dc=net": Unable to acquire
replica: error: permission denied


The access log on the original master contains:
        [27/Jul/2017:17:31:48.338205279 -0400] conn=115 fd=70 slot=70
connection from 10.0.33.200 to 10.0.33.200
        [27/Jul/2017:17:31:48.338602001 -0400] conn=115 op=0 BIND
dn="cn=Directory Manager" method=128 version=2
        [27/Jul/2017:17:31:48.338684940 -0400] conn=115 op=0 RESULT err=0
tag=97 nentries=0 etime=0 dn="cn=directory manager"
        [27/Jul/2017:17:54:32.478121113 -0400] conn=115 fd=121 slot=121
connection from 10.0.33.201 to 10.0.33.200
        [27/Jul/2017:17:54:32.479047230 -0400] conn=115 op=0 BIND dn=""
method=sasl version=3 mech=GSSAPI
        [27/Jul/2017:17:54:32.482605087 -0400] conn=115 op=0 RESULT err=14
tag=97 nentries=0 etime=0, SASL bind in progress
        [27/Jul/2017:17:54:32.483393321 -0400] conn=115 op=1 BIND dn=""
method=sasl version=3 mech=GSSAPI
        [27/Jul/2017:17:54:32.484615090 -0400] conn=115 op=1 RESULT err=14
tag=97 nentries=0 etime=0, SASL bind in progress
        [27/Jul/2017:17:54:32.485067380 -0400] conn=115 op=2 BIND dn=""
method=sasl version=3 mech=GSSAPI
        [27/Jul/2017:17:54:32.486355861 -0400] conn=115 op=2 RESULT err=0
tag=97 nentries=0 etime=0
dn="krbprincipalname=ldap/fll2aipa02stg.ipa-stg.chewy.net@ipa-stg.chewy.net,cn=services,cn=accounts,dc=ipa-stg,dc=chewy,dc=net"
        [27/Jul/2017:17:54:32.486992403 -0400] conn=115 op=3 SRCH base=""
scope=0 filter="(objectClass=*)" attrs="supportedControl supportedExtension"
        [27/Jul/2017:17:54:32.489473132 -0400] conn=115 op=3 RESULT err=0
tag=101 nentries=1 etime=0
        [27/Jul/2017:17:54:32.489967733 -0400] conn=115 op=4 SRCH base=""
scope=0 filter="(objectClass=*)" attrs="supportedControl supportedExtension"
        [27/Jul/2017:17:54:32.492209604 -0400] conn=115 op=4 RESULT err=0
tag=101 nentries=1 etime=0
        [27/Jul/2017:17:54:32.492559529 -0400] conn=115 op=5 EXT
oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
        [27/Jul/2017:17:54:32.494124224 -0400] conn=115 op=5 RESULT err=0
tag=120 nentries=0 etime=0
        [27/Jul/2017:17:54:36.498506345 -0400] conn=115 op=6 EXT
oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
        [27/Jul/2017:17:54:36.500590218 -0400] conn=115 op=6 RESULT err=0
tag=120 nentries=0 etime=0
        [27/Jul/2017:17:54:42.504167583 -0400] conn=115 op=7 EXT
oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
        [27/Jul/2017:17:54:42.507097328 -0400] conn=115 op=7 RESULT err=0
tag=120 nentries=0 etime=0
        [27/Jul/2017:17:54:54.514671476 -0400] conn=115 op=9 EXT
oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
        [27/Jul/2017:17:54:54.516861209 -0400] conn=115 op=9 RESULT err=0
tag=120 nentries=0 etime=0
        [27/Jul/2017:17:55:18.520948176 -0400] conn=115 op=11 EXT
oid="2.16.840.1.113730.3.5.12" name="replication-multimaster-extop"
        [27/Jul/2017:17:55:18.523931139 -0400] conn=115 op=11 RESULT err=0
tag=120 nentries=0 etime=0


The command being used is:

        ipa-replica-install --principal admin -w XXXX -n ipa-stg.chewy.net
-r IPA-STG.CHEWY.NET --setup-dns --no-host-dns --setup-kra --mkhomedir
--forwarder 10.0.2.10 --forwarder 10.0.2.11 --no-ntp --no-dnssec-validation
-U --server=fll2aipa01stg.ipa-stg.chewy.net --setup-ca --skip-conncheck
What is the reason of using --skip-conncheck? Connection checks verify
that replica can contact master and vice-versa skipping it might hide
some env issues which might then fail replica installation. The error:
"Can't contact LDAP server" suggests that.
The conncheck is because there seems to be a bug where if our load balancer probes the temporary server it spins up, the temporary server gets screwed up. It's been a few months since I put that in place, but IIRC, the temp server hangs and doesn't shut down.

It's not the issue anyway as I just figured it out. It seems that for some reason the `ipa-replica-install` script isn't adding an A record for the host to the zone in IPA. It adds all the SSHFP records, just no A record. I added it manually and the installation completed successfully.




Any ideas what's wrong?

I've attached the output of ipa-replica-install as well as
/var/log/ipareplica-install.log. I can provide additional logs if necessary,
just let me know which ones.

-Patrick