So for some reason yesterday my replication broke. Checked out the logs and found
this:Mar 20 14:16:02 freeipa01 systemd: ipa-dnskeysyncd.service: main process exited,
code=exited, status=1/FAILUREMar 20 14:16:02 freeipa01 systemd: Unit
ipa-dnskeysyncd.service entered failed state.Mar 20 14:16:02 freeipa01 systemd:
ipa-dnskeysyncd.service failed.Mar 20 14:17:02 freeipa01 systemd: ipa-dnskeysyncd.service
holdoff time over, scheduling restart.Mar 20 14:17:02 freeipa01 systemd: Started IPA key
daemon.Mar 20 14:17:02 freeipa01 systemd: Starting IPA key daemon...Mar 20 14:17:05
freeipa01 ipa-dnskeysyncd: ipa : INFO LDAP bind...Mar 20 14:17:05 freeipa01
ipa-dnskeysyncd: ipa : INFO Commencing sync processMar 20 14:17:05 freeipa01
ipa-dnskeysyncd: ipa.ipaserver.dnssec.keysyncer.KeySyncer: INFO Initial LDAP dump is
done, sychronizing with ODS and BINDMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: Traceback
(most recent call last):Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: File
"/usr/libexec/ipa/ipa-dnskeysyncd", line 114, in <module>Mar 20 14:17:09
freeipa01 ipa-dnskeysyncd: while ldap_connection.syncrepl_poll(all=1,
msgid=ldap_search):Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd: File
"/usr/lib64/python2.7/site-packages/ldap/syncrepl.py", line 405, in
syncrepl_pollMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: self.syncrepl_refreshdone()Mar 20
14:17:09 freeipa01 ipa-dnskeysyncd: File
"/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 115, in
syncrepl_refreshdoneMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: self.hsm_replica_sync()Mar
20 14:17:09 freeipa01 ipa-dnskeysyncd: File
"/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 181, in
hsm_replica_syncMar 20 14:17:09 freeipa01 ipa-dnskeysyncd:
ipautil.run([paths.IPA_DNSKEYSYNCD_REPLICA])Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd:
File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 512, in
runMar 20 14:17:09 freeipa01 ipa-dnskeysyncd: raise CalledProcessError(p.returncode,
arg_string, str(output))Mar 20 14:17:09 freeipa01 ipa-dnskeysyncd:
subprocess.CalledProcessError: Command '/usr/libexec/ipa/ipa-dnskeysync-replica'
returned non-zero exit status 1Mar 20 14:17:09 freeipa01 systemd: ipa-dnskeysyncd.service:
main process exited, code=exited, status=1/FAILUREMar 20 14:17:09 freeipa01 systemd: Unit
ipa-dnskeysyncd.service entered failed state.Mar 20 14:17:09 freeipa01 systemd:
ipa-dnskeysyncd.service failed.Mar 20 14:17:39 freeipa01 su: (to root) gatewayblend on
pts/0Mar 20 14:17:39 freeipa01 dbus[742]: [system] Activating service
name='org.freedesktop.problems' (using servicehelper)Mar 20 14:17:39 freeipa01
dbus-daemon: dbus[742]: [system] Activating service
name='org.freedesktop.problems' (using servicehelper)Mar 20 14:17:39 freeipa01
dbus[742]: [system] Successfully activated service 'org.freedesktop.problems'Mar
20 14:17:39 freeipa01 dbus-daemon: dbus[742]: [system] Successfully activated service
'org.freedesktop.problems'Mar 20 14:18:09 freeipa01 systemd:
ipa-dnskeysyncd.service holdoff time over, scheduling restart.Mar 20 14:18:09 freeipa01
systemd: Started IPA key daemon.Mar 20 14:18:09 freeipa01 systemd: Starting IPA key
daemon...Mar 20 14:18:13 freeipa01 ipa-dnskeysyncd: ipa : INFO LDAP bind...Mar
20 14:18:13 freeipa01 ipa-dnskeysyncd: ipa : INFO Commencing sync processMar
20 14:18:13 freeipa01 ipa-dnskeysyncd: ipa.ipaserver.dnssec.keysyncer.KeySyncer: INFO
Initial LDAP dump is done, sychronizing with ODS and BINDMar 20 14:18:17 freeipa01
ipa-dnskeysyncd: Traceback (most recent call last):Mar 20 14:18:17 freeipa01
ipa-dnskeysyncd: File "/usr/libexec/ipa/ipa-dnskeysyncd", line 114, in
<module>Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: while
ldap_connection.syncrepl_poll(all=1, msgid=ldap_search):Mar 20 14:18:17 freeipa01
ipa-dnskeysyncd: File "/usr/lib64/python2.7/site-packages/ldap/syncrepl.py",
line 405, in syncrepl_pollMar 20 14:18:17 freeipa01 ipa-dnskeysyncd:
self.syncrepl_refreshdone()Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd: File
"/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 115, in
syncrepl_refreshdoneMar 20 14:18:17 freeipa01 ipa-dnskeysyncd: self.hsm_replica_sync()Mar
20 14:18:17 freeipa01 ipa-dnskeysyncd: File
"/usr/lib/python2.7/site-packages/ipaserver/dnssec/keysyncer.py", line 181, in
hsm_replica_syncMar 20 14:18:17 freeipa01 ipa-dnskeysyncd:
ipautil.run([paths.IPA_DNSKEYSYNCD_REPLICA])Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd:
File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 512, in
runMar 20 14:18:17 freeipa01 ipa-dnskeysyncd: raise CalledProcessError(p.returncode,
arg_string, str(output))Mar 20 14:18:17 freeipa01 ipa-dnskeysyncd:
subprocess.CalledProcessError: Command '/usr/libexec/ipa/ipa-dnskeysync-replica'
returned non-zero exit status 1Mar 20 14:18:17 freeipa01 systemd: ipa-dnskeysyncd.service:
main process exited, code=exited, status=1/FAILUREMar 20 14:18:17 freeipa01 systemd: Unit
ipa-dnskeysyncd.service entered failed state.Mar 20 14:18:17 freeipa01 systemd:
ipa-dnskeysyncd.service failed.
The service says its working just fine:[root@myserver ~]# sudo systemctl status ipa -l●
ipa.service - Identity, Policy, Audit Loaded: loaded
(/usr/lib/systemd/system/ipa.service; enabled; vendor preset: disabled) Active: active
(exited) since Tue 2018-03-20 19:08:00 UTC; 11min ago Process: 17633
ExecStop=/usr/sbin/ipactl stop (code=exited, status=0/SUCCESS) Process: 17725
ExecStart=/usr/sbin/ipactl start (code=exited, status=0/SUCCESS) Main PID: 17725
(code=exited, status=0/SUCCESS) CGroup: /system.slice/ipa.service
Mar 20 19:08:00
myserver.mydomain.com ipactl[17725]: Starting krb5kdc ServiceMar 20
19:08:00
myserver.mydomain.com ipactl[17725]: Starting kadmin ServiceMar 20 19:08:00
myserver.mydomain.com ipactl[17725]: Starting named ServiceMar 20 19:08:00
myserver.mydomain.com ipactl[17725]: Starting httpd ServiceMar 20 19:08:00
myserver.mydomain.com ipactl[17725]: Starting ipa-custodia ServiceMar 20 19:08:00
myserver.mydomain.com ipactl[17725]: Starting ntpd ServiceMar 20 19:08:00
myserver.mydomain.com ipactl[17725]: Starting pki-tomcatd ServiceMar 20 19:08:00
myserver.mydomain.com ipactl[17725]: Starting ipa-otpd ServiceMar 20 19:08:00
myserver.mydomain.com ipactl[17725]: Starting ipa-dnskeysyncd ServiceMar 20 19:08:00
myserver.mydomain.com systemd[1]: Started Identity, Policy, Audit.[root@myserver ~]#
Should I pull replication from another server or is there a simpler way to fix this?
Thanks,Andrew