Hi Florence,
Thank you for taking the time to respond.
We have a number of replicas in various campus locations around our region, and our state
and country is currently locked-down with COVID restrictions, and it would take days to
get to the replicas even without restrictions. We have had wide-scale WAN outages over
weeks for this system.
To describe our problem in a simplified manner;
We had the IPA replicas ‘Location1R1’ and ‘Location2R1’ talking happily when the WAN
between them went away, while it was down we added ‘Location2R2’ to ‘Location2R1’, and
manually added some user-ranges for accounts at Location2. We also added a number of
ipa-clients at Location2. When the WAN came back, ‘Location1R1’ and ‘Location2R1’ stopped
sharing data because the CSN updates were dropped weeks ago. We re-initialized
‘Location2R1’ from ‘Location1R1’, however that has resulted in isolation of ‘Location2R2’.
Most logins at Location2 fail, depending on where the DNS round-robin points SSSD at the
time of login.
We have multiple locations, each with a flavor of this issue. No replica has a complete
picture of all replication agreements, host accounts and user accounts. Though we have had
some success with continually re-initializing replicas.
It was my hope, once the WAN was stabilized to remove and re ipa-server-install at our
first server, then remove and re-run ipa-client-install then ipa-replica-install at each
secondary server. I believe this would re-constitute the system, effectively by
re-installation of that specific service. And yes, this would all be done via SSH.
However, we meet the obstacles described in my initial email when performing the install a
second time.
Sincerely, and with many thanks in advance,
Rob
From: Florence Renaud via FreeIPA-users <freeipa-users(a)lists.fedorahosted.org>
Sent: Wednesday, 2 June 2021 2:16 AM
To: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
Cc: Mattson, Robert (FP) <Robert.Mattson(a)L3Harris.com>; Florence Renaud
<flo(a)redhat.com>
Subject: [EXTERNAL] [Freeipa-users] Re: IPA Reinstall
Hi,
the recommended way to uninstall a replica and reinstall it is described in the doc:
1. Uninstall the replica
(
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/...)
with ipa server-del and ipa-server-install --uninstall
2. re-install the replica as if it was a new one:
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/...
Was there any reason to backup files and restore them? The replica installation should
re-create everything.
flo
On Wed, May 26, 2021 at 7:32 AM Robert.Mattson--- via FreeIPA-users
<freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org>>
wrote:
Dear Community,
I'd like to uninstall and reinstall IPA from a CentOS box because its easier than
reinstalling the OS completely.
We have a number of replicas, and this host is installed using ipa-client-install and then
ipa-replica-install.
To remove it, I backup some data like /var/kerberos/krb5kdc/{cacert.pem,kd*} and
/etc/httpd/conf/password.conf
and then run '/usr/sbin/ipa-server-install --uninstall -U
--ignore-topology-disconnect'.
I then sed '/Environment=K/d', '/ExecStartPre/d',
'/ExecStopPost/d' /etc/systemd/system/httpd.service
I recreate the host-account on another replica using ipa host-add, then ipa
hostgroup-add-member.
On the now-removed host, I do some housekeeping like restoring the backed up files and
then I run;
/usr/sbin/ipa-client-install \
--password=${otp} \
--mkhomedir \
--no-ntp \
--unattended \
--domain=realm.name<http://realm.name> \
--realm=REALM.NAME<http://REALM.NAME> \
--ca-cert-file=/etc/pki/ca-trust/source/ca.crt
then
/usr/sbin/ipa-replica-install \
--dirsrv-cert-file=/etc/pki/tls/private/ipa.pkcs12 \
--http-cert-file=/etc/pki/tls/private/ipa.pkcs12 \
--dirsrv-pin=pwd \
--http-pin=pwd \
--unattended \
--no-pkinit \
--no-ntp
I seem to get the following keytab request problem followed by dirsrv failure. from
ipa-replica-install (4.6.4-10.el7.centos.3.x86_64). If I upgrade to
4.6.8-5.el7.centos.4.noarch.rpm, I get the same problem.[1]
On serverb, the host which receives the binding request for the reinstall, I get
permission denied the bind dn “” does not have permission in dirsrv error log…?
Does anyone have any thoughts,
Cheers and many thanks in advance,
Rob
[1]
2021-05-26T02:50:56Z DEBUG Backing up system configuration file
'/etc/httpd/conf.d/ipa.conf'
2021-05-26T02:50:56Z DEBUG -> Not backing up - '/etc/httpd/conf.d/ipa.conf'
doesn't exist
2021-05-26T02:50:56Z DEBUG Backing up system configuration file
'/etc/httpd/conf.d/ipa-rewrite.conf'
2021-05-26T02:50:56Z DEBUG -> Not backing up -
'/etc/httpd/conf.d/ipa-rewrite.conf' doesn't exist
2021-05-26T02:50:56Z DEBUG duration: 0 seconds
2021-05-26T02:50:56Z DEBUG [10/21]: setting up httpd keytab
2021-05-26T02:50:56Z DEBUG raw:
service_add(u'HTTP/servera.system@REALM.NAME<mailto:servera.system@REALM.NAME>',
force=True, version=u'2.230')
2021-05-26T02:50:56Z DEBUG
service_add(ipapython.kerberos.Principal('HTTP/servera.system@REALM.NAME<mailto:servera.system@REALM.NAME>'),
force=True, all=False, raw=False, version=u'2.230', no_members=False)
2021-05-26T02:50:56Z DEBUG flushing ldapi://%2Fvar%2Frun%2Fslapd-REALM-NAME.socket from
SchemaCache
2021-05-26T02:50:56Z DEBUG retrieving schema for SchemaCache
url=ldapi://%2Fvar%2Frun%2Fslapd-REALM-NAME.socket
conn=<ldap.ldapobject.SimpleLDAPObject instance at 0x7f0d29f50368>
2021-05-26T02:50:57Z DEBUG raw: host_show(u'servera.system',
version=u'2.230')
2021-05-26T02:50:57Z DEBUG host_show(u'servera.system', rights=False, all=False,
raw=False, version=u'2.230', no_members=False)
2021-05-26T02:50:57Z DEBUG Backing up system configuration file
'/var/lib/ipa/gssproxy/http.keytab'
2021-05-26T02:50:57Z DEBUG -> Not backing up -
'/var/lib/ipa/gssproxy/http.keytab' doesn't exist
2021-05-26T02:50:57Z DEBUG Starting external process
2021-05-26T02:50:57Z DEBUG args=/usr/sbin/ipa-getkeytab -k
/var/lib/ipa/gssproxy/http.keytab -p
HTTP/servera.system@REALM.NAME<mailto:HTTP/servera.system@REALM.NAME> -H
ldapi://%2Fvar%2Frun%2Fslapd-REALM-NAME.socket -Y EXTERNAL
2021-05-26T02:50:57Z DEBUG Process finished, return code=9
2021-05-26T02:50:57Z DEBUG stdout=
2021-05-26T02:50:57Z DEBUG stderr=Failed to parse result: unsupported extended operation
Retrying with pre-4.0 keytab retrieval method...
Failed to parse result: unsupported extended operation
Failed to get keytab!
Failed to get keytab
2021-05-26T02:50:57Z DEBUG Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
570, in start_creation
run_step(full_msg, method)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
560, in run_step
method()
File "/usr/lib/python2.7/site-packages/ipaserver/install/httpinstance.py",
line 637, in request_service_keytab
super(HTTPInstance, self).request_service_keytab()
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
742, in request_service_keytab
self.run_getkeytab(self.api.env.ldap_uri, self.keytab, self.principal)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
732, in run_getkeytab
ipautil.run(args, nolog=nolog)
File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 562, in
run
raise CalledProcessError(p.returncode, arg_string, str(output))
CalledProcessError: Command '/usr/sbin/ipa-getkeytab -k
/var/lib/ipa/gssproxy/http.keytab -p
HTTP/servera.system@REALM.NAME<mailto:HTTP/servera.system@REALM.NAME> -H
ldapi://%2Fvar%2Frun%2Fslapd-REALM-NAME.socket -Y EXTERNAL' returned non-zero exit
status 9
2021-05-26T02:50:57Z DEBUG [error] CalledProcessError: Command
'/usr/sbin/ipa-getkeytab -k /var/lib/ipa/gssproxy/http.keytab -p
HTTP/servera.system@REALM.NAME<mailto:HTTP/servera.system@REALM.NAME> -H
ldapi://%2Fvar%2Frun%2Fslapd-REALM-NAME.socket -Y EXTERNAL' returned non-zero exit
status 9
2021-05-26T02:50:57Z DEBUG File
"/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 178, in
execute
return_value = self.run()
File "/usr/lib/python2.7/site-packages/ipapython/install/cli.py", line 319, in
run
return cfgr.run()
File "/usr/lib/python2.7/site-packages/ipapython/install/core.py", line 364,
in run
return self.execute()
exc_handler(exc_info)
<snip />
File "/usr/lib/python2.7/site-packages/ipapython/install/common.py", line 65,
in _install
for unused in self._installer(self.parent):
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/__init__.py",
line 622, in main
replica_install(self)
File
"/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py",
line 406, in decorated
func(installer)
File
"/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py",
line 1487, in install
fstore=fstore)
File
"/usr/lib/python2.7/site-packages/ipaserver/install/server/replicainstall.py",
line 173, in install_http
subject_base=config.subject_base, master_fqdn=config.master_host_name)
File "/usr/lib/python2.7/site-packages/ipaserver/install/httpinstance.py",
line 188, in create_instance
self.start_creation()
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
570, in start_creation
run_step(full_msg, method)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
560, in run_step
method()
File "/usr/lib/python2.7/site-packages/ipaserver/install/httpinstance.py",
line 637, in request_service_keytab
super(HTTPInstance, self).request_service_keytab()
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
742, in request_service_keytab
self.run_getkeytab(self.api.env.ldap_uri, self.keytab, self.principal)
File "/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line
732, in run_getkeytab
ipautil.run(args, nolog=nolog)
File "/usr/lib/python2.7/site-packages/ipapython/ipautil.py", line 562, in
run
raise CalledProcessError(p.returncode, arg_string, str(output))
2021-05-26T02:50:57Z DEBUG The ipa-replica-install command failed, exception:
CalledProcessError: Command '/usr/sbin/ipa-getkeytab -k
/var/lib/ipa/gssproxy/http.keytab -p
HTTP/servera.system@REALM.NAME<mailto:HTTP/servera.system@REALM.NAME> -H
ldapi://%2Fvar%2Frun%2Fslapd-REALM-NAME.socket -Y EXTERNAL' returned non-zero exit
status 9
2021-05-26T02:50:57Z ERROR Command '/usr/sbin/ipa-getkeytab -k
/var/lib/ipa/gssproxy/http.keytab -p
HTTP/servera.system@REALM.NAME<mailto:HTTP/servera.system@REALM.NAME> -H
ldapi://%2Fvar%2Frun%2Fslapd-REALM-NAME.socket -Y EXTERNAL' returned non-zero exit
status 9
2021-05-26T02:50:57Z ERROR The ipa-replica-install command failed. See
/var/log/ipareplica-install.log for more information
[2]
[26/May/2021:12:50:47.240285166 +1000] - WARN - NSMMReplicationPlugin - repl5_inc_run -
agmt="cn=meToservera.system" (servera:389): The remote replica has a different
database generation ID than the local database. You may have to reinitialize the remote
replica, or the local replica.
[26/May/2021:12:50:47.858057379 +1000] - INFO - NSMMReplicationPlugin - repl5_tot_run -
Beginning total update of replica "agmt="cn=meToservera.system"
(servera:389)".
[26/May/2021:12:50:50.679652092 +1000] - INFO - NSMMReplicationPlugin - repl5_tot_run -
Finished total update of replica "agmt="cn=meToservera.system"
(servera:389)". Sent 582 entries.
[26/May/2021:12:50:52.158394667 +1000] - ERR - NSMMReplicationPlugin - acquire_replica -
agmt="cn=meToservera.system" (servera:389): Unable to acquire replica:
permission denied. The bind dn "" does not have permission to supply replication
updates to the replica. Will retry later.
[26/May/2021:12:50:55.079367688 +1000] - ERR - NSMMReplicationPlugin - acquire_replica -
agmt="cn=meToservera.system" (servera:389): Unable to acquire replica:
permission denied. The bind dn "" does not have permission to supply replication
updates to the replica. Will retry later.
[26/May/2021:12:50:58.084381230 +1000] - ERR - NSMMReplicationPlugin - acquire_replica -
agmt="cn=meToservera.system" (servera:389): Unable to acquire replica:
permission denied. The bind dn "" does not have permission to supply replication
updates to the replica. Will retry later.
[26/May/2021:12:51:01.092727541 +1000] - ERR - NSMMReplicationPlugin - acquire_replica -
agmt="cn=meToservera.system" (servera:389): Unable to acquire replica:
permission denied. The bind dn "" does not have permission to supply replication
updates to the replica. Will retry later.
CONFIDENTIALITY NOTICE: This email and any attachments are for the sole use of the
intended recipient and may contain material that is proprietary, confidential, privileged
or otherwise legally protected or restricted under applicable government laws. Any review,
disclosure, distributing or other use without expressed permission of the sender is
strictly prohibited. If you are not the intended recipient, please contact the sender and
delete all copies without reading, printing, or saving.
_______________________________________________
FreeIPA-users mailing list --
freeipa-users@lists.fedorahosted.org<mailto:freeipa-users@lists.fedorahosted.org>
To unsubscribe send an email to
freeipa-users-leave@lists.fedorahosted.org<mailto:freeipa-users-leave@lists.fedorahosted.org>
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure