Re: Cannot enroll a 4.9 client to 4.10 server fails with PrincipalName not found
by Rob Crittenden
Kroon PC, Peter via FreeIPA-users wrote:
> Hi all!
>
> I'm working on updating my freeipa server from rocky 8 to 9. I'm playing around with a virtual machines as playground server and client, since I'd rather not break my everything right away. As part of this, I first installed ipa-server version 4.10.2-8.el9_3 on the server. Then I did an ipa-restore with a backup from my production ipa server (rocky 8, 4.9.12-11.module+el8.9.0+1652+4ee71f6a), followed by an ipa-server-upgrade. All is well so far (I think).
I don't know how you achieved this. ipa-restore attempts to prevent
using restore as a backdoor upgrade mechanism.
> The client is running Debian bookworm with backports, where the latest ipa-client version is 4.9.11-1. Then, I went with the usual ipa-client-install --no-ntp, which fails with "Joining realm failed: Failed to parse result: PrincipalName not found." after retrieving the CA cert.
> The logs don't tell me much more, but the --debug flag does. It negotiates a JSON-RPC response, in which it says '{... "principal": "admin(a)EXAMPLE.COM", ...}'. I note that principal != PrincipalName. Also note, that on the server, the host /is/ added.
>
> So I guess my question is: how much version skew between server and client is supported?
Plenty. There isn't much to client enrollment and the API hasn't changed
significantly in a long time.
rob
1 month, 3 weeks
Cannot enroll a 4.9 client to 4.10 server fails with PrincipalName not found
by Kroon PC, Peter
Hi all!
I'm working on updating my freeipa server from rocky 8 to 9. I'm playing around with a virtual machines as playground server and client, since I'd rather not break my everything right away. As part of this, I first installed ipa-server version 4.10.2-8.el9_3 on the server. Then I did an ipa-restore with a backup from my production ipa server (rocky 8, 4.9.12-11.module+el8.9.0+1652+4ee71f6a), followed by an ipa-server-upgrade. All is well so far (I think).
The client is running Debian bookworm with backports, where the latest ipa-client version is 4.9.11-1. Then, I went with the usual ipa-client-install --no-ntp, which fails with "Joining realm failed: Failed to parse result: PrincipalName not found." after retrieving the CA cert.
The logs don't tell me much more, but the --debug flag does. It negotiates a JSON-RPC response, in which it says '{... "principal": "admin(a)EXAMPLE.COM", ...}'. I note that principal != PrincipalName. Also note, that on the server, the host /is/ added.
So I guess my question is: how much version skew between server and client is supported?
Kind regards, and many thanks for the excellent software and support,
Peter Kroon
1 month, 3 weeks
Failed FreeIPA replica installation
by D S
Hello, I've encountered several issues while installing freeipa replica.
I have freeipa 4.6.8 master and the replica I tried installing is 4.9.12.
During the replica install it seems that the replica is unable to get a CA cert from my master:
DEBUG Configuring Kerberos KDC (krb5kdc)
DEBUG [1/1]: installing X509 Certificate for PKINIT
DEBUG flushing ldapi://%2Frun%2Fslapd-[REDACTED].socket from SchemaCache
DEBUG retrieving schema for SchemaCache url=ldapi://%2Frun%2Fslapd-[REDACTED].socket conn=<ldap.ldapobject.SimpleLDAPObject object at 0x7f6ac1b7d6d8>
DEBUG certmonger request is in state 'NEWLY_ADDED_READING_KEYINFO'
DEBUG certmonger request is in state 'SUBMITTING'
DEBUG certmonger request is in state 'CA_UNREACHABLE'
DEBUG Cert request 20240312144851 failed: CA_UNREACHABLE (Server at https://[REDACTED]/ipa/json failed request, will retry: 903 (an internal error has occurred).)
DEBUG Giving up on cert request 20240312144851
WARNING PKINIT certificate request failed: Certificate issuance failed (CA_UNREACHABLE: Server at https://[REDACTED]/ipa/json failed request, will retry: 903 (an internal error has occurred).)
WARNING Failed to configure PKINIT
DEBUG Full PKINIT configuration did not succeed
DEBUG The setup will only install bits essential to the server functionality
DEBUG You can enable PKINIT after the setup completed using 'ipa-pkinit-manage'
DEBUG certmonger request is in state 'GENERATING_CSR'
DEBUG certmonger request is in state 'MONITORING'
DEBUG Cert request 20240312144853 was successful
DEBUG step duration: krb5kdc setup_pkinit 2.72 sec
DEBUG Done configuring Kerberos KDC (krb5kdc).
(However the the installation succeeds with INFO The ipa-replica-install command was successful)
On master in /var/log/httpd/error_log:
ipa: ERROR: non-public: AttributeError: 'ldap2' object has no attribute 'Object'
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ipaserver/rpcserver.py", line 369, in wsgi_execute
result = command(*args, **options)
File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 450, in __call__
return self.__do_call(*args, **options)
File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 478, in __do_call
ret = self.run(*args, **options)
File "/usr/lib/python2.7/site-packages/ipalib/frontend.py", line 800, in run
return self.execute(*args, **options)
File "/usr/lib/python2.7/site-packages/ipaserver/plugins/cert.py", line 863, in execute
ca_kdc_check(ldap, alt_principal.hostname)
File "/usr/lib/python2.7/site-packages/ipaserver/plugins/cert.py", line 301, in ca_kdc_check
master_dn = api_instance.Object.server.get_dn(unicode(hostname))
AttributeError: 'ldap2' object has no attribute 'Object'
ipa: INFO: [jsonserver_kerb] host/ipa-replica01.[REDACTED]@[REDACTED]: cert_request(u'MIID3DCCAsQCAQAwQjEZMBcGA1UECgwQRkxPUkEuTFRGUy5UT09MUzElMCMGA1UEAxMcaXBhLXNsYXZlMDEuZmxvcmEubHRmcy50b29sczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBANhiTu7x/3sRCBq0spuBM1M2dkAzfMFEtJRx/BUcDi7PrRhJBa+w2o+XfOjtP9YFYpnOYbqV6eNlslGuYbHYIZX5D4lawvcT+IfqBaeWAz3ZRyK+nFVotMm+BRHNeUFse3OUL2kGPLRgXZQ1wdzE+V17rTRtPHDV9cs0ERjTjMyntM24zYxWicrdsoFfwF/BJQRXVO+GZDdhJcmNvCcxywaUD8C/y9TYYLwhFdIRYFmOHvTHJwabPt5/QM8EkrAfkpCHypwf6oz7xvyiqnRFyx5okWexHqE1xXHbvMS+pjd2eNBl4n38H+hbdb83wJFS64M9zerlhU/u5h4cbY0WdIJtOz2dP7A3NvQL0X5oF10ivWYw5+Rbar8h6nCr07ApVku6m5kXePp6rK3c0f+IORxflJFEdRGBG7zBbE1Dt4Kf5Q+uTwiEjN2wEre1s6CFg8oTipncOwHkyettAplFisitfdxs4HlEZpsN3kxh6NQDFAhx4JE1WNGMPVZNKuBz6tHhMwgpDDXS16UjIXZqYllUDnBaf5GdawCgWr2wbXEaUflgOWje/QyvZkYVsHZzUFw9Bqh8B7jxw7h/4KAM43XBDWN9P6J2+gqp2M3SQ=', profile_id=u'KDCs_PKINIT_Certs', principal=u'krbtgt/[REDACTED]@[REDACTED]', add=True): InternalError
That's the issue number one. Number two is that I can't login into web UI of my replica - it gives me "Login failed due to an unknown reason" error. From /var/log/httpd/error_log:
[auth_gssapi:error] GSS ERROR gss_acquire_cred[_from]() failed to get server creds: [Unspecified GSS failure. Minor code may provide more information ( SPNEGO cannot find mechanisms to negotiate)]
[wsgi:error] ipa: INFO: 401 Unauthorized: No session cookie found
Finally, my third issue is that I can't remove replica from my master. ipa-replica-manage del --force --cleanup fails with:
Traceback (most recent call last):
File "/usr/sbin/ipa-replica-manage", line 1624, in <module>
main(options, args)
File "/usr/sbin/ipa-replica-manage", line 1524, in main
api.finalize()
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 740, in finalize
self.__do_if_not_done('load_plugins')
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 431, in __do_if_not_done
getattr(self, name)()
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 620, in load_plugins
self.add_package(package)
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 658, in add_package
self.add_module(module)
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 675, in add_module
self.add_plugin(**kwargs)
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 711, in add_plugin
plugin=plugin,
PluginOverrideError: unexpected override of BaseCertObject.certreq with <class 'ipaserver.plugins.cert.certreq'>
Unexpected error: unexpected override of BaseCertObject.certreq with <class 'ipaserver.plugins.cert.certreq'>
1 month, 3 weeks
Re: ipa-setup-ca
by Pagan, Omar
Not sure if they did use the external CA. How can I check?
//omar
Omar Pagan, CISSP
AAP Sr. DevOps/SysAdmin
[cid:image001.png@01DA76C3.BAE28A80]
1 month, 3 weeks
ipa: ERROR: No valid Negotiate header - from/in container replica
by lejeczek
Hi guys.
This is my first trial/test of replicas in container - here
I added a replica to already existing, bare-metal IPA
domain, which otherwise works a okey - so numerous issues
are possible.
In container, only in this replica, I get:
bash-5.1# ipa dnszone-find
ipa: ERROR: No valid Negotiate header in server response
What that is, might be, a symptom of? Where to go with
troubleshooting?
All thoughts share are much appreciated.
many thanks, L.
1 month, 3 weeks
"Internal server error 'Link'" from ClonesConnectivyAndDataCheck health check on RHEL 8 when talking to RHEL 9 server
by Sam Morris
I tracked down the source of the myserious "Internal server error
'Link'" message when running this health check. It's caused by having a
mixture of both RHEL 8 and RHEL 9 servers.
The error message in context:
# ipa-healthcheck --source=pki.server.healthcheck.clones.connectivity_and_data --check=ClonesConnectivyAndDataCheck --output-type=json --debug
[...]
stderr=
Calling check <pki.server.healthcheck.clones.connectivity_and_data.ClonesConnectivyAndDataCheck object at 0x7f8c87e8cf60>
[...]
About to check the subsystem clones
Starting new HTTPS connection (1): ipa3.example.com:443
https://ipa3.example.com:443 "POST /ca/rest/certs/search?size=3 HTTP/1.1" 200 431
Cert data successfully obtained from clone.
Starting new HTTPS connection (1): ipa5.example.com:443
https://ipa5.example.com:443 "POST /ca/rest/certs/search?size=3 HTTP/1.1" 200 431
Cert data successfully obtained from clone.
Starting new HTTPS connection (1): ipa6.example.com:443
https://ipa6.example.com:443 "POST /ca/rest/certs/search?size=3 HTTP/1.1" 200 317
Internal server error 'Link'
[...]
[
{
"source": "pki.server.healthcheck.clones.connectivity_and_data",
"check": "ClonesConnectivyAndDataCheck",
"result": "ERROR",
"uuid": "f672f185-6251-47e9-a772-8f9796a34986",
"when": "20240312021736Z",
"duration": "0.521790",
"kw": {
"status": "ERROR: pki-tomcat : Internal error testing CA clone. Host: ipa6.example.com Port: 443"
}
}
]
I edited ClonesConnectivyAndDataCheck.check_ca_clones to call
logger.exception in its BaseException handler instead of just
logger.error. This logs the traceback of the original exception:
Internal server error 'Link'
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/pki/server/healthcheck/clones/connectivity_and_data.py", line 35, in check_ca_clones
certs = cert_client.list_certs(size=3)
File "/usr/lib/python3.6/site-packages/pki/__init__.py", line 431, in handler
return fn_call(inst, *args, **kwargs)
File "/usr/lib/python3.6/site-packages/pki/cert.py", line 674, in list_certs
return CertDataInfoCollection.from_json(response.json())
File "/usr/lib/python3.6/site-packages/pki/cert.py", line 179, in from_json
links = json_value['Link']
KeyError: 'Link'
I guess the newer version of Dogtag in RHEL 9 doesn't include this
"Link" attribute, but pki.cert:CertDataInfoCollection.from_json in RHEL
8 expects it to be present.
--
Sam Morris <https://robots.org.uk/>
CAAA AA1A CA69 A83A 892B 1855 D20B 4202 5CDA 27B9
1 month, 3 weeks
ipa-getcert request results in CA_REJECTED, on an enrolled host
by Bo Lind
I'm having a weird one. This has worked well on a number of other, identical hosts, but one is repeatedly giving me trouble:
root@naughtyhost:~# ipa-getcert request -f /etc/pki/tls/certs/xrdp.pem -k /etc/pki/tls/private/xrdp.key -r -w -v
New signing request "20240312125107" added.
State NEWLY_ADDED_READING_KEYINFO, stuck: no.
State SUBMITTING, stuck: no.
State CA_REJECTED, stuck: yes.
root@naughtyhost:~# ipa-getcert list
Number of certificates and requests being tracked: 1.
Request ID '20240312125107':
status: CA_REJECTED
ca-error: Server at https://idm0.example.local/ipa/json denied our request, giving up: 2100 (Insufficient access: SASL(-1): generic failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Credential cache is empty)).
stuck: yes
key pair storage: type=FILE,location='/etc/pki/tls/private/xrdp.key'
certificate: type=FILE,location='/etc/pki/tls/certs/xrdp.pem'
CA: IPA
issuer:
subject:
issued: unknown
expires: unknown
pre-save command:
post-save command:
track: yes
auto-renew: yes
I've tried looking in the logfiles on idm0, but couldn't really find anything useful.
1 month, 3 weeks
Number of concurrent connections are decreased by replication.
by Jaehwan Kim
Hello.
I recently encoutered a poblem that nubmer of concurrent connecitons are decreased in FreeIPA servers.
[Architecutre - replication topology]
My replication topology which is circular (ring-shaped), consists of 13 FreeIPA servers.
These 13 servers are grouped as 3 clusters, of which members are 5, 4, 4 respectively.
NLBs(network load balancers) to share request from clients for ipa login, kerberos authenticaion, ldap connections, are assinged to each cluster.
Therefore 3 NLBs have 5, 4, 4 FreeIPA servers as their nlb backend pool, repectively.
This architecture has been worked successfully for 2 years, but recently I encountered a problem that 867 host_add per hours to one cluster results in "# of concurrent connections decrement" for all clusters.
Command to get # of concurrent connections is
dsconf -D "cn=Directory Manager" ldap://server.example.com monitor server | grep currentconnections:
About 2K connections are observed for each servers, by this command.
I also found that if servers which replication info isn't transfered to, this symptom doesn't happen, even though those are in the same replication topology ring.
Hence, I guess that "# of concurrent connections decrement" symptom is related to replcation.
I tried to tune the parameters like
dtablesize = 65535,
repl-release-timeout = 120,
nnsslapd-threadnumber = authomatic thread tuning,
db and entry cache auto-sizing (nsslapd-cache-autosize = 80,
with failure.
I want to ask help to solve this symptom, if posible.
Thank you.
JHK
1 month, 3 weeks
pki-tomcatd not starting
by Omar Pagan
Hello,
I came back from vacation and noticed that the pki-tomcatd was not running. All other services are running fine, I can kinit admin and search for users, I can also log into the UI and see everything. When I try to start the service I see the following errors:
Mar 11 20:44:44 ldap01.app.uaap.maxar.com ipa-pki-wait-running[7903]: ipa-pki-wait-running: Request failed unexpectedly, 404 Client Error: for url: http://ldap01.app.uaap.maxar.com:8080/ca/admin/ca/getStat>
Mar 11 20:44:44 ldap01.app.uaap.maxar.com systemd[1]: pki-tomcatd(a)pki-tomcat.service: Start-post operation timed out. Stopping.
I have checked all the certs and everything is in order:
$ getcert list | grep expire
expires: 2025-01-22 14:07:35 UTC
expires: 2025-01-22 14:06:46 UTC
expires: 2025-01-22 14:06:45 UTC
expires: 2025-01-22 14:06:45 UTC
expires: 2043-02-02 14:06:44 UTC
expires: 2025-01-22 14:06:45 UTC
expires: 2025-02-02 14:08:10 UTC
I also have checked this:
$ klist -ekt /etc/dirsrv/ds.keytab
Keytab name: FILE:/etc/dirsrv/ds.keytab
KVNO Timestamp Principal
---- ------------------- ------------------------------------------------------
2 02/02/2023 14:06:06 ldap/ldap01.app.uaap.maxar.com(a)APP.UAAP.MAXAR.COM (aes256-cts-hmac-sha1-96)
2 02/02/2023 14:06:06 ldap/ldap01.app.uaap.maxar.com(a)APP.UAAP.MAXAR.COM (aes128-cts-hmac-sha1-96)
2 02/02/2023 14:06:06 ldap/ldap01.app.uaap.maxar.com(a)APP.UAAP.MAXAR.COM (aes128-cts-hmac-sha256-128)
2 02/02/2023 14:06:06 ldap/ldap01.app.uaap.maxar.com(a)APP.UAAP.MAXAR.COM (aes256-cts-hmac-sha384-192)
2 02/02/2023 14:06:06 ldap/ldap01.app.uaap.maxar.com(a)APP.UAAP.MAXAR.COM (camellia128-cts-cmac)
2 02/02/2023 14:06:06 ldap/ldap01.app.uaap.maxar.com(a)APP.UAAP.MAXAR.COM (camellia256-cts-cmac)
not sure if that's correct or not. Please help, I don't see why pki-tomcatd would just die on me for no reason. I haven't run any updates / upgrades on the system and it was working fine before I left. Thanks
1 month, 3 weeks
ipa-healthcheck timeout too short for ClonesConnectivyAndDataCheck
by Sam Morris
All three of my IPA servers have this health check failing:
[root@ipa3 ~]# ipa-healthcheck --source pki.server.healthcheck.clones.connectivity_and_data --check ClonesConnectivyAndDataCheck --output-type=human
Internal server error 'Link'
ERROR: pki.server.healthcheck.clones.connectivity_and_data.ClonesConnectivyAndDataCheck
[root@ipa5 ~]# ipa-healthcheck --source=pki.server.healthcheck.clones.connectivity_and_data --check=ClonesConnectivyAndDataCheck --output-type=human
Internal server error 'Link'
Unable to reach KRA at https://ipa5.ipa.robots.org.uk:443: Request timed out
ERROR: pki.server.healthcheck.clones.connectivity_and_data.ClonesConnectivyAndDataCheck
ERROR: pki.server.healthcheck.clones.connectivity_and_data.ClonesConnectivyAndDataCheck
[root@ipa6 ~]# ipa-healthcheck --source=pki.server.healthcheck.clones.connectivity_and_data --check=ClonesConnectivyAndDataCheck --output-type=human
Unable to reach KRA at https://ipa6.ipa.robots.org.uk:443: Request timed out
ERROR: pki.server.healthcheck.clones.connectivity_and_data.ClonesConnectivyAndDataCheck
[root@ipa6 ~]# java.io.FileNotFoundException: /tmp/tmpc4h9ypzo/password.txt (No such file or directory)
at java.base/java.io.FileInputStream.open0(Native Method)
at java.base/java.io.FileInputStream.open(FileInputStream.java:216)
at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
at java.base/java.io.FileInputStream.<init>(FileInputStream.java:111)
at java.base/java.io.FileReader.<init>(FileReader.java:60)
at com.netscape.cmstools.cli.MainCLI.loadPassword(MainCLI.java:241)
at com.netscape.cmstools.cli.MainCLI.parseOptions(MainCLI.java:416)
at com.netscape.cmstools.cli.MainCLI.execute(MainCLI.java:647)
at com.netscape.cmstools.cli.MainCLI.main(MainCLI.java:698)
The problem is the timeout mechanism implemented in
ipahealthcheck.core.core:run_plugin; it turns out that 10 seconds is not
sufficient in my case.
Plugin <pki.server.healthcheck.clones.connectivity_and_data.ClonesConnectivyAndDataCheck object at 0x7fe02c4992b0> elapsed time: 0:00:16.857277 sec
I've bumped the timeout to 30 seconds in
/etc/ipahealthcheck/ipahealthcheck.conf and all is well.
I wonder if others are running into this problem as well? If so it may
be worth making this change by default in ipa-healtcheck. Or maybe my
IPA servers are slower than everyone else's... :)
--
Sam Morris <https://robots.org.uk/>
PGP: rsa4096/CAAA AA1A CA69 A83A 892B 1855 D20B 4202 5CDA 27B9
1 month, 3 weeks