Our freeipa certificates need to be renewed due to passing their expiry
While some certificates have renewed ok, the ipaCert and
auditSigningCert are renewing but the new certificates have the wrong
serverA (CRL, first, master) RHEL 7.3, ipa 4.4
serverB (replica) RHEL 7.3, ipa 4.4
serverC (replica) RHEL 7.4, ipa 4.5
Once there are renewed certificates with the wrong Subject present,
there are various problems with renewing the remaining certificates,
which I think might be related to the bad Subject:
1) When just ipaCert has the wrong subject no further renewals happen
2) When auditSigningCert has the wrong subject the ipa pki-tomcatd
service will not start and no further renewals happen.
I've been round the following loop many times on ServerA, our first master:
1) Restore good certificates from backup
2) Put the clock back to a time when certificates are all valid
3) Resubmit certificates for renewal
Each time the ipaCert renews it has the same wrong Subject. The wrong
Subject includes the host name of one of our ipa client systems.
Each time the auditSigningCert renews it has the same wrong Subject but
a different subject to the ipaCert. The wrong Subject in this case
includes the host name of a system which has never been an ipa client,
but might have been added and removed with ipa host-add and ipa host-del
for testing something, a while ago.
As far as I can see, the "cert_subject" is set correctly in the file
/var/lib/certmonger/<request id> until the point at which the
certificate is actually renewed.
I'd be very grateful for some pointers as to which configuration options
and logs to check through to resolve this problem on our production system.
If its of any relevance we did change which server is the first master
some time ago.
I have very similiar problem as this one:
ipa-server-upgrade fails as below
Upgrading IPA services
Upgrading the configuration of the IPA services
[Verifying that root certificate is published]
[Migrate CRL publish directory]
CRL tree already moved
[Verifying that CA proxy configuration is correct]
IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
CA did not start in 300.0s
The ipa-server-upgrade command failed. See /var/log/ipaupgrade.log for more information
And the log tells me that CA returns status 500
DEBUG Waiting for CA to start...
DEBUG request POST http://<<ipa1.fqdn>>:8080/ca/admin/ca/getStatus<http://%3c%3cipa1.fqdn%3e%3e:8080/ca/admin/ca/getStatus>
DEBUG request body ''
DEBUG response status 500
DEBUG response headers Server: Apache-Coyote/1.1
Date: Fri, 15 Jun 2018 10:05:29 GMT
DEBUG The CA status is: check interrupted due to error: Retrieving CA status failed with status 500
DEBUG Waiting for CA to start...
ERROR IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
File "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 172, in execute
return_value = self.run()
File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgrade.py", line 48, in run
The ipa-server-upgrade command failed, exception: ScriptError: CA did not start in 300.0s
ERROR CA did not start in 300.0s
With command "ipactl start --ignore-service-failures" I can start all the services but pki-tomcatd.
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
httpd Service: RUNNING
pki-tomcatd Service: STOPPED
smb Service: RUNNING
winbind Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
Suggested resolution to above problem doesn't help me since the LDAP and NSS DB seem to have same certificates (some difference in wrapping but the string is same if I take out the line breaks) and even the serial number matches.
certutil -L -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca' -a
certutil -L -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca' |grep Serial
Serial Number: 4 (0x4)
ldapsearch -LLL -D 'cn=directory manager' -W -b uid=pkidbuser,ou=people,o=ipaca userCertificate description seeAlso
Enter LDAP Password:
description: 2;4;CN=Certificate Authority,O=<<REALM>>;CN=CA Subsystem,
seeAlso: CN=CA Subsystem,O=<<REALM>>
And here's where my actual knowledge of things end. I've been trying to figure out all kind of logs (tomcat, Kerberos, directory server, ...) but haven't found a solid reason for it. I'm starting to believe this is a certificate issue, because although "getcert list" tells me that the certificate status is "Monitoring" on all certificates the expiry date is already in the past (current date 20.6.2018, certificate expiry 21.03.2018) on 4 certificates and it won't update even if I resubmit it or delete certificate and manually redo it (it got the same date as the "old ones"). The "main certs" ("caSigningCert cert-pki-ca", "Server-Cert cert-pki-ca" and two directory server certs) are valid for years (until 2020+).
Request ID '20160331084234':
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
issuer: CN=Certificate Authority,O=<<REALM>>
subject: CN=OCSP Subsystem,O=<<REALM>>
expires: 2018-03-21 09:42:04 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
Request ID '20160331085008':
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
issuer: CN=Certificate Authority,O=<<REALM>>
expires: 2020-03-04 09:58:23 UTC
principal name: HTTP/<<ipasrv1.fqdn>>@<<REALM>>
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
post-save command: /usr/lib64/ipa/certmonger/restart_httpd
Has anyone else bumped into same kind of issues? Any ideas where I should continue looking? I'm starting to run out of ideas...
I had an issue a short while ago with a replica which turned out to be an
expired certificate which I renewed and all seemed good.
It now appears that although the certificate renewed as seen by getcert
-list, it didn't update /etc/httpd/alias and so the httpd and tomcat-pki
services won't start unless I set the date to before the certificate
expired, and even then sometimes the httpd error_log shows:
Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off"
to nss.conf so the server can start until the problem can be resolved.
and the service fails to start.
I've tried resubmitting the certificate, and it doesn't seem to throw an
error, but it doesn't update /alias either.
Trying to access the server via the web page shows the old certificate
still in use.
I see the same certificate error with the replica server, which was freshly
rebuilt and added last week.
I've doubtless dug further into the hole trying to troubleshoot this, so I
probably need to start from the beginning again, and a pointer in the right
direction would be a great help!
A getcert list shows all the certificates expiry dates well into the future.
How can I get the certs back in sync? I've found a few guides and most seem
to be for earlier versions, and I'm not sure if they're still current.
I can post whatever logs you think will help, I'm afraid I'm not familiar
enough with them all to tell which are the most relevant. Is there a guide
for the logs?
Thanks for any help you can give,
I have a synology NAS which hosts some SMB shares on my network. I would
like to be able to use FreeIPA as the LDAP provider it checks against for
authenticating these shares. I have a system user that I created in
FreeIPA for this purpose.
I configured the NAS to connect to my FreeIPA server for LDAP, but I get a
message about a failure to access some users NT passwords and how the Samba
service may not work for these users. It also says it could be either a
lack of NT passwords for the users or insufficient privileges to access
them. After chatting with Synology support they wanted me to enable CIFS
plaintext password authentication. However, if I select that option it
given me a warning about the share not being able to be the remote mount
target of CIFS anymore due to SMB being set to v1 only and disabling the
SMB related Bonjour service. If the system user doesn't have the needed
privileges, how can I fix that since I can't enroll the NAS?
BYU Dept. of Chemistry and Biochemistry
I've been having an issue recently where my servers can't install
FreeIPA due to this error:
Cannot connect to the server due to generic error: error marshalling
data for XML-RPC transport: message: need a <type 'unicode'>; got 'No
valid Negotiate header in server response' (a <type 'str'>)
Installation failed. Rolling back changes.
Unenrolling client from IPA server
Restarting FreeIPA (ipactl stop, ipactl start) solves the solution, but
I captured the FreeIPA logs during that event, which you can see here:
Any ideas what's going wrong? One solution would be to have a cron
restart FreeIPA nightly, but that's not ideal.
Harald Dunkel wrote:
> Hi Robert,
> On 6/26/18 4:45 PM, Rob Crittenden via FreeIPA-users wrote:
>> Harald Dunkel wrote:
>>> I see several files with a key_pin or Key_pin_file inside. I would prefer
>>> to send you these files in an encrypted EMail. What would you suggest? Do
>>> you have PGP?
>> Except for the pin the rest of the content is generally safe. My key is
>> available in the MIT keyserver if you want to send it out of band.
I don't see anything obviously wrong. I'd try launching certmonger from
a shell to see what you get:
# certmonger -d 9
At long last I've got a brand new IPA cluster running in our AWS
footprint with a modern v4.5.4 install and a proper AD Trust in place to
a complex domain forest
In my older cluster I made use of a lot of the info here that Alexander
and Jakob had written up:
Just wanted to check and see if the advice in that post still holds true
for ipa-server-4.5.4-10 etc. -- is there anything I should avoid or
anything new (links, blog posts, URLs) that I should read up on to tune
After installing a PC with Ubuntu 18.04 I'm seeing this problem with
SSH logins. The gssapi-with-mic authentication method does not
work anymore. Strangely enough a system that I upgraded (16.04->18.04)
was working fine.
The debug of sshd shows (fivel being the unqualified hostname):
debug1: Unspecified GSS failure. Minor code may provide more information
No key table entry found matching host/fivel@
After debugging and looking at differences between the installed and upgraded system
I found that the new Ubuntu 18.04 installation has a slightly different krb5 configuration.
dns_canonicalize_hostname = false
fqdn = <kerberos realm>
Now the workaround for the login problem is to comment out dns_canonicalize_hostname.
Can anyone comment on this? Why was this changed? Why doesn't it work out of the box?