Hello Flo,
I'm resending the mail since my first response was rejected because of SPF, and only
found its way to the mailing list...
Many thanks again for your response. First of all, I've figured out that
the package "pki-symkey" was missing, so I've installed it with yum.
Now, according to systemctl, pki-tomcatd is running:
root@mat-ipa-master-1:~$ systemctl status pki-tomcatd(a)pki-tomcat.service
● pki-tomcatd(a)pki-tomcat.service - PKI Tomcat Server pki-tomcat
Loaded: loaded (/lib/systemd/system/pki-tomcatd@.service; enabled;
vendor preset: disabled)
Active: active (running) since Wed 2018-01-10 10:08:25 CET; 3min 55s
ago
(...)
But, ipactl still complains that it is stopped:
root@mat-ipa-master-1:~$ ipactl start --ignore-service-failures
Existing service file detected!
Assuming stale, cleaning and proceeding
Starting Directory Service
(...)
Failed to start pki-tomcatd Service
Forced start, ignoring pki-tomcatd Service, continuing normal operation
(...)
As you suggested, I've checked the debug log
/var/log/pki/pki-tomcat/ca/debug:
(...)
Internal Database Error encountered: Could not connect to LDAP server
host mat-ipa-master-1.materna-com.de port 636 Error
netscape.ldap.LDAPException: Authentication failed (48)
(...)
So, I've examined the config and the certificates, with the commands
from the blog post:
root@mat-ipa-master-1:~$ grep internaldb.ldapauth
/etc/pki/pki-tomcat/ca/CS.cfg
internaldb.ldapauth.authtype=SslClientAuth
internaldb.ldapauth.bindDN=cn=Directory Manager
internaldb.ldapauth.bindPWPrompt=internaldb
internaldb.ldapauth.clientCertNickname=subsystemCert cert-pki-ca
root@mat-ipa-master-1:~$ grep internaldb.ldapconn
/etc/pki/pki-tomcat/ca/CS.cfg
internaldb.ldapconn.host=mat-ipa-master-1.materna-com.de
internaldb.ldapconn.port=636
internaldb.ldapconn.secureConn=true
Ok, we're using LDAPS.
root@mat-ipa-master-1:~$ certutil -L -d /etc/pki/pki-tomcat/alias -n
'subsystemCert cert-pki-ca'
Certificate:
Data:
Version: 3 (0x2)
Serial Number: 33 (0x21)
Signature Algorithm: PKCS #1 SHA-256 With RSA Encryption
Issuer: "CN=Certificate Authority,O=MATERNA-COM.DE"
Validity:
Not Before: Fri Jan 12 23:56:02 2018
Not After : Sat Jan 13 14:45:00 2018
(...)
Interesting, I've reset the date to Jan 10th:
root@mat-ipa-master-1:~$ date
Wed Jan 10 10:05:49 CET 2018
So, the certificate is not expired, but invalid since it's too new?!
Never mind, going further:
root@mat-ipa-master-1:~$ grep internal
/var/lib/pki/pki-tomcat/conf/password.conf | cut -d= -f2 > /tmp/pwdfile.txt
root@mat-ipa-master-1:~$ certutil -K -d /etc/pki/pki-tomcat/alias -f
/tmp/pwdfile.txt -n 'subsystemCert cert-pki-ca'
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private
Key and Certificate Services"
certutil: problem listing keys: SEC_ERROR_UNRECOGNIZED_OID: Unrecognized
Object Identifier.
Do you have an idea why the key for this OID cannot be found?
Thanks, and best regards from Germany,
Harald
--
Dipl.-Ing. (FH)
Harald Husemann
Senior System Administrator
Managed Services
Phone: +49 231 9505-222
Mobile: +49 1570 11 22 22 2
harald.husemann(a)materna.de
www.materna.de | Newsletter | Twitter | XING | Facebook | google+
Materna GmbH | Voßkuhle 37 | D-44141 Dortmund | Germany
Geschäftsführer: Helmut Binder, Michael Knopp
Amtsgericht Dortmund HRB 5839
-----Ursprüngliche Nachricht-----
Von: Florence Blanc-Renaud [mailto:flo@redhat.com]
Gesendet: Dienstag, 30. Januar 2018 14:05
An: FreeIPA users list <freeipa-users(a)lists.fedorahosted.org>
Cc: Husemann, Harald <Harald.Husemann(a)materna.de>
Betreff: Re: [Freeipa-users] Re: Howto renew certificates with external CA?
On 01/24/2018 07:35 PM, Harald.Husemann--- via FreeIPA-users wrote:
Hello Flo,
thanks for your answer, and for the explanation of the certutil output. I have tried your
suggestion, first with sudo:
hhuseman@mat-ipa-master-1:~$ sudo kinit -kt /etc/krb5.keytab
[sudo] password for hhuseman:
Sorry, try again.
[sudo] password for hhuseman:
Sorry, try again.
[sudo] password for hhuseman:
sudo: 2 incorrect password attempts
I'm quite sure my password is correct, so it seems there's something broken here
also, since sudo worked before the certificate update. My next try was running the command
as root:
hhuseman@mat-ipa-master-1:~$ su -
Password:
root@mat-ipa-master-1:~$ kinit -kt /etc/krb5.keytab
root@mat-ipa-master-1:~$ exit
logout
As you see, there is no output at all, so I tried it again with -V:
root@mat-ipa-master-1:~$ kinit -V -kt /etc/krb5.keytab
Using existing cache: persistent:0:krb_ccache_VPUg94b
Using principal: host/mat-ipa-master-1.materna-com.de(a)MATERNA-COM.DE
Using keytab: /etc/krb5.keytab
Authenticated to Kerberos v5
root@mat-ipa-master-1:~$
I have also re-checked the certificate which is issued by the HTTPS-Server in my browser,
it is still the old one.
And, I've tried to get the list of certificates with ipa-getcert:
root@mat-ipa-master-1:~$ ipa-getcert list
Number of certificates and requests being tracked: 5.
Request ID '20170303080146':
status: CA_UNREACHABLE
ca-error: Server at
https://mat-ipa-master-1.materna-com.de/ipa/xml failed
request, will retry: -504 (libcurl failed to execute the HTTP POST transaction,
explaining: Peer's Certificate has expired.).
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-MATERNA-COM-DE',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-MATERNA-COM-DE/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/dirsrv/slapd-MATERNA-COM-DE',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MATERNA-COM.DE
subject: CN=mat-ipa-master-1.materna-com.de,O=MATERNA-COM.DE
expires: 2018-01-13 14:45:00 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MATERNA-COM-DE
track: yes
auto-renew: yes
Interesting, since the date was still reset to January 11th, so, even the old certificate
should be valid:
root@mat-ipa-master-1:~$ date
Thu Jan 11 05:22:21 CET 2018
Nevertheless, I've set the date to actual time by sync'ing it to our
NTP-Server:
root@mat-ipa-master-1:~$ ntpdate omcix
24 Jan 19:09:00 ntpdate[32699]: step time server 172.30.96.6 offset 1172766.789568 sec
root@mat-ipa-master-1:~$ date
Wed Jan 24 19:09:06 CET 2018
But, ipa-getcert list is still falling:
root@mat-ipa-master-1:~$ ipa-getcert list
Number of certificates and requests being tracked: 5.
Request ID '20170303080146':
status: NEED_TO_SUBMIT
ca-error: Server at
https://mat-ipa-master-1.materna-com.de/ipa/xml failed
request, will retry: -504 (libcurl failed to execute the HTTP POST transaction,
explaining: Peer's Certificate has expired.).
stuck: no
key pair storage:
type=NSSDB,location='/etc/dirsrv/slapd-MATERNA-COM-DE',nickname='Server-Cert',token='NSS
Certificate DB',pinfile='/etc/dirsrv/slapd-MATERNA-COM-DE/pwdfile.txt'
certificate:
type=NSSDB,location='/etc/dirsrv/slapd-MATERNA-COM-DE',nickname='Server-Cert',token='NSS
Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=MATERNA-COM.DE
subject: CN=mat-ipa-master-1.materna-com.de,O=MATERNA-COM.DE
expires: 2018-01-13 14:45:00 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command:
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv MATERNA-COM-DE
track: yes
auto-renew: yes
root@mat-ipa-master-1:~$
To ensure everything's running I've issued an ipactl:
root@mat-ipa-master-1:~$ ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
ipa_memcached Service: RUNNING
httpd Service: RUNNING
ipa-custodia Service: RUNNING
pki-tomcatd Service: STOPPED
ipa-otpd Service: RUNNING
ipa: INFO: The ipactl command was successful
root@mat-ipa-master-1:~$
So it seems everything's ok except of the PKI, I've tried to restart it, but it
fails:
root@mat-ipa-master-1:~$ ipactl start pki-tomcatd
You must specify one action
root@mat-ipa-master-1:~$ ipactl start
Existing service file detected!
Assuming stale, cleaning and proceeding
Starting Directory Service
Starting krb5kdc Service
Starting kadmin Service
Starting ipa_memcached Service
Starting httpd Service
Starting ipa-custodia Service
Starting pki-tomcatd Service
Failed to start pki-tomcatd Service
Shutting down
Hint: You can use --ignore-service-failure option for forced start in case that a
non-critical service failed
Aborting ipactl
root@mat-ipa-master-1:~$
I hope this helps to track down the problem a bit...
pki-tomcat may fail to start if it's unable to authenticate to the LDAP
server (LDAP is used as data store by pki-tomcat, and authentication is
done with the cert 'subsystemCert cert-pki-ca' that is stored in
/etc/pki/pki-tomcat/alias).
You will need to check if this cert is still valid. Sometimes during
renewal, the cert properly gets downloaded to the NSS db
/etc/pki/pki-tomcat/alias but is not propagated to the LDAP server. You
need to compare the cert in the NSS DB and the value for the ldap entry
uid=pkidbuser,ou=people,o=ipaca. More information can be found in this
blog:https://floblanc.wordpress.com/2017/09/11/troubleshooting-freeipa-pk...
But before jumping to the conclusion, please read pki-tomcat logs in
/var/log/pki/pki-tomcat/ca/debug and check if the issue is indeed coming
from an expired subsystemCert cert-pki-ca certificate.
Flo
Many thanks and regards from Germany,
Harald