Hi Fraser, I am making some progress. Let's please continue.
[1]
I was able to follow your info and find common date in past for all certs to be valid.
Note, in case this is important, I have four IPA servers and I do this on CA renewal
master.
[2]
Then system clock was set to past time (about 2 weeks before expire time) , stop ntp,
restart krb5kdc, dirsrv, httpd, CA. The I verify that CA is running, with command :
SSL_DIR=/etc/httpd/alias/ curl -v -o /dev/null --cacert /etc/ipa/ca.crt
https://`hostname`:8443/ca/agent/ca/profileReview
* Initializing NSS with certpath: sql:/etc/httpd/alias/
* CAfile: /etc/ipa/ca.crt
CApath: none
* NSS: client certificate not found (nickname not specified)
* SSL connection using TLS_RSA_WITH_AES_256_CBC_SHA
* Server certificate:
* subject:
CN=ca-ldap04,O=REALM.COM
* start date: Aug 01 17:18:06 2018 GMT
* expire date: Jul 21 17:18:06 2020 GMT
* common name: ca-ldap04
* issuer: CN=Certificate
Authority,O=US.ORACLE.COM
GET /ca/agent/ca/profileReview HTTP/1.1
User-Agent: curl/7.29.0
Host: ca-ldap04:8443
Accept: */*
* NSS: client certificate not found (nickname not specified)
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: text/html;charset=UTF-8
< Transfer-Encoding: chunked
< Date: Wed, 01 Aug 2018 18:28:04 GMT
<
{ [data not shown]
100 17641 0 17641 0 0 230k 0 --:--:-- --:--:-- --:--:-- 232k
* Connection #0 to host
ca-ldap04.realm.com left intact
[3]
Restart certmonger, and ONLY ONE cert is renewed, it's "Server-Cert
cert-pki-ca".
status: CA_UNREACHABLE
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
cert-pki-ca',token='NSS Certificate DB'
expires: 2018-08-14 20:49:38 UTC
status: CA_UNREACHABLE
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
cert-pki-ca',token='NSS Certificate DB'
expires: 2018-08-14 20:49:35 UTC
status: CA_UNREACHABLE
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
cert-pki-ca',token='NSS Certificate DB'
expires: 2018-08-14 20:49:36 UTC
status: MONITORING
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
cert-pki-ca',token='NSS Certificate DB'
expires: 2036-08-24 20:49:35 UTC
status: MONITORING
certificate:
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
cert-pki-ca',token='NSS Certificate DB'
expires: 2020-07-21 17:18:06 UTC
status: CA_UNREACHABLE
certificate:
type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS
Certificate DB'
expires: 2018-08-14 20:50:00 UTC
[4]
From "journalctl -fu certmonger --full " basically there is " Insufficient
access: Invalid credentials"
Aug 01 11:04:45
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:45 [7447] Will
revisit CA6('dogtag-ipa-ca-renew-agent').default_profile now.
Aug 01 11:04:45
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:45 [7447]
CA6('dogtag-ipa-ca-renew-agent').default_profile moved to state
'DISABLED'
Aug 01 11:04:45
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:45 [7447] Waiting
for instructions for CA6('dogtag-ipa-ca-renew-agent').default_profile.
Aug 01 11:04:45
ca-ldap04.realm.com dogtag-ipa-ca-renew-agent-submit[7526]: Traceback
(most recent call last):
File
"/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 511, in
<module>
sys.exit(main())
File
"/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 497, in main
if
ca.is_renewal_master():
File
"/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", line 1188,
in is_renewal_master
self.ldap_connect()
File
"/usr/lib/python2.7/site-packages/ipaserver/install/service.py", line 177, in
ldap_connect
conn.do_bind(self.dm_password, autobind=self.autobind)
File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1690, in do_bind
self.do_sasl_gssapi_bind(timeout=timeout)
File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1668, in
do_sasl_gssapi_bind
self.__bind_with_wait(self.gssapi_bind, timeout)
File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1650, in
__bind_with_wait
bind_func(*args, **kwargs)
File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 1108, in
gssapi_bind
'', auth_tokens, server_controls, client_controls)
File
"/usr/lib64/python2.7/contextlib.py", line 35, in __exit__
self.gen.throw(type, value, traceback)
File
"/usr/lib/python2.7/site-packages/ipapython/ipaldap.py", line 973, in
error_handler
raise
errors.ACIError(info="%s %s" % (info, desc))
ACIError:
Insufficient access: Invalid credentials
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
Certificate submission still ongoing.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will
revisit Request1('20161112162146') on traffic from 92.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').profiles retrieval unsupported
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').profiles moved to state 'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will
revisit CA6('dogtag-ipa-ca-renew-agent').profiles now.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').profiles moved to state 'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting
for instructions for CA6('dogtag-ipa-ca-renew-agent').profiles.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').capabilities retrieval unsupported
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').capabilities moved to state 'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will
revisit CA6('dogtag-ipa-ca-renew-agent').capabilities now.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').capabilities moved to state 'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting
for instructions for CA6('dogtag-ipa-ca-renew-agent').capabilities.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').renewal_reqs retrieval unsupported
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').renewal_reqs moved to state 'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will
revisit CA6('dogtag-ipa-ca-renew-agent').renewal_reqs now.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').renewal_reqs moved to state 'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting
for instructions for CA6('dogtag-ipa-ca-renew-agent').renewal_reqs.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs retrieval unsupported
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs moved to state
'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will
revisit CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs now.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs moved to state
'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting
for instructions for CA6('dogtag-ipa-ca-renew-agent').enrollment_reqs.
Aug 01 11:04:46
ca-ldap04.realm.com python2[7533]: GSSAPI client step 1
Aug 01 11:04:46
ca-ldap04.realm.com python2[7533]: GSSAPI client step 1
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').encryption_certs retrieval unsupported
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').encryption_certs moved to state
'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will
revisit CA6('dogtag-ipa-ca-renew-agent').encryption_certs now.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
CA6('dogtag-ipa-ca-renew-agent').encryption_certs moved to state
'DISABLED'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Waiting
for instructions for CA6('dogtag-ipa-ca-renew-agent').encryption_certs.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
Certificate submission attempt complete.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Child
status = 3.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Child
output:
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: "Internal error
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: "
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Internal
error
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
Certificate not (yet?) issued.
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447]
Request1('20161112162146') moved to state 'CA_UNREACHABLE'
Aug 01 11:04:46
ca-ldap04.realm.com certmonger[7447]: 2018-08-01 11:04:46 [7447] Will
revisit Request1('20161112162146') in 566546 seconds.