Hello,

thank you for your support.
We have tried going back date, and certificate still didn't get renewed. We found new error messages.

Looking at the log, it is clear that the issue is a result of us manually adding a new CA (March '18) when previous CA expired on the master and we had lost the very first CA Master.


Current Server-Cert cert-pki-ca has been signed with previous CA and process failed


And renewal expire with following message:

“Error 60 connecting to https://ds01.DOMAIN.com:8443/ca/agent/ca/profileReview: Peer certificate cannot be authenticated with given CA certificates.”

key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set

certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'


Checking log in /var/log/httpd/error_log

Xmlserver is failing to authenticate

“SSL Library Error: -12195 Peer does not recognize and trust the CA that issued your certificate”


Question:

How can we regenerate the certificate with new CA, instead of renewing the certificate?

How can we avoid that error at profileReview ?



Here is the journal content for certomnger, after going back on date.

Jun 11 18:42:38 ds01.domain.com systemd[1]: Stopping Certificate monitoring and PKI enrollment...
Jun 11 18:42:38 ds01.domain.com systemd[1]: Starting Certificate monitoring and PKI enrollment...
Jun 11 18:42:39 ds01.domain.com systemd[1]: Started Certificate monitoring and PKI enrollment.
Jun 11 18:42:51 ds01.domain.com ipa-submit[12982]: GSSAPI client step 1
Jun 11 18:42:51 ds01.domain.com ipa-submit[12982]: GSSAPI client step 1
Jun 11 18:42:52 ds01.domain.com ipa-submit[12982]: GSSAPI client step 1
Jun 11 18:42:52 ds01.domain.com ipa-submit[12982]: GSSAPI client step 1
Jun 11 18:43:10 ds01.domain.com dogtag-ipa-ca-renew-agent-submit[12831]: Forwarding request to dogtag-ipa-renew-agent
Jun 11 18:43:11 ds01.domain.com dogtag-ipa-ca-renew-agent-submit[12831]: dogtag-ipa-renew-agent returned 3
Jun 11 18:43:11 ds01.domain.com certmonger[12742]: 2018-06-11 18:43:11 [12742] Error 60 connecting to https://ds01.domain.com:8443/ca/agent/ca/profileReview: Peeer certificate cannot be authenticated with given CA certificates.
Jun 11 18:43:20 ds01.domain.com dogtag-ipa-ca-renew-agent-submit[12832]: Forwarding request to dogtag-ipa-renew-agent
Jun 11 18:43:20 ds01.domain.com dogtag-ipa-renew-agent-submit[12989]: GET https://ds01.domain.com:8443/ca/agent/ca/profileReview?requestId=49990630&xml=true
Jun 11 18:43:20 ds01.domain.com dogtag-ipa-renew-agent-submit[12989]: (null)
Jun 11 18:43:20 ds01.domain.com dogtag-ipa-ca-renew-agent-submit[12832]: dogtag-ipa-renew-agent returned 3
Jun 11 18:43:20 ds01.domain.com certmonger[12742]: 2018-06-11 18:43:20 [12742] Error 60 connecting to https://ds01.domain.com:8443/ca/agent/ca/profileReview: Peer certificate cannot be authenticated with given CA certificates.

Thank you,
Bhavin

From: Rob Crittenden <rcritten@redhat.com>
Sent: Friday, January 18, 2019 2:42 PM
To: Bhavin Vaidya; FreeIPA users list; Florence Blanc-Renaud
Subject: Re: [Freeipa-users] Re: Expired Certificates.
 
Bhavin Vaidya wrote:
> Thank you Rob.
>
> After falling date more than a day prior to oldest expiring date,
> restarted certmonger, it showed SUBMITTING for sometime and went back
> to CA_UNREACHABLE with Internal Error.

You'll need to look in the CA debug log to try to discern why it isn't
accepting requests. My guess is it isn't really started (don't confuse
tomcat running with the CA running).

>
> WRT Fraser's IdM Blog
> <https://frasertweedale.github.io/blog-redhat/posts/2018-05-31-replacing-lost-ca.html> we
> have the old ra-agents key an certificates at /var/lib/ipa. Can we just
> remove them and do back date again? this is the oldest expired
> certificate we have. this may be due to couple of upgrades we have
> carried out on our Master FreeIPA server (ds01).

No, don't remove any files. This is not related to the CA not answering
requests.

rob

>
>  
> [root@ds01 local]# getcert list -f /var/lib/ipa/ra-agent.pem
> Number of certificates and requests being tracked: 9.
> Request ID '20180315021503':
> status: CA_UNREACHABLE
> ca-error: Internal error
> stuck: no
> key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
> certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
> CA: dogtag-ipa-ca-renew-agent
> issuer: CN=Certificate Authority,O=DOMAIN.COM
> subject: CN=IPA RA,O=DOMAIN.COM
> expires: 2018-06-15 23:15:23 UTC
> key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> eku: id-kp-serverAuth,id-kp-clientAuth
> pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
> post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
> track: yes
> auto-renew: yes
>
> thank you,
> Bhavin
>
>
> ------------------------------------------------------------------------
> *From:* Rob Crittenden <rcritten@redhat.com>
> *Sent:* Thursday, January 17, 2019 12:40 PM
> *To:* FreeIPA users list; Florence Blanc-Renaud
> *Cc:* Bhavin Vaidya
> *Subject:* Re: [Freeipa-users] Re: Expired Certificates.
>  
> Bhavin Vaidya via FreeIPA-users wrote:
>> Thank you Flo.
>>
>> # ipa config-show | grep renewal
>>   IPA CA renewal master: ds01.domain.com            <----- this is the
>> server having 2 expired certificate.
>>
>> One more question.
>> if we just stop NTP (and have other IPA services running as is)  and go
>> back in date to June 14, 2018 date, will there be any issue with other
>> FreeIPA server or services?
>
> You shouldn't have issues with other masters. They will fail to connect
> due to the time mismatch and will be able to re-connect once time is
> restored.
>
> You'll need to manually restart the services after running ipactl stop
> because ipactl start will start NTP.
>
> Once the certs are renew then setting the date back to today and ipactl
> restart should bring everything back up.
>
> rob
>
>>
>> thank you,
>> Bhavin
>>
>>
>> ------------------------------------------------------------------------
>> *From:* Florence Blanc-Renaud <flo@redhat.com>
>> *Sent:* Thursday, January 17, 2019 12:20 AM
>> *To:* FreeIPA users list
>> *Cc:* Bhavin Vaidya
>> *Subject:* Re: [Freeipa-users] Expired Certificates.
>>  
>> On 1/17/19 4:30 AM, Bhavin Vaidya via FreeIPA-users wrote:
>>> Hello,
>>>
>>> We rebooted our Primary FreeIPA server (ds01) and then it will not start
>>> pki-tomcatd, Kerberos will also not work, though it starts.
>>> We realized that 2 certificates have expired.
>>> we tried stopped ipa, stopped NTP, going back to Dec 14th, 2018 and
>>> restarted certmonger, bring back date but still no luck.
>>>
>>> this is our primary, and we do have 2 local and 2 remote FreeIPA server
>>> on them only one of the certificate (June 15th, 2018) is showing expired
>>> and others are good.
>>>
>> Hi,
>>
>> the first step is to find which server is the CA renewal master. This
>> server will need to be repaired first.
>> # ipa config-show | grep renewal
>>    IPA CA renewal master: <hostname>
>>
>> On the renewal master, check which certificates are expired, find a date
>> in the past where all certs are valid, stop NTP, go back to the date and
>> check if certmonger is succeeding in renewing the certs. If it's not the
>> case, you will have to check the journal content for certmonger messages.
>>
>> HTH,
>> flo
>>> Do we have to go back on date before June 15th, 2018 on ds01?
>>> Details are:
>>>
>>> [root@ds01 ~]# cat /etc/centos-release
>>> CentOS Linux release 7.4.1708 (Core)
>>>
>>> [root@ds01 ~]# ipa ca-find
>>> ------------
>>> 1 CA matched
>>> ------------
>>>    Name: ipa
>>>    Description: IPA CA
>>>    Authority ID: 606<...........SNIP..........>450
>>>    Subject DN: CN=Certificate Authority,O=DOMAIN.COM
>>>    Issuer DN: CN=Certificate Authority,O=DOMAIN.COM
>>> ----------------------------
>>> Number of entries returned 1
>>> ----------------------------
>>>
>>> [root@ds02 ~]# ipa ping
>>> -------------------------------------------
>>> IPA server version 4.5.0. API version 2.228
>>>
>>> [root@ds01 ~]# KRB5_TRACE=/dev/stdout kinit admin
>>> [5509] 1547598366.261229: Getting initial credentials for admin@DOMAIN.COM
>>> [5509] 1547598366.267532: Sending request (171 bytes) to DOMAIN.COM
>>> [5509] 1547598366.268593: Resolving hostname ds01.domain.com
>>> [5509] 1547598366.269479: Sending initial UDP request to dgram
>>> 192.1xx.xxx.xxx:88
>>> [5509] 1547598367.270712: Initiating TCP connection to stream
>>> 192.1xx.xxx.xxx:88
>>> [5509] 1547598367.270884: Sending TCP request to stream 192.1xx.xxx.xxx:88
>>> [5509] 1547598372.338780: Received answer (171 bytes) from dgram
>>> 192.1xx.xxx.xxx:88
>>> [5509] 1547598372.338841: Terminating TCP connection to stream
>>> 192.1xx.xxx.xxx:88
>>> [5509] 1547598372.338989: Response was from master KDC
>>> [5509] 1547598372.339095: Received error from KDC: -1765328324/Generic
>>> error (see e-text)
>>> kinit: Generic error (see e-text) while getting initial credentials
>>>
>>>
>>> [root@ds01 ~]# getcert list
>>> Number of certificates and requests being tracked: 9.
>>> Request ID '20180228053337':
>>>          status: MONITORING
>>>          stuck: no
>>>          key pair storage:
>>> type=FILE,location='/var/kerberos/krb5kdc/kdc.key'
>>>          certificate: type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
>>>          CA: SelfSign
>>>          issuer: CN=ds01.domain.com,O=DOMAIN.COM
>>>          subject: CN=ds01.domain.com,O=DOMAIN.COM
>>>          expires: 2019-03-07 06:24:12 UTC
>>>          principal name: krbtgt/DOMAIN.COM@DOMAIN.COM
>>>          certificate template/profile: KDCs_PKINIT_Certs
>>>          pre-save command:
>>>          post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
>>>          track: yes
>>>          auto-renew: yes
>>> Request ID '20180315021457':
>>>          status: MONITORING
>>>          stuck: no
>>>          key pair storage:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>>> cert-pki-ca',token='NSS Certificate DB',pin set
>>>          certificate:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
>>> cert-pki-ca',token='NSS Certificate DB'
>>>          CA: dogtag-ipa-ca-renew-agent
>>>          issuer: CN=Certificate Authority,O=DOMAIN.COM
>>>          subject: CN=CA Audit,O=DOMAIN.COM
>>>          expires: 2020-02-25 04:27:49 UTC
>>>          key usage: digitalSignature,nonRepudiation
>>>          pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>>>          post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
>>> "auditSigningCert cert-pki-ca"
>>>          track: yes
>>>          auto-renew: yes
>>> Request ID '20180315021500':
>>>          status: MONITORING
>>>          stuck: no
>>>          key pair storage:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS
>>> Certificate DB',pin set
>>>          certificate:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS
>>> Certificate DB'
>>>          CA: dogtag-ipa-ca-renew-agent
>>>          issuer: CN=Certificate Authority,O=DOMAIN.COM
>>>          subject: CN=OCSP Subsystem,O=DOMAIN.COM
>>>          expires: 2020-02-25 04:28:38 UTC
>>>          eku: id-kp-OCSPSigning
>>>          pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>>>          post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
>>> "ocspSigningCert cert-pki-ca"
>>>          track: yes
>>>          auto-renew: yes
>>> Request ID '20180315021501':
>>>          status: MONITORING
>>>          stuck: no
>>>          key pair storage:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>>> cert-pki-ca',token='NSS Certificate DB',pin set
>>>          certificate:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
>>> cert-pki-ca',token='NSS Certificate DB'
>>>          CA: dogtag-ipa-ca-renew-agent
>>>          issuer: CN=Certificate Authority,O=DOMAIN.COM
>>>          subject: CN=CA Subsystem,O=DOMAIN.COM
>>>          expires: 2020-02-25 04:31:47 UTC
>>>          key usage:
>>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>>>          eku: id-kp-serverAuth,id-kp-clientAuth
>>>          pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>>>          post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
>>> "subsystemCert cert-pki-ca"
>>>          track: yes
>>>          auto-renew: yes
>>> Request ID '20180315021502':
>>>          status: MONITORING
>>>          stuck: no
>>>          key pair storage:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>>> cert-pki-ca',token='NSS Certificate DB',pin set
>>>          certificate:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
>>> cert-pki-ca',token='NSS Certificate DB'
>>>          CA: dogtag-ipa-ca-renew-agent
>>>          issuer: CN=Certificate Authority,O=DOMAIN.COM
>>>          subject: CN=Certificate Authority,O=DOMAIN.COM
>>>          expires: 2038-03-07 03:47:46 UTC
>>>          key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
>>>          pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>>>          post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
>>> "caSigningCert cert-pki-ca"
>>>          track: yes
>>>          auto-renew: yes
>>> Request ID '20180315021503':
>>>          status: CA_UNREACHABLE
>>>          ca-error: Internal error
>>>          stuck: no
>>>          key pair storage: type=FILE,location='/var/lib/ipa/ra-agent.key'
>>>          certificate: type=FILE,location='/var/lib/ipa/ra-agent.pem'
>>>          CA: dogtag-ipa-ca-renew-agent
>>>          issuer: CN=Certificate Authority,O=DOMAIN.COM
>>>          subject: CN=IPA RA,O=DOMAIN.COM
>>>          expires: 2018-06-15 23:15:23 UTC
>>>          key usage:
>>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>>>          eku: id-kp-serverAuth,id-kp-clientAuth
>>>          pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
>>>          post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
>>>          track: yes
>>>          auto-renew: yes
>>> Request ID '20180315021504':
>>>          status: CA_UNREACHABLE
>>>          ca-error: Internal error
>>>          stuck: no
>>>          key pair storage:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>>> cert-pki-ca',token='NSS Certificate DB',pin set
>>>          certificate:
>>> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
>>> cert-pki-ca',token='NSS Certificate DB'
>>>          CA: dogtag-ipa-ca-renew-agent
>>>          issuer: CN=Certificate Authority,O=DOMAIN.COM
>>>          subject: CN=ds01.domain.com,O=DOMAIN.COM
>>>          expires: 2018-12-16 21:02:44 UTC
>>>          key usage:
>>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>>>          eku: id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
>>>          pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
>>>          post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert
>>> "Server-Cert cert-pki-ca"
>>>          track: yes
>>>          auto-renew: yes
>>> Request ID '20180315021505':
>>>          status: MONITORING
>>>          stuck: no
>>>          key pair storage:
>>> type=NSSDB,location='/etc/dirsrv/slapd-DOMAIN-COM',nickname='Server-Cert',token='NSS
>>> Certificate DB',pinfile='/etc/dirsrv/slapd-DOMAIN-COM/pwdfile.txt'
>>>          certificate:
>>> type=NSSDB,location='/etc/dirsrv/slapd-DOMAIN-COM',nickname='Server-Cert',token='NSS
>>> Certificate DB'
>>>          CA: IPA
>>>          issuer: CN=Certificate Authority,O=DOMAIN.COM
>>>          subject: CN=ds01.domain.com,O=DOMAIN.COM
>>>          expires: 2020-03-07 08:49:36 UTC
>>>          principal name: ldap/ds01.domain.com@DOMAIN.COM
>>>          key usage:
>>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>>>          eku: id-kp-serverAuth,id-kp-clientAuth
>>>          pre-save command:
>>>          post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv
>>> DOMAIN-COM
>>>          track: yes
>>>          auto-renew: yes
>>> Request ID '20180315021510':
>>>          status: MONITORING
>>>          stuck: no
>>>          key pair storage:
>>> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
>>> Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
>>>          certificate:
>>> type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
>>> Certificate DB'
>>>          CA: IPA
>>>          issuer: CN=Certificate Authority,O=DOMAIN.COM
>>>          subject: CN=ds01.domain.com,O=DOMAIN.COM
>>>          expires: 2020-03-07 08:49:51 UTC
>>>          principal name: HTTP/ds01.domain.com@DOMAIN.COM
>>>          key usage:
>>> digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
>>>          eku: id-kp-serverAuth,id-kp-clientAuth
>>>          pre-save command:
>>>          post-save command: /usr/libexec/ipa/certmonger/restart_httpd
>>>          track: yes
>>>          auto-renew: yes
>>>
>>> thank you,
>>> Bhavin
>>>
>>> _______________________________________________
>>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
>>> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
>>> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>>> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
>>>
>>
>>
>>
>> _______________________________________________
>> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
>> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
>> Fedora Code of Conduct: https://getfedora.org/code-of-conduct.html
>> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
>> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
>>
>