... and just to show that I am a man of my word...


Thanks again for all the help guys!
Scott


From: Scott Z. <sudz28@hotmail.com>
Sent: Thursday, August 13, 2020 9:58 AM
To: Fraser Tweedale <ftweedal@redhat.com>; FreeIPA users list <freeipa-users@lists.fedorahosted.org>
Cc: Rob Crittenden <rcritten@redhat.com>; Florence Blanc-Renaud <flo@redhat.com>; Alexander Scheel <ascheel@redhat.com>
Subject: Re: [Freeipa-users] Re: pki-tomcatd not starting
 
Taking it from the top again this morning 🙂 
1) Stopped NTPD
2) Checked status of certmonger (running, several "Error 60 connecting to <server>: Peer certificate cannot be authenticated with given CA certificates)
3) Checked that debug.level=0 in CS.cfg (it was already set there)
4) Checked the 'not after' dates for auditSigningCert, ocspSigningCert, subsystemSigningCert, and Server-Cert... all are still valid except for Server-Cert (Not After Sep. 26 2019), so I set the system date back to Sept. 25, 2019.
5) Stopped all ipa services (ipactl stop)
6) Restated IPA services manually in the specified order (dirsrv@domain, krb5kdc, kadmin, named-pkcs11, httpd, pki-tomcatd@pki-tomcat)
7) Checked the status of all those services to make sure they were showing the new, older date of Sept. 25, 2019.
8) Ran the curl command to make sure it would work (curl --cacert /etc/ipa/ca.crt -v https://`hostname`:8443/ca/ee/ca/getCertChain)
9) Ran "ipa-getcert resubmi -i <reqID>" for the Server-Cert cert-pki-ca cert I was trying to renew.
10) Ran "getcert list" a couple of times; first saw that it was showing "SUBMITTING", the it switched after several seconds to "POST_SAVED_CERT" - WOW!!!  SOMETHING NEW!  Then, a few seconds later still, it's showing as "MONITORING".  Checking the "expires" date, it's FINALLY updated to Sept. 14, 2021!!!!!!

I cannot tell you how exciting this is for me.  What a journey.  Now, I assume I just need to "ipactl stop" and then "ipactl start", and all should be well I guess.  I'm afraid to do it though, I'm scared of the cert reverting or something funky 🙁






From: Fraser Tweedale <ftweedal@redhat.com>
Sent: Wednesday, August 12, 2020 6:02 PM
To: FreeIPA users list <freeipa-users@lists.fedorahosted.org>
Cc: Rob Crittenden <rcritten@redhat.com>; Florence Blanc-Renaud <flo@redhat.com>; Alexander Scheel <ascheel@redhat.com>; Scott Z. <sudz28@hotmail.com>
Subject: Re: [Freeipa-users] Re: pki-tomcatd not starting
 
On Thu, Aug 13, 2020 at 02:43:33AM +0000, Scott Z. via FreeIPA-users wrote:
> Just in case it helps to narrow things down a bit or answers questions...
> 1) The problem IdM server is the CA Master as far as I can tell (ran the command "ipa config-show", saw that the IPA CA renewal master: was the same server with the bad cert.
>
In any case, the CA renewal master setting shouldn't affect renewal
of the Dogtag "Server-Cert cert-pki-ca" certificate.  This is
because the TLS server certificates are not shared; each server
needs their own certificate.

> 2) Followed the steps in the Red Hat knowledge article at https://access.redhat.com/solutions/3357261
> 3) As noted at the bottom of that page, I had pretty good success up until the end.
>
> My current status is that I've done an ipactl restart --ignore-service-failure, my timedate value is once again current, and when I do a "getcert list" I see the offending cert (Server-Cert cert-pki-ca) listed as CA_UNREACHABLE, with a ca-error value of Internal Error and of course still showing an expiration date of Sep. 26, 2019.
>
> If I do a status check on the certmonger service I see lots of "Internal Error" messages along with "Unspecified GSS failure.  Minor code may provide more information, Minor (2529639068): Cannot contact any KDC for realm '<domain>'."
>
Was the KDC running at the time those certmonger GSS errors were
produced?  That could explain this error.

It would help to see the /etc/pki/pki-tomcat/ca/debug log:

- for the startup failures, that may indicate why Dogtag does not
  start up properly
- and for the time period during which renewal of the problematic
  certificate is attempted

Ensure PKI debug logging is at a verbose level.  In
/etc/pki/pki-tomcat/ca/CS.cfg, change the config:

    debug.level=0

It is counterintuitive but /lower/ number = higher verbosity.

It would help to see certmonger journal output (`journalctl -u
certmonger') covering the time period of the renewal attempt.

Also, just seeing the all the certificates in the various location
(especially Dogtag and dirsrv NSSDBs, including the CA certificates)
would be helpful.

I understand that you have security policies that may prevent you
share all this in a public list (or making extra work for you to
redact sensitive data).  If it would allow you to share more
logs/data, perhaps you could consider a commercial support
subscription with Red Hat.

Thanks,
Fraser