Thanks John.

This may give some more insight.
Anyone?

$ getcert list
Number of certificates and requests being tracked: 8.
Request ID '20150316184508':
status: NEED_TO_SUBMIT
ca-error: Error setting up ccache for "host" service on client using default keytab: Cannot contact any KDC for requested realm.
stuck: no
key pair storage: type=NSSDB,location='/etc/dirsrv/slapd-HQ-SPINQUE-COM',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/dirsrv/slapd-HQ-SPINQUE-COM/pwdfile.txt'
certificate: type=NSSDB,location='/etc/dirsrv/slapd-HQ-SPINQUE-COM',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=spinque04.hq.spinque.com,O=HQ.SPINQUE.COM
expires: 2017-03-16 18:45:07 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: 
post-save command: /usr/lib64/ipa/certmonger/restart_dirsrv HQ-SPINQUE-COM
track: yes
auto-renew: yes
Request ID '20150316184529':
status: CA_UNREACHABLE
ca-error: Error setting up ccache for "host" service on client using default keytab: Cannot contact any KDC for requested realm.
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS Certificate DB'
CA: IPA
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=spinque04.hq.spinque.com,O=HQ.SPINQUE.COM
expires: 2017-03-16 18:45:29 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: 
post-save command: /usr/lib64/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
Request ID '20160501114629':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=CA Audit,O=HQ.SPINQUE.COM
expires: 2019-01-26 19:42:21 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20160501114630':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=OCSP Subsystem,O=HQ.SPINQUE.COM
expires: 2019-01-26 19:41:30 UTC
eku: id-kp-OCSPSigning
pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "ocspSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20160501114631':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=CA Subsystem,O=HQ.SPINQUE.COM
expires: 2019-01-26 19:40:12 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20160501114632':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=Certificate Authority,O=HQ.SPINQUE.COM
expires: 2035-03-16 18:44:35 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "caSigningCert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20160501114633':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=IPA RA,O=HQ.SPINQUE.COM
expires: 2019-01-26 19:41:51 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/lib64/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/lib64/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
Request ID '20160501114634':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB',pin set
certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca',token='NSS Certificate DB'
CA: dogtag-ipa-renew-agent
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=spinque04.hq.spinque.com,O=HQ.SPINQUE.COM
expires: 2019-01-26 19:40:06 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/lib64/ipa/certmonger/stop_pkicad
post-save command: /usr/lib64/ipa/certmonger/renew_ca_cert "Server-Cert cert-pki-ca"
track: yes
auto-renew: yes


On Wed, 7 Jun 2017 at 15:16 John Keates via FreeIPA-users <freeipa-users@lists.fedorahosted.org> wrote:
Looks to me like Apache isn’t using the correct certificate, or the correct certificate was never installed. But I don’t know enough about FreeIPA’s certificate replacement process to known which one it is.
Aside from digging deeper and checking to see where Apache is looking for certificates and maybe manually refreshing it to see if the certificate gets replaced correctly this time I’m afraid someone else is going to have to jump in here.

John


On 7 Jun 2017, at 15:03, Roberto Cornacchia via FreeIPA-users <freeipa-users@lists.fedorahosted.org> wrote:

OK, I did so and httpd restarts.

$ openssl s_client -connect 127.0.0.1:443 -showcerts
CONNECTED(00000003)
depth=1 O = HQ.SPINQUE.COM, CN = Certificate Authority
verify return:1
verify error:num=10:certificate has expired
notAfter=Mar 16 18:45:29 2017 GMT
verify return:1
notAfter=Mar 16 18:45:29 2017 GMT
verify return:1
---
Certificate chain
   i:/O=HQ.SPINQUE.COM/CN=Certificate Authority
...

Fair enough, but why does this say it expires in 2019? Are they two different certificates? 

$ getcert list -d /etc/httpd/alias -n ipaCert
Number of certificates and requests being tracked: 8.
Request ID '20160501114633':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=IPA RA,O=HQ.SPINQUE.COM
expires: 2019-01-26 19:41:51 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/lib64/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/lib64/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes

What's the right way to solve this?


On Wed, 7 Jun 2017 at 14:52 John Keates via FreeIPA-users <freeipa-users@lists.fedorahosted.org> wrote:
I would suggest doing what the last line says: 

Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.

Then, you can check the certificates and maybe refresh it if it is actually expired.

John

On 7 Jun 2017, at 14:39, Roberto Cornacchia via FreeIPA-users <freeipa-users@lists.fedorahosted.org> wrote:

Things are getting worse.

First, the version I reported before was incorrect (taken from a client). Here's the server one.

$ ipa --version
VERSION: 4.2.4, API_VERSION: 2.156

I did a dnf update (Fedora 23). The IPA upgrade failed.
I tried running it again, manually, after a reboot:

$ ipa-server-upgrade 
session memcached servers not running
Upgrading IPA:
  [1/8]: saving configuration
  [2/8]: disabling listeners
  [3/8]: enabling DS global lock
  [4/8]: starting directory server
  [5/8]: updating schema
  [6/8]: upgrading server
Add failure attribute "cn" not allowed
  [7/8]: stopping directory server
  [8/8]: restoring configuration
Done.
Update complete
Upgrading IPA services
Upgrading the configuration of the IPA services
[Verifying that root certificate is published]
[Migrate CRL publish directory]
CRL tree already moved
[Verifying that CA proxy configuration is correct]
[Verifying that KDC configuration is using ipa-kdb backend]
[Fix DS schema file syntax]
Syntax already fixed
[Removing RA cert from DS NSS database]
RA cert already removed
[Enable sidgen and extdom plugins by default]
[Updating mod_nss protocol versions]
Protocol versions already updated
[Fixing trust flags in /etc/httpd/alias]
Trust flags already processed
[Exporting KRA agent PEM file]
KRA is not enabled
IPA server upgrade failed: Inspect /var/log/ipaupgrade.log and run command ipa-server-upgrade manually.
Unexpected error - see /var/log/ipaupgrade.log for details:
CalledProcessError: Command ''/bin/systemctl' 'start' 'httpd.service'' returned non-zero exit status 1

The ipaupgrade log only says that starting httpd failed.

HTTPD log says:

[Wed Jun 07 14:32:26.822478 2017] [core:notice] [pid 3182] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Wed Jun 07 14:32:26.823122 2017] [suexec:notice] [pid 3182] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Wed Jun 07 14:32:26.823467 2017] [:warn] [pid 3182] NSSSessionCacheTimeout is deprecated. Ignoring.
[Wed Jun 07 14:32:26.913923 2017] [:error] [pid 3182] SSL Library Error: -8181 Certificate has expired
[Wed Jun 07 14:32:26.913942 2017] [:error] [pid 3182] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.

Any suggestion?

On Wed, 7 Jun 2017 at 13:17 Roberto Cornacchia <roberto.cornacchia@gmail.com> wrote:
Not being able to login to the admin console, I checked the httpd log and found the following errors:

[Wed Jun 07 12:50:59.352022 2017] [:error] [pid 10240] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[Wed Jun 07 12:50:59.353372 2017] [:error] [pid 10237] SSL Library Error: -8181 Certificate has expired
[Wed Jun 07 12:50:59.353395 2017] [:error] [pid 10237] Unable to verify certificate 'Server-Cert'. Add "NSSEnforceValidCerts off" to nss.conf so the server can start until the problem can be resolved.
[Wed Jun 07 12:50:59.986025 2017] [core:error] [pid 11522] AH00546: no record of generation 47 of exiting child 10203

I also get an error during enrollment of a new client (which seems to retrieve a valid certificate anyway):

Password for admin@HQ.SPINQUE.COM
Successfully retrieved CA cert
    Subject:     CN=Certificate Authority,O=HQ.SPINQUE.COM
    Issuer:      CN=Certificate Authority,O=HQ.SPINQUE.COM
    Valid From:  Mon Mar 16 18:44:35 2015 UTC
    Valid Until: Fri Mar 16 18:44:35 2035 UTC

Joining realm failed: libcurl failed to execute the HTTP POST transaction, explaining:  TCP connection reset by peer

Services are up:

$ ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: RUNNING
ipa_memcached Service: RUNNING
httpd Service: RUNNING
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful


Certificate monitoring seems ok:

$ getcert list -d /etc/httpd/alias -n ipaCert
Number of certificates and requests being tracked: 8.
Request ID '20160501114633':
status: MONITORING
stuck: no
key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt'
certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=Certificate Authority,O=HQ.SPINQUE.COM
subject: CN=IPA RA,O=HQ.SPINQUE.COM
expires: 2019-01-26 19:41:51 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/lib64/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/lib64/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes

Version:

$ ipa --version
VERSION: 4.4.3, API_VERSION: 2.215

Could you please point me at what else to check?

_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org

_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org

_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org