Hi All,
I've been trying to work through this issue but can't find the magic formula to get it working so I'm turning to the community for help.
We are currently running VERSION: 4.4.0, API_VERSION: 2.213 in a 4 node multi master environment and the steps listed below were performed on the IPA CA renewal master. Please let me know if any additional information is needed regarding the environment.
As background we had expired certificates (both /etc/httpd/alias and /etc/pki/pki-tomcat/alias) which were renewed by setting the date in the past and restarting certmonger. Now on the IPA CA renewal master all certs have valid 'expires' dates and have a status of MONITORING. Note that the other nodes still have expired certs.
On the IPA CA renewal master when I start up the pki-tomcatd@pki-tomcat.service with the default CS.cfg and password.conf file I get the following error:
Internal Database Error encountered: Could not connect to LDAP server host francolin.local port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
So I changed the CS.cfg file to use Basic Auth and added the directory-manager-password to password.conf, restarted pki-tomcatd@pki-tomcat.service and now I get the following:
/var/log/messages
Jan 6 10:54:30 francolin systemd: Started PKI Tomcat Server pki-tomcat. Jan 6 10:54:30 francolin server: Java virtual machine used: /usr/lib/jvm/jre-1.8.0-openjdk/bin/java Jan 6 10:54:30 francolin server: classpath used: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar Jan 6 10:54:30 francolin server: main class used: org.apache.catalina.startup.Bootstrap Jan 6 10:54:30 francolin server: flags used: -DRESTEASY_LIB=/usr/share/java/resteasy-base -Djava.library.path=/usr/lib64/nuxwdog-jni Jan 6 10:54:30 francolin server: options used: -Dcatalina.base=/var/lib/pki/pki-tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp -Djava.util.logging.config.file=/var/lib/pki/pki-tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.security.manager -Djava.security.policy==/var/lib/pki/pki-tomcat/conf/catalina.policy Jan 6 10:54:30 francolin server: arguments used: start Jan 6 10:54:30 francolin server: WARNING: Problem with JAR file [/usr/share/pki/server/common/lib/symkey.jar], exists: [false], canRead: [false] Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'enableOCSP' to 'false' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspResponderURL' to 'http://francolin.local:9080/ca/ocsp' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspResponderCertNickname' to 'ocspSigningCert cert-pki-ca' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspCacheSize' to '1000' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspMinCacheEntryDuration' to '60' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspMaxCacheEntryDuration' to '120' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspTimeout' to '10' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'strictCiphers' to 'true' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslOptions' to 'ssl2=false,ssl3=false,tls=true' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ssl2Ciphers' to '-SSL2_RC4_128_WITH_MD5,-SSL2_RC4_128_EXPORT40_WITH_MD5,-SSL2_RC2_128_CBC_WITH_MD5,-SSL2_RC2_128_CBC_EXPORT40_WITH_MD5,-SSL2_DES_64_CBC_WITH_MD5,-SSL2_DES_192_EDE3_CBC_WITH_MD5' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ssl3Ciphers' to '-SSL3_FORTEZZA_DMS_WITH_NULL_SHA,-SSL3_FORTEZZA_DMS_WITH_RC4_128_SHA,+SSL3_RSA_WITH_RC4_128_SHA,-SSL3_RSA_EXPORT_WITH_RC4_40_MD5,+SSL3_RSA_WITH_3DES_EDE_CBC_SHA,-SSL3_RSA_WITH_DES_CBC_SHA,-SSL3_RSA_EXPORT_WITH_RC2_CBC_40_MD5,-SSL3_FORTEZZA_DMS_WITH_FORTEZZA_CBC_SHA,-SSL_RSA_FIPS_WITH_DES_CBC_SHA,+SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA,-SSL3_RSA_WITH_NULL_MD5,-TLS_RSA_EXPORT1024_WITH_RC4_56_SHA,-TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,+TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'tlsCiphers' to '-TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,-TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA,+TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,+TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,+TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,-TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,+TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,+TLS_RSA_WITH_3DES_EDE_CBC_SHA,+TLS_RSA_WITH_AES_128_CBC_SHA,+TLS_RSA_WITH_AES_256_CBC_SHA,+TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,+TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,-TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,-TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,+TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,+TLS_DHE_DSS_WITH_AES_128_CBC_SHA,+TLS_DHE_DSS_WITH_AES_256_CBC_SHA,+TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA,+TLS_DHE_RSA_WITH_AES_128_CBC_SHA,+TLS_DHE_RSA_WITH_AES_256_CBC_SHA' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslVersionRangeStream' to 'tls1_0:tls1_2' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslVersionRangeDatagram' to 'tls1_1:tls1_2' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslRangeCiphers' to '-TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,-TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,-TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,-TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,-TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,-TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,-TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,-TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,-TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,-TLS_DHE_DSS_WITH_AES_128_CBC_SHA,-TLS_DHE_DSS_WITH_AES_256_CBC_SHA,-TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA,-TLS_DHE_RSA_WITH_AES_128_CBC_SHA,-TLS_DHE_RSA_WITH_AES_256_CBC_SHA,-TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,-TLS_DHE_RSA_WITH_AES_256_CBC_SHA256,-TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,-TLS_DHE_DSS_WITH_AES_128_GCM_SHA256,-TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,-TLS_ECDHE_RSA_WITH_AES_128_CB C_SHA256,-TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,-TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,-TLS_RSA_WITH_AES_128_CBC_SHA256,-TLS_RSA_WITH_AES_256_CBC_SHA256,-TLS_RSA_WITH_AES_128_GCM_SHA256,+TLS_RSA_WITH_3DES_EDE_CBC_SHA,+TLS_RSA_WITH_AES_128_CBC_SHA,+TLS_RSA_WITH_AES_256_CBC_SHA' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'serverCertNickFile' to '/var/lib/pki/pki-tomcat/conf/serverCertNick.conf' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'passwordFile' to '/var/lib/pki/pki-tomcat/conf/password.conf' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'passwordClass' to 'org.apache.tomcat.util.net.jss.PlainPasswordFile' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'certdbDir' to '/var/lib/pki/pki-tomcat/alias' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'xmlValidation' to 'false' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'xmlNamespaceAware' to 'false' did not find a matching property. Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[before_init] Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[after_init] Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[before_start] Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[configure_start] Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[start] Jan 6 10:54:31 francolin server: SSLAuthenticatorWithFallback: Creating SSL authenticator with fallback Jan 6 10:54:31 francolin server: SSLAuthenticatorWithFallback: Setting container Jan 6 10:54:32 francolin server: SSLAuthenticatorWithFallback: Initializing authenticators Jan 6 10:54:32 francolin server: SSLAuthenticatorWithFallback: Starting authenticators Jan 6 10:54:33 francolin server: CMSEngine.initializePasswordStore() begins Jan 6 10:54:33 francolin server: CMSEngine.initializePasswordStore(): tag=internaldb Jan 6 10:54:33 francolin server: testLDAPConnection connecting to francolin.local:389 Jan 6 10:54:33 francolin server: CMSEngine.initializePasswordStore(): tag=replicationdb Jan 6 10:54:33 francolin server: testLDAPConnection connecting to francolin.local:389 Jan 6 10:54:33 francolin server: testLDAPConnection: The specified user cn=Replication Manager masterAgreement1-francolin.local-pki-tomcat,cn=config does not exist Jan 6 10:54:33 francolin server: CMSEngine: init(): password test execution failed for replicationdbwith NO_SUCH_USER. This may not be a latest instance. Ignoring .. Jan 6 10:54:34 francolin server: SelfTestSubsystem: Disabling "ca" subsystem due to selftest failure. Jan 6 10:54:34 francolin server: ----------------------- Jan 6 10:54:34 francolin server: Disabled "ca" subsystem Jan 6 10:54:34 francolin server: ----------------------- Jan 6 10:54:34 francolin server: Subsystem ID: ca Jan 6 10:54:34 francolin server: Instance ID: pki-tomcat Jan 6 10:54:34 francolin server: Enabled: False Jan 6 10:54:34 francolin server: Invalid class name repositorytop Jan 6 10:54:35 francolin server: Invalid class name repositorytop Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.DBRegistry.createObject(DBRegistry.java:485) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.DBSSession.read(DBSSession.java:167) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.DBSSession.read(DBSSession.java:137) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.Repository.getSerialNumber(Repository.java:125) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.Repository.initCache(Repository.java:244) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.Repository.checkRanges(Repository.java:460) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.apps.CMSEngine.startup(CMSEngine.java:1374) Jan 6 10:54:35 francolin server: at com.netscape.certsrv.apps.CMS.startup(CMS.java:201) Jan 6 10:54:35 francolin server: at com.netscape.certsrv.apps.CMS.start(CMS.java:1622) Jan 6 10:54:35 francolin server: at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) Jan 6 10:54:35 francolin server: at javax.servlet.GenericServlet.init(GenericServlet.java:158) Jan 6 10:54:35 francolin server: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Jan 6 10:54:35 francolin server: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Jan 6 10:54:35 francolin server: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Jan 6 10:54:35 francolin server: at java.lang.reflect.Method.invoke(Method.java:498) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285) Jan 6 10:54:35 francolin server: at java.security.AccessController.doPrivileged(Native Method) Jan 6 10:54:35 francolin server: at javax.security.auth.Subject.doAsPrivileged(Subject.java:549) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610) Jan 6 10:54:35 francolin server: at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145) Jan 6 10:54:35 francolin server: at java.security.AccessController.doPrivileged(Native Method) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652) Jan 6 10:54:35 francolin server: at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679) Jan 6 10:54:35 francolin server: at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966) Jan 6 10:54:35 francolin server: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) Jan 6 10:54:35 francolin server: at java.util.concurrent.FutureTask.run(FutureTask.java:266) Jan 6 10:54:35 francolin server: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) Jan 6 10:54:35 francolin server: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) Jan 6 10:54:35 francolin server: at java.lang.Thread.run(Thread.java:748) Jan 6 10:54:36 francolin server: PKIListener: org.apache.catalina.core.StandardServer[after_start] Jan 6 10:54:36 francolin server: PKIListener: Subsystem CA is disabled. Jan 6 10:54:36 francolin server: PKIListener: Check /var/log/pki/pki-tomcat/ca/selftests.log for possible errors. Jan 6 10:54:36 francolin server: PKIListener: To enable the subsystem: Jan 6 10:54:36 francolin server: PKIListener: pki-server subsystem-enable -i pki-tomcat ca Jan 6 10:54:47 francolin server: SSLAuthenticatorWithFallback: Stopping authenticators Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [LDAPConnThread-3 ldaps://francolin.local:389] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [LDAPConnThread-7 ldaps://francolin.local:389] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [authorityMonitor] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [LDAPConnThread-9 ldaps://francolin.local:389] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [profileChangeMonitor] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SSLAuthenticatorWithFallback: Setting container
/var/log/pki/pki-tomcat/ca/selftests.log
0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: Initializing self test plugins: 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading all self test plugin logger parameters 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading all self test plugin instances 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading all self test plugin instance parameters 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading self test plugins in on-demand order 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading self test plugins in startup order 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: Self test plugins have been successfully loaded! 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: Running self test plugins specified to be executed at startup: 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] CAPresence: CA is present 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SystemCertsVerification: system certs verification failure: Certificate not found: auditSigningCert cert-pki-ca 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: The CRITICAL self test plugin called selftests.container.instance.SystemCertsVerification running at startup FAILED!
I can also confirm that the 'auditSigningCert cert-pka-ca' isn't there when I run certutil -L -d /etc/pki/pki-tomcat/alias/. The output is listed below:
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
subsystemCert cert-pki-ca ,, Server-Cert cert-pki-ca u,u,u caSigningCert cert-pki-ca CTu,Cu,Cu ocspSigningCert cert-pki-ca u,u,u
The 'auditSigningCert cert-pka-ca' shows up on the other nodes however:
auditSigningCert cert-pki-ca u,u,Pu
Let me know if there is more information that is needed. This one is baffling me.
Thanks,
Ferdinand
On 1/6/20 10:12 PM, Ferdinand Babas via FreeIPA-users wrote:
Hi All,
I've been trying to work through this issue but can't find the magic formula to get it working so I'm turning to the community for help.
We are currently running VERSION: 4.4.0, API_VERSION: 2.213 in a 4 node multi master environment and the steps listed below were performed on the IPA CA renewal master. Please let me know if any additional information is needed regarding the environment.
As background we had expired certificates (both /etc/httpd/alias and /etc/pki/pki-tomcat/alias) which were renewed by setting the date in the past and restarting certmonger. Now on the IPA CA renewal master all certs have valid 'expires' dates and have a status of MONITORING. Note that the other nodes still have expired certs.
On the IPA CA renewal master when I start up the pki-tomcatd@pki-tomcat.service with the default CS.cfg and password.conf file I get the following error:
Internal Database Error encountered: Could not connect to LDAP server host francolin.local port 636 Error netscape.ldap.LDAPException: Authentication failed (48)
Hi, this error usually happens when there are issues with the subsystemCert cert-pki-ca. According to your certutil output, the cert is missing u,u,u flags, can you check if the associated key is present? - Get the password to access the key: $ sudo grep internal /var/lib/pki/pki-tomcat/conf/password.conf | cut -d= -f2 > /tmp/pwdfile.txt
- Check if the key is present in the NSSdb: $ sudo certutil -K -d /etc/pki/pki-tomcat/alias -f /tmp/pwdfile.txt | grep subsystemCert
If the key is there, you should see a line with < id> rsa 81c..d04 subsystemCert cert-pki-ca
If the key is missing you can recover it from another node where the CA is installed.
So I changed the CS.cfg file to use Basic Auth and added the directory-manager-password to password.conf, restarted pki-tomcatd@pki-tomcat.service and now I get the following:
/var/log/messages
Jan 6 10:54:30 francolin systemd: Started PKI Tomcat Server pki-tomcat. Jan 6 10:54:30 francolin server: Java virtual machine used: /usr/lib/jvm/jre-1.8.0-openjdk/bin/java Jan 6 10:54:30 francolin server: classpath used: /usr/share/tomcat/bin/bootstrap.jar:/usr/share/tomcat/bin/tomcat-juli.jar:/usr/share/java/commons-daemon.jar Jan 6 10:54:30 francolin server: main class used: org.apache.catalina.startup.Bootstrap Jan 6 10:54:30 francolin server: flags used: -DRESTEASY_LIB=/usr/share/java/resteasy-base -Djava.library.path=/usr/lib64/nuxwdog-jni Jan 6 10:54:30 francolin server: options used: -Dcatalina.base=/var/lib/pki/pki-tomcat -Dcatalina.home=/usr/share/tomcat -Djava.endorsed.dirs= -Djava.io.tmpdir=/var/lib/pki/pki-tomcat/temp -Djava.util.logging.config.file=/var/lib/pki/pki-tomcat/conf/logging.properties -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager -Djava.security.manager -Djava.security.policy==/var/lib/pki/pki-tomcat/conf/catalina.policy Jan 6 10:54:30 francolin server: arguments used: start Jan 6 10:54:30 francolin server: WARNING: Problem with JAR file [/usr/share/pki/server/common/lib/symkey.jar], exists: [false], canRead: [false] Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'enableOCSP' to 'false' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspResponderURL' to 'http://francolin.local:9080/ca/ocsp' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspResponderCertNickname' to 'ocspSigningCert cert-pki-ca' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspCacheSize' to '1000' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspMinCacheEntryDuration' to '60' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspMaxCacheEntryDuration' to '120' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ocspTimeout' to '10' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'strictCiphers' to 'true' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslOptions' to 'ssl2=false,ssl3=false,tls=true' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ssl2Ciphers' to '-SSL2_RC4_128_WITH_MD5,-SSL2_RC4_128_EXPORT40_WITH_MD5,-SSL2_RC2_128_CBC_WITH_MD5,-SSL2_RC2_128_CBC_EXPORT40_WITH_MD5,-SSL2_DES_64_CBC_WITH_MD5,-SSL2_DES_192_EDE3_CBC_WITH_MD5' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'ssl3Ciphers' to '-SSL3_FORTEZZA_DMS_WITH_NULL_SHA,-SSL3_FORTEZZA_DMS_WITH_RC4_128_SHA,+SSL3_RSA_WITH_RC4_128_SHA,-SSL3_RSA_EXPORT_WITH_RC4_40_MD5,+SSL3_RSA_WITH_3DES_EDE_CBC_SHA,-SSL3_RSA_WITH_DES_CBC_SHA,-SSL3_RSA_EXPORT_WITH_RC2_CBC_40_MD5,-SSL3_FORTEZZA_DMS_WITH_FORTEZZA_CBC_SHA,-SSL_RSA_FIPS_WITH_DES_CBC_SHA,+SSL_RSA_FIPS_WITH_3DES_EDE_CBC_SHA,-SSL3_RSA_WITH_NULL_MD5,-TLS_RSA_EXPORT1024_WITH_RC4_56_SHA,-TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,+TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'tlsCiphers' to '-TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,-TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA,+TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,+TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,+TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,-TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,+TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,+TLS_RSA_WITH_3DES_EDE_CBC_SHA,+TLS_RSA_WITH_AES_128_CBC_SHA,+TLS_RSA_WITH_AES_256_CBC_SHA,+TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,+TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,-TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,-TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,+TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,+TLS_DHE_DSS_WITH_AES_128_CBC_SHA,+TLS_DHE_DSS_WITH_AES_256_CBC_SHA,+TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA,+TLS_DHE_RSA_WITH_AES_128_CBC_SHA,+TLS_DHE_RSA_WITH_AES_256_CBC_SHA' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslVersionRangeStream' to 'tls1_0:tls1_2' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslVersionRangeDatagram' to 'tls1_1:tls1_2' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'sslRangeCiphers' to '-TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA,-TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDH_RSA_WITH_AES_128_CBC_SHA,-TLS_ECDH_RSA_WITH_AES_256_CBC_SHA,-TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA,-TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,-TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,-TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA,-TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,-TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,-TLS_DHE_DSS_WITH_3DES_EDE_CBC_SHA,-TLS_DHE_DSS_WITH_AES_128_CBC_SHA,-TLS_DHE_DSS_WITH_AES_256_CBC_SHA,-TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA,-TLS_DHE_RSA_WITH_AES_128_CBC_SHA,-TLS_DHE_RSA_WITH_AES_256_CBC_SHA,-TLS_DHE_RSA_WITH_AES_128_CBC_SHA256,-TLS_DHE_RSA_WITH_AES_256_CBC_SHA256,-TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,-TLS_DHE_DSS_WITH_AES_128_GCM_SHA256,-TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256,-TLS_ECDHE_RSA_WITH_AES_128_CB C_SHA256,-TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,-TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,-TLS_RSA_WITH_AES_128_CBC_SHA256,-TLS_RSA_WITH_AES_256_CBC_SHA256,-TLS_RSA_WITH_AES_128_GCM_SHA256,+TLS_RSA_WITH_3DES_EDE_CBC_SHA,+TLS_RSA_WITH_AES_128_CBC_SHA,+TLS_RSA_WITH_AES_256_CBC_SHA' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'serverCertNickFile' to '/var/lib/pki/pki-tomcat/conf/serverCertNick.conf' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'passwordFile' to '/var/lib/pki/pki-tomcat/conf/password.conf' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'passwordClass' to 'org.apache.tomcat.util.net.jss.PlainPasswordFile' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'certdbDir' to '/var/lib/pki/pki-tomcat/alias' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'xmlValidation' to 'false' did not find a matching property. Jan 6 10:54:31 francolin server: WARNING: [SetPropertiesRule]{Server/Service/Engine/Host} Setting property 'xmlNamespaceAware' to 'false' did not find a matching property. Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[before_init] Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[after_init] Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[before_start] Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[configure_start] Jan 6 10:54:31 francolin server: PKIListener: org.apache.catalina.core.StandardServer[start] Jan 6 10:54:31 francolin server: SSLAuthenticatorWithFallback: Creating SSL authenticator with fallback Jan 6 10:54:31 francolin server: SSLAuthenticatorWithFallback: Setting container Jan 6 10:54:32 francolin server: SSLAuthenticatorWithFallback: Initializing authenticators Jan 6 10:54:32 francolin server: SSLAuthenticatorWithFallback: Starting authenticators Jan 6 10:54:33 francolin server: CMSEngine.initializePasswordStore() begins Jan 6 10:54:33 francolin server: CMSEngine.initializePasswordStore(): tag=internaldb Jan 6 10:54:33 francolin server: testLDAPConnection connecting to francolin.local:389 Jan 6 10:54:33 francolin server: CMSEngine.initializePasswordStore(): tag=replicationdb Jan 6 10:54:33 francolin server: testLDAPConnection connecting to francolin.local:389 Jan 6 10:54:33 francolin server: testLDAPConnection: The specified user cn=Replication Manager masterAgreement1-francolin.local-pki-tomcat,cn=config does not exist Jan 6 10:54:33 francolin server: CMSEngine: init(): password test execution failed for replicationdbwith NO_SUCH_USER. This may not be a latest instance. Ignoring .. Jan 6 10:54:34 francolin server: SelfTestSubsystem: Disabling "ca" subsystem due to selftest failure. Jan 6 10:54:34 francolin server: ----------------------- Jan 6 10:54:34 francolin server: Disabled "ca" subsystem Jan 6 10:54:34 francolin server: ----------------------- Jan 6 10:54:34 francolin server: Subsystem ID: ca Jan 6 10:54:34 francolin server: Instance ID: pki-tomcat Jan 6 10:54:34 francolin server: Enabled: False Jan 6 10:54:34 francolin server: Invalid class name repositorytop Jan 6 10:54:35 francolin server: Invalid class name repositorytop Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.DBRegistry.createObject(DBRegistry.java:485) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.DBSSession.read(DBSSession.java:167) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.DBSSession.read(DBSSession.java:137) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.Repository.getSerialNumber(Repository.java:125) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.Repository.initCache(Repository.java:244) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.dbs.Repository.checkRanges(Repository.java:460) Jan 6 10:54:35 francolin server: at com.netscape.cmscore.apps.CMSEngine.startup(CMSEngine.java:1374) Jan 6 10:54:35 francolin server: at com.netscape.certsrv.apps.CMS.startup(CMS.java:201) Jan 6 10:54:35 francolin server: at com.netscape.certsrv.apps.CMS.start(CMS.java:1622) Jan 6 10:54:35 francolin server: at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:114) Jan 6 10:54:35 francolin server: at javax.servlet.GenericServlet.init(GenericServlet.java:158) Jan 6 10:54:35 francolin server: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) Jan 6 10:54:35 francolin server: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) Jan 6 10:54:35 francolin server: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) Jan 6 10:54:35 francolin server: at java.lang.reflect.Method.invoke(Method.java:498) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285) Jan 6 10:54:35 francolin server: at java.security.AccessController.doPrivileged(Native Method) Jan 6 10:54:35 francolin server: at javax.security.auth.Subject.doAsPrivileged(Subject.java:549) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175) Jan 6 10:54:35 francolin server: at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1270) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1195) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1085) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5318) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5610) Jan 6 10:54:35 francolin server: at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145) Jan 6 10:54:35 francolin server: at java.security.AccessController.doPrivileged(Native Method) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873) Jan 6 10:54:35 francolin server: at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652) Jan 6 10:54:35 francolin server: at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679) Jan 6 10:54:35 francolin server: at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966) Jan 6 10:54:35 francolin server: at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) Jan 6 10:54:35 francolin server: at java.util.concurrent.FutureTask.run(FutureTask.java:266) Jan 6 10:54:35 francolin server: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) Jan 6 10:54:35 francolin server: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) Jan 6 10:54:35 francolin server: at java.lang.Thread.run(Thread.java:748) Jan 6 10:54:36 francolin server: PKIListener: org.apache.catalina.core.StandardServer[after_start] Jan 6 10:54:36 francolin server: PKIListener: Subsystem CA is disabled. Jan 6 10:54:36 francolin server: PKIListener: Check /var/log/pki/pki-tomcat/ca/selftests.log for possible errors. Jan 6 10:54:36 francolin server: PKIListener: To enable the subsystem: Jan 6 10:54:36 francolin server: PKIListener: pki-server subsystem-enable -i pki-tomcat ca Jan 6 10:54:47 francolin server: SSLAuthenticatorWithFallback: Stopping authenticators Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [LDAPConnThread-3 ldaps://francolin.local:389] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [LDAPConnThread-7 ldaps://francolin.local:389] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [authorityMonitor] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [LDAPConnThread-9 ldaps://francolin.local:389] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SEVERE: The web application [/ca] appears to have started a thread named [profileChangeMonitor] but has failed to stop it. This is very likely to create a memory leak. Jan 6 10:54:47 francolin server: SSLAuthenticatorWithFallback: Setting container
/var/log/pki/pki-tomcat/ca/selftests.log
0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: Initializing self test plugins: 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading all self test plugin logger parameters 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading all self test plugin instances 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading all self test plugin instance parameters 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading self test plugins in on-demand order 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: loading self test plugins in startup order 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: Self test plugins have been successfully loaded! 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: Running self test plugins specified to be executed at startup: 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] CAPresence: CA is present 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SystemCertsVerification: system certs verification failure: Certificate not found: auditSigningCert cert-pki-ca 0.localhost-startStop-2 - [06/Jan/2020:10:56:29 HST] [20] [1] SelfTestSubsystem: The CRITICAL self test plugin called selftests.container.instance.SystemCertsVerification running at startup FAILED!
I can also confirm that the 'auditSigningCert cert-pka-ca' isn't there when I run certutil -L -d /etc/pki/pki-tomcat/alias/. The output is listed below:
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
subsystemCert cert-pki-ca ,, Server-Cert cert-pki-ca u,u,u caSigningCert cert-pki-ca CTu,Cu,Cu ocspSigningCert cert-pki-ca u,u,u
The 'auditSigningCert cert-pka-ca' shows up on the other nodes however:
auditSigningCert cert-pki-ca u,u,Pu
The 'auditSigningCert cert-pki-ca' is the same on all the nodes with CA. If you are able to get it from another machine, you can import the cert on this master. First check if the key is present (with certutil -K), and if needed also get the key from another node.
To export cert+key from a NSSDB to a p12file: pk12util -d /etc/pki/pki-tomcat/alias -n 'auditSigningCert cert-pki-ca' -o file.p12 To import cert.key from a p12 file to a NSSDB: pk12util -d /etc/pki/pki-tomcat/alias -i file.p12
HTH, flo
Let me know if there is more information that is needed. This one is baffling me.
Thanks,
Ferdinand _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Hi Flo,
Thanks for the response.
On 1/6/20 10:12 PM, Ferdinand Babas via FreeIPA-users wrote: Hi, this error usually happens when there are issues with the subsystemCert cert-pki-ca. According to your certutil output, the cert is missing u,u,u flags, can you check if the associated key is present?
- Get the password to access the key:
$ sudo grep internal /var/lib/pki/pki-tomcat/conf/password.conf | cut -d= -f2 > /tmp/pwdfile.txt
- Check if the key is present in the NSSdb:
$ sudo certutil -K -d /etc/pki/pki-tomcat/alias -f /tmp/pwdfile.txt | grep subsystemCert
If the key is there, you should see a line with < id> rsa 81c..d04 subsystemCert cert-pki-ca
The above steps failed for me with the following:
Incorrect password/PIN entered. certutil: could not authenticate to token NSS Certificate DB.: SEC_ERROR_BAD_PASSWORD: The security password entered is incorrect.
The above steps did work with the other 3 nodes however:
< 3> rsa 48f4a4...2c126e NSS Certificate DB:subsystemCert cert-pki-ca
If the key is missing you can recover it from another node where the CA is installed.
How would I recover the key from the other nodes?
The 'auditSigningCert cert-pki-ca' is the same on all the nodes with CA. If you are able to get it from another machine, you can import the cert on this master. First check if the key is present (with certutil -K), and if needed also get the key from another node.
To export cert+key from a NSSDB to a p12file: pk12util -d /etc/pki/pki-tomcat/alias -n 'auditSigningCert cert-pki-ca' -o file.p12 To import cert.key from a p12 file to a NSSDB: pk12util -d /etc/pki/pki-tomcat/alias -i file.p12
Regarding the 'auditSigningCert cert-pki-ca' I ran the export command you provided but got stuck when prompted for the password (Enter Password or Pin for "NSS Certificate DB":). What should I do if I have no idea what the password is?
HTH, flo
Thanks so much for your help!!
Ferdinand
On 1/8/20 3:30 AM, Ferdinand Babas via FreeIPA-users wrote:
Hi Flo,
Thanks for the response.
On 1/6/20 10:12 PM, Ferdinand Babas via FreeIPA-users wrote: Hi, this error usually happens when there are issues with the subsystemCert cert-pki-ca. According to your certutil output, the cert is missing u,u,u flags, can you check if the associated key is present?
- Get the password to access the key:
$ sudo grep internal /var/lib/pki/pki-tomcat/conf/password.conf | cut -d= -f2 > /tmp/pwdfile.txt
- Check if the key is present in the NSSdb:
$ sudo certutil -K -d /etc/pki/pki-tomcat/alias -f /tmp/pwdfile.txt | grep subsystemCert
If the key is there, you should see a line with < id> rsa 81c..d04 subsystemCert cert-pki-ca
The above steps failed for me with the following:
Incorrect password/PIN entered. certutil: could not authenticate to token NSS Certificate DB.: SEC_ERROR_BAD_PASSWORD: The security password entered is incorrect.
Do you have the file /var/lib/pki/pki-tomcat/conf/password.conf ? Its content is usually: internal=<password> replicationdb=<number>
If it's empty/missing, you can also check if there is a /etc/pki/pki-tomcat/alias/pwdfile.txt which should contain the password.
You really need this password if you want to manage the private keys inside the NSS DB.
The above steps did work with the other 3 nodes however:
< 3> rsa 48f4a4...2c126e NSS Certificate DB:subsystemCert cert-pki-ca
If the key is missing you can recover it from another node where the CA is installed.
How would I recover the key from the other nodes?
Export key + cert with pk12util from an other node, then import. But you will need the NSSDB password.
The 'auditSigningCert cert-pki-ca' is the same on all the nodes with CA. If you are able to get it from another machine, you can import the cert on this master. First check if the key is present (with certutil -K), and if needed also get the key from another node.
To export cert+key from a NSSDB to a p12file: pk12util -d /etc/pki/pki-tomcat/alias -n 'auditSigningCert cert-pki-ca' -o file.p12 To import cert.key from a p12 file to a NSSDB: pk12util -d /etc/pki/pki-tomcat/alias -i file.p12
Regarding the 'auditSigningCert cert-pki-ca' I ran the export command you provided but got stuck when prompted for the password (Enter Password or Pin for "NSS Certificate DB":). What should I do if I have no idea what the password is?
The password is the NSS DB's password, stored in /var/lib/pki/pki-tomcat/conf/password.conf or /etc/pki/pki-tomcat/alias/pwdfile.txt.
flo
HTH, flo
Thanks so much for your help!!
Ferdinand _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
On 1/8/20 3:30 AM, Ferdinand Babas via FreeIPA-users wrote: Do you have the file /var/lib/pki/pki-tomcat/conf/password.conf ? Its content is usually: internal=<password> replicationdb=<number>
If it's empty/missing, you can also check if there is a /etc/pki/pki-tomcat/alias/pwdfile.txt which should contain the password.
You really need this password if you want to manage the private keys inside the NSS DB.
Export key + cert with pk12util from an other node, then import. But you will need the NSSDB password.
The password is the NSS DB's password, stored in /var/lib/pki/pki-tomcat/conf/password.conf or /etc/pki/pki-tomcat/alias/pwdfile.txt.
flo
Thanks for pointing me in the right direction. I was able to get the NSS DB password and was able to export and import the key and cert like you described.
I had to export and import both the 'auditSigningCert cert-pki-ca' and 'subsystemCert cert-pki-ca' and now on the CA Renewal Master I get the following:
# certutil -L -d /etc/pki/pki-tomcat/alias
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
Server-Cert cert-pki-ca u,u,u auditSigningCert cert-pki-ca u,u,Pu caSigningCert cert-pki-ca CTu,Cu,Cu ocspSigningCert cert-pki-ca u,u,u subsystemCert cert-pki-ca u,u,u
# certutil -K -d /etc/pki/pki-tomcat/alias -f /tmp/pwdfile.txt
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa cc9d67...53abe6 (orphan) < 1> rsa 7b55ea...d259f7 auditSigningCert cert-pki-ca < 2> rsa f06e29...d0fb44 NSS Certificate DB:Server-Cert cert-pki-ca < 3> rsa bcc164...94ef97 NSS Certificate DB:ocspSigningCert cert-pki-ca < 4> rsa e49b1d...a6a30f NSS Certificate DB:caSigningCert cert-pki-ca < 5> rsa 48f4a4...2c126e subsystemCert cert-pki-ca
Now the the certs that were imported are expired:
# certutil -L -d /etc/pki/pki-tomcat/alias/ -n 'auditSigningCert cert-pki-ca'
Validity: Not Before: Wed Jun 14 20:45:05 2017 Not After : Tue Jun 04 20:45:05 2019
# certutil -L -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca'
Validity: Not Before: Thu Jun 29 04:28:11 2017 Not After : Wed Jun 19 04:28:11 2019
So I changed the date to prior to Jun 04 2019
# timedatectl set-time 2019-06-01
Restarted the pki-tomcatd service and the certmonger service and now I get the following:
ipactl status
Directory Service: RUNNING krb5kdc Service: RUNNING kadmin Service: RUNNING ipa_memcached Service: RUNNING httpd Service: RUNNING ipa-custodia Service: RUNNING ntpd Service: STOPPED pki-tomcatd Service: RUNNING <- Running... Yay!! ipa-otpd Service: RUNNING ipa: INFO: The ipactl command was successful
From /var/log/messages:
Jun 1 03:21:49 francolin systemd: Stopping Certificate monitoring and PKI enrollment... Jun 1 03:21:49 francolin certmonger: 2019-06-01 00:17:28 [24501] Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation2019-06-01 00:38:13 [24501] Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation Jun 1 03:21:49 francolin systemd: Starting Certificate monitoring and PKI enrollment... Jun 1 03:21:50 francolin systemd: Started Certificate monitoring and PKI enrollment. Jun 1 03:21:51 francolin ns-slapd: [01/Jun/2019:03:21:51.309397072 -1000] csngen_new_csn - Warning: too much time skew (-19140576 secs). Current seqnum=a Jun 1 03:21:51 francolin ns-slapd: [01/Jun/2019:03:21:51.372397615 -1000] csngen_new_csn - Warning: too much time skew (-19140576 secs). Current seqnum=b Jun 1 03:23:50 francolin ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid) Jun 1 03:23:50 francolin ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid) Jun 1 03:23:50 francolin ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid)
# getcert list ... Request ID '20170614062601': status: MONITORING ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=CA Audit,O=LOCAL expires: 2019-06-04 20:45:05 UTC key usage: digitalSignature,nonRepudiation pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca" track: yes auto-renew: yes ... Request ID '20170614062603': status: MONITORING ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=CA Subsystem,O=LOCAL expires: 2019-06-19 04:28:11 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca" track: yes auto-renew: yes
This is the furthest I've gotten and I appreciate all the help you've provided. Hopefully a few more tips will get me going.
Thanks so much!!
Ferdinand
On 1/9/20 6:44 AM, Ferdinand Babas via FreeIPA-users wrote:
On 1/8/20 3:30 AM, Ferdinand Babas via FreeIPA-users wrote: Do you have the file /var/lib/pki/pki-tomcat/conf/password.conf ? Its content is usually: internal=<password> replicationdb=<number>
If it's empty/missing, you can also check if there is a /etc/pki/pki-tomcat/alias/pwdfile.txt which should contain the password.
You really need this password if you want to manage the private keys inside the NSS DB.
Export key + cert with pk12util from an other node, then import. But you will need the NSSDB password.
The password is the NSS DB's password, stored in /var/lib/pki/pki-tomcat/conf/password.conf or /etc/pki/pki-tomcat/alias/pwdfile.txt.
flo
Thanks for pointing me in the right direction. I was able to get the NSS DB password and was able to export and import the key and cert like you described.
I had to export and import both the 'auditSigningCert cert-pki-ca' and 'subsystemCert cert-pki-ca' and now on the CA Renewal Master I get the following:
# certutil -L -d /etc/pki/pki-tomcat/alias
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
Server-Cert cert-pki-ca u,u,u auditSigningCert cert-pki-ca u,u,Pu caSigningCert cert-pki-ca CTu,Cu,Cu ocspSigningCert cert-pki-ca u,u,u subsystemCert cert-pki-ca u,u,u
# certutil -K -d /etc/pki/pki-tomcat/alias -f /tmp/pwdfile.txt
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa cc9d67...53abe6 (orphan) < 1> rsa 7b55ea...d259f7 auditSigningCert cert-pki-ca < 2> rsa f06e29...d0fb44 NSS Certificate DB:Server-Cert cert-pki-ca < 3> rsa bcc164...94ef97 NSS Certificate DB:ocspSigningCert cert-pki-ca < 4> rsa e49b1d...a6a30f NSS Certificate DB:caSigningCert cert-pki-ca < 5> rsa 48f4a4...2c126e subsystemCert cert-pki-ca
Now the the certs that were imported are expired:
# certutil -L -d /etc/pki/pki-tomcat/alias/ -n 'auditSigningCert cert-pki-ca'
Validity: Not Before: Wed Jun 14 20:45:05 2017 Not After : Tue Jun 04 20:45:05 2019# certutil -L -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca'
Validity: Not Before: Thu Jun 29 04:28:11 2017 Not After : Wed Jun 19 04:28:11 2019So I changed the date to prior to Jun 04 2019
# timedatectl set-time 2019-06-01
Restarted the pki-tomcatd service and the certmonger service and now I get the following:
ipactl status
Directory Service: RUNNING krb5kdc Service: RUNNING kadmin Service: RUNNING ipa_memcached Service: RUNNING httpd Service: RUNNING ipa-custodia Service: RUNNING ntpd Service: STOPPED pki-tomcatd Service: RUNNING <- Running... Yay!! ipa-otpd Service: RUNNING ipa: INFO: The ipactl command was successful
From /var/log/messages:
Jun 1 03:21:49 francolin systemd: Stopping Certificate monitoring and PKI enrollment... Jun 1 03:21:49 francolin certmonger: 2019-06-01 00:17:28 [24501] Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation2019-06-01 00:38:13 [24501] Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation Jun 1 03:21:49 francolin systemd: Starting Certificate monitoring and PKI enrollment... Jun 1 03:21:50 francolin systemd: Started Certificate monitoring and PKI enrollment. Jun 1 03:21:51 francolin ns-slapd: [01/Jun/2019:03:21:51.309397072 -1000] csngen_new_csn - Warning: too much time skew (-19140576 secs). Current seqnum=a Jun 1 03:21:51 francolin ns-slapd: [01/Jun/2019:03:21:51.372397615 -1000] csngen_new_csn - Warning: too much time skew (-19140576 secs). Current seqnum=b Jun 1 03:23:50 francolin ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid) Jun 1 03:23:50 francolin ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid) Jun 1 03:23:50 francolin ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid)
# getcert list ... Request ID '20170614062601': status: MONITORING ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=CA Audit,O=LOCAL expires: 2019-06-04 20:45:05 UTC key usage: digitalSignature,nonRepudiation pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca" track: yes auto-renew: yes ... Request ID '20170614062603': status: MONITORING ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=CA Subsystem,O=LOCAL expires: 2019-06-19 04:28:11 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca" track: yes auto-renew: yes
This is the furthest I've gotten and I appreciate all the help you've provided. Hopefully a few more tips will get me going.
Hi,
you need to carefully pick the date in the past. At that given date, all your certs must be valid (ie notbefore < date < notafter). It's likely that you choose a date before the notbefore date of some of the certs.
flo
Thanks so much!!
Ferdinand _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
On 1/9/20 6:44 AM, Ferdinand Babas via FreeIPA-users wrote: Hi,
you need to carefully pick the date in the past. At that given date, all your certs must be valid (ie notbefore < date < notafter). It's likely that you choose a date before the notbefore date of some of the certs.
flo
Hi flo,
Still working on this and I'm unsure exactly what to do next. Here are the Not Before and Not After dates of all the certs:
/etc/dirsrv/slapd-CFHT-HAWAII-EDU,nickname='Server-Cert' Not Before: Sat May 18 19:15:24 2019 Not After : Tue May 18 19:15:24 2021
/etc/httpd/alias,nickname='Server-Cert' Not Before: Sat May 18 19:15:34 2019 Not After : Tue May 18 19:15:34 2021
/etc/httpd/alias,nickname='ipaCert' Not Before: Wed Jun 14 06:06:40 2017 Not After : Tue Jun 04 06:06:40 2019
/etc/pki/pki-tomcat/alias,nickname='auditSigningCert cert-pki-ca' Not Before: Wed Jun 14 20:45:05 2017 Not After : Tue Jun 04 20:45:05 2019
/etc/pki/pki-tomcat/alias,nickname='ocspSigningCert cert-pki-ca' Not Before: Sat Jun 01 10:29:31 2019 Not After : Fri May 21 10:29:31 2021
/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca Not Before: Thu Jun 29 04:28:11 2017 Not After : Wed Jun 19 04:28:11 2019
/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca Not Before: Wed Jul 22 14:25:13 2015 Not After : Sun Jul 22 14:25:13 2035
/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca' Not Before: Tue May 07 19:15:22 2019 Not After : Mon Apr 26 19:15:22 2021
From what I can tell setting the date to 2019-06-02 should be fine so I did that and restarted pki-tomcatd (which starts up fine when back dated).
When I restart certmonger I'm getting log messages of:
Jun 3 00:00:24 francolin ns-slapd: [03/Jun/2019:00:00:24.572219961 -1000] csngen_new_csn - Warning: too much time skew (-19483202 secs). Current seqnum=1 Jun 3 00:00:24 francolin ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid) Jun 3 00:00:24 francolin ns-slapd: [03/Jun/2019:00:00:24.612098416 -1000] csngen_new_csn - Warning: too much time skew (-19483203 secs). Current seqnum=1 Jun 3 00:00:24 francolin ns-slapd: [03/Jun/2019:00:00:24.628118429 -1000] csngen_new_csn - Warning: too much time skew (-19483204 secs). Current seqnum=1
And getcert list displays the following:
... Request ID '20170614061938': status: NEED_CSR_GEN_TOKEN stuck: yes key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=IPA RA,O=LOCAL expires: 2019-06-04 06:06:40 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes Request ID '20170614062601': status: MONITORING ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=CA Audit,O=LOCAL expires: 2019-06-04 20:45:05 UTC key usage: digitalSignature,nonRepudiation pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca" track: yes auto-renew: yes ... Request ID '20170614062603': status: MONITORING ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=CA Subsystem,O=LOCAL expires: 2019-06-19 04:28:11 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca" track: yes auto-renew: yes ...
Thanks for all of your help.
Ferdinand
On 1/14/20 11:41 PM, Ferdinand Babas via FreeIPA-users wrote:
On 1/9/20 6:44 AM, Ferdinand Babas via FreeIPA-users wrote: Hi,
you need to carefully pick the date in the past. At that given date, all your certs must be valid (ie notbefore < date < notafter). It's likely that you choose a date before the notbefore date of some of the certs.
flo
Hi flo,
Still working on this and I'm unsure exactly what to do next. Here are the Not Before and Not After dates of all the certs:
/etc/dirsrv/slapd-CFHT-HAWAII-EDU,nickname='Server-Cert' Not Before: Sat May 18 19:15:24 2019 Not After : Tue May 18 19:15:24 2021
/etc/httpd/alias,nickname='Server-Cert' Not Before: Sat May 18 19:15:34 2019 Not After : Tue May 18 19:15:34 2021
/etc/httpd/alias,nickname='ipaCert' Not Before: Wed Jun 14 06:06:40 2017 Not After : Tue Jun 04 06:06:40 2019
/etc/pki/pki-tomcat/alias,nickname='auditSigningCert cert-pki-ca' Not Before: Wed Jun 14 20:45:05 2017 Not After : Tue Jun 04 20:45:05 2019
/etc/pki/pki-tomcat/alias,nickname='ocspSigningCert cert-pki-ca' Not Before: Sat Jun 01 10:29:31 2019 Not After : Fri May 21 10:29:31 2021
/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca Not Before: Thu Jun 29 04:28:11 2017 Not After : Wed Jun 19 04:28:11 2019
/etc/pki/pki-tomcat/alias',nickname='caSigningCert cert-pki-ca Not Before: Wed Jul 22 14:25:13 2015 Not After : Sun Jul 22 14:25:13 2035
/etc/pki/pki-tomcat/alias',nickname='Server-Cert cert-pki-ca' Not Before: Tue May 07 19:15:22 2019 Not After : Mon Apr 26 19:15:22 2021
From what I can tell setting the date to 2019-06-02 should be fine so I did that and restarted pki-tomcatd (which starts up fine when back dated).
Agreed, any date between June 1 and June 4 should be ok.
When I restart certmonger I'm getting log messages of:
Jun 3 00:00:24 francolin ns-slapd: [03/Jun/2019:00:00:24.572219961 -1000] csngen_new_csn - Warning: too much time skew (-19483202 secs). Current seqnum=1 Jun 3 00:00:24 francolin ns-slapd: GSSAPI Error: Unspecified GSS failure. Minor code may provide more information (Ticket not yet valid) Jun 3 00:00:24 francolin ns-slapd: [03/Jun/2019:00:00:24.612098416 -1000] csngen_new_csn - Warning: too much time skew (-19483203 secs). Current seqnum=1 Jun 3 00:00:24 francolin ns-slapd: [03/Jun/2019:00:00:24.628118429 -1000] csngen_new_csn - Warning: too much time skew (-19483204 secs). Current seqnum=1
And getcert list displays the following:
... Request ID '20170614061938': status: NEED_CSR_GEN_TOKEN
ipaCert is the most important cert to renew and should be handled first. The man page for getcert-list explains this error as: NEED_CSR_GEN_TOKEN The service was unable to find the token in which the key pair is supposed to be stored.
So I would check if /etc/httpd/alias has the right permissions, if /etc/httpd/alias/pwdfile.txt contains the password for /etc/httpd/alias and has the right permissions. On a RHEL 7.2 system I can see: [root ~]# ls -ld /etc/httpd/alias/ drwxr-xr-x. 2 root root 4096 Jan 15 10:48 /etc/httpd/alias/ [root ~]# ls -l /etc/httpd/alias/ total 188 -r--r--r--. 1 root root 1427 Jan 15 10:48 cacert.asc -r--r--r--. 1 root root 1427 Jan 13 14:19 cacert.asc.orig -rw-rw----. 1 root apache 65536 Jan 15 10:49 cert8.db -rw-rw----. 1 root apache 65536 Jan 13 14:27 cert8.db.orig -rw-------. 1 root root 5872 Nov 16 2018 install.log -rw-rw----. 1 root apache 16384 Jan 15 10:49 key3.db -rw-rw----. 1 root apache 16384 Jan 13 14:27 key3.db.orig -r--r-----. 1 root apache 3481 Jan 15 10:47 kra-agent.pem lrwxrwxrwx. 1 root root 33 Nov 16 2018 libnssckbi.so -> ../../..//usr/lib64/libnssckbi.so -rw-rw----. 1 root apache 20 Jan 15 10:47 pwdfile.txt -rw-rw----. 1 root apache 20 Jan 13 14:18 pwdfile.txt.orig -rw-rw----. 1 root apache 16384 Jan 15 10:47 secmod.db -rw-rw----. 1 root apache 16384 Jan 13 14:18 secmod.db.orig
And to check the password: [root@ ~]# certutil -K -d /etc/httpd/alias/ -f /etc/httpd/alias/pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 1b212c867ebde61fdc295ecd63ef690cd93fc783 NSS Certificate DB:ipaCert < 1> rsa da7a2b61ed951d1cfff1309534a3232258da9487 NSS Certificate DB:Signing-Cert < 2> rsa e5fdd7ec9d8daa97566695ac562fb1697a811401 NSS Certificate DB:Server-Cert
(if the password file is wrong, you will see: Incorrect password/PIN entered)
flo
stuck: yes key pair storage: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB',pinfile='/etc/httpd/alias/pwdfile.txt' certificate: type=NSSDB,location='/etc/httpd/alias',nickname='ipaCert',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=IPA RA,O=LOCAL expires: 2019-06-04 06:06:40 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert track: yes auto-renew: yes Request ID '20170614062601': status: MONITORING ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=CA Audit,O=LOCAL expires: 2019-06-04 20:45:05 UTC key usage: digitalSignature,nonRepudiation pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigningCert cert-pki-ca" track: yes auto-renew: yes ... Request ID '20170614062603': status: MONITORING ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation stuck: no key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB' CA: dogtag-ipa-ca-renew-agent issuer: CN=Certificate Authority,O=LOCAL subject: CN=CA Subsystem,O=LOCAL expires: 2019-06-19 04:28:11 UTC key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment eku: id-kp-serverAuth,id-kp-clientAuth pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCert cert-pki-ca" track: yes auto-renew: yes ...
Thanks for all of your help.
Ferdinand _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
On 1/14/20 11:41 PM, Ferdinand Babas via FreeIPA-users wrote: Agreed, any date between June 1 and June 4 should be ok.
ipaCert is the most important cert to renew and should be handled first. The man page for getcert-list explains this error as: NEED_CSR_GEN_TOKEN The service was unable to find the token in which the key pair is supposed to be stored.
So I would check if /etc/httpd/alias has the right permissions, if /etc/httpd/alias/pwdfile.txt contains the password for /etc/httpd/alias and has the right permissions. On a RHEL 7.2 system I can see: [root ~]# ls -ld /etc/httpd/alias/ drwxr-xr-x. 2 root root 4096 Jan 15 10:48 /etc/httpd/alias/ [root ~]# ls -l /etc/httpd/alias/ total 188 -r--r--r--. 1 root root 1427 Jan 15 10:48 cacert.asc -r--r--r--. 1 root root 1427 Jan 13 14:19 cacert.asc.orig -rw-rw----. 1 root apache 65536 Jan 15 10:49 cert8.db -rw-rw----. 1 root apache 65536 Jan 13 14:27 cert8.db.orig -rw-------. 1 root root 5872 Nov 16 2018 install.log -rw-rw----. 1 root apache 16384 Jan 15 10:49 key3.db -rw-rw----. 1 root apache 16384 Jan 13 14:27 key3.db.orig -r--r-----. 1 root apache 3481 Jan 15 10:47 kra-agent.pem lrwxrwxrwx. 1 root root 33 Nov 16 2018 libnssckbi.so -> ../../..//usr/lib64/libnssckbi.so -rw-rw----. 1 root apache 20 Jan 15 10:47 pwdfile.txt -rw-rw----. 1 root apache 20 Jan 13 14:18 pwdfile.txt.orig -rw-rw----. 1 root apache 16384 Jan 15 10:47 secmod.db -rw-rw----. 1 root apache 16384 Jan 13 14:18 secmod.db.orig
And to check the password: [root@ ~]# certutil -K -d /etc/httpd/alias/ -f /etc/httpd/alias/pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 1b212c867ebde61fdc295ecd63ef690cd93fc783 NSS Certificate DB:ipaCert < 1> rsa da7a2b61ed951d1cfff1309534a3232258da9487 NSS Certificate DB:Signing-Cert < 2> rsa e5fdd7ec9d8daa97566695ac562fb1697a811401 NSS Certificate DB:Server-Cert
(if the password file is wrong, you will see: Incorrect password/PIN entered)
flo
Hi flo,
Everything looks okay as far as permissions but when I run the certutil -K -d /etc/httpd/alias/ -f /etc/httpd/alias/pwdfile.txt command I get the following:
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 0fa7d5a5d032d3d9bb0845425b4ce4c3588c5eba NSS Certificate DB:Server-Cert
ipaCert is missing from the output, but it is there when I run certutil -L -d /etc/httpd/alias/:
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
Server-Cert u,u,u LOCAL IPA CA CT,C,C ipaCert ,,
It is missing the trust attributes though? How would I resolve that?
Thanks,
Ferdinand
On 1/16/20 12:26 AM, Ferdinand Babas via FreeIPA-users wrote:
On 1/14/20 11:41 PM, Ferdinand Babas via FreeIPA-users wrote: Agreed, any date between June 1 and June 4 should be ok.
ipaCert is the most important cert to renew and should be handled first. The man page for getcert-list explains this error as: NEED_CSR_GEN_TOKEN The service was unable to find the token in which the key pair is supposed to be stored.
So I would check if /etc/httpd/alias has the right permissions, if /etc/httpd/alias/pwdfile.txt contains the password for /etc/httpd/alias and has the right permissions. On a RHEL 7.2 system I can see: [root ~]# ls -ld /etc/httpd/alias/ drwxr-xr-x. 2 root root 4096 Jan 15 10:48 /etc/httpd/alias/ [root ~]# ls -l /etc/httpd/alias/ total 188 -r--r--r--. 1 root root 1427 Jan 15 10:48 cacert.asc -r--r--r--. 1 root root 1427 Jan 13 14:19 cacert.asc.orig -rw-rw----. 1 root apache 65536 Jan 15 10:49 cert8.db -rw-rw----. 1 root apache 65536 Jan 13 14:27 cert8.db.orig -rw-------. 1 root root 5872 Nov 16 2018 install.log -rw-rw----. 1 root apache 16384 Jan 15 10:49 key3.db -rw-rw----. 1 root apache 16384 Jan 13 14:27 key3.db.orig -r--r-----. 1 root apache 3481 Jan 15 10:47 kra-agent.pem lrwxrwxrwx. 1 root root 33 Nov 16 2018 libnssckbi.so -> ../../..//usr/lib64/libnssckbi.so -rw-rw----. 1 root apache 20 Jan 15 10:47 pwdfile.txt -rw-rw----. 1 root apache 20 Jan 13 14:18 pwdfile.txt.orig -rw-rw----. 1 root apache 16384 Jan 15 10:47 secmod.db -rw-rw----. 1 root apache 16384 Jan 13 14:18 secmod.db.orig
And to check the password: [root@ ~]# certutil -K -d /etc/httpd/alias/ -f /etc/httpd/alias/pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 1b212c867ebde61fdc295ecd63ef690cd93fc783 NSS Certificate DB:ipaCert < 1> rsa da7a2b61ed951d1cfff1309534a3232258da9487 NSS Certificate DB:Signing-Cert < 2> rsa e5fdd7ec9d8daa97566695ac562fb1697a811401 NSS Certificate DB:Server-Cert
(if the password file is wrong, you will see: Incorrect password/PIN entered)
flo
Hi flo,
Everything looks okay as far as permissions but when I run the certutil -K -d /etc/httpd/alias/ -f /etc/httpd/alias/pwdfile.txt command I get the following:
certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 0fa7d5a5d032d3d9bb0845425b4ce4c3588c5eba NSS Certificate DB:Server-Cert
ipaCert is missing from the output, but it is there when I run certutil -L -d /etc/httpd/alias/:
Certificate Nickname Trust Attributes SSL,S/MIME,JAR/XPI
Server-Cert u,u,u LOCAL IPA CA CT,C,C ipaCert ,,
It is missing the trust attributes though? How would I resolve that?
Hi, the cert is present but its private key is missing. It looks like you lost many of the private keys on that node, do you have a backup somewhere of the NSS database? Otherwise, the private key may be present on other nodes and can be recovered from them using the export and import commands I provided earlier (pk12util).
flo
Thanks,
Ferdinand _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
On 1/16/20 12:26 AM, Ferdinand Babas via FreeIPA-users wrote: Hi, the cert is present but its private key is missing. It looks like you lost many of the private keys on that node, do you have a backup somewhere of the NSS database? Otherwise, the private key may be present on other nodes and can be recovered from them using the export and import commands I provided earlier (pk12util).
flo
Hi flo,
No backup but I was able to export and import the ipaCert from another node and pki-tomcatd starts up now when back dated.
When I stop and start certmonger and run getcert list I get the following errors for a couple of certs:
ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation
The above error is reported for:
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
and
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
But when I run the following everything checks out:
# grep internal /var/lib/pki/pki-tomcat/conf/password.conf | cut -d= -f2 > /tmp/pwdfile.txt # certutil -K -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca' -f /tmp/pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 48f4a4...2c126e subsystemCert cert-pki-ca # certutil -K -d /etc/pki/pki-tomcat/alias -n 'auditSigningCert cert-pki-ca' -f /tmp/pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 7b55ea...d259f7 auditSigningCert cert-pki-ca
Things seem to checkout. I'm unsure what I should do next, I appreciate all the tips and help you've been providing.
Thanks,
Ferdinand
On 1/17/20 8:40 AM, Ferdinand Babas via FreeIPA-users wrote:
On 1/16/20 12:26 AM, Ferdinand Babas via FreeIPA-users wrote: Hi, the cert is present but its private key is missing. It looks like you lost many of the private keys on that node, do you have a backup somewhere of the NSS database? Otherwise, the private key may be present on other nodes and can be recovered from them using the export and import commands I provided earlier (pk12util).
flo
Hi flo,
No backup but I was able to export and import the ipaCert from another node and pki-tomcatd starts up now when back dated.
When I stop and start certmonger and run getcert list I get the following errors for a couple of certs:
ca-error: Server at "https://francolin.local:8443/ca/agent/ca/profileProcess" replied: 1: You did not provide a valid certificate for this operation
Hi,
IPA is using the ipaCert to authenticate to Dogtag when it requests new certs. You need to check that the ipaCert is valid and consistent with the content of uid=ipara,ou=people,o=ipaca. This ldap entry must contain a "userCertificate" attribute (can be multivalued) that contains the certificate + a description attribute with the format "2;<serial number>;<issuer>;<subject>". With all the manual operations done on IPA, it's likely that this entry is inconsistent with the ipaCert and you will need to update it with ldapmodify. When it's done you can check if the cert is valid and allows authentication with the command: # kinit admin # ipa cert-show 1
If it's working, you can resubmit the tracking requests that were showing the error: # getcert resubmit -i <id>
After that, your CA renewal master should be good to go and you can set the date to the current date. HTH, flo
The above error is reported for:
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert cert-pki-ca',token='NSS Certificate DB'
and
key pair storage: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB',pin set certificate: type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert cert-pki-ca',token='NSS Certificate DB'
But when I run the following everything checks out:
# grep internal /var/lib/pki/pki-tomcat/conf/password.conf | cut -d= -f2 > /tmp/pwdfile.txt # certutil -K -d /etc/pki/pki-tomcat/alias -n 'subsystemCert cert-pki-ca' -f /tmp/pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 48f4a4...2c126e subsystemCert cert-pki-ca # certutil -K -d /etc/pki/pki-tomcat/alias -n 'auditSigningCert cert-pki-ca' -f /tmp/pwdfile.txt certutil: Checking token "NSS Certificate DB" in slot "NSS User Private Key and Certificate Services" < 0> rsa 7b55ea...d259f7 auditSigningCert cert-pki-ca
Things seem to checkout. I'm unsure what I should do next, I appreciate all the tips and help you've been providing.
Thanks,
Ferdinand _______________________________________________ FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
freeipa-users@lists.fedorahosted.org
-
Ferdinand Babas -
Florence Blanc-Renaud