On 04/04/18 12:43, Florence Blanc-Renaud wrote:
On 04/04/2018 12:37 PM, lejeczek via FreeIPA-users wrote:
>
>
> On 04/04/18 09:36, Florence Blanc-Renaud wrote:
>> On 04/03/2018 08:37 PM, lejeczek wrote:
>>>
>>>
>>> On 29/03/18 12:43, Florence Blanc-Renaud wrote:
>>>> On 03/28/2018 12:42 PM, lejeczek via FreeIPA-users wrote:
>>>>> hi guys,
>>>>>
>>>>> I fail to troubleshoot this here:
>>>>>
>>>>> $ ipactl start --ignore-service-failures
>>>>> Starting Directory Service
>>>>> Starting krb5kdc Service
>>>>> Starting kadmin Service
>>>>> Starting named Service
>>>>> Starting httpd Service
>>>>> Starting ipa-custodia Service
>>>>> Starting ntpd Service
>>>>> Starting pki-tomcatd Service
>>>>> Failed to start pki-tomcatd Service
>>>>> Forced start, ignoring pki-tomcatd Service,
>>>>> continuing normal operation
>>>>
>>>> Hi,
>>>>
>>>> pki-tomcatd may fail to start when the subsystemCert
>>>> cert-pki-ca did not properly get renewed. Please find
>>>> more information in this blog:
>>>>
https://floblanc.wordpress.com/2017/09/11/troubleshooting-freeipa-pki-tom...
>>>>
>>>>
>>>> Flo
>>>>
>>>>> Starting ipa-otpd Service
>>>>> Starting ipa-dnskeysyncd Service
>>>>> ipa: INFO: The ipactl command was successful
>>>>>
>>>>> logs in /var/log/pki/pki-tomcat:
>>>>> localhost.2018-03-28.log
>>>>> ...
>>>>> Mar 28, 2018 11:35:14 AM
>>>>> org.apache.catalina.core.StandardHostValve invoke
>>>>> SEVERE: Exception Processing /ca/admin/ca/getStatus
>>>>> javax.ws.rs.ServiceUnavailableException: Subsystem
>>>>> unavailable
>>>>> ?????????????? at
>>>>>
com.netscape.cms.tomcat.ProxyRealm.findSecurityConstraints(ProxyRealm.java:145)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:500)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:962)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1087)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:637)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)
>>>>>
>>>>> ?????????????? at
>>>>>
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>>>>>
>>>>> ?????????????? at
>>>>>
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>>>>>
>>>>> ?????????????? at
>>>>>
org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
>>>>>
>>>>> ?????????????? at java.lang.Thread.run(Thread.java:748)
>>>>>
>>>>>
>>>>> in catalina.2018-03-28.log:
>>>>> ...
>>>>> Mar 28, 2018 11:41:35 AM
>>>>> org.apache.catalina.core.ContainerBase backgroundProcess
>>>>> WARNING: Exception processing realm
>>>>> com.netscape.cms.tomcat.ProxyRealm@1e572093
>>>>> background process
>>>>> javax.ws.rs.ServiceUnavailableException: Subsystem
>>>>> unavailable
>>>>> ?????? at
>>>>>
com.netscape.cms.tomcat.ProxyRealm.backgroundProcess(ProxyRealm.java:137)
>>>>>
>>>>> ?????? at
>>>>>
org.apache.catalina.core.ContainerBase.backgroundProcess(ContainerBase.java:1356)
>>>>>
>>>>> ?????? at
>>>>>
org.apache.catalina.core.StandardContext.backgroundProcess(StandardContext.java:5958)
>>>>>
>>>>> ?????? at
>>>>>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1542)
>>>>>
>>>>> ?????? at
>>>>>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552)
>>>>>
>>>>> ?????? at
>>>>>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.processChildren(ContainerBase.java:1552)
>>>>>
>>>>> ?????? at
>>>>>
org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1520)
>>>>>
>>>>> ?????? at java.lang.Thread.run(Thread.java:748)
>>>>>
>>>>>
>>>>> Would you able to conclude anything from those
>>>>> errors? What might be a problem?
>>>>>
>>>>> many thanks, L.
>>>>> _______________________________________________
>>>>> FreeIPA-users mailing list --
>>>>> freeipa-users(a)lists.fedorahosted.org
>>>>> To unsubscribe send an email to
>>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>>
>>>
>>> I have followed those instructions from the link and it
>>> seems that both certutil & ldap have the same certificate.
>>> However I also see:
>>>
>>> $ sudo journalctl -lf -o cat -u dirsrv@
>>> ...
>>> GSSAPI client step 1
>>> GSSAPI client step 1
>>> GSSAPI client step 2
>>> [03/Apr/2018:19:30:53.962565693 +0100] - ERR -
>>> slapi_ldap_bind - Error: could not send startTLS
>>> request: error -11 (Connect error)
>>> [03/Apr/2018:19:30:53.965606137 +0100] - ERR -
>>> slapi_ldap_bind - Error: could not send startTLS
>>> request: error -11 (Connect error)
>>> GSSAPI client step 1
>>> GSSAPI client step 1
>>> GSSAPI client step 1
>>> ..
>>>
>>> and in /var/log/pki/pki-tomcat/ca/debug
>>>
>>> [03/Apr/2018:19:09:45][localhost-startStop-1]:
>>> SSLClientCertificatSelectionCB: Entering!
>>> [03/Apr/2018:19:09:45][localhost-startStop-1]:
>>> Candidate cert: Server-Cert cert-pki-ca
>>> [03/Apr/2018:19:09:45][localhost-startStop-1]:
>>> Candidate cert: caSigningCert cert-pki-ca
>>> [03/Apr/2018:19:09:45][localhost-startStop-1]:
>>> SSLClientCertificateSelectionCB: returning: null
>>
>> Hi,
>> it looks like the subsystemCert is not picked to
>> authenticate to the LDAP server. Can you check if the
>> content of /etc/pki/pki-tomcat/ca/CS.cfg is also
>> consistent: this file contains an entry for
>> ca.subsystem.cert=MII.. that should match the cert
>> 'subsystemCert cert-pki-ca' stored in
>> /etc/pki/pki-tomcat/alias/ and in LDAP.
>>
>> Flo
>>
>>> [03/Apr/2018:19:09:45][localhost-startStop-1]: SSL
>>> handshake happened
>>> Could not connect to LDAP server host rider.private
>>> port 636 Error netscape.ldap.LDAPException:
>>> Authentication failed (48)
>>> at
>>>
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapBoundConnFactory.java:205)
>>>
>>> at
>>>
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:166)
>>>
>>> at
>>>
com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFactory.java:130)
>>>
>>> at
>>> com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:654)
>>>
>>> at
>>> com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1176)
>>>
>>> at
>>> com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:1082)
>>>
>>> at
>>> com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:572)
>>>
>>> at
>>> com.netscape.certsrv.apps.CMS.init(CMS.java:189)
>>> at
>>> com.netscape.certsrv.apps.CMS.start(CMS.java:1631)
>>> at
>>> com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java:117)
>>>
>>> at
>>> javax.servlet.GenericServlet.init(GenericServlet.java:158)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>
>>> at
>>>
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>
>>> at
>>> java.lang.reflect.Method.invoke(Method.java:498)
>>> at
>>> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
>>>
>>> at
>>> org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
>>>
>>> at
>>> java.security.AccessController.doPrivileged(Native Method)
>>> at
>>> javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
>>>
>>> at
>>> org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
>>>
>>> at
>>>
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:175)
>>>
>>> at
>>>
org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.java:124)
>>>
>>> at
>>>
org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1257)
>>>
>>> at
>>>
org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1182)
>>>
>>> at
>>> org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1072)
>>>
>>> at
>>>
org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5368)
>>>
>>> at
>>>
org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5660)
>>>
>>> at
>>> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
>>>
>>> at
>>>
org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:899)
>>>
>>> at
>>> org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:133)
>>>
>>> at
>>>
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:156)
>>>
>>> at
>>>
org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
>>>
>>> at
>>> java.security.AccessController.doPrivileged(Native Method)
>>> at
>>> org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
>>>
>>> at
>>> org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
>>>
>>> at
>>> org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:679)
>>>
>>> at
>>>
org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1966)
>>>
>>> at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>
>>> at
>>> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>>> ..
>>
>
>
>
> could this have some causation in expired certs?
>
Hi,
CA_WORKING means that certmonger's helper is trying to
download the certificate from LDAP, but does not find new
certs.
In topologies with multiple servers, only one server is
the renewal master. When one of auditSigningCert
cert-pki-ca, ocspSigningCert cert-pki-ca, subsystemCert
cert-pki-ca or caSigningCert cert-pki-ca expires, the
renewal master is the one that actually handles the
renewal, and the other masters simply download the new
certs from LDAP.
You need to check which server is your renewal master (ipa
config-show | grep 'IPA CA renewal master'), then make
sure that the certs were properly renewed on this master
(check consistency between /etc/pki/pki-tomcat/alias, the
certs in cn=certificates,cn=ipa,cn=etc,$BASEDN, and the
content in /etc/pki/pki-tomcat/ca/CS.cfg).
Then check that replication is working between the renewal
master and the other masters. If the replication is
broken, the certs will not be copied on the other masters
and the download will not detect new certificates.
HTH,
Flo
I had no renewal master at all! Was it gone with a replica a
removed from IPA? If/when CA renewal master is removed from
IPA(if it's allowed in the first place?) does another server
assume this role automatically?
Now I've set up a renewal server as per the doc but still
cannot start that new CA renewal server.
How to renew those expired certs? Maybe this has to be done
manually somehow prior, in order to start it all?
> $ getcert list | grep -E "Request
> ID|status|certificate|expires"
> Number of certificates and requests being tracked: 9.
> Request ID '20170920090053':
> status: MONITORING
> certificate:
> type=FILE,location='/var/kerberos/krb5kdc/kdc.crt'
> expires: 2018-09-20 09:00:53 UTC
> certificate template/profile: KDCs_PKINIT_Certs
> Request ID '20171221120303':
> status: CA_WORKING
> certificate:
>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='auditSigningCert
> cert-pki-ca',token='NSS Certificate DB'
> expires: 2018-03-27 14:07:51 UTC
> Request ID '20171221120304':
> status: CA_WORKING
> certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='ocspSigningCert
> cert-pki-ca',token='NSS Certificate DB'
> expires: 2018-03-27 14:07:50 UTC
> Request ID '20171221120305':
> status: CA_WORKING
> certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='subsystemCert
> cert-pki-ca',token='NSS Certificate DB'
> expires: 2018-03-27 14:07:51 UTC
> Request ID '20171221120306':
> status: MONITORING
> certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='caSigningCert
> cert-pki-ca',token='NSS Certificate DB'
> expires: 2036-04-06 14:07:49 UTC
> Request ID '20171221120307':
> status: CA_WORKING
> certificate:
> type=FILE,location='/var/lib/ipa/ra-agent.pem'
> expires: 2018-03-27 14:08:18 UTC
> Request ID '20171221120308':
> status: MONITORING
> certificate:
> type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> cert-pki-ca',token='NSS Certificate DB'
> expires: 2019-07-25 16:13:23 UTC
> Request ID '20171221120309':
> status: MONITORING
> certificate:
>
type=NSSDB,location='/etc/dirsrv/slapd-PRIVATE-CCNR-CEB-PRIVATE-CAM-AC-UK',nickname='Server-Cert',token='NSS
> Certificate DB'
> expires: 2019-04-09 12:11:52 UTC
> Request ID '20171221120310':
> status: MONITORING
> certificate:
>
type=NSSDB,location='/etc/httpd/alias',nickname='Server-Cert',token='NSS
> Certificate DB'
> expires: 2019-04-09 12:11:54 UTC
> _______________________________________________
> FreeIPA-users mailing list --
> freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to
> freeipa-users-leave(a)lists.fedorahosted.org