From manuel.gugliucci at scaiitec.it Mon Feb 8 10:59:46 2021
Content-Type: multipart/mixed; boundary="===============5892905460819013252=="
MIME-Version: 1.0
From: Manuel Gugliucci
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] IPA certs expired, pki-tomcatd fails to start
Date: Mon, 08 Feb 2021 10:59:36 +0000
Message-ID: <20210208105936.31782.85508@mailman01.iad2.fedoraproject.org>
--===============5892905460819013252==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Hello,
I'm running a freeipa server over a cloudera cluster, on 2020-12-31 all the=
certs expired and did not renew by itself.
After I set the system date before the expiration date, I tried ipa-cacert-=
renew but returns an error saying that ca cert are not managed by certmonge=
r so I did a getcert resubmit for every cert.
Almos all went on "Monitoring" state, except for one that says "NEED_CSR_GE=
N_PIN".
If I try to do 'ipactl start', it starts to first upgrade IPA and fails bec=
ause of the pki-tomcat service:
```
2019-12-31T19:12:01Z DEBUG response body 'Apache Tomcat/=
7.0.76 - Error report HTTP Status 500 - Sub=
system unavailable
type Ex=
ception report
message Subsystem unavailable
d=
escription The server encountered an internal error that prevented i=
t from fulfilling this requ
est.
exception
javax.ws.rs.ServiceUnavailableExcepti=
on: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecurit=
yConstraints(ProxyRealm.java:145)\n\torg.apache.catalina.authenticator.Auth=
enticatorBase.invoke(AuthenticatorBase.java:492)\n\torg.apache.catalina.val=
ves.ErrorReportValve.invoke(ErrorReportValve.java:103)\n\torg.apache.catali=
na.valves.AccessLogValve.invoke(AccessLogValve.java:962)\n\torg.apache.cata=
lina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)\n\torg.apache.=
coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:=
1091)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.proce=
ss(AbstractProtocol.java:637)\n\torg.apache.tomcat.util.net.JIoEndpoint$Soc=
ketProcessor.run(JIoEndpoint.java:316)\n\tjava.util.concurrent.ThreadPoolEx=
ecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.Thre=
adPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tomcat=
.util.threads.TaskThrea
d$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thread.=
java:748)\n
note The full stack trace of the root cau=
se is available in the Apache Tomcat/7.0.76 logs.
Apache Tomcat/7.0.76
'
2019-12-31T19:12:01Z DEBUG The CA status is: check interrupted due to error=
: Retrieving CA status failed with status 500
2019-12-31T19:12:01Z DEBUG Waiting for CA to start...
```
I also looked for the previous threads listed on this forum, but none of th=
em provided a solution
--===============5892905460819013252==--
From flo at redhat.com Mon Feb 8 12:53:29 2021
Content-Type: multipart/mixed; boundary="===============2472195656912242657=="
MIME-Version: 1.0
From: Florence Blanc-Renaud
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Mon, 08 Feb 2021 13:53:07 +0100
Message-ID: <1a01c8c7-1a97-0fc0-db36-07c08a37bb63@redhat.com>
In-Reply-To: 20210208105936.31782.85508@mailman01.iad2.fedoraproject.org
--===============2472195656912242657==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
On 2/8/21 11:59 AM, Manuel Gugliucci via FreeIPA-users wrote:
> Hello,
> =
> I'm running a freeipa server over a cloudera cluster, on 2020-12-31 all t=
he certs expired and did not renew by itself.
> =
> After I set the system date before the expiration date, I tried ipa-cacer=
t-renew but returns an error saying that ca cert are not managed by certmon=
ger so I did a getcert resubmit for every cert.
Hi,
the command "ipa-cacert-manage renew" is used to renew the CA =
certificate (in case IPA was installed with an embedded CA), not the =
other certs.
Before giving any advice, I would like to know more about the =
deployment. Are there a single or multiple IPA servers? Is the CA role =
deployed on multiple servers? Which version is installed?
# kinit admin; ipa server-role-find
# rpm -qa *ipa-server
Which certificates are valid or expired?
# getcert list
Thanks,
flo
> =
> Almos all went on "Monitoring" state, except for one that says "NEED_CSR_=
GEN_PIN".
> =
> If I try to do 'ipactl start', it starts to first upgrade IPA and fails b=
ecause of the pki-tomcat service:
> =
> ```
> 2019-12-31T19:12:01Z DEBUG response body 'Apache Tomca=
t/7.0.76 - Error report HTTP Status 500 - S=
ubsystem unavailable
type =
Exception report
message Subsystem unavailable
description The server encountered an internal error that prevented=
it from fulfilling this requ
> est.
exception
javax.ws.rs.ServiceUnavailableExce=
ption: Subsystem unavailable\n\tcom.netscape.cms.tomcat.ProxyRealm.findSecu=
rityConstraints(ProxyRealm.java:145)\n\torg.apache.catalina.authenticator.A=
uthenticatorBase.invoke(AuthenticatorBase.java:492)\n\torg.apache.catalina.=
valves.ErrorReportValve.invoke(ErrorReportValve.java:103)\n\torg.apache.cat=
alina.valves.AccessLogValve.invoke(AccessLogValve.java:962)\n\torg.apache.c=
atalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:445)\n\torg.apac=
he.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.ja=
va:1091)\n\torg.apache.coyote.AbstractProtocol$AbstractConnectionHandler.pr=
ocess(AbstractProtocol.java:637)\n\torg.apache.tomcat.util.net.JIoEndpoint$=
SocketProcessor.run(JIoEndpoint.java:316)\n\tjava.util.concurrent.ThreadPoo=
lExecutor.runWorker(ThreadPoolExecutor.java:1149)\n\tjava.util.concurrent.T=
hreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)\n\torg.apache.tom=
cat.util.threads.TaskThrea
> d$WrappingRunnable.run(TaskThread.java:61)\n\tjava.lang.Thread.run(Thre=
ad.java:748)\n
note The full stack trace of the root =
cause is available in the Apache Tomcat/7.0.76 logs.
Apache Tomcat/7.0.76
'
> 2019-12-31T19:12:01Z DEBUG The CA status is: check interrupted due to err=
or: Retrieving CA status failed with status 500
> 2019-12-31T19:12:01Z DEBUG Waiting for CA to start...
> ```
> I also looked for the previous threads listed on this forum, but none of =
them provided a solution
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.=
org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code=
-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users=
(a)lists.fedorahosted.org
> =
--===============2472195656912242657==--
From manuel.gugliucci at scaiitec.it Mon Feb 8 13:04:01 2021
Content-Type: multipart/mixed; boundary="===============0930619118656160908=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Mon, 08 Feb 2021 13:03:51 +0000
Message-ID: <20210208130351.21528.80269@mailman01.iad2.fedoraproject.org>
In-Reply-To: 1a01c8c7-1a97-0fc0-db36-07c08a37bb63@redhat.com
--===============0930619118656160908==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Hi Florence, thanks for the answer
it's a single IPA server, VERSION: 4.6.8, API_VERSION: 2.237
I kinit as admin without problems, then:
[root(a)ipa1 ~]# ipa server-role-find
ipa: ERROR: cannot connect to 'https://ipa1.itec.lab/ipa/json': Internal Se=
rver Error
[root(a)ipa1 ~]# rpm -qa *ipa-server
ipa-server-4.6.8-5.el7.centos.x86_64
# getcert list
Number of certificates and requests being tracked: 7.
Request ID '20191231201955':
status: MONITORING
stuck: no
key pair storage: type=3DFILE,location=3D'/var/kerberos/krb5kdc/kdc.key'
certificate: type=3DFILE,location=3D'/var/kerberos/krb5kdc/kdc.crt'
CA: SelfSign
issuer: CN=3Dipa1.itec.lab,O=3DITEC.LAB
subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
expires: 2020-12-31 20:19:55 UTC
principal name: krbtgt/ITEC.LAB(a)ITEC.LAB
certificate template/profile: KDCs_PKINIT_Certs
pre-save command: =
post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
track: yes
auto-renew: yes
Request ID '20201102185036':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'auditSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'auditSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DCA Audit,O=3DITEC.LAB
expires: 2020-12-08 09:35:14 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigning=
Cert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201102185037':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'ocspSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'ocspSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DOCSP Subsystem,O=3DITEC.LAB
expires: 2020-12-08 09:38:07 UTC
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningC=
ert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201102185038':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'subsystemCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'subsystemCert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DCA Subsystem,O=3DITEC.LAB
expires: 2020-12-08 09:37:36 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCer=
t cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201102185039':
status: NEED_CSR_GEN_PIN
stuck: yes
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'caSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'caSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent-selfsigned
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DCertificate Authority,O=3DITEC.LAB
expires: 2037-01-25 14:22:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCer=
t cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201102185040':
status: MONITORING
stuck: no
key pair storage: type=3DFILE,location=3D'/var/lib/ipa/ra-agent.key'
certificate: type=3DFILE,location=3D'/var/lib/ipa/ra-agent.pem'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DIPA RA,O=3DITEC.LAB
expires: 2020-12-08 09:37:47 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
Request ID '20201102185042':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'Server-Cert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'Server-Cert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
expires: 2020-12-08 09:35:05 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert =
cert-pki-ca"
track: yes
auto-renew: yes
I had to set my date in several weeks before the expiring to renew them via=
certmonger, but it does not auto-renew past 30-12-2020
Thanks for the support,
Manuel
--===============0930619118656160908==--
From flo at redhat.com Mon Feb 8 13:47:39 2021
Content-Type: multipart/mixed; boundary="===============7017014442913194653=="
MIME-Version: 1.0
From: Florence Blanc-Renaud
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Mon, 08 Feb 2021 14:47:20 +0100
Message-ID:
In-Reply-To: 20210208130351.21528.80269@mailman01.iad2.fedoraproject.org
--===============7017014442913194653==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
On 2/8/21 2:03 PM, Manuel Gujo via FreeIPA-users wrote:
> Hi Florence, thanks for the answer
> =
> it's a single IPA server, VERSION: 4.6.8, API_VERSION: 2.237
> =
Hi,
The CA is self-signed and still valid, and you are lucky because this =
ipa version already provides a new tool called ipa-cert-fix that should =
be able to help renew the certificates.
For more information please refer to the doc [1]. ipa-cert-fix analyzes =
the existing certificates and lists the ones that need to be renewed, =
then prompts you for confirmation and proceeds. No need to move the date =
in the past or do manual steps.
HTH,
flo
[1] =
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/ht=
ml/linux_domain_identity_authentication_and_policy_guide/cert-renewal#renew=
ing-expired-system-certificate-when-idm-is-offline
> I kinit as admin without problems, then:
> =
> [root(a)ipa1 ~]# ipa server-role-find
> ipa: ERROR: cannot connect to 'https://ipa1.itec.lab/ipa/json': Internal =
Server Error
> [root(a)ipa1 ~]# rpm -qa *ipa-server
> ipa-server-4.6.8-5.el7.centos.x86_64
> =
> # getcert list
> Number of certificates and requests being tracked: 7.
> Request ID '20191231201955':
> status: MONITORING
> stuck: no
> key pair storage: type=3DFILE,location=3D'/var/kerberos/krb5kdc/kdc.key'
> certificate: type=3DFILE,location=3D'/var/kerberos/krb5kdc/kdc.crt'
> CA: SelfSign
> issuer: CN=3Dipa1.itec.lab,O=3DITEC.LAB
> subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
> expires: 2020-12-31 20:19:55 UTC
> principal name: krbtgt/ITEC.LAB(a)ITEC.LAB
> certificate template/profile: KDCs_PKINIT_Certs
> pre-save command:
> post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
> track: yes
> auto-renew: yes
> Request ID '20201102185036':
> status: MONITORING
> stuck: no
> key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',ni=
ckname=3D'auditSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
> certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nicknam=
e=3D'auditSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
> CA: dogtag-ipa-ca-renew-agent
> issuer: CN=3DCertificate Authority,O=3DITEC.LAB
> subject: CN=3DCA Audit,O=3DITEC.LAB
> expires: 2020-12-08 09:35:14 UTC
> key usage: digitalSignature,nonRepudiation
> pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigni=
ngCert cert-pki-ca"
> track: yes
> auto-renew: yes
> Request ID '20201102185037':
> status: MONITORING
> stuck: no
> key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',ni=
ckname=3D'ocspSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
> certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nicknam=
e=3D'ocspSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
> CA: dogtag-ipa-ca-renew-agent
> issuer: CN=3DCertificate Authority,O=3DITEC.LAB
> subject: CN=3DOCSP Subsystem,O=3DITEC.LAB
> expires: 2020-12-08 09:38:07 UTC
> eku: id-kp-OCSPSigning
> pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSignin=
gCert cert-pki-ca"
> track: yes
> auto-renew: yes
> Request ID '20201102185038':
> status: MONITORING
> stuck: no
> key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',ni=
ckname=3D'subsystemCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
> certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nicknam=
e=3D'subsystemCert cert-pki-ca',token=3D'NSS Certificate DB'
> CA: dogtag-ipa-ca-renew-agent
> issuer: CN=3DCertificate Authority,O=3DITEC.LAB
> subject: CN=3DCA Subsystem,O=3DITEC.LAB
> expires: 2020-12-08 09:37:36 UTC
> key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherm=
ent
> eku: id-kp-serverAuth,id-kp-clientAuth
> pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemC=
ert cert-pki-ca"
> track: yes
> auto-renew: yes
> Request ID '20201102185039':
> status: NEED_CSR_GEN_PIN
> stuck: yes
> key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',ni=
ckname=3D'caSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
> certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nicknam=
e=3D'caSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
> CA: dogtag-ipa-ca-renew-agent-selfsigned
> issuer: CN=3DCertificate Authority,O=3DITEC.LAB
> subject: CN=3DCertificate Authority,O=3DITEC.LAB
> expires: 2037-01-25 14:22:25 UTC
> key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
> pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningC=
ert cert-pki-ca"
> track: yes
> auto-renew: yes
> Request ID '20201102185040':
> status: MONITORING
> stuck: no
> key pair storage: type=3DFILE,location=3D'/var/lib/ipa/ra-agent.key'
> certificate: type=3DFILE,location=3D'/var/lib/ipa/ra-agent.pem'
> CA: dogtag-ipa-ca-renew-agent
> issuer: CN=3DCertificate Authority,O=3DITEC.LAB
> subject: CN=3DIPA RA,O=3DITEC.LAB
> expires: 2020-12-08 09:37:47 UTC
> key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherm=
ent
> eku: id-kp-serverAuth,id-kp-clientAuth
> pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
> post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
> track: yes
> auto-renew: yes
> Request ID '20201102185042':
> status: MONITORING
> stuck: no
> key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',ni=
ckname=3D'Server-Cert cert-pki-ca',token=3D'NSS Certificate DB',pin set
> certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nicknam=
e=3D'Server-Cert cert-pki-ca',token=3D'NSS Certificate DB'
> CA: dogtag-ipa-ca-renew-agent
> issuer: CN=3DCertificate Authority,O=3DITEC.LAB
> subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
> expires: 2020-12-08 09:35:05 UTC
> key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherm=
ent
> eku: id-kp-serverAuth,id-kp-clientAuth
> pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
> post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cer=
t cert-pki-ca"
> track: yes
> auto-renew: yes
> =
> I had to set my date in several weeks before the expiring to renew them v=
ia certmonger, but it does not auto-renew past 30-12-2020
> =
> Thanks for the support,
> Manuel
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.=
org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code=
-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users=
(a)lists.fedorahosted.org
>=20
--===============7017014442913194653==--
From manuel.gugliucci at scaiitec.it Mon Feb 8 13:56:42 2021
Content-Type: multipart/mixed; boundary="===============6890672297103561626=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Mon, 08 Feb 2021 13:56:33 +0000
Message-ID: <20210208135633.18730.11573@mailman01.iad2.fedoraproject.org>
In-Reply-To: ca050885-b5c8-c67e-acde-c8f451cf7bdc@redhat.com
--===============6890672297103561626==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Hi,
I re-sync the date to today and ran ipa-cert-fix but it returns an error
[root(a)ipa1 ~]# ipa-cert-fix
WARNING
ipa-cert-fix is intended for recovery when expired certificates
prevent the normal operation of IPA. It should ONLY be used
in such scenarios, and backup of the system, especially certificates
and keys, is STRONGLY RECOMMENDED.
The following certificates will be renewed: =
Dogtag sslserver certificate:
Subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
Serial: 17
Expires: 2020-12-08 09:35:05
Dogtag subsystem certificate:
Subject: CN=3DCA Subsystem,O=3DITEC.LAB
Serial: 19
Expires: 2020-12-08 09:37:36
Dogtag ca_ocsp_signing certificate:
Subject: CN=3DOCSP Subsystem,O=3DITEC.LAB
Serial: 21
Expires: 2020-12-08 09:38:07
Dogtag ca_audit_signing certificate:
Subject: CN=3DCA Audit,O=3DITEC.LAB
Serial: 18
Expires: 2020-12-08 09:35:14
IPA IPA RA certificate:
Subject: CN=3DIPA RA,O=3DITEC.LAB
Serial: 20
Expires: 2020-12-08 09:37:47
IPA Apache HTTPS certificate:
Subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
Serial: 24
Expires: 2020-12-30 09:35:04
IPA LDAP certificate:
Subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
Serial: 25
Expires: 2020-12-30 09:35:16
IPA KDC certificate:
Subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
Serial: 1
Expires: 2020-12-31 20:19:55
Enter "yes" to proceed: yes
Proceeding.
[Errno 2] No such file or directory: '/etc/pki/pki-tomcat/certs/sslserver.c=
rt'
The ipa-cert-fix command failed.
--===============6890672297103561626==--
From flo at redhat.com Tue Feb 9 08:36:49 2021
Content-Type: multipart/mixed; boundary="===============6556618621630545301=="
MIME-Version: 1.0
From: Florence Blanc-Renaud
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Tue, 09 Feb 2021 09:31:19 +0100
Message-ID:
In-Reply-To: 20210208135633.18730.11573@mailman01.iad2.fedoraproject.org
--===============6556618621630545301==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
On 2/8/21 2:56 PM, Manuel Gujo via FreeIPA-users wrote:
> Hi,
> =
> I re-sync the date to today and ran ipa-cert-fix but it returns an error
> =
> [root(a)ipa1 ~]# ipa-cert-fix
> =
> WARNING
> =
> ipa-cert-fix is intended for recovery when expired certificates
> prevent the normal operation of IPA. It should ONLY be used
> in such scenarios, and backup of the system, especially certificates
> and keys, is STRONGLY RECOMMENDED.
> =
> =
> The following certificates will be renewed:
> =
> Dogtag sslserver certificate:
> Subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
> Serial: 17
> Expires: 2020-12-08 09:35:05
> =
> Dogtag subsystem certificate:
> Subject: CN=3DCA Subsystem,O=3DITEC.LAB
> Serial: 19
> Expires: 2020-12-08 09:37:36
> =
> Dogtag ca_ocsp_signing certificate:
> Subject: CN=3DOCSP Subsystem,O=3DITEC.LAB
> Serial: 21
> Expires: 2020-12-08 09:38:07
> =
> Dogtag ca_audit_signing certificate:
> Subject: CN=3DCA Audit,O=3DITEC.LAB
> Serial: 18
> Expires: 2020-12-08 09:35:14
> =
> IPA IPA RA certificate:
> Subject: CN=3DIPA RA,O=3DITEC.LAB
> Serial: 20
> Expires: 2020-12-08 09:37:47
> =
> IPA Apache HTTPS certificate:
> Subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
> Serial: 24
> Expires: 2020-12-30 09:35:04
> =
> IPA LDAP certificate:
> Subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
> Serial: 25
> Expires: 2020-12-30 09:35:16
> =
> IPA KDC certificate:
> Subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
> Serial: 1
> Expires: 2020-12-31 20:19:55
> =
> Enter "yes" to proceed: yes
> Proceeding.
> [Errno 2] No such file or directory: '/etc/pki/pki-tomcat/certs/sslserver=
.crt'
> The ipa-cert-fix command failed.
Hi,
which version of pki-server is installed? You may be hitting =
https://bugzilla.redhat.com/show_bug.cgi?id=3D1897120
Looks like you will need to manually fix the renewal issue by following =
the old good method with changing date etc...
The first expiration date is 2020-12-08, the system date needs to be =
moved before that date. Please try and let me know if there are issues.
flo
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.=
org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code=
-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users=
(a)lists.fedorahosted.org
>=20
--===============6556618621630545301==--
From manuel.gugliucci at scaiitec.it Tue Feb 9 09:06:28 2021
Content-Type: multipart/mixed; boundary="===============4807286496341520518=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Tue, 09 Feb 2021 09:06:18 +0000
Message-ID: <20210209090618.13571.83744@mailman01.iad2.fedoraproject.org>
In-Reply-To: d0c8d7da-7814-e0f0-b615-b64b10102f47@redhat.com
--===============4807286496341520518==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Hi,
pki-server version is 10.5.17, if I hit pki-server-upgrade it says this:
# pki-server-upgrade
Upgrading from version 10.5.17 to 10.5.18:
1. Fix EC admin certificate profile (Yes/No) [Y]:
could be helpful?
I also found this: https://www.dogtagpki.org/wiki/Tomcat_SSL_Configuration_=
with_OpenSSL
could generating a self signed sslserver.crt a good idea?
in the bugzilla link you sent, do I have to download and run that pyscript?
About the old method with turn the date before expiration, what I have to d=
o precisely? Because I already done this:
#getcert list
//list all the cert IDs
#ipa-getcert resubmit -i
//for every cert listed before
#systemctl certmonger restart
Thanks for the support Flo
Manuel
--===============4807286496341520518==--
From manuel.gugliucci at scaiitec.it Fri Feb 12 09:05:20 2021
Content-Type: multipart/mixed; boundary="===============2927378700239064078=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Fri, 12 Feb 2021 09:05:10 +0000
Message-ID: <20210212090510.25276.17909@mailman01.iad2.fedoraproject.org>
In-Reply-To: d0c8d7da-7814-e0f0-b615-b64b10102f47@redhat.com
--===============2927378700239064078==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Hi,
I've retried to move date three weeks before 2020-12-08 and renew cert manu=
ally
# ipa-getcert resubmit -i "ID"
Resubmitting "20201102185036" to "dogtag-ipa-ca-renew-agent".
Here's one of the output log from journalctl -xe
# journalctl -xe
nov 17 18:08:27 ipa1.itec.lab certmonger[27108]: 2020-11-17 18:08:27 [27108=
] Internal error
nov 17 18:08:29 ipa1.itec.lab dogtag-ipa-ca-renew-agent-submit[28053]: Trac=
eback (most recent call last):
Fi=
le "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 533, in=
=
sys.exit(main())
Fi=
le "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 507, in=
main
=
kinit_keytab(principal, paths.KRB5_KEYTAB, ccache_filename)
Fi=
le "/usr/lib/python2.7/site-packages/ipalib/install/kinit.py", line 47, in =
kinit_keytab
=
cred =3D gssapi.Credentials(name=3Dname, store=3Dstore, usage=3D'initiate')
Fi=
le "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 64, in __new__
=
store=3Dstore)
Fi=
le "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 148, in acqui=
re
=
usage)
Fi=
le "ext_cred_store.pyx", line 182, in gssapi.raw.ext_cred_store.acquire_cre=
d_from (gssapi/raw/ext_cred
GSSE=
rror: Major (851968): Unspecified GSS failure. Minor code may provide more=
information, Minor (252963
now all the certs (except from kerberos and CA ones) are status: CA_UNREACH=
ABLE.
CA cert is status: NEED_CSR_GEN_PIN=20
--===============2927378700239064078==--
From rcritten at redhat.com Fri Feb 12 15:37:21 2021
Content-Type: multipart/mixed; boundary="===============5128773531211025706=="
MIME-Version: 1.0
From: Rob Crittenden
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Fri, 12 Feb 2021 10:37:05 -0500
Message-ID: <4d3a716a-4dae-6e55-a450-833da4daf128@redhat.com>
In-Reply-To: 20210212090510.25276.17909@mailman01.iad2.fedoraproject.org
--===============5128773531211025706==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Manuel Gujo via FreeIPA-users wrote:
> Hi,
> =
> I've retried to move date three weeks before 2020-12-08 and renew cert ma=
nually
> =
> # ipa-getcert resubmit -i "ID"
> Resubmitting "20201102185036" to "dogtag-ipa-ca-renew-agent".
> =
> Here's one of the output log from journalctl -xe
> =
> # journalctl -xe
> nov 17 18:08:27 ipa1.itec.lab certmonger[27108]: 2020-11-17 18:08:27 [271=
08] Internal error
> nov 17 18:08:29 ipa1.itec.lab dogtag-ipa-ca-renew-agent-submit[28053]: Tr=
aceback (most recent call last):
> =
File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 533, =
in
> =
sys.exit(main())
> =
File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 507, =
in main
> =
kinit_keytab(principal, paths.KRB5_KEYTAB, ccache_filename)
> =
File "/usr/lib/python2.7/site-packages/ipalib/install/kinit.py", line 47, i=
n kinit_keytab
> =
cred =3D gssapi.Credentials(name=3Dname, store=3Dstore, usage=3D'initiate=
')
> =
File "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 64, in __ne=
w__
> =
store=3Dstore)
> =
File "/usr/lib64/python2.7/site-packages/gssapi/creds.py", line 148, in acq=
uire
> =
usage)
> =
File "ext_cred_store.pyx", line 182, in gssapi.raw.ext_cred_store.acquire_c=
red_from (gssapi/raw/ext_cred
> GS=
SError: Major (851968): Unspecified GSS failure. Minor code may provide mo=
re information, Minor (252963
> =
> now all the certs (except from kerberos and CA ones) are status: CA_UNREA=
CHABLE.
> =
> CA cert is status: NEED_CSR_GEN_PIN =
When you are moving back in time are you bringing the IPA services back
up? You need to do this manually if you have an NTP server enabled
(which it is by default).
Minimum you need to restart, in order, dirsrv.target, krb5kdc, named,
httpd, pki-tomcatd. If you restart certmonger it may kick off the
renewals for you (or it might not).
If you can get the services running back in time then runipa config-show
to determine whether this server is configured as the CA renewal server.
Only one in the cluster will have this role and the renewals need to
take place on that server. If this one isn't it and none of the others
report it then you can run: ipa config-mod --ca-renewal-master-server=3D
As Flo said the NEED_CSR_GEN_PIN is from your using ipa-cacert-manage.
It doesn't affect anything in the short term.
rob
--===============5128773531211025706==--
From manuel.gugliucci at scaiitec.it Fri Feb 12 15:42:49 2021
Content-Type: multipart/mixed; boundary="===============7830807065629117288=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Fri, 12 Feb 2021 15:42:39 +0000
Message-ID: <20210212154239.21274.63116@mailman01.iad2.fedoraproject.org>
In-Reply-To: 4d3a716a-4dae-6e55-a450-833da4daf128@redhat.com
--===============7830807065629117288==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Hi Rob,
do I have to stop all the IPA services before i move back the date? Now I'm=
only moving back date and restarting certmonger.
pki-tomcatd is failed so i can't stop/restart it
--===============7830807065629117288==--
From rcritten at redhat.com Fri Feb 12 16:05:04 2021
Content-Type: multipart/mixed; boundary="===============3017406877216900466=="
MIME-Version: 1.0
From: Rob Crittenden
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Fri, 12 Feb 2021 11:04:49 -0500
Message-ID:
In-Reply-To: 20210212154239.21274.63116@mailman01.iad2.fedoraproject.org
--===============3017406877216900466==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Manuel Gujo via FreeIPA-users wrote:
> Hi Rob,
> =
> do I have to stop all the IPA services before i move back the date? Now I=
'm only moving back date and restarting certmonger.
It wouldn't hurt.
You absolutely need to restart things in the past because they can't run
in current time with expired certs.
> pki-tomcatd is failed so i can't stop/restart it
Then go back in the past and we can try to figure out why it won't start
then. It won't start now due to expired certs.
You can't renew the certs without a working CA.
rob
--===============3017406877216900466==--
From manuel.gugliucci at scaiitec.it Fri Feb 12 16:11:30 2021
Content-Type: multipart/mixed; boundary="===============0474500137371675838=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Fri, 12 Feb 2021 16:11:21 +0000
Message-ID: <20210212161121.30811.95648@mailman01.iad2.fedoraproject.org>
In-Reply-To: c234d6e7-cdf0-2d3d-7bef-da1b01bebc2b@redhat.com
--===============0474500137371675838==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
I moved the date before the expiring and restarted the services one by one =
as you listed (systemctl restart dirsrv(a)my-domain, systemctl restart krb5=
kdc etc.)
then:
[root(a)ipa1 ~]# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: STOPPED (if I do systemctl status named it says running)
httpd Service: RUNNING
ipa-custodia Service: RUNNING
ntpd Service: RUNNING
pki-tomcatd Service: STOPPED
ipa-otpd Service: STOPPED
ipa-dnskeysyncd Service: STOPPED
ipa: INFO: The ipactl command was successful
pki-tomcatd failed to start:
# systemctl restart pki-tomcatd(a)ITEC-LAB
Job for pki-tomcatd(a)ITEC-LAB.service failed because the control process e=
xited with error code. See "systemctl status pki-tomcatd(a)ITEC-LAB.service=
" and "journalctl -xe" for details.
# journalctl -xe =
nov 17 18:22:31 ipa1.itec.lab systemd[1]: Unit pki-tomcatd(a)ITEC-LAB.servi=
ce entered failed state.
nov 17 18:22:31 ipa1.itec.lab audispd[24456]: node=3Dipa1.itec.lab type=3DS=
ERVICE_START msg=3Daudit(1605637351.916:7091): pid=3D1 uid=3D0 auid=3D42949=
67295 ses=3D4294967295 msg=3D'unit=3Dpki-tomc
nov 17 18:22:31 ipa1.itec.lab systemd[1]: pki-tomcatd(a)ITEC-LAB.service fa=
iled.
nov 17 18:22:31 ipa1.itec.lab polkitd[30556]: Unregistered Authentication A=
gent for unix-process:17970:71502359 (system bus name :1.1719, object path =
/org/freedesktop/PolicyKit1
--===============0474500137371675838==--
From rcritten at redhat.com Fri Feb 12 16:40:01 2021
Content-Type: multipart/mixed; boundary="===============9024709199035763294=="
MIME-Version: 1.0
From: Rob Crittenden
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Fri, 12 Feb 2021 11:39:47 -0500
Message-ID: <3d142af9-aa1a-01c6-0864-013f4f0b2832@redhat.com>
In-Reply-To: 20210212161121.30811.95648@mailman01.iad2.fedoraproject.org
--===============9024709199035763294==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Manuel Gujo via FreeIPA-users wrote:
> I moved the date before the expiring and restarted the services one by on=
e as you listed (systemctl restart dirsrv(a)my-domain, systemctl restart kr=
b5kdc etc.)
> =
> then:
> [root(a)ipa1 ~]# ipactl status
> Directory Service: RUNNING
> krb5kdc Service: RUNNING
> kadmin Service: RUNNING
> named Service: STOPPED (if I do systemctl status named it says running)
> httpd Service: RUNNING
> ipa-custodia Service: RUNNING
> ntpd Service: RUNNING
> pki-tomcatd Service: STOPPED
> ipa-otpd Service: STOPPED
> ipa-dnskeysyncd Service: STOPPED
> ipa: INFO: The ipactl command was successful
> =
> pki-tomcatd failed to start:
> =
> # systemctl restart pki-tomcatd(a)ITEC-LAB
> Job for pki-tomcatd(a)ITEC-LAB.service failed because the control process=
exited with error code. See "systemctl status pki-tomcatd(a)ITEC-LAB.servi=
ce" and "journalctl -xe" for details.
> =
> # journalctl -xe =
> nov 17 18:22:31 ipa1.itec.lab systemd[1]: Unit pki-tomcatd(a)ITEC-LAB.ser=
vice entered failed state.
> nov 17 18:22:31 ipa1.itec.lab audispd[24456]: node=3Dipa1.itec.lab type=
=3DSERVICE_START msg=3Daudit(1605637351.916:7091): pid=3D1 uid=3D0 auid=3D4=
294967295 ses=3D4294967295 msg=3D'unit=3Dpki-tomc
> nov 17 18:22:31 ipa1.itec.lab systemd[1]: pki-tomcatd(a)ITEC-LAB.service =
failed.
> nov 17 18:22:31 ipa1.itec.lab polkitd[30556]: Unregistered Authentication=
Agent for unix-process:17970:71502359 (system bus name :1.1719, object pat=
h /org/freedesktop/PolicyKit1
Look in /var/log/pki/pki-tomcat/ca/debug. Find where it tries to start
the service and go down from there. Looking at the end of the log is
almost always fruitless.
rob
--===============9024709199035763294==--
From manuel.gugliucci at scaiitec.it Fri Feb 12 18:08:03 2021
Content-Type: multipart/mixed; boundary="===============3252552859192356450=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Fri, 12 Feb 2021 18:07:51 +0000
Message-ID: <20210212180751.11429.12894@mailman01.iad2.fedoraproject.org>
In-Reply-To: 3d142af9-aa1a-01c6-0864-013f4f0b2832@redhat.com
--===============3252552859192356450==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Here's what I found in /var/log/pki/pki-tomcat/ca/debug
Could not connect to LDAP server host ipa1.itec.lab port 636 Error netscape=
.ldap.LDAPException: Unable to create socket: java.net.ConnectException: Co=
nnection refused (Connection refused) (-1)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.makeConnection(LdapB=
oundConnFactory.java:205)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFa=
ctory.java:166)
at com.netscape.cmscore.ldapconn.LdapBoundConnFactory.init(LdapBoundConnFa=
ctory.java:130)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:667)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1056)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:962)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:568)
at com.netscape.certsrv.apps.CMS.init(CMS.java:191)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1458)
at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java=
:117)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja=
va:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso=
rImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.ja=
va:175)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.ja=
va:124)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.ja=
va:1218)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.ja=
va:1174)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1066)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.=
java:5377)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.=
java:5669)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.j=
ava:899)
at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:13=
3)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(Container=
Base.java:156)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(Container=
Base.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java=
:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.=
java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja=
va:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j=
ava:624)
at java.lang.Thread.run(Thread.java:748)
Internal Database Error encountered: Could not connect to LDAP server host =
ipa1.itec.lab port 636 Error netscape.ldap.LDAPException: Unable to create =
socket: java.net.ConnectException: Connection refused (Connection refused) =
(-1)
at com.netscape.cmscore.dbs.DBSubsystem.init(DBSubsystem.java:689)
at com.netscape.cmscore.apps.CMSEngine.initSubsystem(CMSEngine.java:1056)
at com.netscape.cmscore.apps.CMSEngine.initSubsystems(CMSEngine.java:962)
at com.netscape.cmscore.apps.CMSEngine.init(CMSEngine.java:568)
at com.netscape.certsrv.apps.CMS.init(CMS.java:191)
at com.netscape.certsrv.apps.CMS.start(CMS.java:1458)
at com.netscape.cms.servlet.base.CMSStartServlet.init(CMSStartServlet.java=
:117)
at javax.servlet.GenericServlet.init(GenericServlet.java:158)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.ja=
va:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccesso=
rImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:288)
at org.apache.catalina.security.SecurityUtil$1.run(SecurityUtil.java:285)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAsPrivileged(Subject.java:549)
at org.apache.catalina.security.SecurityUtil.execute(SecurityUtil.java:320)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.ja=
va:175)
at org.apache.catalina.security.SecurityUtil.doAsPrivilege(SecurityUtil.ja=
va:124)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.ja=
va:1218)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.ja=
va:1174)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1066)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.=
java:5377)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.=
java:5669)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.j=
ava:899)
at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:13=
3)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(Container=
Base.java:156)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(Container=
Base.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:873)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:652)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java=
:679)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.=
java:1966)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.ja=
va:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j=
ava:624)
at java.lang.Thread.run(Thread.java:748)
[31/Dec/2019:19:55:52][localhost-startStop-1]: CMS.start(): shutdown server
[31/Dec/2019:19:55:52][localhost-startStop-1]: CMSEngine.shutdown()
--===============3252552859192356450==--
From rcritten at redhat.com Fri Feb 12 18:23:35 2021
Content-Type: multipart/mixed; boundary="===============5455666091608152997=="
MIME-Version: 1.0
From: Rob Crittenden
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Fri, 12 Feb 2021 13:23:20 -0500
Message-ID:
In-Reply-To: 20210212180751.11429.12894@mailman01.iad2.fedoraproject.org
--===============5455666091608152997==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Manuel Gujo via FreeIPA-users wrote:
> Here's what I found in /var/log/pki/pki-tomcat/ca/debug
> =
> Could not connect to LDAP server host ipa1.itec.lab port 636 Error netsca=
pe.ldap.LDAPException: Unable to create socket: java.net.ConnectException: =
Connection refused (Connection refused) (-1)
Double-check that dirsrv is started and listening on port 636. I assume
the host name is correct.
rob
--===============5455666091608152997==--
From manuel.gugliucci at scaiitec.it Mon Feb 15 08:33:47 2021
Content-Type: multipart/mixed; boundary="===============2415229631649351648=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Mon, 15 Feb 2021 08:33:37 +0000
Message-ID: <20210215083337.28700.99515@mailman01.iad2.fedoraproject.org>
In-Reply-To: bd5ef15c-5ff6-399e-b788-205ea1c8ad10@redhat.com
--===============2415229631649351648==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
# systemctl status dirsrv(a)ITEC-LAB
=E2=97=8F dirsrv(a)ITEC-LAB.service - 389 Directory Server ITEC-LAB.
Loaded: loaded (/usr/lib/systemd/system/dirsrv(a).service; enabled; vend=
or preset: disabled)
Active: active (running) since mar 2020-11-17 18:00:26 UTC; 2 months 28 =
days ago
Main PID: 15817 (ns-slapd)
Status: "slapd started: Ready to process requests"
CGroup: /system.slice/system-dirsrv.slice/dirsrv(a)ITEC-LAB.service
=E2=94=94=E2=94=8015817 /usr/sbin/ns-slapd -D /etc/dirsrv/slapd-=
ITEC-LAB -i /var/run/dirsrv/slapd-ITEC-LAB.pid
feb 12 17:26:03 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 3
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 1
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 2
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 3
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 1
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 2
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 3
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 1
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 2
feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 3
# netstat -tulpn | grep LISTEN
tcp 0 0 0.0.0.0:749 0.0.0.0:* LISTEN =
1222/kadmind =
tcp 0 0 0.0.0.0:464 0.0.0.0:* LISTEN =
1222/kadmind =
tcp 0 0 192.168.20.3:53 0.0.0.0:* LISTEN =
17818/named =
tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTEN =
17818/named =
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN =
24121/sshd =
tcp 0 0 0.0.0.0:88 0.0.0.0:* LISTEN =
15869/krb5kdc =
tcp 0 0 127.0.0.1:953 0.0.0.0:* LISTEN =
17818/named =
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN =
24232/master =
tcp6 0 0 :::749 :::* LISTEN =
1222/kadmind =
tcp6 0 0 :::80 :::* LISTEN =
16122/httpd =
tcp6 0 0 :::464 :::* LISTEN =
1222/kadmind =
tcp6 0 0 :::53 :::* LISTEN =
17818/named =
tcp6 0 0 :::22 :::* LISTEN =
24121/sshd =
tcp6 0 0 :::88 :::* LISTEN =
15869/krb5kdc =
tcp6 0 0 ::1:953 :::* LISTEN =
17818/named =
tcp6 0 0 ::1:25 :::* LISTEN =
24232/master =
tcp6 0 0 :::443 :::* LISTEN =
16122/httpd =
dirsrv is started but I don't see port 636 on this list, how can I open it =
for dirsrv?
--===============2415229631649351648==--
From rcritten at redhat.com Mon Feb 15 14:30:48 2021
Content-Type: multipart/mixed; boundary="===============4483985877845769890=="
MIME-Version: 1.0
From: Rob Crittenden
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Mon, 15 Feb 2021 09:30:34 -0500
Message-ID:
In-Reply-To: 20210215083337.28700.99515@mailman01.iad2.fedoraproject.org
--===============4483985877845769890==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Manuel Gujo via FreeIPA-users wrote:
> # systemctl status dirsrv(a)ITEC-LAB
> =E2=97=8F dirsrv(a)ITEC-LAB.service - 389 Directory Server ITEC-LAB.
> Loaded: loaded (/usr/lib/systemd/system/dirsrv(a).service; enabled; ve=
ndor preset: disabled)
> Active: active (running) since mar 2020-11-17 18:00:26 UTC; 2 months 2=
8 days ago
> Main PID: 15817 (ns-slapd)
> Status: "slapd started: Ready to process requests"
> CGroup: /system.slice/system-dirsrv.slice/dirsrv(a)ITEC-LAB.service
> =E2=94=94=E2=94=8015817 /usr/sbin/ns-slapd -D /etc/dirsrv/slap=
d-ITEC-LAB -i /var/run/dirsrv/slapd-ITEC-LAB.pid
> =
> feb 12 17:26:03 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 3
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 1
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 2
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 3
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 1
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 2
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 3
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 1
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 2
> feb 14 03:35:04 ipa1.itec.lab ns-slapd[15817]: GSSAPI server step 3
> =
> # netstat -tulpn | grep LISTEN
> tcp 0 0 0.0.0.0:749 0.0.0.0:* LISTE=
N 1222/kadmind =
> tcp 0 0 0.0.0.0:464 0.0.0.0:* LISTE=
N 1222/kadmind =
> tcp 0 0 192.168.20.3:53 0.0.0.0:* LISTE=
N 17818/named =
> tcp 0 0 127.0.0.1:53 0.0.0.0:* LISTE=
N 17818/named =
> tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTE=
N 24121/sshd =
> tcp 0 0 0.0.0.0:88 0.0.0.0:* LISTE=
N 15869/krb5kdc =
> tcp 0 0 127.0.0.1:953 0.0.0.0:* LISTE=
N 17818/named =
> tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTE=
N 24232/master =
> tcp6 0 0 :::749 :::* LISTE=
N 1222/kadmind =
> tcp6 0 0 :::80 :::* LISTE=
N 16122/httpd =
> tcp6 0 0 :::464 :::* LISTE=
N 1222/kadmind =
> tcp6 0 0 :::53 :::* LISTE=
N 17818/named =
> tcp6 0 0 :::22 :::* LISTE=
N 24121/sshd =
> tcp6 0 0 :::88 :::* LISTE=
N 15869/krb5kdc =
> tcp6 0 0 ::1:953 :::* LISTE=
N 17818/named =
> tcp6 0 0 ::1:25 :::* LISTE=
N 24232/master =
> tcp6 0 0 :::443 :::* LISTE=
N 16122/httpd =
> =
> dirsrv is started but I don't see port 636 on this list, how can I open i=
t for dirsrv?
In fact it isn't listening on any port.
Stop dirsrv and look in /etc/dirsrv/slapd-ITEC-LAB/dse.ldif for
nsslapd-port which should be 389 and nsslapd-security which should be
on. If not then fix it and restart dirsrv. That should fix it.
rob
--===============4483985877845769890==--
From manuel.gugliucci at scaiitec.it Mon Feb 15 15:38:29 2021
Content-Type: multipart/mixed; boundary="===============5827145078568548144=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Mon, 15 Feb 2021 15:38:18 +0000
Message-ID: <20210215153818.11220.71701@mailman01.iad2.fedoraproject.org>
In-Reply-To: d2fcd4d5-42d7-b1a0-20e9-390c414479e2@redhat.com
--===============5827145078568548144==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Hi Rob,
so in "/etc/dirsrv/slapd-ITEC-LAB/dse.ldif", nsslapd-port was '0' and nssla=
pd-security was off, I fixed it and now it's listening on port 389 and 636:
# netstat -tulpn | grep LISTEN | grep ns-slapd
tcp6 0 0 :::636 :::* LISTEN =
30606/ns-slapd =
tcp6 0 0 :::389 :::* LISTEN =
30606/ns-slapd =
Then I tried to restart all the ipactl services one by one. pki-tomcatd kee=
ps failing and /var/log/pki/pki-tomcat/ca/debug does not log anymore (last =
log is the one i sent you above, 31 Dec 2019)
I resubmitted all the expired certs and restarting cermonger but certs keep=
being unreachable.
from certmonger logs:
nov 17 18:11:47 ipa1.itec.lab dogtag-ipa-ca-renew-agent-submit[30764]: Forw=
arding request to dogtag-ipa-renew-agent
nov 17 18:11:47 ipa1.itec.lab dogtag-ipa-renew-agent-submit[31183]: GET htt=
p://ipa1.itec.lab:8080/ca/ee/ca/profileSubmit?profileId=3DcaServerCert&seri=
al_num=3D20&renewal=3Dtrue&xml=3Dtrue&
nov 17 18:11:47 ipa1.itec.lab dogtag-ipa-renew-agent-submit[31183]: (null)
nov 17 18:11:47 ipa1.itec.lab dogtag-ipa-ca-renew-agent-submit[30764]: dogt=
ag-ipa-renew-agent returned 3
nov 17 18:11:47 ipa1.itec.lab certmonger[30685]: 2020-11-17 18:11:47 [30685=
] Error 7 connecting to http://ipa1.itec.lab:8080/ca/ee/ca/profileSubmit: C=
ouldn't connect to server.
in certmonger's log I also saw these:
nov 17 18:11:01 ipa1.itec.lab dogtag-ipa-ca-renew-agent-submit[30741]: Trac=
eback (most recent call last):
Fi=
le "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 533, in=
=
sys.exit(main())
Fi=
le "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 495, in=
main
=
api.finalize()
Fi=
le "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 740, in fina=
lize
=
self.__do_if_not_done('load_plugins')
Fi=
le "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 431, in __do=
_if_not_done
=
getattr(self, name)()
Fi=
le "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 620, in load=
_plugins
=
self.add_package(package)
Fi=
le "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 647, in add_=
package
=
module =3D importlib.import_module(name)
Fi=
le "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
=
__import__(name)
Fi=
le "/usr/lib/python2.7/site-packages/ipaserver/plugins/server.py", line 32,=
in
=
from ipaserver.install import bindinstance, dnskeysyncinstance
Fi=
le "/usr/lib/python2.7/site-packages/ipaserver/install/dnskeysyncinstance.p=
y", line 17, in
=
from ipaserver import p11helper as _ipap11helper
Fi=
le "/usr/lib/python2.7/site-packages/ipaserver/p11helper.py", line 342, in =
=
_libp11_kit =3D _ffi.dlopen(ctypes.util.find_library('p11-kit'))
Fi=
le "/usr/lib64/python2.7/ctypes/util.py", line 244, in find_library
=
return _findSoname_ldconfig(name) or _get_soname(_findLib_gcc(name))
Fi=
le "/usr/lib64/python2.7/ctypes/util.py", line 233, in _findSoname_ldconfig
=
f =3D os.popen('/sbin/ldconfig -p 2>/dev/null')
OSEr=
ror: [Errno 12] Cannot allocate memory
--===============5827145078568548144==--
From rcritten at redhat.com Mon Feb 15 20:02:21 2021
Content-Type: multipart/mixed; boundary="===============8024469623496485149=="
MIME-Version: 1.0
From: Rob Crittenden
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Mon, 15 Feb 2021 15:02:07 -0500
Message-ID: <97a36513-8c3b-23fa-bff6-5a6a9a4c06fc@redhat.com>
In-Reply-To: 20210215153818.11220.71701@mailman01.iad2.fedoraproject.org
--===============8024469623496485149==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Manuel Gujo via FreeIPA-users wrote:
> Hi Rob,
> =
> so in "/etc/dirsrv/slapd-ITEC-LAB/dse.ldif", nsslapd-port was '0' and nss=
lapd-security was off, I fixed it and now it's listening on port 389 and 63=
6:
> =
> # netstat -tulpn | grep LISTEN | grep ns-slapd
> tcp6 0 0 :::636 :::* LISTE=
N 30606/ns-slapd =
> tcp6 0 0 :::389 :::* LISTE=
N 30606/ns-slapd =
> =
> Then I tried to restart all the ipactl services one by one. pki-tomcatd k=
eeps failing and /var/log/pki/pki-tomcat/ca/debug does not log anymore (las=
t log is the one i sent you above, 31 Dec 2019)
> =
> I resubmitted all the expired certs and restarting cermonger but certs ke=
ep being unreachable.
If the CA isn't running then there is no point in resubmitting the
certmonger requests. It is guaranteed to fail with UNREACHABLE.
Check the journalctl output and the other logs, like catalina, in
/var/log/pki/pki-tomcat for more information on why it failed to start.
> =
> from certmonger logs:
> =
> nov 17 18:11:47 ipa1.itec.lab dogtag-ipa-ca-renew-agent-submit[30764]: Fo=
rwarding request to dogtag-ipa-renew-agent
> nov 17 18:11:47 ipa1.itec.lab dogtag-ipa-renew-agent-submit[31183]: GET h=
ttp://ipa1.itec.lab:8080/ca/ee/ca/profileSubmit?profileId=3DcaServerCert&se=
rial_num=3D20&renewal=3Dtrue&xml=3Dtrue&
> nov 17 18:11:47 ipa1.itec.lab dogtag-ipa-renew-agent-submit[31183]: (null)
> nov 17 18:11:47 ipa1.itec.lab dogtag-ipa-ca-renew-agent-submit[30764]: do=
gtag-ipa-renew-agent returned 3
> nov 17 18:11:47 ipa1.itec.lab certmonger[30685]: 2020-11-17 18:11:47 [306=
85] Error 7 connecting to http://ipa1.itec.lab:8080/ca/ee/ca/profileSubmit:=
Couldn't connect to server.
> =
> in certmonger's log I also saw these:
> =
> nov 17 18:11:01 ipa1.itec.lab dogtag-ipa-ca-renew-agent-submit[30741]: Tr=
aceback (most recent call last):
> =
File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 533, =
in
> =
sys.exit(main())
> =
File "/usr/libexec/certmonger/dogtag-ipa-ca-renew-agent-submit", line 495, =
in main
> =
api.finalize()
> =
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 740, in fi=
nalize
> =
self.__do_if_not_done('load_plugins')
> =
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 431, in __=
do_if_not_done
> =
getattr(self, name)()
> =
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 620, in lo=
ad_plugins
> =
self.add_package(package)
> =
File "/usr/lib/python2.7/site-packages/ipalib/plugable.py", line 647, in ad=
d_package
> =
module =3D importlib.import_module(name)
> =
File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
> =
__import__(name)
> =
File "/usr/lib/python2.7/site-packages/ipaserver/plugins/server.py", line 3=
2, in
> =
from ipaserver.install import bindinstance, dnskeysyncinstance
> =
File "/usr/lib/python2.7/site-packages/ipaserver/install/dnskeysyncinstance=
.py", line 17, in
> =
from ipaserver import p11helper as _ipap11helper
> =
File "/usr/lib/python2.7/site-packages/ipaserver/p11helper.py", line 342, i=
n
> =
_libp11_kit =3D _ffi.dlopen(ctypes.util.find_library('p11-kit'))
> =
File "/usr/lib64/python2.7/ctypes/util.py", line 244, in find_library
> =
return _findSoname_ldconfig(name) or _get_soname(_findLib_gcc(name))
> =
File "/usr/lib64/python2.7/ctypes/util.py", line 233, in _findSoname_ldconf=
ig
> =
f =3D os.popen('/sbin/ldconfig -p 2>/dev/null')
> OS=
Error: [Errno 12] Cannot allocate memory
Is this host memory-constrained? How much RAM does it have?
rob
--===============8024469623496485149==--
From manuel.gugliucci at scaiitec.it Tue Feb 16 08:58:22 2021
Content-Type: multipart/mixed; boundary="===============0353205060416642309=="
MIME-Version: 1.0
From: Manuel Gujo
To: freeipa-users at lists.fedorahosted.org
Subject: [Freeipa-users] Re: IPA certs expired, pki-tomcatd fails to start
Date: Tue, 16 Feb 2021 08:58:13 +0000
Message-ID: <20210216085813.14637.6920@mailman01.iad2.fedoraproject.org>
In-Reply-To: 97a36513-8c3b-23fa-bff6-5a6a9a4c06fc@redhat.com
--===============0353205060416642309==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
> Manuel Gujo via FreeIPA-users wrote:
> =
> If the CA isn't running then there is no point in resubmitting the
> certmonger requests. It is guaranteed to fail with UNREACHABLE.
> =
> Check the journalctl output and the other logs, like catalina, in
> /var/log/pki/pki-tomcat for more information on why it failed to start.
> =
> =
> =
> Is this host memory-constrained? How much RAM does it have?
> =
> rob
there's new log on debug. Catalina does not log anything (0kb per file).
in debug:
Could not connect to LDAP server host ipa1.itec.lab port 636 Error netscape=
.ldap.LDAPException: Unable to create socket: org.mozilla.jss.ssl.SSLSocket=
Exception: org.mozilla.jss.ssl.SSLSocketExc
eption: SSL_ForceHandshake failed: (-8181) Peer's Certificate has expired. =
(-1)
in "system" logs says the same thing of debugs'
When I try to run 'ipactl start' without -f option, it says this:
# ipactl start
IPA version error: data needs to be upgraded (expected version '4.6.8-5.el7=
.centos', current version '4.4.0-14.el7.centos.4')
then after a while it fails and in /var/log/ipaupgrade.log says:
2020-11-17T18:25:05Z DEBUG httplib request failed:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/ipapython/dogtag.py", line 220, in=
_httplib_request
conn.request(method, path, body=3Drequest_body, headers=3Dheaders)
File "/usr/lib64/python2.7/httplib.py", line 1056, in request
self._send_request(method, url, body, headers)
File "/usr/lib64/python2.7/httplib.py", line 1090, in _send_request
self.endheaders(body)
File "/usr/lib64/python2.7/httplib.py", line 1052, in endheaders
self._send_output(message_body)
File "/usr/lib64/python2.7/httplib.py", line 890, in _send_output
self.send(msg)
File "/usr/lib64/python2.7/httplib.py", line 852, in send
self.connect()
File "/usr/lib64/python2.7/httplib.py", line 1266, in connect
HTTPConnection.connect(self)
File "/usr/lib64/python2.7/httplib.py", line 833, in connect
self.timeout, self.source_address)
File "/usr/lib64/python2.7/socket.py", line 571, in create_connection
raise err
error: [Errno 111] Connection refused
2020-11-17T18:25:05Z ERROR IPA server upgrade failed: Inspect /var/log/ipau=
pgrade.log and run command ipa-server-upgrade manually.
2020-11-17T18:25:05Z DEBUG File "/usr/lib/python2.7/site-packages/ipapyth=
on/admintool.py", line 178, in execute
return_value =3D self.run()
File "/usr/lib/python2.7/site-packages/ipaserver/install/ipa_server_upgra=
de.py", line 54, in run
server.upgrade()
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.p=
y", line 2176, in upgrade
upgrade_configuration()
File "/usr/lib/python2.7/site-packages/ipaserver/install/server/upgrade.p=
y", line 2059, in upgrade_configuration
cainstance.repair_profile_caIPAserviceCert()
File "/usr/lib/python2.7/site-packages/ipaserver/install/cainstance.py", =
line 1949, in repair_profile_caIPAserviceCert
with api.Backend.ra_certprofile as profile_api:
File "/usr/lib/python2.7/site-packages/ipaserver/plugins/dogtag.py", line=
1311, in __enter__
method=3D'GET'
File "/usr/lib/python2.7/site-packages/ipapython/dogtag.py", line 167, in=
https_request
method=3Dmethod, headers=3Dheaders)
File "/usr/lib/python2.7/site-packages/ipapython/dogtag.py", line 229, in=
_httplib_request
raise NetworkError(uri=3Duri, error=3Dstr(e))
2020-11-17T18:25:05Z DEBUG The ipa-server-upgrade command failed, exception=
: NetworkError: cannot connect to 'https://ipa1.itec.lab:8443/ca/rest/accou=
nt/login': [Errno 111] Connection refused
2020-11-17T18:25:05Z ERROR Unexpected error - see /var/log/ipaupgrade.log f=
or details:
After this run, I noticed that some of the certs went on Monitoring state
# getcert list
Number of certificates and requests being tracked: 9.
Request ID '20191231201955':
status: MONITORING
stuck: no
key pair storage: type=3DFILE,location=3D'/var/kerberos/krb5kdc/kdc.key'
certificate: type=3DFILE,location=3D'/var/kerberos/krb5kdc/kdc.crt'
CA: SelfSign
issuer: CN=3Dipa1.itec.lab,O=3DITEC.LAB
subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
expires: 2022-02-08 15:59:12 UTC
principal name: krbtgt/ITEC.LAB(a)ITEC.LAB
certificate template/profile: KDCs_PKINIT_Certs
pre-save command: =
post-save command: /usr/libexec/ipa/certmonger/renew_kdc_cert
track: yes
auto-renew: yes
Request ID '20201117182331':
status: CA_UNREACHABLE
ca-error: Error 7 connecting to http://ipa1.itec.lab:8080/ca/ee/ca/profile=
Submit: Couldn't connect to server.
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'auditSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'auditSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DCA Audit,O=3DITEC.LAB
expires: 2020-12-08 09:35:14 UTC
key usage: digitalSignature,nonRepudiation
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "auditSigning=
Cert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201117182333':
status: CA_UNREACHABLE
ca-error: Error 7 connecting to http://ipa1.itec.lab:8080/ca/ee/ca/profile=
Submit: Couldn't connect to server.
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'ocspSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'ocspSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DOCSP Subsystem,O=3DITEC.LAB
expires: 2020-12-08 09:38:07 UTC
eku: id-kp-OCSPSigning
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "ocspSigningC=
ert cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201117182335':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'subsystemCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'subsystemCert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DCA Subsystem,O=3DITEC.LAB
expires: 2022-11-07 18:24:47 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "subsystemCer=
t cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201117182336':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'caSigningCert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'caSigningCert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DCertificate Authority,O=3DITEC.LAB
expires: 2037-01-25 14:22:25 UTC
key usage: digitalSignature,nonRepudiation,keyCertSign,cRLSign
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "caSigningCer=
t cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201117182338':
status: CA_UNREACHABLE
ca-error: Error 7 connecting to http://ipa1.itec.lab:8080/ca/ee/ca/profile=
Submit: Couldn't connect to server.
stuck: no
key pair storage: type=3DFILE,location=3D'/var/lib/ipa/ra-agent.key'
certificate: type=3DFILE,location=3D'/var/lib/ipa/ra-agent.pem'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3DIPA RA,O=3DITEC.LAB
expires: 2020-12-08 09:37:47 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/renew_ra_cert_pre
post-save command: /usr/libexec/ipa/certmonger/renew_ra_cert
track: yes
auto-renew: yes
Request ID '20201117182339':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nick=
name=3D'Server-Cert cert-pki-ca',token=3D'NSS Certificate DB',pin set
certificate: type=3DNSSDB,location=3D'/etc/pki/pki-tomcat/alias',nickname=
=3D'Server-Cert cert-pki-ca',token=3D'NSS Certificate DB'
CA: dogtag-ipa-ca-renew-agent
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
expires: 2022-11-07 18:24:56 UTC
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: /usr/libexec/ipa/certmonger/stop_pkicad
post-save command: /usr/libexec/ipa/certmonger/renew_ca_cert "Server-Cert =
cert-pki-ca"
track: yes
auto-renew: yes
Request ID '20201117182342':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/dirsrv/slapd-ITEC-LAB',nic=
kname=3D'Server-Cert',token=3D'NSS Certificate DB',pinfile=3D'/etc/dirsrv/s=
lapd-ITEC-LAB/pwdfile.txt'
certificate: type=3DNSSDB,location=3D'/etc/dirsrv/slapd-ITEC-LAB',nickname=
=3D'Server-Cert',token=3D'NSS Certificate DB'
CA: IPA
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
expires: 2020-12-30 09:35:16 UTC
principal name: ldap/ipa1.itec.lab(a)ITEC.LAB
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: =
post-save command: /usr/libexec/ipa/certmonger/restart_dirsrv ITEC-LAB
track: yes
auto-renew: yes
Request ID '20201117182351':
status: MONITORING
stuck: no
key pair storage: type=3DNSSDB,location=3D'/etc/httpd/alias',nickname=3D'S=
erver-Cert',token=3D'NSS Certificate DB',pinfile=3D'/etc/httpd/alias/pwdfil=
e.txt'
certificate: type=3DNSSDB,location=3D'/etc/httpd/alias',nickname=3D'Server=
-Cert',token=3D'NSS Certificate DB'
CA: IPA
issuer: CN=3DCertificate Authority,O=3DITEC.LAB
subject: CN=3Dipa1.itec.lab,O=3DITEC.LAB
expires: 2020-12-30 09:35:04 UTC
principal name: HTTP/ipa1.itec.lab(a)ITEC.LAB
key usage: digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
eku: id-kp-serverAuth,id-kp-clientAuth
pre-save command: =
post-save command: /usr/libexec/ipa/certmonger/restart_httpd
track: yes
auto-renew: yes
# ipactl status
Directory Service: RUNNING
krb5kdc Service: RUNNING
kadmin Service: RUNNING
named Service: STOPPED
httpd Service: RUNNING
ipa-custodia Service: RUNNING
ntpd Service: STOPPED
pki-tomcatd Service: RUNNING
ipa-otpd Service: RUNNING
ipa-dnskeysyncd Service: RUNNING
ipa: INFO: The ipactl command was successful
but pki-tomcatd still fails if I try to restart it and in the debug logs:
[17/Nov/2020:18:32:34][http-bio-8080-exec-7]: CMSServlet:service() uri =3D =
/ca/admin/ca/getStatus
[17/Nov/2020:18:32:34][http-bio-8080-exec-7]: CMSServlet: caGetStatus start=
to service.
[17/Nov/2020:18:32:34][http-bio-8080-exec-7]: Failed to read product versio=
n String. java.io.FileNotFoundException: /usr/share/pki/CS_SERVER_VERSION (=
No such file or directory)
[17/Nov/2020:18:32:34][http-bio-8080-exec-7]: CMSServlet: curDate=3DTue Nov=
17 18:32:34 UTC 2020 id=3DcaGetStatus time=3D9
IPA VM has 2 CPU and 4GB of RAM, it never goes up to 90% of the usage
--===============0353205060416642309==--