Thanks again for the response! So, this is interesting. an
ldapsearch actually does find a record, yet if I use something like
Apache Directory Studio to try and look at it, it doesn't show up.
----
[root@orldc-prod-ipa01 alias]# ldapsearch -h localhost -p 389 -D
'cn=Directory Manager' -w "B\$ankers1" -b
"cn=268304420,ou=certificateRepository,ou=ca,o=ipaca" -LLL
dn: cn=268304420,ou=certificateRepository,ou=ca,o=ipaca
cn: 268304420
issuedBy: ipara
autoRenew: ENABLED
certStatus: VALID
dateOfModify: 20161216163020Z
dateOfCreate: 20161216163020Z
signingAlgorithmId: 1.2.840.113549.1.1.11
algorithmId: 1.2.840.113549.1.1.1
version: 2
userCertificate;binary:: MIIEIjxxxxxx
....
....
extension: 1.3.6.1.5.5.7.1.1
extension: 2.5.29.14
extension: 2.5.29.37
extension: 2.5.29.35
extension: 2.5.29.31
extension: 2.5.29.15
publicKeyData:: MIIBIjA...
....
....
issuerName: CN=Certificate Authority,O=PASSUR.LOCAL
subjectName: CN=orldc-prod-ipa01.passur.local,O=PASSUR.LOCAL
duration: 1163158400000
notAfter: 20181217163020Z
notBefore: 20161216163020Z
metaInfo: requestId:9980041
metaInfo: profileId:caIPAserviceCert
serialno: 09268304420
objectClass: top
objectClass: certificateRecord
----
Strange. I'm wondering if there is some permissions problem in the
directory? I have no idea how I would fix that if it were, however
this is, in itself, revealing.
On Tue, Dec 4, 2018 at 10:57 PM Fraser Tweedale <ftweedal(a)redhat.com> wrote:
>
> Hi Christopher,
>
> I agree with Rob that replication issue is the most likely cause.
> If there were replication issues, depending on your topology there
> may be serial/request ID range conflicts too. But the most critical
> issue is the about-to-expire certificate.
>
> A couple of quick points/questions:
>
> - The expiring certificate is the Server-Cert, other CA replicas
> will have different Server-Certs so they will continue to
> function.
>
> - Are there any other certs on this replica, or others, that are
> close to expiry?
>
> Now to the error:
>
> "http://orldc-prod-ipa01.passur.local:8080/ca/ee/ca/profileSubmit"
> replied: Record not found
>
> It is not clear exactly what record is missing. It is likely either
> the certificate record, or its corresponding request record. The
> Dogtag debug log (/var/log/pki/pki-tomcat/ca/debug) may reveal more.
>
> In any case, have a hunt for
>
> cn=268304422,ou=certificateRepository,ou=ca,o=ipaca
>
> If found, in the entry there should be an attribute:
>
> metaInfo: requestId:<N>
>
> for some value of <N>. Now also look for the entry:
>
> cn=<N>,ou=ca,ou=requests,o=ipaca
>
> If any of these entries can be found on other replicas but not the
> database on the replica where the cert is expiring, you can manually
> export/import them, and it might solve the issue.
>
> Otherwise, I recall a recent issue where the workaround was to make
> the Certmonger renewal helper do a "new issuance" rather than a
> "renewal"-based operation against the Dogtag CA. This could help in
> your situation too. I am not sure whether or where the steps were
> recorded so Rob, Florence - do you know?
>
> Anyhow it is possible I have gone down the garden path so it would
> really help to see the relevant portion of the Dogtag debug log.
> (Be aware Dogtag timestamps are in local time, when you are looking
> for the relevant output).
>
> Cheers,
> Fraser
>
> On Tue, Dec 04, 2018 at 09:47:11PM -0500, Christopher Young via FreeIPA-users wrote:
> > Another thing I notice that confuses me... (see attached)
> >
> >
> > Is it normal to have this many certificate with the same Subject for
> > an IPA server? I'm wondering if somewhere along it renewed and yet
> > didn't update locally or something. I'm really not sure what's
going
> > on here, but what I'm confused about is IF I wanted to generate a new
> > server for the IPA server, how would I go about doing that in a manner
> > where the certificate would have all the right attributes? (and would
> > I want to do that?)
> >
> > Sorry for all the questions. I'm figuring the pieces out as I go.
> > On Tue, Dec 4, 2018 at 9:04 PM Christopher Young <mexigabacho(a)gmail.com>
wrote:
> > >
> > > Output:
> > > ----
> > > [root@orldc-prod-ipa01 alias]# ipa-csreplica-manage list -v
> > > `hostname`.passur.local
> > > Directory Manager password:
> > >
> > > orldc-prod-ipa02.passur.local
> > > last init status: None
> > > last init ended: 1970-01-01 00:00:00+00:00
> > > last update status: Error (-1) Problem connecting to replica - LDAP
> > > error: Can't contact LDAP server (connection error)
> > > last update ended: 1970-01-01 00:00:00+00:00
> > > ----
> > >
> > > Granted, it's replication partner (orldc-prod-ipa02) is the one that I
> > > mentioned has the issue starting at this point. So, that likely has
> > > something to do with this output. Having said that, I'm not quite
> > > sure what I should do here. I have definitely been issuing certs from
> > > this system. One note is that the 'hostname' command on my
systems
> > > return only the short hostname. I'm not sure if that would be an
> > > issue or not, but it is worth noting. (To add 'domainname' does
show
> > > the proper domainname for the system and what-not.
> > >
> > > Any ideas on the best way to fix just this host? I don't mind the
> > > idea of removing the CA replicas on the others after fixing this and
> > > re-replicating or anything. I just want to get this functioning
> > > before something exprires and I end up in a really bad spot (which is
> > > sadly only a day away!). (Why, oh why, do we always 'find' these
> > > type of problems under a time crunch in this business?) :)
> > >
> > >
> > >
> > >
> > > On Tue, Dec 4, 2018 at 5:57 PM Rob Crittenden <rcritten(a)redhat.com>
wrote:
> > > >
> > > > Christopher Young via FreeIPA-users wrote:
> > > > > Yeah. I definitely lost on this one at this point. As far as I
can
> > > > > tell, SOMEHOW I'm missing these certs in the directory? Does
that
> > > > > sound right?
> > > > >
> > > > > How would one go about making sure is corrected? I'm guess
I'd need
> > > > > to regenerate some type of certificate on the IPA host, but
I'm afraid
> > > > > of breaking things worse. I have one more day before this one
> > > > > expires, so I'm trying to troubleshoot and fix it before
then. This
> > > > > all started when I noticed that another IPA server/replica failed
to
> > > > > restart. I think these two issues are related, but right now,
my
> > > > > users are functional with just the 'ipa01' system (which
has this
> > > > > ca-error' issue and the cert not found. I'm afraid to
restart
> > > > > anything on that system because of that.
> > > > >
> > > > > I'm still reading and trying to understand and put the
pieces
> > > > > together, however I'm worried about this issue.
> > > >
> > > > It sounds like one of your CA's is not replicating. You can use
> > > > ipa-csreplica-manage list -v `hostname` on each CA master to get the
status.
> > > >
> > > > Any given replica can only store so much data to replicate so
depending
> > > > on how long they have been disconnected could impact whether this is
> > > > easily recoverable.
> > > >
> > > > Given that, on any master it should always use its own CA (if there
is
> > > > one) when issuing certs so this is a bit strange.
> > > >
> > > > rob
> > > >
> > > > >
> > > > > Anyway, if anyone has any thoughts or tips here, I'd really
appreciate
> > > > > it as I feel lost at this exact moment.
> > > > > On Tue, Dec 4, 2018 at 2:33 PM Christopher Young
<mexigabacho(a)gmail.com> wrote:
> > > > >>
> > > > >> IPA 4.5.4 (has been upgraded for years just to understand
that there
> > > > >> is a history)
> > > > >> This system (ipa01) is the renewal master (in case that
matters)
> > > > >>
> > > > >> I'm getting the following error on 'getcert'. My
gut tells me this is
> > > > >> kinda a big deal. :) I really could use some help figuring
this one
> > > > >> out as I'm not the most CA-versed. I have been learning
quite a bit
> > > > >> reading some of the blogs, but there's definitely alot of
ignorance of
> > > > >> the details on my part.
> > > > >>
> > > > >> The error:
> > > > >> -----------
> > > > >> [root@orldc-prod-ipa01 log]# getcert list | grep -A12 -B1
error
> > > > >> status: MONITORING
> > > > >> ca-error: Server at
> > > > >>
"http://orldc-prod-ipa01.passur.local:8080/ca/ee/ca/profileSubmit"
> > > > >> replied: Record not found
> > > > >> stuck: no
> > > > >> key pair storage:
> > > > >>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> > > > >> cert-pki-ca',token='NSS Certificate DB',pin set
> > > > >> certificate:
> > > > >>
type=NSSDB,location='/etc/pki/pki-tomcat/alias',nickname='Server-Cert
> > > > >> cert-pki-ca',token='NSS Certificate DB'
> > > > >> CA: dogtag-ipa-ca-renew-agent
> > > > >> issuer: CN=Certificate Authority,O=PASSUR.LOCAL
> > > > >> subject:
CN=orldc-prod-ipa01.passur.local,O=PASSUR.LOCAL
> > > > >> expires: 2018-12-06 21:43:50 UTC
> > > > >> key usage:
> > > > >>
digitalSignature,nonRepudiation,keyEncipherment,dataEncipherment
> > > > >> eku:
id-kp-serverAuth,id-kp-clientAuth,id-kp-emailProtection
> > > > >> pre-save command:
/usr/libexec/ipa/certmonger/stop_pkicad
> > > > >> post-save command:
/usr/libexec/ipa/certmonger/renew_ca_cert
> > > > >> "Server-Cert cert-pki-ca"
> > > > >> track: yes
> > > > >> -----------
> > > > >>
> > > > >>
> > > > >> If I look at the cert referenced locally in the NSS DB:
> > > > >> ------
> > > > >> [root@orldc-prod-ipa01 log]# certutil -L -d
/etc/pki/pki-tomcat/alias
> > > > >> -f /etc/httpd/alias/pwdfile.txt
> > > > >>
> > > > >> Certificate Nickname
Trust Attributes
> > > > >>
SSL,S/MIME,JAR/XPI
> > > > >>
> > > > >> Server-Cert cert-pki-ca
u,u,u
> > > > >> auditSigningCert cert-pki-ca
u,u,Pu
> > > > >> caSigningCert cert-pki-ca
CTu,Cu,Cu
> > > > >> subsystemCert cert-pki-ca
u,u,u
> > > > >> ocspSigningCert cert-pki-ca
u,u,u
> > > > >> ------
> > > > >> [root@orldc-prod-ipa01 log]# certutil -L -d
/etc/pki/pki-tomcat/alias
> > > > >> -f /etc/httpd/alias/pwdfile.txt -n 'Server-Cert
cert-pki-ca' | grep
> > > > >> "Subject:\|Serial"
> > > > >> Serial Number: 268304422 (0xffe0026)
> > > > >> Subject:
"CN=orldc-prod-ipa01.passur.local,O=PASSUR.LOCAL"
> > > > >> -----
> > > > >> [root@orldc-prod-ipa01 log]# ipa cert-find
--min-serial-number
> > > > >> 268304422 --max-serial-number 268304423
> > > > >> ----------------------
> > > > >> 0 certificates matched
> > > > >> ----------------------
> > > > >> -----
> > > > >>
> > > > >> I'm trying to figure out how to find this certificate.
And IF somehow
> > > > >> it is wrong or missing, how do I fix such a scenario?
> > > > >>
> > > > >> Any help here is always appreciated! Unfortunately, I'm
running out
> > > > >> of time based on the expiration date I see on
'getcert'. I'm not sure
> > > > >> of the ramifications, but this seems pretty critical on the
surface.
> > > > >>
> > > > >> Thanks again for any help and direction!
> > > > >>
> > > > >> -- Chris
> > > > > _______________________________________________
> > > > > FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
> > > > > To unsubscribe send an email to
freeipa-users-leave(a)lists.fedorahosted.org
> > > > > Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> > > > > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > > > List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> > > > >
> > > >
>
>
> > _______________________________________________
> > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> > Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
>