On 8/16/17 5:24 AM, thierry bordaz via FreeIPA-users wrote:
Hi Ian
> Hmm.. This looks odd.
>
> [root@seattlenfs ianh]# ipa-csreplica-manage re-initialize --from
> freeipa-sea.bpt.rocks
> Directory Manager password:
>
> ipa: ERROR: Found multiple agreements for seattlenfs.bpt.rocks. Only
> initializing the first one returned:
> cn=cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat,cn=replica,cn=o\=ipaca,cn=mapping
> tree,cn=config
> Update in progress, 15 seconds elapsed
> [freeipa-sea.bpt.rocks] reports: Update failed! Status: [32 - LDAP
> error: No such object]
Here there are several replica agreement from freeipa-sea.bpt.rocks->
seattlenfs (for o=ipaca suffix).
I am not sure how this can happen.
It could be some remaining entries from previous install/uninstall or
multiple initialization from different sources.
If it's possible to do
something wrong I am sure I have done it. I have
tried the various clean-ruv commands for the apparently existing
agreements with nonexistent servers many times but it seems ot have no
effect.
Looking at the error below, I wonder if there were actually 2 replica
agreements freeipa-sea->seattlenfs that are
* cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat
* masterAgreement1-seattlenfs.bpt.rocks-pki-tomcat
Would you dump (on all servers freeipa-sea, seattlenfs and freeipa-dal):
ldapsearch -LLL -D 'cn=directory manager' -W -b
"cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" nscpentrywsi
freeipa-dal no longer exists. Also, since they also appear below...
neither do edinburghnfs, fremontnis, bellevuenfs or bpt-nyc1-nfs. The
one causing me the most obvious headaches is freeipa-dal.
[root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory manager' -W -b
"cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" nscpentrywsi
Enter LDAP Password:
dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
nscpentrywsi: dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
nscpentrywsi: cn: replica
nscpentrywsi: createTimestamp: 20160814234939Z
nscpentrywsi: creatorsName: cn=directory manager
nscpentrywsi: modifiersName: cn=Multimaster Replication
Plugin,cn=plugins,cn=c
onfig
nscpentrywsi: modifyTimestamp: 20170816141634Z
nscpentrywsi: nsDS5Flags: 1
nscpentrywsi: nsDS5ReplicaBindDN: cn=Replication Manager
cloneAgreement1-freei
pa-sea.bpt.rocks-pki-tomcat,ou=csusers,cn=config
nscpentrywsi: nsDS5ReplicaBindDN: cn=Replication Manager
masterAgreement1-seat
tlenfs.bpt.rocks-pki-tomcat,ou=csusers,cn=config
nscpentrywsi: nsDS5ReplicaId: 1065
nscpentrywsi: nsDS5ReplicaName: b21a1f1e-627911e6-93e6ef4b-69dcc2d1
nscpentrywsi: nsDS5ReplicaRoot: o=ipaca
nscpentrywsi: nsDS5ReplicaType: 3
nscpentrywsi: nsState::
KQQAAAAAAADAU5RZAAAAAAAAAAAAAAAAKgAAAAAAAAAAAAAAAAAAAA
==
nscpentrywsi: nsds5replicabinddngroup: cn=replication
managers,cn=sysaccounts,
cn=etc,dc=bpt,dc=rocks
nscpentrywsi: nsds5replicabinddngroupcheckinterval: 60
nscpentrywsi: objectClass: top
nscpentrywsi: objectClass: nsDS5Replica
nscpentrywsi: objectClass: extensibleobject
nscpentrywsi: numSubordinates: 2
nscpentrywsi: nsds5ReplicaChangeCount: 961
nscpentrywsi: nsds5replicareapactive: 0
dn:
cn=cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat,cn=replica,cn=o\3Dipac
a,cn=mapping tree,cn=config
nscpentrywsi: dn:
cn=cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat,cn=repli
ca,cn=o\3Dipaca,cn=mapping tree,cn=config
nscpentrywsi: cn: cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat
nscpentrywsi: createTimestamp: 20160814234940Z
nscpentrywsi: creatorsName: cn=directory manager
nscpentrywsi: description: cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat
nscpentrywsi: modifiersName: cn=Multimaster Replication
Plugin,cn=plugins,cn=c
onfig
nscpentrywsi: modifyTimestamp: 20170810183437Z
nscpentrywsi: nsDS5ReplicaBindDN: cn=Replication Manager
masterAgreement1-free
ipa-sea.bpt.rocks-pki-tomcat,ou=csusers,cn=config
nscpentrywsi: nsDS5ReplicaBindMethod: Simple
nscpentrywsi: nsDS5ReplicaCredentials:
{AES-TUhNR0NTcUdTSWIzRFFFRkRUQm1NRVVHQ1
NxR1NJYjNEUUVGRERBNEJDUXhNemsyTlRka05TMHhZMlE1TTJKbA0KT1MwNU1HTm1PV0V6T1MwMk5
qazNNV0V3TndBQ0FRSUNBU0F3Q2dZSUtvWklodmNOQWdjd0hRWUpZSVpJQVdVRA0KQkFFcUJCQ0da
bE1zY2Nod0FQeFlXMkF0Y3RhTA==}AIuTmb+o3/MWgo0mc8r8fw==
nscpentrywsi: nsDS5ReplicaHost: seattlenfs.bpt.rocks
nscpentrywsi: nsDS5ReplicaPort: 389
nscpentrywsi: nsDS5ReplicaRoot: o=ipaca
nscpentrywsi: nsDS5ReplicaTransportInfo: TLS
nscpentrywsi: nsds50ruv: {replicageneration} 55c8f3ae000000600000
nscpentrywsi: nsds50ruv: {replica 81 ldap://seattlenfs.bpt.rocks:389}
568ac431
000000510000 57b103ce000500510000
nscpentrywsi: nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
57b10
3d4000004290000 57b254df000004290000
nscpentrywsi: nsds50ruv: {replica 1095 ldap://freeipa-sea.bpt.rocks:389}
579a9
63c000004470000 57a42389000104470000
nscpentrywsi: nsds50ruv: {replica 96 ldap://freeipa-sea.bpt.rocks:389}
55c8f3b
d000000600000 5799a02e000000600000
nscpentrywsi: nsds5ReplicaEnabled: on
nscpentrywsi: nsruvReplicaLastModified: {replica 81
ldap://seattlenfs.bpt.rock
s:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1065
ldap://freeipa-sea.bpt.r
ocks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1095
ldap://freeipa-sea.bpt.r
ocks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 96
ldap://freeipa-sea.bpt.roc
ks:389} 00000000
nscpentrywsi: objectClass: top
nscpentrywsi: objectClass: nsds5replicationagreement
nscpentrywsi: nsds5replicareapactive: 0
nscpentrywsi: nsds5replicaLastUpdateStart: 19700101000000Z
nscpentrywsi: nsds5replicaLastUpdateEnd: 19700101000000Z
nscpentrywsi: nsds5replicaChangesSentSinceStartup:
nscpentrywsi: nsds5replicaLastUpdateStatus: Error (32) Problem
connecting to r
eplica - LDAP error: No such object (connection error)
nscpentrywsi: nsds5replicaUpdateInProgress: FALSE
nscpentrywsi: nsds5replicaLastInitStart: 19700101000000Z
nscpentrywsi: nsds5replicaLastInitEnd: 19700101000000Z
dn:
cn=masterAgreement1-seattlenfs.bpt.rocks-pki-tomcat,cn=replica,cn=o\3Dipac
a,cn=mapping tree,cn=config
nscpentrywsi: dn:
cn=masterAgreement1-seattlenfs.bpt.rocks-pki-tomcat,cn=repli
ca,cn=o\3Dipaca,cn=mapping tree,cn=config
nscpentrywsi: cn: masterAgreement1-seattlenfs.bpt.rocks-pki-tomcat
nscpentrywsi: createTimestamp: 20160825052011Z
nscpentrywsi: creatorsName: uid=pkidbuser,ou=people,o=ipaca
nscpentrywsi: description: masterAgreement1-seattlenfs.bpt.rocks-pki-tomcat
nscpentrywsi: modifiersName: cn=Multimaster Replication
Plugin,cn=plugins,cn=c
onfig
nscpentrywsi: modifyTimestamp: 20170706211220Z
nscpentrywsi: nsDS5ReplicaBindDN: cn=Replication Manager
cloneAgreement1-seatt
lenfs.bpt.rocks-pki-tomcat,ou=csusers,cn=config
nscpentrywsi: nsDS5ReplicaBindMethod: Simple
nscpentrywsi: nsDS5ReplicaCredentials:
{AES-TUhNR0NTcUdTSWIzRFFFRkRUQm1NRVVHQ1
NxR1NJYjNEUUVGRERBNEJDUXhNemsyTlRka05TMHhZMlE1TTJKbA0KT1MwNU1HTm1PV0V6T1MwMk5
qazNNV0V3TndBQ0FRSUNBU0F3Q2dZSUtvWklodmNOQWdjd0hRWUpZSVpJQVdVRA0KQkFFcUJCQnZ6
Z2tGUzErSjlvdEVYRlFJaVQyMA==}yhEAKKElqudqql31/kWsUw==
nscpentrywsi: nsDS5ReplicaHost: seattlenfs.bpt.rocks
nscpentrywsi: nsDS5ReplicaPort: 389
nscpentrywsi: nsDS5ReplicaRoot: o=ipaca
nscpentrywsi: nsDS5ReplicaTransportInfo: TLS
nscpentrywsi: nsds50ruv: {replicageneration} 55c8f3ae000000600000
nscpentrywsi: nsds50ruv: {replica 1290 ldap://seattlenfs.bpt.rocks:389}
57be80
4c0000050a0000 587236150000050a0000
nscpentrywsi: nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
57b10
3d4000004290000 57be8043000704290000
nscpentrywsi: nsds50ruv: {replica 1070 ldap://bellevuenfs.bpt.rocks:389}
57a4f
2700000042e0000 57a4f2700007042e0000
nscpentrywsi: nsds50ruv: {replica 1075
ldap://bpt-nyc1-nfs.bpt.rocks:389} 57a4
7865000004330000 57b111c8000404330000
nscpentrywsi: nsds50ruv: {replica 1085 ldap://fremontnis.bpt.rocks:389}
57a403
e60000043d0000 57a643ea0000043d0000
nscpentrywsi: nsds50ruv: {replica 1090 ldap://freeipa-dal.bpt.rocks:389}
57a2d
d35000004420000 57a2dd35000404420000
nscpentrywsi: nsds50ruv: {replica 1195
ldap://edinburghnfs.bpt.rocks:389} 57a4
2390000004ab0000 57a42390000404ab0000
nscpentrywsi: nsruvReplicaLastModified: {replica 1290
ldap://seattlenfs.bpt.ro
cks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1065
ldap://freeipa-sea.bpt.r
ocks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1070
ldap://bellevuenfs.bpt.r
ocks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1075
ldap://bpt-nyc1-nfs.bpt.
rocks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1085
ldap://fremontnis.bpt.ro
cks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1090
ldap://freeipa-dal.bpt.r
ocks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1195
ldap://edinburghnfs.bpt.
rocks:389} 00000000
nscpentrywsi: objectClass: top
nscpentrywsi: objectClass: nsds5replicationagreement
nscpentrywsi: nsds5replicareapactive: 0
nscpentrywsi: nsds5replicaLastUpdateStart: 19700101000000Z
nscpentrywsi: nsds5replicaLastUpdateEnd: 19700101000000Z
nscpentrywsi: nsds5replicaChangesSentSinceStartup:
nscpentrywsi: nsds5replicaLastUpdateStatus: Error (19) Replication error
acqui
ring replica: Replica has different database generation ID, remote
replica ma
y need to be initialized (RUV error)
nscpentrywsi: nsds5replicaUpdateInProgress: FALSE
nscpentrywsi: nsds5replicaLastInitStart: 19700101000000Z
nscpentrywsi: nsds5replicaLastInitEnd: 19700101000000Z
##################
[root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory manager' -W -b
"cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config" nscpentrywsi
Enter LDAP Password:
dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
nscpentrywsi: dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
nscpentrywsi: cn: replica
nscpentrywsi: createTimestamp: 20160825052011Z
nscpentrywsi: creatorsName: cn=directory manager
nscpentrywsi: modifiersName: cn=Multimaster Replication
Plugin,cn=plugins,cn=c
onfig
nscpentrywsi: modifyTimestamp: 20170810182749Z
nscpentrywsi: nsDS5Flags: 1
nscpentrywsi: nsDS5ReplicaBindDN: cn=Replication Manager
cloneAgreement1-seatt
lenfs.bpt.rocks-pki-tomcat,ou=csusers,cn=config
nscpentrywsi: nsDS5ReplicaId: 1290
nscpentrywsi: nsDS5ReplicaName: 8706ab1e-6a8311e6-a9bfdbbf-74dc7323
nscpentrywsi: nsDS5ReplicaRoot: o=ipaca
nscpentrywsi: nsDS5ReplicaType: 3
nscpentrywsi: nsState::
CgUAAAAAAACkpYxZAAAAAAAAAAAAAAAAKgAAAAAAAAABAAAAAAAAAA
==
nscpentrywsi: nsds5replicabinddngroup: cn=replication
managers,cn=sysaccounts,
cn=etc,dc=bpt,dc=rocks
nscpentrywsi: nsds5replicabinddngroupcheckinterval: 60
nscpentrywsi: objectClass: top
nscpentrywsi: objectClass: nsDS5Replica
nscpentrywsi: objectClass: extensibleobject
nscpentrywsi: numSubordinates: 1
nscpentrywsi: nsds5ReplicaChangeCount: 3
nscpentrywsi: nsds5replicareapactive: 0
dn:
cn=cloneAgreement1-seattlenfs.bpt.rocks-pki-tomcat,cn=replica,cn=o\3Dipaca
,cn=mapping tree,cn=config
nscpentrywsi: dn:
cn=cloneAgreement1-seattlenfs.bpt.rocks-pki-tomcat,cn=replic
a,cn=o\3Dipaca,cn=mapping tree,cn=config
nscpentrywsi: cn: cloneAgreement1-seattlenfs.bpt.rocks-pki-tomcat
nscpentrywsi: createTimestamp: 20160825052011Z
nscpentrywsi: creatorsName: cn=directory manager
nscpentrywsi: description: cloneAgreement1-seattlenfs.bpt.rocks-pki-tomcat
nscpentrywsi: modifiersName: cn=directory manager
nscpentrywsi: modifyTimestamp: 20170810183422Z
nscpentrywsi: nsDS5ReplicaBindDN: cn=Replication Manager
masterAgreement1-seat
tlenfs.bpt.rocks-pki-tomcat,ou=csusers,cn=config
nscpentrywsi: nsDS5ReplicaBindMethod: Simple
nscpentrywsi: nsDS5ReplicaCredentials:
{AES-TUhNR0NTcUdTSWIzRFFFRkRUQm1NRVVHQ1
NxR1NJYjNEUUVGRERBNEJDUXhNemsyTlRka05TMHhZMlE1TTJKbA0KT1MwNU1HTm1PV0V6T1MwMk5
qazNNV0V3TndBQ0FRSUNBU0F3Q2dZSUtvWklodmNOQWdjd0hRWUpZSVpJQVdVRA0KQkFFcUJCQzZI
d20zcHNMYTZKT3dQVDg4RW9XRA==}PpeZp0WYyNiA7DWqrsJfBw==
nscpentrywsi: nsDS5ReplicaHost: freeipa-sea.bpt.rocks
nscpentrywsi: nsDS5ReplicaPort: 389
nscpentrywsi: nsDS5ReplicaRoot: o=ipaca
nscpentrywsi: nsDS5ReplicaTransportInfo: TLS
nscpentrywsi: nsds50ruv: {replicageneration} 57c291d9000004290000
nscpentrywsi: nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
57f84
0bf000004290000 598ca31f000104290000
nscpentrywsi: nsds50ruv: {replica 1290 ldap://seattlenfs.bpt.rocks:389}
nscpentrywsi: nsruvReplicaLastModified: {replica 1065
ldap://freeipa-sea.bpt.r
ocks:389} 00000000
nscpentrywsi: nsruvReplicaLastModified: {replica 1290
ldap://seattlenfs.bpt.ro
cks:389} 00000000
nscpentrywsi: objectClass: top
nscpentrywsi: objectClass: nsds5replicationagreement
nscpentrywsi: nsds5ReplicaEnabled: on
nscpentrywsi: nsds5replicareapactive: 0
nscpentrywsi: nsds5replicaLastUpdateStart: 19700101000000Z
nscpentrywsi: nsds5replicaLastUpdateEnd: 19700101000000Z
nscpentrywsi: nsds5replicaChangesSentSinceStartup:
nscpentrywsi: nsds5replicaLastUpdateStatus: Error (19) Replication error
acqui
ring replica: Replica has different database generation ID, remote
replica ma
y need to be initialized (RUV error)
nscpentrywsi: nsds5replicaUpdateInProgress: FALSE
nscpentrywsi: nsds5replicaLastInitStart: 19700101000000Z
nscpentrywsi: nsds5replicaLastInitEnd: 19700101000000Z
best regards
thierry
>
>> On freeipa-sea and seattlenfs you may check the ipaca replica
>> agreements status before/after reinit. I would guess they are
>> currently reporting failures between these two replicas.
>>
>> ldapsearch -LLL -D 'cn=directory manager' -W -b
>> "cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config"
>> "objectClass=nsds5replicationagreement" nsds5replicaLastUpdateStatus
>>
> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory manager' -W
> -b "cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config"
> "objectClass=nsds5replicationagreement" nsds5replicaLastUpdateStatus
> Enter LDAP Password:
> dn:
> cn=cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat,cn=replica,cn=o\3Dipac
> a,cn=mapping tree,cn=config
> nsds5replicaLastUpdateStatus: Error (32) Problem connecting to
> replica - LDAP
> error: No such object (connection error)
>
> dn:
> cn=masterAgreement1-seattlenfs.bpt.rocks-pki-tomcat,cn=replica,cn=o\3Dipac
> a,cn=mapping tree,cn=config
> nsds5replicaLastUpdateStatus: Error (19) Replication error acquiring
> replica:
> Replica has different database generation ID, remote replica may
> need to be i
> nitialized (RUV error)
>
> and
>
> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory manager' -W
> -b "cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config"
> "objectClass=nsds5replicationagreement" nsds5replicaLastUpdateStatus
> Enter LDAP Password:
> dn:
> cn=cloneAgreement1-seattlenfs.bpt.rocks-pki-tomcat,cn=replica,cn=o\3Dipaca
> ,cn=mapping tree,cn=config
> nsds5replicaLastUpdateStatus: Error (19) Replication error acquiring
> replica:
> Replica has different database generation ID, remote replica may
> need to be i
> nitialized (RUV error)
>
>
>>>
>>> several times over the years when replication has stopped.
>>>
>>> Replication actually is working, as far as user and machine
>>> accounts and attributes are concerned anyway, and has been for a while.
>>>
>>> There is a zombie server freeipa-dal.bpt.rocks that I can't get rid
>>> of... it shows up in the GUI on the Topology page but generates a
>>> Server not found error. I don't know if that's related.
>>
>> Zombie servers have been discussed a lot on this mailing list. You
>> may get rid of them with list-ruv/clean-ruv subcommand. Replication
>> usually manage to work fine even if some zombie entries exist but it
>> is a good practice to clean them.
> Yeah, I've been through every possible combination of
> list-ruv/clean-ruv. I have something odd going on in that no
> replication agreements show up anywhere for freeipa-dal.
>>
>> regards
>> thierry
>>>>
>>>> On 08/08/2017 10:33 PM, Ian Harding via FreeIPA-users wrote:
>>>>>
>>>>> On 8/7/17 1:44 AM, thierry bordaz wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On 08/07/2017 09:22 AM, Florence Blanc-Renaud via FreeIPA-users
>>>>>> wrote:
>>>>>>> On 08/04/2017 11:02 PM, Ian Harding via FreeIPA-users wrote:
>>>>>>>> On 8/4/17 2:16 AM, Florence Blanc-Renaud wrote:
>>>>>>>>
>>>>>>>>> On 08/03/2017 11:13 PM, Ian Harding via FreeIPA-users
wrote:
>>>>>>>>>> On 08/03/2017 12:28 AM, Florence Blanc-Renaud
wrote:
>>>>>>>>>>> On 08/02/2017 11:51 PM, Ian Harding via
FreeIPA-users wrote:
>>>>>>>>>>>> On 08/02/2017 12:11 AM, Florence
Blanc-Renaud wrote:
>>>>>>>>>>>>> On 08/02/2017 01:43 AM, Ian Harding
wrote:
>>>>>>>>>>>>>> On 08/01/2017 12:03 PM, Rob
Crittenden wrote:
>>>>>>>>>>>>>>> Ian Harding wrote:
>>>>>>>>>>>>>>>> On 08/01/2017 07:39 AM,
Florence Blanc-Renaud wrote:
>>>>>>>>>>>>>>>>> On 08/01/2017 03:11
PM, Ian Harding wrote:
>>>>>>>>>>>>>>>>>> On 08/01/2017
01:48 AM, Florence Blanc-Renaud wrote:
>>>>>>>>>>>>>>>>>>> On 08/01/2017
01:32 AM, Ian Harding via
>>>>>>>>>>>>>>>>>>> FreeIPA-users
wrote:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> On
07/31/2017 11:34 AM, Rob Crittenden wrote:
>>>>>>>>>>>>>>>>>>>>> Ian
Harding via FreeIPA-users wrote:
>>>>>>>>>>>>>>>>>>>>>> I
had an unexpected restart of an IPA server
>>>>>>>>>>>>>>>>>>>>>>
that had apparently had
>>>>>>>>>>>>>>>>>>>>>>
updates run but had not been restarted. ipactl
>>>>>>>>>>>>>>>>>>>>>>
says pki-tomcatd
>>>>>>>>>>>>>>>>>>>>>>
would
>>>>>>>>>>>>>>>>>>>>>>
not start.
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>
Strangely, the actual service appears to be
>>>>>>>>>>>>>>>>>>>>>>
running:
>>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>
dogtag is an application within tomcat so tomcat
>>>>>>>>>>>>>>>>>>>>> can
run without
>>>>>>>>>>>>>>>>>>>>>
dogtag
>>>>>>>>>>>>>>>>>>>>>
running.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>> We
need to see more of the dogtag debug log to
>>>>>>>>>>>>>>>>>>>>> see
what is going on.
>>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> It looks
like an authentication problem...
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
[28/Jul/2017:10:08:47][localhost-startStop-1]: SSL
>>>>>>>>>>>>>>>>>>>> handshake
happened
>>>>>>>>>>>>>>>>>>>> Could not
connect to LDAP server host
>>>>>>>>>>>>>>>>>>>>
seattlenfs.bpt.rocks port 636
>>>>>>>>>>>>>>>>>>>> Error
netscape.ldap.LDAPException: Authentication
>>>>>>>>>>>>>>>>>>>> failed
(49)
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> dogtag stores
its internal data in the LDAP server
>>>>>>>>>>>>>>>>>>> and needs to
>>>>>>>>>>>>>>>>>>> establish a
secure LDAP connection. You can check
>>>>>>>>>>>>>>>>>>> how this
>>>>>>>>>>>>>>>>>>> connection is
configured in
>>>>>>>>>>>>>>>>>>>
/etc/pki/pki-tomcat/ca/CS.cfg, look for
>>>>>>>>>>>>>>>>>>> the lines:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
internaldb.ldapauth.authtype=SslClientAuth
>>>>>>>>>>>>>>>>>>>
internaldb.ldapauth.bindDN=cn=Directory Manager
>>>>>>>>>>>>>>>>>>>
internaldb.ldapauth.bindPWPrompt=internaldb
>>>>>>>>>>>>>>>>>>>
internaldb.ldapauth.clientCertNickname=subsystemCert
>>>>>>>>>>>>>>>>>>> cert-pki-ca
>>>>>>>>>>>>>>>>>>>
internaldb.ldapconn.host=vm-...
>>>>>>>>>>>>>>>>>>>
internaldb.ldapconn.port=636
>>>>>>>>>>>>>>>>>>>
internaldb.ldapconn.secureConn
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> authtype can
be SslClientAuth (authentication with
>>>>>>>>>>>>>>>>>>> a ssl
>>>>>>>>>>>>>>>>>>> certificate)
or BasicAuth (authentication with a
>>>>>>>>>>>>>>>>>>> bind DN and
>>>>>>>>>>>>>>>>>>> password
stored in
>>>>>>>>>>>>>>>>>>>
/var/lib/pki/pki-tomcat/conf/password.conf).
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> You can use
this information to manually check the
>>>>>>>>>>>>>>>>>>> credentials.
For
>>>>>>>>>>>>>>>>>>> instance with
sslclientauth:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> export
LDAPTLS_CACERTDIR=/etc/pki/pki-tomcat/alias
>>>>>>>>>>>>>>>>>>> export
LDAPTLS_CERT='subsystemCert cert-pki-ca'
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ldapsearch -H
ldaps://`hostname`:636 -b "" -s base
>>>>>>>>>>>>>>>>>>> -Y EXTERNAL
>>>>>>>>>>>>>>>>>>> (provide the
password from
>>>>>>>>>>>>>>>>>>>
/etc/pki/pki-tomcat/alias/pwdfile.txt)
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> I found this:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
internaldb.ldapauth.authtype=SslClientAuth
>>>>>>>>>>>>>>>>>>
internaldb.ldapauth.bindDN=uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
internaldb.ldapauth.bindPWPrompt=internaldb
>>>>>>>>>>>>>>>>>>
internaldb.ldapauth.clientCertNickname=subsystemCert
>>>>>>>>>>>>>>>>>> cert-pki-ca
>>>>>>>>>>>>>>>>>>
internaldb.ldapconn.cloneReplicationPort=389
>>>>>>>>>>>>>>>>>> ...
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> and when I try
the ldapsearch I am presented with a
>>>>>>>>>>>>>>>>>> prompt to
provide
>>>>>>>>>>>>>>>>>> a pin/password
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Please enter pin,
password, or pass phrase for
>>>>>>>>>>>>>>>>>> security token
'ldap(0)':
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> but there is no
password file...
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> you are right, in
4.4. there is no pwdfile.txt and
>>>>>>>>>>>>>>>>> the password can be
>>>>>>>>>>>>>>>>> found in
/var/lib/pki/pki-tomcat/conf/password.conf
>>>>>>>>>>>>>>>>> (with the tag
>>>>>>>>>>>>>>>>> internal=...)
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Can you check if the
password with the tag
>>>>>>>>>>>>>>>>> internal=... allows
to read
>>>>>>>>>>>>>>>>> the keys from the NSS
db?
>>>>>>>>>>>>>>>>> certutil -K -d
/etc/pki/pki-tomcat/alias
>>>>>>>>>>>>>>>>> (provide password)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> That works...
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> # certutil -K -d
/etc/pki/pki-tomcat/alias
>>>>>>>>>>>>>>>> certutil: Checking token
"NSS Certificate DB" in slot
>>>>>>>>>>>>>>>> "NSS User Private
>>>>>>>>>>>>>>>> Key and Certificate
Services"
>>>>>>>>>>>>>>>> Enter Password or Pin for
"NSS Certificate DB":
>>>>>>>>>>>>>>>> < 0> rsa
0f327e760a7eecdcf6973f5dc57ca5367c592d64
>>>>>>>>>>>>>>>> (orphan)
>>>>>>>>>>>>>>>> < 1> rsa
b12580c7c696cfcd8aefc9405a7a870b24b7b96a NSS
>>>>>>>>>>>>>>>> Certificate
>>>>>>>>>>>>>>>> DB:auditSigningCert
cert-pki-ca
>>>>>>>>>>>>>>>> < 2> rsa
881b7254c40fa40bc50681bcc8d37bb3eb49937e
>>>>>>>>>>>>>>>> caSigningCert
>>>>>>>>>>>>>>>> cert-pki-ca
>>>>>>>>>>>>>>>> < 3> rsa
fa9a255a1d15585ac28064c0f4986e416bc48403 NSS
>>>>>>>>>>>>>>>> Certificate
>>>>>>>>>>>>>>>> DB:ocspSigningCert
cert-pki-ca
>>>>>>>>>>>>>>>> < 4> rsa
3fb609d0f7d72c2d325d6a2dc16577a7f7e5a01f
>>>>>>>>>>>>>>>> Server-Cert
>>>>>>>>>>>>>>>> cert-pki-ca
>>>>>>>>>>>>>>>> < 5> rsa
1e9479a9556af9339bb5e4552ccbd381d3c38856 NSS
>>>>>>>>>>>>>>>> Certificate
>>>>>>>>>>>>>>>> DB:subsystemCert
cert-pki-ca
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> But this doesn't
(with the same password from
>>>>>>>>>>>>>>>> password.conf)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> # ldapsearch -H
ldaps://`hostname`:636 -b "" -s base
>>>>>>>>>>>>>>>> -Y EXTERNAL
>>>>>>>>>>>>>>>> Please enter pin,
password, or pass phrase for
>>>>>>>>>>>>>>>> security token
'ldap(0)':
>>>>>>>>>>>>>>>> SASL/EXTERNAL
authentication started
>>>>>>>>>>>>>>>>
ldap_sasl_interactive_bind_s: Invalid credentials (49)
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> That password is getting
me somewhere though, since if
>>>>>>>>>>>>>>>> I put in a
>>>>>>>>>>>>>>>> nonsense or incorrect
password it just prompts over
>>>>>>>>>>>>>>>> and over.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Let's step back a second.
You upgraded from what to what?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> There wasn't much of a
change... I just assumed someone
>>>>>>>>>>>>>> ran yum upgrade and didn't
restart, then the power
>>>>>>>>>>>>>> outage... it looks like not much
of a version change
>>>>>>>>>>>>>> though.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> # grep ipa /var/log/yum.log
>>>>>>>>>>>>>> Jan 08 04:45:32 Installed:
>>>>>>>>>>>>>>
ipa-common-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:45:32 Installed:
>>>>>>>>>>>>>>
ipa-client-common-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:46:06 Updated:
>>>>>>>>>>>>>>
libipa_hbac-1.14.0-43.el7_3.4.x86_64
>>>>>>>>>>>>>> Jan 08 04:46:07 Updated:
>>>>>>>>>>>>>>
python-libipa_hbac-1.14.0-43.el7_3.4.x86_64
>>>>>>>>>>>>>> Jan 08 04:46:08 Installed:
>>>>>>>>>>>>>>
python-ipaddress-1.0.16-2.el7.noarch
>>>>>>>>>>>>>> Jan 08 04:47:04 Installed:
>>>>>>>>>>>>>>
python2-ipalib-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:47:05 Installed:
>>>>>>>>>>>>>>
python2-ipaclient-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:47:05 Updated:
>>>>>>>>>>>>>>
ipa-admintools-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:47:05 Installed:
>>>>>>>>>>>>>>
ipa-server-common-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:47:06 Installed:
>>>>>>>>>>>>>>
python2-ipaserver-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:47:07 Updated:
sssd-ipa-1.14.0-43.el7_3.4.x86_64
>>>>>>>>>>>>>> Jan 08 04:47:17 Updated:
>>>>>>>>>>>>>>
ipa-client-4.4.0-14.el7.centos.1.1.x86_64
>>>>>>>>>>>>>> Jan 08 04:47:23 Updated:
>>>>>>>>>>>>>>
ipa-server-4.4.0-14.el7.centos.1.1.x86_64
>>>>>>>>>>>>>> Jan 08 04:47:27 Updated:
>>>>>>>>>>>>>>
ipa-server-dns-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:47:36 Installed:
>>>>>>>>>>>>>>
ipa-python-compat-4.4.0-14.el7.centos.1.1.noarch
>>>>>>>>>>>>>> Jan 08 04:50:07 Erased:
>>>>>>>>>>>>>>
ipa-python-4.2.0-15.0.1.el7.centos.19.x86_64
>>>>>>>>>>>>>> Jul 28 09:55:41 Updated:
>>>>>>>>>>>>>>
ipa-common-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>> Jul 28 09:55:42 Updated:
>>>>>>>>>>>>>>
ipa-client-common-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>> Jul 28 09:55:43 Updated:
>>>>>>>>>>>>>>
ipa-server-common-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>> Jul 28 09:55:43 Updated:
>>>>>>>>>>>>>>
python2-ipalib-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>> Jul 28 09:55:44 Updated:
>>>>>>>>>>>>>>
python2-ipaclient-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>> Jul 28 09:55:45 Updated:
>>>>>>>>>>>>>>
python2-ipaserver-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>> Jul 28 09:55:45 Updated:
>>>>>>>>>>>>>>
ipa-admintools-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>> Jul 28 09:55:46 Updated:
>>>>>>>>>>>>>>
ipa-client-4.4.0-14.el7.centos.7.x86_64
>>>>>>>>>>>>>> Jul 28 09:55:48 Updated:
>>>>>>>>>>>>>>
ipa-server-4.4.0-14.el7.centos.7.x86_64
>>>>>>>>>>>>>> Jul 28 09:55:48 Updated:
>>>>>>>>>>>>>>
ipa-server-dns-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>> Jul 28 09:55:48 Updated:
>>>>>>>>>>>>>>
ipa-python-compat-4.4.0-14.el7.centos.7.noarch
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> # grep pki /var/log/yum.log
>>>>>>>>>>>>>> Jan 08 04:46:05 Updated:
pki-base-10.3.3-14.el7_3.noarch
>>>>>>>>>>>>>> Jan 08 04:46:09 Updated:
krb5-pkinit-1.14.1-27.el7_3.x86_64
>>>>>>>>>>>>>> Jan 08 04:47:19 Installed:
>>>>>>>>>>>>>>
pki-base-java-10.3.3-14.el7_3.noarch
>>>>>>>>>>>>>> Jan 08 04:47:19 Updated:
pki-tools-10.3.3-14.el7_3.x86_64
>>>>>>>>>>>>>> Jan 08 04:47:21 Updated:
pki-server-10.3.3-14.el7_3.noarch
>>>>>>>>>>>>>> Jan 08 04:47:22 Updated:
pki-kra-10.3.3-14.el7_3.noarch
>>>>>>>>>>>>>> Jan 08 04:47:22 Updated:
pki-ca-10.3.3-14.el7_3.noarch
>>>>>>>>>>>>>> Jul 28 10:13:16 Updated:
pki-base-10.3.3-19.el7_3.noarch
>>>>>>>>>>>>>> Jul 28 10:13:17 Updated:
>>>>>>>>>>>>>>
pki-base-java-10.3.3-19.el7_3.noarch
>>>>>>>>>>>>>> Jul 28 10:13:17 Updated:
pki-tools-10.3.3-19.el7_3.x86_64
>>>>>>>>>>>>>> Jul 28 10:13:21 Updated:
pki-server-10.3.3-19.el7_3.noarch
>>>>>>>>>>>>>> Jul 28 10:13:21 Updated:
pki-kra-10.3.3-19.el7_3.noarch
>>>>>>>>>>>>>> Jul 28 10:13:21 Updated:
pki-ca-10.3.3-19.el7_3.noarch
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Are you running in SELinux
enforcing mode?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> # getenforce
>>>>>>>>>>>>>> Enforcing
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If so can you run:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> # ausearch -m AVC
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> # ausearch -m AVC
>>>>>>>>>>>>>> The NOLOG option to log_format is
deprecated. Please use
>>>>>>>>>>>>>> the write_logs option.
>>>>>>>>>>>>>> The NOLOG option is overriding
the write_logs current
>>>>>>>>>>>>>> setting.
>>>>>>>>>>>>>> ----
>>>>>>>>>>>>>> time->Mon Mar 14 10:39:01
2016
>>>>>>>>>>>>>> type=SYSCALL
msg=audit(1457977141.767:17537):
>>>>>>>>>>>>>> arch=c000003e syscall=2
success=no exit=-13
>>>>>>>>>>>>>> a0=7fbcab038d10 a1=800 a2=1
a3=7fbca60f42e0 items=0
>>>>>>>>>>>>>> ppid=1406 pid=12965
auid=4294967295 uid=0 gid=0
>>>>>>>>>>>>>> euid=581400001 suid=0
fsuid=581400001 egid=581400001
>>>>>>>>>>>>>> sgid=0 fsgid=581400001 tty=(none)
ses=4294967295
>>>>>>>>>>>>>> comm="sshd"
exe="/usr/sbin/sshd"
>>>>>>>>>>>>>>
subj=system_u:system_r:sshd_t:s0-s0:c0.c1023 key=(null)
>>>>>>>>>>>>>> type=AVC
msg=audit(1457977141.767:17537): avc: denied {
>>>>>>>>>>>>>> read } for pid=12965
comm="sshd" name="authorized_keys"
>>>>>>>>>>>>>> dev="dm-2"
ino=116393517
>>>>>>>>>>>>>>
scontext=system_u:system_r:sshd_t:s0-s0:c0.c1023
>>>>>>>>>>>>>>
tcontext=system_u:object_r:unlabeled_t:s0 tclass=file
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I suspect that the subsystem
cert was renewed and
>>>>>>>>>>>>>>> everything wasn't
>>>>>>>>>>>>>>> updated properly. If I'm
right this is unrelated to the
>>>>>>>>>>>>>>> upgrade it just
>>>>>>>>>>>>>>> shone a spotlight on it.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Can you run these commands:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> $ ldapsearch -LLL -D
'cn=directory manager' -W -b
>>>>>>>>>>>>>>>
uid=pkidbuser,ou=people,o=ipaca userCertificate
>>>>>>>>>>>>>>> description
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> This returns
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> $ ldapsearch -LLL -D
'cn=directory manager' -W -b
>>>>>>>>>>>>>> uid=pkidbuser,ou=people,o=ipaca
userCertificate description
>>>>>>>>>>>>>> Enter LDAP Password:
>>>>>>>>>>>>>> dn:
uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>>>>>>> userCertificate::
>>>>>>>>>>>>>>
MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ... stuff ...
>>>>>>>>>>>>>>
ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>>>>>>>>>>> description: 2;4;CN=Certificate
>>>>>>>>>>>>>> Authority,O=BPT.ROCKS;CN=CA
Subsystem,O=BPT.ROCKS
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> and
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> # certutil -L -d
/etc/pki/pki-tomcat/alias -n
>>>>>>>>>>>>>>> 'subsystemCert
>>>>>>>>>>>>>>> cert-pki-ca' | grep
Serial
>>>>>>>>>>>>>>> # certutil -L -d
/etc/pki/pki-tomcat/alias -n
>>>>>>>>>>>>>>> 'subsystemCert
cert-pki-ca' -a
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> These both return
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> # certutil -K -d
/etc/pki/pki-tomcat/alias -n
>>>>>>>>>>>>>> 'subsystemCert
cert-pki-ca' | grep Serial
>>>>>>>>>>>>>> Enter Password or Pin for
"NSS Certificate DB":
>>>>>>>>>>>>>> certutil: problem listing keys:
>>>>>>>>>>>>>> SEC_ERROR_UNRECOGNIZED_OID:
Unrecognized Object Identifier.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> # certutil -K -d
/etc/pki/pki-tomcat/alias -n
>>>>>>>>>>>>>> 'subsystemCert
cert-pki-ca' -a
>>>>>>>>>>>>>> certutil: Checking token
"NSS Certificate DB" in slot
>>>>>>>>>>>>>> "NSS User Private Key and
Certificate Services"
>>>>>>>>>>>>>> Enter Password or Pin for
"NSS Certificate DB":
>>>>>>>>>>>>>> certutil: problem listing keys:
>>>>>>>>>>>>>> SEC_ERROR_UNRECOGNIZED_OID:
Unrecognized Object Identifier.
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>
>>>>>>>>>>>>> you made a typo and requested the
keys (-K) instead of
>>>>>>>>>>>>> the certs (-L). Please retry with -L
and you should be
>>>>>>>>>>>>> able to see the certificate serial.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> [root@seattlenfs ianh]# ldapsearch -LLL
-D 'cn=directory
>>>>>>>>>>>> manager' -W -b
uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>>>>> userCertificate description
>>>>>>>>>>>> Enter LDAP Password:
>>>>>>>>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>>>>> userCertificate::
>>>>>>>>>>>>
MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>>>>>> ...
>>>>>>>>>>>>
ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>>>>>>>>> description: 2;4;CN=Certificate
>>>>>>>>>>>> Authority,O=BPT.ROCKS;CN=CA
Subsystem,O=BPT.ROCKS
>>>>>>>>>>>>
>>>>>>>>>>>> So the serial is 4?
>>>>>>>>>>>>
>>>>>>>>>>>> [root@seattlenfs ianh]# certutil -L -d
>>>>>>>>>>>> /etc/pki/pki-tomcat/alias -n
'subsystemCert cert-pki-ca' |
>>>>>>>>>>>> grep Serial
>>>>>>>>>>>> Serial Number: 1341718541
(0x4ff9000d)
>>>>>>>>>>>>
>>>>>>>>>>>> Which is NOT 4...
>>>>>>>>>>>>
>>>>>>>>>>>> [root@seattlenfs ianh]# certutil -L -d
>>>>>>>>>>>> /etc/pki/pki-tomcat/alias -n
'subsystemCert cert-pki-ca' -a
>>>>>>>>>>>> -----BEGIN CERTIFICATE-----
>>>>>>>>>>>>
MIIDbjCCAlagAwIBAgIET/kADTANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>>>>>>
>>>>>>>>>>>> ...
>>>>>>>>>>>> sjeiFbUFtoMnfmHLIYpe5QAR
>>>>>>>>>>>> -----END CERTIFICATE-----
>>>>>>>>>>>>
>>>>>>>>>>>> And this cert is NOT the same.
>>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> when certmonger renews a certificate, it runs
pre-save and
>>>>>>>>>>> post-save commands (which you can see in the
output of
>>>>>>>>>>> getcert list). In your case, the post-save
command for
>>>>>>>>>>> subsystemCert cert-pki-ca probably failed.
>>>>>>>>>>>
>>>>>>>>>>> This command (on the renewal master) is
responsible for
>>>>>>>>>>> copying the new cert from the NSS db to the
LDAP server.
>>>>>>>>>>> You need first to check which IPA server is
the renewal
>>>>>>>>>>> master (as the serial numbers are in a
completely different
>>>>>>>>>>> range, I assume that you have multiple IPA
servers
>>>>>>>>>>> configured with a CA instance):
>>>>>>>>>>>
>>>>>>>>>>> $ export BASEDN=`grep basedn
/etc/ipa/default.conf | cut
>>>>>>>>>>> -d= -f2-`
>>>>>>>>>>> $ kinit admin
>>>>>>>>>>> $ ldapsearch -LLL -Q -h `hostname` -p 389 -Y
GSSAPI -b
>>>>>>>>>>> "cn=masters,cn=ipa,cn=etc,$BASEDN"
>>>>>>>>>>>
'(&(cn=CA)(ipaConfigString=caRenewalMaster))' dn
>>>>>>>>>>> dn:
>>>>>>>>>>>
cn=CA,cn=vm-ipaserver.ipadomain.com,cn=masters,cn=ipa,cn=etc,dc=ipadomain,dc=com
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> In this example the renewal master is
vm-ipaserver.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I get this
>>>>>>>>>>
>>>>>>>>>> $ ldapsearch -LLL -Q -h `hostname` -p 389 -Y
GSSAPI -b
>>>>>>>>>> "cn=masters,cn=ipa,cn=etc,$BASEDN"
>>>>>>>>>>
'(&(cn=CA)(ipaConfigString=caRenewalMaster))' dn
>>>>>>>>>> dn:
>>>>>>>>>>
cn=CA,cn=freeipa-sea.bpt.rocks,cn=masters,cn=ipa,cn=etc,dc=bpt,dc=rocks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Which is the other FreeIPA server.
>>>>>>>>>>
>>>>>>>>> Hi Ian,
>>>>>>>>>
>>>>>>>>> it's getting difficult to follow this thread, so
could we
>>>>>>>>> step back and summarize?
>>>>>>>> Yes, it is. Sorry.
>>>>>>>>>
>>>>>>>>> - On server freeipa-sea (=renewal master), the CA
component
>>>>>>>>> is installed and 'subsystemCert cert-pki-ca'
has Serial
>>>>>>>>> number 4 in the NSSDB /etc/pki/pki-tomcat/alias
>>>>>>>> No?
>>>>>>>>
>>>>>>>> [root@freeipa-sea ianh]# certutil -L -d
>>>>>>>> /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca' |
>>>>>>>> grep Serial
>>>>>>>> Serial Number: 1341718541 (0x4ff9000d)
>>>>>>>>> - On server seattlenfs (where PKI fails to restart),
the CA
>>>>>>>>> component is installed and 'subsystemCert
cert-pki-ca' has
>>>>>>>>> Serial number 1341718541 in the NSSDB
/etc/pki/pki-tomcat/alias.
>>>>>>>> Yes
>>>>>>>>
>>>>>>>> [root@seattlenfs ianh]# certutil -L -d
>>>>>>>> /etc/pki/pki-tomcat/alias -n 'subsystemCert
cert-pki-ca' |
>>>>>>>> grep Serial
>>>>>>>> Serial Number: 1341718541 (0x4ff9000d)
>>>>>>>>
>>>>>>>>>
>>>>>>>>> - In the LDAP servers, the cert is stored in the
entry
>>>>>>>>> uid=pkidbuser,ou=people,o=ipaca with Serial number 4
(i.e. it
>>>>>>>>> is the cert from freeipa-sea).
>>>>>>>> This is where the difference is. There are two
certificates
>>>>>>>> on the master and only one on the other. I wonder if
this is
>>>>>>>> a result of the slow motion replication trainwreck I have
>>>>>>>> going on due to a zombie replica in the ldap database
that no
>>>>>>>> longer exists in the real world and I can't seem to
get rid of.
>>>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> the remaining issue is a replication problem. There is a
>>>>>>> "Troubleshooting replication" section [1] in IdM
Admin guide
>>>>>>> that may help you.
>>>>>>>
>>>>>>> Regarding your zombie replica, which commands did you use to
>>>>>>> try to remove it, and what was the output?
>>>>>>>
>>>>>>> Flo
>>>>>>>
>>>>>>> [1]
>>>>>>>
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/...
>>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> To debug replication latency or problem we would need several
>>>>>> info but starting with the results of the following commands
>>>>>>
>>>>>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory
>>>>>> manager' -W -b "uid=pkidbuser,ou=people,o=ipaca"
nscpentrywsi
>>>>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory
>>>>> manager' -W -b "uid=pkidbuser,ou=people,o=ipaca"
nscpentrywsi
>>>>> Enter LDAP Password:
>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>> nscpentrywsi: dn: uid=pkidbuser,ou=people,o=ipaca
>>>>> nscpentrywsi:
>>>>> entryusn;adcsn-5938e688000104290005;vucsn-5938e688000104290005:
>>>>> 75274897
>>>>> nscpentrywsi:
>>>>> modifyTimestamp;adcsn-5938e688000104290004;vucsn-5938e6880001042
>>>>> 90004: 20170608055334Z
>>>>> nscpentrywsi:
>>>>> modifiersName;adcsn-5938e688000104290003;vucsn-5938e688000104290
>>>>> 003: cn=Directory Manager
>>>>> nscpentrywsi: nsUniqueId: 48809b40-2bb911e5-96a0b8b8-1fdd8074
>>>>> nscpentrywsi: cn: pkidbuser
>>>>> nscpentrywsi: createTimestamp: 20150716125146Z
>>>>> nscpentrywsi: creatorsName: cn=directory manager
>>>>> nscpentrywsi: description;vucsn-5938e688000104290001:
>>>>> 2;1341718541;CN=Certific
>>>>> ate Authority,O=BPT.ROCKS;CN=CA Subsystem,O=BPT.ROCKS
>>>>> nscpentrywsi: description;vdcsn-5938e688000104290002;deleted:
>>>>> 2;4;CN=Certifica
>>>>> te Authority,O=BPT.ROCKS;CN=CA Subsystem,O=BPT.ROCKS
>>>>> nscpentrywsi: mail:
>>>>> nscpentrywsi: objectClass: top
>>>>> nscpentrywsi: objectClass: person
>>>>> nscpentrywsi: objectClass: organizationalPerson
>>>>> nscpentrywsi: objectClass: inetOrgPerson
>>>>> nscpentrywsi: objectClass: cmsuser
>>>>> nscpentrywsi: seeAlso: CN=CA Subsystem,O=BPT.ROCKS
>>>>> nscpentrywsi: sn: pkidbuser
>>>>> nscpentrywsi: uid: pkidbuser
>>>>> nscpentrywsi: userCertificate::
>>>>> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MR
>>>>> <snip>
>>>>> cBd4nOV4m9A+qRZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>> nscpentrywsi: userCertificate;vucsn-5938e688000104290000::
>>>>> MIIDbjCCAlagAwIBAgI
>>>>> <snip>
>>>>> AR
>>>>> nscpentrywsi: userPassword:: <snip>
>>>>> nscpentrywsi: userstate: 1
>>>>> nscpentrywsi: usertype: agentType
>>>>> nscpentrywsi: parentid: 2
>>>>> nscpentrywsi: entryid: 51
>>>>>> and
>>>>>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory
>>>>>> manager' -W -b uid=pkidbuser,ou=people,o=ipaca nscpentrywsi
>>>>>
>>>>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory
manager'
>>>>> -W -b uid=pkidbuser,ou=people,o=ipaca nscpentrywsi
>>>>> Enter LDAP Password:
>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>> nscpentrywsi: dn: uid=pkidbuser,ou=people,o=ipaca
>>>>> nscpentrywsi: seeAlso: CN=CA Subsystem,O=BPT.ROCKS
>>>>> nscpentrywsi: userCertificate::
>>>>> MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MR
>>>>> <snip>
>>>>> cBd4nOV4m9A+qRZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>> nscpentrywsi: description: 2;4;CN=Certificate
>>>>> Authority,O=BPT.ROCKS;CN=CA Subs
>>>>> ystem,O=BPT.ROCKS
>>>>> nscpentrywsi: entryid: 51
>>>>> nscpentrywsi: parentid: 2
>>>>> nscpentrywsi: modifyTimestamp: 20150716125146Z
>>>>> nscpentrywsi: createTimestamp: 20150716125146Z
>>>>> nscpentrywsi: modifiersName: cn=directory manager
>>>>> nscpentrywsi: creatorsName: cn=directory manager
>>>>> nscpentrywsi: userPassword:: <snip>
>>>>> nscpentrywsi: userstate: 1
>>>>> nscpentrywsi: usertype: agentType
>>>>> nscpentrywsi: mail:
>>>>> nscpentrywsi: cn: pkidbuser
>>>>> nscpentrywsi: sn: pkidbuser
>>>>> nscpentrywsi: uid: pkidbuser
>>>>> nscpentrywsi: objectClass: top
>>>>> nscpentrywsi: objectClass: person
>>>>> nscpentrywsi: objectClass: organizationalPerson
>>>>> nscpentrywsi: objectClass: inetOrgPerson
>>>>> nscpentrywsi: objectClass: cmsuser
>>>>> nscpentrywsi: nsUniqueId: 48809b40-2bb911e5-96a0b8b8-1fdd8074
>>>>> nscpentrywsi: entryusn: 86
>>>>>>
>>>>>> From the output you may only copy/past the ones related to
>>>>>> userCertificate
>>>>>>
>>>>>>
>>>>>> Also to dump the current replication status, send the result of
>>>>>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory
>>>>>> manager' -W -b
>>>>>>
"o=ipaca"//"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
>>>>> [root@freeipa-sea ianh]# ldapsearch -LLL -D 'cn=directory
>>>>> manager' -W -b "o=ipaca"
>>>>>
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
>>>>> Enter LDAP Password:
>>>>> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
>>>>> cn: replica
>>>>> nsDS5Flags: 1
>>>>> nsDS5ReplicaBindDN: cn=Replication Manager
>>>>> cloneAgreement1-freeipa-sea.bpt.roc
>>>>> ks-pki-tomcat,ou=csusers,cn=config
>>>>> nsDS5ReplicaBindDN: cn=Replication Manager
>>>>> masterAgreement1-seattlenfs.bpt.roc
>>>>> ks-pki-tomcat,ou=csusers,cn=config
>>>>> nsDS5ReplicaId: 1065
>>>>> nsDS5ReplicaName: b21a1f1e-627911e6-93e6ef4b-69dcc2d1
>>>>> nsDS5ReplicaRoot: o=ipaca
>>>>> nsDS5ReplicaType: 3
>>>>> nsState:: KQQAAAAAAAAhHIpZAAAAAAEAAAAAAAAAKgAAAAAAAAABAAAAAAAAAA==
>>>>> nsds5replicabinddngroup: cn=replication
>>>>> managers,cn=sysaccounts,cn=etc,dc=bpt,
>>>>> dc=rocks
>>>>> nsds5replicabinddngroupcheckinterval: 60
>>>>> objectClass: top
>>>>> objectClass: nsDS5Replica
>>>>> objectClass: extensibleobject
>>>>> nsds50ruv: {replicageneration} 57c291d9000004290000
>>>>> nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
>>>>> 57f840bf00000429000
>>>>> 0 598a1c40000104290000
>>>>> nsds50ruv: {replica 1290 ldap://seattlenfs.bpt.rocks:389}
>>>>> nsds5agmtmaxcsn:
>>>>> o=ipaca;cloneAgreement1-freeipa-sea.bpt.rocks-pki-tomcat;seat
>>>>> tlenfs.bpt.rocks;389;unavailable
>>>>> nsds5agmtmaxcsn:
>>>>> o=ipaca;masterAgreement1-seattlenfs.bpt.rocks-pki-tomcat;seat
>>>>> tlenfs.bpt.rocks;389;unavailable
>>>>> nsruvReplicaLastModified: {replica 1065
>>>>> ldap://freeipa-sea.bpt.rocks:389} 598a
>>>>> 1c16
>>>>> nsruvReplicaLastModified: {replica 1290
>>>>> ldap://seattlenfs.bpt.rocks:389} 00000
>>>>> 000
>>>>> nsds5ReplicaChangeCount: 885
>>>>> nsds5replicareapactive: 0
>>>>>> and
>>>>>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory
>>>>>> manager' -W -b
>>>>>>
"o=ipaca"//"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
>>>>>
>>>>>
>>>>> [root@seattlenfs ianh]# ldapsearch -LLL -D 'cn=directory
manager'
>>>>> -W -b "o=ipaca"
>>>>>
"(&(objectclass=nstombstone)(nsUniqueId=ffffffff-ffffffff-ffffffff-ffffffff))"
>>>>> Enter LDAP Password:
>>>>> dn: cn=replica,cn=o\3Dipaca,cn=mapping tree,cn=config
>>>>> cn: replica
>>>>> nsDS5Flags: 1
>>>>> nsDS5ReplicaBindDN: cn=Replication Manager
>>>>> cloneAgreement1-seattlenfs.bpt.rock
>>>>> s-pki-tomcat,ou=csusers,cn=config
>>>>> nsDS5ReplicaId: 1290
>>>>> nsDS5ReplicaName: 8706ab1e-6a8311e6-a9bfdbbf-74dc7323
>>>>> nsDS5ReplicaRoot: o=ipaca
>>>>> nsDS5ReplicaType: 3
>>>>> nsState:: CgUAAAAAAAAKFYpZAAAAAAAAAAAAAAAAKgAAAAAAAAABAAAAAAAAAA==
>>>>> nsds5replicabinddngroup: cn=replication
>>>>> managers,cn=sysaccounts,cn=etc,dc=bpt,
>>>>> dc=rocks
>>>>> nsds5replicabinddngroupcheckinterval: 60
>>>>> objectClass: top
>>>>> objectClass: nsDS5Replica
>>>>> objectClass: extensibleobject
>>>>> nsds50ruv: {replicageneration} 55c8f3ae000000600000
>>>>> nsds50ruv: {replica 1290 ldap://seattlenfs.bpt.rocks:389}
>>>>> 57be804c0000050a0000
>>>>> 587236150000050a0000
>>>>> nsds50ruv: {replica 1065 ldap://freeipa-sea.bpt.rocks:389}
>>>>> 57b103d400000429000
>>>>> 0 57be8043000704290000
>>>>> nsds5agmtmaxcsn:
>>>>> o=ipaca;cloneAgreement1-seattlenfs.bpt.rocks-pki-tomcat;freei
>>>>> pa-sea.bpt.rocks;389;1065;587236150000050a0000
>>>>> nsruvReplicaLastModified: {replica 1290
>>>>> ldap://seattlenfs.bpt.rocks:389} 00000
>>>>> 000
>>>>> nsruvReplicaLastModified: {replica 1065
>>>>> ldap://freeipa-sea.bpt.rocks:389} 0000
>>>>> 0000
>>>>> nsds5ReplicaChangeCount: 3
>>>>> nsds5replicareapactive: 0
>>>>>>
>>>>>> Regards
>>>>>> thierry
>>>>>>>> [root@freeipa-sea ianh]# ldapsearch -LLL -D
'cn=directory
>>>>>>>> manager' -W -b uid=pkidbuser,ou=people,o=ipaca
userCertificate
>>>>>>>> description
>>>>>>>> Enter LDAP Password:
>>>>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>>>>> userCertificate::
>>>>>>>>
MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>> <truncated>
>>>>>>>> ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>>>>> userCertificate::
>>>>>>>>
MIIDbjCCAlagAwIBAgIET/kADTANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQK
>>>>>>>> <truncated> fuQQZmP1XrKkrfsjeiFbUFtoMnfmHLIYpe5QAR
>>>>>>>> description: 2;1341718541;CN=Certificate
>>>>>>>> Authority,O=BPT.ROCKS;CN=CA Subsystem
>>>>>>>> ,O=BPT.ROCKS
>>>>>>>>
>>>>>>>>
>>>>>>>> [root@seattlenfs ianh]# ldapsearch -LLL -D
'cn=directory
>>>>>>>> manager' -W -b uid=pkidbuser,ou=people,o=ipaca
userCertificate
>>>>>>>> description
>>>>>>>> Enter LDAP Password:
>>>>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>>>>> userCertificate::
>>>>>>>>
MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>> <truncated>
>>>>>>>> ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>>>>> description: 2;4;CN=Certificate
Authority,O=BPT.ROCKS;CN=CA
>>>>>>>> Subsystem,O=BPT.RO
>>>>>>>> CKS
>>>>>>>>
>>>>>>>>> If you check the certificates dates, I guess that the
most
>>>>>>>>> recent one is on server seattlenfs (because of serial
>>>>>>>>> numbers). Can you confirm this (because our goal is
to have
>>>>>>>>> the most recent cert stored in all the NSSDBs and in
LDAP)?
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>> Flo
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>> If your renewal master is the machine on
which the NSS DB
>>>>>>>>>>> /etc/pki/pki-tomcat/alias contains the newer
cert, then you
>>>>>>>>>>> can manually run the scripts:
>>>>>>>>>>
>>>>>>>>>> If I'm interpreting this correctly, then it
is... so I ran
>>>>>>>>>> this on the broken machine (seattlenfs)
>>>>>>>>>>
>>>>>>>>>>> $ sudo
/usr/libexec/ipa/certmonger/stop_pkicad
>>>>>>>>>>> $ sudo
/usr/libexec/ipa/certmonger/renew_ca_cert
>>>>>>>>>>> "subsystemCert cert-pki-ca"
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I ran this, it took a while, then I re-checked
and the
>>>>>>>>>> serial still seems to be 4
>>>>>>>>>>
>>>>>>>>>> $ ldapsearch -LLL -D 'cn=directory
manager' -W -b
>>>>>>>>>> uid=pkidbuser,ou=people,o=ipaca userCertificate
description
>>>>>>>>>> Enter LDAP Password:
>>>>>>>>>> dn: uid=pkidbuser,ou=people,o=ipaca
>>>>>>>>>> userCertificate::
>>>>>>>>>>
MIIDczCCAlugAwIBAgIBBDANBgkqhkiG9w0BAQsFADA0MRIwEAYDVQQKDAlC
>>>>>>>>>> ...
>>>>>>>>>> ZNo4eX7jNT8Aa23gDl6xuLVa1SxlyE8XEo8dfAPXMJoFc=
>>>>>>>>>> description: 2;4;CN=Certificate
Authority,O=BPT.ROCKS;CN=CA
>>>>>>>>>> Subsystem,O=BPT.ROCKS
>>>>>>>>>>
>>>>>>>>>> Here's what was in
/var/log/pki/pki-tomcat/ca/debug
>>>>>>>>>>
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> ============================================
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
===== DEBUG
>>>>>>>>>> SUBSYSTEM INITIALIZED =======
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> ============================================
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> restart at autoShutdown? false
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> autoShutdown crumb file path?
>>>>>>>>>> /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> about to look for cert for auto-shutdown
>>>>>>>>>> support:auditSigningCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> found cert:auditSigningCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> done init id=debug
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> initialized debug
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> initSubsystem id=log
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> ready to init id=log
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
Creating
>>>>>>>>>>
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/signedAudit/ca_audit)
>>>>>>>>>>
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
Creating
>>>>>>>>>>
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/system)
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
Creating
>>>>>>>>>>
RollingLogFile(/var/lib/pki/pki-tomcat/logs/ca/transactions)
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> restart at autoShutdown? false
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> autoShutdown crumb file path?
>>>>>>>>>> /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> about to look for cert for auto-shutdown
>>>>>>>>>> support:auditSigningCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> found cert:auditSigningCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> done init id=log
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> initialized log
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> initSubsystem id=jss
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> ready to init id=jss
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> restart at autoShutdown? false
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> autoShutdown crumb file path?
>>>>>>>>>> /var/lib/pki/pki-tomcat/logs/autoShutdown.crumb
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> about to look for cert for auto-shutdown
>>>>>>>>>> support:auditSigningCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> found cert:auditSigningCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> done init id=jss
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> initialized jss
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> initSubsystem id=dbs
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
CMSEngine:
>>>>>>>>>> ready to init id=dbs
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
DBSubsystem:
>>>>>>>>>> init() mEnableSerialMgmt=true
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
Creating
>>>>>>>>>> LdapBoundConnFactor(DBSubsystem)
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> LdapBoundConnFactory: init
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> LdapBoundConnFactory:doCloning true
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
LdapAuthInfo:
>>>>>>>>>> init()
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
LdapAuthInfo:
>>>>>>>>>> init begins
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
LdapAuthInfo:
>>>>>>>>>> init ends
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
init: before
>>>>>>>>>> makeConnection errorIfDown is true
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> makeConnection: errorIfDown true
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
TCP
>>>>>>>>>> Keep-Alive: true
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> SSLClientCertificateSelectionCB: Setting desired
cert
>>>>>>>>>> nickname to: subsystemCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> LdapJssSSLSocket: set client auth cert nickname
>>>>>>>>>> subsystemCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> SSLClientCertificatSelectionCB: Entering!
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
Candidate
>>>>>>>>>> cert: ocspSigningCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
Candidate
>>>>>>>>>> cert: subsystemCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> SSLClientCertificateSelectionCB: desired cert
found in list:
>>>>>>>>>> subsystemCert cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> SSLClientCertificateSelectionCB: returning:
subsystemCert
>>>>>>>>>> cert-pki-ca
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
SSL handshake
>>>>>>>>>> happened
>>>>>>>>>> [03/Aug/2017:13:52:51][localhost-startStop-1]:
>>>>>>>>>> CMSEngine.shutdown()
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> I really appreciate you sticking with me through
this. I
>>>>>>>>>> owe you.
>>>>>>>>>>
>>>>>>>>>> - Ian
>>>>>>>>>>
>>>>>>>>>>> After that, check that the LDAP entry has
been updated. If
>>>>>>>>>>> it is the case, you should be able to restart
pki.
>>>>>>>>>>>
>>>>>>>>>>> Flo
>>>>>>>>>>>
>>>>>>>>>>>> Thank you again for your time!
>>>>>>>>>>>>
>>>>>>>>>>>> Ian
>>>>>>>>>>>>
>>>>>>>>>>>>> Flo.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> which seems weird.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The description attribute in
the ldapsearch output is
>>>>>>>>>>>>>>> of the form 2;#;DN;DN
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The # should match the serial
number in the first
>>>>>>>>>>>>>>> certutil output
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The ASCII blob (minus the
BEGIN/END headers) should
>>>>>>>>>>>>>>> match one of the
>>>>>>>>>>>>>>> userCertificate attributes.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> rob
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> FreeIPA-users mailing list --
>>>>>>>> freeipa-users(a)lists.fedorahosted.org
>>>>>>>> To unsubscribe send an email to
>>>>>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>>>>> _______________________________________________
>>>>>>> FreeIPA-users mailing list --
freeipa-users(a)lists.fedorahosted.org
>>>>>>> To unsubscribe send an email to
>>>>>>> freeipa-users-leave(a)lists.fedorahosted.org
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> FreeIPA-users mailing list --freeipa-users(a)lists.fedorahosted.org
>>>>> To unsubscribe send an email
tofreeipa-users-leave(a)lists.fedorahosted.org
>>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> FreeIPA-users mailing list --freeipa-users(a)lists.fedorahosted.org
>>> To unsubscribe send an email tofreeipa-users-leave(a)lists.fedorahosted.org
>>
>
_______________________________________________
FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org