On Wed, Mar 13, 2024 at 1:58 PM Ian Kumlien <ian.kumlien(a)gmail.com> wrote:
On Wed, Mar 13, 2024 at 11:39 AM Florence Blanc-Renaud <flo(a)redhat.com> wrote:
>
> Hi,
>
> On Wed, Mar 13, 2024 at 10:06 AM Ian Kumlien <ian.kumlien(a)gmail.com> wrote:
>>
>> On Tue, Mar 12, 2024 at 10:36 PM Florence Blanc-Renaud <flo(a)redhat.com>
wrote:
>> >
>> > Hi,
>> >
>> > On Tue, Mar 12, 2024 at 12:54 PM Ian Kumlien via FreeIPA-users
<freeipa-users(a)lists.fedorahosted.org> wrote:
>> >>
>> >> Hi,
>> >>
>> >> So i have spent quite some time trying to get out of the swamp that is
>> >> centos stream 8 and back to something with a actual upgrade path,
>> >> fedora =)
>> >>
>> >> Everything works except the ipa-ca-install on the replica - mostly
>> >> fails at the same step
>> >>
>> >> At some point the conncheck failed, dropping me in to a prompt asking
>> >> for the password of a admin-<machine> account
>> >>
>> >> Anyway, I do know about the issue with - vs _ and validated on master,
>> >> changed on replica as detailed here:
>> >>
https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedorah...
>> >>
>> >> But it still fails..
>> >>
>> >> Oh and btw, none of the machines are running any firewalls =)
>> >>
>> >> Anyone that has a clue of what to test next?
>> >>
>> >> Btw, it's 4.9 to 4.11 if there is other issues with
interoperability
>> >>
>> >> ipa-ca-install --skip-conncheck
>> >> Directory Manager (existing master) password:
>> >>
>> >> Configuring certificate server (pki-tomcatd). Estimated time: 3
minutes
>> >> [1/28]: creating certificate server db
>> >> [2/28]: setting up initial replication
>> >> Starting replication, please wait until this has completed.
>> >> Update in progress, 7 seconds elapsed
>> >> Update succeeded
>> >>
>> >> [3/28]: creating ACIs for admin
>> >> [4/28]: creating installation admin user
>> >> ipaserver.install.dogtaginstance: ERROR Unable to log in as
>> >> uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca on
>> >> ldap://freeipa-1.xerces.lan:389
>> >> [error] NotFound: uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca
>> >> did not replicate to ldap://freeipa-1.xerces.lan:389
>> >>
>> >> Your system may be partly configured.
>> >> Run /usr/sbin/ipa-server-install --uninstall to clean up.
>> >>
>> >> Unexpected error - see /var/log/ipareplica-ca-install.log for details:
>> >> NotFound: uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca did not
>> >> replicate to ldap://freeipa-1.xerces.lan:389
>> >>
>> > The installation of a CA clone creates this user on the replica, lets the
replication copy the entry on the master and then checks by doing a ldap bind from the
replica to the master that the entry has been properly replicated.
>> > When this error happens, it can either mean that the entry was not
replicated or that the bind failed.
>> >
>> > In order to know exactly what is happening for you, you can check
>> > - on the master freeipa-1.xerces.lan, do a ldapsearch for this entry and
check if it exists. If the entry is present, the replication properly propagated the entry
from replica to master and you are probably hitting the 2nd issue.
>> > # ldapsearch -D "cn=directory manager" -W -b
uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca
>> >
>> > - on the replica, do a ldapsearch for this entry and check the userpassword
attribute. It is base64-encoded, and you can decode it in order to find the password
storage scheme that was used to encrypt the password.
>> > For instance on my machine:
>> >
>> > dn: uid=admin-replica.ipa.test,ou=people,o=ipaca
>> > userPassword::
e1BCS0RGMl9TSEEyNTZ9QUFBSUFCWVMrWHUxVEJzb0VTcjJLQVl4RlZHWGRHWWZ
>> >
NTmxFN3dCZHRRV1IxUTNxaTdKTXord2duLzIrc1NKMDZJbXhBeng5ZkR2VEIrMCsvQkZyMmRiL1pT
>> >
dy96YzdhNWlVNGVCYnZHem9FODM0VHpIbHBweS9UeFRhc0Facm81OG1iT05OaUdBbml1c3pVcE5nb
>> >
055R3dLYkpqQzZQeEpNeStnUklOa2xaOHJjTHBQSkZLam9jR0UvQ1NoeWFQYWN0b1ZZQlZVWHAzM3
>> >
pyeWtZVlBIL0pIUjNQb2pnZnNUb2pRL2w5UWg1UGEwVjVVZ0VyUGpFK0dsNWtLS3FMaWE0d296Rk4
>> >
wM3ozZjVwRGZDRnZOSi9CVEdENHhpcmNhcFZSVG5jTTRBZ0xPQlBCa2hoVm1vbEZBZHZ0OVUxY1ZL
>> >
ZHVDZWRhWVUzZXZrS1hHcWx3alpTbEpPdkQ5SllJb0FHRlBwOXJERlJscU1MWEhUckx2aVoxTWgyM
>> > 2Roa0hrR0VXM3pna3VuK2FIcnNvYUZMWWQwZi95NjlweDBRMzJvci9vOXBZV1F6S1ppNUFp
>> >
>> >
>> > If I base64 decode the value:
>> >
>> > # echo
e1BCS0RGMl9TSEEyNTZ9QUFBSUFCWVMrWHUxVEJzb0VTcjJLQVl4RlZHWGRHWWZNTmxFN3dCZHRRV1IxUTNxaTdKTXord2duLzIrc1NKMDZJbXhBeng5ZkR2VEIrMCsvQkZyMmRiL1pTdy96YzdhNWlVNGVCYnZHem9FODM0VHpIbHBweS9UeFRhc0Facm81OG1iT05OaUdBbml1c3pVcE5nb055R3dLYkpqQzZQeEpNeStnUklOa2xaOHJjTHBQSkZLam9jR0UvQ1NoeWFQYWN0b1ZZQlZVWHAzM3pyeWtZVlBIL0pIUjNQb2pnZnNUb2pRL2w5UWg1UGEwVjVVZ0VyUGpFK0dsNWtLS3FMaWE0d296Rk4wM3ozZjVwRGZDRnZOSi9CVEdENHhpcmNhcFZSVG5jTTRBZ0xPQlBCa2hoVm1vbEZBZHZ0OVUxY1ZLZHVDZWRhWVUzZXZrS1hHcWx3alpTbEpPdkQ5SllJb0FHRlBwOXJERlJscU1MWEhUckx2aVoxTWgyM2Roa0hrR0VXM3pna3VuK2FIcnNvYUZMWWQwZi95NjlweDBRMzJvci9vOXBZV1F6S1ppNUFp
| base64 -d
>> >
{PBKDF2_SHA256}AAAIABYS+Xu1TBsoESr2KAYxFVGXdGYfMNlE7wBdtQWR1Q3qi7JMz+wgn/2+sSJ06ImxAzx9fDvTB+0+/BFr2db/ZSw/zc7a5iU4eBbvGzoE834TzHlppy/TxTasAZro58mbONNiGAniuszUpNgoNyGwKbJjC6PxJMy+gRINklZ8rcLpPJFKjocGE/CShyaPactoVYBVUXp33zrykYVPH/JHR3PojgfsTojQ/l9Qh5Pa0V5UgErPjE+Gl5kKKqLia4wozFN03z3f5pDfCFvNJ/BTGD4xircapVRTncM4AgLOBPBkhhVmolFAdvt9U1cVKduCedaYU3evkKXGqlwjZSlJOvD9JYIoAGFPp9rDFRlqMLXHTrLviZ1Mh23dhkHkGEW3zgkun+aHrsoaFLYd0f/y69px0Q32or/o9pYWQzKZi5Ai
>>
>> Yes, and the value is the same on both replicas, both the encoded
>> base64 and the password scheme: {PBKDF2_SHA256}AAAIAGIHopZZSHY8.....
>>
>> Since I changed it as described in the link i included...
>>
>> > which means that the replica used PBKDF2_SHA256 as password storage
scheme.
>> > You need to check if this password storage scheme is supported on the
master (we had issues in the past with a password storage scheme used by the replica that
was not supported on the master and caused the bind to fail,
https://bugzilla.redhat.com/show_bug.cgi?id=2151071). The list of supported password
storage schemes is available with the following command:
>> > # ldapsearch -D "cn=directory manager" -W -LLL -o ldif-wrap=no -b
"cn=Password Storage Schemes,cn=plugins,cn=config" -s one dn
>>
>> Yes, and they both support PBKDF2_SHA256 both as plugin and password
>> storage scheme
>>
>> > If the replica is using a password scheme not supported on the master, you
are probably hitting the above BZ. There were fixes for multiple versions of 389-ds, we
would need to know your exact versions on the replica and the master to point you to the
right advisory.
>>
>> 4.9.10 and 4.11.1
>>
>> (fedora is just now updating it to 4.11.1-2 will look at the changes)
>>
>> Anyway, thanks for the help so far, i can now see the account
>> replicated but i don't quite understand why it doesn't work...
>
>
> Your ipa-ca-install command failed at 2024-03-11T15:05:24Z. Can you check on the
master around this date if there is a connection from replica to master with a BIND
attempt that would be failing? The logs are in /var/log/dirsrv/slapd-YOURDOMAIN/access.
Look for something similar to BIND
dn="uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca". Then note the conn number
and op number and look for the RESULT with the same conn and op, for instance:
>
> [13/Mar/2024:09:10:01.331583308 +0000] conn=106 op=2 BIND
dn="uid=admin-replica0.ipa.test,ou=people,o=ipaca" method=128 version=3
> [13/Mar/2024:09:10:01.355201520 +0000] conn=106 op=2 RESULT err=0 tag=97 nentries=0
wtime=0.000106547 optime=0.023642452 etime=0.023744831
dn="uid=admin-replica0.ipa.test,ou=people,o=ipaca"
>
>
> The lines may be separated by other logs, and the err=xxx will show if the bind is
successful or failed. err=0 means success, err=49 means invalid credentials.
Yes, it does fail with invalid credentials...
[11/Mar/2024:15:25:34.887970467 +0100] conn=118429 op=2 BIND
dn="uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca" method=128
version=3
[11/Mar/2024:15:25:34.888893033 +0100] conn=118429 op=2 RESULT err=49
tag=97 nentries=0 wtime=0.000265028 optime=0.000950489
etime=0.001212411 - Invalid credentials
Latest test, states
[13/Mar/2024:10:41:45.063122671 +0100] conn=192008 op=2 RESULT err=49
tag=97 nentries=0 wtime=0.000244892 optime=0.001078282
etime=0.001320065 - No such entry
I did remove it. as in uninstalled freeipa-4 removed the server and
removed that account, just to make sure it was replicated properly...:
[--8<--]
As a side node, the conncheck for ipa-ca-install fails all the time
now, when executing check on remote master it ends with this:
2024-03-14T07:42:26Z DEBUG Destroyed connection
context.rpcclient_139905569284576
2024-03-14T07:42:26Z ERROR ERROR: Remote master check failed with
following error message(s):
invalid 'cn': must be "freeipa-4.xerces.lan"
2024-03-14T07:42:26Z DEBUG Stopping listening thread.
Which seems really strange...