Hi,
On Thu, Mar 14, 2024 at 8:55 AM Ian Kumlien <ian.kumlien(a)gmail.com> wrote:
On Wed, Mar 13, 2024 at 1:58 PM Ian Kumlien
<ian.kumlien(a)gmail.com> wrote:
>
> On Wed, Mar 13, 2024 at 11:39 AM Florence Blanc-Renaud <flo(a)redhat.com>
wrote:
> >
> > Hi,
> >
> > On Wed, Mar 13, 2024 at 10:06 AM Ian Kumlien <ian.kumlien(a)gmail.com>
wrote:
> >>
> >> On Tue, Mar 12, 2024 at 10:36 PM Florence Blanc-Renaud <
flo(a)redhat.com> wrote:
> >> >
> >> > Hi,
> >> >
> >> > On Tue, Mar 12, 2024 at 12:54 PM Ian Kumlien via FreeIPA-users <
freeipa-users(a)lists.fedorahosted.org> wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> So i have spent quite some time trying to get out of the swamp
that is
> >> >> centos stream 8 and back to something with a actual upgrade path,
> >> >> fedora =)
> >> >>
> >> >> Everything works except the ipa-ca-install on the replica -
mostly
> >> >> fails at the same step
> >> >>
> >> >> At some point the conncheck failed, dropping me in to a prompt
asking
> >> >> for the password of a admin-<machine> account
> >> >>
> >> >> Anyway, I do know about the issue with - vs _ and validated on
master,
> >> >> changed on replica as detailed here:
> >> >>
https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedorah...
> >> >>
> >> >> But it still fails..
> >> >>
> >> >> Oh and btw, none of the machines are running any firewalls =)
> >> >>
> >> >> Anyone that has a clue of what to test next?
> >> >>
> >> >> Btw, it's 4.9 to 4.11 if there is other issues with
interoperability
> >> >>
> >> >> ipa-ca-install --skip-conncheck
> >> >> Directory Manager (existing master) password:
> >> >>
> >> >> Configuring certificate server (pki-tomcatd). Estimated time: 3
minutes
> >> >> [1/28]: creating certificate server db
> >> >> [2/28]: setting up initial replication
> >> >> Starting replication, please wait until this has completed.
> >> >> Update in progress, 7 seconds elapsed
> >> >> Update succeeded
> >> >>
> >> >> [3/28]: creating ACIs for admin
> >> >> [4/28]: creating installation admin user
> >> >> ipaserver.install.dogtaginstance: ERROR Unable to log in as
> >> >> uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca on
> >> >> ldap://freeipa-1.xerces.lan:389
> >> >> [error] NotFound:
uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca
> >> >> did not replicate to ldap://freeipa-1.xerces.lan:389
> >> >>
> >> >> Your system may be partly configured.
> >> >> Run /usr/sbin/ipa-server-install --uninstall to clean up.
> >> >>
> >> >> Unexpected error - see /var/log/ipareplica-ca-install.log for
details:
> >> >> NotFound: uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca did
not
> >> >> replicate to ldap://freeipa-1.xerces.lan:389
> >> >>
> >> > The installation of a CA clone creates this user on the replica,
lets the replication copy the entry on the master and then checks by doing
a ldap bind from the replica to the master that the entry has been properly
replicated.
> >> > When this error happens, it can either mean that the entry was not
replicated or that the bind failed.
> >> >
> >> > In order to know exactly what is happening for you, you can check
> >> > - on the master freeipa-1.xerces.lan, do a ldapsearch for this
entry and check if it exists. If the entry is present, the replication
properly propagated the entry from replica to master and you are probably
hitting the 2nd issue.
> >> > # ldapsearch -D "cn=directory manager" -W -b
uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca
> >> >
> >> > - on the replica, do a ldapsearch for this entry and check the
userpassword attribute. It is base64-encoded, and you can decode it in
order to find the password storage scheme that was used to encrypt the
password.
> >> > For instance on my machine:
> >> >
> >> > dn: uid=admin-replica.ipa.test,ou=people,o=ipaca
> >> > userPassword::
e1BCS0RGMl9TSEEyNTZ9QUFBSUFCWVMrWHUxVEJzb0VTcjJLQVl4RlZHWGRHWWZ
> >> >
NTmxFN3dCZHRRV1IxUTNxaTdKTXord2duLzIrc1NKMDZJbXhBeng5ZkR2VEIrMCsvQkZyMmRiL1pT
> >> >
dy96YzdhNWlVNGVCYnZHem9FODM0VHpIbHBweS9UeFRhc0Facm81OG1iT05OaUdBbml1c3pVcE5nb
> >> >
055R3dLYkpqQzZQeEpNeStnUklOa2xaOHJjTHBQSkZLam9jR0UvQ1NoeWFQYWN0b1ZZQlZVWHAzM3
> >> >
pyeWtZVlBIL0pIUjNQb2pnZnNUb2pRL2w5UWg1UGEwVjVVZ0VyUGpFK0dsNWtLS3FMaWE0d296Rk4
> >> >
wM3ozZjVwRGZDRnZOSi9CVEdENHhpcmNhcFZSVG5jTTRBZ0xPQlBCa2hoVm1vbEZBZHZ0OVUxY1ZL
> >> >
ZHVDZWRhWVUzZXZrS1hHcWx3alpTbEpPdkQ5SllJb0FHRlBwOXJERlJscU1MWEhUckx2aVoxTWgyM
> >> >
2Roa0hrR0VXM3pna3VuK2FIcnNvYUZMWWQwZi95NjlweDBRMzJvci9vOXBZV1F6S1ppNUFp
> >> >
> >> >
> >> > If I base64 decode the value:
> >> >
> >> > # echo
e1BCS0RGMl9TSEEyNTZ9QUFBSUFCWVMrWHUxVEJzb0VTcjJLQVl4RlZHWGRHWWZNTmxFN3dCZHRRV1IxUTNxaTdKTXord2duLzIrc1NKMDZJbXhBeng5ZkR2VEIrMCsvQkZyMmRiL1pTdy96YzdhNWlVNGVCYnZHem9FODM0VHpIbHBweS9UeFRhc0Facm81OG1iT05OaUdBbml1c3pVcE5nb055R3dLYkpqQzZQeEpNeStnUklOa2xaOHJjTHBQSkZLam9jR0UvQ1NoeWFQYWN0b1ZZQlZVWHAzM3pyeWtZVlBIL0pIUjNQb2pnZnNUb2pRL2w5UWg1UGEwVjVVZ0VyUGpFK0dsNWtLS3FMaWE0d296Rk4wM3ozZjVwRGZDRnZOSi9CVEdENHhpcmNhcFZSVG5jTTRBZ0xPQlBCa2hoVm1vbEZBZHZ0OVUxY1ZLZHVDZWRhWVUzZXZrS1hHcWx3alpTbEpPdkQ5SllJb0FHRlBwOXJERlJscU1MWEhUckx2aVoxTWgyM2Roa0hrR0VXM3pna3VuK2FIcnNvYUZMWWQwZi95NjlweDBRMzJvci9vOXBZV1F6S1ppNUFp
| base64 -d
> >> >
{PBKDF2_SHA256}AAAIABYS+Xu1TBsoESr2KAYxFVGXdGYfMNlE7wBdtQWR1Q3qi7JMz+wgn/2+sSJ06ImxAzx9fDvTB+0+/BFr2db/ZSw/zc7a5iU4eBbvGzoE834TzHlppy/TxTasAZro58mbONNiGAniuszUpNgoNyGwKbJjC6PxJMy+gRINklZ8rcLpPJFKjocGE/CShyaPactoVYBVUXp33zrykYVPH/JHR3PojgfsTojQ/l9Qh5Pa0V5UgErPjE+Gl5kKKqLia4wozFN03z3f5pDfCFvNJ/BTGD4xircapVRTncM4AgLOBPBkhhVmolFAdvt9U1cVKduCedaYU3evkKXGqlwjZSlJOvD9JYIoAGFPp9rDFRlqMLXHTrLviZ1Mh23dhkHkGEW3zgkun+aHrsoaFLYd0f/y69px0Q32or/o9pYWQzKZi5Ai
> >>
> >> Yes, and the value is the same on both replicas, both the encoded
> >> base64 and the password scheme: {PBKDF2_SHA256}AAAIAGIHopZZSHY8.....
> >>
> >> Since I changed it as described in the link i included...
> >>
> >> > which means that the replica used PBKDF2_SHA256 as password storage
scheme.
> >> > You need to check if this password storage scheme is supported on
the master (we had issues in the past with a password storage scheme used
by the replica that was not supported on the master and caused the bind to
fail,
https://bugzilla.redhat.com/show_bug.cgi?id=2151071). The list of
supported password storage schemes is available with the following command:
> >> > # ldapsearch -D "cn=directory manager" -W -LLL -o
ldif-wrap=no -b
"cn=Password Storage Schemes,cn=plugins,cn=config" -s one dn
> >>
> >> Yes, and they both support PBKDF2_SHA256 both as plugin and password
> >> storage scheme
> >>
> >> > If the replica is using a password scheme not supported on the
master, you are probably hitting the above BZ. There were fixes for
multiple versions of 389-ds, we would need to know your exact versions on
the replica and the master to point you to the right advisory.
> >>
> >> 4.9.10 and 4.11.1
> >>
> >> (fedora is just now updating it to 4.11.1-2 will look at the changes)
> >>
> >> Anyway, thanks for the help so far, i can now see the account
> >> replicated but i don't quite understand why it doesn't work...
> >
> >
> > Your ipa-ca-install command failed at 2024-03-11T15:05:24Z. Can you
check on the master around this date if there is a connection from replica
to master with a BIND attempt that would be failing? The logs are in
/var/log/dirsrv/slapd-YOURDOMAIN/access. Look for something similar to BIND
dn="uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca". Then note the conn
number and op number and look for the RESULT with the same conn and op, for
instance:
> >
> > [13/Mar/2024:09:10:01.331583308 +0000] conn=106 op=2 BIND
dn="uid=admin-replica0.ipa.test,ou=people,o=ipaca" method=128 version=3
> > [13/Mar/2024:09:10:01.355201520 +0000] conn=106 op=2 RESULT err=0
tag=97 nentries=0 wtime=0.000106547 optime=0.023642452 etime=0.023744831
dn="uid=admin-replica0.ipa.test,ou=people,o=ipaca"
> >
> >
> > The lines may be separated by other logs, and the err=xxx will show if
the bind is successful or failed. err=0 means success, err=49 means invalid
credentials.
>
> Yes, it does fail with invalid credentials...
>
> [11/Mar/2024:15:25:34.887970467 +0100] conn=118429 op=2 BIND
> dn="uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca" method=128
> version=3
> [11/Mar/2024:15:25:34.888893033 +0100] conn=118429 op=2 RESULT err=49
> tag=97 nentries=0 wtime=0.000265028 optime=0.000950489
> etime=0.001212411 - Invalid credentials
>
> Latest test, states
> [13/Mar/2024:10:41:45.063122671 +0100] conn=192008 op=2 RESULT err=49
> tag=97 nentries=0 wtime=0.000244892 optime=0.001078282
> etime=0.001320065 - No such entry
>
> I did remove it. as in uninstalled freeipa-4 removed the server and
> removed that account, just to make sure it was replicated properly...:
>
>
> [--8<--]
As a side node, the conncheck for ipa-ca-install fails all the time
now, when executing check on remote master it ends with this:
2024-03-14T07:42:26Z DEBUG Destroyed connection
context.rpcclient_139905569284576
2024-03-14T07:42:26Z ERROR ERROR: Remote master check failed with
following error message(s):
invalid 'cn': must be "freeipa-4.xerces.lan"
2024-03-14T07:42:26Z DEBUG Stopping listening thread.
Which seems really strange...
The message is highly misleading, but basically means that the conncheck
part tried
to check the connection to freeipa-4.xerces.lan, there was an
issue and then a different server was tried. Discussed in this thread:
Can you provide the exact OS + ipa / 389-ds-base versions that you have on
your server and on your replica? And any relevant info about the history of
the master (for instance, was the master initially installed with this
version or was it upgraded from older versions).
flo