Hi,
On Thu, Mar 14, 2024 at 8:55 AM Ian Kumlien ian.kumlien@gmail.com wrote:
On Wed, Mar 13, 2024 at 1:58 PM Ian Kumlien ian.kumlien@gmail.com wrote:
On Wed, Mar 13, 2024 at 11:39 AM Florence Blanc-Renaud flo@redhat.com
wrote:
Hi,
On Wed, Mar 13, 2024 at 10:06 AM Ian Kumlien ian.kumlien@gmail.com
wrote:
On Tue, Mar 12, 2024 at 10:36 PM Florence Blanc-Renaud <
flo@redhat.com> wrote:
Hi,
On Tue, Mar 12, 2024 at 12:54 PM Ian Kumlien via FreeIPA-users <
freeipa-users@lists.fedorahosted.org> wrote:
Hi,
So i have spent quite some time trying to get out of the swamp
that is
centos stream 8 and back to something with a actual upgrade path, fedora =)
Everything works except the ipa-ca-install on the replica - mostly fails at the same step
At some point the conncheck failed, dropping me in to a prompt
asking
for the password of a admin-<machine> account
Anyway, I do know about the issue with - vs _ and validated on
master,
changed on replica as detailed here:
https://lists.fedoraproject.org/archives/list/freeipa-users@lists.fedorahost...
But it still fails..
Oh and btw, none of the machines are running any firewalls =)
Anyone that has a clue of what to test next?
Btw, it's 4.9 to 4.11 if there is other issues with
interoperability
ipa-ca-install --skip-conncheck Directory Manager (existing master) password:
Configuring certificate server (pki-tomcatd). Estimated time: 3
minutes
[1/28]: creating certificate server db [2/28]: setting up initial replication Starting replication, please wait until this has completed. Update in progress, 7 seconds elapsed Update succeeded
[3/28]: creating ACIs for admin [4/28]: creating installation admin user ipaserver.install.dogtaginstance: ERROR Unable to log in as uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca on ldap://freeipa-1.xerces.lan:389 [error] NotFound:
uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca
did not replicate to ldap://freeipa-1.xerces.lan:389
Your system may be partly configured. Run /usr/sbin/ipa-server-install --uninstall to clean up.
Unexpected error - see /var/log/ipareplica-ca-install.log for
details:
NotFound: uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca did not replicate to ldap://freeipa-1.xerces.lan:389
The installation of a CA clone creates this user on the replica,
lets the replication copy the entry on the master and then checks by doing a ldap bind from the replica to the master that the entry has been properly replicated.
When this error happens, it can either mean that the entry was not
replicated or that the bind failed.
In order to know exactly what is happening for you, you can check
- on the master freeipa-1.xerces.lan, do a ldapsearch for this
entry and check if it exists. If the entry is present, the replication properly propagated the entry from replica to master and you are probably hitting the 2nd issue.
# ldapsearch -D "cn=directory manager" -W -b
uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca
- on the replica, do a ldapsearch for this entry and check the
userpassword attribute. It is base64-encoded, and you can decode it in order to find the password storage scheme that was used to encrypt the password.
For instance on my machine:
dn: uid=admin-replica.ipa.test,ou=people,o=ipaca userPassword::
e1BCS0RGMl9TSEEyNTZ9QUFBSUFCWVMrWHUxVEJzb0VTcjJLQVl4RlZHWGRHWWZ
NTmxFN3dCZHRRV1IxUTNxaTdKTXord2duLzIrc1NKMDZJbXhBeng5ZkR2VEIrMCsvQkZyMmRiL1pT
dy96YzdhNWlVNGVCYnZHem9FODM0VHpIbHBweS9UeFRhc0Facm81OG1iT05OaUdBbml1c3pVcE5nb
055R3dLYkpqQzZQeEpNeStnUklOa2xaOHJjTHBQSkZLam9jR0UvQ1NoeWFQYWN0b1ZZQlZVWHAzM3
pyeWtZVlBIL0pIUjNQb2pnZnNUb2pRL2w5UWg1UGEwVjVVZ0VyUGpFK0dsNWtLS3FMaWE0d296Rk4
wM3ozZjVwRGZDRnZOSi9CVEdENHhpcmNhcFZSVG5jTTRBZ0xPQlBCa2hoVm1vbEZBZHZ0OVUxY1ZL
ZHVDZWRhWVUzZXZrS1hHcWx3alpTbEpPdkQ5SllJb0FHRlBwOXJERlJscU1MWEhUckx2aVoxTWgyM
2Roa0hrR0VXM3pna3VuK2FIcnNvYUZMWWQwZi95NjlweDBRMzJvci9vOXBZV1F6S1ppNUFp
If I base64 decode the value:
# echo
e1BCS0RGMl9TSEEyNTZ9QUFBSUFCWVMrWHUxVEJzb0VTcjJLQVl4RlZHWGRHWWZNTmxFN3dCZHRRV1IxUTNxaTdKTXord2duLzIrc1NKMDZJbXhBeng5ZkR2VEIrMCsvQkZyMmRiL1pTdy96YzdhNWlVNGVCYnZHem9FODM0VHpIbHBweS9UeFRhc0Facm81OG1iT05OaUdBbml1c3pVcE5nb055R3dLYkpqQzZQeEpNeStnUklOa2xaOHJjTHBQSkZLam9jR0UvQ1NoeWFQYWN0b1ZZQlZVWHAzM3pyeWtZVlBIL0pIUjNQb2pnZnNUb2pRL2w5UWg1UGEwVjVVZ0VyUGpFK0dsNWtLS3FMaWE0d296Rk4wM3ozZjVwRGZDRnZOSi9CVEdENHhpcmNhcFZSVG5jTTRBZ0xPQlBCa2hoVm1vbEZBZHZ0OVUxY1ZLZHVDZWRhWVUzZXZrS1hHcWx3alpTbEpPdkQ5SllJb0FHRlBwOXJERlJscU1MWEhUckx2aVoxTWgyM2Roa0hrR0VXM3pna3VuK2FIcnNvYUZMWWQwZi95NjlweDBRMzJvci9vOXBZV1F6S1ppNUFp | base64 -d
{PBKDF2_SHA256}AAAIABYS+Xu1TBsoESr2KAYxFVGXdGYfMNlE7wBdtQWR1Q3qi7JMz+wgn/2+sSJ06ImxAzx9fDvTB+0+/BFr2db/ZSw/zc7a5iU4eBbvGzoE834TzHlppy/TxTasAZro58mbONNiGAniuszUpNgoNyGwKbJjC6PxJMy+gRINklZ8rcLpPJFKjocGE/CShyaPactoVYBVUXp33zrykYVPH/JHR3PojgfsTojQ/l9Qh5Pa0V5UgErPjE+Gl5kKKqLia4wozFN03z3f5pDfCFvNJ/BTGD4xircapVRTncM4AgLOBPBkhhVmolFAdvt9U1cVKduCedaYU3evkKXGqlwjZSlJOvD9JYIoAGFPp9rDFRlqMLXHTrLviZ1Mh23dhkHkGEW3zgkun+aHrsoaFLYd0f/y69px0Q32or/o9pYWQzKZi5Ai
Yes, and the value is the same on both replicas, both the encoded base64 and the password scheme: {PBKDF2_SHA256}AAAIAGIHopZZSHY8.....
Since I changed it as described in the link i included...
which means that the replica used PBKDF2_SHA256 as password storage
scheme.
You need to check if this password storage scheme is supported on
the master (we had issues in the past with a password storage scheme used by the replica that was not supported on the master and caused the bind to fail, https://bugzilla.redhat.com/show_bug.cgi?id=2151071). The list of supported password storage schemes is available with the following command:
# ldapsearch -D "cn=directory manager" -W -LLL -o ldif-wrap=no -b
"cn=Password Storage Schemes,cn=plugins,cn=config" -s one dn
Yes, and they both support PBKDF2_SHA256 both as plugin and password storage scheme
If the replica is using a password scheme not supported on the
master, you are probably hitting the above BZ. There were fixes for multiple versions of 389-ds, we would need to know your exact versions on the replica and the master to point you to the right advisory.
4.9.10 and 4.11.1
(fedora is just now updating it to 4.11.1-2 will look at the changes)
Anyway, thanks for the help so far, i can now see the account replicated but i don't quite understand why it doesn't work...
Your ipa-ca-install command failed at 2024-03-11T15:05:24Z. Can you
check on the master around this date if there is a connection from replica to master with a BIND attempt that would be failing? The logs are in /var/log/dirsrv/slapd-YOURDOMAIN/access. Look for something similar to BIND dn="uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca". Then note the conn number and op number and look for the RESULT with the same conn and op, for instance:
[13/Mar/2024:09:10:01.331583308 +0000] conn=106 op=2 BIND
dn="uid=admin-replica0.ipa.test,ou=people,o=ipaca" method=128 version=3
[13/Mar/2024:09:10:01.355201520 +0000] conn=106 op=2 RESULT err=0
tag=97 nentries=0 wtime=0.000106547 optime=0.023642452 etime=0.023744831 dn="uid=admin-replica0.ipa.test,ou=people,o=ipaca"
The lines may be separated by other logs, and the err=xxx will show if
the bind is successful or failed. err=0 means success, err=49 means invalid credentials.
Yes, it does fail with invalid credentials...
[11/Mar/2024:15:25:34.887970467 +0100] conn=118429 op=2 BIND dn="uid=admin-freeipa-4.xerces.lan,ou=people,o=ipaca" method=128 version=3 [11/Mar/2024:15:25:34.888893033 +0100] conn=118429 op=2 RESULT err=49 tag=97 nentries=0 wtime=0.000265028 optime=0.000950489 etime=0.001212411 - Invalid credentials
Latest test, states [13/Mar/2024:10:41:45.063122671 +0100] conn=192008 op=2 RESULT err=49 tag=97 nentries=0 wtime=0.000244892 optime=0.001078282 etime=0.001320065 - No such entry
I did remove it. as in uninstalled freeipa-4 removed the server and removed that account, just to make sure it was replicated properly...:
[--8<--]
As a side node, the conncheck for ipa-ca-install fails all the time now, when executing check on remote master it ends with this: 2024-03-14T07:42:26Z DEBUG Destroyed connection context.rpcclient_139905569284576 2024-03-14T07:42:26Z ERROR ERROR: Remote master check failed with following error message(s): invalid 'cn': must be "freeipa-4.xerces.lan" 2024-03-14T07:42:26Z DEBUG Stopping listening thread.
Which seems really strange...
The message is highly misleading, but basically means that the conncheck
part tried to check the connection to freeipa-4.xerces.lan, there was an issue and then a different server was tried. Discussed in this thread: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahoste...
Can you provide the exact OS + ipa / 389-ds-base versions that you have on your server and on your replica? And any relevant info about the history of the master (for instance, was the master initially installed with this version or was it upgraded from older versions).
flo