Hi,
I have a small two node FreeIPA setup. (auth1 & auth2) I noticed today that I was getting a replication error on node 2 (auth2) about missing CSN in the changelog.
I reinitialized the two nodes replicating auth1 -> auth2 and this has fixed replication issues in the past, but the error persists.
I can create users and delete users from each side of the replication and it appears to be replicating those changes and they seem (non-definitively) to be in sync, but this error concerns me and reinitializing doesn't appear to solve it.
Here are the logs.
Any help is resolving this would be fantastic as I'm not finding much help via web searches.
Thanks, David
The sanitized error:
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.682835677 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized.
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.684721395 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged
Apr 16 10:50:31 auth2 ns-slapd[8419]: [16/Apr/2025:10:50:31.685877312 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
Santitized re-initialization
ipa topologysegment-reinitialize domain auth1....-to-auth2.... --right
--------------------------------------------------------------------------------------------
Replication refresh for segment: "auth1....-to-auth2...." requested.
--------------------------------------------------------------------------------------------
The sanitized logs of the re-initialization
Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.963872886 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_changebackend userRoot is going offline; inactivate plugin
Apr 16 10:45:59 auth2 ns-slapd[8419]: [16/Apr/2025:10:45:59.966877380 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is going offline; disabling replication
Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.189646600 -0400] - INFO - bdb_instance_start - Import is running with nsslapd-db-private-import-mem on; No other process is allowed to access the database
Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.891598813 -0400] - ERR - agmt="cn=caToauth1...." (auth1:389) - clcache_load_buffer - Can't locate CSN 66587eaa000100050000 in the changelog (DB rc=-12797). If replication stops, the consumer may need to be reinitialized.
Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.893083190 -0400] - ERR - NSMMReplicationPlugin - changelog program - repl_plugin_name_cl - agmt="cn=caToauth1...." (auth1:389): CSN 66587eaa000100050000 not found, we aren't as up to date, or we purged
Apr 16 10:46:00 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:00.894538030 -0400] - ERR - NSMMReplicationPlugin - send_updates - agmt="cn=caToauth1...." (auth1:389): Data required to update replica has been purged from the changelog. If the error persists the replica must be reinitialized.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.403074677 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers finished; cleaning up...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.605238770 -0400] - INFO - bdb_import_monitor_threads - import userRoot: Workers cleaned up.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.606678188 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Indexing complete. Post-processing...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.607860375 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numsubordinates (this may take several minutes to complete)...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.618231549 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Generating numSubordinates complete.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.620014951 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Gathering ancestorid non-leaf IDs...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.621434375 -0400] - INFO - bdb_get_nonleaf_ids - import userRoot: Finished gathering ancestorid non-leaf IDs.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.622574267 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Starting sort of ancestorid non-leaf IDs...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.623757818 -0400] - INFO - ldbm_get_nonleaf_ids - import userRoot: Finished sort of ancestorid non-leaf IDs.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.626748319 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Creating ancestorid index (new idl)...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.632623820 -0400] - INFO - bdb_ancestorid_new_idl_create_index - import userRoot: Created ancestorid index (new idl).
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.633896253 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Flushing caches...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.635305588 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Closing files...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.728196646 -0400] - INFO - bdb_public_bdb_import_main - import userRoot: Import complete. Processed 729 entries in 3 seconds. (243.00 entries/sec)
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.737959417 -0400] - ERR - ipa-topology-plugin - ipa_topo_be_state_change - backend userRoot is coming online; checking domain level and init shared topology
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.744152900 -0400] - NOTICE - NSMMReplicationPlugin - multisupplier_be_state_change - Replica dc=...,dc=... is coming online; enabling replication
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.754743353 -0400] - WARN - NSMMReplicationPlugin - replica_reload_ruv - New data for replica dc=...,dc=... does not match the data in the changelog.
Apr 16 10:46:02 auth2 ns-slapd[8419]: Recreating the changelog file. This could affect replication with replica's consumers in which case the consumers should be reinitialized.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.862749463 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.864263319 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.872479720 -0400] - ERR - cos-plugin - cos_dn_defs_cb - Skipping CoS Definition cn=Password Policy,cn=accounts,dc=...,dc=...--no CoS Templates found, which should be added before the CoS Definition.
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.874025309 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=groups,cn=compat,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.875303781 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=computers,cn=compat,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.876489711 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=ng,cn=compat,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.877770904 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target ou=sudoers,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.879231097 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=users,cn=compat,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.880458410 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.881648891 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.882722133 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.884124162 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.885222292 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.886404863 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.887615474 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.889102423 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.890327963 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.891412886 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.892586141 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=vaults,cn=kra,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.899706161 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.901020418 -0400] - WARN - NSACLPlugin - acl_parse - The ACL target cn=casigningcert cert-pki-ca,cn=ca_renewal,cn=ipa,cn=etc,dc=...,dc=... does not exist
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.907037194 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding the replication changelog RUV, this may take several minutes...
Apr 16 10:46:02 auth2 ns-slapd[8419]: [16/Apr/2025:10:46:02.908357262 -0400] - NOTICE - NSMMReplicationPlugin - changelog program - _cl5ConstructRUVs - Rebuilding replication changelog RUV complete. Result 0 (Success)