Sign In
Sign Up
Sign In
Sign Up
Manage this list
×
Keyboard Shortcuts
Thread View
j
: Next unread message
k
: Previous unread message
j a
: Jump to all threads
j l
: Jump to MailingList overview
2024
December
November
October
September
August
July
June
May
April
March
February
January
2023
December
November
October
September
August
July
June
May
April
March
February
January
2022
December
November
October
September
August
July
June
May
April
March
February
January
2021
December
November
October
September
August
July
June
May
April
March
February
January
2020
December
November
October
September
August
July
June
May
April
March
February
January
2019
December
November
October
September
August
July
June
May
April
March
February
January
2018
December
November
October
September
August
July
June
May
April
March
February
January
2017
December
November
October
September
August
July
June
May
April
March
February
January
2016
December
November
October
September
August
July
June
May
April
March
February
January
2015
December
November
October
September
August
July
June
May
April
March
February
January
2014
December
November
October
September
August
July
June
May
April
March
February
January
2013
December
November
October
September
August
July
June
May
April
March
February
January
2012
December
November
October
September
August
July
June
May
April
March
February
January
2011
December
November
October
September
August
July
June
May
April
March
February
January
2010
December
November
October
September
August
July
June
May
April
March
February
January
2009
December
November
October
September
August
July
June
May
April
March
February
January
2008
December
November
October
September
August
July
June
May
April
March
February
January
2007
December
November
October
September
August
July
June
May
April
March
February
January
2006
December
November
October
September
August
July
June
May
April
March
February
January
2005
December
November
October
September
August
July
June
List overview
Download
389-commits
May 2014
----- 2024 -----
December 2024
November 2024
October 2024
September 2024
August 2024
July 2024
June 2024
May 2024
April 2024
March 2024
February 2024
January 2024
----- 2023 -----
December 2023
November 2023
October 2023
September 2023
August 2023
July 2023
June 2023
May 2023
April 2023
March 2023
February 2023
January 2023
----- 2022 -----
December 2022
November 2022
October 2022
September 2022
August 2022
July 2022
June 2022
May 2022
April 2022
March 2022
February 2022
January 2022
----- 2021 -----
December 2021
November 2021
October 2021
September 2021
August 2021
July 2021
June 2021
May 2021
April 2021
March 2021
February 2021
January 2021
----- 2020 -----
December 2020
November 2020
October 2020
September 2020
August 2020
July 2020
June 2020
May 2020
April 2020
March 2020
February 2020
January 2020
----- 2019 -----
December 2019
November 2019
October 2019
September 2019
August 2019
July 2019
June 2019
May 2019
April 2019
March 2019
February 2019
January 2019
----- 2018 -----
December 2018
November 2018
October 2018
September 2018
August 2018
July 2018
June 2018
May 2018
April 2018
March 2018
February 2018
January 2018
----- 2017 -----
December 2017
November 2017
October 2017
September 2017
August 2017
July 2017
June 2017
May 2017
April 2017
March 2017
February 2017
January 2017
----- 2016 -----
December 2016
November 2016
October 2016
September 2016
August 2016
July 2016
June 2016
May 2016
April 2016
March 2016
February 2016
January 2016
----- 2015 -----
December 2015
November 2015
October 2015
September 2015
August 2015
July 2015
June 2015
May 2015
April 2015
March 2015
February 2015
January 2015
----- 2014 -----
December 2014
November 2014
October 2014
September 2014
August 2014
July 2014
June 2014
May 2014
April 2014
March 2014
February 2014
January 2014
----- 2013 -----
December 2013
November 2013
October 2013
September 2013
August 2013
July 2013
June 2013
May 2013
April 2013
March 2013
February 2013
January 2013
----- 2012 -----
December 2012
November 2012
October 2012
September 2012
August 2012
July 2012
June 2012
May 2012
April 2012
March 2012
February 2012
January 2012
----- 2011 -----
December 2011
November 2011
October 2011
September 2011
August 2011
July 2011
June 2011
May 2011
April 2011
March 2011
February 2011
January 2011
----- 2010 -----
December 2010
November 2010
October 2010
September 2010
August 2010
July 2010
June 2010
May 2010
April 2010
March 2010
February 2010
January 2010
----- 2009 -----
December 2009
November 2009
October 2009
September 2009
August 2009
July 2009
June 2009
May 2009
April 2009
March 2009
February 2009
January 2009
----- 2008 -----
December 2008
November 2008
October 2008
September 2008
August 2008
July 2008
June 2008
May 2008
April 2008
March 2008
February 2008
January 2008
----- 2007 -----
December 2007
November 2007
October 2007
September 2007
August 2007
July 2007
June 2007
May 2007
April 2007
March 2007
February 2007
January 2007
----- 2006 -----
December 2006
November 2006
October 2006
September 2006
August 2006
July 2006
June 2006
May 2006
April 2006
March 2006
February 2006
January 2006
----- 2005 -----
December 2005
November 2005
October 2005
September 2005
August 2005
July 2005
June 2005
389-commits@lists.fedoraproject.org
4 participants
64 discussions
Start a n
N
ew thread
Branch '389-ds-base-1.3.2' - dirsrvtests/tickets ldap/servers
by thierry bordaz
13 May '14
13 May '14
dirsrvtests/tickets/ticket47787_test.py | 630 +++++++++++++++++++++++++++++ ldap/servers/slapd/back-ldbm/ldbm_modify.c | 2 2 files changed, 631 insertions(+), 1 deletion(-) New commits: commit c13feeefc47f399199f611e0e42db546c950a83c Author: Thierry bordaz (tbordaz) <tbordaz(a)redhat.com> Date: Fri Apr 25 15:07:08 2014 +0200 Ticket 47787 - A replicated MOD fails (Unwilling to perform) if it targets a tombstone Bug Description: A fix for
https://fedorahosted.org/389/ticket/47396
prevents a MOD operation on a tombstone. A problem is that legacy version may allowed it or if a MOD entry and DEL entry occurs on different master, it may happen that a consumer receive a MOD on a tombstone. The server return 'Unwilling to perform' that may break replication. Fix Description: Allows a MOD on tombstone, only if it comes from a replication session
https://fedorahosted.org/389/ticket/47787
Reviewed by: Rich Megginson (thanks) Platforms tested: F17, F19 Flag Day: no Doc impact: no diff --git a/dirsrvtests/tickets/ticket47787_test.py b/dirsrvtests/tickets/ticket47787_test.py new file mode 100644 index 0000000..d9d3ca9 --- /dev/null +++ b/dirsrvtests/tickets/ticket47787_test.py @@ -0,0 +1,630 @@ +''' +Created on April 14, 2014 + +@author: tbordaz +''' +import os +import sys +import time +import ldap +import logging +import socket +import time +import logging +import pytest +import re +from lib389 import DirSrv, Entry, tools, NoSuchEntryError +from lib389.tools import DirSrvTools +from lib389._constants import * +from lib389.properties import * +from constants import * +from lib389._constants import REPLICAROLE_MASTER + +logging.getLogger(__name__).setLevel(logging.DEBUG) +log = logging.getLogger(__name__) + +# +# important part. We can deploy Master1 and Master2 on different versions +# +installation1_prefix = None +installation2_prefix = None + +# set this flag to False so that it will assert on failure _status_entry_both_server +DEBUG_FLAG = False + +TEST_REPL_DN = "cn=test_repl, %s" % SUFFIX + +STAGING_CN = "staged user" +PRODUCTION_CN = "accounts" +EXCEPT_CN = "excepts" + +STAGING_DN = "cn=%s,%s" % (STAGING_CN, SUFFIX) +PRODUCTION_DN = "cn=%s,%s" % (PRODUCTION_CN, SUFFIX) +PROD_EXCEPT_DN = "cn=%s,%s" % (EXCEPT_CN, PRODUCTION_DN) + +STAGING_PATTERN = "cn=%s*,%s" % (STAGING_CN[:2], SUFFIX) +PRODUCTION_PATTERN = "cn=%s*,%s" % (PRODUCTION_CN[:2], SUFFIX) +BAD_STAGING_PATTERN = "cn=bad*,%s" % (SUFFIX) +BAD_PRODUCTION_PATTERN = "cn=bad*,%s" % (SUFFIX) + +BIND_CN = "bind_entry" +BIND_DN = "cn=%s,%s" % (BIND_CN, SUFFIX) +BIND_PW = "password" + +NEW_ACCOUNT = "new_account" +MAX_ACCOUNTS = 20 + +CONFIG_MODDN_ACI_ATTR = "nsslapd-moddn-aci" + +class TopologyMaster1Master2(object): + def __init__(self, master1, master2): + master1.open() + self.master1 = master1 + + master2.open() + self.master2 = master2 + + +(a)pytest.fixture(scope="module") +def topology(request): + ''' + This fixture is used to create a replicated topology for the 'module'. + The replicated topology is MASTER1 <-> Master2. + At the beginning, It may exists a master2 instance and/or a master2 instance. + It may also exists a backup for the master1 and/or the master2. + + Principle: + If master1 instance exists: + restart it + If master2 instance exists: + restart it + If backup of master1 AND backup of master2 exists: + create or rebind to master1 + create or rebind to master2 + + restore master1 from backup + restore master2 from backup + else: + Cleanup everything + remove instances + remove backups + Create instances + Initialize replication + Create backups + ''' + global installation1_prefix + global installation2_prefix + + # allocate master1 on a given deployement + master1 = DirSrv(verbose=False) + if installation1_prefix: + args_instance[SER_DEPLOYED_DIR] = installation1_prefix + + # Args for the master1 instance + args_instance[SER_HOST] = HOST_MASTER_1 + args_instance[SER_PORT] = PORT_MASTER_1 + args_instance[SER_SERVERID_PROP] = SERVERID_MASTER_1 + args_master = args_instance.copy() + master1.allocate(args_master) + + # allocate master1 on a given deployement + master2 = DirSrv(verbose=False) + if installation2_prefix: + args_instance[SER_DEPLOYED_DIR] = installation2_prefix + + # Args for the consumer instance + args_instance[SER_HOST] = HOST_MASTER_2 + args_instance[SER_PORT] = PORT_MASTER_2 + args_instance[SER_SERVERID_PROP] = SERVERID_MASTER_2 + args_master = args_instance.copy() + master2.allocate(args_master) + + + # Get the status of the backups + backup_master1 = master1.checkBackupFS() + backup_master2 = master2.checkBackupFS() + + # Get the status of the instance and restart it if it exists + instance_master1 = master1.exists() + if instance_master1: + master1.stop(timeout=10) + master1.start(timeout=10) + + instance_master2 = master2.exists() + if instance_master2: + master2.stop(timeout=10) + master2.start(timeout=10) + + if backup_master1 and backup_master2: + # The backups exist, assuming they are correct + # we just re-init the instances with them + if not instance_master1: + master1.create() + # Used to retrieve configuration information (dbdir, confdir...) + master1.open() + + if not instance_master2: + master2.create() + # Used to retrieve configuration information (dbdir, confdir...) + master2.open() + + # restore master1 from backup + master1.stop(timeout=10) + master1.restoreFS(backup_master1) + master1.start(timeout=10) + + # restore master2 from backup + master2.stop(timeout=10) + master2.restoreFS(backup_master2) + master2.start(timeout=10) + else: + # We should be here only in two conditions + # - This is the first time a test involve master-consumer + # so we need to create everything + # - Something weird happened (instance/backup destroyed) + # so we discard everything and recreate all + + # Remove all the backups. So even if we have a specific backup file + # (e.g backup_master) we clear all backups that an instance my have created + if backup_master1: + master1.clearBackupFS() + if backup_master2: + master2.clearBackupFS() + + # Remove all the instances + if instance_master1: + master1.delete() + if instance_master2: + master2.delete() + + # Create the instances + master1.create() + master1.open() + master2.create() + master2.open() + + # + # Now prepare the Master-Consumer topology + # + # First Enable replication + master1.replica.enableReplication(suffix=SUFFIX, role=REPLICAROLE_MASTER, replicaId=REPLICAID_MASTER_1) + master2.replica.enableReplication(suffix=SUFFIX, role=REPLICAROLE_MASTER, replicaId=REPLICAID_MASTER_2) + + # Initialize the supplier->consumer + + properties = {RA_NAME: r'meTo_$host:$port', + RA_BINDDN: defaultProperties[REPLICATION_BIND_DN], + RA_BINDPW: defaultProperties[REPLICATION_BIND_PW], + RA_METHOD: defaultProperties[REPLICATION_BIND_METHOD], + RA_TRANSPORT_PROT: defaultProperties[REPLICATION_TRANSPORT]} + repl_agreement = master1.agreement.create(suffix=SUFFIX, host=master2.host, port=master2.port, properties=properties) + + if not repl_agreement: + log.fatal("Fail to create a replica agreement") + sys.exit(1) + + log.debug("%s created" % repl_agreement) + + properties = {RA_NAME: r'meTo_$host:$port', + RA_BINDDN: defaultProperties[REPLICATION_BIND_DN], + RA_BINDPW: defaultProperties[REPLICATION_BIND_PW], + RA_METHOD: defaultProperties[REPLICATION_BIND_METHOD], + RA_TRANSPORT_PROT: defaultProperties[REPLICATION_TRANSPORT]} + master2.agreement.create(suffix=SUFFIX, host=master1.host, port=master1.port, properties=properties) + + master1.agreement.init(SUFFIX, HOST_MASTER_2, PORT_MASTER_2) + master1.waitForReplInit(repl_agreement) + + # Check replication is working fine + master1.add_s(Entry((TEST_REPL_DN, { + 'objectclass': "top person".split(), + 'sn': 'test_repl', + 'cn': 'test_repl'}))) + loop = 0 + while loop <= 10: + try: + ent = master2.getEntry(TEST_REPL_DN, ldap.SCOPE_BASE, "(objectclass=*)") + break + except ldap.NO_SUCH_OBJECT: + time.sleep(1) + loop += 1 + + # Time to create the backups + master1.stop(timeout=10) + master1.backupfile = master1.backupFS() + master1.start(timeout=10) + + master2.stop(timeout=10) + master2.backupfile = master2.backupFS() + master2.start(timeout=10) + + # + # Here we have two instances master and consumer + # with replication working. Either coming from a backup recovery + # or from a fresh (re)init + # Time to return the topology + return TopologyMaster1Master2(master1, master2) + + + +def _bind_manager(server): + server.log.info("Bind as %s " % DN_DM) + server.simple_bind_s(DN_DM, PASSWORD) + +def _bind_normal(server): + server.log.info("Bind as %s " % BIND_DN) + server.simple_bind_s(BIND_DN, BIND_PW) + +def _header(topology, label): + topology.master1.log.info("\n\n###############################################") + topology.master1.log.info("#######") + topology.master1.log.info("####### %s" % label) + topology.master1.log.info("#######") + topology.master1.log.info("###############################################") + +def _status_entry_both_server(topology, name=None, desc=None, debug=True): + if not name: + return + topology.master1.log.info("\n\n######################### Tombstone on M1 ######################\n") + ent_m1 = _find_tombstone(topology.master1, SUFFIX, 'sn', name) + assert ent_m1 + + topology.master1.log.info("\n\n######################### Tombstone on M2 ######################\n") + ent_m2 = _find_tombstone(topology.master2, SUFFIX, 'sn', name) + assert ent_m2 + + topology.master1.log.info("\n\n######################### Description ######################\n%s\n" % desc) + topology.master1.log.info("M1 only\n") + for attr in ent_m1.getAttrs(): + + if not debug: + assert attr in ent_m2.getAttrs() + + if not attr in ent_m2.getAttrs(): + topology.master1.log.info(" %s" % attr) + for val in ent_m1.getValues(attr): + topology.master1.log.info(" %s" % val) + + topology.master1.log.info("M2 only\n") + for attr in ent_m2.getAttrs(): + + if not debug: + assert attr in ent_m1.getAttrs() + + if not attr in ent_m1.getAttrs(): + topology.master1.log.info(" %s" % attr) + for val in ent_m2.getValues(attr): + topology.master1.log.info(" %s" % val) + + topology.master1.log.info("M1 differs M2\n") + + if not debug: + assert ent_m1.dn == ent_m2.dn + + if ent_m1.dn != ent_m2.dn: + topology.master1.log.info(" M1[dn] = %s\n M2[dn] = %s" % (ent_m1.dn, ent_m2.dn)) + + for attr1 in ent_m1.getAttrs(): + if attr1 in ent_m2.getAttrs(): + for val1 in ent_m1.getValues(attr1): + found = False + for val2 in ent_m2.getValues(attr1): + if val1 == val2: + found = True + break + + if not debug: + assert found + + if not found: + topology.master1.log.info(" M1[%s] = %s" % (attr1, val1)) + + + for attr2 in ent_m2.getAttrs(): + if attr2 in ent_m1.getAttrs(): + for val2 in ent_m2.getValues(attr2): + found = False + for val1 in ent_m1.getValues(attr2): + if val2 == val1: + found = True + break + + if not debug: + assert found + + if not found: + topology.master1.log.info(" M2[%s] = %s" % (attr2, val2)) + +def _pause_RAs(topology): + topology.master1.log.info("\n\n######################### Pause RA M1<->M2 ######################\n") + ents = topology.master1.agreement.list(suffix=SUFFIX) + assert len(ents) == 1 + topology.master1.agreement.pause(ents[0].dn) + + ents = topology.master2.agreement.list(suffix=SUFFIX) + assert len(ents) == 1 + topology.master2.agreement.pause(ents[0].dn) + +def _resume_RAs(topology): + topology.master1.log.info("\n\n######################### resume RA M1<->M2 ######################\n") + ents = topology.master1.agreement.list(suffix=SUFFIX) + assert len(ents) == 1 + topology.master1.agreement.resume(ents[0].dn) + + ents = topology.master2.agreement.list(suffix=SUFFIX) + assert len(ents) == 1 + topology.master2.agreement.resume(ents[0].dn) + +def _find_tombstone(instance, base, attr, value): + # + # we can not use a filter with a (&(objeclass=nsTombstone)(sn=name)) because + # tombstone are not index in 'sn' so 'sn=name' will return NULL + # and even if tombstone are indexed for objectclass the '&' will set + # the candidate list to NULL + # + filt = '(objectclass=%s)' % REPLICA_OC_TOMBSTONE + ents = instance.search_s(base, ldap.SCOPE_SUBTREE, filt) + found = False + for ent in ents: + if ent.hasAttr(attr): + for val in ent.getValues(attr): + if val == value: + instance.log.debug("tombstone found: %r" % ent) + return ent + return None + + +def _delete_entry(instance, entry_dn, name): + instance.log.info("\n\n######################### DELETE %s (M1) ######################\n" % name) + + # delete the entry + instance.delete_s(entry_dn) + assert _find_tombstone(instance, SUFFIX, 'sn', name) != None + + + +def _mod_entry(instance, entry_dn, attr, value): + instance.log.info("\n\n######################### MOD %s (M2) ######################\n" % entry_dn) + mod = [(ldap.MOD_REPLACE, attr, value)] + instance.modify_s(entry_dn, mod) + +def _modrdn_entry(instance=None, entry_dn=None, new_rdn=None, del_old=0, new_superior=None): + assert instance != None + assert entry_dn != None + + if not new_rdn: + pattern = 'cn=(.*),(.*)' + rdnre = re.compile(pattern) + match = rdnre.match(entry_dn) + old_value = match.group(1) + new_rdn_val = "%s_modrdn" % old_value + new_rdn = "cn=%s" % new_rdn_val + + + instance.log.info("\n\n######################### MODRDN %s (M2) ######################\n" % new_rdn) + if new_superior: + instance.rename_s(entry_dn, new_rdn, newsuperior=new_superior, delold=del_old) + else: + instance.rename_s(entry_dn, new_rdn, delold=del_old) + +def _check_entry_exists(instance, entry_dn): + loop = 0 + while loop <= 10: + try: + ent = instance.getEntry(entry_dn, ldap.SCOPE_BASE, "(objectclass=*)") + break + except ldap.NO_SUCH_OBJECT: + time.sleep(1) + loop += 1 + assert loop <= 10 + +def _check_mod_received(instance, base, filt, attr, value): + instance.log.info("\n\n######################### Check MOD replicated on %s ######################\n" % instance.serverid) + loop = 0 + while loop <= 10: + ent = instance.getEntry(base, ldap.SCOPE_SUBTREE, filt) + if ent.hasAttr(attr) and ent.getValue(attr) == value: + break + time.sleep(1) + loop += 1 + assert loop <= 10 + +def _check_replication(topology, entry_dn): + # prepare the filter to retrieve the entry + filt = entry_dn.split(',')[0] + + topology.master1.log.info("\n######################### Check replicat M1->M2 ######################\n") + loop = 0 + while loop <= 10: + attr = 'description' + value = 'test_value_%d' % loop + mod = [(ldap.MOD_REPLACE, attr, value)] + topology.master1.modify_s(entry_dn, mod) + _check_mod_received(topology.master2, SUFFIX, filt, attr, value) + loop += 1 + + topology.master1.log.info("\n######################### Check replicat M2->M1 ######################\n") + loop = 0 + while loop <= 10: + attr = 'description' + value = 'test_value_%d' % loop + mod = [(ldap.MOD_REPLACE, attr, value)] + topology.master2.modify_s(entry_dn, mod) + _check_mod_received(topology.master1, SUFFIX, filt, attr, value) + loop += 1 + + +def test_ticket47787_init(topology): + """ + Creates + - a staging DIT + - a production DIT + - add accounts in staging DIT + + """ + + topology.master1.log.info("\n\n######################### INITIALIZATION ######################\n") + + # entry used to bind with + topology.master1.log.info("Add %s" % BIND_DN) + topology.master1.add_s(Entry((BIND_DN, { + 'objectclass': "top person".split(), + 'sn': BIND_CN, + 'cn': BIND_CN, + 'userpassword': BIND_PW}))) + + # DIT for staging + topology.master1.log.info("Add %s" % STAGING_DN) + topology.master1.add_s(Entry((STAGING_DN, { + 'objectclass': "top organizationalRole".split(), + 'cn': STAGING_CN, + 'description': "staging DIT"}))) + + # DIT for production + topology.master1.log.info("Add %s" % PRODUCTION_DN) + topology.master1.add_s(Entry((PRODUCTION_DN, { + 'objectclass': "top organizationalRole".split(), + 'cn': PRODUCTION_CN, + 'description': "production DIT"}))) + + + # enable replication error logging + mod = [(ldap.MOD_REPLACE, 'nsslapd-errorlog-level', '8192')] + topology.master1.modify_s(DN_CONFIG, mod) + topology.master2.modify_s(DN_CONFIG, mod) + + + + + + # add dummy entries in the staging DIT + for cpt in range(MAX_ACCOUNTS): + name = "%s%d" % (NEW_ACCOUNT, cpt) + topology.master1.add_s(Entry(("cn=%s,%s" % (name, STAGING_DN), { + 'objectclass': "top person".split(), + 'sn': name, + 'cn': name}))) + + +def test_ticket47787_2(topology): + ''' + Disable replication so that updates are not replicated + Delete an entry on M1. Modrdn it on M2 (chg rdn + delold=0 + same superior). + update a test entry on M2 + Reenable the RA. + checks that entry was deleted on M2 (with the modified RDN) + checks that test entry was replicated on M1 (replication M2->M1 not broken by modrdn) + ''' + + _header(topology, "test_ticket47787_2") + _bind_manager(topology.master1) + _bind_manager(topology.master2) + + #entry to test the replication is still working + name = "%s%d" % (NEW_ACCOUNT, MAX_ACCOUNTS -1) + test_rdn = "cn=%s" % (name) + testentry_dn = "%s,%s" % (test_rdn, STAGING_DN) + + name = "%s%d" % (NEW_ACCOUNT, MAX_ACCOUNTS - 2) + test2_rdn = "cn=%s" % (name) + testentry2_dn = "%s,%s" % (test2_rdn, STAGING_DN) + + # value of updates to test the replication both ways + attr = 'description' + value = 'test_ticket47787_2' + + # entry for the modrdn + name = "%s%d" % (NEW_ACCOUNT, 1) + rdn = "cn=%s" % (name) + entry_dn = "%s,%s" % (rdn, STAGING_DN) + + # created on M1, wait the entry exists on M2 + _check_entry_exists(topology.master2, entry_dn) + _check_entry_exists(topology.master2, testentry_dn) + + _pause_RAs(topology) + + # Delete 'entry_dn' on M1. + # dummy update is only have a first CSN before the DEL + # else the DEL will be in min_csn RUV and make diagnostic a bit more complex + _mod_entry(topology.master1, testentry2_dn, attr, 'dummy') + _delete_entry(topology.master1, entry_dn, name) + _mod_entry(topology.master1, testentry2_dn, attr, value) + + time.sleep(1) # important to have MOD.csn != DEL.csn + + # MOD 'entry_dn' on M1. + # dummy update is only have a first CSN before the MOD entry_dn + # else the DEL will be in min_csn RUV and make diagnostic a bit more complex + _mod_entry(topology.master2, testentry_dn, attr, 'dummy') + _mod_entry(topology.master2, entry_dn, attr, value) + _mod_entry(topology.master2, testentry_dn, attr, value) + + + _resume_RAs(topology) + + topology.master1.log.info("\n\n######################### Check DEL replicated on M2 ######################\n") + loop = 0 + while loop <= 10: + ent = _find_tombstone(topology.master2, SUFFIX, 'sn', name) + if ent: + break + time.sleep(1) + loop += 1 + assert loop <= 10 + assert ent + + # the following checks are not necessary + # as this bug is only for failing replicated MOD (entry_dn) on M1 + #_check_mod_received(topology.master1, SUFFIX, "(%s)" % (test_rdn), attr, value) + #_check_mod_received(topology.master2, SUFFIX, "(%s)" % (test2_rdn), attr, value) + # + #_check_replication(topology, testentry_dn) + + _status_entry_both_server(topology, name=name, desc="DEL M1 - MOD M2", debug=DEBUG_FLAG) + + topology.master1.log.info("\n\n######################### Check MOD replicated on M1 ######################\n") + loop = 0 + while loop <= 10: + ent = _find_tombstone(topology.master1, SUFFIX, 'sn', name) + if ent: + break + time.sleep(1) + loop += 1 + assert loop <= 10 + assert ent + assert ent.hasAttr(attr) + assert ent.getValue(attr) == value + + +def test_ticket47787_final(topology): + topology.master1.stop(timeout=10) + topology.master2.stop(timeout=10) + +def run_isolated(): + ''' + run_isolated is used to run these test cases independently of a test scheduler (xunit, py.test..) + To run isolated without py.test, you need to + - edit this file and comment '@pytest.fixture' line before 'topology' function. + - set the installation prefix + - run this program + ''' + global installation1_prefix + global installation2_prefix + installation1_prefix = None + installation2_prefix = None + + topo = topology(True) + topo.master1.log.info("\n\n######################### Ticket 47787 ######################\n") + test_ticket47787_init(topo) + + test_ticket47787_2(topo) + + test_ticket47787_final(topo) + + + + +if __name__ == '__main__': + run_isolated() + diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modify.c b/ldap/servers/slapd/back-ldbm/ldbm_modify.c index 3fd5683..d2dc983 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modify.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modify.c @@ -568,7 +568,7 @@ ldbm_back_modify( Slapi_PBlock *pb ) if ( !is_fixup_operation ) { - if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) ) { + if (!repl_op && slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) ) { ldap_result_code = LDAP_UNWILLING_TO_PERFORM; ldap_result_message = "Operation not allowed on tombstone entry."; slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modify",
1
0
0
0
Branch '389-ds-base-1.2.11' - ldap/servers
by Noriko Hosoi
12 May '14
12 May '14
ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c | 92 ++++++++++++++++++--------- 1 file changed, 64 insertions(+), 28 deletions(-) New commits: commit ca407f8a684e58067440f57049794c47255cf716 Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Fri May 9 10:10:49 2014 -0700 Ticket #47764 - Problem with deletion while replicated Bug description: When checking a child entry on a node, it only checked the first position, which was normally "deleted" if there were no more children. But in some cases, a tombstoned child was placed there. If it occurred, even though there were no live child any more, _entryrdn_delete_key returned "has children" and the delete operation failed. Fix description: This patch checks all the children of the to-be- deleted node and if there is no child or all of them are tombstones, it goes to the next process. Also, the fixed a typo reported by chatfield (Thank you!!)
https://fedorahosted.org/389/ticket/47764
Reviewed by chatfield and mreynolds(a)redhat.com (Thank you both!!) (cherry picked from commit 832253ea96b83e7ebc16fe507d77090e87aed1c2) (cherry picked from commit 86b34ca0a002715a263a56b1e8c870dd3035bce4) (cherry picked from commit efa23ced2a6e3de3389d9b801329066f511bc38c) diff --git a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c index 6426fb7..38b407d 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c @@ -2795,6 +2795,8 @@ _entryrdn_delete_key(backend *be, int issuffix = 0; Slapi_RDN *tmpsrdn = NULL; int db_retry = 0; + int done = 0; + char buffer[RDN_BULK_FETCH_BUFFER_SIZE]; slapi_log_error(SLAPI_LOG_TRACE, ENTRYRDN_TAG, "--> _entryrdn_delete_key\n"); @@ -2828,45 +2830,79 @@ _entryrdn_delete_key(backend *be, /* check if the target element has a child or not */ keybuf = slapi_ch_smprintf("%c%u", RDN_INDEX_CHILD, id); key.data = (void *)keybuf; - key.size = key.ulen = strlen(nrdn) + 1; + key.size = key.ulen = strlen(keybuf) + 1; key.flags = DB_DBT_USERMEM; + /* Setting the bulk fetch buffer */ memset(&data, 0, sizeof(data)); - data.flags = DB_DBT_MALLOC; + data.ulen = sizeof(buffer); + data.size = sizeof(buffer); + data.data = buffer; + data.flags = DB_DBT_USERMEM; - for (db_retry = 0; db_retry < RETRY_TIMES; db_retry++) { - rc = cursor->c_get(cursor, &key, &data, DB_SET); - if (rc) { + done = 0; + while (!done) { + rc = cursor->c_get(cursor, &key, &data, DB_SET|DB_MULTIPLE); + if (DB_LOCK_DEADLOCK == rc) { + slapi_log_error(ENTRYRDN_LOGLEVEL(rc), ENTRYRDN_TAG, + "_entryrdn_delete_key: cursor get deadlock\n"); +#ifdef FIX_TXN_DEADLOCKS +#error if txn != NULL, have to retry the entire transaction +#endif + /* try again */ + continue; + } else if (DB_NOTFOUND == rc) { + /* no children; ok */ + done = 1; + continue; + } else if (rc) { + _entryrdn_cursor_print_error("_entryrdn_delete_key", + key.data, data.size, data.ulen, rc); + goto bail; + } + + do { + rdn_elem *childelem = NULL; + DBT dataret; + void *ptr; + DB_MULTIPLE_INIT(ptr, &data); + do { + memset(&dataret, 0, sizeof(dataret)); + DB_MULTIPLE_NEXT(ptr, &data, dataret.data, dataret.size); + if (NULL == dataret.data || NULL == ptr) { + break; + } + childelem = (rdn_elem *)dataret.data; + if (!slapi_is_special_rdn(childelem->rdn_elem_nrdn_rdn, RDN_IS_TOMBSTONE)) { + /* there's at least one live child */ + slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, + "_entryrdn_delete_key: Failed to remove %s; " + "has a child %s\n", nrdn, + (char *)childelem->rdn_elem_nrdn_rdn); + rc = -1; + goto bail; + } + } while (NULL != dataret.data && NULL != ptr); +retry_get: + rc = cursor->c_get(cursor, &key, &data, DB_NEXT_DUP|DB_MULTIPLE); if (DB_LOCK_DEADLOCK == rc) { slapi_log_error(ENTRYRDN_LOGLEVEL(rc), ENTRYRDN_TAG, - "_entryrdn_delete_key: cursor get deadlock\n"); + "_entryrdn_delete_key: retry cursor get deadlock\n"); +#ifdef FIX_TXN_DEADLOCKS +#error if txn != NULL, have to retry the entire transaction +#endif /* try again */ - if (db_txn) { - goto bail; /* have to abort/retry the entire transaction */ - } else { - ENTRYRDN_DELAY; /* sleep for a bit then retry immediately */ - } - } else if (DB_NOTFOUND != rc) { + goto retry_get; + } else if (DB_NOTFOUND == rc) { + rc = 0; + done = 1; + break; + } else if (rc) { _entryrdn_cursor_print_error("_entryrdn_delete_key", key.data, data.size, data.ulen, rc); goto bail; - } else { - break; /* DB_NOTFOUND - ok */ } - } else { - slapi_ch_free(&data.data); - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, - "_entryrdn_delete_key: Failed to remove %s; " - "has children\n", nrdn); - rc = -1; - goto bail; - } - } - if (RETRY_TIMES == db_retry) { - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, - "_entryrdn_delete_key: failed after [%d] iterations\n", db_retry); - rc = DB_LOCK_DEADLOCK; - goto bail; + } while (0 == rc); } workid = id;
1
0
0
0
Branch '389-ds-base-1.3.1' - ldap/servers
by Noriko Hosoi
12 May '14
12 May '14
ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c | 92 ++++++++++++++++++--------- 1 file changed, 64 insertions(+), 28 deletions(-) New commits: commit efa23ced2a6e3de3389d9b801329066f511bc38c Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Fri May 9 10:10:49 2014 -0700 Ticket #47764 - Problem with deletion while replicated Bug description: When checking a child entry on a node, it only checked the first position, which was normally "deleted" if there were no more children. But in some cases, a tombstoned child was placed there. If it occurred, even though there were no live child any more, _entryrdn_delete_key returned "has children" and the delete operation failed. Fix description: This patch checks all the children of the to-be- deleted node and if there is no child or all of them are tombstones, it goes to the next process. Also, the fixed a typo reported by chatfield (Thank you!!)
https://fedorahosted.org/389/ticket/47764
Reviewed by chatfield and mreynolds(a)redhat.com (Thank you both!!) (cherry picked from commit 832253ea96b83e7ebc16fe507d77090e87aed1c2) (cherry picked from commit 86b34ca0a002715a263a56b1e8c870dd3035bce4) diff --git a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c index 34b7f70..292f4fd 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c @@ -2802,6 +2802,8 @@ _entryrdn_delete_key(backend *be, int issuffix = 0; Slapi_RDN *tmpsrdn = NULL; int db_retry = 0; + int done = 0; + char buffer[RDN_BULK_FETCH_BUFFER_SIZE]; slapi_log_error(SLAPI_LOG_TRACE, ENTRYRDN_TAG, "--> _entryrdn_delete_key\n"); @@ -2835,45 +2837,79 @@ _entryrdn_delete_key(backend *be, /* check if the target element has a child or not */ keybuf = slapi_ch_smprintf("%c%u", RDN_INDEX_CHILD, id); key.data = (void *)keybuf; - key.size = key.ulen = strlen(nrdn) + 1; + key.size = key.ulen = strlen(keybuf) + 1; key.flags = DB_DBT_USERMEM; + /* Setting the bulk fetch buffer */ memset(&data, 0, sizeof(data)); - data.flags = DB_DBT_MALLOC; + data.ulen = sizeof(buffer); + data.size = sizeof(buffer); + data.data = buffer; + data.flags = DB_DBT_USERMEM; - for (db_retry = 0; db_retry < RETRY_TIMES; db_retry++) { - rc = cursor->c_get(cursor, &key, &data, DB_SET); - if (rc) { + done = 0; + while (!done) { + rc = cursor->c_get(cursor, &key, &data, DB_SET|DB_MULTIPLE); + if (DB_LOCK_DEADLOCK == rc) { + slapi_log_error(ENTRYRDN_LOGLEVEL(rc), ENTRYRDN_TAG, + "_entryrdn_delete_key: cursor get deadlock\n"); +#ifdef FIX_TXN_DEADLOCKS +#error if txn != NULL, have to retry the entire transaction +#endif + /* try again */ + continue; + } else if (DB_NOTFOUND == rc) { + /* no children; ok */ + done = 1; + continue; + } else if (rc) { + _entryrdn_cursor_print_error("_entryrdn_delete_key", + key.data, data.size, data.ulen, rc); + goto bail; + } + + do { + rdn_elem *childelem = NULL; + DBT dataret; + void *ptr; + DB_MULTIPLE_INIT(ptr, &data); + do { + memset(&dataret, 0, sizeof(dataret)); + DB_MULTIPLE_NEXT(ptr, &data, dataret.data, dataret.size); + if (NULL == dataret.data || NULL == ptr) { + break; + } + childelem = (rdn_elem *)dataret.data; + if (!slapi_is_special_rdn(childelem->rdn_elem_nrdn_rdn, RDN_IS_TOMBSTONE)) { + /* there's at least one live child */ + slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, + "_entryrdn_delete_key: Failed to remove %s; " + "has a child %s\n", nrdn, + (char *)childelem->rdn_elem_nrdn_rdn); + rc = -1; + goto bail; + } + } while (NULL != dataret.data && NULL != ptr); +retry_get: + rc = cursor->c_get(cursor, &key, &data, DB_NEXT_DUP|DB_MULTIPLE); if (DB_LOCK_DEADLOCK == rc) { slapi_log_error(ENTRYRDN_LOGLEVEL(rc), ENTRYRDN_TAG, - "_entryrdn_delete_key: cursor get deadlock\n"); + "_entryrdn_delete_key: retry cursor get deadlock\n"); +#ifdef FIX_TXN_DEADLOCKS +#error if txn != NULL, have to retry the entire transaction +#endif /* try again */ - if (db_txn) { - goto bail; /* have to abort/retry the entire transaction */ - } else { - ENTRYRDN_DELAY; /* sleep for a bit then retry immediately */ - } - } else if (DB_NOTFOUND != rc) { + goto retry_get; + } else if (DB_NOTFOUND == rc) { + rc = 0; + done = 1; + break; + } else if (rc) { _entryrdn_cursor_print_error("_entryrdn_delete_key", key.data, data.size, data.ulen, rc); goto bail; - } else { - break; /* DB_NOTFOUND - ok */ } - } else { - slapi_ch_free(&data.data); - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, - "_entryrdn_delete_key: Failed to remove %s; " - "has children\n", nrdn); - rc = -1; - goto bail; - } - } - if (RETRY_TIMES == db_retry) { - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, - "_entryrdn_delete_key: failed after [%d] iterations\n", db_retry); - rc = DB_LOCK_DEADLOCK; - goto bail; + } while (0 == rc); } workid = id;
1
0
0
0
Branch '389-ds-base-1.3.2' - ldap/servers
by Noriko Hosoi
12 May '14
12 May '14
ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c | 92 ++++++++++++++++++--------- 1 file changed, 64 insertions(+), 28 deletions(-) New commits: commit 86b34ca0a002715a263a56b1e8c870dd3035bce4 Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Fri May 9 10:10:49 2014 -0700 Ticket #47764 - Problem with deletion while replicated Bug description: When checking a child entry on a node, it only checked the first position, which was normally "deleted" if there were no more children. But in some cases, a tombstoned child was placed there. If it occurred, even though there were no live child any more, _entryrdn_delete_key returned "has children" and the delete operation failed. Fix description: This patch checks all the children of the to-be- deleted node and if there is no child or all of them are tombstones, it goes to the next process. Also, the fixed a typo reported by chatfield (Thank you!!)
https://fedorahosted.org/389/ticket/47764
Reviewed by chatfield and mreynolds(a)redhat.com (Thank you both!!) (cherry picked from commit 832253ea96b83e7ebc16fe507d77090e87aed1c2) diff --git a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c index 34b7f70..292f4fd 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c @@ -2802,6 +2802,8 @@ _entryrdn_delete_key(backend *be, int issuffix = 0; Slapi_RDN *tmpsrdn = NULL; int db_retry = 0; + int done = 0; + char buffer[RDN_BULK_FETCH_BUFFER_SIZE]; slapi_log_error(SLAPI_LOG_TRACE, ENTRYRDN_TAG, "--> _entryrdn_delete_key\n"); @@ -2835,45 +2837,79 @@ _entryrdn_delete_key(backend *be, /* check if the target element has a child or not */ keybuf = slapi_ch_smprintf("%c%u", RDN_INDEX_CHILD, id); key.data = (void *)keybuf; - key.size = key.ulen = strlen(nrdn) + 1; + key.size = key.ulen = strlen(keybuf) + 1; key.flags = DB_DBT_USERMEM; + /* Setting the bulk fetch buffer */ memset(&data, 0, sizeof(data)); - data.flags = DB_DBT_MALLOC; + data.ulen = sizeof(buffer); + data.size = sizeof(buffer); + data.data = buffer; + data.flags = DB_DBT_USERMEM; - for (db_retry = 0; db_retry < RETRY_TIMES; db_retry++) { - rc = cursor->c_get(cursor, &key, &data, DB_SET); - if (rc) { + done = 0; + while (!done) { + rc = cursor->c_get(cursor, &key, &data, DB_SET|DB_MULTIPLE); + if (DB_LOCK_DEADLOCK == rc) { + slapi_log_error(ENTRYRDN_LOGLEVEL(rc), ENTRYRDN_TAG, + "_entryrdn_delete_key: cursor get deadlock\n"); +#ifdef FIX_TXN_DEADLOCKS +#error if txn != NULL, have to retry the entire transaction +#endif + /* try again */ + continue; + } else if (DB_NOTFOUND == rc) { + /* no children; ok */ + done = 1; + continue; + } else if (rc) { + _entryrdn_cursor_print_error("_entryrdn_delete_key", + key.data, data.size, data.ulen, rc); + goto bail; + } + + do { + rdn_elem *childelem = NULL; + DBT dataret; + void *ptr; + DB_MULTIPLE_INIT(ptr, &data); + do { + memset(&dataret, 0, sizeof(dataret)); + DB_MULTIPLE_NEXT(ptr, &data, dataret.data, dataret.size); + if (NULL == dataret.data || NULL == ptr) { + break; + } + childelem = (rdn_elem *)dataret.data; + if (!slapi_is_special_rdn(childelem->rdn_elem_nrdn_rdn, RDN_IS_TOMBSTONE)) { + /* there's at least one live child */ + slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, + "_entryrdn_delete_key: Failed to remove %s; " + "has a child %s\n", nrdn, + (char *)childelem->rdn_elem_nrdn_rdn); + rc = -1; + goto bail; + } + } while (NULL != dataret.data && NULL != ptr); +retry_get: + rc = cursor->c_get(cursor, &key, &data, DB_NEXT_DUP|DB_MULTIPLE); if (DB_LOCK_DEADLOCK == rc) { slapi_log_error(ENTRYRDN_LOGLEVEL(rc), ENTRYRDN_TAG, - "_entryrdn_delete_key: cursor get deadlock\n"); + "_entryrdn_delete_key: retry cursor get deadlock\n"); +#ifdef FIX_TXN_DEADLOCKS +#error if txn != NULL, have to retry the entire transaction +#endif /* try again */ - if (db_txn) { - goto bail; /* have to abort/retry the entire transaction */ - } else { - ENTRYRDN_DELAY; /* sleep for a bit then retry immediately */ - } - } else if (DB_NOTFOUND != rc) { + goto retry_get; + } else if (DB_NOTFOUND == rc) { + rc = 0; + done = 1; + break; + } else if (rc) { _entryrdn_cursor_print_error("_entryrdn_delete_key", key.data, data.size, data.ulen, rc); goto bail; - } else { - break; /* DB_NOTFOUND - ok */ } - } else { - slapi_ch_free(&data.data); - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, - "_entryrdn_delete_key: Failed to remove %s; " - "has children\n", nrdn); - rc = -1; - goto bail; - } - } - if (RETRY_TIMES == db_retry) { - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, - "_entryrdn_delete_key: failed after [%d] iterations\n", db_retry); - rc = DB_LOCK_DEADLOCK; - goto bail; + } while (0 == rc); } workid = id;
1
0
0
0
ldap/servers
by Noriko Hosoi
12 May '14
12 May '14
ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c | 92 ++++++++++++++++++--------- 1 file changed, 64 insertions(+), 28 deletions(-) New commits: commit 832253ea96b83e7ebc16fe507d77090e87aed1c2 Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Fri May 9 10:10:49 2014 -0700 Ticket #47764 - Problem with deletion while replicated Bug description: When checking a child entry on a node, it only checked the first position, which was normally "deleted" if there were no more children. But in some cases, a tombstoned child was placed there. If it occurred, even though there were no live child any more, _entryrdn_delete_key returned "has children" and the delete operation failed. Fix description: This patch checks all the children of the to-be- deleted node and if there is no child or all of them are tombstones, it goes to the next process. Also, the fixed a typo reported by chatfield (Thank you!!)
https://fedorahosted.org/389/ticket/47764
Reviewed by chatfield and mreynolds(a)redhat.com (Thank you both!!) diff --git a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c index 8ac4fa5..46d30eb 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c @@ -2811,6 +2811,8 @@ _entryrdn_delete_key(backend *be, int issuffix = 0; Slapi_RDN *tmpsrdn = NULL; int db_retry = 0; + int done = 0; + char buffer[RDN_BULK_FETCH_BUFFER_SIZE]; slapi_log_error(SLAPI_LOG_TRACE, ENTRYRDN_TAG, "--> _entryrdn_delete_key\n"); @@ -2844,45 +2846,79 @@ _entryrdn_delete_key(backend *be, /* check if the target element has a child or not */ keybuf = slapi_ch_smprintf("%c%u", RDN_INDEX_CHILD, id); key.data = (void *)keybuf; - key.size = key.ulen = strlen(nrdn) + 1; + key.size = key.ulen = strlen(keybuf) + 1; key.flags = DB_DBT_USERMEM; + /* Setting the bulk fetch buffer */ memset(&data, 0, sizeof(data)); - data.flags = DB_DBT_MALLOC; + data.ulen = sizeof(buffer); + data.size = sizeof(buffer); + data.data = buffer; + data.flags = DB_DBT_USERMEM; - for (db_retry = 0; db_retry < RETRY_TIMES; db_retry++) { - rc = cursor->c_get(cursor, &key, &data, DB_SET); - if (rc) { + done = 0; + while (!done) { + rc = cursor->c_get(cursor, &key, &data, DB_SET|DB_MULTIPLE); + if (DB_LOCK_DEADLOCK == rc) { + slapi_log_error(ENTRYRDN_LOGLEVEL(rc), ENTRYRDN_TAG, + "_entryrdn_delete_key: cursor get deadlock\n"); +#ifdef FIX_TXN_DEADLOCKS +#error if txn != NULL, have to retry the entire transaction +#endif + /* try again */ + continue; + } else if (DB_NOTFOUND == rc) { + /* no children; ok */ + done = 1; + continue; + } else if (rc) { + _entryrdn_cursor_print_error("_entryrdn_delete_key", + key.data, data.size, data.ulen, rc); + goto bail; + } + + do { + rdn_elem *childelem = NULL; + DBT dataret; + void *ptr; + DB_MULTIPLE_INIT(ptr, &data); + do { + memset(&dataret, 0, sizeof(dataret)); + DB_MULTIPLE_NEXT(ptr, &data, dataret.data, dataret.size); + if (NULL == dataret.data || NULL == ptr) { + break; + } + childelem = (rdn_elem *)dataret.data; + if (!slapi_is_special_rdn(childelem->rdn_elem_nrdn_rdn, RDN_IS_TOMBSTONE)) { + /* there's at least one live child */ + slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, + "_entryrdn_delete_key: Failed to remove %s; " + "has a child %s\n", nrdn, + (char *)childelem->rdn_elem_nrdn_rdn); + rc = -1; + goto bail; + } + } while (NULL != dataret.data && NULL != ptr); +retry_get: + rc = cursor->c_get(cursor, &key, &data, DB_NEXT_DUP|DB_MULTIPLE); if (DB_LOCK_DEADLOCK == rc) { slapi_log_error(ENTRYRDN_LOGLEVEL(rc), ENTRYRDN_TAG, - "_entryrdn_delete_key: cursor get deadlock\n"); + "_entryrdn_delete_key: retry cursor get deadlock\n"); +#ifdef FIX_TXN_DEADLOCKS +#error if txn != NULL, have to retry the entire transaction +#endif /* try again */ - if (db_txn) { - goto bail; /* have to abort/retry the entire transaction */ - } else { - ENTRYRDN_DELAY; /* sleep for a bit then retry immediately */ - } - } else if (DB_NOTFOUND != rc) { + goto retry_get; + } else if (DB_NOTFOUND == rc) { + rc = 0; + done = 1; + break; + } else if (rc) { _entryrdn_cursor_print_error("_entryrdn_delete_key", key.data, data.size, data.ulen, rc); goto bail; - } else { - break; /* DB_NOTFOUND - ok */ } - } else { - slapi_ch_free(&data.data); - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, - "_entryrdn_delete_key: Failed to remove %s; " - "has children\n", nrdn); - rc = -1; - goto bail; - } - } - if (RETRY_TIMES == db_retry) { - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, - "_entryrdn_delete_key: failed after [%d] iterations\n", db_retry); - rc = DB_LOCK_DEADLOCK; - goto bail; + } while (0 == rc); } workid = id;
1
0
0
0
Branch '389-ds-base-1.3.1' - ldap/servers
by Noriko Hosoi
12 May '14
12 May '14
ldap/servers/plugins/replication/repl5.h | 1 ldap/servers/plugins/replication/urp.c | 94 +++--- ldap/servers/plugins/replication/urp.h | 5 ldap/servers/plugins/replication/urp_glue.c | 32 +- ldap/servers/plugins/replication/urp_tombstone.c | 56 +++ ldap/servers/slapd/add.c | 15 - ldap/servers/slapd/back-ldbm/back-ldbm.h | 3 ldap/servers/slapd/back-ldbm/backentry.c | 1 ldap/servers/slapd/back-ldbm/cache.c | 103 ++++-- ldap/servers/slapd/back-ldbm/ldbm_add.c | 338 ++++++++++++++-------- ldap/servers/slapd/back-ldbm/ldbm_delete.c | 343 +++++++++++++++-------- ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c | 12 ldap/servers/slapd/back-ldbm/ldbm_modify.c | 54 ++- ldap/servers/slapd/back-ldbm/ldbm_modrdn.c | 318 +++++++++++++++------ ldap/servers/slapd/back-ldbm/ldbm_search.c | 2 ldap/servers/slapd/back-ldbm/ldif2ldbm.c | 2 ldap/servers/slapd/back-ldbm/parents.c | 31 +- ldap/servers/slapd/back-ldbm/proto-back-ldbm.h | 5 ldap/servers/slapd/entry.c | 38 ++ ldap/servers/slapd/rdn.c | 108 ++++++- ldap/servers/slapd/slapi-plugin.h | 45 +++ ldap/servers/slapd/slapi-private.h | 7 ldap/servers/slapd/util.c | 67 ++++ 23 files changed, 1196 insertions(+), 484 deletions(-) New commits: commit 68b52b72a4a682e5778938a3c780f4d5ff08ca0f Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Fri May 9 09:59:41 2014 -0700 Ticket #47750 - Creating a glue fails if one above level is a conflict or missing; Ticket #47696 - Large Searches Hang - Possibly entryrdn related Bug Descriptions: There were multiple issues to break the parent-child relationship when a resurrect occurred. When resurrecting a tombstone, there is a possibility that an entry having the same name was already added. In the case, the code consdered the entry is already there, which confused the numsubordinates and tombstone-num- subordinates count. If it is detected, it creates a conflict glue entry instead of a pure glue entry. The 2 numsubordinates counts were also not correctly updated in the resurrection case. When the entryrdn was introduced, Slapi_RDN was held in the entry, which was not correctly updated in the resurrect case. Fix Descriptions: [replication/urp.c] urp_fixup_rename_entry sets the parent uniqueid if it is a RESURRECT mode (called from tombstone_to_glue) as being done in urp_fixup_add_entry, which is necessary to retrieve the correct parent in ldbm_back_modrdn. [replication/urp_glue.c] In create_glue_entry, . there was a possibility to pass a tombstone DN (nsuniqueid=...,<DN>) is used to create a glue entry. If found, the part is removed before passing it to create a glue entry. . there is a possibility to create a glue entry fails due to the parent entry not found. This occurs when the parent entry has turned to be a conflict entry. In the case, instead of retrying the glue creation, it logs an error and returns. [replication/urp_tombstone.c] In tombstone_to_glue, . turning a tombstone to an ordinary glue entry could fail with the entry already exists. (Note: ldbm_back_add guarantees that the found entry has a different nsuniqueid from this tombstone has.) This situation occurs an entry having the same dn could have been added/replicated by this moment. And the entry cannot be used for this tombstone to resurrect since the subordinates/tombstonesubordinates relationships get corrupted. To maintain the relationship properly, the tombstone is resurrected as a conflict, glue entry using urp_fixup_rename_entry. If it still returns LDAP_ALREADY_EXISTS, it is assured the expected glue entry exists since the same nsuniqueid is shared. [slapd/back-ldbm/ldbm_add.c] In ldbm_back_add, . in is_resurect_operation case, a SLAPI_ADD_EXISTING_DN_ENTRY entry and a SLAPI_ADD_EXISTING_UNIQUEID_ENTRY entry are identical, it returns SUCCESS instead of LDAP_ALREADY_EXISTS. . in is_resurect_operation case, the DN is modified from the tombstone DN to the ordinary DN. The DN is stored in e_sdn as well as in e_srdn in the Slapi_Entry. The e_srdn part was not updated. This patch fixes it and e_sdn and e_srdn are in sync. . parent_update_on_childchange adjusts the count of numsubordinates and tombstonenumsubordinates. It only took PARENTUPDATE_ADD|_DELETE operations. This patch adds PARENTUPDATE_RESURECT operation to increment numsubordinates and decrement tombstonenumsubordinates at the same time. . in is_resurect_operation case, a tombstone to be replaced with a glue entry needs to be deleted from the entryrdn index, which was missing. [slapd/back-ldbm/ldbm_delete.c] . Check the target DN if it starts with "nsuniqueid=...,", if it is, log an error and returns. [slapd/back-ldbm/ldbm_modrdn.c] . If it is is_resurect_operation case, it allows to rename a tombstone entry. This is used to support to change a tombstone entry to a conflict glue entry in tombstone_to_glue. . ldbm_back_modify checks if the renamed entry has children or not and if there are, they are updated to point to the renamed entry. The check function slapi_entry_has_children was just checking the numsubordinates attribute type. If a positive value is returned, it took care all the children including the tombstones, but if the renamed entry had tombstoned children only, it considered there were no children and the tombstoned children were not updated to point the renamed entry. This patch fixes it. [slapd/back-ldbm/ldbm_entryrdn.c] . Added a flag to entryrdn_rename_subtree and entryrdn_get_subordinates to treat the given DN a tombstoned DN if it is. In addition, helper APIs such as slapi_rdn_is_multivalued, slapi_rdn_is_ conflict, and slapi_is_special_rdn are added. Ported from 389-ds-base-1.2.11.
https://fedorahosted.org/389/ticket/47750
Reviewed by mreynolds(a)redhat.com (Thank you, Mark!!) (cherry picked from commit 160cb3f686e433c01532d28770b2977ec957e73e) (cherry picked from commit 3f933fe1b930feae3f76a729a7314a621c8119b9) diff --git a/ldap/servers/plugins/replication/repl5.h b/ldap/servers/plugins/replication/repl5.h index 4809441..29f00d4 100644 --- a/ldap/servers/plugins/replication/repl5.h +++ b/ldap/servers/plugins/replication/repl5.h @@ -60,6 +60,7 @@ #include "llist.h" #include "repl5_ruv.h" #include "cl4.h" +#include "plstr.h" #define START_UPDATE_DELAY 2 /* 2 second */ #define REPLICA_TYPE_WINDOWS 1 diff --git a/ldap/servers/plugins/replication/urp.c b/ldap/servers/plugins/replication/urp.c index b6fe2ff..d867274 100644 --- a/ldap/servers/plugins/replication/urp.c +++ b/ldap/servers/plugins/replication/urp.c @@ -58,7 +58,6 @@ static int urp_naming_conflict_removal (Slapi_PBlock *pb, char *sessionid, CSN * static int mod_namingconflict_attr (const char *uniqueid, const Slapi_DN *entrysdn, const Slapi_DN *conflictsdn, CSN *opcsn, const char *optype); static int del_replconflict_attr (const Slapi_Entry *entry, CSN *opcsn, int opflags); static char *get_dn_plus_uniqueid(char *sessionid,const Slapi_DN *oldsdn,const char *uniqueid); -static char *get_rdn_plus_uniqueid(char *sessionid,const char *olddn,const char *uniqueid); static int is_suffix_entry (Slapi_PBlock *pb, Slapi_Entry *entry, Slapi_DN **parenddn); /* @@ -388,6 +387,9 @@ urp_modrdn_operation( Slapi_PBlock *pb ) slapi_entry_get_sdn (target_entry), "renameTombstone", opcsn); */ op_result = LDAP_NO_SUCH_OBJECT; + slapi_log_error(SLAPI_LOG_REPL, sessionid, + "urp_modrdn: target_entry %s is a tombstone; returning LDAP_NO_SUCH_OBJECT.\n", + slapi_entry_get_dn((Slapi_Entry *)target_entry)); slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); if (op_result == 0) { @@ -473,10 +475,9 @@ urp_modrdn_operation( Slapi_PBlock *pb ) mod_namingconflict_attr (op_uniqueid, target_sdn, existing_sdn, opcsn, "MODRDN"); slapi_pblock_set(pb, SLAPI_MODRDN_NEWRDN, newrdn_with_uniqueid); slapi_log_error(slapi_log_urp, sessionid, - "urp_modrdn: Naming conflict MODRDN. Rename target entry to %s\n", - newrdn_with_uniqueid ); - - rc= slapi_setbit_int(rc,SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); + "urp_modrdn: Naming conflict MODRDN. Rename target entry from %s to %s\n", + newrdn, newrdn_with_uniqueid ); + rc= slapi_setbit_int(rc, SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); PROFILE_POINT; /* ModRDN Conflict; Entry with Target DN Exists; Rename Operation Entry */ goto bailout; } @@ -628,12 +629,15 @@ urp_delete_operation( Slapi_PBlock *pb ) op_result= LDAP_NO_SUCH_OBJECT; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc = SLAPI_PLUGIN_FAILURE; /* Don't apply the Delete */ + slapi_log_error(slapi_log_urp, sessionid, + "Entry %s does not exist; returning NO_SUCH_OBJECT.\n", + slapi_entry_get_dn((Slapi_Entry *)deleteentry)); PROFILE_POINT; /* Delete Operation; Entry not exist. */ } else if(is_tombstone_entry(deleteentry)) { /* The entry is already a Tombstone, ignore this delete. */ - op_result= LDAP_SUCCESS; + op_result= LDAP_ALREADY_EXISTS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc = SLAPI_PLUGIN_NOOP; /* Don't apply the Delete */ slapi_log_error(slapi_log_urp, sessionid, @@ -811,7 +815,7 @@ urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *pa } int -urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflags) +urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags) { Slapi_PBlock *newpb; Slapi_Operation *op; @@ -839,7 +843,12 @@ urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflag opcsn = (CSN *)entry_get_dncsn (entry); slapi_pblock_get (newpb, SLAPI_OPERATION, &op); operation_set_csn (op, opcsn); - + if (parentuniqueid) + { + struct slapi_operation_parameters *op_params; + slapi_pblock_get( newpb, SLAPI_OPERATION_PARAMETERS, &op_params ); + op_params->p.p_modrdn.modrdn_newsuperior_address.uniqueid = (char*)parentuniqueid; + } slapi_modrdn_internal_pb(newpb); slapi_pblock_get(newpb, SLAPI_PLUGIN_INTOP_RESULT, &op_result); @@ -1035,16 +1044,17 @@ urp_annotate_dn (char *sessionid, const Slapi_Entry *entry, CSN *opcsn, const ch basesdn = slapi_entry_get_sdn_const (entry); basedn = slapi_entry_get_dn_const (entry); newrdn = get_rdn_plus_uniqueid ( sessionid, basedn, uniqueid ); - if(newrdn!=NULL) - { + if(newrdn) { mod_namingconflict_attr (uniqueid, basesdn, basesdn, opcsn, optype); - op_result = urp_fixup_rename_entry ( entry, newrdn, 0 ); + slapi_log_error(slapi_log_urp, sessionid, + "urp_annotate_dn: %s --> %s\n", basedn, newrdn); + op_result = urp_fixup_rename_entry ( entry, newrdn, NULL, 0 ); switch(op_result) { case LDAP_SUCCESS: slapi_log_error(slapi_log_urp, sessionid, - "Naming conflict %s. Renamed existing entry to %s\n", - optype, newrdn); + "Naming conflict %s. Renamed existing entry to %s\n", + optype, newrdn); rc = 1; break; case LDAP_NO_SUCH_OBJECT: @@ -1064,16 +1074,21 @@ urp_annotate_dn (char *sessionid, const Slapi_Entry *entry, CSN *opcsn, const ch * for a conflict!! After fix for 558293, this * state can't be reproduced anymore (5-Oct-01) */ - slapi_log_error( SLAPI_LOG_FATAL, sessionid, - "Entry %s exists in cache but not in DB\n", - basedn ); + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Entry %s exists in cache but not in DB\n", + basedn); rc = LDAP_NO_SUCH_OBJECT; break; default: - slapi_log_error( slapi_log_urp, sessionid, - "Failed to annotate %s, err=%d\n", newrdn, op_result); + slapi_log_error(slapi_log_urp, sessionid, + "Failed to annotate %s, err=%d\n", + newrdn, op_result); } slapi_ch_free ( (void**)&newrdn ); + } else { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to create conflict DN from basedn: %s and uniqueid: %s\n", + basedn, uniqueid ); } return rc; } @@ -1218,7 +1233,7 @@ urp_naming_conflict_removal ( Slapi_PBlock *pb, char *sessionid, CSN *opcsn, con * is done after DB lock was released. The backend modrdn * will acquire the DB lock if it sees this flag. */ - op_result = urp_fixup_rename_entry((const Slapi_Entry *)min_naming_conflict_entry, newrdnstr, OP_FLAG_ACTION_INVOKE_FOR_REPLOP); + op_result = urp_fixup_rename_entry((const Slapi_Entry *)min_naming_conflict_entry, newrdnstr, NULL, OP_FLAG_ACTION_INVOKE_FOR_REPLOP); if ( op_result != LDAP_SUCCESS ) { slapi_log_error (slapi_log_urp, sessionid, @@ -1251,15 +1266,19 @@ bailout: static char * get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *uniqueid) { - Slapi_RDN *rdn= slapi_rdn_new(); - char *newdn; + Slapi_RDN *rdn = slapi_rdn_new(); + char *newdn = NULL; + int rc = slapi_rdn_init_all_sdn_ext(rdn, oldsdn, 1); + if (rc) { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to convert %s to RDN\n", slapi_sdn_get_dn(oldsdn)); + goto bail; + } PR_ASSERT(uniqueid!=NULL); /* Check if the RDN already contains the Unique ID */ - slapi_rdn_set_dn(rdn, slapi_sdn_get_dn(oldsdn)); - if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid))) - { + if (slapi_rdn_is_conflict(rdn)) { /* The Unique ID is already in the RDN. * This is a highly improbable collision. * It suggests that a duplicate UUID was generated. @@ -1268,7 +1287,6 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique */ slapi_log_error(SLAPI_LOG_FATAL, sessionid, "Annotated DN %s has naming conflict\n", slapi_sdn_get_dn(oldsdn) ); - newdn= NULL; } else { @@ -1282,21 +1300,27 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique newdn = slapi_ch_smprintf("%s,%s", slapi_rdn_get_rdn(rdn), parentdn); slapi_ch_free_string(&parentdn); } +bail: slapi_rdn_free(&rdn); return newdn; } -static char * +char * get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid) { - char *newrdn; + char *newrdn = NULL; /* Check if the RDN already contains the Unique ID */ - Slapi_DN *sdn= slapi_sdn_new_dn_byval(olddn); - Slapi_RDN *rdn= slapi_rdn_new(); - slapi_sdn_get_rdn(sdn,rdn); + Slapi_DN *sdn = slapi_sdn_new_dn_byval(olddn); + Slapi_RDN *rdn = slapi_rdn_new(); + int rc = slapi_rdn_init_all_sdn_ext(rdn, sdn, 1); + if (rc) { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to convert %s to RDN\n", olddn); + goto bail; + } + PR_ASSERT(uniqueid!=NULL); - if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid))) - { + if (slapi_rdn_is_conflict(rdn)) { /* The Unique ID is already in the RDN. * This is a highly improbable collision. * It suggests that a duplicate UUID was generated. @@ -1304,14 +1328,14 @@ get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid) * require admin intercession */ slapi_log_error(SLAPI_LOG_FATAL, sessionid, - "Annotated DN %s has naming conflict\n", olddn ); - newrdn= NULL; + "Annotated RDN %s has naming conflict\n", olddn); } else { slapi_rdn_add(rdn,SLAPI_ATTR_UNIQUEID,uniqueid); - newrdn= slapi_ch_strdup(slapi_rdn_get_rdn(rdn)); + newrdn = slapi_ch_strdup(slapi_rdn_get_rdn(rdn)); } +bail: slapi_sdn_free(&sdn); slapi_rdn_free(&rdn); return newrdn; diff --git a/ldap/servers/plugins/replication/urp.h b/ldap/servers/plugins/replication/urp.h index 26d7c75..8afd7d1 100644 --- a/ldap/servers/plugins/replication/urp.h +++ b/ldap/servers/plugins/replication/urp.h @@ -55,11 +55,12 @@ int urp_delete_operation( Slapi_PBlock *pb ); int urp_post_delete_operation( Slapi_PBlock *pb ); int urp_modrdn_operation( Slapi_PBlock *pb ); int urp_post_modrdn_operation( Slapi_PBlock *pb ); +char *get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid); /* urp internal ops */ int urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *parentuniqueid, CSN *opcsn, int opflags); int urp_fixup_delete_entry (const char *uniqueid, const char *dn, CSN *opcsn, int opflags); -int urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflags); +int urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags); int urp_fixup_modify_entry (const char *uniqueid, const Slapi_DN *sdn, CSN *opcsn, Slapi_Mods *smods, int opflags); int is_suffix_dn (Slapi_PBlock *pb, const Slapi_DN *dn, Slapi_DN **parenddn); @@ -78,6 +79,6 @@ PRBool get_glue_csn(const Slapi_Entry *entry, const CSN **gluecsn); * urp_tombstone.c */ int is_tombstone_entry(const Slapi_Entry* entry); -int tombstone_to_glue(Slapi_PBlock *pb, const char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn); +int tombstone_to_glue(Slapi_PBlock *pb, char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn); int entry_to_tombstone ( Slapi_PBlock *pb, Slapi_Entry *entry ); PRBool get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn); diff --git a/ldap/servers/plugins/replication/urp_glue.c b/ldap/servers/plugins/replication/urp_glue.c index 6eeb826..199d4de 100644 --- a/ldap/servers/plugins/replication/urp_glue.c +++ b/ldap/servers/plugins/replication/urp_glue.c @@ -153,7 +153,7 @@ static const char *glue_entry = "objectclass: top\n" "objectclass: extensibleObject\n" /* JCMREPL - To avoid schema checking. */ "objectclass: glue\n" - "nsuniqueid: %s\n" + "nsuniqueid: %s\n" /* this uniqueid is set to Slapi_Entry in slapi_str2entry. */ "%s: %s\n"; /* Add why it's been created */ static int @@ -166,15 +166,15 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha Slapi_DN *sdn = NULL; Slapi_RDN *newrdn = slapi_rdn_new_rdn(rdn); char *estr, *rdnstr, *rdntype, *rdnval, *rdnpair; - sdn = slapi_sdn_new_dn_byval(slapi_sdn_get_ndn(superiordn)); + sdn = slapi_sdn_new_ndn_byval(slapi_sdn_get_ndn(superiordn)); slapi_sdn_add_rdn(sdn,rdn); /* must take care of multi-valued rdn: split rdn into different lines introducing * '\n' between each type/value pair. - */ + */ alloc_len = RDNBUFSIZE; rdnstr = slapi_ch_malloc(alloc_len); - rdnpair = rdnstr; + rdnpair = rdnstr; *rdnpair = '\0'; /* so that strlen(rdnstr) may return 0 the first time it's called */ while ((rdnval_index = slapi_rdn_get_next(newrdn, rdnval_index, &rdntype, &rdnval)) != -1) { rdntype_len = strlen(rdntype); @@ -188,7 +188,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha } slapi_ldif_put_type_and_value_with_options(&rdnpair, rdntype, rdnval, rdnval_len, LDIF_OPT_NOWRAP); *rdnpair = '\0'; - } + } estr= slapi_ch_smprintf(glue_entry, slapi_sdn_get_ndn(sdn), rdnstr, uniqueid, ATTR_NSDS5_REPLCONFLICT, reason); slapi_ch_free((void**)&rdnstr); @@ -196,9 +196,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha slapi_ch_free((void**)&newrdn); e = slapi_str2entry( estr, 0 ); PR_ASSERT(e!=NULL); - if ( e!=NULL ) - { - slapi_entry_set_uniqueid (e, slapi_ch_strdup(uniqueid)); + if (e) { op_result = urp_fixup_add_entry (e, NULL, NULL, opcsn, 0); } slapi_ch_free_string(&estr); @@ -232,7 +230,7 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char slapi_pblock_get( pb, SLAPI_BACKEND, &backend ); slapi_sdn_get_backend_parent ( dn, superiordn, backend ); - slapi_sdn_get_rdn ( dn, rdn ); + slapi_rdn_set_dn_ext(rdn, slapi_sdn_get_dn(dn), 1/* skip nsuniqeid=..., in dn */); while(!done) { @@ -246,16 +244,30 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char done= 1; break; case LDAP_ALREADY_EXISTS: + { + struct slapi_operation_parameters *op_params; + /* This is okay. While creating a glue, a real entry was added. */ slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name, "%s: Skipped creating glue entry %s uniqueid=%s reason Entry Already Exists\n", sessionid, dnstr, uniqueid); + op_result = LDAP_SUCCESS; + /* If we could not create a glue having the nsuniqueid, + * we have to abandon it. */ + slapi_pblock_get(pb, SLAPI_OPERATION_PARAMETERS, &op_params); + slapi_ch_free_string(&op_params->p.p_add.parentuniqueid); done= 1; break; + } case LDAP_NO_SUCH_OBJECT: /* The parent is missing */ { /* JCMREPL - Create the parent ... recursion?... but what's the uniqueid? */ - PR_ASSERT(0); /* JCMREPL */ + slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, + "%s: Can't created glue entry %s uniqueid=%s, error %d; " + "Possibly, parent entry is a conflict entry.\n", + sessionid, dnstr, uniqueid, op_result); + done= 1; + break; } default: slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name, diff --git a/ldap/servers/plugins/replication/urp_tombstone.c b/ldap/servers/plugins/replication/urp_tombstone.c index a61f1e5..5a37ca8 100644 --- a/ldap/servers/plugins/replication/urp_tombstone.c +++ b/ldap/servers/plugins/replication/urp_tombstone.c @@ -84,7 +84,7 @@ get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn) static int tombstone_to_glue_resolve_parent ( Slapi_PBlock *pb, - const char *sessionid, + char *sessionid, const Slapi_DN *parentdn, const char *parentuniqueid, CSN *opcsn) @@ -144,7 +144,7 @@ tombstone_to_glue_resolve_parent ( int tombstone_to_glue ( Slapi_PBlock *pb, - const char *sessionid, + char *sessionid, Slapi_Entry *tombstoneentry, const Slapi_DN *tombstonedn, const char *reason, @@ -156,6 +156,7 @@ tombstone_to_glue ( Slapi_Entry *addingentry; const char *addingdn; int op_result; + int rdn_is_conflict = 0; /* JCMREPL * Nothing logged to the 5.0 Change Log @@ -170,18 +171,19 @@ tombstone_to_glue ( * which won't help us identify the correct backend to search. */ is_suffix_dn_ext (pb, tombstonedn, &parentdn, 1 /* is_tombstone */); - parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, - SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */ + parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */ tombstone_to_glue_resolve_parent (pb, sessionid, parentdn, parentuniqueid, opcsn); - slapi_sdn_free(&parentdn); - /* Submit an Add operation to turn the tombstone entry into glue. */ + /* Submit an Add operation to turn the tombstone entry into glue. */ /* * The tombstone is stored with an invalid DN, we must fix this. */ addingentry = slapi_entry_dup(tombstoneentry); addingdn = slapi_sdn_get_dn(tombstonedn); slapi_entry_set_sdn(addingentry, tombstonedn); + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry), slapi_entry_get_dn_const(addingentry)); + rdn_is_conflict = slapi_rdn_is_conflict(slapi_entry_get_srdn(addingentry)); if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) { @@ -189,18 +191,58 @@ tombstone_to_glue ( slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason); } tombstoneuniqueid= slapi_entry_get_uniqueid(tombstoneentry); - op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, parentuniqueid, opcsn, OP_FLAG_RESURECT_ENTRY); + /* + * addingentry and parentuniqueid are consumed in urp_fixup_add_entry, + * regardless of the result. + * Note: addingentry is not really consumed in ldbm_back_add. + * tombstoneentry from DB/entry cache is duplicated and turned to be a glue. + * This addingentry is freed in op_shared_add. + */ + op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, slapi_ch_strdup(parentuniqueid), opcsn, OP_FLAG_RESURECT_ENTRY); + if ((LDAP_ALREADY_EXISTS == op_result) && !rdn_is_conflict) { + /* conflict -- there's already the same named entry added. + * But this to-be-glued entry needs to be added since this is a parent of child entries... + * So, rename this tombstone parententry a conflict, glue entry. + * Instead of "fixup_add", we have to "fixup_rename"... + * */ + char *conflictrdn = get_rdn_plus_uniqueid(sessionid, addingdn, tombstoneuniqueid); + if (conflictrdn) { + addingentry = slapi_entry_dup(tombstoneentry); + if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) { + /* Add the reason of turning it to glue - The backend code will use it*/ + slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason); + } + slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, + "%s: Can't resurrect tombstone to glue reason '%s'. " + "Try with conflict dn %s, error=%d\n", + sessionid, reason, addingdn, op_result); + op_result = urp_fixup_rename_entry(addingentry, (const char *)conflictrdn, parentuniqueid, + OP_FLAG_RESURECT_ENTRY|OP_FLAG_TOMBSTONE_ENTRY); + slapi_ch_free_string(&conflictrdn); + slapi_entry_free(addingentry); + addingentry = NULL; + } + } + slapi_ch_free_string(&parentuniqueid); if (op_result == LDAP_SUCCESS) { slapi_log_error (slapi_log_urp, repl_plugin_name, "%s: Resurrected tombstone %s to glue reason '%s'\n", sessionid, addingdn, reason); } + else if (LDAP_ALREADY_EXISTS == op_result) + { + slapi_log_error(slapi_log_urp, repl_plugin_name, + "%s: No need to turn tombstone %s to glue; it was already resurrected.\n", + sessionid, addingdn); + op_result = LDAP_SUCCESS; + } else { slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, "%s: Can't resurrect tombstone %s to glue reason '%s', error=%d\n", sessionid, addingdn, reason, op_result); } + slapi_sdn_free(&parentdn); return op_result; } diff --git a/ldap/servers/slapd/add.c b/ldap/servers/slapd/add.c index e98f197..fadc56b 100644 --- a/ldap/servers/slapd/add.c +++ b/ldap/servers/slapd/add.c @@ -753,10 +753,17 @@ static void op_shared_add (Slapi_PBlock *pb) slapi_pblock_get(pb, SLAPI_ENTRY_POST_OP, &pse); do_ps_service(pse, NULL, LDAP_CHANGETYPE_ADD, 0); - - /* If be_add succeeded, then e is consumed. We - * set e to NULL to prevent freeing it ourselves. */ - e = NULL; + /* + * If be_add succeeded, then e is consumed except the resurect case. + * If it is resurect, the corresponding tombstone entry is resurected + * and put into the cache. + * Otherwise, we set e to NULL to prevent freeing it ourselves. + */ + if (operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY) && save_e) { + e = save_e; + } else { + e = NULL; + } } else { diff --git a/ldap/servers/slapd/back-ldbm/back-ldbm.h b/ldap/servers/slapd/back-ldbm/back-ldbm.h index d633243..8aa9825 100644 --- a/ldap/servers/slapd/back-ldbm/back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/back-ldbm.h @@ -891,7 +891,8 @@ typedef struct _back_search_result_set /* operation for parent_update_on_childchange */ #define PARENTUPDATE_ADD 0x1 #define PARENTUPDATE_DEL 0x2 -#define PARENTUPDATE_MASK (PARENTUPDATE_ADD|PARENTUPDATE_DEL) +#define PARENTUPDATE_RESURECT 0x4 +#define PARENTUPDATE_MASK (PARENTUPDATE_ADD|PARENTUPDATE_DEL|PARENTUPDATE_RESURECT) #define PARENTUPDATE_CREATE_TOMBSTONE 0x10 #define PARENTUPDATE_DELETE_TOMBSTONE 0x20 diff --git a/ldap/servers/slapd/back-ldbm/backentry.c b/ldap/servers/slapd/back-ldbm/backentry.c index 52468ad..114c2df 100644 --- a/ldap/servers/slapd/back-ldbm/backentry.c +++ b/ldap/servers/slapd/back-ldbm/backentry.c @@ -52,6 +52,7 @@ backentry_free( struct backentry **bep ) return; } ep = *bep; + PR_ASSERT(ep->ep_state & (ENTRY_STATE_DELETED|ENTRY_STATE_NOTINCACHE)); if ( ep->ep_entry != NULL ) { slapi_entry_free( ep->ep_entry ); } diff --git a/ldap/servers/slapd/back-ldbm/cache.c b/ldap/servers/slapd/back-ldbm/cache.c index 7f3f025..0309fcf 100644 --- a/ldap/servers/slapd/back-ldbm/cache.c +++ b/ldap/servers/slapd/back-ldbm/cache.c @@ -890,7 +890,7 @@ entrycache_remove_int(struct cache *cache, struct backentry *e) } else { - LOG("remove %d from id hash failed\n", e->ep_id, 0, 0); + LOG("remove %s (%d) from id hash failed\n", ndn, e->ep_id, 0); } } #ifdef UUIDCACHE_ON @@ -984,7 +984,12 @@ int cache_replace(struct cache *cache, void *oldptr, void *newptr) static int entrycache_replace(struct cache *cache, struct backentry *olde, struct backentry *newe) { - int found; + int found = 0; + int found_in_dn = 0; + int found_in_id = 0; +#ifdef UUIDCACHE_ON + int found_in_uuid = 0; +#endif const char *oldndn; const char *newndn; #ifdef UUIDCACHE_ON @@ -992,6 +997,7 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, const char *newuuid; #endif size_t entry_size = 0; + struct backentry *alte = NULL; LOG("=> entrycache_replace (%s) -> (%s)\n", backentry_get_ndn(olde), backentry_get_ndn(newe), 0); @@ -1015,31 +1021,21 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, * cache tables, operation error */ if ( (olde->ep_state & ENTRY_STATE_NOTINCACHE) == 0 ) { - int found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn)); - int found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID)); + found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn)); + found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID)); #ifdef UUIDCACHE_ON - int found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid)); + found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid)); #endif found = found_in_dn && found_in_id; #ifdef UUIDCACHE_ON found = found && found_in_uuid; #endif - if (!found) { -#ifdef UUIDCACHE_ON - LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n", - found_in_dn, found_in_id, found_in_uuid); -#else - LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d]\n", - found_in_dn, found_in_id, 0); -#endif - PR_Unlock(cache->c_mutex); - return 1; - } } - if (! entry_same_dn(newe, (void *)oldndn) && - (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) { - /* if we're doing a modrdn, the new entry can be in the dn table - * already, so we need to remove that too. + /* If fails, we have to make sure the both entires are removed from the cache, + * otherwise, we have no idea what's left in the cache or not... */ + if (!entry_same_dn(newe, (void *)oldndn) && (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) { + /* if we're doing a modrdn or turning an entry to a tombstone, + * the new entry can be in the dn table already, so we need to remove that too. */ if (remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn))) { @@ -1048,20 +1044,44 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, LOG("entry cache replace remove entry size %lu\n", newe->ep_size, 0, 0); } } - + /* + * The old entry could have been "removed" between the add and this replace, + * The entry is NOT freed, but NOT in the dn hash. + * which could happen since the entry is not necessarily locked. + * This is ok. + */ + olde->ep_state = ENTRY_STATE_DELETED; /* olde is removed from the cache, so set DELETED here. */ + if (!found) { + if (olde->ep_state & ENTRY_STATE_DELETED) { + LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]; but the entry is alreay deleted.\n", + oldndn, found_in_dn, found_in_id); + } else { +#ifdef UUIDCACHE_ON + LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n", + found_in_dn, found_in_id, found_in_uuid); +#else + LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]\n", + oldndn, found_in_dn, found_in_id); +#endif + PR_Unlock(cache->c_mutex); + return 1; + } + } /* now, add the new entry to the hashtables */ /* (probably don't need such extensive error handling, once this has been * tested enough that we believe it works.) */ - if (!add_hash(cache->c_dntable, (void *)newndn, strlen(newndn), newe, NULL)) { - LOG("entry cache replace: can't add dn\n", 0, 0, 0); + if (!add_hash(cache->c_dntable, (void *)newndn, strlen(newndn), newe, (void **)&alte)) { + LOG("entry cache replace (%s): can't add to dn table (returned %s)\n", + newndn, alte?slapi_entry_get_dn(alte->ep_entry):"none", 0); PR_Unlock(cache->c_mutex); return 1; } - if (!add_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID), newe, NULL)) { - LOG("entry cache replace: can't add id\n", 0, 0, 0); + if (!add_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID), newe, (void **)&alte)) { + LOG("entry cache replace (%s): can't add to id table (returned %s)\n", + newndn, alte?slapi_entry_get_dn(alte->ep_entry):"none", 0); if(remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)) == 0){ - LOG("entry cache replace: failed to remove dn table\n", 0, 0, 0); + LOG("entry cache replace: failed to remove dn table\n", 0, 0, 0); } PR_Unlock(cache->c_mutex); return 1; @@ -1071,10 +1091,10 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, newe, NULL)) { LOG("entry cache replace: can't add uuid\n", 0, 0, 0); if(remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)) == 0){ - LOG("entry cache replace: failed to remove dn table(uuid cache)\n", 0, 0, 0); + LOG("entry cache replace: failed to remove dn table(uuid cache)\n", 0, 0, 0); } if(remove_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID)) == 0){ - LOG("entry cache replace: failed to remove id table(uuid cache)\n", 0, 0, 0); + LOG("entry cache replace: failed to remove id table(uuid cache)\n", 0, 0, 0); } PR_Unlock(cache->c_mutex); return 1; @@ -1088,7 +1108,6 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, } else if (newe->ep_size < olde->ep_size) { slapi_counter_subtract(cache->c_cursize, olde->ep_size - newe->ep_size); } - olde->ep_state = ENTRY_STATE_DELETED; newe->ep_state = 0; PR_Unlock(cache->c_mutex); LOG("<= entrycache_replace OK, cache size now %lu cache count now %ld\n", @@ -1282,7 +1301,7 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, PR_Lock(cache->c_mutex); if (! add_hash(cache->c_dntable, (void *)ndn, strlen(ndn), e, (void **)&my_alt)) { - LOG("entry \"%s\" already in dn cache\n", backentry_get_ndn(e), 0, 0); + LOG("entry \"%s\" already in dn cache\n", ndn, 0, 0); /* add_hash filled in 'my_alt' if necessary */ if (my_alt == e) { @@ -1322,14 +1341,15 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, { if (my_alt->ep_state & ENTRY_STATE_CREATING) { - LOG("the entry is reserved\n", 0, 0, 0); + LOG("the entry %s is reserved (ep_state: 0x%x, state: 0x%x\n", ndn, e->ep_state, state); e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); return -1; } else if (state != 0) { - LOG("the entry already exists. cannot reserve it.\n", 0, 0, 0); + LOG("the entry %s already exists. cannot reserve it. (ep_state: 0x%x, state: 0x%x\n", + ndn, e->ep_state, state); e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); return -1; @@ -1341,6 +1361,11 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, if ((*alt)->ep_refcnt == 0) lru_delete(cache, (void *)*alt); (*alt)->ep_refcnt++; + LOG("the entry %s already exists. returning existing entry %s (state: 0x%x)\n", + ndn, backentry_get_ndn(my_alt), state); + } else { + LOG("the entry %s already exists. Not returning existing entry %s (state: 0x%x)\n", + ndn, backentry_get_ndn(my_alt), state); } PR_Unlock(cache->c_mutex); return 1; @@ -1355,7 +1380,7 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, if (state == 0) { /* neither of these should fail, or something is very wrong. */ if (! add_hash(cache->c_idtable, &(e->ep_id), sizeof(ID), e, NULL)) { - LOG("entry %s already in id cache!\n", backentry_get_ndn(e), 0, 0); + LOG("entry %s already in id cache!\n", ndn, 0, 0); if (already_in) { /* there's a bug in the implementatin of 'modify' and 'modrdn' * that i'm working around here. basically they do a @@ -1377,10 +1402,12 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, return 0; } if(remove_hash(cache->c_dntable, (void *)ndn, strlen(ndn)) == 0){ - LOG("entrycache_add_int: failed to remove dn table\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove %s from dn table\n", 0, 0, 0); } e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); + LOG("entrycache_add_int: failed to add %s to cache (ep_state: %x, already_in: %d)\n", + ndn, e->ep_state, already_in); return -1; } #ifdef UUIDCACHE_ON @@ -1391,10 +1418,10 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, LOG("entry %s already in uuid cache!\n", backentry_get_ndn(e), 0, 0); if(remove_hash(cache->c_dntable, (void *)ndn, strlen(ndn)) == 0){ - LOG("entrycache_add_int: failed to remove dn table(uuid cache)\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove dn table(uuid cache)\n", 0, 0, 0); } if(remove_hash(cache->c_idtable, &(e->ep_id), sizeof(ID)) == 0){ - LOG("entrycache_add_int: failed to remove id table(uuid cache)\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove id table(uuid cache)\n", 0, 0, 0); } e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); @@ -1499,8 +1526,8 @@ int cache_lock_entry(struct cache *cache, struct backentry *e) if (!e->ep_mutexp) { LOG("<= cache_lock_entry (DELETED)\n", 0, 0, 0); LDAPDebug1Arg(LDAP_DEBUG_ANY, - "cache_lock_entry: failed to create a lock for %s\n", - backentry_get_ndn(e)); + "cache_lock_entry: failed to create a lock for %s\n", + backentry_get_ndn(e)); LOG("<= cache_lock_entry (FAILED)\n", 0, 0, 0); return 1; } diff --git a/ldap/servers/slapd/back-ldbm/ldbm_add.c b/ldap/servers/slapd/back-ldbm/ldbm_add.c index 3dcceff..0a4557b 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_add.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_add.c @@ -103,9 +103,9 @@ ldbm_back_add( Slapi_PBlock *pb ) int parent_found = 0; int ruv_c_init = 0; int rc = 0; - int addingentry_id_assigned= 0; - int addingentry_in_cache= 0; - int tombstone_in_cache= 0; + int addingentry_id_assigned = 0; + int addingentry_in_cache = 0; + int tombstone_in_cache = 0; Slapi_DN *sdn = NULL; Slapi_DN parentsdn; Slapi_Operation *operation; @@ -118,6 +118,7 @@ ldbm_back_add( Slapi_PBlock *pb ) entry_address addr = {0}; int not_an_error = 0; int parent_switched = 0; + int noabort = 1; slapi_pblock_get( pb, SLAPI_PLUGIN_PRIVATE, &li ); slapi_pblock_get( pb, SLAPI_ADD_ENTRY, &e ); @@ -194,7 +195,6 @@ ldbm_back_add( Slapi_PBlock *pb ) goto error_return; } - /* * Originally (in the U-M LDAP 3.3 code), there was a comment near this * code about a race condition. The race was that a 2nd entry could be @@ -214,11 +214,16 @@ ldbm_back_add( Slapi_PBlock *pb ) if (txn.back_txn_txn && (txn.back_txn_txn != parent_txn)) { /* Don't release SERIAL LOCK */ dblayer_txn_abort_ext(li, &txn, PR_FALSE); + noabort = 1; slapi_pblock_set(pb, SLAPI_TXN, parent_txn); if (addingentry_in_cache) { /* addingentry is in cache. Remove it once. */ - CACHE_REMOVE(&inst->inst_cache, addingentry); + retval = CACHE_REMOVE(&inst->inst_cache, addingentry); + if (retval) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "ldbm_add: cache_remove %s failed.\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); + } CACHE_RETURN(&inst->inst_cache, &addingentry); } else { backentry_free(&addingentry); @@ -231,11 +236,11 @@ ldbm_back_add( Slapi_PBlock *pb ) } if (addingentry_in_cache) { /* Adding the resetted addingentry to the cache. */ - if (cache_add_tentative(&inst->inst_cache, - addingentry, NULL) != 0) { - LDAPDebug0Args(LDAP_DEBUG_CACHE, - "cache_add_tentative concurrency detected\n"); + if (cache_add_tentative(&inst->inst_cache, addingentry, NULL) != 0) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code = LDAP_ALREADY_EXISTS; + addingentry_in_cache = 0; goto error_return; } } @@ -260,6 +265,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if (0 == retry_count) { /* First time, hold SERIAL LOCK */ retval = dblayer_txn_begin(be, parent_txn, &txn); + noabort = 0; if (!is_tombstone_operation) { rc= slapi_setbit_int(rc,SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); @@ -306,6 +312,10 @@ ldbm_back_add( Slapi_PBlock *pb ) } } + /* + * If the parent is conflict, slapi_sdn_get_backend_parent does not support it. + * That is, adding children to a conflict entry is not allowed. + */ slapi_sdn_get_backend_parent(sdn, &parentsdn, pb->pb_backend); /* Check if an entry with the intended DN already exists. */ done_with_pblock_entry(pb,SLAPI_ADD_EXISTING_DN_ENTRY); /* Could be through this multiple times */ @@ -313,28 +323,25 @@ ldbm_back_add( Slapi_PBlock *pb ) addr.udn = NULL; addr.uniqueid = NULL; ldap_result_code= get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_EXISTING_DN_ENTRY, !is_replicated_operation); - if(ldap_result_code==LDAP_OPERATIONS_ERROR || - ldap_result_code==LDAP_INVALID_DN_SYNTAX) - { + if(ldap_result_code==LDAP_OPERATIONS_ERROR || ldap_result_code==LDAP_INVALID_DN_SYNTAX) { goto error_return; } } /* if we can find the parent by dn or uniqueid, and the operation has requested the parent then get it */ - if(have_parent_address(&parentsdn, operation->o_params.p.p_add.parentuniqueid) && - slapi_isbitset_int(rc,SLAPI_RTN_BIT_FETCH_PARENT_ENTRY)) - { + if (have_parent_address(&parentsdn, operation->o_params.p.p_add.parentuniqueid) && + slapi_isbitset_int(rc,SLAPI_RTN_BIT_FETCH_PARENT_ENTRY)) { done_with_pblock_entry(pb,SLAPI_ADD_PARENT_ENTRY); /* Could be through this multiple times */ addr.sdn = &parentsdn; addr.udn = NULL; addr.uniqueid = operation->o_params.p.p_add.parentuniqueid; - ldap_result_code= get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_PARENT_ENTRY, !is_replicated_operation); - /* need to set parentsdn or parentuniqueid if either is not set? */ + ldap_result_code = get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_PARENT_ENTRY, !is_replicated_operation); } /* Call the Backend Pre Add plugins */ + ldap_result_code = LDAP_SUCCESS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); - rc= plugin_call_plugins(pb, SLAPI_PLUGIN_BE_PRE_ADD_FN); + rc = plugin_call_plugins(pb, SLAPI_PLUGIN_BE_PRE_ADD_FN); if (rc < 0) { int opreturn = 0; if (SLAPI_PLUGIN_NOOP == rc) { @@ -380,6 +387,7 @@ ldbm_back_add( Slapi_PBlock *pb ) ldap_result_code= LDAP_OPERATIONS_ERROR; goto error_return; } + noabort = 0; /* stash the transaction for plugins */ slapi_pblock_set(pb, SLAPI_TXN, txn.back_txn_txn); @@ -402,10 +410,14 @@ ldbm_back_add( Slapi_PBlock *pb ) operation->o_params.p.p_add.parentuniqueid = slapi_ch_strdup(slapi_entry_get_uniqueid(parententry->ep_entry)); } - if (slapi_sdn_isempty(&parentsdn)) { + if (slapi_sdn_isempty(&parentsdn) || + slapi_sdn_compare(&parentsdn, slapi_entry_get_sdn(parententry->ep_entry))) { /* Set the parentsdn now */ slapi_sdn_set_dn_byval(&parentsdn, slapi_entry_get_dn_const(parententry->ep_entry)); } + } else { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "find_entry2modify_only returned NULL parententry pdn: %s, uniqueid: %s\n", + slapi_sdn_get_dn(&parentsdn), slapi_sdn_get_dn(&parentsdn), addr.uniqueid?addr.uniqueid:"none"); } modify_init(&parent_modify_c,parententry); } @@ -413,11 +425,36 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Check if the entry we have been asked to add already exists */ { Slapi_Entry *entry; - slapi_pblock_get( pb, SLAPI_ADD_EXISTING_DN_ENTRY, &entry); - if ( entry != NULL ) - { - /* The entry already exists */ - ldap_result_code= LDAP_ALREADY_EXISTS; + slapi_pblock_get(pb, SLAPI_ADD_EXISTING_DN_ENTRY, &entry); + if (entry) { + if (is_resurect_operation) { + Slapi_Entry *uniqentry; + slapi_pblock_get(pb, SLAPI_ADD_EXISTING_UNIQUEID_ENTRY, &uniqentry); + if (uniqentry == entry) { + /* + * adding entry having the uniqueid exists. + * No need to resurrect. + */ + ldap_result_code = LDAP_SUCCESS; + } else { + /* The entry having the DN already exists */ + if (uniqentry) { + if (PL_strcmp(slapi_entry_get_uniqueid(entry), + slapi_entry_get_uniqueid(uniqentry))) { + /* Not match; conflict. */ + ldap_result_code = LDAP_ALREADY_EXISTS; + } else { + /* Same entry; no need to resurrect. */ + ldap_result_code = LDAP_SUCCESS; + } + } else { + ldap_result_code = LDAP_ALREADY_EXISTS; + } + } + } else { + /* The entry already exists */ + ldap_result_code = LDAP_ALREADY_EXISTS; + } goto error_return; } else @@ -428,7 +465,7 @@ ldbm_back_add( Slapi_PBlock *pb ) * entry we did match has a referral we should return * instead. we do this only if managedsait is not on. */ - if ( !managedsait && !is_tombstone_operation ) + if (!managedsait && !is_tombstone_operation && !is_resurect_operation) { int err= 0; Slapi_DN ancestorsdn; @@ -438,9 +475,7 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_sdn_done(&ancestorsdn); if ( ancestorentry != NULL ) { - int sentreferral = - check_entry_for_referral(pb, ancestorentry->ep_entry, - backentry_get_ndn(ancestorentry), "ldbm_back_add"); + int sentreferral= check_entry_for_referral(pb, ancestorentry->ep_entry, backentry_get_ndn(ancestorentry), "ldbm_back_add"); CACHE_RETURN( &inst->inst_cache, &ancestorentry ); if(sentreferral) { @@ -453,25 +488,25 @@ ldbm_back_add( Slapi_PBlock *pb ) } /* no need to check the schema as this is a replication add */ - if(!is_replicated_operation){ + if (!is_replicated_operation) { if ((operation_is_flag_set(operation,OP_FLAG_ACTION_SCHEMA_CHECK)) && (slapi_entry_schema_check(pb, e) != 0)) - { - LDAPDebug(LDAP_DEBUG_TRACE, "entry failed schema check\n", 0, 0, 0); - ldap_result_code = LDAP_OBJECT_CLASS_VIOLATION; - slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); - goto error_return; - } + { + LDAPDebug(LDAP_DEBUG_TRACE, "entry failed schema check\n", 0, 0, 0); + ldap_result_code = LDAP_OBJECT_CLASS_VIOLATION; + slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + goto error_return; + } - /* Check attribute syntax */ - if (slapi_entry_syntax_check(pb, e, 0) != 0) - { - LDAPDebug(LDAP_DEBUG_TRACE, "entry failed syntax check\n", 0, 0, 0); - ldap_result_code = LDAP_INVALID_SYNTAX; - slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); - goto error_return; + /* Check attribute syntax */ + if (slapi_entry_syntax_check(pb, e, 0) != 0) + { + LDAPDebug(LDAP_DEBUG_TRACE, "entry failed syntax check\n", 0, 0, 0); + ldap_result_code = LDAP_INVALID_SYNTAX; + slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + goto error_return; + } } - } opcsn = operation_get_csn (operation); if(is_resurect_operation) @@ -488,7 +523,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if ( tombstoneentry==NULL ) { ldap_result_code= -1; - goto error_return; /* error result sent by find_entry2modify() */ + goto error_return; /* error result sent by find_entry2modify() */ } tombstone_in_cache = 1; @@ -507,10 +542,14 @@ ldbm_back_add( Slapi_PBlock *pb ) */ if (NULL == sdn) { LDAPDebug0Args(LDAP_DEBUG_ANY, "ldbm_back_add: Null target dn\n"); + ldap_result_code = LDAP_OPERATIONS_ERROR; goto error_return; } dn = slapi_sdn_get_dn(sdn); slapi_entry_set_sdn(addingentry->ep_entry, sdn); /* The DN is passed into the entry. */ + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry->ep_entry), + slapi_entry_get_dn_const(addingentry->ep_entry)); /* LPREPL: the DN is normalized...Somehow who should get a not normalized one */ addingentry->ep_id = slapi_entry_attr_get_ulong(addingentry->ep_entry,"entryid"); slapi_entry_attr_delete(addingentry->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); @@ -674,10 +713,8 @@ ldbm_back_add( Slapi_PBlock *pb ) Slapi_DN ancestorsdn; struct backentry *ancestorentry; - LDAPDebug( LDAP_DEBUG_TRACE, - "parent does not exist, pdn = %s\n", - slapi_sdn_get_dn(&parentsdn), 0, 0 ); - + LDAPDebug1Arg(LDAP_DEBUG_BACKLDBM, "ldbm_add: Parent \"%s\" does not exist. " + "It might be a conflict entry.\n", slapi_sdn_get_dn(&parentsdn)); slapi_sdn_init(&ancestorsdn); ancestorentry = dn2ancestor(be, &parentsdn, &ancestorsdn, &txn, &err ); CACHE_RETURN( &inst->inst_cache, &ancestorentry ); @@ -688,22 +725,37 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_sdn_done(&ancestorsdn); goto error_return; } - ldap_result_code = plugin_call_acl_plugin (pb, e, NULL, NULL, SLAPI_ACL_ADD, - ACLPLUGIN_ACCESS_DEFAULT, &errbuf ); + ldap_result_code = plugin_call_acl_plugin(pb, e, NULL, NULL, SLAPI_ACL_ADD, + ACLPLUGIN_ACCESS_DEFAULT, &errbuf); if ( ldap_result_code != LDAP_SUCCESS ) { - LDAPDebug( LDAP_DEBUG_TRACE, "no access to parent\n", 0, 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "no access to parent, pdn = %s\n", + slapi_sdn_get_dn(&parentsdn)); ldap_result_message= errbuf; goto error_return; } pid = parententry->ep_id; + + /* We may need to adjust the DN since parent could be a resrected conflict entry... */ + if (!slapi_sdn_isparent(slapi_entry_get_sdn_const(parententry->ep_entry), + slapi_entry_get_sdn_const(addingentry->ep_entry))) { + Slapi_DN adjustedsdn = {0}; + char *adjusteddn = slapi_ch_smprintf("%s,%s", + slapi_entry_get_rdn_const(addingentry->ep_entry), + slapi_entry_get_dn_const(parententry->ep_entry)); + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "ldbm_add: adjusting dn: %s --> %s\n", + slapi_entry_get_dn(addingentry->ep_entry), adjusteddn); + slapi_sdn_set_normdn_passin(&adjustedsdn, adjusteddn); + slapi_entry_set_sdn(addingentry->ep_entry, &adjustedsdn); + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry->ep_entry), adjusteddn); + slapi_sdn_done(&adjustedsdn); + } } else { /* no parent */ - if ( !isroot && !is_replicated_operation) - { - LDAPDebug( LDAP_DEBUG_TRACE, "no parent & not root\n", - 0, 0, 0 ); + if (!isroot && !is_replicated_operation) { + LDAPDebug0Args(LDAP_DEBUG_TRACE, "no parent & not root\n"); ldap_result_code= LDAP_INSUFFICIENT_ACCESS; goto error_return; } @@ -732,11 +784,12 @@ ldbm_back_add( Slapi_PBlock *pb ) * operational attributes to ensure that the cache is sized correctly. */ if ( cache_add_tentative( &inst->inst_cache, addingentry, NULL )!= 0 ) { - LDAPDebug( LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected\n", 0, 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code= LDAP_ALREADY_EXISTS; goto error_return; } - addingentry_in_cache= 1; + addingentry_in_cache = 1; /* * Before we add the entry, find out if the syntax of the aci @@ -744,7 +797,8 @@ ldbm_back_add( Slapi_PBlock *pb ) * the entry if the syntax is incorrect. */ if ( plugin_call_acl_verify_syntax (pb, addingentry->ep_entry, &errbuf) != 0 ) { - LDAPDebug( LDAP_DEBUG_TRACE, "ACL syntax error\n", 0,0,0); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "ACL syntax error: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code= LDAP_INVALID_SYNTAX; ldap_result_message= errbuf; goto error_return; @@ -753,12 +807,14 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Having decided that we're really going to do the operation, let's modify the in-memory state of the parent to reflect the new child (update subordinate count specifically */ - if (NULL != parententry) - { + if (parententry) { retval = parent_update_on_childchange(&parent_modify_c, - PARENTUPDATE_ADD, NULL); + is_resurect_operation?PARENTUPDATE_RESURECT:PARENTUPDATE_ADD, + NULL); /* The modify context now contains info needed later */ - if (0 != retval) { + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "parent_update_on_childchange: %s, rc=%d\n", + slapi_entry_get_dn_const(addingentry->ep_entry), retval); ldap_result_code= LDAP_OPERATIONS_ERROR; goto error_return; } @@ -781,14 +837,12 @@ ldbm_back_add( Slapi_PBlock *pb ) not_an_error = 1; rc = retval = LDAP_SUCCESS; } - LDAPDebug1Arg( LDAP_DEBUG_TRACE, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " - "returned error code %d\n", retval ); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " + "returned error code %d\n", retval ); if (!ldap_result_code) { slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } if (!ldap_result_code) { - LDAPDebug0Args( LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " - "returned error but did not setSLAPI_RESULT_CODE \n" ); ldap_result_code = LDAP_OPERATIONS_ERROR; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); } @@ -797,6 +851,8 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_set(pb, SLAPI_PLUGIN_OPRETURN, ldap_result_code ? &ldap_result_code : &retval); } slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + LDAPDebug1Arg(LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin failed: %d", + ldap_result_code ? ldap_result_code : retval); goto error_return; } @@ -807,9 +863,10 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_TRACE, "id2entry_add failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + if (retval) { + LDAPDebug(LDAP_DEBUG_TRACE, "id2entry_add(%s) failed, err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; @@ -817,19 +874,18 @@ ldbm_back_add( Slapi_PBlock *pb ) } goto error_return; } - if(is_resurect_operation) - { + if (is_resurect_operation) { retval = index_addordel_string(be,SLAPI_ATTR_OBJECTCLASS,SLAPI_ATTR_VALUE_TOMBSTONE,addingentry->ep_id,BE_INDEX_DEL,&txn); if (DB_LOCK_DEADLOCK == retval) { LDAPDebug( LDAP_DEBUG_ARGS, "add 2 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 1 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string TOMBSTONE (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -843,10 +899,10 @@ ldbm_back_add( Slapi_PBlock *pb ) continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 2 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string UNIQUEID (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -864,17 +920,29 @@ ldbm_back_add( Slapi_PBlock *pb ) continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 3 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string ENTRYDN (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; } goto error_return; } - } + /* Need to delete the entryrdn index of the resurrected tombstone... */ + if (entryrdn_get_switch()) { /* subtree-rename: on */ + if (tombstoneentry) { + retval = entryrdn_index_entry(be, tombstoneentry, BE_INDEX_DEL, &txn); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, "Resurrecting %s: failed to remove entryrdn index, err=%d %s\n", + slapi_entry_get_dn_const(tombstoneentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + goto error_return; + } + } + } + } if (is_tombstone_operation) { retval = index_addordel_entry( be, addingentry, BE_INDEX_ADD | BE_INDEX_TOMBSTONE, &txn ); @@ -889,9 +957,8 @@ ldbm_back_add( Slapi_PBlock *pb ) /* retry txn */ continue; } - if (retval != 0) { - LDAPDebug2Args(LDAP_DEBUG_ANY, - "add: attempt to index %lu failed (rc=%d)\n", + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_ANY, "add: attempt to index %lu failed; rc=%d\n", (u_long)addingentry->ep_id, retval); ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { @@ -909,11 +976,10 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 1 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "modify_update_all: %s (%lu) failed; rc=%d\n", + slapi_entry_get_dn(addingentry->ep_entry), (u_long)addingentry->ep_id, retval); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -928,17 +994,16 @@ ldbm_back_add( Slapi_PBlock *pb ) { retval= vlv_update_all_indexes(&txn, be, pb, NULL, addingentry); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, - "add DEADLOCK vlv_update_index\n", 0, 0, 0 ); + LDAPDebug(LDAP_DEBUG_ARGS, + "add DEADLOCK vlv_update_index\n", 0, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, - "vlv_update_index failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_TRACE, + "vlv_update_index failed, err=%d %s\n", + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -984,7 +1049,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if (retry_count == RETRY_TIMES) { /* Failed */ LDAPDebug( LDAP_DEBUG_ANY, "Retry count exceeded in add\n", 0, 0, 0 ); - ldap_result_code= LDAP_BUSY; + ldap_result_code= LDAP_BUSY; goto error_return; } @@ -995,38 +1060,53 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_set( pb, SLAPI_ENTRY_PRE_OP, NULL ); slapi_pblock_set( pb, SLAPI_ENTRY_POST_OP, slapi_entry_dup( addingentry->ep_entry )); - if(is_resurect_operation) - { + if (is_resurect_operation) { /* * We can now switch the tombstone entry with the real entry. */ - if (cache_replace( &inst->inst_cache, tombstoneentry, addingentry ) != 0 ) - { + retval = cache_replace(&inst->inst_cache, tombstoneentry, addingentry); + if (retval) { /* This happens if the dn of addingentry already exists */ - cache_unlock_entry( &inst->inst_cache, tombstoneentry ); ADD_SET_ERROR(ldap_result_code, LDAP_ALREADY_EXISTS, retry_count); + LDAPDebug2Args(LDAP_DEBUG_CACHE, "ldap_add: cache_replace concurrency detected: %s (rc: %d)\n", + slapi_entry_get_dn_const(addingentry->ep_entry), retval); + retval = -1; goto error_return; } + if (addingentry_in_cache) { /* decrease the refcnt added by tentative */ + CACHE_RETURN( &inst->inst_cache, &addingentry ); + } + addingentry_in_cache = 1; /* reset it to make it sure... */ /* * The tombstone was locked down in the cache... we can * get rid of the entry in the cache now. + * We cannot expect tombstoneentry exists from now on. */ - cache_unlock_entry( &inst->inst_cache, tombstoneentry ); - CACHE_RETURN( &inst->inst_cache, &tombstoneentry ); - tombstone_in_cache = 0; /* deleted */ + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, delete the tombstone dn from the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, + tombstoneentry->ep_id); + if (bdn) { /* in the dncache, remove it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + cache_unlock_entry(&inst->inst_cache, tombstoneentry); + CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + tombstone_in_cache = 0; } if (parent_found) { /* switch the parent entry copy into play */ - modify_switch_entries( &parent_modify_c,be); + modify_switch_entries(&parent_modify_c,be); parent_switched = 1; } if (ruv_c_init) { if (modify_switch_entries(&ruv_c, be) != 0 ) { ldap_result_code= LDAP_OPERATIONS_ERROR; - LDAPDebug( LDAP_DEBUG_ANY, - "ldbm_back_add: modify_switch_entries failed\n", 0, 0, 0); + LDAPDebug0Args(LDAP_DEBUG_ANY, + "ldbm_back_add: modify_switch_entries failed\n"); goto error_return; } } @@ -1040,8 +1120,6 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } if (!ldap_result_code) { - LDAPDebug0Args( LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_POST_ADD_FN plugin " - "returned error but did not set SLAPI_RESULT_CODE\n" ); ldap_result_code = LDAP_OPERATIONS_ERROR; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); } @@ -1066,6 +1144,7 @@ ldbm_back_add( Slapi_PBlock *pb ) } goto error_return; } + noabort = 1; rc= 0; goto common_return; @@ -1075,23 +1154,22 @@ error_return: { next_id_return( be, addingentry->ep_id ); } - if ( NULL != addingentry ) + if ( addingentry ) { if ( addingentry_in_cache ) { if (inst) { CACHE_REMOVE(&inst->inst_cache, addingentry); } - addingentry_in_cache = 0; } - backentry_clear_entry(addingentry); /* e is released in the frontend */ - backentry_free( &addingentry ); /* release the backend wrapper, here */ - } - if(tombstone_in_cache && inst) - { - CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + else + { + if (!is_resurect_operation) { /* if resurect, tombstoneentry is dupped. */ + backentry_clear_entry(addingentry); /* e is released in the frontend */ + } + backentry_free( &addingentry ); /* release the backend wrapper, here */ + } } - if (rc == DB_RUNRECOVERY) { dblayer_remember_disk_filled(li); ldbm_nasty("Add",80,rc); @@ -1109,7 +1187,7 @@ error_return: } diskfull_return: if (disk_full) { - rc= return_on_disk_full(li); + rc = return_on_disk_full(li); } else { /* It is safer not to abort when the transaction is not started. */ if (txn.back_txn_txn && (txn.back_txn_txn != parent_txn)) { @@ -1143,7 +1221,9 @@ diskfull_return: } /* Release SERIAL LOCK */ - dblayer_txn_abort(be, &txn); /* abort crashes in case disk full */ + if (!noabort) { + dblayer_txn_abort(be, &txn); /* abort crashes in case disk full */ + } /* txn is no longer valid - reset the txn pointer to the parent */ slapi_pblock_set(pb, SLAPI_TXN, parent_txn); } @@ -1154,8 +1234,12 @@ diskfull_return: common_return: if (inst) { + if(tombstone_in_cache && tombstoneentry) { + cache_unlock_entry(&inst->inst_cache, tombstoneentry); + CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + } if (addingentry_in_cache && addingentry) { - if (entryrdn_get_switch()) { /* subtree-rename: on */ + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ /* since adding the entry to the entry cache was successful, * let's add the dn to dncache, if not yet done. */ struct backdn *bdn = dncache_find_id(&inst->inst_dncache, diff --git a/ldap/servers/slapd/back-ldbm/ldbm_delete.c b/ldap/servers/slapd/back-ldbm/ldbm_delete.c index 4c75fb9..0ea1bee 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_delete.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_delete.c @@ -90,6 +90,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) /* replication, we must create a new tombstone entry */ int tombstone_in_cache = 0; int e_in_cache = 0; + int remove_e_from_cache = 0; entry_address *addr; int addordel_flags = 0; /* passed to index_addordel */ char *entryusn_str = NULL; @@ -220,6 +221,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (tombstone_in_cache) { CACHE_REMOVE(&inst->inst_cache, tombstone); CACHE_RETURN(&inst->inst_cache, &tombstone); + tombstone = NULL; tombstone_in_cache = 0; } else { backentry_free(&tombstone); @@ -230,13 +232,6 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } } - - /* reset original entry in cache */ - if (!e_in_cache) { - CACHE_ADD(&inst->inst_cache, e, NULL); - e_in_cache = 1; - } - if (ruv_c_init) { /* reset the ruv txn stuff */ modify_term(&ruv_c, be); @@ -272,17 +267,28 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (0 == retry_count) { /* just once */ /* find and lock the entry we are about to modify */ + /* + * A corner case: + * If a conflict occurred in a MMR topology, a replicated delete + * op from another master could target a conflict entry; while the + * corresponding entry on this server could have been already + * deleted. That is, the entry 'e' found with "addr" is a tomb- + * stone. If it is the case, we need to back off. + */ if ( (e = find_entry2modify( pb, be, addr, &txn )) == NULL ) { ldap_result_code= LDAP_NO_SUCH_OBJECT; retval = -1; + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "ldbm_back_delete: Deleting entry is already deleted.\n"); goto error_return; /* error result sent by find_entry2modify() */ } e_in_cache = 1; /* e is cached */ - if ( slapi_entry_has_children( e->ep_entry ) ) - { + retval = slapi_entry_has_children(e->ep_entry); + if (retval) { ldap_result_code= LDAP_NOT_ALLOWED_ON_NONLEAF; + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_delete", "Deleting entry %s has %d children.\n", + slapi_entry_get_dn(e->ep_entry), retval); retval = -1; goto error_return; } @@ -318,7 +324,17 @@ ldbm_back_delete( Slapi_PBlock *pb ) * Plugin indicated some kind of failure, * or that this Operation became a No-Op. */ + slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); if (!ldap_result_code) { + if (LDAP_ALREADY_EXISTS == ldap_result_code) { + /* + * The target entry is already a tombstone. + * We need to treat this as a success, + * but we need to remove the entry e from the entry cache. + */ + remove_e_from_cache = 1; + ldap_result_code = LDAP_SUCCESS; + } slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } /* restore original entry so the front-end delete code can free it */ @@ -330,8 +346,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } /* the flag could be set in a preop plugin (e.g., USN) */ - delete_tombstone_entry = operation_is_flag_set(operation, - OP_FLAG_TOMBSTONE_ENTRY); + delete_tombstone_entry = operation_is_flag_set(operation, OP_FLAG_TOMBSTONE_ENTRY); } /* call the transaction pre delete plugins just after the @@ -493,13 +508,15 @@ ldbm_back_delete( Slapi_PBlock *pb ) } if (NULL == parent) { entry_address parent_addr; - + if (is_tombstone_entry) { + parent_addr.uniqueid = slapi_entry_attr_get_charptr(e->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + } else { + parent_addr.uniqueid = NULL; + } parent_addr.sdn = &parentsdn; - parent_addr.uniqueid = NULL; - parent = find_entry2modify_only_ext(pb, be, &parent_addr, - TOMBSTONE_INCLUDED, &txn); + parent = find_entry2modify_only_ext(pb, be, &parent_addr, TOMBSTONE_INCLUDED, &txn); } - if (NULL != parent) { + if (parent) { int isglue; size_t haschildren = 0; int op = PARENTUPDATE_DEL; @@ -516,11 +533,11 @@ ldbm_back_delete( Slapi_PBlock *pb ) } else if (delete_tombstone_entry) { op |= PARENTUPDATE_DELETE_TOMBSTONE; } - retval = parent_update_on_childchange(&parent_modify_c, - op, &haschildren); + retval = parent_update_on_childchange(&parent_modify_c, op, &haschildren); /* The modify context now contains info needed later */ if (0 != retval) { ldap_result_code= LDAP_OPERATIONS_ERROR; + slapi_sdn_done(&parentsdn); retval = -1; goto error_return; } @@ -550,9 +567,17 @@ ldbm_back_delete( Slapi_PBlock *pb ) */ const char *childuniqueid= slapi_entry_get_uniqueid(e->ep_entry); const char *parentuniqueid= NULL; - char *tombstone_dn = compute_entry_tombstone_dn(slapi_entry_get_dn(e->ep_entry), - childuniqueid); + char *edn = slapi_entry_get_dn(e->ep_entry); + char *tombstone_dn; Slapi_Value *tomb_value; + + if (slapi_is_special_rdn(edn, RDN_IS_TOMBSTONE)) { + LDAPDebug1Arg(LDAP_DEBUG_ANY, "Turning a tombstone into a tombstone! \"%s\"\n", edn); + ldap_result_code= LDAP_OPERATIONS_ERROR; + retval = -1; + goto error_return; + } + tombstone_dn = compute_entry_tombstone_dn(edn, childuniqueid); slapi_sdn_set_ndn_byval(&nscpEntrySDN, slapi_sdn_get_ndn(slapi_entry_get_sdn(e->ep_entry))); @@ -674,41 +699,85 @@ ldbm_back_delete( Slapi_PBlock *pb ) * tentatively for now, then cache_add again when the original * entry is removed from the cache. */ - if (cache_add_tentative( &inst->inst_cache, tombstone, NULL) == 0) { + retval = cache_add_tentative(&inst->inst_cache, tombstone, NULL); + if (0 == retval) { tombstone_in_cache = 1; - } else if (!(tombstone->ep_state & ENTRY_STATE_NOTINCACHE)) { - LDAPDebug1Arg(LDAP_DEBUG_CACHE, - "id2entry_add tombstone (%s) is in cache\n", - slapi_entry_get_dn(tombstone->ep_entry)); - tombstone_in_cache = 1; + } else { + LDAPDebug2Args(LDAP_DEBUG_ANY, + "tombstone entry %s failed to add to the cache: %d\n", + slapi_entry_get_dn(tombstone->ep_entry), retval); + tombstone_in_cache = 0; + if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; + DEL_SET_ERROR(ldap_result_code, + LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; } retval = id2entry_add( be, tombstone, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 1 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete 1 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); /* Abort and re-try */ continue; } - if (0 != retval) { + if (retval) { LDAPDebug( LDAP_DEBUG_ANY, "id2entry_add failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } - if (cache_replace( &inst->inst_cache, e, tombstone ) != 0 ) { - LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "ldbm_back_delete cache_replace failed\n"); - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); - retval= -1; - goto error_return; + if (tombstone_in_cache) { + retval = cache_replace(&inst->inst_cache, e, tombstone); + if (retval) { + LDAPDebug(LDAP_DEBUG_CACHE, "ldbm_back_delete: cache_replace failed (%d): %s --> %s\n", + retval, slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn(tombstone->ep_entry)); + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } } else { - e_in_cache = 0; /* e un-cached */ + struct backentry *imposter = NULL; + retval = CACHE_ADD(&inst->inst_cache, tombstone, &imposter); + if (retval > 0) { + if (imposter) { + /* + * The same tombstone entry (different Slapi_Entry) is already + * generated and set to cache. Back off. + */ + CACHE_RETURN(&inst->inst_cache, &imposter); + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: same DN tombstone in cache: %s\n", + slapi_entry_get_dn(tombstone->ep_entry)); + } else { + /* + * The same tombstone entry (same Slapi_Entry) is being created. + * Something is wrong. We should clean it up from the cache, + * and back off. + */ + tombstone_in_cache = 1; + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: same tombstone in cache: %s\n", + slapi_entry_get_dn(tombstone->ep_entry)); + } + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } else if (retval < 0) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: Add %s failed.\n", + slapi_entry_get_dn(tombstone->ep_entry)); + /* Complete add error */ + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } } - /* tombstone was already added to the cache via cache_add_tentative (to reserve its spot in the cache) - and/or id2entry_add - so it already had one refcount - cache_replace adds another refcount - - drop the extra ref added by cache_replace */ - CACHE_RETURN( &inst->inst_cache, &tombstone ); + if (tombstone_in_cache) { + /* tombstone was already added to the cache via cache_add_tentative (to reserve its spot in the cache) + and/or id2entry_add - so it already had one refcount - cache_replace adds another refcount - + drop the extra ref added by cache_replace */ + CACHE_RETURN( &inst->inst_cache, &tombstone ); + } + tombstone_in_cache = 1; } else { @@ -716,13 +785,12 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = id2entry_delete( be, e, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 2 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "delete 2 DEADLOCK\n"); /* Retry txn */ continue; } - if (retval != 0 ) { - if (retval == DB_RUNRECOVERY || - LDBM_OS_ERR_IS_DISKFULL(retval)) { + if (retval) { + if (retval == DB_RUNRECOVERY || LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; } DEL_SET_ERROR(ldap_result_code, @@ -739,12 +807,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = index_addordel_entry( be, e, addordel_flags, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 1 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "delete 1 DEADLOCK\n"); /* Retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_TRACE, "index_del_entry failed\n", 0, 0, 0 ); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, "index_del_entry(%s, 0x%x) failed (%d)\n", + slapi_entry_get_dn(e->ep_entry), addordel_flags, retval); DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } @@ -759,14 +828,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) SLAPI_ATTR_VALUE_TOMBSTONE, tombstone->ep_id,BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_VALUE_TOMBSTONE, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_VALUE_TOMBSTONE, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -779,14 +848,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_entry_get_uniqueid(tombstone->ep_entry), tombstone->ep_id,BE_INDEX_ADD,&txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_UNIQUEID, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_UNIQUEID, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -799,14 +868,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_sdn_get_ndn(&nscpEntrySDN), tombstone->ep_id, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_NSCP_ENTRYDN, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_NSCP_ENTRYDN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -823,14 +892,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) entryusn_str, tombstone->ep_id, BE_INDEX_ADD, &txn); slapi_ch_free_string(&entryusn_str); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_ENTRYUSN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_ENTRYUSN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -854,13 +923,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) svals, NULL, e->ep_id, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (updating " LDBM_PARENTID_STR ") DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if ( retval ) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", LDBM_PARENTID_STR, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -870,15 +939,16 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } } +#if 0 /* The entryrdn element is already deleted in the index_addordel_entry */ retval = entryrdn_index_entry(be, e, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (deleting entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, + LDAPDebug2Args( LDAP_DEBUG_ANY, "delete (deleting entryrdn) failed, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -887,21 +957,21 @@ ldbm_back_delete( Slapi_PBlock *pb ) LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } +#endif retval = entryrdn_index_entry(be, tombstone, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "adding (adding tombstone entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, - "adding (adding tombstone entryrdn) failed, err=%d %s\n", - retval, - (msg = dblayer_strerror( retval )) ? msg : "" ); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, + "adding (adding tombstone entryrdn %s) failed, err=%d %s\n", + slapi_entry_get_dn(tombstone->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : "" ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } } @@ -918,14 +988,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) SLAPI_ATTR_VALUE_TOMBSTONE, e->ep_id, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_VALUE_TOMBSTONE, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_VALUE_TOMBSTONE, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -938,14 +1008,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_entry_get_uniqueid(e->ep_entry), e->ep_id, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_UNIQUEID, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_UNIQUEID, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -962,14 +1032,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) nscpedn, e->ep_id, BE_INDEX_DEL, &txn); slapi_ch_free((void **)&nscpedn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_NSCP_ENTRYDN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_NSCP_ENTRYDN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -988,14 +1058,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); slapi_ch_free_string(&entryusn_str); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_ENTRYUSN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_ENTRYUSN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -1009,13 +1079,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) { retval = entryrdn_index_entry(be, e, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (deleting entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, + LDAPDebug2Args( LDAP_DEBUG_ANY, "delete (deleting entryrdn) failed, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -1032,12 +1102,12 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = modify_update_all(be,pb,&parent_modify_c,&txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "del 4 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "del 4 DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "delete 3 BAD, err=%d %s\n", + LDAPDebug( LDAP_DEBUG_ANY, "delete 3 BAD, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; DEL_SET_ERROR(ldap_result_code, @@ -1055,7 +1125,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete DEADLOCK vlv_update_index\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete DEADLOCK vlv_update_index\n", 0, 0, 0 ); /* Retry txn */ continue; } @@ -1148,14 +1218,24 @@ ldbm_back_delete( Slapi_PBlock *pb ) /* delete from cache and clean up */ if (e) { + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, delete the tombstone dn from the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); + if (bdn) { /* in the dncache, remove it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } if (e_in_cache) { CACHE_REMOVE(&inst->inst_cache, e); + cache_unlock_entry(&inst->inst_cache, e); + CACHE_RETURN(&inst->inst_cache, &e); + } else { + cache_unlock_entry(&inst->inst_cache, e); } - cache_unlock_entry(&inst->inst_cache, e); - CACHE_RETURN(&inst->inst_cache, &e); e = NULL; } - + if (ruv_c_init) { if (modify_switch_entries(&ruv_c, be) != 0 ) { ldap_result_code= LDAP_OPERATIONS_ERROR; @@ -1176,16 +1256,36 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto common_return; error_return: - if (inst && tombstone_in_cache) - { - CACHE_REMOVE( &inst->inst_cache, tombstone ); - CACHE_RETURN( &inst->inst_cache, &tombstone ); - tombstone = NULL; - tombstone_in_cache = 0; + if (tombstone) { + if (entryrdn_get_switch()) { /* subtree-rename: on */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, tombstone->ep_id); + if (bdn) { /* already in the dncache. Delete it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + if (tombstone_in_cache) { /* successfully replaced */ + CACHE_REMOVE( &inst->inst_cache, tombstone ); + CACHE_RETURN( &inst->inst_cache, &tombstone ); + tombstone = NULL; + tombstone_in_cache = 0; + } else { + backentry_free( &tombstone ); + } } - else - { - backentry_free( &tombstone ); + + /* Need to return to cache after post op plugins are called */ + if (e) { + if (e_in_cache) { + if (remove_e_from_cache) { + /* The entry is already transformed to a tombstone. */ + CACHE_REMOVE( &inst->inst_cache, e ); + } + cache_unlock_entry( &inst->inst_cache, e ); + CACHE_RETURN( &inst->inst_cache, &e ); + } else { + cache_unlock_entry( &inst->inst_cache, e ); + } } if (retval == DB_RUNRECOVERY) { @@ -1246,15 +1346,33 @@ common_return: for the post op plugins */ slapi_pblock_set( pb, SLAPI_DELETE_BEPREOP_ENTRY, orig_entry ); } - if (inst && tombstone_in_cache) - { - CACHE_RETURN( &inst->inst_cache, &tombstone ); - tombstone = NULL; - tombstone_in_cache = 0; - } - else - { - backentry_free( &tombstone ); + if (inst && tombstone) { + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, + tombstone->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *tombstonesdn = slapi_sdn_dup(slapi_entry_get_sdn(tombstone->ep_entry)); + if (tombstonesdn) { + bdn = backdn_init(tombstonesdn, tombstone->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_delete", + "set %s to dn cache\n", slapi_sdn_get_dn(tombstonesdn)); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + } + } + if (tombstone_in_cache) { /* successfully replaced */ + CACHE_RETURN( &inst->inst_cache, &tombstone ); + tombstone = NULL; + tombstone_in_cache = 0; + } else { + backentry_free( &tombstone ); + } } /* result code could be used in the bepost plugin functions. */ @@ -1267,17 +1385,6 @@ common_return: plugin_call_plugins (pb, SLAPI_PLUGIN_BE_POST_DELETE_FN); } - /* Need to return to cache after post op plugins are called */ - if (inst) { - if (retval && e) { /* error case */ - cache_unlock_entry( &inst->inst_cache, e ); - CACHE_RETURN( &inst->inst_cache, &e ); - } - if (inst->inst_ref_count) { - slapi_counter_decrement(inst->inst_ref_count); - } - } - if (ruv_c_init) { modify_term(&ruv_c, be); } @@ -1291,7 +1398,7 @@ diskfull_return: } slapi_send_ldap_result( pb, ldap_result_code, NULL, ldap_result_message, 0, NULL ); } - modify_term(&parent_modify_c,be); + modify_term(&parent_modify_c, be); if (rc == 0 && opcsn && !is_fixup_operation && !delete_tombstone_entry) { /* URP Naming Collision diff --git a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c index d381166..34b7f70 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c @@ -387,7 +387,7 @@ entryrdn_index_read_ext(backend *be, rc = slapi_rdn_init_all_sdn(&srdn, sdn); if (rc < 0) { - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, + slapi_log_error(SLAPI_LOG_BACKLDBM, ENTRYRDN_TAG, "entryrdn_index_read: Param error: Failed to convert " "%s to Slapi_RDN\n", slapi_sdn_get_dn(sdn)); rc = LDAP_INVALID_DN_SYNTAX; @@ -501,7 +501,8 @@ entryrdn_rename_subtree(backend *be, Slapi_RDN *newsrdn, /* new rdn */ const Slapi_DN *newsupsdn, /* new superior dn */ ID id, - back_txn *txn) + back_txn *txn, + int flags) { int rc = -1; struct attrinfo *ai = NULL; @@ -543,7 +544,7 @@ entryrdn_rename_subtree(backend *be, goto bail; } - rc = slapi_rdn_init_all_sdn(&oldsrdn, oldsdn); + rc = slapi_rdn_init_all_sdn_ext(&oldsrdn, oldsdn, flags); if (rc < 0) { slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, "entryrdn_rename_subtree: Failed to convert olddn " @@ -969,7 +970,8 @@ entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, - back_txn *txn) + back_txn *txn, + int flags) { int rc = -1; struct attrinfo *ai = NULL; @@ -1001,7 +1003,7 @@ entryrdn_get_subordinates(backend *be, goto bail; } - rc = slapi_rdn_init_all_sdn(&srdn, sdn); + rc = slapi_rdn_init_all_sdn_ext(&srdn, sdn, flags); if (rc) { if (rc < 0) { slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modify.c b/ldap/servers/slapd/back-ldbm/ldbm_modify.c index 0f01b2f..ac84ccb 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modify.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modify.c @@ -116,8 +116,14 @@ int modify_switch_entries(modify_context *mc,backend *be) ldbm_instance *inst = (ldbm_instance *) be->be_instance_info; int ret = 0; if (mc->old_entry!=NULL && mc->new_entry!=NULL) { - ret = cache_replace(&(inst->inst_cache), mc->old_entry, mc->new_entry); - if (ret == 0) mc->new_entry_in_cache = 1; + ret = cache_replace(&(inst->inst_cache), mc->old_entry, mc->new_entry); + if (ret == 0) { + mc->new_entry_in_cache = 1; + } else { + LDAPDebug(LDAP_DEBUG_CACHE, "modify_switch_entries: replacing %s with %s failed (%d)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), + slapi_entry_get_dn(mc->new_entry->ep_entry), ret); + } } return ret; } @@ -154,6 +160,10 @@ modify_unswitch_entries(modify_context *mc,backend *be) CACHE_RETURN( &(inst->inst_cache), &(mc->old_entry) ); mc->new_entry_in_cache = 1; mc->old_entry = NULL; + } else { + LDAPDebug(LDAP_DEBUG_CACHE, "modify_unswitch_entries: replacing %s with %s failed (%d)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), + slapi_entry_get_dn(mc->new_entry->ep_entry), ret); } } @@ -754,6 +764,12 @@ ldbm_back_modify( Slapi_PBlock *pb ) MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } + /* e uncached */ + /* we must return both e (which has been deleted) and new entry ec */ + /* cache_replace removes e from the caches */ + cache_unlock_entry( &inst->inst_cache, e ); + CACHE_RETURN( &inst->inst_cache, &e ); + /* lock new entry in cache to prevent usage until we are complete */ cache_lock_entry( &inst->inst_cache, ec ); ec_in_cache = 1; @@ -764,10 +780,6 @@ ldbm_back_modify( Slapi_PBlock *pb ) /* invalidate virtual cache */ ec->ep_entry->e_virtual_watermark = 0; - /* we must return both e (which has been deleted) and new entry ec */ - /* cache_replace removes e from the caches */ - cache_unlock_entry( &inst->inst_cache, e ); - CACHE_RETURN( &inst->inst_cache, &e ); /* * LP Fix of crash when the commit will fail: * If the commit fail, the common error path will @@ -870,28 +882,28 @@ error_return: CACHE_REMOVE( &inst->inst_cache, ec ); /* if ec was in cache, e was not - add back e */ if (e) { - CACHE_ADD( &inst->inst_cache, e, NULL ); - cache_lock_entry( &inst->inst_cache, e ); + if (CACHE_ADD( &inst->inst_cache, e, NULL )) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "ldbm_modify: CACHE_ADD %s failed\n", + slapi_entry_get_dn(e->ep_entry)); + } } } common_return: slapi_mods_done(&smods); - if (inst && ec_in_cache) - { - cache_unlock_entry( &inst->inst_cache, ec); - CACHE_RETURN( &inst->inst_cache, &ec ); - } - else - { - backentry_free(&ec); - } - if (inst) { - if (e) { - cache_unlock_entry( &inst->inst_cache, e); - CACHE_RETURN( &inst->inst_cache, &e); + if (ec_in_cache) { + cache_unlock_entry( &inst->inst_cache, ec); + CACHE_RETURN( &inst->inst_cache, &ec ); + } else { + backentry_free(&ec); + /* if ec was not in cache, cache_replace was not done. + * i.e., e was not unlocked. */ + if (e) { + cache_unlock_entry( &inst->inst_cache, e); + CACHE_RETURN( &inst->inst_cache, &e); + } } if (inst->inst_ref_count) { slapi_counter_decrement(inst->inst_ref_count); diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c index 1162fdb..78c4148 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c @@ -45,10 +45,10 @@ #include "back-ldbm.h" -static const char *moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn); +static const char *moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn, int is_tombstone); static void moddn_unlock_and_return_entry(backend *be,struct backentry **targetentry); static int moddn_newrdn_mods(Slapi_PBlock *pb, const char *olddn, struct backentry *ec, Slapi_Mods *smods_wsi, int is_repl_op); -static IDList *moddn_get_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, struct backentry *parententry, Slapi_DN *parentdn, struct backentry ***child_entries, struct backdn ***child_dns); +static IDList *moddn_get_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, struct backentry *parententry, Slapi_DN *parentdn, struct backentry ***child_entries, struct backdn ***child_dns, int is_resurect_operation); static int moddn_rename_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, IDList *children, Slapi_DN *dn_parentdn, Slapi_DN *dn_newsuperiordn, struct backentry *child_entries[]); static int modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbminfo *li, struct backentry *e, struct backentry **ec, Slapi_Mods *smods1, Slapi_Mods *smods2, Slapi_Mods *smods3, int *e_in_cache, int *ec_in_cache); static void mods_remove_nsuniqueid(Slapi_Mods *smods); @@ -110,6 +110,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) Slapi_Operation *operation; int is_replicated_operation= 0; int is_fixup_operation = 0; + int is_resurect_operation = 0; + int is_tombstone = 0; entry_address new_addr; entry_address *old_addr; entry_address oldparent_addr; @@ -136,6 +138,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_IS_REPLICATED_OPERATION, &is_replicated_operation ); is_ruv = operation_is_flag_set(operation, OP_FLAG_REPL_RUV); is_fixup_operation = operation_is_flag_set(operation, OP_FLAG_REPL_FIXUP); + is_resurect_operation = operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY); + is_tombstone = operation_is_flag_set(operation,OP_FLAG_TOMBSTONE_ENTRY); /* tombstone_to_glue on parent entry*/ if (NULL == sdn) { slapi_send_ldap_result( pb, LDAP_INVALID_DN_SYNTAX, NULL, @@ -169,7 +173,13 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, &dn_newsuperiordn ); slapi_sdn_init_dn_byref(&dn_newrdn, newrdn); /* slapi_sdn_init_normdn_byref(&dn_newsuperiordn, newsuperiordn); */ - slapi_sdn_get_parent(sdn, &dn_parentdn); + if (is_resurect_operation) { + /* no need to free this pdn. */ + const char *pdn = slapi_dn_find_parent_ext(slapi_sdn_get_dn(sdn), is_resurect_operation); + slapi_sdn_set_dn_byval(&dn_parentdn, pdn); + } else { + slapi_sdn_get_parent(sdn, &dn_parentdn); + } } /* if old and new superior are equals, newsuperior should not be set @@ -267,7 +277,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } /* make sure the original entry is back in the cache if it was removed */ if (!e_in_cache) { - CACHE_ADD(&inst->inst_cache, e, NULL); + if (CACHE_ADD(&inst->inst_cache, e, NULL)) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_back_modrdn: CACHE_ADD %s to cache failed\n", + slapi_entry_get_dn_const(e->ep_entry)); + } e_in_cache = 1; } slapi_pblock_get( pb, SLAPI_MODRDN_EXISTING_ENTRY, &ent ); @@ -310,11 +324,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; } - if (ruv_c_init) { - /* reset the ruv txn stuff */ - modify_term(&ruv_c, be); - ruv_c_init = 0; - } + if (ruv_c_init) { + /* reset the ruv txn stuff */ + modify_term(&ruv_c, be); + ruv_c_init = 0; + } /* We're re-trying */ LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "Modrdn Retrying Transaction\n"); @@ -365,7 +379,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) * it's passed to slapi_sdn_init_normdn_byref */ slapi_pblock_get(pb, SLAPI_MODRDN_NEWRDN, &newrdn); slapi_sdn_init_dn_byref(&dn_newrdn, newrdn); - newdn = moddn_get_newdn(pb,sdn, &dn_newrdn, dn_newsuperiordn); + newdn = moddn_get_newdn(pb,sdn, &dn_newrdn, dn_newsuperiordn, is_tombstone); slapi_sdn_set_dn_passin(&dn_newdn,newdn); new_addr.sdn = &dn_newdn; new_addr.udn = NULL; @@ -486,13 +500,15 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; /* error result sent by find_entry2modify() */ } e_in_cache = 1; /* e is in the cache and locked */ - if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) ) { - ldap_result_code = LDAP_UNWILLING_TO_PERFORM; + if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) && + !is_resurect_operation) { + ldap_result_code = LDAP_UNWILLING_TO_PERFORM; ldap_result_message = "Operation not allowed on tombstone entry."; - slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", - "Attempt to rename a tombstone entry %s\n", slapi_sdn_get_dn(slapi_entry_get_sdn_const( e->ep_entry ))); - goto error_return; - } + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "Attempt to rename a tombstone entry %s\n", + slapi_sdn_get_dn(slapi_entry_get_sdn_const( e->ep_entry ))); + goto error_return; + } /* Check that an entry with the same DN doesn't already exist. */ { Slapi_Entry *entry; @@ -509,7 +525,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* Fetch and lock the parent of the entry that is moving */ oldparent_addr.sdn = &dn_parentdn; - oldparent_addr.uniqueid = NULL; + if (is_resurect_operation) { + oldparent_addr.uniqueid = operation->o_params.p.p_modrdn.modrdn_newsuperior_address.uniqueid; + } else { + oldparent_addr.uniqueid = NULL; + } parententry = find_entry2modify_only( pb, be, &oldparent_addr, &txn ); modify_init(&parent_modify_context,parententry); @@ -517,7 +537,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) if(slapi_sdn_get_ndn(dn_newsuperiordn) != NULL) { slapi_pblock_get (pb, SLAPI_MODRDN_NEWSUPERIOR_ADDRESS, &newsuperior_addr); + if (is_resurect_operation) { + newsuperior_addr->uniqueid = slapi_entry_attr_get_charptr(e->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + } newparententry = find_entry2modify_only( pb, be, newsuperior_addr, &txn ); + slapi_ch_free_string(&newsuperior_addr->uniqueid); modify_init(&newparent_modify_context,newparententry); } @@ -560,8 +584,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } slapi_pblock_set( pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, dn_newsuperiordn ); } - slapi_sdn_set_dn_passin(&dn_newdn, - moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn)); + slapi_sdn_set_dn_passin(&dn_newdn, moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn, is_tombstone)); /* Check that we're allowed to add an entry below the new superior */ if ( newparententry == NULL ) @@ -662,6 +685,57 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_rdn_done(&srdn); } + if(is_resurect_operation) { + slapi_log_error(SLAPI_LOG_REPL, "ldbm_back_modrdn", + "Resurrecting an entry %s\n", slapi_entry_get_dn(ec->ep_entry)); + slapi_entry_attr_delete(ec->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + slapi_entry_delete_string(ec->ep_entry, SLAPI_ATTR_OBJECTCLASS, SLAPI_ATTR_VALUE_TOMBSTONE); + /* Now also remove the nscpEntryDN */ + if (slapi_entry_attr_delete(ec->ep_entry, SLAPI_ATTR_NSCP_ENTRYDN) != 0){ + LDAPDebug(LDAP_DEBUG_REPL, "Resurrection of %s - Couldn't remove %s\n", + slapi_entry_get_dn(ec->ep_entry), SLAPI_ATTR_NSCP_ENTRYDN, 0); + } + + /* Set the reason (this is only a reason why modrdn is needed for resurrection) */ + slapi_entry_add_string(ec->ep_entry, "nsds5ReplConflict", "deletedEntryHasChildren"); + + /* Clear the Tombstone Flag in the entry */ + slapi_entry_clear_flag(ec->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE); + + /* make sure the objectclass + - does not contain any duplicate values + - has CSNs for the new values we added + */ + { + Slapi_Attr *sa = NULL; + Slapi_Value sv; + const struct berval *svbv = NULL; + + /* add the extensibleobject objectclass with csn if not present */ + slapi_entry_attr_find(ec->ep_entry, SLAPI_ATTR_OBJECTCLASS, &sa); + slapi_value_init_string(&sv, "extensibleobject"); + svbv = slapi_value_get_berval(&sv); + if (slapi_attr_value_find(sa, svbv)) { /* not found, so add it */ + if (opcsn) { + value_update_csn(&sv, CSN_TYPE_VALUE_UPDATED, opcsn); + } + slapi_attr_add_value(sa, &sv); + } + value_done(&sv); + + /* add the glue objectclass with csn if not present */ + slapi_value_init_string(&sv, "glue"); + svbv = slapi_value_get_berval(&sv); + if (slapi_attr_value_find(sa, svbv)) { /* not found, so add it */ + if (opcsn) { + value_update_csn(&sv, CSN_TYPE_VALUE_UPDATED, opcsn); + } + slapi_attr_add_value(sa, &sv); + } + value_done(&sv); + } + } + /* create it in the cache - prevents others from creating it */ if (( cache_add_tentative( &inst->inst_cache, ec, NULL ) != 0 ) ) { ec_in_cache = 0; /* not in cache */ @@ -671,6 +745,10 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* somebody must've created it between dn2entry() and here */ /* JCMREPL - Hmm... we can't permit this to happen...? */ ldap_result_code= LDAP_ALREADY_EXISTS; + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "cache_add_tentative failed: %s\n", slapi_entry_get_dn(ec->ep_entry)); + } goto error_return; } /* so if the old dn is the same as the new dn, the entry will not be cached @@ -718,8 +796,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } } - slapi_pblock_get( pb, SLAPI_MODIFY_MODS, &mods ); - slapi_mods_init_byref(&smods_operation_wsi,mods); + slapi_pblock_get( pb, SLAPI_MODIFY_MODS, &mods ); + slapi_mods_init_byref(&smods_operation_wsi,mods); /* * We are about to pass the last abandon test, so from now on we are @@ -809,48 +887,81 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* * Update the subordinate count of the parents to reflect the moved child. */ - if ( parententry!=NULL ) - { + if (parententry) { retval = parent_update_on_childchange(&parent_modify_context, PARENTUPDATE_DEL, NULL); /* The parent modify context now contains info needed later */ - if (0 != retval) - { + if (retval) { goto error_return; } } - if ( newparententry!=NULL ) - { + if (newparententry) { retval = parent_update_on_childchange(&newparent_modify_context, PARENTUPDATE_ADD, NULL); /* The newparent modify context now contains info needed later */ - if (0 != retval) - { + if (retval) { goto error_return; } } } + /* is_resurect_operation case, there's no new superior. Just rename. */ + if (is_resurect_operation && parententry) { + retval = parent_update_on_childchange(&parent_modify_context, PARENTUPDATE_RESURECT, NULL); + if (retval) { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "parent_update_on_childchange parent %s of %s failed, rc=%d\n", + slapi_entry_get_dn_const(parent_modify_context.old_entry->ep_entry), + slapi_entry_get_dn_const(ec->ep_entry), retval); + goto error_return; + } + } /* - * If the entry has children then we're going to have to rename them all. + * If the entry has children including tombstones, + * then we're going to have to rename them all. */ - if (slapi_entry_has_children( e->ep_entry )) - { + if (slapi_entry_has_children_ext(e->ep_entry, 1)) { /* JCM - This is where the subtree lock will appear */ if (entryrdn_get_switch()) /* subtree-rename: on */ { + if (is_resurect_operation) { +#if defined(DEBUG) + /* Get the present value of the subcount attr, or 0 if not present */ + Slapi_Attr *read_attr = NULL; + int sub_count = -1; + if (0 == slapi_entry_attr_find(parent_modify_context.old_entry->ep_entry, + "numsubordinates", &read_attr)) { + /* decode the value */ + Slapi_Value *sval; + slapi_attr_first_value(read_attr, &sval); + if (sval) { + const struct berval *bval = slapi_value_get_berval(sval); + if(bval) { + sub_count = atol(bval->bv_val); + } + } + } + LDAPDebug(LDAP_DEBUG_ANY, "parent_update_on_childchange parent %s of %s numsub=%d\n", + slapi_entry_get_dn_const(parent_modify_context.old_entry->ep_entry), + slapi_entry_get_dn_const(e->ep_entry), sub_count); +#endif + slapi_log_error(SLAPI_LOG_BACKLDBM, "ldbm_back_modrdn", + "%s has children\n", slapi_entry_get_dn(e->ep_entry)); + } children = moddn_get_children(&txn, pb, be, e, sdn, - &child_entries, &child_dns); + &child_entries, &child_dns, is_resurect_operation); } else { children = moddn_get_children(&txn, pb, be, e, sdn, - &child_entries, NULL); + &child_entries, NULL, 0); } /* JCM - Shouldn't we perform an access control check on all the children. */ /* JCMREPL - But, the replication client has total rights over its subtree, so no access check needed. */ /* JCM - A subtree move could break ACIs, static groups, and dynamic groups. */ + } else if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "%s has NO children\n", slapi_entry_get_dn(e->ep_entry)); } /* @@ -904,10 +1015,10 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (retval != 0 ) + if (retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "modrdn_rename_entry_update_indexes failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + LDAPDebug(LDAP_DEBUG_ANY, "modrdn_rename_entry_update_indexes %s --> %s failed, err=%d\n", + slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn(ec->ep_entry), retval); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; @@ -1027,14 +1138,17 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_rdn_init_sdn(&newsrdn, (const Slapi_DN *)&dn_newdn); retval = entryrdn_rename_subtree(be, (const Slapi_DN *)sdn, &newsrdn, (const Slapi_DN *)dn_newsuperiordn, - e->ep_id, &txn); + e->ep_id, &txn, is_tombstone); slapi_rdn_done(&newsrdn); if (retval != 0) { if (retval == DB_LOCK_DEADLOCK) continue; if (retval == DB_RUNRECOVERY || LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - MOD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "entryrdn_rename_subtree failed (%d); dn: %s, newsrdn: %s, dn_newsuperiordn: %s\n", + retval, slapi_sdn_get_dn(sdn), slapi_rdn_get_rdn(&newsrdn), + slapi_sdn_get_dn(dn_newsuperiordn)); goto error_return; } } @@ -1060,7 +1174,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; } - if (!is_ruv && !is_fixup_operation) { + if (!is_ruv && !is_fixup_operation) { ruv_c_init = ldbm_txn_ruv_modify_context( pb, &ruv_c ); if (-1 == ruv_c_init) { LDAPDebug( LDAP_DEBUG_ANY, @@ -1160,6 +1274,12 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) * "renamed" dn is generated based upon the moved subtree. */ for (i = 0; child_entries[i] != NULL; i++) { + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "Calling cache remove & return %s (refcnt: %d)\n", + slapi_entry_get_dn(child_entries[i]->ep_entry), + child_entries[i]->ep_refcnt); + } CACHE_REMOVE( &inst->inst_cache, child_entries[i] ); cache_unlock_entry( &inst->inst_cache, child_entries[i] ); CACHE_RETURN( &inst->inst_cache, &child_entries[i] ); @@ -1196,21 +1316,13 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) error_return: /* result already sent above - just free stuff */ - if ( NULL != postentry ) - { + if (postentry) { slapi_entry_free( postentry ); postentry= NULL; /* make sure caller doesn't attempt to free this */ slapi_pblock_set( pb, SLAPI_ENTRY_POST_OP, postentry ); } - if (e && entryrdn_get_switch() && inst) - { - struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); - CACHE_REMOVE(&inst->inst_dncache, bdn); - CACHE_RETURN(&inst->inst_dncache, &bdn); - } - if(children) - { + if(children) { int i = 0; if (child_entries && *child_entries && inst) { @@ -1308,9 +1420,32 @@ common_return: /* Free up the resource we don't need any more */ if (ec) { + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_REPL, "ldbm_back_modrdn", + "Resurrecting an entry %s: result: %d, %d\n", + slapi_entry_get_dn(ec->ep_entry), ldap_result_code, retval); + } + if (inst && (0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, ec->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *ecsdn = slapi_sdn_dup(slapi_entry_get_sdn(ec->ep_entry)); + if (ecsdn) { + bdn = backdn_init(ecsdn, ec->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + CACHE_RETURN(&inst->inst_dncache, &bdn); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "set %s to dn cache\n", slapi_sdn_get_dn(sdn)); + } + } + } + } /* remove the new entry from the cache if the op failed - otherwise, leave it in */ - if (ec_in_cache && inst) { + if (ec_in_cache && ec && inst) { if (retval) { CACHE_REMOVE( &inst->inst_cache, ec ); } @@ -1322,15 +1457,15 @@ common_return: ec_in_cache = 0; } - /* put e back in the cache if the modrdn failed */ - if (e) { - if (!e_in_cache && retval && inst) { - CACHE_ADD(&inst->inst_cache, e, NULL); - e_in_cache = 1; + if (inst) { + if (e && entryrdn_get_switch() && (0 == retval)) { + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + if (inst->inst_ref_count) { + slapi_counter_decrement(inst->inst_ref_count); } - } - if (inst && inst->inst_ref_count) { - slapi_counter_decrement(inst->inst_ref_count); } moddn_unlock_and_return_entry(be,&e); @@ -1341,11 +1476,11 @@ common_return: if (ldap_result_code!=-1) { - if (not_an_error) { - /* This is mainly used by urp. Solved conflict is not an error. - * And we don't want the supplier to halt sending the updates. */ - ldap_result_code = LDAP_SUCCESS; - } + if (not_an_error) { + /* This is mainly used by urp. Solved conflict is not an error. + * And we don't want the supplier to halt sending the updates. */ + ldap_result_code = LDAP_SUCCESS; + } slapi_send_ldap_result( pb, ldap_result_code, ldap_result_matcheddn, ldap_result_message, 0,NULL ); } @@ -1394,7 +1529,7 @@ common_return: * Work out what the new DN of the entry will be. */ static const char * -moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn) +moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn, int is_tombstone) { char *newdn; const char *newrdn= slapi_sdn_get_dn(dn_newrdn); @@ -1415,18 +1550,18 @@ moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi else { /* construct the new dn */ - char *pdn; - const char *dn= slapi_sdn_get_dn(dn_olddn); - pdn = slapi_dn_beparent( pb, dn ); - if ( pdn != NULL ) - { - newdn= slapi_dn_plus_rdn(pdn, newrdn); /* JCM - Use Slapi_RDN */ - } - else - { - newdn= slapi_ch_strdup(newrdn); + const char *dn = slapi_sdn_get_dn((const Slapi_DN *)dn_olddn); + if (slapi_dn_isbesuffix(pb, dn)) { + newdn = slapi_ch_strdup(newrdn); + } else { + /* no need to free this pdn. */ + const char *pdn = slapi_dn_find_parent_ext(dn, is_tombstone); + if (pdn) { + newdn = slapi_dn_plus_rdn(pdn, newrdn); + } else { + newdn = slapi_ch_strdup(newrdn); + } } - slapi_ch_free( (void**)&pdn ); } return newdn; } @@ -1653,6 +1788,7 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm Slapi_Operation *operation; int is_ruv = 0; /* True if the current entry is RUV */ int orig_ec_in_cache = 0; + int cache_rc = 0; slapi_pblock_get( pb, SLAPI_BACKEND, &be ); slapi_pblock_get( pb, SLAPI_OPERATION, &operation ); @@ -1664,7 +1800,12 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm * Update the ID to Entry index. * Note that id2entry_add replaces the entry, so the Entry ID stays the same. */ - retval = id2entry_add( be, *ec, ptxn ); + retval = id2entry_add_ext(be, *ec, ptxn, 1, &cache_rc); + if (cache_rc) { + LDAPDebug(LDAP_DEBUG_CACHE, + "modrdn_rename_entry_update_indexes: adding %s failed to add to the cache (rc: %d, cache_rc: %d)\n", + slapi_entry_get_dn(e->ep_entry), retval, cache_rc); + } if (DB_LOCK_DEADLOCK == retval) { /* Retry txn */ @@ -1758,11 +1899,11 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm } } if (cache_replace( &inst->inst_cache, e, *ec ) != 0 ) { - LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "modrdn_rename_entry_update_indexes cache_replace failed\n"); + LDAPDebug2Args(LDAP_DEBUG_CACHE, + "modrdn_rename_entry_update_indexes cache_replace %s -> %s failed\n", + slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn((*ec)->ep_entry)); retval= -1; goto error_return; - } else { - *e_in_cache = 0; /* e un-cached */ } if (orig_ec_in_cache) { /* ec was already added to the cache via cache_add_tentative (to reserve its spot in the cache) @@ -1978,7 +2119,8 @@ moddn_get_children(back_txn *ptxn, struct backentry *parententry, Slapi_DN *dn_parentdn, struct backentry ***child_entries, - struct backdn ***child_dns) + struct backdn ***child_dns, + int is_resurect_operation) { ldbm_instance *inst = (ldbm_instance *) be->be_instance_info; int err= 0; @@ -2006,7 +2148,7 @@ moddn_get_children(back_txn *ptxn, { err = entryrdn_get_subordinates(be, slapi_entry_get_sdn_const(parententry->ep_entry), - parententry->ep_id, &candidates, ptxn); + parententry->ep_id, &candidates, ptxn, is_resurect_operation); if (err) { LDAPDebug1Arg( LDAP_DEBUG_ANY, "moddn_get_children: " "entryrdn_get_subordinates returned %d\n", err); @@ -2025,8 +2167,13 @@ moddn_get_children(back_txn *ptxn, slapi_filter_free(filter,1); } - if (candidates!=NULL) - { + if (candidates) { + Slapi_DN parentsdn = {0}; + if (is_resurect_operation) { + slapi_sdn_get_parent(dn_parentdn, &parentsdn); + dn_parentdn = &parentsdn; + } + sr_current = idl_iterator_init(candidates); result_idl= idl_alloc(candidates->b_nids); do @@ -2059,6 +2206,7 @@ moddn_get_children(back_txn *ptxn, } } while (id!=NOID); idl_free(candidates); + slapi_sdn_done(&parentsdn); } nids = result_idl ? result_idl->b_nids : 0; diff --git a/ldap/servers/slapd/back-ldbm/ldbm_search.c b/ldap/servers/slapd/back-ldbm/ldbm_search.c index 1dcf698..d0563bd 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_search.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_search.c @@ -1209,7 +1209,7 @@ subtree_candidates( /* subtree-rename: on && no ancestorid */ *err = entryrdn_get_subordinates(be, slapi_entry_get_sdn_const(e->ep_entry), - e->ep_id, &descendants, &txn); + e->ep_id, &descendants, &txn, 0); idl_insert(&descendants, e->ep_id); candidates = idl_intersection(be, candidates, descendants); idl_free(tmp); diff --git a/ldap/servers/slapd/back-ldbm/ldif2ldbm.c b/ldap/servers/slapd/back-ldbm/ldif2ldbm.c index 7d93fa0..d803f5d 100644 --- a/ldap/servers/slapd/back-ldbm/ldif2ldbm.c +++ b/ldap/servers/slapd/back-ldbm/ldif2ldbm.c @@ -900,7 +900,7 @@ static IDList *ldbm_fetch_subtrees(backend *be, char **include, int *err) */ if (entryrdn_get_noancestorid()) { /* subtree-rename: on && no ancestorid */ - *err = entryrdn_get_subordinates(be, &sdn, id, &idl, txn); + *err = entryrdn_get_subordinates(be, &sdn, id, &idl, txn, 0); } else { *err = ldbm_ancestorid_read(be, txn, id, &idl); } diff --git a/ldap/servers/slapd/back-ldbm/parents.c b/ldap/servers/slapd/back-ldbm/parents.c index 59e8763..d375519 100644 --- a/ldap/servers/slapd/back-ldbm/parents.c +++ b/ldap/servers/slapd/back-ldbm/parents.c @@ -58,11 +58,17 @@ char *tombstone_numsubordinates = LDBM_TOMBSTONE_NUMSUBORDINATES_STR; * The routine is allowed to modify the parent entry, and to return a set of * LDAPMods reflecting the changes it made. The LDAPMods array must be freed * by the called by calling ldap_free_mods(p,1) + * + * PARENTUPDATE_RESURECT == turning a tombstone into an entry + * tombstone_numsubordinates-- + * numsubordinates++ */ /* - * PARENTUPDATE_CREATE_TOMBSTONE: increment tombstone_numsubordinates + * PARENTUPDATE_CREATE_TOMBSTONE: turning an entry into a tombstone + * numsubordinates-- + * tombstone_numsubordinates++ * PARENTUPDATE_DELETE_TOMBSTONE: don't touch numsubordinates, and - * decrement tombstone_numsubordinates + * tombstone_numsubordinates-- */ int @@ -84,7 +90,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) op &= PARENTUPDATE_MASK; /* Check nobody is trying to use op == 3, it's not implemented yet */ - PR_ASSERT( (op == PARENTUPDATE_ADD) || (op == PARENTUPDATE_DEL)); + PR_ASSERT((op == PARENTUPDATE_ADD) || (op == PARENTUPDATE_DEL) || (op == PARENTUPDATE_RESURECT)); /* We want to invent a mods set to be passed to modify_apply_mods() */ @@ -114,16 +120,17 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) if (PARENTUPDATE_DELETE_TOMBSTONE != repl_op) { /* are we adding ? */ - if ( (PARENTUPDATE_ADD == op) && !already_present) { + if (((PARENTUPDATE_ADD == op) || (PARENTUPDATE_RESURECT == op)) && !already_present) { /* If so, and the parent entry does not already have a subcount * attribute, we need to add it */ mod_op = LDAP_MOD_ADD; } else if (PARENTUPDATE_DEL == op) { if (!already_present) { /* This means that something is wrong---deleting a child but no subcount present on parent */ - LDAPDebug0Args( LDAP_DEBUG_ANY, - "numsubordinates assertion failure\n" ); + LDAPDebug(LDAP_DEBUG_ANY, "Parent %s has no children. (op 0x%x, repl_op 0x%x)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), op, repl_op); slapi_mods_free(&smods); + PR_ASSERT(0); return -1; } else { if (current_sub_count == 1) { @@ -138,7 +145,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) } /* Now compute the new value */ - if (PARENTUPDATE_ADD == op) { + if ((PARENTUPDATE_ADD == op) || (PARENTUPDATE_RESURECT == op)) { current_sub_count++; } else { current_sub_count--; @@ -159,8 +166,8 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) /* tombstoneNumSubordinates is needed only when this is repl op * and a child is being deleted */ - if (repl_op && (PARENTUPDATE_DEL == op)) { - current_sub_count = LDAP_MAXINT; + current_sub_count = LDAP_MAXINT; + if ((repl_op && (PARENTUPDATE_DEL == op)) || (PARENTUPDATE_RESURECT == op)) { ret = slapi_entry_attr_find(mc->old_entry->ep_entry, tombstone_numsubordinates, &read_attr); if (0 == ret) { @@ -175,7 +182,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) } } - if (PARENTUPDATE_DELETE_TOMBSTONE == repl_op) { + if ((PARENTUPDATE_DELETE_TOMBSTONE == repl_op) || (PARENTUPDATE_RESURECT == op)) { /* deleting a tombstone entry: * reaping or manually deleting it */ if ((current_sub_count != LDAP_MAXINT) && @@ -187,9 +194,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) tombstone_numsubordinates, strlen(value_buffer), value_buffer); } - } - - if (PARENTUPDATE_CREATE_TOMBSTONE == repl_op) { + } else if (PARENTUPDATE_CREATE_TOMBSTONE == repl_op) { /* creating a tombstone entry */ if (current_sub_count != LDAP_MAXINT) { current_sub_count++; diff --git a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h index d41ce1c..df3b6f3 100644 --- a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h @@ -196,6 +196,7 @@ struct backentry *dn2entry_ext(Slapi_Backend *be, const Slapi_DN *sdn, back_txn struct backentry *dn2entry_or_ancestor(Slapi_Backend *be, const Slapi_DN *sdn, Slapi_DN *ancestor, back_txn *txn, int *err); struct backentry *dn2ancestor(Slapi_Backend *be,const Slapi_DN *sdn,Slapi_DN *ancestordn,back_txn *txn,int *err); int get_copy_of_entry(Slapi_PBlock *pb, const entry_address *addr, back_txn *txn, int plock_parameter, int must_exist); +int get_copy_of_entry_ext(Slapi_PBlock *pb, ID id, const entry_address *addr, back_txn *txn, int plock_parameter, int must_exist); void done_with_pblock_entry(Slapi_PBlock *pb, int plock_parameter); /* @@ -711,8 +712,8 @@ int entryrdn_index_entry(backend *be, struct backentry *e, int flags, back_txn * int entryrdn_index_read(backend *be, const Slapi_DN *sdn, ID *id, back_txn *txn); int entryrdn_index_read_ext(backend *be, const Slapi_DN *sdn, ID *id, int flags, back_txn *txn); -int entryrdn_rename_subtree(backend *be, const Slapi_DN *oldsdn, Slapi_RDN *newsrdn, const Slapi_DN *newsupsdn, ID id, back_txn *txn); -int entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, back_txn *txn); +int entryrdn_rename_subtree(backend *be, const Slapi_DN *oldsdn, Slapi_RDN *newsrdn, const Slapi_DN *newsupsdn, ID id, back_txn *txn, int flags); +int entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, back_txn *txn, int flags); int entryrdn_lookup_dn(backend *be, const char *rdn, ID id, char **dn, Slapi_RDN **psrdn, back_txn *txn); int entryrdn_get_parent(backend *be, const char *rdn, ID id, char **prdn, ID *pid, back_txn *txn); #endif diff --git a/ldap/servers/slapd/entry.c b/ldap/servers/slapd/entry.c index f65bea7..f91c986 100644 --- a/ldap/servers/slapd/entry.c +++ b/ldap/servers/slapd/entry.c @@ -3168,9 +3168,10 @@ slapi_entry_add_rdn_values( Slapi_Entry *e ) * Author/Modifier: RJP */ int -slapi_entry_has_children(const Slapi_Entry *entry) +slapi_entry_has_children_ext(const Slapi_Entry *entry, int include_tombstone) { Slapi_Attr *attr; + int count = 0; LDAPDebug( LDAP_DEBUG_TRACE, "=> slapi_has_children( %s )\n", slapi_entry_get_dn_const(entry), 0, 0); @@ -3180,15 +3181,34 @@ slapi_entry_has_children(const Slapi_Entry *entry) Slapi_Value *sval; slapi_attr_first_value( attr, &sval ); if(sval!=NULL) - { + { const struct berval *bval = slapi_value_get_berval( sval ); if(bval!=NULL) { /* The entry has the attribute, and it's non-zero */ - if (strcmp(bval->bv_val, "0") != 0) - { - LDAPDebug( LDAP_DEBUG_TRACE, "<= slapi_has_children 1\n", 0, 0, 0 ); - return(1); + count = strtol(bval->bv_val, (char **)NULL, 10); + if (count > 0) { + LDAPDebug1Arg( LDAP_DEBUG_TRACE, "<= slapi_has_children %d\n", count); + return count; + } + } + } + } + /*If the subordinatecount exists, and it's nonzero, then return 1.*/ + if (include_tombstone && (slapi_entry_attr_find( entry, "tombstonenumsubordinates", &attr) == 0)) + { + Slapi_Value *sval; + slapi_attr_first_value( attr, &sval ); + if(sval!=NULL) + { + const struct berval *bval = slapi_value_get_berval( sval ); + if(bval!=NULL) + { + /* The entry has the attribute, and it's non-zero */ + count = strtol(bval->bv_val, (char **)NULL, 10); + if (count > 0) { + LDAPDebug1Arg( LDAP_DEBUG_TRACE, "<= slapi_has_tombstone_children %d\n", count); + return count; } } } @@ -3197,6 +3217,12 @@ slapi_entry_has_children(const Slapi_Entry *entry) return(0); } +int +slapi_entry_has_children(const Slapi_Entry *entry) +{ + return slapi_entry_has_children_ext(entry, 0); +} + /* * Renames an entry to simulate a MODRDN operation */ diff --git a/ldap/servers/slapd/rdn.c b/ldap/servers/slapd/rdn.c index fe2fae0..6e40ed8 100644 --- a/ldap/servers/slapd/rdn.c +++ b/ldap/servers/slapd/rdn.c @@ -124,7 +124,7 @@ slapi_rdn_init_dn(Slapi_RDN *rdn,const char *dn) * 1 -- "dn" does not belong to the database; could be "rdn" */ static int -_slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn) +_slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone) { const char *dn = NULL; const char *ndn= NULL; @@ -202,9 +202,16 @@ _slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn) } /* Get the last matched position */ - if(dns) - { - rdn->rdn = slapi_ch_strdup(dns[0]); + if (dns) { + if (is_tombstone && slapi_is_special_rdn(dns[0], RDN_IS_TOMBSTONE)) { + /* merge nsuniqueid=...,<rdn> into one rdn */ + rdn->rdn = slapi_ch_smprintf("%s,%s", dns[0], dns[1]); + slapi_ch_free_string(&dns[0]); + dns[0] = slapi_ch_strdup(rdn->rdn); + charray_remove(dns, dns[1], 1); + } else { + rdn->rdn = slapi_ch_strdup(dns[0]); + } rdn->all_rdns = dns; slapi_setbit_uchar(rdn->flag,FLAG_ALL_RDNS); } @@ -238,7 +245,7 @@ slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn) slapi_rdn_init(rdn); slapi_sdn_init(&sdn); slapi_sdn_set_dn_byval(&sdn, dn); - rc = _slapi_rdn_init_all_dn_ext(rdn, (const Slapi_DN *)&sdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, (const Slapi_DN *)&sdn, 0); slapi_sdn_done(&sdn); return rc; } @@ -257,6 +264,33 @@ slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn) * 1 -- dn does not belong to the database */ int +slapi_rdn_init_all_sdn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone) +{ + int rc = 0; /* success */ + + if (NULL == rdn || NULL == sdn) + { + return -1; + } + slapi_rdn_init(rdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, sdn, is_tombstone); + return rc; +} + +/* + * This function sets DN from sdn to Slapi_RDN. + * Note: The underlying function _slapi_rdn_init_all_dn_ext checks if the DN + * is in the root or sub suffix the server owns. If it is, the root or sub + * suffix is treated as one "rdn" (e.g., "dc=sub,dc=example,dc=com") and 0 is + * returned. If it is not, the DN is separated by ',' and each string is set + * to RDN array. (e.g., input: "uid=A,ou=does_not_exist" ==> "uid=A", "ou= + * does_not_exist") and 1 is returned. + * + * Return Value: 0 -- Success + * -1 -- Error + * 1 -- dn does not belong to the database + */ +int slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn) { int rc = 0; /* success */ @@ -266,7 +300,7 @@ slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn) return -1; } slapi_rdn_init(rdn); - rc = _slapi_rdn_init_all_dn_ext(rdn, sdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, sdn, 0); return rc; } @@ -291,6 +325,17 @@ slapi_rdn_init_rdn(Slapi_RDN *rdn,const Slapi_RDN *fromrdn) } void +slapi_rdn_set_dn_ext(Slapi_RDN *rdn,const char *dn, int skip_tombstone) +{ + const char *mydn = dn; + slapi_rdn_done(rdn); + if (skip_tombstone && slapi_is_special_rdn(dn, RDN_IS_TOMBSTONE)) { + mydn = dn + slapi_uniqueIDRdnSize() + 1/*,*/; + } + slapi_rdn_init_dn(rdn, mydn); +} + +void slapi_rdn_set_dn(Slapi_RDN *rdn,const char *dn) { slapi_rdn_done(rdn); @@ -473,6 +518,33 @@ slapi_rdn_contains(Slapi_RDN *rdn, const char *type, const char *value, size_t l } int +slapi_rdn_is_multivalued(Slapi_RDN *rdn) +{ + char *p = NULL; + if (rdn && rdn->rdn) { + p = PL_strchr(rdn->rdn, '+'); + } + if (p) { + return 1; + } else { + return 0; + } +} + +/* + * Return value 1: if rdn is a conflict rdn + * 0: otherwise + */ +int +slapi_rdn_is_conflict(Slapi_RDN *rdn) +{ + if (!rdn) { + return 0; + } + return slapi_is_special_rdn(slapi_rdn_get_nrdn(rdn), RDN_IS_CONFLICT); +} + +int slapi_rdn_add(Slapi_RDN *rdn, const char *type, const char *value) { PR_ASSERT(NULL != type); @@ -484,12 +556,32 @@ slapi_rdn_add(Slapi_RDN *rdn, const char *type, const char *value) } else { - /* type=value+rdn '\0' */ - char *newrdn = slapi_create_dn_string("%s=%s+%s", type, value, rdn->rdn); + char *newrdn = NULL; + char *rp = rdn->rdn; + PRUint32 uniqueidlen = slapi_uniqueIDRdnSize(); + + if (slapi_is_special_rdn(rp, RDN_IS_TOMBSTONE)) { + char *corerp = rp + uniqueidlen + 1; + *(rp + uniqueidlen) = '\0'; + newrdn = slapi_create_dn_string("%s,%s=%s+%s", rp, type, value, corerp); + } else { + /* type=value+rdn '\0' */ + newrdn = slapi_create_dn_string("%s=%s+%s", type, value, rp); + } slapi_ch_free_string(&rdn->rdn); rdn->rdn = newrdn; } slapi_unsetbit_uchar(rdn->flag,FLAG_RDNS); + + if (rdn->all_rdns && rdn->all_rdns[0]) { + slapi_ch_free_string(&rdn->all_rdns[0]); + rdn->all_rdns[0] = slapi_ch_strdup(rdn->rdn); + } + if (rdn->all_nrdns && rdn->all_nrdns[0]) { + slapi_ch_free_string(&rdn->all_nrdns[0]); + rdn->all_nrdns[0] = slapi_ch_strdup(rdn->rdn); + slapi_dn_ignore_case(rdn->all_nrdns[0]); + } return 1; } diff --git a/ldap/servers/slapd/slapi-plugin.h b/ldap/servers/slapd/slapi-plugin.h index 8a7640e..64a5780 100644 --- a/ldap/servers/slapd/slapi-plugin.h +++ b/ldap/servers/slapd/slapi-plugin.h @@ -1922,6 +1922,17 @@ int slapi_entry_attr_has_syntax_value(const Slapi_Entry *e, const char *type, co int slapi_entry_has_children(const Slapi_Entry *e); /** + * This function determines if the specified entry has child entries + * including the tombstoned descendents. + * + * \param e Entry that you want to test for child entries. + * \param include_tombstone If non-zero, check tombstone_subordinates, too. + * \return \c 1 if the entry you supply has child entries. + * \return \c 0 if the entry you supply has child entries. + */ +int slapi_entry_has_children_ext(const Slapi_Entry *e, int include_tombstone); + +/** * This function determines if an entry is the root DSE. * * The root DSE is a special entry that contains information about the Directory @@ -3033,9 +3044,11 @@ void slapi_rdn_init_rdn(Slapi_RDN *rdn,const Slapi_RDN *fromrdn); * \see slapi_rdn_set_rdn() */ void slapi_rdn_set_dn(Slapi_RDN *rdn,const char *dn); +void slapi_rdn_set_dn_ext(Slapi_RDN *rdn,const char *dn, int skip_tombstone); Slapi_RDN *slapi_rdn_new_all_dn(const char *dn); int slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn); int slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn); +int slapi_rdn_init_all_sdn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone); /** * Sets the RDN value in a \c Slapi_RDN structure from a \c Slapi_DN. @@ -3460,6 +3473,24 @@ size_t slapi_rdn_get_size(Slapi_RDN *srdn); */ char * slapi_rdn_get_value(const char *rdn); +/** + * Check if the rdn is multivalued or not + * + * \param rdn A pointer to rdn to exam. + * \return 1, if the rdn is multi valued. + * 0, if the rdn is simgle valued. + */ +int slapi_rdn_is_multivalued(Slapi_RDN *rdn); + +/** + * Check if the rdn is a conflict rdn or not + * + * \param rdn A pointer to rdn to exam. + * \return 1, if the rdn is a conflict rdn + * 0, if the rdn is not a conflict rdn + */ +int slapi_rdn_is_conflict(Slapi_RDN *rdn); + /* * utility routines for dealing with DNs */ @@ -7519,6 +7550,20 @@ char *slapi_pr_strerror( const PRErrorCode prerrno ); */ const char *slapi_system_strerror( const int syserrno ); +/** Check if rdn is a slecial rdn/dn or not. + * + * \param rdn rdn/dn to check + * \param flags specify the type: RDN_IS_TOMBSTONE or RDN_IS_CONFLICT or 0 + * + * \return 1 if rdn matches the flag. + * If flag is IS_TOMBSTONE, returns 1 if rdn is a tombstone rdn/dn. + * If flag is IS_CONFLICT, returns 1 if rdn is a conflict rdn/dn. + * \return 0 otherwise + */ +#define RDN_IS_TOMBSTONE 0x1 +#define RDN_IS_CONFLICT 0x2 +int slapi_is_special_rdn(const char *rdn, int flag); + #ifdef __cplusplus } #endif diff --git a/ldap/servers/slapd/slapi-private.h b/ldap/servers/slapd/slapi-private.h index 5fa172a..3712e65 100644 --- a/ldap/servers/slapd/slapi-private.h +++ b/ldap/servers/slapd/slapi-private.h @@ -1012,6 +1012,13 @@ int slapi_uniqueIDIsUUID(const Slapi_UniqueID *uId); */ int slapi_uniqueIDSize( void ); +/* Name: slapi_uniqueIDRdnSize + Description: returns size of SLAPI_ATTR_UNIQUEID=slapi_uniqueIDSize() + Parameters: none + Return: size of the string version of "SLAPI_ATTR_UNIQUEID=uniqueID" in bytes + */ +int slapi_uniqueIDRdnSize( void ); + /* Name: slapi_uniqueIDDup Description: duplicates an UniqueID object Parameters: uId - id to duplicate diff --git a/ldap/servers/slapd/util.c b/ldap/servers/slapd/util.c index cd69a45..880700c 100644 --- a/ldap/servers/slapd/util.c +++ b/ldap/servers/slapd/util.c @@ -1405,3 +1405,70 @@ slapi_str_to_u64(const char *s) (v6 << 36) | (v7 << 32) | (v8 << 28) | (v9 << 24) | (v10 << 20) | (v11 << 16) | (v12 << 12) | (v13 << 8) | (v14 << 4) | v15; } + +/* + * Check if rdn is a slecial rdn/dn or not. + * If flag is IS_TOMBSTONE, returns 1 if rdn is a tombstone rdn/dn. + * If flag is IS_CONFLICT, returns 1 if rdn is a conflict rdn/dn. + * Otherwise returns 0. + */ +static int util_uniqueidlen = 0; +int +slapi_is_special_rdn(const char *rdn, int flag) +{ + char *rp; + int plus = 0; + if (!util_uniqueidlen) { + util_uniqueidlen = SLAPI_ATTR_UNIQUEID_LENGTH + slapi_uniqueIDSize() + 1/*=*/; + } + + if ((RDN_IS_TOMBSTONE != flag) && (RDN_IS_CONFLICT != flag)) { + LDAPDebug1Arg(LDAP_DEBUG_ANY, "slapi_is_special_rdn: invalid flag %d\n", flag); + return 0; /* not a special rdn/dn */ + } + if (!rdn) { + LDAPDebug0Args(LDAP_DEBUG_ANY, "slapi_is_special_rdn: NULL rdn\n"); + return 0; /* not a special rdn/dn */ + } + + if (strlen(rdn) < util_uniqueidlen) { + return 0; /* not a special rdn/dn */ + } + rp = (char *)rdn; + while (rp) { + char *comma = NULL; + if (!PL_strncasecmp(rp, SLAPI_ATTR_UNIQUEID, SLAPI_ATTR_UNIQUEID_LENGTH) && + (*(rp + SLAPI_ATTR_UNIQUEID_LENGTH) == '=')) { + if (RDN_IS_TOMBSTONE == flag) { + if ((*(rp + util_uniqueidlen) == ',') || + (*(rp + util_uniqueidlen) == '\0')) { + return 1; + } else { + return 0; + } + } else { + if ((*(rp + util_uniqueidlen) == '+') || + (plus && ((*(rp + util_uniqueidlen) == ',') || + (*(rp + util_uniqueidlen) == '\0')))) { + return 1; + } + } + } + comma = PL_strchr(rp, ','); + rp = PL_strchr(rp, '+'); + if (rp && (rp < comma)) { + plus = 1; + rp++; + } + } + return 0; +} + +int +slapi_uniqueIDRdnSize() +{ + if (!util_uniqueidlen) { + util_uniqueidlen = SLAPI_ATTR_UNIQUEID_LENGTH + slapi_uniqueIDSize() + 1/*=*/; + } + return util_uniqueidlen; +}
1
0
0
0
Branch '389-ds-base-1.3.2' - ldap/servers
by Noriko Hosoi
12 May '14
12 May '14
ldap/servers/plugins/replication/repl5.h | 1 ldap/servers/plugins/replication/urp.c | 94 +++--- ldap/servers/plugins/replication/urp.h | 5 ldap/servers/plugins/replication/urp_glue.c | 32 +- ldap/servers/plugins/replication/urp_tombstone.c | 56 +++ ldap/servers/slapd/add.c | 15 - ldap/servers/slapd/back-ldbm/back-ldbm.h | 3 ldap/servers/slapd/back-ldbm/backentry.c | 1 ldap/servers/slapd/back-ldbm/cache.c | 103 ++++-- ldap/servers/slapd/back-ldbm/ldbm_add.c | 338 ++++++++++++++-------- ldap/servers/slapd/back-ldbm/ldbm_delete.c | 343 +++++++++++++++-------- ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c | 12 ldap/servers/slapd/back-ldbm/ldbm_modify.c | 54 ++- ldap/servers/slapd/back-ldbm/ldbm_modrdn.c | 320 +++++++++++++++------ ldap/servers/slapd/back-ldbm/ldbm_search.c | 2 ldap/servers/slapd/back-ldbm/ldif2ldbm.c | 2 ldap/servers/slapd/back-ldbm/parents.c | 31 +- ldap/servers/slapd/back-ldbm/proto-back-ldbm.h | 5 ldap/servers/slapd/entry.c | 38 ++ ldap/servers/slapd/rdn.c | 108 ++++++- ldap/servers/slapd/slapi-plugin.h | 45 +++ ldap/servers/slapd/slapi-private.h | 7 ldap/servers/slapd/util.c | 67 ++++ 23 files changed, 1197 insertions(+), 485 deletions(-) New commits: commit 3f933fe1b930feae3f76a729a7314a621c8119b9 Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Fri May 9 09:59:41 2014 -0700 Ticket #47750 - Creating a glue fails if one above level is a conflict or missing; Ticket #47696 - Large Searches Hang - Possibly entryrdn related Bug Descriptions: There were multiple issues to break the parent-child relationship when a resurrect occurred. When resurrecting a tombstone, there is a possibility that an entry having the same name was already added. In the case, the code consdered the entry is already there, which confused the numsubordinates and tombstone-num- subordinates count. If it is detected, it creates a conflict glue entry instead of a pure glue entry. The 2 numsubordinates counts were also not correctly updated in the resurrection case. When the entryrdn was introduced, Slapi_RDN was held in the entry, which was not correctly updated in the resurrect case. Fix Descriptions: [replication/urp.c] urp_fixup_rename_entry sets the parent uniqueid if it is a RESURRECT mode (called from tombstone_to_glue) as being done in urp_fixup_add_entry, which is necessary to retrieve the correct parent in ldbm_back_modrdn. [replication/urp_glue.c] In create_glue_entry, . there was a possibility to pass a tombstone DN (nsuniqueid=...,<DN>) is used to create a glue entry. If found, the part is removed before passing it to create a glue entry. . there is a possibility to create a glue entry fails due to the parent entry not found. This occurs when the parent entry has turned to be a conflict entry. In the case, instead of retrying the glue creation, it logs an error and returns. [replication/urp_tombstone.c] In tombstone_to_glue, . turning a tombstone to an ordinary glue entry could fail with the entry already exists. (Note: ldbm_back_add guarantees that the found entry has a different nsuniqueid from this tombstone has.) This situation occurs an entry having the same dn could have been added/replicated by this moment. And the entry cannot be used for this tombstone to resurrect since the subordinates/tombstonesubordinates relationships get corrupted. To maintain the relationship properly, the tombstone is resurrected as a conflict, glue entry using urp_fixup_rename_entry. If it still returns LDAP_ALREADY_EXISTS, it is assured the expected glue entry exists since the same nsuniqueid is shared. [slapd/back-ldbm/ldbm_add.c] In ldbm_back_add, . in is_resurect_operation case, a SLAPI_ADD_EXISTING_DN_ENTRY entry and a SLAPI_ADD_EXISTING_UNIQUEID_ENTRY entry are identical, it returns SUCCESS instead of LDAP_ALREADY_EXISTS. . in is_resurect_operation case, the DN is modified from the tombstone DN to the ordinary DN. The DN is stored in e_sdn as well as in e_srdn in the Slapi_Entry. The e_srdn part was not updated. This patch fixes it and e_sdn and e_srdn are in sync. . parent_update_on_childchange adjusts the count of numsubordinates and tombstonenumsubordinates. It only took PARENTUPDATE_ADD|_DELETE operations. This patch adds PARENTUPDATE_RESURECT operation to increment numsubordinates and decrement tombstonenumsubordinates at the same time. . in is_resurect_operation case, a tombstone to be replaced with a glue entry needs to be deleted from the entryrdn index, which was missing. [slapd/back-ldbm/ldbm_delete.c] . Check the target DN if it starts with "nsuniqueid=...,", if it is, log an error and returns. [slapd/back-ldbm/ldbm_modrdn.c] . If it is is_resurect_operation case, it allows to rename a tombstone entry. This is used to support to change a tombstone entry to a conflict glue entry in tombstone_to_glue. . ldbm_back_modify checks if the renamed entry has children or not and if there are, they are updated to point to the renamed entry. The check function slapi_entry_has_children was just checking the numsubordinates attribute type. If a positive value is returned, it took care all the children including the tombstones, but if the renamed entry had tombstoned children only, it considered there were no children and the tombstoned children were not updated to point the renamed entry. This patch fixes it. [slapd/back-ldbm/ldbm_entryrdn.c] . Added a flag to entryrdn_rename_subtree and entryrdn_get_subordinates to treat the given DN a tombstoned DN if it is. In addition, helper APIs such as slapi_rdn_is_multivalued, slapi_rdn_is_ conflict, and slapi_is_special_rdn are added. Ported from 389-ds-base-1.2.11.
https://fedorahosted.org/389/ticket/47750
Reviewed by mreynolds(a)redhat.com (Thank you, Mark!!) (cherry picked from commit 160cb3f686e433c01532d28770b2977ec957e73e) diff --git a/ldap/servers/plugins/replication/repl5.h b/ldap/servers/plugins/replication/repl5.h index f841afe..d91dbeb 100644 --- a/ldap/servers/plugins/replication/repl5.h +++ b/ldap/servers/plugins/replication/repl5.h @@ -60,6 +60,7 @@ #include "llist.h" #include "repl5_ruv.h" #include "cl4.h" +#include "plstr.h" #define START_UPDATE_DELAY 2 /* 2 second */ #define REPLICA_TYPE_WINDOWS 1 diff --git a/ldap/servers/plugins/replication/urp.c b/ldap/servers/plugins/replication/urp.c index b6fe2ff..d867274 100644 --- a/ldap/servers/plugins/replication/urp.c +++ b/ldap/servers/plugins/replication/urp.c @@ -58,7 +58,6 @@ static int urp_naming_conflict_removal (Slapi_PBlock *pb, char *sessionid, CSN * static int mod_namingconflict_attr (const char *uniqueid, const Slapi_DN *entrysdn, const Slapi_DN *conflictsdn, CSN *opcsn, const char *optype); static int del_replconflict_attr (const Slapi_Entry *entry, CSN *opcsn, int opflags); static char *get_dn_plus_uniqueid(char *sessionid,const Slapi_DN *oldsdn,const char *uniqueid); -static char *get_rdn_plus_uniqueid(char *sessionid,const char *olddn,const char *uniqueid); static int is_suffix_entry (Slapi_PBlock *pb, Slapi_Entry *entry, Slapi_DN **parenddn); /* @@ -388,6 +387,9 @@ urp_modrdn_operation( Slapi_PBlock *pb ) slapi_entry_get_sdn (target_entry), "renameTombstone", opcsn); */ op_result = LDAP_NO_SUCH_OBJECT; + slapi_log_error(SLAPI_LOG_REPL, sessionid, + "urp_modrdn: target_entry %s is a tombstone; returning LDAP_NO_SUCH_OBJECT.\n", + slapi_entry_get_dn((Slapi_Entry *)target_entry)); slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); if (op_result == 0) { @@ -473,10 +475,9 @@ urp_modrdn_operation( Slapi_PBlock *pb ) mod_namingconflict_attr (op_uniqueid, target_sdn, existing_sdn, opcsn, "MODRDN"); slapi_pblock_set(pb, SLAPI_MODRDN_NEWRDN, newrdn_with_uniqueid); slapi_log_error(slapi_log_urp, sessionid, - "urp_modrdn: Naming conflict MODRDN. Rename target entry to %s\n", - newrdn_with_uniqueid ); - - rc= slapi_setbit_int(rc,SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); + "urp_modrdn: Naming conflict MODRDN. Rename target entry from %s to %s\n", + newrdn, newrdn_with_uniqueid ); + rc= slapi_setbit_int(rc, SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); PROFILE_POINT; /* ModRDN Conflict; Entry with Target DN Exists; Rename Operation Entry */ goto bailout; } @@ -628,12 +629,15 @@ urp_delete_operation( Slapi_PBlock *pb ) op_result= LDAP_NO_SUCH_OBJECT; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc = SLAPI_PLUGIN_FAILURE; /* Don't apply the Delete */ + slapi_log_error(slapi_log_urp, sessionid, + "Entry %s does not exist; returning NO_SUCH_OBJECT.\n", + slapi_entry_get_dn((Slapi_Entry *)deleteentry)); PROFILE_POINT; /* Delete Operation; Entry not exist. */ } else if(is_tombstone_entry(deleteentry)) { /* The entry is already a Tombstone, ignore this delete. */ - op_result= LDAP_SUCCESS; + op_result= LDAP_ALREADY_EXISTS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc = SLAPI_PLUGIN_NOOP; /* Don't apply the Delete */ slapi_log_error(slapi_log_urp, sessionid, @@ -811,7 +815,7 @@ urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *pa } int -urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflags) +urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags) { Slapi_PBlock *newpb; Slapi_Operation *op; @@ -839,7 +843,12 @@ urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflag opcsn = (CSN *)entry_get_dncsn (entry); slapi_pblock_get (newpb, SLAPI_OPERATION, &op); operation_set_csn (op, opcsn); - + if (parentuniqueid) + { + struct slapi_operation_parameters *op_params; + slapi_pblock_get( newpb, SLAPI_OPERATION_PARAMETERS, &op_params ); + op_params->p.p_modrdn.modrdn_newsuperior_address.uniqueid = (char*)parentuniqueid; + } slapi_modrdn_internal_pb(newpb); slapi_pblock_get(newpb, SLAPI_PLUGIN_INTOP_RESULT, &op_result); @@ -1035,16 +1044,17 @@ urp_annotate_dn (char *sessionid, const Slapi_Entry *entry, CSN *opcsn, const ch basesdn = slapi_entry_get_sdn_const (entry); basedn = slapi_entry_get_dn_const (entry); newrdn = get_rdn_plus_uniqueid ( sessionid, basedn, uniqueid ); - if(newrdn!=NULL) - { + if(newrdn) { mod_namingconflict_attr (uniqueid, basesdn, basesdn, opcsn, optype); - op_result = urp_fixup_rename_entry ( entry, newrdn, 0 ); + slapi_log_error(slapi_log_urp, sessionid, + "urp_annotate_dn: %s --> %s\n", basedn, newrdn); + op_result = urp_fixup_rename_entry ( entry, newrdn, NULL, 0 ); switch(op_result) { case LDAP_SUCCESS: slapi_log_error(slapi_log_urp, sessionid, - "Naming conflict %s. Renamed existing entry to %s\n", - optype, newrdn); + "Naming conflict %s. Renamed existing entry to %s\n", + optype, newrdn); rc = 1; break; case LDAP_NO_SUCH_OBJECT: @@ -1064,16 +1074,21 @@ urp_annotate_dn (char *sessionid, const Slapi_Entry *entry, CSN *opcsn, const ch * for a conflict!! After fix for 558293, this * state can't be reproduced anymore (5-Oct-01) */ - slapi_log_error( SLAPI_LOG_FATAL, sessionid, - "Entry %s exists in cache but not in DB\n", - basedn ); + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Entry %s exists in cache but not in DB\n", + basedn); rc = LDAP_NO_SUCH_OBJECT; break; default: - slapi_log_error( slapi_log_urp, sessionid, - "Failed to annotate %s, err=%d\n", newrdn, op_result); + slapi_log_error(slapi_log_urp, sessionid, + "Failed to annotate %s, err=%d\n", + newrdn, op_result); } slapi_ch_free ( (void**)&newrdn ); + } else { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to create conflict DN from basedn: %s and uniqueid: %s\n", + basedn, uniqueid ); } return rc; } @@ -1218,7 +1233,7 @@ urp_naming_conflict_removal ( Slapi_PBlock *pb, char *sessionid, CSN *opcsn, con * is done after DB lock was released. The backend modrdn * will acquire the DB lock if it sees this flag. */ - op_result = urp_fixup_rename_entry((const Slapi_Entry *)min_naming_conflict_entry, newrdnstr, OP_FLAG_ACTION_INVOKE_FOR_REPLOP); + op_result = urp_fixup_rename_entry((const Slapi_Entry *)min_naming_conflict_entry, newrdnstr, NULL, OP_FLAG_ACTION_INVOKE_FOR_REPLOP); if ( op_result != LDAP_SUCCESS ) { slapi_log_error (slapi_log_urp, sessionid, @@ -1251,15 +1266,19 @@ bailout: static char * get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *uniqueid) { - Slapi_RDN *rdn= slapi_rdn_new(); - char *newdn; + Slapi_RDN *rdn = slapi_rdn_new(); + char *newdn = NULL; + int rc = slapi_rdn_init_all_sdn_ext(rdn, oldsdn, 1); + if (rc) { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to convert %s to RDN\n", slapi_sdn_get_dn(oldsdn)); + goto bail; + } PR_ASSERT(uniqueid!=NULL); /* Check if the RDN already contains the Unique ID */ - slapi_rdn_set_dn(rdn, slapi_sdn_get_dn(oldsdn)); - if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid))) - { + if (slapi_rdn_is_conflict(rdn)) { /* The Unique ID is already in the RDN. * This is a highly improbable collision. * It suggests that a duplicate UUID was generated. @@ -1268,7 +1287,6 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique */ slapi_log_error(SLAPI_LOG_FATAL, sessionid, "Annotated DN %s has naming conflict\n", slapi_sdn_get_dn(oldsdn) ); - newdn= NULL; } else { @@ -1282,21 +1300,27 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique newdn = slapi_ch_smprintf("%s,%s", slapi_rdn_get_rdn(rdn), parentdn); slapi_ch_free_string(&parentdn); } +bail: slapi_rdn_free(&rdn); return newdn; } -static char * +char * get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid) { - char *newrdn; + char *newrdn = NULL; /* Check if the RDN already contains the Unique ID */ - Slapi_DN *sdn= slapi_sdn_new_dn_byval(olddn); - Slapi_RDN *rdn= slapi_rdn_new(); - slapi_sdn_get_rdn(sdn,rdn); + Slapi_DN *sdn = slapi_sdn_new_dn_byval(olddn); + Slapi_RDN *rdn = slapi_rdn_new(); + int rc = slapi_rdn_init_all_sdn_ext(rdn, sdn, 1); + if (rc) { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to convert %s to RDN\n", olddn); + goto bail; + } + PR_ASSERT(uniqueid!=NULL); - if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid))) - { + if (slapi_rdn_is_conflict(rdn)) { /* The Unique ID is already in the RDN. * This is a highly improbable collision. * It suggests that a duplicate UUID was generated. @@ -1304,14 +1328,14 @@ get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid) * require admin intercession */ slapi_log_error(SLAPI_LOG_FATAL, sessionid, - "Annotated DN %s has naming conflict\n", olddn ); - newrdn= NULL; + "Annotated RDN %s has naming conflict\n", olddn); } else { slapi_rdn_add(rdn,SLAPI_ATTR_UNIQUEID,uniqueid); - newrdn= slapi_ch_strdup(slapi_rdn_get_rdn(rdn)); + newrdn = slapi_ch_strdup(slapi_rdn_get_rdn(rdn)); } +bail: slapi_sdn_free(&sdn); slapi_rdn_free(&rdn); return newrdn; diff --git a/ldap/servers/plugins/replication/urp.h b/ldap/servers/plugins/replication/urp.h index 26d7c75..8afd7d1 100644 --- a/ldap/servers/plugins/replication/urp.h +++ b/ldap/servers/plugins/replication/urp.h @@ -55,11 +55,12 @@ int urp_delete_operation( Slapi_PBlock *pb ); int urp_post_delete_operation( Slapi_PBlock *pb ); int urp_modrdn_operation( Slapi_PBlock *pb ); int urp_post_modrdn_operation( Slapi_PBlock *pb ); +char *get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid); /* urp internal ops */ int urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *parentuniqueid, CSN *opcsn, int opflags); int urp_fixup_delete_entry (const char *uniqueid, const char *dn, CSN *opcsn, int opflags); -int urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflags); +int urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags); int urp_fixup_modify_entry (const char *uniqueid, const Slapi_DN *sdn, CSN *opcsn, Slapi_Mods *smods, int opflags); int is_suffix_dn (Slapi_PBlock *pb, const Slapi_DN *dn, Slapi_DN **parenddn); @@ -78,6 +79,6 @@ PRBool get_glue_csn(const Slapi_Entry *entry, const CSN **gluecsn); * urp_tombstone.c */ int is_tombstone_entry(const Slapi_Entry* entry); -int tombstone_to_glue(Slapi_PBlock *pb, const char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn); +int tombstone_to_glue(Slapi_PBlock *pb, char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn); int entry_to_tombstone ( Slapi_PBlock *pb, Slapi_Entry *entry ); PRBool get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn); diff --git a/ldap/servers/plugins/replication/urp_glue.c b/ldap/servers/plugins/replication/urp_glue.c index 6eeb826..199d4de 100644 --- a/ldap/servers/plugins/replication/urp_glue.c +++ b/ldap/servers/plugins/replication/urp_glue.c @@ -153,7 +153,7 @@ static const char *glue_entry = "objectclass: top\n" "objectclass: extensibleObject\n" /* JCMREPL - To avoid schema checking. */ "objectclass: glue\n" - "nsuniqueid: %s\n" + "nsuniqueid: %s\n" /* this uniqueid is set to Slapi_Entry in slapi_str2entry. */ "%s: %s\n"; /* Add why it's been created */ static int @@ -166,15 +166,15 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha Slapi_DN *sdn = NULL; Slapi_RDN *newrdn = slapi_rdn_new_rdn(rdn); char *estr, *rdnstr, *rdntype, *rdnval, *rdnpair; - sdn = slapi_sdn_new_dn_byval(slapi_sdn_get_ndn(superiordn)); + sdn = slapi_sdn_new_ndn_byval(slapi_sdn_get_ndn(superiordn)); slapi_sdn_add_rdn(sdn,rdn); /* must take care of multi-valued rdn: split rdn into different lines introducing * '\n' between each type/value pair. - */ + */ alloc_len = RDNBUFSIZE; rdnstr = slapi_ch_malloc(alloc_len); - rdnpair = rdnstr; + rdnpair = rdnstr; *rdnpair = '\0'; /* so that strlen(rdnstr) may return 0 the first time it's called */ while ((rdnval_index = slapi_rdn_get_next(newrdn, rdnval_index, &rdntype, &rdnval)) != -1) { rdntype_len = strlen(rdntype); @@ -188,7 +188,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha } slapi_ldif_put_type_and_value_with_options(&rdnpair, rdntype, rdnval, rdnval_len, LDIF_OPT_NOWRAP); *rdnpair = '\0'; - } + } estr= slapi_ch_smprintf(glue_entry, slapi_sdn_get_ndn(sdn), rdnstr, uniqueid, ATTR_NSDS5_REPLCONFLICT, reason); slapi_ch_free((void**)&rdnstr); @@ -196,9 +196,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha slapi_ch_free((void**)&newrdn); e = slapi_str2entry( estr, 0 ); PR_ASSERT(e!=NULL); - if ( e!=NULL ) - { - slapi_entry_set_uniqueid (e, slapi_ch_strdup(uniqueid)); + if (e) { op_result = urp_fixup_add_entry (e, NULL, NULL, opcsn, 0); } slapi_ch_free_string(&estr); @@ -232,7 +230,7 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char slapi_pblock_get( pb, SLAPI_BACKEND, &backend ); slapi_sdn_get_backend_parent ( dn, superiordn, backend ); - slapi_sdn_get_rdn ( dn, rdn ); + slapi_rdn_set_dn_ext(rdn, slapi_sdn_get_dn(dn), 1/* skip nsuniqeid=..., in dn */); while(!done) { @@ -246,16 +244,30 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char done= 1; break; case LDAP_ALREADY_EXISTS: + { + struct slapi_operation_parameters *op_params; + /* This is okay. While creating a glue, a real entry was added. */ slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name, "%s: Skipped creating glue entry %s uniqueid=%s reason Entry Already Exists\n", sessionid, dnstr, uniqueid); + op_result = LDAP_SUCCESS; + /* If we could not create a glue having the nsuniqueid, + * we have to abandon it. */ + slapi_pblock_get(pb, SLAPI_OPERATION_PARAMETERS, &op_params); + slapi_ch_free_string(&op_params->p.p_add.parentuniqueid); done= 1; break; + } case LDAP_NO_SUCH_OBJECT: /* The parent is missing */ { /* JCMREPL - Create the parent ... recursion?... but what's the uniqueid? */ - PR_ASSERT(0); /* JCMREPL */ + slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, + "%s: Can't created glue entry %s uniqueid=%s, error %d; " + "Possibly, parent entry is a conflict entry.\n", + sessionid, dnstr, uniqueid, op_result); + done= 1; + break; } default: slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name, diff --git a/ldap/servers/plugins/replication/urp_tombstone.c b/ldap/servers/plugins/replication/urp_tombstone.c index a61f1e5..5a37ca8 100644 --- a/ldap/servers/plugins/replication/urp_tombstone.c +++ b/ldap/servers/plugins/replication/urp_tombstone.c @@ -84,7 +84,7 @@ get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn) static int tombstone_to_glue_resolve_parent ( Slapi_PBlock *pb, - const char *sessionid, + char *sessionid, const Slapi_DN *parentdn, const char *parentuniqueid, CSN *opcsn) @@ -144,7 +144,7 @@ tombstone_to_glue_resolve_parent ( int tombstone_to_glue ( Slapi_PBlock *pb, - const char *sessionid, + char *sessionid, Slapi_Entry *tombstoneentry, const Slapi_DN *tombstonedn, const char *reason, @@ -156,6 +156,7 @@ tombstone_to_glue ( Slapi_Entry *addingentry; const char *addingdn; int op_result; + int rdn_is_conflict = 0; /* JCMREPL * Nothing logged to the 5.0 Change Log @@ -170,18 +171,19 @@ tombstone_to_glue ( * which won't help us identify the correct backend to search. */ is_suffix_dn_ext (pb, tombstonedn, &parentdn, 1 /* is_tombstone */); - parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, - SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */ + parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */ tombstone_to_glue_resolve_parent (pb, sessionid, parentdn, parentuniqueid, opcsn); - slapi_sdn_free(&parentdn); - /* Submit an Add operation to turn the tombstone entry into glue. */ + /* Submit an Add operation to turn the tombstone entry into glue. */ /* * The tombstone is stored with an invalid DN, we must fix this. */ addingentry = slapi_entry_dup(tombstoneentry); addingdn = slapi_sdn_get_dn(tombstonedn); slapi_entry_set_sdn(addingentry, tombstonedn); + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry), slapi_entry_get_dn_const(addingentry)); + rdn_is_conflict = slapi_rdn_is_conflict(slapi_entry_get_srdn(addingentry)); if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) { @@ -189,18 +191,58 @@ tombstone_to_glue ( slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason); } tombstoneuniqueid= slapi_entry_get_uniqueid(tombstoneentry); - op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, parentuniqueid, opcsn, OP_FLAG_RESURECT_ENTRY); + /* + * addingentry and parentuniqueid are consumed in urp_fixup_add_entry, + * regardless of the result. + * Note: addingentry is not really consumed in ldbm_back_add. + * tombstoneentry from DB/entry cache is duplicated and turned to be a glue. + * This addingentry is freed in op_shared_add. + */ + op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, slapi_ch_strdup(parentuniqueid), opcsn, OP_FLAG_RESURECT_ENTRY); + if ((LDAP_ALREADY_EXISTS == op_result) && !rdn_is_conflict) { + /* conflict -- there's already the same named entry added. + * But this to-be-glued entry needs to be added since this is a parent of child entries... + * So, rename this tombstone parententry a conflict, glue entry. + * Instead of "fixup_add", we have to "fixup_rename"... + * */ + char *conflictrdn = get_rdn_plus_uniqueid(sessionid, addingdn, tombstoneuniqueid); + if (conflictrdn) { + addingentry = slapi_entry_dup(tombstoneentry); + if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) { + /* Add the reason of turning it to glue - The backend code will use it*/ + slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason); + } + slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, + "%s: Can't resurrect tombstone to glue reason '%s'. " + "Try with conflict dn %s, error=%d\n", + sessionid, reason, addingdn, op_result); + op_result = urp_fixup_rename_entry(addingentry, (const char *)conflictrdn, parentuniqueid, + OP_FLAG_RESURECT_ENTRY|OP_FLAG_TOMBSTONE_ENTRY); + slapi_ch_free_string(&conflictrdn); + slapi_entry_free(addingentry); + addingentry = NULL; + } + } + slapi_ch_free_string(&parentuniqueid); if (op_result == LDAP_SUCCESS) { slapi_log_error (slapi_log_urp, repl_plugin_name, "%s: Resurrected tombstone %s to glue reason '%s'\n", sessionid, addingdn, reason); } + else if (LDAP_ALREADY_EXISTS == op_result) + { + slapi_log_error(slapi_log_urp, repl_plugin_name, + "%s: No need to turn tombstone %s to glue; it was already resurrected.\n", + sessionid, addingdn); + op_result = LDAP_SUCCESS; + } else { slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, "%s: Can't resurrect tombstone %s to glue reason '%s', error=%d\n", sessionid, addingdn, reason, op_result); } + slapi_sdn_free(&parentdn); return op_result; } diff --git a/ldap/servers/slapd/add.c b/ldap/servers/slapd/add.c index e98f197..fadc56b 100644 --- a/ldap/servers/slapd/add.c +++ b/ldap/servers/slapd/add.c @@ -753,10 +753,17 @@ static void op_shared_add (Slapi_PBlock *pb) slapi_pblock_get(pb, SLAPI_ENTRY_POST_OP, &pse); do_ps_service(pse, NULL, LDAP_CHANGETYPE_ADD, 0); - - /* If be_add succeeded, then e is consumed. We - * set e to NULL to prevent freeing it ourselves. */ - e = NULL; + /* + * If be_add succeeded, then e is consumed except the resurect case. + * If it is resurect, the corresponding tombstone entry is resurected + * and put into the cache. + * Otherwise, we set e to NULL to prevent freeing it ourselves. + */ + if (operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY) && save_e) { + e = save_e; + } else { + e = NULL; + } } else { diff --git a/ldap/servers/slapd/back-ldbm/back-ldbm.h b/ldap/servers/slapd/back-ldbm/back-ldbm.h index 0834a1c..bd84e0f 100644 --- a/ldap/servers/slapd/back-ldbm/back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/back-ldbm.h @@ -902,7 +902,8 @@ typedef struct _back_search_result_set /* operation for parent_update_on_childchange */ #define PARENTUPDATE_ADD 0x1 #define PARENTUPDATE_DEL 0x2 -#define PARENTUPDATE_MASK (PARENTUPDATE_ADD|PARENTUPDATE_DEL) +#define PARENTUPDATE_RESURECT 0x4 +#define PARENTUPDATE_MASK (PARENTUPDATE_ADD|PARENTUPDATE_DEL|PARENTUPDATE_RESURECT) #define PARENTUPDATE_CREATE_TOMBSTONE 0x10 #define PARENTUPDATE_DELETE_TOMBSTONE 0x20 diff --git a/ldap/servers/slapd/back-ldbm/backentry.c b/ldap/servers/slapd/back-ldbm/backentry.c index 52468ad..114c2df 100644 --- a/ldap/servers/slapd/back-ldbm/backentry.c +++ b/ldap/servers/slapd/back-ldbm/backentry.c @@ -52,6 +52,7 @@ backentry_free( struct backentry **bep ) return; } ep = *bep; + PR_ASSERT(ep->ep_state & (ENTRY_STATE_DELETED|ENTRY_STATE_NOTINCACHE)); if ( ep->ep_entry != NULL ) { slapi_entry_free( ep->ep_entry ); } diff --git a/ldap/servers/slapd/back-ldbm/cache.c b/ldap/servers/slapd/back-ldbm/cache.c index 7f3f025..0309fcf 100644 --- a/ldap/servers/slapd/back-ldbm/cache.c +++ b/ldap/servers/slapd/back-ldbm/cache.c @@ -890,7 +890,7 @@ entrycache_remove_int(struct cache *cache, struct backentry *e) } else { - LOG("remove %d from id hash failed\n", e->ep_id, 0, 0); + LOG("remove %s (%d) from id hash failed\n", ndn, e->ep_id, 0); } } #ifdef UUIDCACHE_ON @@ -984,7 +984,12 @@ int cache_replace(struct cache *cache, void *oldptr, void *newptr) static int entrycache_replace(struct cache *cache, struct backentry *olde, struct backentry *newe) { - int found; + int found = 0; + int found_in_dn = 0; + int found_in_id = 0; +#ifdef UUIDCACHE_ON + int found_in_uuid = 0; +#endif const char *oldndn; const char *newndn; #ifdef UUIDCACHE_ON @@ -992,6 +997,7 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, const char *newuuid; #endif size_t entry_size = 0; + struct backentry *alte = NULL; LOG("=> entrycache_replace (%s) -> (%s)\n", backentry_get_ndn(olde), backentry_get_ndn(newe), 0); @@ -1015,31 +1021,21 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, * cache tables, operation error */ if ( (olde->ep_state & ENTRY_STATE_NOTINCACHE) == 0 ) { - int found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn)); - int found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID)); + found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn)); + found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID)); #ifdef UUIDCACHE_ON - int found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid)); + found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid)); #endif found = found_in_dn && found_in_id; #ifdef UUIDCACHE_ON found = found && found_in_uuid; #endif - if (!found) { -#ifdef UUIDCACHE_ON - LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n", - found_in_dn, found_in_id, found_in_uuid); -#else - LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d]\n", - found_in_dn, found_in_id, 0); -#endif - PR_Unlock(cache->c_mutex); - return 1; - } } - if (! entry_same_dn(newe, (void *)oldndn) && - (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) { - /* if we're doing a modrdn, the new entry can be in the dn table - * already, so we need to remove that too. + /* If fails, we have to make sure the both entires are removed from the cache, + * otherwise, we have no idea what's left in the cache or not... */ + if (!entry_same_dn(newe, (void *)oldndn) && (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) { + /* if we're doing a modrdn or turning an entry to a tombstone, + * the new entry can be in the dn table already, so we need to remove that too. */ if (remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn))) { @@ -1048,20 +1044,44 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, LOG("entry cache replace remove entry size %lu\n", newe->ep_size, 0, 0); } } - + /* + * The old entry could have been "removed" between the add and this replace, + * The entry is NOT freed, but NOT in the dn hash. + * which could happen since the entry is not necessarily locked. + * This is ok. + */ + olde->ep_state = ENTRY_STATE_DELETED; /* olde is removed from the cache, so set DELETED here. */ + if (!found) { + if (olde->ep_state & ENTRY_STATE_DELETED) { + LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]; but the entry is alreay deleted.\n", + oldndn, found_in_dn, found_in_id); + } else { +#ifdef UUIDCACHE_ON + LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n", + found_in_dn, found_in_id, found_in_uuid); +#else + LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]\n", + oldndn, found_in_dn, found_in_id); +#endif + PR_Unlock(cache->c_mutex); + return 1; + } + } /* now, add the new entry to the hashtables */ /* (probably don't need such extensive error handling, once this has been * tested enough that we believe it works.) */ - if (!add_hash(cache->c_dntable, (void *)newndn, strlen(newndn), newe, NULL)) { - LOG("entry cache replace: can't add dn\n", 0, 0, 0); + if (!add_hash(cache->c_dntable, (void *)newndn, strlen(newndn), newe, (void **)&alte)) { + LOG("entry cache replace (%s): can't add to dn table (returned %s)\n", + newndn, alte?slapi_entry_get_dn(alte->ep_entry):"none", 0); PR_Unlock(cache->c_mutex); return 1; } - if (!add_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID), newe, NULL)) { - LOG("entry cache replace: can't add id\n", 0, 0, 0); + if (!add_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID), newe, (void **)&alte)) { + LOG("entry cache replace (%s): can't add to id table (returned %s)\n", + newndn, alte?slapi_entry_get_dn(alte->ep_entry):"none", 0); if(remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)) == 0){ - LOG("entry cache replace: failed to remove dn table\n", 0, 0, 0); + LOG("entry cache replace: failed to remove dn table\n", 0, 0, 0); } PR_Unlock(cache->c_mutex); return 1; @@ -1071,10 +1091,10 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, newe, NULL)) { LOG("entry cache replace: can't add uuid\n", 0, 0, 0); if(remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)) == 0){ - LOG("entry cache replace: failed to remove dn table(uuid cache)\n", 0, 0, 0); + LOG("entry cache replace: failed to remove dn table(uuid cache)\n", 0, 0, 0); } if(remove_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID)) == 0){ - LOG("entry cache replace: failed to remove id table(uuid cache)\n", 0, 0, 0); + LOG("entry cache replace: failed to remove id table(uuid cache)\n", 0, 0, 0); } PR_Unlock(cache->c_mutex); return 1; @@ -1088,7 +1108,6 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, } else if (newe->ep_size < olde->ep_size) { slapi_counter_subtract(cache->c_cursize, olde->ep_size - newe->ep_size); } - olde->ep_state = ENTRY_STATE_DELETED; newe->ep_state = 0; PR_Unlock(cache->c_mutex); LOG("<= entrycache_replace OK, cache size now %lu cache count now %ld\n", @@ -1282,7 +1301,7 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, PR_Lock(cache->c_mutex); if (! add_hash(cache->c_dntable, (void *)ndn, strlen(ndn), e, (void **)&my_alt)) { - LOG("entry \"%s\" already in dn cache\n", backentry_get_ndn(e), 0, 0); + LOG("entry \"%s\" already in dn cache\n", ndn, 0, 0); /* add_hash filled in 'my_alt' if necessary */ if (my_alt == e) { @@ -1322,14 +1341,15 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, { if (my_alt->ep_state & ENTRY_STATE_CREATING) { - LOG("the entry is reserved\n", 0, 0, 0); + LOG("the entry %s is reserved (ep_state: 0x%x, state: 0x%x\n", ndn, e->ep_state, state); e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); return -1; } else if (state != 0) { - LOG("the entry already exists. cannot reserve it.\n", 0, 0, 0); + LOG("the entry %s already exists. cannot reserve it. (ep_state: 0x%x, state: 0x%x\n", + ndn, e->ep_state, state); e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); return -1; @@ -1341,6 +1361,11 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, if ((*alt)->ep_refcnt == 0) lru_delete(cache, (void *)*alt); (*alt)->ep_refcnt++; + LOG("the entry %s already exists. returning existing entry %s (state: 0x%x)\n", + ndn, backentry_get_ndn(my_alt), state); + } else { + LOG("the entry %s already exists. Not returning existing entry %s (state: 0x%x)\n", + ndn, backentry_get_ndn(my_alt), state); } PR_Unlock(cache->c_mutex); return 1; @@ -1355,7 +1380,7 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, if (state == 0) { /* neither of these should fail, or something is very wrong. */ if (! add_hash(cache->c_idtable, &(e->ep_id), sizeof(ID), e, NULL)) { - LOG("entry %s already in id cache!\n", backentry_get_ndn(e), 0, 0); + LOG("entry %s already in id cache!\n", ndn, 0, 0); if (already_in) { /* there's a bug in the implementatin of 'modify' and 'modrdn' * that i'm working around here. basically they do a @@ -1377,10 +1402,12 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, return 0; } if(remove_hash(cache->c_dntable, (void *)ndn, strlen(ndn)) == 0){ - LOG("entrycache_add_int: failed to remove dn table\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove %s from dn table\n", 0, 0, 0); } e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); + LOG("entrycache_add_int: failed to add %s to cache (ep_state: %x, already_in: %d)\n", + ndn, e->ep_state, already_in); return -1; } #ifdef UUIDCACHE_ON @@ -1391,10 +1418,10 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, LOG("entry %s already in uuid cache!\n", backentry_get_ndn(e), 0, 0); if(remove_hash(cache->c_dntable, (void *)ndn, strlen(ndn)) == 0){ - LOG("entrycache_add_int: failed to remove dn table(uuid cache)\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove dn table(uuid cache)\n", 0, 0, 0); } if(remove_hash(cache->c_idtable, &(e->ep_id), sizeof(ID)) == 0){ - LOG("entrycache_add_int: failed to remove id table(uuid cache)\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove id table(uuid cache)\n", 0, 0, 0); } e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); @@ -1499,8 +1526,8 @@ int cache_lock_entry(struct cache *cache, struct backentry *e) if (!e->ep_mutexp) { LOG("<= cache_lock_entry (DELETED)\n", 0, 0, 0); LDAPDebug1Arg(LDAP_DEBUG_ANY, - "cache_lock_entry: failed to create a lock for %s\n", - backentry_get_ndn(e)); + "cache_lock_entry: failed to create a lock for %s\n", + backentry_get_ndn(e)); LOG("<= cache_lock_entry (FAILED)\n", 0, 0, 0); return 1; } diff --git a/ldap/servers/slapd/back-ldbm/ldbm_add.c b/ldap/servers/slapd/back-ldbm/ldbm_add.c index 9087a62..0a39f71 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_add.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_add.c @@ -103,9 +103,9 @@ ldbm_back_add( Slapi_PBlock *pb ) int parent_found = 0; int ruv_c_init = 0; int rc = 0; - int addingentry_id_assigned= 0; - int addingentry_in_cache= 0; - int tombstone_in_cache= 0; + int addingentry_id_assigned = 0; + int addingentry_in_cache = 0; + int tombstone_in_cache = 0; Slapi_DN *sdn = NULL; Slapi_DN parentsdn; Slapi_Operation *operation; @@ -118,6 +118,7 @@ ldbm_back_add( Slapi_PBlock *pb ) entry_address addr = {0}; int not_an_error = 0; int parent_switched = 0; + int noabort = 1; slapi_pblock_get( pb, SLAPI_PLUGIN_PRIVATE, &li ); slapi_pblock_get( pb, SLAPI_ADD_ENTRY, &e ); @@ -194,7 +195,6 @@ ldbm_back_add( Slapi_PBlock *pb ) goto error_return; } - /* * Originally (in the U-M LDAP 3.3 code), there was a comment near this * code about a race condition. The race was that a 2nd entry could be @@ -214,11 +214,16 @@ ldbm_back_add( Slapi_PBlock *pb ) if (txn.back_txn_txn && (txn.back_txn_txn != parent_txn)) { /* Don't release SERIAL LOCK */ dblayer_txn_abort_ext(li, &txn, PR_FALSE); + noabort = 1; slapi_pblock_set(pb, SLAPI_TXN, parent_txn); if (addingentry_in_cache) { /* addingentry is in cache. Remove it once. */ - CACHE_REMOVE(&inst->inst_cache, addingentry); + retval = CACHE_REMOVE(&inst->inst_cache, addingentry); + if (retval) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "ldbm_add: cache_remove %s failed.\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); + } CACHE_RETURN(&inst->inst_cache, &addingentry); } else { backentry_free(&addingentry); @@ -231,11 +236,11 @@ ldbm_back_add( Slapi_PBlock *pb ) } if (addingentry_in_cache) { /* Adding the resetted addingentry to the cache. */ - if (cache_add_tentative(&inst->inst_cache, - addingentry, NULL) != 0) { - LDAPDebug0Args(LDAP_DEBUG_CACHE, - "cache_add_tentative concurrency detected\n"); + if (cache_add_tentative(&inst->inst_cache, addingentry, NULL) != 0) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code = LDAP_ALREADY_EXISTS; + addingentry_in_cache = 0; goto error_return; } } @@ -260,6 +265,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if (0 == retry_count) { /* First time, hold SERIAL LOCK */ retval = dblayer_txn_begin(be, parent_txn, &txn); + noabort = 0; if (!is_tombstone_operation) { rc= slapi_setbit_int(rc,SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); @@ -306,6 +312,10 @@ ldbm_back_add( Slapi_PBlock *pb ) } } + /* + * If the parent is conflict, slapi_sdn_get_backend_parent does not support it. + * That is, adding children to a conflict entry is not allowed. + */ slapi_sdn_get_backend_parent(sdn, &parentsdn, pb->pb_backend); /* Check if an entry with the intended DN already exists. */ done_with_pblock_entry(pb,SLAPI_ADD_EXISTING_DN_ENTRY); /* Could be through this multiple times */ @@ -313,28 +323,25 @@ ldbm_back_add( Slapi_PBlock *pb ) addr.udn = NULL; addr.uniqueid = NULL; ldap_result_code= get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_EXISTING_DN_ENTRY, !is_replicated_operation); - if(ldap_result_code==LDAP_OPERATIONS_ERROR || - ldap_result_code==LDAP_INVALID_DN_SYNTAX) - { + if(ldap_result_code==LDAP_OPERATIONS_ERROR || ldap_result_code==LDAP_INVALID_DN_SYNTAX) { goto error_return; } } /* if we can find the parent by dn or uniqueid, and the operation has requested the parent then get it */ - if(have_parent_address(&parentsdn, operation->o_params.p.p_add.parentuniqueid) && - slapi_isbitset_int(rc,SLAPI_RTN_BIT_FETCH_PARENT_ENTRY)) - { + if (have_parent_address(&parentsdn, operation->o_params.p.p_add.parentuniqueid) && + slapi_isbitset_int(rc,SLAPI_RTN_BIT_FETCH_PARENT_ENTRY)) { done_with_pblock_entry(pb,SLAPI_ADD_PARENT_ENTRY); /* Could be through this multiple times */ addr.sdn = &parentsdn; addr.udn = NULL; addr.uniqueid = operation->o_params.p.p_add.parentuniqueid; - ldap_result_code= get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_PARENT_ENTRY, !is_replicated_operation); - /* need to set parentsdn or parentuniqueid if either is not set? */ + ldap_result_code = get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_PARENT_ENTRY, !is_replicated_operation); } /* Call the Backend Pre Add plugins */ + ldap_result_code = LDAP_SUCCESS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); - rc= plugin_call_plugins(pb, SLAPI_PLUGIN_BE_PRE_ADD_FN); + rc = plugin_call_plugins(pb, SLAPI_PLUGIN_BE_PRE_ADD_FN); if (rc < 0) { int opreturn = 0; if (SLAPI_PLUGIN_NOOP == rc) { @@ -380,6 +387,7 @@ ldbm_back_add( Slapi_PBlock *pb ) ldap_result_code= LDAP_OPERATIONS_ERROR; goto error_return; } + noabort = 0; /* stash the transaction for plugins */ slapi_pblock_set(pb, SLAPI_TXN, txn.back_txn_txn); @@ -402,10 +410,14 @@ ldbm_back_add( Slapi_PBlock *pb ) operation->o_params.p.p_add.parentuniqueid = slapi_ch_strdup(slapi_entry_get_uniqueid(parententry->ep_entry)); } - if (slapi_sdn_isempty(&parentsdn)) { + if (slapi_sdn_isempty(&parentsdn) || + slapi_sdn_compare(&parentsdn, slapi_entry_get_sdn(parententry->ep_entry))) { /* Set the parentsdn now */ slapi_sdn_set_dn_byval(&parentsdn, slapi_entry_get_dn_const(parententry->ep_entry)); } + } else { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "find_entry2modify_only returned NULL parententry pdn: %s, uniqueid: %s\n", + slapi_sdn_get_dn(&parentsdn), slapi_sdn_get_dn(&parentsdn), addr.uniqueid?addr.uniqueid:"none"); } modify_init(&parent_modify_c,parententry); } @@ -413,11 +425,36 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Check if the entry we have been asked to add already exists */ { Slapi_Entry *entry; - slapi_pblock_get( pb, SLAPI_ADD_EXISTING_DN_ENTRY, &entry); - if ( entry != NULL ) - { - /* The entry already exists */ - ldap_result_code= LDAP_ALREADY_EXISTS; + slapi_pblock_get(pb, SLAPI_ADD_EXISTING_DN_ENTRY, &entry); + if (entry) { + if (is_resurect_operation) { + Slapi_Entry *uniqentry; + slapi_pblock_get(pb, SLAPI_ADD_EXISTING_UNIQUEID_ENTRY, &uniqentry); + if (uniqentry == entry) { + /* + * adding entry having the uniqueid exists. + * No need to resurrect. + */ + ldap_result_code = LDAP_SUCCESS; + } else { + /* The entry having the DN already exists */ + if (uniqentry) { + if (PL_strcmp(slapi_entry_get_uniqueid(entry), + slapi_entry_get_uniqueid(uniqentry))) { + /* Not match; conflict. */ + ldap_result_code = LDAP_ALREADY_EXISTS; + } else { + /* Same entry; no need to resurrect. */ + ldap_result_code = LDAP_SUCCESS; + } + } else { + ldap_result_code = LDAP_ALREADY_EXISTS; + } + } + } else { + /* The entry already exists */ + ldap_result_code = LDAP_ALREADY_EXISTS; + } goto error_return; } else @@ -428,7 +465,7 @@ ldbm_back_add( Slapi_PBlock *pb ) * entry we did match has a referral we should return * instead. we do this only if managedsait is not on. */ - if ( !managedsait && !is_tombstone_operation ) + if (!managedsait && !is_tombstone_operation && !is_resurect_operation) { int err= 0; Slapi_DN ancestorsdn; @@ -438,9 +475,7 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_sdn_done(&ancestorsdn); if ( ancestorentry != NULL ) { - int sentreferral = - check_entry_for_referral(pb, ancestorentry->ep_entry, - backentry_get_ndn(ancestorentry), "ldbm_back_add"); + int sentreferral= check_entry_for_referral(pb, ancestorentry->ep_entry, backentry_get_ndn(ancestorentry), "ldbm_back_add"); CACHE_RETURN( &inst->inst_cache, &ancestorentry ); if(sentreferral) { @@ -453,25 +488,25 @@ ldbm_back_add( Slapi_PBlock *pb ) } /* no need to check the schema as this is a replication add */ - if(!is_replicated_operation){ + if (!is_replicated_operation) { if ((operation_is_flag_set(operation,OP_FLAG_ACTION_SCHEMA_CHECK)) && (slapi_entry_schema_check(pb, e) != 0)) - { - LDAPDebug(LDAP_DEBUG_TRACE, "entry failed schema check\n", 0, 0, 0); - ldap_result_code = LDAP_OBJECT_CLASS_VIOLATION; - slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); - goto error_return; - } + { + LDAPDebug(LDAP_DEBUG_TRACE, "entry failed schema check\n", 0, 0, 0); + ldap_result_code = LDAP_OBJECT_CLASS_VIOLATION; + slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + goto error_return; + } - /* Check attribute syntax */ - if (slapi_entry_syntax_check(pb, e, 0) != 0) - { - LDAPDebug(LDAP_DEBUG_TRACE, "entry failed syntax check\n", 0, 0, 0); - ldap_result_code = LDAP_INVALID_SYNTAX; - slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); - goto error_return; + /* Check attribute syntax */ + if (slapi_entry_syntax_check(pb, e, 0) != 0) + { + LDAPDebug(LDAP_DEBUG_TRACE, "entry failed syntax check\n", 0, 0, 0); + ldap_result_code = LDAP_INVALID_SYNTAX; + slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + goto error_return; + } } - } opcsn = operation_get_csn (operation); if(is_resurect_operation) @@ -488,7 +523,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if ( tombstoneentry==NULL ) { ldap_result_code= -1; - goto error_return; /* error result sent by find_entry2modify() */ + goto error_return; /* error result sent by find_entry2modify() */ } tombstone_in_cache = 1; @@ -507,10 +542,14 @@ ldbm_back_add( Slapi_PBlock *pb ) */ if (NULL == sdn) { LDAPDebug0Args(LDAP_DEBUG_ANY, "ldbm_back_add: Null target dn\n"); + ldap_result_code = LDAP_OPERATIONS_ERROR; goto error_return; } dn = slapi_sdn_get_dn(sdn); slapi_entry_set_sdn(addingentry->ep_entry, sdn); /* The DN is passed into the entry. */ + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry->ep_entry), + slapi_entry_get_dn_const(addingentry->ep_entry)); /* LPREPL: the DN is normalized...Somehow who should get a not normalized one */ addingentry->ep_id = slapi_entry_attr_get_ulong(addingentry->ep_entry,"entryid"); slapi_entry_attr_delete(addingentry->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); @@ -674,10 +713,8 @@ ldbm_back_add( Slapi_PBlock *pb ) Slapi_DN ancestorsdn; struct backentry *ancestorentry; - LDAPDebug( LDAP_DEBUG_TRACE, - "parent does not exist, pdn = %s\n", - slapi_sdn_get_dn(&parentsdn), 0, 0 ); - + LDAPDebug1Arg(LDAP_DEBUG_BACKLDBM, "ldbm_add: Parent \"%s\" does not exist. " + "It might be a conflict entry.\n", slapi_sdn_get_dn(&parentsdn)); slapi_sdn_init(&ancestorsdn); ancestorentry = dn2ancestor(be, &parentsdn, &ancestorsdn, &txn, &err ); CACHE_RETURN( &inst->inst_cache, &ancestorentry ); @@ -688,22 +725,37 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_sdn_done(&ancestorsdn); goto error_return; } - ldap_result_code = plugin_call_acl_plugin (pb, e, NULL, NULL, SLAPI_ACL_ADD, - ACLPLUGIN_ACCESS_DEFAULT, &errbuf ); + ldap_result_code = plugin_call_acl_plugin(pb, e, NULL, NULL, SLAPI_ACL_ADD, + ACLPLUGIN_ACCESS_DEFAULT, &errbuf); if ( ldap_result_code != LDAP_SUCCESS ) { - LDAPDebug( LDAP_DEBUG_TRACE, "no access to parent\n", 0, 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "no access to parent, pdn = %s\n", + slapi_sdn_get_dn(&parentsdn)); ldap_result_message= errbuf; goto error_return; } pid = parententry->ep_id; + + /* We may need to adjust the DN since parent could be a resrected conflict entry... */ + if (!slapi_sdn_isparent(slapi_entry_get_sdn_const(parententry->ep_entry), + slapi_entry_get_sdn_const(addingentry->ep_entry))) { + Slapi_DN adjustedsdn = {0}; + char *adjusteddn = slapi_ch_smprintf("%s,%s", + slapi_entry_get_rdn_const(addingentry->ep_entry), + slapi_entry_get_dn_const(parententry->ep_entry)); + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "ldbm_add: adjusting dn: %s --> %s\n", + slapi_entry_get_dn(addingentry->ep_entry), adjusteddn); + slapi_sdn_set_normdn_passin(&adjustedsdn, adjusteddn); + slapi_entry_set_sdn(addingentry->ep_entry, &adjustedsdn); + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry->ep_entry), adjusteddn); + slapi_sdn_done(&adjustedsdn); + } } else { /* no parent */ - if ( !isroot && !is_replicated_operation) - { - LDAPDebug( LDAP_DEBUG_TRACE, "no parent & not root\n", - 0, 0, 0 ); + if (!isroot && !is_replicated_operation) { + LDAPDebug0Args(LDAP_DEBUG_TRACE, "no parent & not root\n"); ldap_result_code= LDAP_INSUFFICIENT_ACCESS; goto error_return; } @@ -732,11 +784,12 @@ ldbm_back_add( Slapi_PBlock *pb ) * operational attributes to ensure that the cache is sized correctly. */ if ( cache_add_tentative( &inst->inst_cache, addingentry, NULL )!= 0 ) { - LDAPDebug( LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected\n", 0, 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code= LDAP_ALREADY_EXISTS; goto error_return; } - addingentry_in_cache= 1; + addingentry_in_cache = 1; /* * Before we add the entry, find out if the syntax of the aci @@ -744,7 +797,8 @@ ldbm_back_add( Slapi_PBlock *pb ) * the entry if the syntax is incorrect. */ if ( plugin_call_acl_verify_syntax (pb, addingentry->ep_entry, &errbuf) != 0 ) { - LDAPDebug( LDAP_DEBUG_TRACE, "ACL syntax error\n", 0,0,0); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "ACL syntax error: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code= LDAP_INVALID_SYNTAX; ldap_result_message= errbuf; goto error_return; @@ -753,12 +807,14 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Having decided that we're really going to do the operation, let's modify the in-memory state of the parent to reflect the new child (update subordinate count specifically */ - if (NULL != parententry) - { + if (parententry) { retval = parent_update_on_childchange(&parent_modify_c, - PARENTUPDATE_ADD, NULL); + is_resurect_operation?PARENTUPDATE_RESURECT:PARENTUPDATE_ADD, + NULL); /* The modify context now contains info needed later */ - if (0 != retval) { + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "parent_update_on_childchange: %s, rc=%d\n", + slapi_entry_get_dn_const(addingentry->ep_entry), retval); ldap_result_code= LDAP_OPERATIONS_ERROR; goto error_return; } @@ -781,14 +837,12 @@ ldbm_back_add( Slapi_PBlock *pb ) not_an_error = 1; rc = retval = LDAP_SUCCESS; } - LDAPDebug1Arg( LDAP_DEBUG_TRACE, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " - "returned error code %d\n", retval ); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " + "returned error code %d\n", retval ); if (!ldap_result_code) { slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } if (!ldap_result_code) { - LDAPDebug0Args( LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " - "returned error but did not setSLAPI_RESULT_CODE \n" ); ldap_result_code = LDAP_OPERATIONS_ERROR; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); } @@ -797,6 +851,8 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_set(pb, SLAPI_PLUGIN_OPRETURN, ldap_result_code ? &ldap_result_code : &retval); } slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + LDAPDebug1Arg(LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin failed: %d", + ldap_result_code ? ldap_result_code : retval); goto error_return; } @@ -807,9 +863,10 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_TRACE, "id2entry_add failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + if (retval) { + LDAPDebug(LDAP_DEBUG_TRACE, "id2entry_add(%s) failed, err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; @@ -817,19 +874,18 @@ ldbm_back_add( Slapi_PBlock *pb ) } goto error_return; } - if(is_resurect_operation) - { + if (is_resurect_operation) { retval = index_addordel_string(be,SLAPI_ATTR_OBJECTCLASS,SLAPI_ATTR_VALUE_TOMBSTONE,addingentry->ep_id,BE_INDEX_DEL|BE_INDEX_EQUALITY,&txn); if (DB_LOCK_DEADLOCK == retval) { LDAPDebug( LDAP_DEBUG_ARGS, "add 2 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 1 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string TOMBSTONE (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -843,10 +899,10 @@ ldbm_back_add( Slapi_PBlock *pb ) continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 2 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string UNIQUEID (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -864,17 +920,29 @@ ldbm_back_add( Slapi_PBlock *pb ) continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 3 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string ENTRYDN (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; } goto error_return; } - } + /* Need to delete the entryrdn index of the resurrected tombstone... */ + if (entryrdn_get_switch()) { /* subtree-rename: on */ + if (tombstoneentry) { + retval = entryrdn_index_entry(be, tombstoneentry, BE_INDEX_DEL, &txn); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, "Resurrecting %s: failed to remove entryrdn index, err=%d %s\n", + slapi_entry_get_dn_const(tombstoneentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + goto error_return; + } + } + } + } if (is_tombstone_operation) { retval = index_addordel_entry( be, addingentry, BE_INDEX_ADD | BE_INDEX_TOMBSTONE, &txn ); @@ -889,9 +957,8 @@ ldbm_back_add( Slapi_PBlock *pb ) /* retry txn */ continue; } - if (retval != 0) { - LDAPDebug2Args(LDAP_DEBUG_ANY, - "add: attempt to index %lu failed (rc=%d)\n", + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_ANY, "add: attempt to index %lu failed; rc=%d\n", (u_long)addingentry->ep_id, retval); ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { @@ -909,11 +976,10 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 1 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "modify_update_all: %s (%lu) failed; rc=%d\n", + slapi_entry_get_dn(addingentry->ep_entry), (u_long)addingentry->ep_id, retval); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -928,17 +994,16 @@ ldbm_back_add( Slapi_PBlock *pb ) { retval= vlv_update_all_indexes(&txn, be, pb, NULL, addingentry); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, - "add DEADLOCK vlv_update_index\n", 0, 0, 0 ); + LDAPDebug(LDAP_DEBUG_ARGS, + "add DEADLOCK vlv_update_index\n", 0, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, - "vlv_update_index failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_TRACE, + "vlv_update_index failed, err=%d %s\n", + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -984,7 +1049,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if (retry_count == RETRY_TIMES) { /* Failed */ LDAPDebug( LDAP_DEBUG_ANY, "Retry count exceeded in add\n", 0, 0, 0 ); - ldap_result_code= LDAP_BUSY; + ldap_result_code= LDAP_BUSY; goto error_return; } @@ -995,38 +1060,53 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_set( pb, SLAPI_ENTRY_PRE_OP, NULL ); slapi_pblock_set( pb, SLAPI_ENTRY_POST_OP, slapi_entry_dup( addingentry->ep_entry )); - if(is_resurect_operation) - { + if (is_resurect_operation) { /* * We can now switch the tombstone entry with the real entry. */ - if (cache_replace( &inst->inst_cache, tombstoneentry, addingentry ) != 0 ) - { + retval = cache_replace(&inst->inst_cache, tombstoneentry, addingentry); + if (retval) { /* This happens if the dn of addingentry already exists */ - cache_unlock_entry( &inst->inst_cache, tombstoneentry ); ADD_SET_ERROR(ldap_result_code, LDAP_ALREADY_EXISTS, retry_count); + LDAPDebug2Args(LDAP_DEBUG_CACHE, "ldap_add: cache_replace concurrency detected: %s (rc: %d)\n", + slapi_entry_get_dn_const(addingentry->ep_entry), retval); + retval = -1; goto error_return; } + if (addingentry_in_cache) { /* decrease the refcnt added by tentative */ + CACHE_RETURN( &inst->inst_cache, &addingentry ); + } + addingentry_in_cache = 1; /* reset it to make it sure... */ /* * The tombstone was locked down in the cache... we can * get rid of the entry in the cache now. + * We cannot expect tombstoneentry exists from now on. */ - cache_unlock_entry( &inst->inst_cache, tombstoneentry ); - CACHE_RETURN( &inst->inst_cache, &tombstoneentry ); - tombstone_in_cache = 0; /* deleted */ + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, delete the tombstone dn from the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, + tombstoneentry->ep_id); + if (bdn) { /* in the dncache, remove it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + cache_unlock_entry(&inst->inst_cache, tombstoneentry); + CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + tombstone_in_cache = 0; } if (parent_found) { /* switch the parent entry copy into play */ - modify_switch_entries( &parent_modify_c,be); + modify_switch_entries(&parent_modify_c,be); parent_switched = 1; } if (ruv_c_init) { if (modify_switch_entries(&ruv_c, be) != 0 ) { ldap_result_code= LDAP_OPERATIONS_ERROR; - LDAPDebug( LDAP_DEBUG_ANY, - "ldbm_back_add: modify_switch_entries failed\n", 0, 0, 0); + LDAPDebug0Args(LDAP_DEBUG_ANY, + "ldbm_back_add: modify_switch_entries failed\n"); goto error_return; } } @@ -1040,8 +1120,6 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } if (!ldap_result_code) { - LDAPDebug0Args( LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_POST_ADD_FN plugin " - "returned error but did not set SLAPI_RESULT_CODE\n" ); ldap_result_code = LDAP_OPERATIONS_ERROR; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); } @@ -1066,6 +1144,7 @@ ldbm_back_add( Slapi_PBlock *pb ) } goto error_return; } + noabort = 1; rc= 0; goto common_return; @@ -1075,23 +1154,22 @@ error_return: { next_id_return( be, addingentry->ep_id ); } - if ( NULL != addingentry ) + if ( addingentry ) { if ( addingentry_in_cache ) { if (inst) { CACHE_REMOVE(&inst->inst_cache, addingentry); } - addingentry_in_cache = 0; } - backentry_clear_entry(addingentry); /* e is released in the frontend */ - backentry_free( &addingentry ); /* release the backend wrapper, here */ - } - if(tombstone_in_cache && inst) - { - CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + else + { + if (!is_resurect_operation) { /* if resurect, tombstoneentry is dupped. */ + backentry_clear_entry(addingentry); /* e is released in the frontend */ + } + backentry_free( &addingentry ); /* release the backend wrapper, here */ + } } - if (rc == DB_RUNRECOVERY) { dblayer_remember_disk_filled(li); ldbm_nasty("Add",80,rc); @@ -1109,7 +1187,7 @@ error_return: } diskfull_return: if (disk_full) { - rc= return_on_disk_full(li); + rc = return_on_disk_full(li); } else { /* It is safer not to abort when the transaction is not started. */ if (txn.back_txn_txn && (txn.back_txn_txn != parent_txn)) { @@ -1143,7 +1221,9 @@ diskfull_return: } /* Release SERIAL LOCK */ - dblayer_txn_abort(be, &txn); /* abort crashes in case disk full */ + if (!noabort) { + dblayer_txn_abort(be, &txn); /* abort crashes in case disk full */ + } /* txn is no longer valid - reset the txn pointer to the parent */ slapi_pblock_set(pb, SLAPI_TXN, parent_txn); } @@ -1154,8 +1234,12 @@ diskfull_return: common_return: if (inst) { + if(tombstone_in_cache && tombstoneentry) { + cache_unlock_entry(&inst->inst_cache, tombstoneentry); + CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + } if (addingentry_in_cache && addingentry) { - if (entryrdn_get_switch()) { /* subtree-rename: on */ + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ /* since adding the entry to the entry cache was successful, * let's add the dn to dncache, if not yet done. */ struct backdn *bdn = dncache_find_id(&inst->inst_dncache, diff --git a/ldap/servers/slapd/back-ldbm/ldbm_delete.c b/ldap/servers/slapd/back-ldbm/ldbm_delete.c index ea9473e..915ea08 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_delete.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_delete.c @@ -90,6 +90,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) /* replication, we must create a new tombstone entry */ int tombstone_in_cache = 0; int e_in_cache = 0; + int remove_e_from_cache = 0; entry_address *addr; int addordel_flags = 0; /* passed to index_addordel */ char *entryusn_str = NULL; @@ -220,6 +221,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (tombstone_in_cache) { CACHE_REMOVE(&inst->inst_cache, tombstone); CACHE_RETURN(&inst->inst_cache, &tombstone); + tombstone = NULL; tombstone_in_cache = 0; } else { backentry_free(&tombstone); @@ -230,13 +232,6 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } } - - /* reset original entry in cache */ - if (!e_in_cache) { - CACHE_ADD(&inst->inst_cache, e, NULL); - e_in_cache = 1; - } - if (ruv_c_init) { /* reset the ruv txn stuff */ modify_term(&ruv_c, be); @@ -272,17 +267,28 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (0 == retry_count) { /* just once */ /* find and lock the entry we are about to modify */ + /* + * A corner case: + * If a conflict occurred in a MMR topology, a replicated delete + * op from another master could target a conflict entry; while the + * corresponding entry on this server could have been already + * deleted. That is, the entry 'e' found with "addr" is a tomb- + * stone. If it is the case, we need to back off. + */ if ( (e = find_entry2modify( pb, be, addr, &txn )) == NULL ) { ldap_result_code= LDAP_NO_SUCH_OBJECT; retval = -1; + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "ldbm_back_delete: Deleting entry is already deleted.\n"); goto error_return; /* error result sent by find_entry2modify() */ } e_in_cache = 1; /* e is cached */ - if ( slapi_entry_has_children( e->ep_entry ) ) - { + retval = slapi_entry_has_children(e->ep_entry); + if (retval) { ldap_result_code= LDAP_NOT_ALLOWED_ON_NONLEAF; + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_delete", "Deleting entry %s has %d children.\n", + slapi_entry_get_dn(e->ep_entry), retval); retval = -1; goto error_return; } @@ -318,7 +324,17 @@ ldbm_back_delete( Slapi_PBlock *pb ) * Plugin indicated some kind of failure, * or that this Operation became a No-Op. */ + slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); if (!ldap_result_code) { + if (LDAP_ALREADY_EXISTS == ldap_result_code) { + /* + * The target entry is already a tombstone. + * We need to treat this as a success, + * but we need to remove the entry e from the entry cache. + */ + remove_e_from_cache = 1; + ldap_result_code = LDAP_SUCCESS; + } slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } /* restore original entry so the front-end delete code can free it */ @@ -330,8 +346,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } /* the flag could be set in a preop plugin (e.g., USN) */ - delete_tombstone_entry = operation_is_flag_set(operation, - OP_FLAG_TOMBSTONE_ENTRY); + delete_tombstone_entry = operation_is_flag_set(operation, OP_FLAG_TOMBSTONE_ENTRY); } /* call the transaction pre delete plugins just after the @@ -493,13 +508,15 @@ ldbm_back_delete( Slapi_PBlock *pb ) } if (NULL == parent) { entry_address parent_addr; - + if (is_tombstone_entry) { + parent_addr.uniqueid = slapi_entry_attr_get_charptr(e->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + } else { + parent_addr.uniqueid = NULL; + } parent_addr.sdn = &parentsdn; - parent_addr.uniqueid = NULL; - parent = find_entry2modify_only_ext(pb, be, &parent_addr, - TOMBSTONE_INCLUDED, &txn); + parent = find_entry2modify_only_ext(pb, be, &parent_addr, TOMBSTONE_INCLUDED, &txn); } - if (NULL != parent) { + if (parent) { int isglue; size_t haschildren = 0; int op = PARENTUPDATE_DEL; @@ -516,11 +533,11 @@ ldbm_back_delete( Slapi_PBlock *pb ) } else if (delete_tombstone_entry) { op |= PARENTUPDATE_DELETE_TOMBSTONE; } - retval = parent_update_on_childchange(&parent_modify_c, - op, &haschildren); + retval = parent_update_on_childchange(&parent_modify_c, op, &haschildren); /* The modify context now contains info needed later */ if (0 != retval) { ldap_result_code= LDAP_OPERATIONS_ERROR; + slapi_sdn_done(&parentsdn); retval = -1; goto error_return; } @@ -550,9 +567,17 @@ ldbm_back_delete( Slapi_PBlock *pb ) */ const char *childuniqueid= slapi_entry_get_uniqueid(e->ep_entry); const char *parentuniqueid= NULL; - char *tombstone_dn = compute_entry_tombstone_dn(slapi_entry_get_dn(e->ep_entry), - childuniqueid); + char *edn = slapi_entry_get_dn(e->ep_entry); + char *tombstone_dn; Slapi_Value *tomb_value; + + if (slapi_is_special_rdn(edn, RDN_IS_TOMBSTONE)) { + LDAPDebug1Arg(LDAP_DEBUG_ANY, "Turning a tombstone into a tombstone! \"%s\"\n", edn); + ldap_result_code= LDAP_OPERATIONS_ERROR; + retval = -1; + goto error_return; + } + tombstone_dn = compute_entry_tombstone_dn(edn, childuniqueid); slapi_sdn_set_ndn_byval(&nscpEntrySDN, slapi_sdn_get_ndn(slapi_entry_get_sdn(e->ep_entry))); @@ -674,41 +699,85 @@ ldbm_back_delete( Slapi_PBlock *pb ) * tentatively for now, then cache_add again when the original * entry is removed from the cache. */ - if (cache_add_tentative( &inst->inst_cache, tombstone, NULL) == 0) { + retval = cache_add_tentative(&inst->inst_cache, tombstone, NULL); + if (0 == retval) { tombstone_in_cache = 1; - } else if (!(tombstone->ep_state & ENTRY_STATE_NOTINCACHE)) { - LDAPDebug1Arg(LDAP_DEBUG_CACHE, - "id2entry_add tombstone (%s) is in cache\n", - slapi_entry_get_dn(tombstone->ep_entry)); - tombstone_in_cache = 1; + } else { + LDAPDebug2Args(LDAP_DEBUG_ANY, + "tombstone entry %s failed to add to the cache: %d\n", + slapi_entry_get_dn(tombstone->ep_entry), retval); + tombstone_in_cache = 0; + if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; + DEL_SET_ERROR(ldap_result_code, + LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; } retval = id2entry_add( be, tombstone, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 1 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete 1 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); /* Abort and re-try */ continue; } - if (0 != retval) { + if (retval) { LDAPDebug( LDAP_DEBUG_ANY, "id2entry_add failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } - if (cache_replace( &inst->inst_cache, e, tombstone ) != 0 ) { - LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "ldbm_back_delete cache_replace failed\n"); - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); - retval= -1; - goto error_return; + if (tombstone_in_cache) { + retval = cache_replace(&inst->inst_cache, e, tombstone); + if (retval) { + LDAPDebug(LDAP_DEBUG_CACHE, "ldbm_back_delete: cache_replace failed (%d): %s --> %s\n", + retval, slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn(tombstone->ep_entry)); + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } } else { - e_in_cache = 0; /* e un-cached */ + struct backentry *imposter = NULL; + retval = CACHE_ADD(&inst->inst_cache, tombstone, &imposter); + if (retval > 0) { + if (imposter) { + /* + * The same tombstone entry (different Slapi_Entry) is already + * generated and set to cache. Back off. + */ + CACHE_RETURN(&inst->inst_cache, &imposter); + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: same DN tombstone in cache: %s\n", + slapi_entry_get_dn(tombstone->ep_entry)); + } else { + /* + * The same tombstone entry (same Slapi_Entry) is being created. + * Something is wrong. We should clean it up from the cache, + * and back off. + */ + tombstone_in_cache = 1; + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: same tombstone in cache: %s\n", + slapi_entry_get_dn(tombstone->ep_entry)); + } + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } else if (retval < 0) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: Add %s failed.\n", + slapi_entry_get_dn(tombstone->ep_entry)); + /* Complete add error */ + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } } - /* tombstone was already added to the cache via cache_add_tentative (to reserve its spot in the cache) - and/or id2entry_add - so it already had one refcount - cache_replace adds another refcount - - drop the extra ref added by cache_replace */ - CACHE_RETURN( &inst->inst_cache, &tombstone ); + if (tombstone_in_cache) { + /* tombstone was already added to the cache via cache_add_tentative (to reserve its spot in the cache) + and/or id2entry_add - so it already had one refcount - cache_replace adds another refcount - + drop the extra ref added by cache_replace */ + CACHE_RETURN( &inst->inst_cache, &tombstone ); + } + tombstone_in_cache = 1; } else { @@ -716,13 +785,12 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = id2entry_delete( be, e, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 2 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "delete 2 DEADLOCK\n"); /* Retry txn */ continue; } - if (retval != 0 ) { - if (retval == DB_RUNRECOVERY || - LDBM_OS_ERR_IS_DISKFULL(retval)) { + if (retval) { + if (retval == DB_RUNRECOVERY || LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; } DEL_SET_ERROR(ldap_result_code, @@ -739,12 +807,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = index_addordel_entry( be, e, addordel_flags, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 1 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "delete 1 DEADLOCK\n"); /* Retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_TRACE, "index_del_entry failed\n", 0, 0, 0 ); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, "index_del_entry(%s, 0x%x) failed (%d)\n", + slapi_entry_get_dn(e->ep_entry), addordel_flags, retval); DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } @@ -759,14 +828,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) SLAPI_ATTR_VALUE_TOMBSTONE, tombstone->ep_id,BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_VALUE_TOMBSTONE, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_VALUE_TOMBSTONE, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -779,14 +848,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_entry_get_uniqueid(tombstone->ep_entry), tombstone->ep_id,BE_INDEX_ADD,&txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_UNIQUEID, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_UNIQUEID, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -799,14 +868,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_sdn_get_ndn(&nscpEntrySDN), tombstone->ep_id, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_NSCP_ENTRYDN, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_NSCP_ENTRYDN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -823,14 +892,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) entryusn_str, tombstone->ep_id, BE_INDEX_ADD, &txn); slapi_ch_free_string(&entryusn_str); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_ENTRYUSN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_ENTRYUSN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -854,13 +923,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) svals, NULL, e->ep_id, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (updating " LDBM_PARENTID_STR ") DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if ( retval ) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", LDBM_PARENTID_STR, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -870,15 +939,16 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } } +#if 0 /* The entryrdn element is already deleted in the index_addordel_entry */ retval = entryrdn_index_entry(be, e, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (deleting entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, + LDAPDebug2Args( LDAP_DEBUG_ANY, "delete (deleting entryrdn) failed, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -887,21 +957,21 @@ ldbm_back_delete( Slapi_PBlock *pb ) LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } +#endif retval = entryrdn_index_entry(be, tombstone, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "adding (adding tombstone entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, - "adding (adding tombstone entryrdn) failed, err=%d %s\n", - retval, - (msg = dblayer_strerror( retval )) ? msg : "" ); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, + "adding (adding tombstone entryrdn %s) failed, err=%d %s\n", + slapi_entry_get_dn(tombstone->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : "" ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } } @@ -918,14 +988,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) SLAPI_ATTR_VALUE_TOMBSTONE, e->ep_id, BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_VALUE_TOMBSTONE, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_VALUE_TOMBSTONE, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -938,14 +1008,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_entry_get_uniqueid(e->ep_entry), e->ep_id, BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_UNIQUEID, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_UNIQUEID, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -962,14 +1032,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) nscpedn, e->ep_id, BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); slapi_ch_free((void **)&nscpedn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_NSCP_ENTRYDN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_NSCP_ENTRYDN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -988,14 +1058,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); slapi_ch_free_string(&entryusn_str); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_ENTRYUSN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_ENTRYUSN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -1009,13 +1079,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) { retval = entryrdn_index_entry(be, e, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (deleting entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, + LDAPDebug2Args( LDAP_DEBUG_ANY, "delete (deleting entryrdn) failed, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -1032,12 +1102,12 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = modify_update_all(be,pb,&parent_modify_c,&txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "del 4 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "del 4 DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "delete 3 BAD, err=%d %s\n", + LDAPDebug( LDAP_DEBUG_ANY, "delete 3 BAD, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; DEL_SET_ERROR(ldap_result_code, @@ -1055,7 +1125,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete DEADLOCK vlv_update_index\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete DEADLOCK vlv_update_index\n", 0, 0, 0 ); /* Retry txn */ continue; } @@ -1148,14 +1218,24 @@ ldbm_back_delete( Slapi_PBlock *pb ) /* delete from cache and clean up */ if (e) { + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, delete the tombstone dn from the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); + if (bdn) { /* in the dncache, remove it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } if (e_in_cache) { CACHE_REMOVE(&inst->inst_cache, e); + cache_unlock_entry(&inst->inst_cache, e); + CACHE_RETURN(&inst->inst_cache, &e); + } else { + cache_unlock_entry(&inst->inst_cache, e); } - cache_unlock_entry(&inst->inst_cache, e); - CACHE_RETURN(&inst->inst_cache, &e); e = NULL; } - + if (ruv_c_init) { if (modify_switch_entries(&ruv_c, be) != 0 ) { ldap_result_code= LDAP_OPERATIONS_ERROR; @@ -1176,16 +1256,36 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto common_return; error_return: - if (inst && tombstone_in_cache) - { - CACHE_REMOVE( &inst->inst_cache, tombstone ); - CACHE_RETURN( &inst->inst_cache, &tombstone ); - tombstone = NULL; - tombstone_in_cache = 0; + if (tombstone) { + if (entryrdn_get_switch()) { /* subtree-rename: on */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, tombstone->ep_id); + if (bdn) { /* already in the dncache. Delete it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + if (tombstone_in_cache) { /* successfully replaced */ + CACHE_REMOVE( &inst->inst_cache, tombstone ); + CACHE_RETURN( &inst->inst_cache, &tombstone ); + tombstone = NULL; + tombstone_in_cache = 0; + } else { + backentry_free( &tombstone ); + } } - else - { - backentry_free( &tombstone ); + + /* Need to return to cache after post op plugins are called */ + if (e) { + if (e_in_cache) { + if (remove_e_from_cache) { + /* The entry is already transformed to a tombstone. */ + CACHE_REMOVE( &inst->inst_cache, e ); + } + cache_unlock_entry( &inst->inst_cache, e ); + CACHE_RETURN( &inst->inst_cache, &e ); + } else { + cache_unlock_entry( &inst->inst_cache, e ); + } } if (retval == DB_RUNRECOVERY) { @@ -1246,15 +1346,33 @@ common_return: for the post op plugins */ slapi_pblock_set( pb, SLAPI_DELETE_BEPREOP_ENTRY, orig_entry ); } - if (inst && tombstone_in_cache) - { - CACHE_RETURN( &inst->inst_cache, &tombstone ); - tombstone = NULL; - tombstone_in_cache = 0; - } - else - { - backentry_free( &tombstone ); + if (inst && tombstone) { + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, + tombstone->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *tombstonesdn = slapi_sdn_dup(slapi_entry_get_sdn(tombstone->ep_entry)); + if (tombstonesdn) { + bdn = backdn_init(tombstonesdn, tombstone->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_delete", + "set %s to dn cache\n", slapi_sdn_get_dn(tombstonesdn)); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + } + } + if (tombstone_in_cache) { /* successfully replaced */ + CACHE_RETURN( &inst->inst_cache, &tombstone ); + tombstone = NULL; + tombstone_in_cache = 0; + } else { + backentry_free( &tombstone ); + } } /* result code could be used in the bepost plugin functions. */ @@ -1267,17 +1385,6 @@ common_return: plugin_call_plugins (pb, SLAPI_PLUGIN_BE_POST_DELETE_FN); } - /* Need to return to cache after post op plugins are called */ - if (inst) { - if (retval && e) { /* error case */ - cache_unlock_entry( &inst->inst_cache, e ); - CACHE_RETURN( &inst->inst_cache, &e ); - } - if (inst->inst_ref_count) { - slapi_counter_decrement(inst->inst_ref_count); - } - } - if (ruv_c_init) { modify_term(&ruv_c, be); } @@ -1291,7 +1398,7 @@ diskfull_return: } slapi_send_ldap_result( pb, ldap_result_code, NULL, ldap_result_message, 0, NULL ); } - modify_term(&parent_modify_c,be); + modify_term(&parent_modify_c, be); if (rc == 0 && opcsn && !is_fixup_operation && !delete_tombstone_entry) { /* URP Naming Collision diff --git a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c index d381166..34b7f70 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c @@ -387,7 +387,7 @@ entryrdn_index_read_ext(backend *be, rc = slapi_rdn_init_all_sdn(&srdn, sdn); if (rc < 0) { - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, + slapi_log_error(SLAPI_LOG_BACKLDBM, ENTRYRDN_TAG, "entryrdn_index_read: Param error: Failed to convert " "%s to Slapi_RDN\n", slapi_sdn_get_dn(sdn)); rc = LDAP_INVALID_DN_SYNTAX; @@ -501,7 +501,8 @@ entryrdn_rename_subtree(backend *be, Slapi_RDN *newsrdn, /* new rdn */ const Slapi_DN *newsupsdn, /* new superior dn */ ID id, - back_txn *txn) + back_txn *txn, + int flags) { int rc = -1; struct attrinfo *ai = NULL; @@ -543,7 +544,7 @@ entryrdn_rename_subtree(backend *be, goto bail; } - rc = slapi_rdn_init_all_sdn(&oldsrdn, oldsdn); + rc = slapi_rdn_init_all_sdn_ext(&oldsrdn, oldsdn, flags); if (rc < 0) { slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, "entryrdn_rename_subtree: Failed to convert olddn " @@ -969,7 +970,8 @@ entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, - back_txn *txn) + back_txn *txn, + int flags) { int rc = -1; struct attrinfo *ai = NULL; @@ -1001,7 +1003,7 @@ entryrdn_get_subordinates(backend *be, goto bail; } - rc = slapi_rdn_init_all_sdn(&srdn, sdn); + rc = slapi_rdn_init_all_sdn_ext(&srdn, sdn, flags); if (rc) { if (rc < 0) { slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modify.c b/ldap/servers/slapd/back-ldbm/ldbm_modify.c index f8fb168..3fd5683 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modify.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modify.c @@ -116,8 +116,14 @@ int modify_switch_entries(modify_context *mc,backend *be) ldbm_instance *inst = (ldbm_instance *) be->be_instance_info; int ret = 0; if (mc->old_entry!=NULL && mc->new_entry!=NULL) { - ret = cache_replace(&(inst->inst_cache), mc->old_entry, mc->new_entry); - if (ret == 0) mc->new_entry_in_cache = 1; + ret = cache_replace(&(inst->inst_cache), mc->old_entry, mc->new_entry); + if (ret == 0) { + mc->new_entry_in_cache = 1; + } else { + LDAPDebug(LDAP_DEBUG_CACHE, "modify_switch_entries: replacing %s with %s failed (%d)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), + slapi_entry_get_dn(mc->new_entry->ep_entry), ret); + } } return ret; } @@ -154,6 +160,10 @@ modify_unswitch_entries(modify_context *mc,backend *be) CACHE_RETURN( &(inst->inst_cache), &(mc->old_entry) ); mc->new_entry_in_cache = 1; mc->old_entry = NULL; + } else { + LDAPDebug(LDAP_DEBUG_CACHE, "modify_unswitch_entries: replacing %s with %s failed (%d)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), + slapi_entry_get_dn(mc->new_entry->ep_entry), ret); } } @@ -795,6 +805,12 @@ ldbm_back_modify( Slapi_PBlock *pb ) MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } + /* e uncached */ + /* we must return both e (which has been deleted) and new entry ec */ + /* cache_replace removes e from the caches */ + cache_unlock_entry( &inst->inst_cache, e ); + CACHE_RETURN( &inst->inst_cache, &e ); + /* lock new entry in cache to prevent usage until we are complete */ cache_lock_entry( &inst->inst_cache, ec ); ec_in_cache = 1; @@ -805,10 +821,6 @@ ldbm_back_modify( Slapi_PBlock *pb ) /* invalidate virtual cache */ ec->ep_entry->e_virtual_watermark = 0; - /* we must return both e (which has been deleted) and new entry ec */ - /* cache_replace removes e from the caches */ - cache_unlock_entry( &inst->inst_cache, e ); - CACHE_RETURN( &inst->inst_cache, &e ); /* * LP Fix of crash when the commit will fail: * If the commit fail, the common error path will @@ -911,28 +923,28 @@ error_return: CACHE_REMOVE( &inst->inst_cache, ec ); /* if ec was in cache, e was not - add back e */ if (e) { - CACHE_ADD( &inst->inst_cache, e, NULL ); - cache_lock_entry( &inst->inst_cache, e ); + if (CACHE_ADD( &inst->inst_cache, e, NULL )) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "ldbm_modify: CACHE_ADD %s failed\n", + slapi_entry_get_dn(e->ep_entry)); + } } } common_return: slapi_mods_done(&smods); - if (inst && ec_in_cache) - { - cache_unlock_entry( &inst->inst_cache, ec); - CACHE_RETURN( &inst->inst_cache, &ec ); - } - else - { - backentry_free(&ec); - } - if (inst) { - if (e) { - cache_unlock_entry( &inst->inst_cache, e); - CACHE_RETURN( &inst->inst_cache, &e); + if (ec_in_cache) { + cache_unlock_entry( &inst->inst_cache, ec); + CACHE_RETURN( &inst->inst_cache, &ec ); + } else { + backentry_free(&ec); + /* if ec was not in cache, cache_replace was not done. + * i.e., e was not unlocked. */ + if (e) { + cache_unlock_entry( &inst->inst_cache, e); + CACHE_RETURN( &inst->inst_cache, &e); + } } if (inst->inst_ref_count) { slapi_counter_decrement(inst->inst_ref_count); diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c index 09f1cea..02374cd 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c @@ -45,10 +45,10 @@ #include "back-ldbm.h" -static const char *moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn); +static const char *moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn, int is_tombstone); static void moddn_unlock_and_return_entry(backend *be,struct backentry **targetentry); static int moddn_newrdn_mods(Slapi_PBlock *pb, const char *olddn, struct backentry *ec, Slapi_Mods *smods_wsi, int is_repl_op); -static IDList *moddn_get_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, struct backentry *parententry, Slapi_DN *parentdn, struct backentry ***child_entries, struct backdn ***child_dns); +static IDList *moddn_get_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, struct backentry *parententry, Slapi_DN *parentdn, struct backentry ***child_entries, struct backdn ***child_dns, int is_resurect_operation); static int moddn_rename_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, IDList *children, Slapi_DN *dn_parentdn, Slapi_DN *dn_newsuperiordn, struct backentry *child_entries[]); static int modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbminfo *li, struct backentry *e, struct backentry **ec, Slapi_Mods *smods1, Slapi_Mods *smods2, Slapi_Mods *smods3, int *e_in_cache, int *ec_in_cache); static void mods_remove_nsuniqueid(Slapi_Mods *smods); @@ -110,6 +110,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) Slapi_Operation *operation; int is_replicated_operation= 0; int is_fixup_operation = 0; + int is_resurect_operation = 0; + int is_tombstone = 0; entry_address new_addr; entry_address *old_addr; entry_address oldparent_addr; @@ -136,6 +138,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_IS_REPLICATED_OPERATION, &is_replicated_operation ); is_ruv = operation_is_flag_set(operation, OP_FLAG_REPL_RUV); is_fixup_operation = operation_is_flag_set(operation, OP_FLAG_REPL_FIXUP); + is_resurect_operation = operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY); + is_tombstone = operation_is_flag_set(operation,OP_FLAG_TOMBSTONE_ENTRY); /* tombstone_to_glue on parent entry*/ if (NULL == sdn) { slapi_send_ldap_result( pb, LDAP_INVALID_DN_SYNTAX, NULL, @@ -169,7 +173,13 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, &dn_newsuperiordn ); slapi_sdn_init_dn_byref(&dn_newrdn, newrdn); /* slapi_sdn_init_normdn_byref(&dn_newsuperiordn, newsuperiordn); */ - slapi_sdn_get_parent(sdn, &dn_parentdn); + if (is_resurect_operation) { + /* no need to free this pdn. */ + const char *pdn = slapi_dn_find_parent_ext(slapi_sdn_get_dn(sdn), is_resurect_operation); + slapi_sdn_set_dn_byval(&dn_parentdn, pdn); + } else { + slapi_sdn_get_parent(sdn, &dn_parentdn); + } } /* if old and new superior are equals, newsuperior should not be set @@ -267,7 +277,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } /* make sure the original entry is back in the cache if it was removed */ if (!e_in_cache) { - CACHE_ADD(&inst->inst_cache, e, NULL); + if (CACHE_ADD(&inst->inst_cache, e, NULL)) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_back_modrdn: CACHE_ADD %s to cache failed\n", + slapi_entry_get_dn_const(e->ep_entry)); + } e_in_cache = 1; } slapi_pblock_get( pb, SLAPI_MODRDN_EXISTING_ENTRY, &ent ); @@ -310,11 +324,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; } - if (ruv_c_init) { - /* reset the ruv txn stuff */ - modify_term(&ruv_c, be); - ruv_c_init = 0; - } + if (ruv_c_init) { + /* reset the ruv txn stuff */ + modify_term(&ruv_c, be); + ruv_c_init = 0; + } /* We're re-trying */ LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "Modrdn Retrying Transaction\n"); @@ -365,7 +379,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) * it's passed to slapi_sdn_init_normdn_byref */ slapi_pblock_get(pb, SLAPI_MODRDN_NEWRDN, &newrdn); slapi_sdn_init_dn_byref(&dn_newrdn, newrdn); - newdn = moddn_get_newdn(pb,sdn, &dn_newrdn, dn_newsuperiordn); + newdn = moddn_get_newdn(pb,sdn, &dn_newrdn, dn_newsuperiordn, is_tombstone); slapi_sdn_set_dn_passin(&dn_newdn,newdn); new_addr.sdn = &dn_newdn; new_addr.udn = NULL; @@ -486,13 +500,15 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; /* error result sent by find_entry2modify() */ } e_in_cache = 1; /* e is in the cache and locked */ - if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) ) { - ldap_result_code = LDAP_UNWILLING_TO_PERFORM; + if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) && + !is_resurect_operation) { + ldap_result_code = LDAP_UNWILLING_TO_PERFORM; ldap_result_message = "Operation not allowed on tombstone entry."; - slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", - "Attempt to rename a tombstone entry %s\n", slapi_sdn_get_dn(slapi_entry_get_sdn_const( e->ep_entry ))); - goto error_return; - } + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "Attempt to rename a tombstone entry %s\n", + slapi_sdn_get_dn(slapi_entry_get_sdn_const( e->ep_entry ))); + goto error_return; + } /* Check that an entry with the same DN doesn't already exist. */ { Slapi_Entry *entry; @@ -509,7 +525,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* Fetch and lock the parent of the entry that is moving */ oldparent_addr.sdn = &dn_parentdn; - oldparent_addr.uniqueid = NULL; + if (is_resurect_operation) { + oldparent_addr.uniqueid = operation->o_params.p.p_modrdn.modrdn_newsuperior_address.uniqueid; + } else { + oldparent_addr.uniqueid = NULL; + } parententry = find_entry2modify_only( pb, be, &oldparent_addr, &txn ); modify_init(&parent_modify_context,parententry); @@ -517,7 +537,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) if(slapi_sdn_get_ndn(dn_newsuperiordn) != NULL) { slapi_pblock_get (pb, SLAPI_MODRDN_NEWSUPERIOR_ADDRESS, &newsuperior_addr); + if (is_resurect_operation) { + newsuperior_addr->uniqueid = slapi_entry_attr_get_charptr(e->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + } newparententry = find_entry2modify_only( pb, be, newsuperior_addr, &txn ); + slapi_ch_free_string(&newsuperior_addr->uniqueid); modify_init(&newparent_modify_context,newparententry); } @@ -560,8 +584,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } slapi_pblock_set( pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, dn_newsuperiordn ); } - slapi_sdn_set_dn_passin(&dn_newdn, - moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn)); + slapi_sdn_set_dn_passin(&dn_newdn, moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn, is_tombstone)); /* Check that we're allowed to add an entry below the new superior */ if ( newparententry == NULL ) @@ -662,6 +685,57 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_rdn_done(&srdn); } + if(is_resurect_operation) { + slapi_log_error(SLAPI_LOG_REPL, "ldbm_back_modrdn", + "Resurrecting an entry %s\n", slapi_entry_get_dn(ec->ep_entry)); + slapi_entry_attr_delete(ec->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + slapi_entry_delete_string(ec->ep_entry, SLAPI_ATTR_OBJECTCLASS, SLAPI_ATTR_VALUE_TOMBSTONE); + /* Now also remove the nscpEntryDN */ + if (slapi_entry_attr_delete(ec->ep_entry, SLAPI_ATTR_NSCP_ENTRYDN) != 0){ + LDAPDebug(LDAP_DEBUG_REPL, "Resurrection of %s - Couldn't remove %s\n", + slapi_entry_get_dn(ec->ep_entry), SLAPI_ATTR_NSCP_ENTRYDN, 0); + } + + /* Set the reason (this is only a reason why modrdn is needed for resurrection) */ + slapi_entry_add_string(ec->ep_entry, "nsds5ReplConflict", "deletedEntryHasChildren"); + + /* Clear the Tombstone Flag in the entry */ + slapi_entry_clear_flag(ec->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE); + + /* make sure the objectclass + - does not contain any duplicate values + - has CSNs for the new values we added + */ + { + Slapi_Attr *sa = NULL; + Slapi_Value sv; + const struct berval *svbv = NULL; + + /* add the extensibleobject objectclass with csn if not present */ + slapi_entry_attr_find(ec->ep_entry, SLAPI_ATTR_OBJECTCLASS, &sa); + slapi_value_init_string(&sv, "extensibleobject"); + svbv = slapi_value_get_berval(&sv); + if (slapi_attr_value_find(sa, svbv)) { /* not found, so add it */ + if (opcsn) { + value_update_csn(&sv, CSN_TYPE_VALUE_UPDATED, opcsn); + } + slapi_attr_add_value(sa, &sv); + } + value_done(&sv); + + /* add the glue objectclass with csn if not present */ + slapi_value_init_string(&sv, "glue"); + svbv = slapi_value_get_berval(&sv); + if (slapi_attr_value_find(sa, svbv)) { /* not found, so add it */ + if (opcsn) { + value_update_csn(&sv, CSN_TYPE_VALUE_UPDATED, opcsn); + } + slapi_attr_add_value(sa, &sv); + } + value_done(&sv); + } + } + /* create it in the cache - prevents others from creating it */ if (( cache_add_tentative( &inst->inst_cache, ec, NULL ) != 0 ) ) { ec_in_cache = 0; /* not in cache */ @@ -671,6 +745,10 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* somebody must've created it between dn2entry() and here */ /* JCMREPL - Hmm... we can't permit this to happen...? */ ldap_result_code= LDAP_ALREADY_EXISTS; + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "cache_add_tentative failed: %s\n", slapi_entry_get_dn(ec->ep_entry)); + } goto error_return; } /* so if the old dn is the same as the new dn, the entry will not be cached @@ -718,8 +796,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } } - slapi_pblock_get( pb, SLAPI_MODIFY_MODS, &mods ); - slapi_mods_init_byref(&smods_operation_wsi,mods); + slapi_pblock_get( pb, SLAPI_MODIFY_MODS, &mods ); + slapi_mods_init_byref(&smods_operation_wsi,mods); /* * We are about to pass the last abandon test, so from now on we are @@ -809,48 +887,81 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* * Update the subordinate count of the parents to reflect the moved child. */ - if ( parententry!=NULL ) - { + if (parententry) { retval = parent_update_on_childchange(&parent_modify_context, PARENTUPDATE_DEL, NULL); /* The parent modify context now contains info needed later */ - if (0 != retval) - { + if (retval) { goto error_return; } } - if ( newparententry!=NULL ) - { + if (newparententry) { retval = parent_update_on_childchange(&newparent_modify_context, PARENTUPDATE_ADD, NULL); /* The newparent modify context now contains info needed later */ - if (0 != retval) - { + if (retval) { goto error_return; } } } + /* is_resurect_operation case, there's no new superior. Just rename. */ + if (is_resurect_operation && parententry) { + retval = parent_update_on_childchange(&parent_modify_context, PARENTUPDATE_RESURECT, NULL); + if (retval) { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "parent_update_on_childchange parent %s of %s failed, rc=%d\n", + slapi_entry_get_dn_const(parent_modify_context.old_entry->ep_entry), + slapi_entry_get_dn_const(ec->ep_entry), retval); + goto error_return; + } + } /* - * If the entry has children then we're going to have to rename them all. + * If the entry has children including tombstones, + * then we're going to have to rename them all. */ - if (slapi_entry_has_children( e->ep_entry )) - { + if (slapi_entry_has_children_ext(e->ep_entry, 1)) { /* JCM - This is where the subtree lock will appear */ if (entryrdn_get_switch()) /* subtree-rename: on */ { + if (is_resurect_operation) { +#if defined(DEBUG) + /* Get the present value of the subcount attr, or 0 if not present */ + Slapi_Attr *read_attr = NULL; + int sub_count = -1; + if (0 == slapi_entry_attr_find(parent_modify_context.old_entry->ep_entry, + "numsubordinates", &read_attr)) { + /* decode the value */ + Slapi_Value *sval; + slapi_attr_first_value(read_attr, &sval); + if (sval) { + const struct berval *bval = slapi_value_get_berval(sval); + if(bval) { + sub_count = atol(bval->bv_val); + } + } + } + LDAPDebug(LDAP_DEBUG_ANY, "parent_update_on_childchange parent %s of %s numsub=%d\n", + slapi_entry_get_dn_const(parent_modify_context.old_entry->ep_entry), + slapi_entry_get_dn_const(e->ep_entry), sub_count); +#endif + slapi_log_error(SLAPI_LOG_BACKLDBM, "ldbm_back_modrdn", + "%s has children\n", slapi_entry_get_dn(e->ep_entry)); + } children = moddn_get_children(&txn, pb, be, e, sdn, - &child_entries, &child_dns); + &child_entries, &child_dns, is_resurect_operation); } else { children = moddn_get_children(&txn, pb, be, e, sdn, - &child_entries, NULL); + &child_entries, NULL, 0); } /* JCM - Shouldn't we perform an access control check on all the children. */ /* JCMREPL - But, the replication client has total rights over its subtree, so no access check needed. */ /* JCM - A subtree move could break ACIs, static groups, and dynamic groups. */ + } else if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "%s has NO children\n", slapi_entry_get_dn(e->ep_entry)); } /* @@ -904,10 +1015,10 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (retval != 0 ) + if (retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "modrdn_rename_entry_update_indexes failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + LDAPDebug(LDAP_DEBUG_ANY, "modrdn_rename_entry_update_indexes %s --> %s failed, err=%d\n", + slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn(ec->ep_entry), retval); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; @@ -1027,14 +1138,17 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_rdn_init_sdn(&newsrdn, (const Slapi_DN *)&dn_newdn); retval = entryrdn_rename_subtree(be, (const Slapi_DN *)sdn, &newsrdn, (const Slapi_DN *)dn_newsuperiordn, - e->ep_id, &txn); + e->ep_id, &txn, is_tombstone); slapi_rdn_done(&newsrdn); if (retval != 0) { if (retval == DB_LOCK_DEADLOCK) continue; if (retval == DB_RUNRECOVERY || LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - MOD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "entryrdn_rename_subtree failed (%d); dn: %s, newsrdn: %s, dn_newsuperiordn: %s\n", + retval, slapi_sdn_get_dn(sdn), slapi_rdn_get_rdn(&newsrdn), + slapi_sdn_get_dn(dn_newsuperiordn)); goto error_return; } } @@ -1060,7 +1174,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; } - if (!is_ruv && !is_fixup_operation && !NO_RUV_UPDATE(li)) { + if (!is_ruv && !is_fixup_operation && !NO_RUV_UPDATE(li)) { ruv_c_init = ldbm_txn_ruv_modify_context( pb, &ruv_c ); if (-1 == ruv_c_init) { LDAPDebug( LDAP_DEBUG_ANY, @@ -1160,6 +1274,12 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) * "renamed" dn is generated based upon the moved subtree. */ for (i = 0; child_entries[i] != NULL; i++) { + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "Calling cache remove & return %s (refcnt: %d)\n", + slapi_entry_get_dn(child_entries[i]->ep_entry), + child_entries[i]->ep_refcnt); + } CACHE_REMOVE( &inst->inst_cache, child_entries[i] ); cache_unlock_entry( &inst->inst_cache, child_entries[i] ); CACHE_RETURN( &inst->inst_cache, &child_entries[i] ); @@ -1196,21 +1316,13 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) error_return: /* result already sent above - just free stuff */ - if ( NULL != postentry ) - { + if (postentry) { slapi_entry_free( postentry ); postentry= NULL; /* make sure caller doesn't attempt to free this */ slapi_pblock_set( pb, SLAPI_ENTRY_POST_OP, postentry ); } - if (e && entryrdn_get_switch() && inst) - { - struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); - CACHE_REMOVE(&inst->inst_dncache, bdn); - CACHE_RETURN(&inst->inst_dncache, &bdn); - } - if(children) - { + if(children) { int i = 0; if (child_entries && *child_entries && inst) { @@ -1308,9 +1420,32 @@ common_return: /* Free up the resource we don't need any more */ if (ec) { + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_REPL, "ldbm_back_modrdn", + "Resurrecting an entry %s: result: %d, %d\n", + slapi_entry_get_dn(ec->ep_entry), ldap_result_code, retval); + } + if (inst && (0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, ec->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *ecsdn = slapi_sdn_dup(slapi_entry_get_sdn(ec->ep_entry)); + if (ecsdn) { + bdn = backdn_init(ecsdn, ec->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + CACHE_RETURN(&inst->inst_dncache, &bdn); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "set %s to dn cache\n", slapi_sdn_get_dn(sdn)); + } + } + } + } /* remove the new entry from the cache if the op failed - otherwise, leave it in */ - if (ec_in_cache && inst) { + if (ec_in_cache && ec && inst) { if (retval) { CACHE_REMOVE( &inst->inst_cache, ec ); } @@ -1322,16 +1457,16 @@ common_return: ec_in_cache = 0; } - /* put e back in the cache if the modrdn failed */ - if (e) { - if (!e_in_cache && retval && inst) { - CACHE_ADD(&inst->inst_cache, e, NULL); - e_in_cache = 1; + if (inst) { + if (e && entryrdn_get_switch() && (0 == retval)) { + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); } - } - if (inst && inst->inst_ref_count) { - slapi_counter_decrement(inst->inst_ref_count); - } + if (inst->inst_ref_count) { + slapi_counter_decrement(inst->inst_ref_count); + } + } moddn_unlock_and_return_entry(be,&e); @@ -1341,11 +1476,11 @@ common_return: if (ldap_result_code!=-1) { - if (not_an_error) { - /* This is mainly used by urp. Solved conflict is not an error. - * And we don't want the supplier to halt sending the updates. */ - ldap_result_code = LDAP_SUCCESS; - } + if (not_an_error) { + /* This is mainly used by urp. Solved conflict is not an error. + * And we don't want the supplier to halt sending the updates. */ + ldap_result_code = LDAP_SUCCESS; + } slapi_send_ldap_result( pb, ldap_result_code, ldap_result_matcheddn, ldap_result_message, 0,NULL ); } @@ -1394,7 +1529,7 @@ common_return: * Work out what the new DN of the entry will be. */ static const char * -moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn) +moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn, int is_tombstone) { char *newdn; const char *newrdn= slapi_sdn_get_dn(dn_newrdn); @@ -1415,18 +1550,18 @@ moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi else { /* construct the new dn */ - char *pdn; - const char *dn= slapi_sdn_get_dn(dn_olddn); - pdn = slapi_dn_beparent( pb, dn ); - if ( pdn != NULL ) - { - newdn= slapi_dn_plus_rdn(pdn, newrdn); /* JCM - Use Slapi_RDN */ - } - else - { - newdn= slapi_ch_strdup(newrdn); + const char *dn = slapi_sdn_get_dn((const Slapi_DN *)dn_olddn); + if (slapi_dn_isbesuffix(pb, dn)) { + newdn = slapi_ch_strdup(newrdn); + } else { + /* no need to free this pdn. */ + const char *pdn = slapi_dn_find_parent_ext(dn, is_tombstone); + if (pdn) { + newdn = slapi_dn_plus_rdn(pdn, newrdn); + } else { + newdn = slapi_ch_strdup(newrdn); + } } - slapi_ch_free( (void**)&pdn ); } return newdn; } @@ -1653,6 +1788,7 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm Slapi_Operation *operation; int is_ruv = 0; /* True if the current entry is RUV */ int orig_ec_in_cache = 0; + int cache_rc = 0; slapi_pblock_get( pb, SLAPI_BACKEND, &be ); slapi_pblock_get( pb, SLAPI_OPERATION, &operation ); @@ -1664,7 +1800,12 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm * Update the ID to Entry index. * Note that id2entry_add replaces the entry, so the Entry ID stays the same. */ - retval = id2entry_add( be, *ec, ptxn ); + retval = id2entry_add_ext(be, *ec, ptxn, 1, &cache_rc); + if (cache_rc) { + LDAPDebug(LDAP_DEBUG_CACHE, + "modrdn_rename_entry_update_indexes: adding %s failed to add to the cache (rc: %d, cache_rc: %d)\n", + slapi_entry_get_dn(e->ep_entry), retval, cache_rc); + } if (DB_LOCK_DEADLOCK == retval) { /* Retry txn */ @@ -1758,11 +1899,11 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm } } if (cache_replace( &inst->inst_cache, e, *ec ) != 0 ) { - LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "modrdn_rename_entry_update_indexes cache_replace failed\n"); + LDAPDebug2Args(LDAP_DEBUG_CACHE, + "modrdn_rename_entry_update_indexes cache_replace %s -> %s failed\n", + slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn((*ec)->ep_entry)); retval= -1; goto error_return; - } else { - *e_in_cache = 0; /* e un-cached */ } if (orig_ec_in_cache) { /* ec was already added to the cache via cache_add_tentative (to reserve its spot in the cache) @@ -1978,7 +2119,8 @@ moddn_get_children(back_txn *ptxn, struct backentry *parententry, Slapi_DN *dn_parentdn, struct backentry ***child_entries, - struct backdn ***child_dns) + struct backdn ***child_dns, + int is_resurect_operation) { ldbm_instance *inst = (ldbm_instance *) be->be_instance_info; int err= 0; @@ -2006,7 +2148,7 @@ moddn_get_children(back_txn *ptxn, { err = entryrdn_get_subordinates(be, slapi_entry_get_sdn_const(parententry->ep_entry), - parententry->ep_id, &candidates, ptxn); + parententry->ep_id, &candidates, ptxn, is_resurect_operation); if (err) { LDAPDebug1Arg( LDAP_DEBUG_ANY, "moddn_get_children: " "entryrdn_get_subordinates returned %d\n", err); @@ -2025,8 +2167,13 @@ moddn_get_children(back_txn *ptxn, slapi_filter_free(filter,1); } - if (candidates!=NULL) - { + if (candidates) { + Slapi_DN parentsdn = {0}; + if (is_resurect_operation) { + slapi_sdn_get_parent(dn_parentdn, &parentsdn); + dn_parentdn = &parentsdn; + } + sr_current = idl_iterator_init(candidates); result_idl= idl_alloc(candidates->b_nids); do @@ -2059,6 +2206,7 @@ moddn_get_children(back_txn *ptxn, } } while (id!=NOID); idl_free(candidates); + slapi_sdn_done(&parentsdn); } nids = result_idl ? result_idl->b_nids : 0; diff --git a/ldap/servers/slapd/back-ldbm/ldbm_search.c b/ldap/servers/slapd/back-ldbm/ldbm_search.c index 1dcf698..d0563bd 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_search.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_search.c @@ -1209,7 +1209,7 @@ subtree_candidates( /* subtree-rename: on && no ancestorid */ *err = entryrdn_get_subordinates(be, slapi_entry_get_sdn_const(e->ep_entry), - e->ep_id, &descendants, &txn); + e->ep_id, &descendants, &txn, 0); idl_insert(&descendants, e->ep_id); candidates = idl_intersection(be, candidates, descendants); idl_free(tmp); diff --git a/ldap/servers/slapd/back-ldbm/ldif2ldbm.c b/ldap/servers/slapd/back-ldbm/ldif2ldbm.c index 7d93fa0..d803f5d 100644 --- a/ldap/servers/slapd/back-ldbm/ldif2ldbm.c +++ b/ldap/servers/slapd/back-ldbm/ldif2ldbm.c @@ -900,7 +900,7 @@ static IDList *ldbm_fetch_subtrees(backend *be, char **include, int *err) */ if (entryrdn_get_noancestorid()) { /* subtree-rename: on && no ancestorid */ - *err = entryrdn_get_subordinates(be, &sdn, id, &idl, txn); + *err = entryrdn_get_subordinates(be, &sdn, id, &idl, txn, 0); } else { *err = ldbm_ancestorid_read(be, txn, id, &idl); } diff --git a/ldap/servers/slapd/back-ldbm/parents.c b/ldap/servers/slapd/back-ldbm/parents.c index 59e8763..d375519 100644 --- a/ldap/servers/slapd/back-ldbm/parents.c +++ b/ldap/servers/slapd/back-ldbm/parents.c @@ -58,11 +58,17 @@ char *tombstone_numsubordinates = LDBM_TOMBSTONE_NUMSUBORDINATES_STR; * The routine is allowed to modify the parent entry, and to return a set of * LDAPMods reflecting the changes it made. The LDAPMods array must be freed * by the called by calling ldap_free_mods(p,1) + * + * PARENTUPDATE_RESURECT == turning a tombstone into an entry + * tombstone_numsubordinates-- + * numsubordinates++ */ /* - * PARENTUPDATE_CREATE_TOMBSTONE: increment tombstone_numsubordinates + * PARENTUPDATE_CREATE_TOMBSTONE: turning an entry into a tombstone + * numsubordinates-- + * tombstone_numsubordinates++ * PARENTUPDATE_DELETE_TOMBSTONE: don't touch numsubordinates, and - * decrement tombstone_numsubordinates + * tombstone_numsubordinates-- */ int @@ -84,7 +90,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) op &= PARENTUPDATE_MASK; /* Check nobody is trying to use op == 3, it's not implemented yet */ - PR_ASSERT( (op == PARENTUPDATE_ADD) || (op == PARENTUPDATE_DEL)); + PR_ASSERT((op == PARENTUPDATE_ADD) || (op == PARENTUPDATE_DEL) || (op == PARENTUPDATE_RESURECT)); /* We want to invent a mods set to be passed to modify_apply_mods() */ @@ -114,16 +120,17 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) if (PARENTUPDATE_DELETE_TOMBSTONE != repl_op) { /* are we adding ? */ - if ( (PARENTUPDATE_ADD == op) && !already_present) { + if (((PARENTUPDATE_ADD == op) || (PARENTUPDATE_RESURECT == op)) && !already_present) { /* If so, and the parent entry does not already have a subcount * attribute, we need to add it */ mod_op = LDAP_MOD_ADD; } else if (PARENTUPDATE_DEL == op) { if (!already_present) { /* This means that something is wrong---deleting a child but no subcount present on parent */ - LDAPDebug0Args( LDAP_DEBUG_ANY, - "numsubordinates assertion failure\n" ); + LDAPDebug(LDAP_DEBUG_ANY, "Parent %s has no children. (op 0x%x, repl_op 0x%x)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), op, repl_op); slapi_mods_free(&smods); + PR_ASSERT(0); return -1; } else { if (current_sub_count == 1) { @@ -138,7 +145,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) } /* Now compute the new value */ - if (PARENTUPDATE_ADD == op) { + if ((PARENTUPDATE_ADD == op) || (PARENTUPDATE_RESURECT == op)) { current_sub_count++; } else { current_sub_count--; @@ -159,8 +166,8 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) /* tombstoneNumSubordinates is needed only when this is repl op * and a child is being deleted */ - if (repl_op && (PARENTUPDATE_DEL == op)) { - current_sub_count = LDAP_MAXINT; + current_sub_count = LDAP_MAXINT; + if ((repl_op && (PARENTUPDATE_DEL == op)) || (PARENTUPDATE_RESURECT == op)) { ret = slapi_entry_attr_find(mc->old_entry->ep_entry, tombstone_numsubordinates, &read_attr); if (0 == ret) { @@ -175,7 +182,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) } } - if (PARENTUPDATE_DELETE_TOMBSTONE == repl_op) { + if ((PARENTUPDATE_DELETE_TOMBSTONE == repl_op) || (PARENTUPDATE_RESURECT == op)) { /* deleting a tombstone entry: * reaping or manually deleting it */ if ((current_sub_count != LDAP_MAXINT) && @@ -187,9 +194,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) tombstone_numsubordinates, strlen(value_buffer), value_buffer); } - } - - if (PARENTUPDATE_CREATE_TOMBSTONE == repl_op) { + } else if (PARENTUPDATE_CREATE_TOMBSTONE == repl_op) { /* creating a tombstone entry */ if (current_sub_count != LDAP_MAXINT) { current_sub_count++; diff --git a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h index f602b2b..97636b8 100644 --- a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h @@ -200,6 +200,7 @@ struct backentry *dn2entry_ext(Slapi_Backend *be, const Slapi_DN *sdn, back_txn struct backentry *dn2entry_or_ancestor(Slapi_Backend *be, const Slapi_DN *sdn, Slapi_DN *ancestor, back_txn *txn, int *err); struct backentry *dn2ancestor(Slapi_Backend *be,const Slapi_DN *sdn,Slapi_DN *ancestordn,back_txn *txn,int *err); int get_copy_of_entry(Slapi_PBlock *pb, const entry_address *addr, back_txn *txn, int plock_parameter, int must_exist); +int get_copy_of_entry_ext(Slapi_PBlock *pb, ID id, const entry_address *addr, back_txn *txn, int plock_parameter, int must_exist); void done_with_pblock_entry(Slapi_PBlock *pb, int plock_parameter); /* @@ -715,8 +716,8 @@ int entryrdn_index_entry(backend *be, struct backentry *e, int flags, back_txn * int entryrdn_index_read(backend *be, const Slapi_DN *sdn, ID *id, back_txn *txn); int entryrdn_index_read_ext(backend *be, const Slapi_DN *sdn, ID *id, int flags, back_txn *txn); -int entryrdn_rename_subtree(backend *be, const Slapi_DN *oldsdn, Slapi_RDN *newsrdn, const Slapi_DN *newsupsdn, ID id, back_txn *txn); -int entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, back_txn *txn); +int entryrdn_rename_subtree(backend *be, const Slapi_DN *oldsdn, Slapi_RDN *newsrdn, const Slapi_DN *newsupsdn, ID id, back_txn *txn, int flags); +int entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, back_txn *txn, int flags); int entryrdn_lookup_dn(backend *be, const char *rdn, ID id, char **dn, Slapi_RDN **psrdn, back_txn *txn); int entryrdn_get_parent(backend *be, const char *rdn, ID id, char **prdn, ID *pid, back_txn *txn); #endif diff --git a/ldap/servers/slapd/entry.c b/ldap/servers/slapd/entry.c index 12a9317..0679eb7 100644 --- a/ldap/servers/slapd/entry.c +++ b/ldap/servers/slapd/entry.c @@ -3310,9 +3310,10 @@ slapi_entry_add_rdn_values( Slapi_Entry *e ) * Author/Modifier: RJP */ int -slapi_entry_has_children(const Slapi_Entry *entry) +slapi_entry_has_children_ext(const Slapi_Entry *entry, int include_tombstone) { Slapi_Attr *attr; + int count = 0; LDAPDebug( LDAP_DEBUG_TRACE, "=> slapi_has_children( %s )\n", slapi_entry_get_dn_const(entry), 0, 0); @@ -3322,15 +3323,34 @@ slapi_entry_has_children(const Slapi_Entry *entry) Slapi_Value *sval; slapi_attr_first_value( attr, &sval ); if(sval!=NULL) - { + { const struct berval *bval = slapi_value_get_berval( sval ); if(bval!=NULL) { /* The entry has the attribute, and it's non-zero */ - if (strcmp(bval->bv_val, "0") != 0) - { - LDAPDebug( LDAP_DEBUG_TRACE, "<= slapi_has_children 1\n", 0, 0, 0 ); - return(1); + count = strtol(bval->bv_val, (char **)NULL, 10); + if (count > 0) { + LDAPDebug1Arg( LDAP_DEBUG_TRACE, "<= slapi_has_children %d\n", count); + return count; + } + } + } + } + /*If the subordinatecount exists, and it's nonzero, then return 1.*/ + if (include_tombstone && (slapi_entry_attr_find( entry, "tombstonenumsubordinates", &attr) == 0)) + { + Slapi_Value *sval; + slapi_attr_first_value( attr, &sval ); + if(sval!=NULL) + { + const struct berval *bval = slapi_value_get_berval( sval ); + if(bval!=NULL) + { + /* The entry has the attribute, and it's non-zero */ + count = strtol(bval->bv_val, (char **)NULL, 10); + if (count > 0) { + LDAPDebug1Arg( LDAP_DEBUG_TRACE, "<= slapi_has_tombstone_children %d\n", count); + return count; } } } @@ -3339,6 +3359,12 @@ slapi_entry_has_children(const Slapi_Entry *entry) return(0); } +int +slapi_entry_has_children(const Slapi_Entry *entry) +{ + return slapi_entry_has_children_ext(entry, 0); +} + /* * Renames an entry to simulate a MODRDN operation */ diff --git a/ldap/servers/slapd/rdn.c b/ldap/servers/slapd/rdn.c index fe2fae0..6e40ed8 100644 --- a/ldap/servers/slapd/rdn.c +++ b/ldap/servers/slapd/rdn.c @@ -124,7 +124,7 @@ slapi_rdn_init_dn(Slapi_RDN *rdn,const char *dn) * 1 -- "dn" does not belong to the database; could be "rdn" */ static int -_slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn) +_slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone) { const char *dn = NULL; const char *ndn= NULL; @@ -202,9 +202,16 @@ _slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn) } /* Get the last matched position */ - if(dns) - { - rdn->rdn = slapi_ch_strdup(dns[0]); + if (dns) { + if (is_tombstone && slapi_is_special_rdn(dns[0], RDN_IS_TOMBSTONE)) { + /* merge nsuniqueid=...,<rdn> into one rdn */ + rdn->rdn = slapi_ch_smprintf("%s,%s", dns[0], dns[1]); + slapi_ch_free_string(&dns[0]); + dns[0] = slapi_ch_strdup(rdn->rdn); + charray_remove(dns, dns[1], 1); + } else { + rdn->rdn = slapi_ch_strdup(dns[0]); + } rdn->all_rdns = dns; slapi_setbit_uchar(rdn->flag,FLAG_ALL_RDNS); } @@ -238,7 +245,7 @@ slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn) slapi_rdn_init(rdn); slapi_sdn_init(&sdn); slapi_sdn_set_dn_byval(&sdn, dn); - rc = _slapi_rdn_init_all_dn_ext(rdn, (const Slapi_DN *)&sdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, (const Slapi_DN *)&sdn, 0); slapi_sdn_done(&sdn); return rc; } @@ -257,6 +264,33 @@ slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn) * 1 -- dn does not belong to the database */ int +slapi_rdn_init_all_sdn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone) +{ + int rc = 0; /* success */ + + if (NULL == rdn || NULL == sdn) + { + return -1; + } + slapi_rdn_init(rdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, sdn, is_tombstone); + return rc; +} + +/* + * This function sets DN from sdn to Slapi_RDN. + * Note: The underlying function _slapi_rdn_init_all_dn_ext checks if the DN + * is in the root or sub suffix the server owns. If it is, the root or sub + * suffix is treated as one "rdn" (e.g., "dc=sub,dc=example,dc=com") and 0 is + * returned. If it is not, the DN is separated by ',' and each string is set + * to RDN array. (e.g., input: "uid=A,ou=does_not_exist" ==> "uid=A", "ou= + * does_not_exist") and 1 is returned. + * + * Return Value: 0 -- Success + * -1 -- Error + * 1 -- dn does not belong to the database + */ +int slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn) { int rc = 0; /* success */ @@ -266,7 +300,7 @@ slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn) return -1; } slapi_rdn_init(rdn); - rc = _slapi_rdn_init_all_dn_ext(rdn, sdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, sdn, 0); return rc; } @@ -291,6 +325,17 @@ slapi_rdn_init_rdn(Slapi_RDN *rdn,const Slapi_RDN *fromrdn) } void +slapi_rdn_set_dn_ext(Slapi_RDN *rdn,const char *dn, int skip_tombstone) +{ + const char *mydn = dn; + slapi_rdn_done(rdn); + if (skip_tombstone && slapi_is_special_rdn(dn, RDN_IS_TOMBSTONE)) { + mydn = dn + slapi_uniqueIDRdnSize() + 1/*,*/; + } + slapi_rdn_init_dn(rdn, mydn); +} + +void slapi_rdn_set_dn(Slapi_RDN *rdn,const char *dn) { slapi_rdn_done(rdn); @@ -473,6 +518,33 @@ slapi_rdn_contains(Slapi_RDN *rdn, const char *type, const char *value, size_t l } int +slapi_rdn_is_multivalued(Slapi_RDN *rdn) +{ + char *p = NULL; + if (rdn && rdn->rdn) { + p = PL_strchr(rdn->rdn, '+'); + } + if (p) { + return 1; + } else { + return 0; + } +} + +/* + * Return value 1: if rdn is a conflict rdn + * 0: otherwise + */ +int +slapi_rdn_is_conflict(Slapi_RDN *rdn) +{ + if (!rdn) { + return 0; + } + return slapi_is_special_rdn(slapi_rdn_get_nrdn(rdn), RDN_IS_CONFLICT); +} + +int slapi_rdn_add(Slapi_RDN *rdn, const char *type, const char *value) { PR_ASSERT(NULL != type); @@ -484,12 +556,32 @@ slapi_rdn_add(Slapi_RDN *rdn, const char *type, const char *value) } else { - /* type=value+rdn '\0' */ - char *newrdn = slapi_create_dn_string("%s=%s+%s", type, value, rdn->rdn); + char *newrdn = NULL; + char *rp = rdn->rdn; + PRUint32 uniqueidlen = slapi_uniqueIDRdnSize(); + + if (slapi_is_special_rdn(rp, RDN_IS_TOMBSTONE)) { + char *corerp = rp + uniqueidlen + 1; + *(rp + uniqueidlen) = '\0'; + newrdn = slapi_create_dn_string("%s,%s=%s+%s", rp, type, value, corerp); + } else { + /* type=value+rdn '\0' */ + newrdn = slapi_create_dn_string("%s=%s+%s", type, value, rp); + } slapi_ch_free_string(&rdn->rdn); rdn->rdn = newrdn; } slapi_unsetbit_uchar(rdn->flag,FLAG_RDNS); + + if (rdn->all_rdns && rdn->all_rdns[0]) { + slapi_ch_free_string(&rdn->all_rdns[0]); + rdn->all_rdns[0] = slapi_ch_strdup(rdn->rdn); + } + if (rdn->all_nrdns && rdn->all_nrdns[0]) { + slapi_ch_free_string(&rdn->all_nrdns[0]); + rdn->all_nrdns[0] = slapi_ch_strdup(rdn->rdn); + slapi_dn_ignore_case(rdn->all_nrdns[0]); + } return 1; } diff --git a/ldap/servers/slapd/slapi-plugin.h b/ldap/servers/slapd/slapi-plugin.h index 596d4db..af4508f 100644 --- a/ldap/servers/slapd/slapi-plugin.h +++ b/ldap/servers/slapd/slapi-plugin.h @@ -1979,6 +1979,17 @@ int slapi_entry_attr_has_syntax_value(const Slapi_Entry *e, const char *type, co int slapi_entry_has_children(const Slapi_Entry *e); /** + * This function determines if the specified entry has child entries + * including the tombstoned descendents. + * + * \param e Entry that you want to test for child entries. + * \param include_tombstone If non-zero, check tombstone_subordinates, too. + * \return \c 1 if the entry you supply has child entries. + * \return \c 0 if the entry you supply has child entries. + */ +int slapi_entry_has_children_ext(const Slapi_Entry *e, int include_tombstone); + +/** * This function determines if an entry is the root DSE. * * The root DSE is a special entry that contains information about the Directory @@ -3126,9 +3137,11 @@ void slapi_rdn_init_rdn(Slapi_RDN *rdn,const Slapi_RDN *fromrdn); * \see slapi_rdn_set_rdn() */ void slapi_rdn_set_dn(Slapi_RDN *rdn,const char *dn); +void slapi_rdn_set_dn_ext(Slapi_RDN *rdn,const char *dn, int skip_tombstone); Slapi_RDN *slapi_rdn_new_all_dn(const char *dn); int slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn); int slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn); +int slapi_rdn_init_all_sdn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone); /** * Sets the RDN value in a \c Slapi_RDN structure from a \c Slapi_DN. @@ -3563,6 +3576,24 @@ size_t slapi_rdn_get_size(Slapi_RDN *srdn); */ char * slapi_rdn_get_value(const char *rdn); +/** + * Check if the rdn is multivalued or not + * + * \param rdn A pointer to rdn to exam. + * \return 1, if the rdn is multi valued. + * 0, if the rdn is simgle valued. + */ +int slapi_rdn_is_multivalued(Slapi_RDN *rdn); + +/** + * Check if the rdn is a conflict rdn or not + * + * \param rdn A pointer to rdn to exam. + * \return 1, if the rdn is a conflict rdn + * 0, if the rdn is not a conflict rdn + */ +int slapi_rdn_is_conflict(Slapi_RDN *rdn); + /* * utility routines for dealing with DNs */ @@ -7798,6 +7829,20 @@ char *slapi_pw_get_scheme_name(PWScheme *pass_scheme); */ void slapi_free_pw_scheme(PWScheme *pass_scheme); +/** Check if rdn is a slecial rdn/dn or not. + * + * \param rdn rdn/dn to check + * \param flags specify the type: RDN_IS_TOMBSTONE or RDN_IS_CONFLICT or 0 + * + * \return 1 if rdn matches the flag. + * If flag is IS_TOMBSTONE, returns 1 if rdn is a tombstone rdn/dn. + * If flag is IS_CONFLICT, returns 1 if rdn is a conflict rdn/dn. + * \return 0 otherwise + */ +#define RDN_IS_TOMBSTONE 0x1 +#define RDN_IS_CONFLICT 0x2 +int slapi_is_special_rdn(const char *rdn, int flag); + #ifdef __cplusplus } #endif diff --git a/ldap/servers/slapd/slapi-private.h b/ldap/servers/slapd/slapi-private.h index 0094bea..1b22d9e 100644 --- a/ldap/servers/slapd/slapi-private.h +++ b/ldap/servers/slapd/slapi-private.h @@ -1015,6 +1015,13 @@ int slapi_uniqueIDIsUUID(const Slapi_UniqueID *uId); */ int slapi_uniqueIDSize( void ); +/* Name: slapi_uniqueIDRdnSize + Description: returns size of SLAPI_ATTR_UNIQUEID=slapi_uniqueIDSize() + Parameters: none + Return: size of the string version of "SLAPI_ATTR_UNIQUEID=uniqueID" in bytes + */ +int slapi_uniqueIDRdnSize( void ); + /* Name: slapi_uniqueIDDup Description: duplicates an UniqueID object Parameters: uId - id to duplicate diff --git a/ldap/servers/slapd/util.c b/ldap/servers/slapd/util.c index c37f6c4..4c23af2 100644 --- a/ldap/servers/slapd/util.c +++ b/ldap/servers/slapd/util.c @@ -1444,3 +1444,70 @@ slapi_get_plugin_name(const char *path, const char *lib) return fullname; } + +/* + * Check if rdn is a slecial rdn/dn or not. + * If flag is IS_TOMBSTONE, returns 1 if rdn is a tombstone rdn/dn. + * If flag is IS_CONFLICT, returns 1 if rdn is a conflict rdn/dn. + * Otherwise returns 0. + */ +static int util_uniqueidlen = 0; +int +slapi_is_special_rdn(const char *rdn, int flag) +{ + char *rp; + int plus = 0; + if (!util_uniqueidlen) { + util_uniqueidlen = SLAPI_ATTR_UNIQUEID_LENGTH + slapi_uniqueIDSize() + 1/*=*/; + } + + if ((RDN_IS_TOMBSTONE != flag) && (RDN_IS_CONFLICT != flag)) { + LDAPDebug1Arg(LDAP_DEBUG_ANY, "slapi_is_special_rdn: invalid flag %d\n", flag); + return 0; /* not a special rdn/dn */ + } + if (!rdn) { + LDAPDebug0Args(LDAP_DEBUG_ANY, "slapi_is_special_rdn: NULL rdn\n"); + return 0; /* not a special rdn/dn */ + } + + if (strlen(rdn) < util_uniqueidlen) { + return 0; /* not a special rdn/dn */ + } + rp = (char *)rdn; + while (rp) { + char *comma = NULL; + if (!PL_strncasecmp(rp, SLAPI_ATTR_UNIQUEID, SLAPI_ATTR_UNIQUEID_LENGTH) && + (*(rp + SLAPI_ATTR_UNIQUEID_LENGTH) == '=')) { + if (RDN_IS_TOMBSTONE == flag) { + if ((*(rp + util_uniqueidlen) == ',') || + (*(rp + util_uniqueidlen) == '\0')) { + return 1; + } else { + return 0; + } + } else { + if ((*(rp + util_uniqueidlen) == '+') || + (plus && ((*(rp + util_uniqueidlen) == ',') || + (*(rp + util_uniqueidlen) == '\0')))) { + return 1; + } + } + } + comma = PL_strchr(rp, ','); + rp = PL_strchr(rp, '+'); + if (rp && (rp < comma)) { + plus = 1; + rp++; + } + } + return 0; +} + +int +slapi_uniqueIDRdnSize() +{ + if (!util_uniqueidlen) { + util_uniqueidlen = SLAPI_ATTR_UNIQUEID_LENGTH + slapi_uniqueIDSize() + 1/*=*/; + } + return util_uniqueidlen; +}
1
0
0
0
ldap/servers
by Noriko Hosoi
12 May '14
12 May '14
ldap/servers/plugins/replication/repl5.h | 1 ldap/servers/plugins/replication/urp.c | 94 +++--- ldap/servers/plugins/replication/urp.h | 5 ldap/servers/plugins/replication/urp_glue.c | 32 +- ldap/servers/plugins/replication/urp_tombstone.c | 56 +++ ldap/servers/slapd/add.c | 15 - ldap/servers/slapd/back-ldbm/back-ldbm.h | 3 ldap/servers/slapd/back-ldbm/backentry.c | 1 ldap/servers/slapd/back-ldbm/cache.c | 103 ++++-- ldap/servers/slapd/back-ldbm/ldbm_add.c | 341 ++++++++++++++-------- ldap/servers/slapd/back-ldbm/ldbm_delete.c | 343 +++++++++++++++-------- ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c | 12 ldap/servers/slapd/back-ldbm/ldbm_modify.c | 54 ++- ldap/servers/slapd/back-ldbm/ldbm_modrdn.c | 320 +++++++++++++++------ ldap/servers/slapd/back-ldbm/ldbm_search.c | 2 ldap/servers/slapd/back-ldbm/ldif2ldbm.c | 2 ldap/servers/slapd/back-ldbm/parents.c | 31 +- ldap/servers/slapd/back-ldbm/proto-back-ldbm.h | 5 ldap/servers/slapd/entry.c | 38 ++ ldap/servers/slapd/rdn.c | 108 ++++++- ldap/servers/slapd/slapi-plugin.h | 45 +++ ldap/servers/slapd/slapi-private.h | 7 ldap/servers/slapd/util.c | 67 ++++ 23 files changed, 1199 insertions(+), 486 deletions(-) New commits: commit 160cb3f686e433c01532d28770b2977ec957e73e Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Fri May 9 09:59:41 2014 -0700 Ticket #47750 - Creating a glue fails if one above level is a conflict or missing; Ticket #47696 - Large Searches Hang - Possibly entryrdn related Bug Descriptions: There were multiple issues to break the parent-child relationship when a resurrect occurred. When resurrecting a tombstone, there is a possibility that an entry having the same name was already added. In the case, the code consdered the entry is already there, which confused the numsubordinates and tombstone-num- subordinates count. If it is detected, it creates a conflict glue entry instead of a pure glue entry. The 2 numsubordinates counts were also not correctly updated in the resurrection case. When the entryrdn was introduced, Slapi_RDN was held in the entry, which was not correctly updated in the resurrect case. Fix Descriptions: [replication/urp.c] urp_fixup_rename_entry sets the parent uniqueid if it is a RESURRECT mode (called from tombstone_to_glue) as being done in urp_fixup_add_entry, which is necessary to retrieve the correct parent in ldbm_back_modrdn. [replication/urp_glue.c] In create_glue_entry, . there was a possibility to pass a tombstone DN (nsuniqueid=...,<DN>) is used to create a glue entry. If found, the part is removed before passing it to create a glue entry. . there is a possibility to create a glue entry fails due to the parent entry not found. This occurs when the parent entry has turned to be a conflict entry. In the case, instead of retrying the glue creation, it logs an error and returns. [replication/urp_tombstone.c] In tombstone_to_glue, . turning a tombstone to an ordinary glue entry could fail with the entry already exists. (Note: ldbm_back_add guarantees that the found entry has a different nsuniqueid from this tombstone has.) This situation occurs an entry having the same dn could have been added/replicated by this moment. And the entry cannot be used for this tombstone to resurrect since the subordinates/tombstonesubordinates relationships get corrupted. To maintain the relationship properly, the tombstone is resurrected as a conflict, glue entry using urp_fixup_rename_entry. If it still returns LDAP_ALREADY_EXISTS, it is assured the expected glue entry exists since the same nsuniqueid is shared. [slapd/back-ldbm/ldbm_add.c] In ldbm_back_add, . in is_resurect_operation case, a SLAPI_ADD_EXISTING_DN_ENTRY entry and a SLAPI_ADD_EXISTING_UNIQUEID_ENTRY entry are identical, it returns SUCCESS instead of LDAP_ALREADY_EXISTS. . in is_resurect_operation case, the DN is modified from the tombstone DN to the ordinary DN. The DN is stored in e_sdn as well as in e_srdn in the Slapi_Entry. The e_srdn part was not updated. This patch fixes it and e_sdn and e_srdn are in sync. . parent_update_on_childchange adjusts the count of numsubordinates and tombstonenumsubordinates. It only took PARENTUPDATE_ADD|_DELETE operations. This patch adds PARENTUPDATE_RESURECT operation to increment numsubordinates and decrement tombstonenumsubordinates at the same time. . in is_resurect_operation case, a tombstone to be replaced with a glue entry needs to be deleted from the entryrdn index, which was missing. [slapd/back-ldbm/ldbm_delete.c] . Check the target DN if it starts with "nsuniqueid=...,", if it is, log an error and returns. [slapd/back-ldbm/ldbm_modrdn.c] . If it is is_resurect_operation case, it allows to rename a tombstone entry. This is used to support to change a tombstone entry to a conflict glue entry in tombstone_to_glue. . ldbm_back_modify checks if the renamed entry has children or not and if there are, they are updated to point to the renamed entry. The check function slapi_entry_has_children was just checking the numsubordinates attribute type. If a positive value is returned, it took care all the children including the tombstones, but if the renamed entry had tombstoned children only, it considered there were no children and the tombstoned children were not updated to point the renamed entry. This patch fixes it. [slapd/back-ldbm/ldbm_entryrdn.c] . Added a flag to entryrdn_rename_subtree and entryrdn_get_subordinates to treat the given DN a tombstoned DN if it is. In addition, helper APIs such as slapi_rdn_is_multivalued, slapi_rdn_is_ conflict, and slapi_is_special_rdn are added. Ported from 389-ds-base-1.2.11.
https://fedorahosted.org/389/ticket/47750
Reviewed by mreynolds(a)redhat.com (Thank you, Mark!!) diff --git a/ldap/servers/plugins/replication/repl5.h b/ldap/servers/plugins/replication/repl5.h index 674627b..a66e7a0 100644 --- a/ldap/servers/plugins/replication/repl5.h +++ b/ldap/servers/plugins/replication/repl5.h @@ -60,6 +60,7 @@ #include "llist.h" #include "repl5_ruv.h" #include "cl4.h" +#include "plstr.h" #define START_UPDATE_DELAY 2 /* 2 second */ #define REPLICA_TYPE_WINDOWS 1 diff --git a/ldap/servers/plugins/replication/urp.c b/ldap/servers/plugins/replication/urp.c index b6fe2ff..d867274 100644 --- a/ldap/servers/plugins/replication/urp.c +++ b/ldap/servers/plugins/replication/urp.c @@ -58,7 +58,6 @@ static int urp_naming_conflict_removal (Slapi_PBlock *pb, char *sessionid, CSN * static int mod_namingconflict_attr (const char *uniqueid, const Slapi_DN *entrysdn, const Slapi_DN *conflictsdn, CSN *opcsn, const char *optype); static int del_replconflict_attr (const Slapi_Entry *entry, CSN *opcsn, int opflags); static char *get_dn_plus_uniqueid(char *sessionid,const Slapi_DN *oldsdn,const char *uniqueid); -static char *get_rdn_plus_uniqueid(char *sessionid,const char *olddn,const char *uniqueid); static int is_suffix_entry (Slapi_PBlock *pb, Slapi_Entry *entry, Slapi_DN **parenddn); /* @@ -388,6 +387,9 @@ urp_modrdn_operation( Slapi_PBlock *pb ) slapi_entry_get_sdn (target_entry), "renameTombstone", opcsn); */ op_result = LDAP_NO_SUCH_OBJECT; + slapi_log_error(SLAPI_LOG_REPL, sessionid, + "urp_modrdn: target_entry %s is a tombstone; returning LDAP_NO_SUCH_OBJECT.\n", + slapi_entry_get_dn((Slapi_Entry *)target_entry)); slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); if (op_result == 0) { @@ -473,10 +475,9 @@ urp_modrdn_operation( Slapi_PBlock *pb ) mod_namingconflict_attr (op_uniqueid, target_sdn, existing_sdn, opcsn, "MODRDN"); slapi_pblock_set(pb, SLAPI_MODRDN_NEWRDN, newrdn_with_uniqueid); slapi_log_error(slapi_log_urp, sessionid, - "urp_modrdn: Naming conflict MODRDN. Rename target entry to %s\n", - newrdn_with_uniqueid ); - - rc= slapi_setbit_int(rc,SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); + "urp_modrdn: Naming conflict MODRDN. Rename target entry from %s to %s\n", + newrdn, newrdn_with_uniqueid ); + rc= slapi_setbit_int(rc, SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); PROFILE_POINT; /* ModRDN Conflict; Entry with Target DN Exists; Rename Operation Entry */ goto bailout; } @@ -628,12 +629,15 @@ urp_delete_operation( Slapi_PBlock *pb ) op_result= LDAP_NO_SUCH_OBJECT; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc = SLAPI_PLUGIN_FAILURE; /* Don't apply the Delete */ + slapi_log_error(slapi_log_urp, sessionid, + "Entry %s does not exist; returning NO_SUCH_OBJECT.\n", + slapi_entry_get_dn((Slapi_Entry *)deleteentry)); PROFILE_POINT; /* Delete Operation; Entry not exist. */ } else if(is_tombstone_entry(deleteentry)) { /* The entry is already a Tombstone, ignore this delete. */ - op_result= LDAP_SUCCESS; + op_result= LDAP_ALREADY_EXISTS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc = SLAPI_PLUGIN_NOOP; /* Don't apply the Delete */ slapi_log_error(slapi_log_urp, sessionid, @@ -811,7 +815,7 @@ urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *pa } int -urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflags) +urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags) { Slapi_PBlock *newpb; Slapi_Operation *op; @@ -839,7 +843,12 @@ urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflag opcsn = (CSN *)entry_get_dncsn (entry); slapi_pblock_get (newpb, SLAPI_OPERATION, &op); operation_set_csn (op, opcsn); - + if (parentuniqueid) + { + struct slapi_operation_parameters *op_params; + slapi_pblock_get( newpb, SLAPI_OPERATION_PARAMETERS, &op_params ); + op_params->p.p_modrdn.modrdn_newsuperior_address.uniqueid = (char*)parentuniqueid; + } slapi_modrdn_internal_pb(newpb); slapi_pblock_get(newpb, SLAPI_PLUGIN_INTOP_RESULT, &op_result); @@ -1035,16 +1044,17 @@ urp_annotate_dn (char *sessionid, const Slapi_Entry *entry, CSN *opcsn, const ch basesdn = slapi_entry_get_sdn_const (entry); basedn = slapi_entry_get_dn_const (entry); newrdn = get_rdn_plus_uniqueid ( sessionid, basedn, uniqueid ); - if(newrdn!=NULL) - { + if(newrdn) { mod_namingconflict_attr (uniqueid, basesdn, basesdn, opcsn, optype); - op_result = urp_fixup_rename_entry ( entry, newrdn, 0 ); + slapi_log_error(slapi_log_urp, sessionid, + "urp_annotate_dn: %s --> %s\n", basedn, newrdn); + op_result = urp_fixup_rename_entry ( entry, newrdn, NULL, 0 ); switch(op_result) { case LDAP_SUCCESS: slapi_log_error(slapi_log_urp, sessionid, - "Naming conflict %s. Renamed existing entry to %s\n", - optype, newrdn); + "Naming conflict %s. Renamed existing entry to %s\n", + optype, newrdn); rc = 1; break; case LDAP_NO_SUCH_OBJECT: @@ -1064,16 +1074,21 @@ urp_annotate_dn (char *sessionid, const Slapi_Entry *entry, CSN *opcsn, const ch * for a conflict!! After fix for 558293, this * state can't be reproduced anymore (5-Oct-01) */ - slapi_log_error( SLAPI_LOG_FATAL, sessionid, - "Entry %s exists in cache but not in DB\n", - basedn ); + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Entry %s exists in cache but not in DB\n", + basedn); rc = LDAP_NO_SUCH_OBJECT; break; default: - slapi_log_error( slapi_log_urp, sessionid, - "Failed to annotate %s, err=%d\n", newrdn, op_result); + slapi_log_error(slapi_log_urp, sessionid, + "Failed to annotate %s, err=%d\n", + newrdn, op_result); } slapi_ch_free ( (void**)&newrdn ); + } else { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to create conflict DN from basedn: %s and uniqueid: %s\n", + basedn, uniqueid ); } return rc; } @@ -1218,7 +1233,7 @@ urp_naming_conflict_removal ( Slapi_PBlock *pb, char *sessionid, CSN *opcsn, con * is done after DB lock was released. The backend modrdn * will acquire the DB lock if it sees this flag. */ - op_result = urp_fixup_rename_entry((const Slapi_Entry *)min_naming_conflict_entry, newrdnstr, OP_FLAG_ACTION_INVOKE_FOR_REPLOP); + op_result = urp_fixup_rename_entry((const Slapi_Entry *)min_naming_conflict_entry, newrdnstr, NULL, OP_FLAG_ACTION_INVOKE_FOR_REPLOP); if ( op_result != LDAP_SUCCESS ) { slapi_log_error (slapi_log_urp, sessionid, @@ -1251,15 +1266,19 @@ bailout: static char * get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *uniqueid) { - Slapi_RDN *rdn= slapi_rdn_new(); - char *newdn; + Slapi_RDN *rdn = slapi_rdn_new(); + char *newdn = NULL; + int rc = slapi_rdn_init_all_sdn_ext(rdn, oldsdn, 1); + if (rc) { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to convert %s to RDN\n", slapi_sdn_get_dn(oldsdn)); + goto bail; + } PR_ASSERT(uniqueid!=NULL); /* Check if the RDN already contains the Unique ID */ - slapi_rdn_set_dn(rdn, slapi_sdn_get_dn(oldsdn)); - if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid))) - { + if (slapi_rdn_is_conflict(rdn)) { /* The Unique ID is already in the RDN. * This is a highly improbable collision. * It suggests that a duplicate UUID was generated. @@ -1268,7 +1287,6 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique */ slapi_log_error(SLAPI_LOG_FATAL, sessionid, "Annotated DN %s has naming conflict\n", slapi_sdn_get_dn(oldsdn) ); - newdn= NULL; } else { @@ -1282,21 +1300,27 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique newdn = slapi_ch_smprintf("%s,%s", slapi_rdn_get_rdn(rdn), parentdn); slapi_ch_free_string(&parentdn); } +bail: slapi_rdn_free(&rdn); return newdn; } -static char * +char * get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid) { - char *newrdn; + char *newrdn = NULL; /* Check if the RDN already contains the Unique ID */ - Slapi_DN *sdn= slapi_sdn_new_dn_byval(olddn); - Slapi_RDN *rdn= slapi_rdn_new(); - slapi_sdn_get_rdn(sdn,rdn); + Slapi_DN *sdn = slapi_sdn_new_dn_byval(olddn); + Slapi_RDN *rdn = slapi_rdn_new(); + int rc = slapi_rdn_init_all_sdn_ext(rdn, sdn, 1); + if (rc) { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to convert %s to RDN\n", olddn); + goto bail; + } + PR_ASSERT(uniqueid!=NULL); - if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid))) - { + if (slapi_rdn_is_conflict(rdn)) { /* The Unique ID is already in the RDN. * This is a highly improbable collision. * It suggests that a duplicate UUID was generated. @@ -1304,14 +1328,14 @@ get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid) * require admin intercession */ slapi_log_error(SLAPI_LOG_FATAL, sessionid, - "Annotated DN %s has naming conflict\n", olddn ); - newrdn= NULL; + "Annotated RDN %s has naming conflict\n", olddn); } else { slapi_rdn_add(rdn,SLAPI_ATTR_UNIQUEID,uniqueid); - newrdn= slapi_ch_strdup(slapi_rdn_get_rdn(rdn)); + newrdn = slapi_ch_strdup(slapi_rdn_get_rdn(rdn)); } +bail: slapi_sdn_free(&sdn); slapi_rdn_free(&rdn); return newrdn; diff --git a/ldap/servers/plugins/replication/urp.h b/ldap/servers/plugins/replication/urp.h index 26d7c75..8afd7d1 100644 --- a/ldap/servers/plugins/replication/urp.h +++ b/ldap/servers/plugins/replication/urp.h @@ -55,11 +55,12 @@ int urp_delete_operation( Slapi_PBlock *pb ); int urp_post_delete_operation( Slapi_PBlock *pb ); int urp_modrdn_operation( Slapi_PBlock *pb ); int urp_post_modrdn_operation( Slapi_PBlock *pb ); +char *get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid); /* urp internal ops */ int urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *parentuniqueid, CSN *opcsn, int opflags); int urp_fixup_delete_entry (const char *uniqueid, const char *dn, CSN *opcsn, int opflags); -int urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, int opflags); +int urp_fixup_rename_entry (const Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags); int urp_fixup_modify_entry (const char *uniqueid, const Slapi_DN *sdn, CSN *opcsn, Slapi_Mods *smods, int opflags); int is_suffix_dn (Slapi_PBlock *pb, const Slapi_DN *dn, Slapi_DN **parenddn); @@ -78,6 +79,6 @@ PRBool get_glue_csn(const Slapi_Entry *entry, const CSN **gluecsn); * urp_tombstone.c */ int is_tombstone_entry(const Slapi_Entry* entry); -int tombstone_to_glue(Slapi_PBlock *pb, const char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn); +int tombstone_to_glue(Slapi_PBlock *pb, char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn); int entry_to_tombstone ( Slapi_PBlock *pb, Slapi_Entry *entry ); PRBool get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn); diff --git a/ldap/servers/plugins/replication/urp_glue.c b/ldap/servers/plugins/replication/urp_glue.c index 6eeb826..199d4de 100644 --- a/ldap/servers/plugins/replication/urp_glue.c +++ b/ldap/servers/plugins/replication/urp_glue.c @@ -153,7 +153,7 @@ static const char *glue_entry = "objectclass: top\n" "objectclass: extensibleObject\n" /* JCMREPL - To avoid schema checking. */ "objectclass: glue\n" - "nsuniqueid: %s\n" + "nsuniqueid: %s\n" /* this uniqueid is set to Slapi_Entry in slapi_str2entry. */ "%s: %s\n"; /* Add why it's been created */ static int @@ -166,15 +166,15 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha Slapi_DN *sdn = NULL; Slapi_RDN *newrdn = slapi_rdn_new_rdn(rdn); char *estr, *rdnstr, *rdntype, *rdnval, *rdnpair; - sdn = slapi_sdn_new_dn_byval(slapi_sdn_get_ndn(superiordn)); + sdn = slapi_sdn_new_ndn_byval(slapi_sdn_get_ndn(superiordn)); slapi_sdn_add_rdn(sdn,rdn); /* must take care of multi-valued rdn: split rdn into different lines introducing * '\n' between each type/value pair. - */ + */ alloc_len = RDNBUFSIZE; rdnstr = slapi_ch_malloc(alloc_len); - rdnpair = rdnstr; + rdnpair = rdnstr; *rdnpair = '\0'; /* so that strlen(rdnstr) may return 0 the first time it's called */ while ((rdnval_index = slapi_rdn_get_next(newrdn, rdnval_index, &rdntype, &rdnval)) != -1) { rdntype_len = strlen(rdntype); @@ -188,7 +188,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha } slapi_ldif_put_type_and_value_with_options(&rdnpair, rdntype, rdnval, rdnval_len, LDIF_OPT_NOWRAP); *rdnpair = '\0'; - } + } estr= slapi_ch_smprintf(glue_entry, slapi_sdn_get_ndn(sdn), rdnstr, uniqueid, ATTR_NSDS5_REPLCONFLICT, reason); slapi_ch_free((void**)&rdnstr); @@ -196,9 +196,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha slapi_ch_free((void**)&newrdn); e = slapi_str2entry( estr, 0 ); PR_ASSERT(e!=NULL); - if ( e!=NULL ) - { - slapi_entry_set_uniqueid (e, slapi_ch_strdup(uniqueid)); + if (e) { op_result = urp_fixup_add_entry (e, NULL, NULL, opcsn, 0); } slapi_ch_free_string(&estr); @@ -232,7 +230,7 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char slapi_pblock_get( pb, SLAPI_BACKEND, &backend ); slapi_sdn_get_backend_parent ( dn, superiordn, backend ); - slapi_sdn_get_rdn ( dn, rdn ); + slapi_rdn_set_dn_ext(rdn, slapi_sdn_get_dn(dn), 1/* skip nsuniqeid=..., in dn */); while(!done) { @@ -246,16 +244,30 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char done= 1; break; case LDAP_ALREADY_EXISTS: + { + struct slapi_operation_parameters *op_params; + /* This is okay. While creating a glue, a real entry was added. */ slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name, "%s: Skipped creating glue entry %s uniqueid=%s reason Entry Already Exists\n", sessionid, dnstr, uniqueid); + op_result = LDAP_SUCCESS; + /* If we could not create a glue having the nsuniqueid, + * we have to abandon it. */ + slapi_pblock_get(pb, SLAPI_OPERATION_PARAMETERS, &op_params); + slapi_ch_free_string(&op_params->p.p_add.parentuniqueid); done= 1; break; + } case LDAP_NO_SUCH_OBJECT: /* The parent is missing */ { /* JCMREPL - Create the parent ... recursion?... but what's the uniqueid? */ - PR_ASSERT(0); /* JCMREPL */ + slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, + "%s: Can't created glue entry %s uniqueid=%s, error %d; " + "Possibly, parent entry is a conflict entry.\n", + sessionid, dnstr, uniqueid, op_result); + done= 1; + break; } default: slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name, diff --git a/ldap/servers/plugins/replication/urp_tombstone.c b/ldap/servers/plugins/replication/urp_tombstone.c index a61f1e5..5a37ca8 100644 --- a/ldap/servers/plugins/replication/urp_tombstone.c +++ b/ldap/servers/plugins/replication/urp_tombstone.c @@ -84,7 +84,7 @@ get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn) static int tombstone_to_glue_resolve_parent ( Slapi_PBlock *pb, - const char *sessionid, + char *sessionid, const Slapi_DN *parentdn, const char *parentuniqueid, CSN *opcsn) @@ -144,7 +144,7 @@ tombstone_to_glue_resolve_parent ( int tombstone_to_glue ( Slapi_PBlock *pb, - const char *sessionid, + char *sessionid, Slapi_Entry *tombstoneentry, const Slapi_DN *tombstonedn, const char *reason, @@ -156,6 +156,7 @@ tombstone_to_glue ( Slapi_Entry *addingentry; const char *addingdn; int op_result; + int rdn_is_conflict = 0; /* JCMREPL * Nothing logged to the 5.0 Change Log @@ -170,18 +171,19 @@ tombstone_to_glue ( * which won't help us identify the correct backend to search. */ is_suffix_dn_ext (pb, tombstonedn, &parentdn, 1 /* is_tombstone */); - parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, - SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */ + parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */ tombstone_to_glue_resolve_parent (pb, sessionid, parentdn, parentuniqueid, opcsn); - slapi_sdn_free(&parentdn); - /* Submit an Add operation to turn the tombstone entry into glue. */ + /* Submit an Add operation to turn the tombstone entry into glue. */ /* * The tombstone is stored with an invalid DN, we must fix this. */ addingentry = slapi_entry_dup(tombstoneentry); addingdn = slapi_sdn_get_dn(tombstonedn); slapi_entry_set_sdn(addingentry, tombstonedn); + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry), slapi_entry_get_dn_const(addingentry)); + rdn_is_conflict = slapi_rdn_is_conflict(slapi_entry_get_srdn(addingentry)); if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) { @@ -189,18 +191,58 @@ tombstone_to_glue ( slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason); } tombstoneuniqueid= slapi_entry_get_uniqueid(tombstoneentry); - op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, parentuniqueid, opcsn, OP_FLAG_RESURECT_ENTRY); + /* + * addingentry and parentuniqueid are consumed in urp_fixup_add_entry, + * regardless of the result. + * Note: addingentry is not really consumed in ldbm_back_add. + * tombstoneentry from DB/entry cache is duplicated and turned to be a glue. + * This addingentry is freed in op_shared_add. + */ + op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, slapi_ch_strdup(parentuniqueid), opcsn, OP_FLAG_RESURECT_ENTRY); + if ((LDAP_ALREADY_EXISTS == op_result) && !rdn_is_conflict) { + /* conflict -- there's already the same named entry added. + * But this to-be-glued entry needs to be added since this is a parent of child entries... + * So, rename this tombstone parententry a conflict, glue entry. + * Instead of "fixup_add", we have to "fixup_rename"... + * */ + char *conflictrdn = get_rdn_plus_uniqueid(sessionid, addingdn, tombstoneuniqueid); + if (conflictrdn) { + addingentry = slapi_entry_dup(tombstoneentry); + if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) { + /* Add the reason of turning it to glue - The backend code will use it*/ + slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason); + } + slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, + "%s: Can't resurrect tombstone to glue reason '%s'. " + "Try with conflict dn %s, error=%d\n", + sessionid, reason, addingdn, op_result); + op_result = urp_fixup_rename_entry(addingentry, (const char *)conflictrdn, parentuniqueid, + OP_FLAG_RESURECT_ENTRY|OP_FLAG_TOMBSTONE_ENTRY); + slapi_ch_free_string(&conflictrdn); + slapi_entry_free(addingentry); + addingentry = NULL; + } + } + slapi_ch_free_string(&parentuniqueid); if (op_result == LDAP_SUCCESS) { slapi_log_error (slapi_log_urp, repl_plugin_name, "%s: Resurrected tombstone %s to glue reason '%s'\n", sessionid, addingdn, reason); } + else if (LDAP_ALREADY_EXISTS == op_result) + { + slapi_log_error(slapi_log_urp, repl_plugin_name, + "%s: No need to turn tombstone %s to glue; it was already resurrected.\n", + sessionid, addingdn); + op_result = LDAP_SUCCESS; + } else { slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, "%s: Can't resurrect tombstone %s to glue reason '%s', error=%d\n", sessionid, addingdn, reason, op_result); } + slapi_sdn_free(&parentdn); return op_result; } diff --git a/ldap/servers/slapd/add.c b/ldap/servers/slapd/add.c index e98f197..fadc56b 100644 --- a/ldap/servers/slapd/add.c +++ b/ldap/servers/slapd/add.c @@ -753,10 +753,17 @@ static void op_shared_add (Slapi_PBlock *pb) slapi_pblock_get(pb, SLAPI_ENTRY_POST_OP, &pse); do_ps_service(pse, NULL, LDAP_CHANGETYPE_ADD, 0); - - /* If be_add succeeded, then e is consumed. We - * set e to NULL to prevent freeing it ourselves. */ - e = NULL; + /* + * If be_add succeeded, then e is consumed except the resurect case. + * If it is resurect, the corresponding tombstone entry is resurected + * and put into the cache. + * Otherwise, we set e to NULL to prevent freeing it ourselves. + */ + if (operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY) && save_e) { + e = save_e; + } else { + e = NULL; + } } else { diff --git a/ldap/servers/slapd/back-ldbm/back-ldbm.h b/ldap/servers/slapd/back-ldbm/back-ldbm.h index 0834a1c..bd84e0f 100644 --- a/ldap/servers/slapd/back-ldbm/back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/back-ldbm.h @@ -902,7 +902,8 @@ typedef struct _back_search_result_set /* operation for parent_update_on_childchange */ #define PARENTUPDATE_ADD 0x1 #define PARENTUPDATE_DEL 0x2 -#define PARENTUPDATE_MASK (PARENTUPDATE_ADD|PARENTUPDATE_DEL) +#define PARENTUPDATE_RESURECT 0x4 +#define PARENTUPDATE_MASK (PARENTUPDATE_ADD|PARENTUPDATE_DEL|PARENTUPDATE_RESURECT) #define PARENTUPDATE_CREATE_TOMBSTONE 0x10 #define PARENTUPDATE_DELETE_TOMBSTONE 0x20 diff --git a/ldap/servers/slapd/back-ldbm/backentry.c b/ldap/servers/slapd/back-ldbm/backentry.c index 52468ad..114c2df 100644 --- a/ldap/servers/slapd/back-ldbm/backentry.c +++ b/ldap/servers/slapd/back-ldbm/backentry.c @@ -52,6 +52,7 @@ backentry_free( struct backentry **bep ) return; } ep = *bep; + PR_ASSERT(ep->ep_state & (ENTRY_STATE_DELETED|ENTRY_STATE_NOTINCACHE)); if ( ep->ep_entry != NULL ) { slapi_entry_free( ep->ep_entry ); } diff --git a/ldap/servers/slapd/back-ldbm/cache.c b/ldap/servers/slapd/back-ldbm/cache.c index 7f3f025..0309fcf 100644 --- a/ldap/servers/slapd/back-ldbm/cache.c +++ b/ldap/servers/slapd/back-ldbm/cache.c @@ -890,7 +890,7 @@ entrycache_remove_int(struct cache *cache, struct backentry *e) } else { - LOG("remove %d from id hash failed\n", e->ep_id, 0, 0); + LOG("remove %s (%d) from id hash failed\n", ndn, e->ep_id, 0); } } #ifdef UUIDCACHE_ON @@ -984,7 +984,12 @@ int cache_replace(struct cache *cache, void *oldptr, void *newptr) static int entrycache_replace(struct cache *cache, struct backentry *olde, struct backentry *newe) { - int found; + int found = 0; + int found_in_dn = 0; + int found_in_id = 0; +#ifdef UUIDCACHE_ON + int found_in_uuid = 0; +#endif const char *oldndn; const char *newndn; #ifdef UUIDCACHE_ON @@ -992,6 +997,7 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, const char *newuuid; #endif size_t entry_size = 0; + struct backentry *alte = NULL; LOG("=> entrycache_replace (%s) -> (%s)\n", backentry_get_ndn(olde), backentry_get_ndn(newe), 0); @@ -1015,31 +1021,21 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, * cache tables, operation error */ if ( (olde->ep_state & ENTRY_STATE_NOTINCACHE) == 0 ) { - int found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn)); - int found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID)); + found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn)); + found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID)); #ifdef UUIDCACHE_ON - int found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid)); + found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid)); #endif found = found_in_dn && found_in_id; #ifdef UUIDCACHE_ON found = found && found_in_uuid; #endif - if (!found) { -#ifdef UUIDCACHE_ON - LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n", - found_in_dn, found_in_id, found_in_uuid); -#else - LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d]\n", - found_in_dn, found_in_id, 0); -#endif - PR_Unlock(cache->c_mutex); - return 1; - } } - if (! entry_same_dn(newe, (void *)oldndn) && - (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) { - /* if we're doing a modrdn, the new entry can be in the dn table - * already, so we need to remove that too. + /* If fails, we have to make sure the both entires are removed from the cache, + * otherwise, we have no idea what's left in the cache or not... */ + if (!entry_same_dn(newe, (void *)oldndn) && (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) { + /* if we're doing a modrdn or turning an entry to a tombstone, + * the new entry can be in the dn table already, so we need to remove that too. */ if (remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn))) { @@ -1048,20 +1044,44 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, LOG("entry cache replace remove entry size %lu\n", newe->ep_size, 0, 0); } } - + /* + * The old entry could have been "removed" between the add and this replace, + * The entry is NOT freed, but NOT in the dn hash. + * which could happen since the entry is not necessarily locked. + * This is ok. + */ + olde->ep_state = ENTRY_STATE_DELETED; /* olde is removed from the cache, so set DELETED here. */ + if (!found) { + if (olde->ep_state & ENTRY_STATE_DELETED) { + LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]; but the entry is alreay deleted.\n", + oldndn, found_in_dn, found_in_id); + } else { +#ifdef UUIDCACHE_ON + LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n", + found_in_dn, found_in_id, found_in_uuid); +#else + LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]\n", + oldndn, found_in_dn, found_in_id); +#endif + PR_Unlock(cache->c_mutex); + return 1; + } + } /* now, add the new entry to the hashtables */ /* (probably don't need such extensive error handling, once this has been * tested enough that we believe it works.) */ - if (!add_hash(cache->c_dntable, (void *)newndn, strlen(newndn), newe, NULL)) { - LOG("entry cache replace: can't add dn\n", 0, 0, 0); + if (!add_hash(cache->c_dntable, (void *)newndn, strlen(newndn), newe, (void **)&alte)) { + LOG("entry cache replace (%s): can't add to dn table (returned %s)\n", + newndn, alte?slapi_entry_get_dn(alte->ep_entry):"none", 0); PR_Unlock(cache->c_mutex); return 1; } - if (!add_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID), newe, NULL)) { - LOG("entry cache replace: can't add id\n", 0, 0, 0); + if (!add_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID), newe, (void **)&alte)) { + LOG("entry cache replace (%s): can't add to id table (returned %s)\n", + newndn, alte?slapi_entry_get_dn(alte->ep_entry):"none", 0); if(remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)) == 0){ - LOG("entry cache replace: failed to remove dn table\n", 0, 0, 0); + LOG("entry cache replace: failed to remove dn table\n", 0, 0, 0); } PR_Unlock(cache->c_mutex); return 1; @@ -1071,10 +1091,10 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, newe, NULL)) { LOG("entry cache replace: can't add uuid\n", 0, 0, 0); if(remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)) == 0){ - LOG("entry cache replace: failed to remove dn table(uuid cache)\n", 0, 0, 0); + LOG("entry cache replace: failed to remove dn table(uuid cache)\n", 0, 0, 0); } if(remove_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID)) == 0){ - LOG("entry cache replace: failed to remove id table(uuid cache)\n", 0, 0, 0); + LOG("entry cache replace: failed to remove id table(uuid cache)\n", 0, 0, 0); } PR_Unlock(cache->c_mutex); return 1; @@ -1088,7 +1108,6 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, } else if (newe->ep_size < olde->ep_size) { slapi_counter_subtract(cache->c_cursize, olde->ep_size - newe->ep_size); } - olde->ep_state = ENTRY_STATE_DELETED; newe->ep_state = 0; PR_Unlock(cache->c_mutex); LOG("<= entrycache_replace OK, cache size now %lu cache count now %ld\n", @@ -1282,7 +1301,7 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, PR_Lock(cache->c_mutex); if (! add_hash(cache->c_dntable, (void *)ndn, strlen(ndn), e, (void **)&my_alt)) { - LOG("entry \"%s\" already in dn cache\n", backentry_get_ndn(e), 0, 0); + LOG("entry \"%s\" already in dn cache\n", ndn, 0, 0); /* add_hash filled in 'my_alt' if necessary */ if (my_alt == e) { @@ -1322,14 +1341,15 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, { if (my_alt->ep_state & ENTRY_STATE_CREATING) { - LOG("the entry is reserved\n", 0, 0, 0); + LOG("the entry %s is reserved (ep_state: 0x%x, state: 0x%x\n", ndn, e->ep_state, state); e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); return -1; } else if (state != 0) { - LOG("the entry already exists. cannot reserve it.\n", 0, 0, 0); + LOG("the entry %s already exists. cannot reserve it. (ep_state: 0x%x, state: 0x%x\n", + ndn, e->ep_state, state); e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); return -1; @@ -1341,6 +1361,11 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, if ((*alt)->ep_refcnt == 0) lru_delete(cache, (void *)*alt); (*alt)->ep_refcnt++; + LOG("the entry %s already exists. returning existing entry %s (state: 0x%x)\n", + ndn, backentry_get_ndn(my_alt), state); + } else { + LOG("the entry %s already exists. Not returning existing entry %s (state: 0x%x)\n", + ndn, backentry_get_ndn(my_alt), state); } PR_Unlock(cache->c_mutex); return 1; @@ -1355,7 +1380,7 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, if (state == 0) { /* neither of these should fail, or something is very wrong. */ if (! add_hash(cache->c_idtable, &(e->ep_id), sizeof(ID), e, NULL)) { - LOG("entry %s already in id cache!\n", backentry_get_ndn(e), 0, 0); + LOG("entry %s already in id cache!\n", ndn, 0, 0); if (already_in) { /* there's a bug in the implementatin of 'modify' and 'modrdn' * that i'm working around here. basically they do a @@ -1377,10 +1402,12 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, return 0; } if(remove_hash(cache->c_dntable, (void *)ndn, strlen(ndn)) == 0){ - LOG("entrycache_add_int: failed to remove dn table\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove %s from dn table\n", 0, 0, 0); } e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); + LOG("entrycache_add_int: failed to add %s to cache (ep_state: %x, already_in: %d)\n", + ndn, e->ep_state, already_in); return -1; } #ifdef UUIDCACHE_ON @@ -1391,10 +1418,10 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, LOG("entry %s already in uuid cache!\n", backentry_get_ndn(e), 0, 0); if(remove_hash(cache->c_dntable, (void *)ndn, strlen(ndn)) == 0){ - LOG("entrycache_add_int: failed to remove dn table(uuid cache)\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove dn table(uuid cache)\n", 0, 0, 0); } if(remove_hash(cache->c_idtable, &(e->ep_id), sizeof(ID)) == 0){ - LOG("entrycache_add_int: failed to remove id table(uuid cache)\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove id table(uuid cache)\n", 0, 0, 0); } e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); @@ -1499,8 +1526,8 @@ int cache_lock_entry(struct cache *cache, struct backentry *e) if (!e->ep_mutexp) { LOG("<= cache_lock_entry (DELETED)\n", 0, 0, 0); LDAPDebug1Arg(LDAP_DEBUG_ANY, - "cache_lock_entry: failed to create a lock for %s\n", - backentry_get_ndn(e)); + "cache_lock_entry: failed to create a lock for %s\n", + backentry_get_ndn(e)); LOG("<= cache_lock_entry (FAILED)\n", 0, 0, 0); return 1; } diff --git a/ldap/servers/slapd/back-ldbm/ldbm_add.c b/ldap/servers/slapd/back-ldbm/ldbm_add.c index fbc1064..582e06a 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_add.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_add.c @@ -103,9 +103,9 @@ ldbm_back_add( Slapi_PBlock *pb ) int parent_found = 0; int ruv_c_init = 0; int rc = 0; - int addingentry_id_assigned= 0; - int addingentry_in_cache= 0; - int tombstone_in_cache= 0; + int addingentry_id_assigned = 0; + int addingentry_in_cache = 0; + int tombstone_in_cache = 0; Slapi_DN *sdn = NULL; Slapi_DN parentsdn; Slapi_Operation *operation; @@ -119,6 +119,7 @@ ldbm_back_add( Slapi_PBlock *pb ) entry_address addr = {0}; int not_an_error = 0; int parent_switched = 0; + int noabort = 1; slapi_pblock_get( pb, SLAPI_PLUGIN_PRIVATE, &li ); slapi_pblock_get( pb, SLAPI_ADD_ENTRY, &e ); @@ -196,7 +197,6 @@ ldbm_back_add( Slapi_PBlock *pb ) goto error_return; } - /* * Originally (in the U-M LDAP 3.3 code), there was a comment near this * code about a race condition. The race was that a 2nd entry could be @@ -216,11 +216,16 @@ ldbm_back_add( Slapi_PBlock *pb ) if (txn.back_txn_txn && (txn.back_txn_txn != parent_txn)) { /* Don't release SERIAL LOCK */ dblayer_txn_abort_ext(li, &txn, PR_FALSE); + noabort = 1; slapi_pblock_set(pb, SLAPI_TXN, parent_txn); if (addingentry_in_cache) { /* addingentry is in cache. Remove it once. */ - CACHE_REMOVE(&inst->inst_cache, addingentry); + retval = CACHE_REMOVE(&inst->inst_cache, addingentry); + if (retval) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "ldbm_add: cache_remove %s failed.\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); + } CACHE_RETURN(&inst->inst_cache, &addingentry); } else { backentry_free(&addingentry); @@ -233,11 +238,11 @@ ldbm_back_add( Slapi_PBlock *pb ) } if (addingentry_in_cache) { /* Adding the resetted addingentry to the cache. */ - if (cache_add_tentative(&inst->inst_cache, - addingentry, NULL) != 0) { - LDAPDebug0Args(LDAP_DEBUG_CACHE, - "cache_add_tentative concurrency detected\n"); + if (cache_add_tentative(&inst->inst_cache, addingentry, NULL) != 0) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code = LDAP_ALREADY_EXISTS; + addingentry_in_cache = 0; goto error_return; } } @@ -262,6 +267,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if (0 == retry_count) { /* First time, hold SERIAL LOCK */ retval = dblayer_txn_begin(be, parent_txn, &txn); + noabort = 0; if (!is_tombstone_operation) { rc= slapi_setbit_int(rc,SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); @@ -308,6 +314,10 @@ ldbm_back_add( Slapi_PBlock *pb ) } } + /* + * If the parent is conflict, slapi_sdn_get_backend_parent does not support it. + * That is, adding children to a conflict entry is not allowed. + */ slapi_sdn_get_backend_parent(sdn, &parentsdn, pb->pb_backend); /* Check if an entry with the intended DN already exists. */ done_with_pblock_entry(pb,SLAPI_ADD_EXISTING_DN_ENTRY); /* Could be through this multiple times */ @@ -315,28 +325,25 @@ ldbm_back_add( Slapi_PBlock *pb ) addr.udn = NULL; addr.uniqueid = NULL; ldap_result_code= get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_EXISTING_DN_ENTRY, !is_replicated_operation); - if(ldap_result_code==LDAP_OPERATIONS_ERROR || - ldap_result_code==LDAP_INVALID_DN_SYNTAX) - { + if(ldap_result_code==LDAP_OPERATIONS_ERROR || ldap_result_code==LDAP_INVALID_DN_SYNTAX) { goto error_return; } } /* if we can find the parent by dn or uniqueid, and the operation has requested the parent then get it */ - if(have_parent_address(&parentsdn, operation->o_params.p.p_add.parentuniqueid) && - slapi_isbitset_int(rc,SLAPI_RTN_BIT_FETCH_PARENT_ENTRY)) - { + if (have_parent_address(&parentsdn, operation->o_params.p.p_add.parentuniqueid) && + slapi_isbitset_int(rc,SLAPI_RTN_BIT_FETCH_PARENT_ENTRY)) { done_with_pblock_entry(pb,SLAPI_ADD_PARENT_ENTRY); /* Could be through this multiple times */ addr.sdn = &parentsdn; addr.udn = NULL; addr.uniqueid = operation->o_params.p.p_add.parentuniqueid; - ldap_result_code= get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_PARENT_ENTRY, !is_replicated_operation); - /* need to set parentsdn or parentuniqueid if either is not set? */ + ldap_result_code = get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_PARENT_ENTRY, !is_replicated_operation); } /* Call the Backend Pre Add plugins */ + ldap_result_code = LDAP_SUCCESS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); - rc= plugin_call_plugins(pb, SLAPI_PLUGIN_BE_PRE_ADD_FN); + rc = plugin_call_plugins(pb, SLAPI_PLUGIN_BE_PRE_ADD_FN); if (rc < 0) { int opreturn = 0; if (SLAPI_PLUGIN_NOOP == rc) { @@ -382,6 +389,7 @@ ldbm_back_add( Slapi_PBlock *pb ) ldap_result_code= LDAP_OPERATIONS_ERROR; goto error_return; } + noabort = 0; /* stash the transaction for plugins */ slapi_pblock_set(pb, SLAPI_TXN, txn.back_txn_txn); @@ -404,10 +412,14 @@ ldbm_back_add( Slapi_PBlock *pb ) operation->o_params.p.p_add.parentuniqueid = slapi_ch_strdup(slapi_entry_get_uniqueid(parententry->ep_entry)); } - if (slapi_sdn_isempty(&parentsdn)) { + if (slapi_sdn_isempty(&parentsdn) || + slapi_sdn_compare(&parentsdn, slapi_entry_get_sdn(parententry->ep_entry))) { /* Set the parentsdn now */ slapi_sdn_set_dn_byval(&parentsdn, slapi_entry_get_dn_const(parententry->ep_entry)); } + } else { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "find_entry2modify_only returned NULL parententry pdn: %s, uniqueid: %s\n", + slapi_sdn_get_dn(&parentsdn), slapi_sdn_get_dn(&parentsdn), addr.uniqueid?addr.uniqueid:"none"); } modify_init(&parent_modify_c,parententry); } @@ -415,11 +427,36 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Check if the entry we have been asked to add already exists */ { Slapi_Entry *entry; - slapi_pblock_get( pb, SLAPI_ADD_EXISTING_DN_ENTRY, &entry); - if ( entry != NULL ) - { - /* The entry already exists */ - ldap_result_code= LDAP_ALREADY_EXISTS; + slapi_pblock_get(pb, SLAPI_ADD_EXISTING_DN_ENTRY, &entry); + if (entry) { + if (is_resurect_operation) { + Slapi_Entry *uniqentry; + slapi_pblock_get(pb, SLAPI_ADD_EXISTING_UNIQUEID_ENTRY, &uniqentry); + if (uniqentry == entry) { + /* + * adding entry having the uniqueid exists. + * No need to resurrect. + */ + ldap_result_code = LDAP_SUCCESS; + } else { + /* The entry having the DN already exists */ + if (uniqentry) { + if (PL_strcmp(slapi_entry_get_uniqueid(entry), + slapi_entry_get_uniqueid(uniqentry))) { + /* Not match; conflict. */ + ldap_result_code = LDAP_ALREADY_EXISTS; + } else { + /* Same entry; no need to resurrect. */ + ldap_result_code = LDAP_SUCCESS; + } + } else { + ldap_result_code = LDAP_ALREADY_EXISTS; + } + } + } else { + /* The entry already exists */ + ldap_result_code = LDAP_ALREADY_EXISTS; + } goto error_return; } else @@ -430,7 +467,7 @@ ldbm_back_add( Slapi_PBlock *pb ) * entry we did match has a referral we should return * instead. we do this only if managedsait is not on. */ - if ( !managedsait && !is_tombstone_operation ) + if (!managedsait && !is_tombstone_operation && !is_resurect_operation) { int err= 0; Slapi_DN ancestorsdn; @@ -440,9 +477,7 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_sdn_done(&ancestorsdn); if ( ancestorentry != NULL ) { - int sentreferral = - check_entry_for_referral(pb, ancestorentry->ep_entry, - backentry_get_ndn(ancestorentry), "ldbm_back_add"); + int sentreferral= check_entry_for_referral(pb, ancestorentry->ep_entry, backentry_get_ndn(ancestorentry), "ldbm_back_add"); CACHE_RETURN( &inst->inst_cache, &ancestorentry ); if(sentreferral) { @@ -455,25 +490,25 @@ ldbm_back_add( Slapi_PBlock *pb ) } /* no need to check the schema as this is a replication add */ - if(!is_replicated_operation){ + if (!is_replicated_operation) { if ((operation_is_flag_set(operation,OP_FLAG_ACTION_SCHEMA_CHECK)) && (slapi_entry_schema_check(pb, e) != 0)) - { - LDAPDebug(LDAP_DEBUG_TRACE, "entry failed schema check\n", 0, 0, 0); - ldap_result_code = LDAP_OBJECT_CLASS_VIOLATION; - slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); - goto error_return; - } + { + LDAPDebug(LDAP_DEBUG_TRACE, "entry failed schema check\n", 0, 0, 0); + ldap_result_code = LDAP_OBJECT_CLASS_VIOLATION; + slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + goto error_return; + } - /* Check attribute syntax */ - if (slapi_entry_syntax_check(pb, e, 0) != 0) - { - LDAPDebug(LDAP_DEBUG_TRACE, "entry failed syntax check\n", 0, 0, 0); - ldap_result_code = LDAP_INVALID_SYNTAX; - slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); - goto error_return; + /* Check attribute syntax */ + if (slapi_entry_syntax_check(pb, e, 0) != 0) + { + LDAPDebug(LDAP_DEBUG_TRACE, "entry failed syntax check\n", 0, 0, 0); + ldap_result_code = LDAP_INVALID_SYNTAX; + slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + goto error_return; + } } - } opcsn = operation_get_csn (operation); if(is_resurect_operation) @@ -490,7 +525,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if ( tombstoneentry==NULL ) { ldap_result_code= -1; - goto error_return; /* error result sent by find_entry2modify() */ + goto error_return; /* error result sent by find_entry2modify() */ } tombstone_in_cache = 1; @@ -509,10 +544,14 @@ ldbm_back_add( Slapi_PBlock *pb ) */ if (NULL == sdn) { LDAPDebug0Args(LDAP_DEBUG_ANY, "ldbm_back_add: Null target dn\n"); + ldap_result_code = LDAP_OPERATIONS_ERROR; goto error_return; } dn = slapi_sdn_get_dn(sdn); slapi_entry_set_sdn(addingentry->ep_entry, sdn); /* The DN is passed into the entry. */ + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry->ep_entry), + slapi_entry_get_dn_const(addingentry->ep_entry)); /* LPREPL: the DN is normalized...Somehow who should get a not normalized one */ addingentry->ep_id = slapi_entry_attr_get_ulong(addingentry->ep_entry,"entryid"); slapi_entry_attr_delete(addingentry->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); @@ -676,10 +715,8 @@ ldbm_back_add( Slapi_PBlock *pb ) Slapi_DN ancestorsdn; struct backentry *ancestorentry; - LDAPDebug( LDAP_DEBUG_TRACE, - "parent does not exist, pdn = %s\n", - slapi_sdn_get_dn(&parentsdn), 0, 0 ); - + LDAPDebug1Arg(LDAP_DEBUG_BACKLDBM, "ldbm_add: Parent \"%s\" does not exist. " + "It might be a conflict entry.\n", slapi_sdn_get_dn(&parentsdn)); slapi_sdn_init(&ancestorsdn); ancestorentry = dn2ancestor(be, &parentsdn, &ancestorsdn, &txn, &err ); CACHE_RETURN( &inst->inst_cache, &ancestorentry ); @@ -690,22 +727,37 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_sdn_done(&ancestorsdn); goto error_return; } - ldap_result_code = plugin_call_acl_plugin (pb, e, NULL, NULL, SLAPI_ACL_ADD, - ACLPLUGIN_ACCESS_DEFAULT, &errbuf ); + ldap_result_code = plugin_call_acl_plugin(pb, e, NULL, NULL, SLAPI_ACL_ADD, + ACLPLUGIN_ACCESS_DEFAULT, &errbuf); if ( ldap_result_code != LDAP_SUCCESS ) { - LDAPDebug( LDAP_DEBUG_TRACE, "no access to parent\n", 0, 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "no access to parent, pdn = %s\n", + slapi_sdn_get_dn(&parentsdn)); ldap_result_message= errbuf; goto error_return; } pid = parententry->ep_id; + + /* We may need to adjust the DN since parent could be a resrected conflict entry... */ + if (!slapi_sdn_isparent(slapi_entry_get_sdn_const(parententry->ep_entry), + slapi_entry_get_sdn_const(addingentry->ep_entry))) { + Slapi_DN adjustedsdn = {0}; + char *adjusteddn = slapi_ch_smprintf("%s,%s", + slapi_entry_get_rdn_const(addingentry->ep_entry), + slapi_entry_get_dn_const(parententry->ep_entry)); + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "ldbm_add: adjusting dn: %s --> %s\n", + slapi_entry_get_dn(addingentry->ep_entry), adjusteddn); + slapi_sdn_set_normdn_passin(&adjustedsdn, adjusteddn); + slapi_entry_set_sdn(addingentry->ep_entry, &adjustedsdn); + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry->ep_entry), adjusteddn); + slapi_sdn_done(&adjustedsdn); + } } else { /* no parent */ - if ( !isroot && !is_replicated_operation) - { - LDAPDebug( LDAP_DEBUG_TRACE, "no parent & not root\n", - 0, 0, 0 ); + if (!isroot && !is_replicated_operation) { + LDAPDebug0Args(LDAP_DEBUG_TRACE, "no parent & not root\n"); ldap_result_code= LDAP_INSUFFICIENT_ACCESS; goto error_return; } @@ -734,11 +786,12 @@ ldbm_back_add( Slapi_PBlock *pb ) * operational attributes to ensure that the cache is sized correctly. */ if ( cache_add_tentative( &inst->inst_cache, addingentry, NULL )!= 0 ) { - LDAPDebug( LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected\n", 0, 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code= LDAP_ALREADY_EXISTS; goto error_return; } - addingentry_in_cache= 1; + addingentry_in_cache = 1; /* * Before we add the entry, find out if the syntax of the aci @@ -746,7 +799,8 @@ ldbm_back_add( Slapi_PBlock *pb ) * the entry if the syntax is incorrect. */ if ( plugin_call_acl_verify_syntax (pb, addingentry->ep_entry, &errbuf) != 0 ) { - LDAPDebug( LDAP_DEBUG_TRACE, "ACL syntax error\n", 0,0,0); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "ACL syntax error: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code= LDAP_INVALID_SYNTAX; ldap_result_message= errbuf; goto error_return; @@ -755,12 +809,14 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Having decided that we're really going to do the operation, let's modify the in-memory state of the parent to reflect the new child (update subordinate count specifically */ - if (NULL != parententry) - { + if (parententry) { retval = parent_update_on_childchange(&parent_modify_c, - PARENTUPDATE_ADD, NULL); + is_resurect_operation?PARENTUPDATE_RESURECT:PARENTUPDATE_ADD, + NULL); /* The modify context now contains info needed later */ - if (0 != retval) { + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "parent_update_on_childchange: %s, rc=%d\n", + slapi_entry_get_dn_const(addingentry->ep_entry), retval); ldap_result_code= LDAP_OPERATIONS_ERROR; goto error_return; } @@ -783,14 +839,12 @@ ldbm_back_add( Slapi_PBlock *pb ) not_an_error = 1; rc = retval = LDAP_SUCCESS; } - LDAPDebug1Arg( LDAP_DEBUG_TRACE, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " - "returned error code %d\n", retval ); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " + "returned error code %d\n", retval ); if (!ldap_result_code) { slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } if (!ldap_result_code) { - LDAPDebug0Args( LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " - "returned error but did not setSLAPI_RESULT_CODE \n" ); ldap_result_code = LDAP_OPERATIONS_ERROR; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); } @@ -799,6 +853,8 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_set(pb, SLAPI_PLUGIN_OPRETURN, ldap_result_code ? &ldap_result_code : &retval); } slapi_pblock_get(pb, SLAPI_PB_RESULT_TEXT, &ldap_result_message); + LDAPDebug1Arg(LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin failed: %d", + ldap_result_code ? ldap_result_code : retval); goto error_return; } @@ -809,9 +865,10 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_TRACE, "id2entry_add failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + if (retval) { + LDAPDebug(LDAP_DEBUG_TRACE, "id2entry_add(%s) failed, err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; @@ -819,19 +876,18 @@ ldbm_back_add( Slapi_PBlock *pb ) } goto error_return; } - if(is_resurect_operation) - { + if (is_resurect_operation) { retval = index_addordel_string(be,SLAPI_ATTR_OBJECTCLASS,SLAPI_ATTR_VALUE_TOMBSTONE,addingentry->ep_id,BE_INDEX_DEL|BE_INDEX_EQUALITY,&txn); if (DB_LOCK_DEADLOCK == retval) { LDAPDebug( LDAP_DEBUG_ARGS, "add 2 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 1 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string TOMBSTONE (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -845,10 +901,10 @@ ldbm_back_add( Slapi_PBlock *pb ) continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 2 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string UNIQUEID (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -866,17 +922,29 @@ ldbm_back_add( Slapi_PBlock *pb ) continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 3 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string ENTRYDN (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; } goto error_return; } - } + /* Need to delete the entryrdn index of the resurrected tombstone... */ + if (entryrdn_get_switch()) { /* subtree-rename: on */ + if (tombstoneentry) { + retval = entryrdn_index_entry(be, tombstoneentry, BE_INDEX_DEL, &txn); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, "Resurrecting %s: failed to remove entryrdn index, err=%d %s\n", + slapi_entry_get_dn_const(tombstoneentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + goto error_return; + } + } + } + } if (is_tombstone_operation) { retval = index_addordel_entry( be, addingentry, BE_INDEX_ADD | BE_INDEX_TOMBSTONE, &txn ); @@ -891,9 +959,8 @@ ldbm_back_add( Slapi_PBlock *pb ) /* retry txn */ continue; } - if (retval != 0) { - LDAPDebug2Args(LDAP_DEBUG_ANY, - "add: attempt to index %lu failed (rc=%d)\n", + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_ANY, "add: attempt to index %lu failed; rc=%d\n", (u_long)addingentry->ep_id, retval); ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { @@ -911,11 +978,10 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 1 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "modify_update_all: %s (%lu) failed; rc=%d\n", + slapi_entry_get_dn(addingentry->ep_entry), (u_long)addingentry->ep_id, retval); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -930,17 +996,16 @@ ldbm_back_add( Slapi_PBlock *pb ) { retval= vlv_update_all_indexes(&txn, be, pb, NULL, addingentry); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, - "add DEADLOCK vlv_update_index\n", 0, 0, 0 ); + LDAPDebug(LDAP_DEBUG_ARGS, + "add DEADLOCK vlv_update_index\n", 0, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, - "vlv_update_index failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_TRACE, + "vlv_update_index failed, err=%d %s\n", + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -986,7 +1051,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if (retry_count == RETRY_TIMES) { /* Failed */ LDAPDebug( LDAP_DEBUG_ANY, "Retry count exceeded in add\n", 0, 0, 0 ); - ldap_result_code= LDAP_BUSY; + ldap_result_code= LDAP_BUSY; goto error_return; } @@ -997,38 +1062,53 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_set( pb, SLAPI_ENTRY_PRE_OP, NULL ); slapi_pblock_set( pb, SLAPI_ENTRY_POST_OP, slapi_entry_dup( addingentry->ep_entry )); - if(is_resurect_operation) - { + if (is_resurect_operation) { /* * We can now switch the tombstone entry with the real entry. */ - if (cache_replace( &inst->inst_cache, tombstoneentry, addingentry ) != 0 ) - { + retval = cache_replace(&inst->inst_cache, tombstoneentry, addingentry); + if (retval) { /* This happens if the dn of addingentry already exists */ - cache_unlock_entry( &inst->inst_cache, tombstoneentry ); ADD_SET_ERROR(ldap_result_code, LDAP_ALREADY_EXISTS, retry_count); + LDAPDebug2Args(LDAP_DEBUG_CACHE, "ldap_add: cache_replace concurrency detected: %s (rc: %d)\n", + slapi_entry_get_dn_const(addingentry->ep_entry), retval); + retval = -1; goto error_return; } + if (addingentry_in_cache) { /* decrease the refcnt added by tentative */ + CACHE_RETURN( &inst->inst_cache, &addingentry ); + } + addingentry_in_cache = 1; /* reset it to make it sure... */ /* * The tombstone was locked down in the cache... we can * get rid of the entry in the cache now. + * We cannot expect tombstoneentry exists from now on. */ - cache_unlock_entry( &inst->inst_cache, tombstoneentry ); - CACHE_RETURN( &inst->inst_cache, &tombstoneentry ); - tombstone_in_cache = 0; /* deleted */ + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, delete the tombstone dn from the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, + tombstoneentry->ep_id); + if (bdn) { /* in the dncache, remove it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + cache_unlock_entry(&inst->inst_cache, tombstoneentry); + CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + tombstone_in_cache = 0; } if (parent_found) { /* switch the parent entry copy into play */ - modify_switch_entries( &parent_modify_c,be); + modify_switch_entries(&parent_modify_c,be); parent_switched = 1; } if (ruv_c_init) { if (modify_switch_entries(&ruv_c, be) != 0 ) { ldap_result_code= LDAP_OPERATIONS_ERROR; - LDAPDebug( LDAP_DEBUG_ANY, - "ldbm_back_add: modify_switch_entries failed\n", 0, 0, 0); + LDAPDebug0Args(LDAP_DEBUG_ANY, + "ldbm_back_add: modify_switch_entries failed\n"); goto error_return; } } @@ -1042,8 +1122,6 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } if (!ldap_result_code) { - LDAPDebug0Args( LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_POST_ADD_FN plugin " - "returned error but did not set SLAPI_RESULT_CODE\n" ); ldap_result_code = LDAP_OPERATIONS_ERROR; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); } @@ -1068,6 +1146,7 @@ ldbm_back_add( Slapi_PBlock *pb ) } goto error_return; } + noabort = 1; rc= 0; goto common_return; @@ -1077,23 +1156,22 @@ error_return: { next_id_return( be, addingentry->ep_id ); } - if ( NULL != addingentry ) + if ( addingentry ) { if ( addingentry_in_cache ) { if (inst) { CACHE_REMOVE(&inst->inst_cache, addingentry); } - addingentry_in_cache = 0; } - backentry_clear_entry(addingentry); /* e is released in the frontend */ - backentry_free( &addingentry ); /* release the backend wrapper, here */ - } - if(tombstone_in_cache && inst) - { - CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + else + { + if (!is_resurect_operation) { /* if resurect, tombstoneentry is dupped. */ + backentry_clear_entry(addingentry); /* e is released in the frontend */ + } + backentry_free( &addingentry ); /* release the backend wrapper, here */ + } } - if (rc == DB_RUNRECOVERY) { dblayer_remember_disk_filled(li); ldbm_nasty("Add",80,rc); @@ -1111,7 +1189,7 @@ error_return: } diskfull_return: if (disk_full) { - rc= return_on_disk_full(li); + rc = return_on_disk_full(li); } else { /* It is safer not to abort when the transaction is not started. */ if (txn.back_txn_txn && (txn.back_txn_txn != parent_txn)) { @@ -1145,7 +1223,9 @@ diskfull_return: } /* Release SERIAL LOCK */ - dblayer_txn_abort(be, &txn); /* abort crashes in case disk full */ + if (!noabort) { + dblayer_txn_abort(be, &txn); /* abort crashes in case disk full */ + } /* txn is no longer valid - reset the txn pointer to the parent */ slapi_pblock_set(pb, SLAPI_TXN, parent_txn); } @@ -1156,8 +1236,12 @@ diskfull_return: common_return: if (inst) { + if(tombstone_in_cache && tombstoneentry) { + cache_unlock_entry(&inst->inst_cache, tombstoneentry); + CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + } if (addingentry_in_cache && addingentry) { - if (entryrdn_get_switch()) { /* subtree-rename: on */ + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ /* since adding the entry to the entry cache was successful, * let's add the dn to dncache, if not yet done. */ struct backdn *bdn = dncache_find_id(&inst->inst_dncache, @@ -1178,8 +1262,9 @@ common_return: } } } - if (is_remove_from_cache) + if (is_remove_from_cache) { CACHE_REMOVE(&inst->inst_cache, addingentry); + } CACHE_RETURN( &inst->inst_cache, &addingentry ); } if (inst->inst_ref_count) { diff --git a/ldap/servers/slapd/back-ldbm/ldbm_delete.c b/ldap/servers/slapd/back-ldbm/ldbm_delete.c index ea9473e..915ea08 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_delete.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_delete.c @@ -90,6 +90,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) /* replication, we must create a new tombstone entry */ int tombstone_in_cache = 0; int e_in_cache = 0; + int remove_e_from_cache = 0; entry_address *addr; int addordel_flags = 0; /* passed to index_addordel */ char *entryusn_str = NULL; @@ -220,6 +221,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (tombstone_in_cache) { CACHE_REMOVE(&inst->inst_cache, tombstone); CACHE_RETURN(&inst->inst_cache, &tombstone); + tombstone = NULL; tombstone_in_cache = 0; } else { backentry_free(&tombstone); @@ -230,13 +232,6 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } } - - /* reset original entry in cache */ - if (!e_in_cache) { - CACHE_ADD(&inst->inst_cache, e, NULL); - e_in_cache = 1; - } - if (ruv_c_init) { /* reset the ruv txn stuff */ modify_term(&ruv_c, be); @@ -272,17 +267,28 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (0 == retry_count) { /* just once */ /* find and lock the entry we are about to modify */ + /* + * A corner case: + * If a conflict occurred in a MMR topology, a replicated delete + * op from another master could target a conflict entry; while the + * corresponding entry on this server could have been already + * deleted. That is, the entry 'e' found with "addr" is a tomb- + * stone. If it is the case, we need to back off. + */ if ( (e = find_entry2modify( pb, be, addr, &txn )) == NULL ) { ldap_result_code= LDAP_NO_SUCH_OBJECT; retval = -1; + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "ldbm_back_delete: Deleting entry is already deleted.\n"); goto error_return; /* error result sent by find_entry2modify() */ } e_in_cache = 1; /* e is cached */ - if ( slapi_entry_has_children( e->ep_entry ) ) - { + retval = slapi_entry_has_children(e->ep_entry); + if (retval) { ldap_result_code= LDAP_NOT_ALLOWED_ON_NONLEAF; + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_delete", "Deleting entry %s has %d children.\n", + slapi_entry_get_dn(e->ep_entry), retval); retval = -1; goto error_return; } @@ -318,7 +324,17 @@ ldbm_back_delete( Slapi_PBlock *pb ) * Plugin indicated some kind of failure, * or that this Operation became a No-Op. */ + slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); if (!ldap_result_code) { + if (LDAP_ALREADY_EXISTS == ldap_result_code) { + /* + * The target entry is already a tombstone. + * We need to treat this as a success, + * but we need to remove the entry e from the entry cache. + */ + remove_e_from_cache = 1; + ldap_result_code = LDAP_SUCCESS; + } slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } /* restore original entry so the front-end delete code can free it */ @@ -330,8 +346,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } /* the flag could be set in a preop plugin (e.g., USN) */ - delete_tombstone_entry = operation_is_flag_set(operation, - OP_FLAG_TOMBSTONE_ENTRY); + delete_tombstone_entry = operation_is_flag_set(operation, OP_FLAG_TOMBSTONE_ENTRY); } /* call the transaction pre delete plugins just after the @@ -493,13 +508,15 @@ ldbm_back_delete( Slapi_PBlock *pb ) } if (NULL == parent) { entry_address parent_addr; - + if (is_tombstone_entry) { + parent_addr.uniqueid = slapi_entry_attr_get_charptr(e->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + } else { + parent_addr.uniqueid = NULL; + } parent_addr.sdn = &parentsdn; - parent_addr.uniqueid = NULL; - parent = find_entry2modify_only_ext(pb, be, &parent_addr, - TOMBSTONE_INCLUDED, &txn); + parent = find_entry2modify_only_ext(pb, be, &parent_addr, TOMBSTONE_INCLUDED, &txn); } - if (NULL != parent) { + if (parent) { int isglue; size_t haschildren = 0; int op = PARENTUPDATE_DEL; @@ -516,11 +533,11 @@ ldbm_back_delete( Slapi_PBlock *pb ) } else if (delete_tombstone_entry) { op |= PARENTUPDATE_DELETE_TOMBSTONE; } - retval = parent_update_on_childchange(&parent_modify_c, - op, &haschildren); + retval = parent_update_on_childchange(&parent_modify_c, op, &haschildren); /* The modify context now contains info needed later */ if (0 != retval) { ldap_result_code= LDAP_OPERATIONS_ERROR; + slapi_sdn_done(&parentsdn); retval = -1; goto error_return; } @@ -550,9 +567,17 @@ ldbm_back_delete( Slapi_PBlock *pb ) */ const char *childuniqueid= slapi_entry_get_uniqueid(e->ep_entry); const char *parentuniqueid= NULL; - char *tombstone_dn = compute_entry_tombstone_dn(slapi_entry_get_dn(e->ep_entry), - childuniqueid); + char *edn = slapi_entry_get_dn(e->ep_entry); + char *tombstone_dn; Slapi_Value *tomb_value; + + if (slapi_is_special_rdn(edn, RDN_IS_TOMBSTONE)) { + LDAPDebug1Arg(LDAP_DEBUG_ANY, "Turning a tombstone into a tombstone! \"%s\"\n", edn); + ldap_result_code= LDAP_OPERATIONS_ERROR; + retval = -1; + goto error_return; + } + tombstone_dn = compute_entry_tombstone_dn(edn, childuniqueid); slapi_sdn_set_ndn_byval(&nscpEntrySDN, slapi_sdn_get_ndn(slapi_entry_get_sdn(e->ep_entry))); @@ -674,41 +699,85 @@ ldbm_back_delete( Slapi_PBlock *pb ) * tentatively for now, then cache_add again when the original * entry is removed from the cache. */ - if (cache_add_tentative( &inst->inst_cache, tombstone, NULL) == 0) { + retval = cache_add_tentative(&inst->inst_cache, tombstone, NULL); + if (0 == retval) { tombstone_in_cache = 1; - } else if (!(tombstone->ep_state & ENTRY_STATE_NOTINCACHE)) { - LDAPDebug1Arg(LDAP_DEBUG_CACHE, - "id2entry_add tombstone (%s) is in cache\n", - slapi_entry_get_dn(tombstone->ep_entry)); - tombstone_in_cache = 1; + } else { + LDAPDebug2Args(LDAP_DEBUG_ANY, + "tombstone entry %s failed to add to the cache: %d\n", + slapi_entry_get_dn(tombstone->ep_entry), retval); + tombstone_in_cache = 0; + if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; + DEL_SET_ERROR(ldap_result_code, + LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; } retval = id2entry_add( be, tombstone, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 1 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete 1 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); /* Abort and re-try */ continue; } - if (0 != retval) { + if (retval) { LDAPDebug( LDAP_DEBUG_ANY, "id2entry_add failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } - if (cache_replace( &inst->inst_cache, e, tombstone ) != 0 ) { - LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "ldbm_back_delete cache_replace failed\n"); - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); - retval= -1; - goto error_return; + if (tombstone_in_cache) { + retval = cache_replace(&inst->inst_cache, e, tombstone); + if (retval) { + LDAPDebug(LDAP_DEBUG_CACHE, "ldbm_back_delete: cache_replace failed (%d): %s --> %s\n", + retval, slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn(tombstone->ep_entry)); + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } } else { - e_in_cache = 0; /* e un-cached */ + struct backentry *imposter = NULL; + retval = CACHE_ADD(&inst->inst_cache, tombstone, &imposter); + if (retval > 0) { + if (imposter) { + /* + * The same tombstone entry (different Slapi_Entry) is already + * generated and set to cache. Back off. + */ + CACHE_RETURN(&inst->inst_cache, &imposter); + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: same DN tombstone in cache: %s\n", + slapi_entry_get_dn(tombstone->ep_entry)); + } else { + /* + * The same tombstone entry (same Slapi_Entry) is being created. + * Something is wrong. We should clean it up from the cache, + * and back off. + */ + tombstone_in_cache = 1; + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: same tombstone in cache: %s\n", + slapi_entry_get_dn(tombstone->ep_entry)); + } + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } else if (retval < 0) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: Add %s failed.\n", + slapi_entry_get_dn(tombstone->ep_entry)); + /* Complete add error */ + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } } - /* tombstone was already added to the cache via cache_add_tentative (to reserve its spot in the cache) - and/or id2entry_add - so it already had one refcount - cache_replace adds another refcount - - drop the extra ref added by cache_replace */ - CACHE_RETURN( &inst->inst_cache, &tombstone ); + if (tombstone_in_cache) { + /* tombstone was already added to the cache via cache_add_tentative (to reserve its spot in the cache) + and/or id2entry_add - so it already had one refcount - cache_replace adds another refcount - + drop the extra ref added by cache_replace */ + CACHE_RETURN( &inst->inst_cache, &tombstone ); + } + tombstone_in_cache = 1; } else { @@ -716,13 +785,12 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = id2entry_delete( be, e, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 2 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "delete 2 DEADLOCK\n"); /* Retry txn */ continue; } - if (retval != 0 ) { - if (retval == DB_RUNRECOVERY || - LDBM_OS_ERR_IS_DISKFULL(retval)) { + if (retval) { + if (retval == DB_RUNRECOVERY || LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; } DEL_SET_ERROR(ldap_result_code, @@ -739,12 +807,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = index_addordel_entry( be, e, addordel_flags, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 1 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "delete 1 DEADLOCK\n"); /* Retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_TRACE, "index_del_entry failed\n", 0, 0, 0 ); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, "index_del_entry(%s, 0x%x) failed (%d)\n", + slapi_entry_get_dn(e->ep_entry), addordel_flags, retval); DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } @@ -759,14 +828,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) SLAPI_ATTR_VALUE_TOMBSTONE, tombstone->ep_id,BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_VALUE_TOMBSTONE, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_VALUE_TOMBSTONE, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -779,14 +848,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_entry_get_uniqueid(tombstone->ep_entry), tombstone->ep_id,BE_INDEX_ADD,&txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_UNIQUEID, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_UNIQUEID, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -799,14 +868,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_sdn_get_ndn(&nscpEntrySDN), tombstone->ep_id, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_NSCP_ENTRYDN, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_NSCP_ENTRYDN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -823,14 +892,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) entryusn_str, tombstone->ep_id, BE_INDEX_ADD, &txn); slapi_ch_free_string(&entryusn_str); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_ENTRYUSN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_ENTRYUSN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -854,13 +923,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) svals, NULL, e->ep_id, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (updating " LDBM_PARENTID_STR ") DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if ( retval ) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", LDBM_PARENTID_STR, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -870,15 +939,16 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } } +#if 0 /* The entryrdn element is already deleted in the index_addordel_entry */ retval = entryrdn_index_entry(be, e, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (deleting entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, + LDAPDebug2Args( LDAP_DEBUG_ANY, "delete (deleting entryrdn) failed, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -887,21 +957,21 @@ ldbm_back_delete( Slapi_PBlock *pb ) LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } +#endif retval = entryrdn_index_entry(be, tombstone, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "adding (adding tombstone entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, - "adding (adding tombstone entryrdn) failed, err=%d %s\n", - retval, - (msg = dblayer_strerror( retval )) ? msg : "" ); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, + "adding (adding tombstone entryrdn %s) failed, err=%d %s\n", + slapi_entry_get_dn(tombstone->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : "" ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } } @@ -918,14 +988,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) SLAPI_ATTR_VALUE_TOMBSTONE, e->ep_id, BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_VALUE_TOMBSTONE, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_VALUE_TOMBSTONE, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -938,14 +1008,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_entry_get_uniqueid(e->ep_entry), e->ep_id, BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_UNIQUEID, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_UNIQUEID, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -962,14 +1032,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) nscpedn, e->ep_id, BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); slapi_ch_free((void **)&nscpedn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_NSCP_ENTRYDN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_NSCP_ENTRYDN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -988,14 +1058,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); slapi_ch_free_string(&entryusn_str); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_ENTRYUSN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_ENTRYUSN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -1009,13 +1079,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) { retval = entryrdn_index_entry(be, e, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (deleting entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, + LDAPDebug2Args( LDAP_DEBUG_ANY, "delete (deleting entryrdn) failed, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -1032,12 +1102,12 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = modify_update_all(be,pb,&parent_modify_c,&txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "del 4 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "del 4 DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "delete 3 BAD, err=%d %s\n", + LDAPDebug( LDAP_DEBUG_ANY, "delete 3 BAD, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; DEL_SET_ERROR(ldap_result_code, @@ -1055,7 +1125,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete DEADLOCK vlv_update_index\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete DEADLOCK vlv_update_index\n", 0, 0, 0 ); /* Retry txn */ continue; } @@ -1148,14 +1218,24 @@ ldbm_back_delete( Slapi_PBlock *pb ) /* delete from cache and clean up */ if (e) { + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, delete the tombstone dn from the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); + if (bdn) { /* in the dncache, remove it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } if (e_in_cache) { CACHE_REMOVE(&inst->inst_cache, e); + cache_unlock_entry(&inst->inst_cache, e); + CACHE_RETURN(&inst->inst_cache, &e); + } else { + cache_unlock_entry(&inst->inst_cache, e); } - cache_unlock_entry(&inst->inst_cache, e); - CACHE_RETURN(&inst->inst_cache, &e); e = NULL; } - + if (ruv_c_init) { if (modify_switch_entries(&ruv_c, be) != 0 ) { ldap_result_code= LDAP_OPERATIONS_ERROR; @@ -1176,16 +1256,36 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto common_return; error_return: - if (inst && tombstone_in_cache) - { - CACHE_REMOVE( &inst->inst_cache, tombstone ); - CACHE_RETURN( &inst->inst_cache, &tombstone ); - tombstone = NULL; - tombstone_in_cache = 0; + if (tombstone) { + if (entryrdn_get_switch()) { /* subtree-rename: on */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, tombstone->ep_id); + if (bdn) { /* already in the dncache. Delete it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + if (tombstone_in_cache) { /* successfully replaced */ + CACHE_REMOVE( &inst->inst_cache, tombstone ); + CACHE_RETURN( &inst->inst_cache, &tombstone ); + tombstone = NULL; + tombstone_in_cache = 0; + } else { + backentry_free( &tombstone ); + } } - else - { - backentry_free( &tombstone ); + + /* Need to return to cache after post op plugins are called */ + if (e) { + if (e_in_cache) { + if (remove_e_from_cache) { + /* The entry is already transformed to a tombstone. */ + CACHE_REMOVE( &inst->inst_cache, e ); + } + cache_unlock_entry( &inst->inst_cache, e ); + CACHE_RETURN( &inst->inst_cache, &e ); + } else { + cache_unlock_entry( &inst->inst_cache, e ); + } } if (retval == DB_RUNRECOVERY) { @@ -1246,15 +1346,33 @@ common_return: for the post op plugins */ slapi_pblock_set( pb, SLAPI_DELETE_BEPREOP_ENTRY, orig_entry ); } - if (inst && tombstone_in_cache) - { - CACHE_RETURN( &inst->inst_cache, &tombstone ); - tombstone = NULL; - tombstone_in_cache = 0; - } - else - { - backentry_free( &tombstone ); + if (inst && tombstone) { + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, + tombstone->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *tombstonesdn = slapi_sdn_dup(slapi_entry_get_sdn(tombstone->ep_entry)); + if (tombstonesdn) { + bdn = backdn_init(tombstonesdn, tombstone->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_delete", + "set %s to dn cache\n", slapi_sdn_get_dn(tombstonesdn)); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + } + } + if (tombstone_in_cache) { /* successfully replaced */ + CACHE_RETURN( &inst->inst_cache, &tombstone ); + tombstone = NULL; + tombstone_in_cache = 0; + } else { + backentry_free( &tombstone ); + } } /* result code could be used in the bepost plugin functions. */ @@ -1267,17 +1385,6 @@ common_return: plugin_call_plugins (pb, SLAPI_PLUGIN_BE_POST_DELETE_FN); } - /* Need to return to cache after post op plugins are called */ - if (inst) { - if (retval && e) { /* error case */ - cache_unlock_entry( &inst->inst_cache, e ); - CACHE_RETURN( &inst->inst_cache, &e ); - } - if (inst->inst_ref_count) { - slapi_counter_decrement(inst->inst_ref_count); - } - } - if (ruv_c_init) { modify_term(&ruv_c, be); } @@ -1291,7 +1398,7 @@ diskfull_return: } slapi_send_ldap_result( pb, ldap_result_code, NULL, ldap_result_message, 0, NULL ); } - modify_term(&parent_modify_c,be); + modify_term(&parent_modify_c, be); if (rc == 0 && opcsn && !is_fixup_operation && !delete_tombstone_entry) { /* URP Naming Collision diff --git a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c index 2550bc9..8ac4fa5 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c @@ -396,7 +396,7 @@ entryrdn_index_read_ext(backend *be, rc = slapi_rdn_init_all_sdn(&srdn, sdn); if (rc < 0) { - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, + slapi_log_error(SLAPI_LOG_BACKLDBM, ENTRYRDN_TAG, "entryrdn_index_read: Param error: Failed to convert " "%s to Slapi_RDN\n", slapi_sdn_get_dn(sdn)); rc = LDAP_INVALID_DN_SYNTAX; @@ -510,7 +510,8 @@ entryrdn_rename_subtree(backend *be, Slapi_RDN *newsrdn, /* new rdn */ const Slapi_DN *newsupsdn, /* new superior dn */ ID id, - back_txn *txn) + back_txn *txn, + int flags) { int rc = -1; struct attrinfo *ai = NULL; @@ -552,7 +553,7 @@ entryrdn_rename_subtree(backend *be, goto bail; } - rc = slapi_rdn_init_all_sdn(&oldsrdn, oldsdn); + rc = slapi_rdn_init_all_sdn_ext(&oldsrdn, oldsdn, flags); if (rc < 0) { slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, "entryrdn_rename_subtree: Failed to convert olddn " @@ -978,7 +979,8 @@ entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, - back_txn *txn) + back_txn *txn, + int flags) { int rc = -1; struct attrinfo *ai = NULL; @@ -1010,7 +1012,7 @@ entryrdn_get_subordinates(backend *be, goto bail; } - rc = slapi_rdn_init_all_sdn(&srdn, sdn); + rc = slapi_rdn_init_all_sdn_ext(&srdn, sdn, flags); if (rc) { if (rc < 0) { slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modify.c b/ldap/servers/slapd/back-ldbm/ldbm_modify.c index e618c6b..d2dc983 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modify.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modify.c @@ -116,8 +116,14 @@ int modify_switch_entries(modify_context *mc,backend *be) ldbm_instance *inst = (ldbm_instance *) be->be_instance_info; int ret = 0; if (mc->old_entry!=NULL && mc->new_entry!=NULL) { - ret = cache_replace(&(inst->inst_cache), mc->old_entry, mc->new_entry); - if (ret == 0) mc->new_entry_in_cache = 1; + ret = cache_replace(&(inst->inst_cache), mc->old_entry, mc->new_entry); + if (ret == 0) { + mc->new_entry_in_cache = 1; + } else { + LDAPDebug(LDAP_DEBUG_CACHE, "modify_switch_entries: replacing %s with %s failed (%d)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), + slapi_entry_get_dn(mc->new_entry->ep_entry), ret); + } } return ret; } @@ -154,6 +160,10 @@ modify_unswitch_entries(modify_context *mc,backend *be) CACHE_RETURN( &(inst->inst_cache), &(mc->old_entry) ); mc->new_entry_in_cache = 1; mc->old_entry = NULL; + } else { + LDAPDebug(LDAP_DEBUG_CACHE, "modify_unswitch_entries: replacing %s with %s failed (%d)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), + slapi_entry_get_dn(mc->new_entry->ep_entry), ret); } } @@ -795,6 +805,12 @@ ldbm_back_modify( Slapi_PBlock *pb ) MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } + /* e uncached */ + /* we must return both e (which has been deleted) and new entry ec */ + /* cache_replace removes e from the caches */ + cache_unlock_entry( &inst->inst_cache, e ); + CACHE_RETURN( &inst->inst_cache, &e ); + /* lock new entry in cache to prevent usage until we are complete */ cache_lock_entry( &inst->inst_cache, ec ); ec_in_cache = 1; @@ -805,10 +821,6 @@ ldbm_back_modify( Slapi_PBlock *pb ) /* invalidate virtual cache */ ec->ep_entry->e_virtual_watermark = 0; - /* we must return both e (which has been deleted) and new entry ec */ - /* cache_replace removes e from the caches */ - cache_unlock_entry( &inst->inst_cache, e ); - CACHE_RETURN( &inst->inst_cache, &e ); /* * LP Fix of crash when the commit will fail: * If the commit fail, the common error path will @@ -911,28 +923,28 @@ error_return: CACHE_REMOVE( &inst->inst_cache, ec ); /* if ec was in cache, e was not - add back e */ if (e) { - CACHE_ADD( &inst->inst_cache, e, NULL ); - cache_lock_entry( &inst->inst_cache, e ); + if (CACHE_ADD( &inst->inst_cache, e, NULL )) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "ldbm_modify: CACHE_ADD %s failed\n", + slapi_entry_get_dn(e->ep_entry)); + } } } common_return: slapi_mods_done(&smods); - if (inst && ec_in_cache) - { - cache_unlock_entry( &inst->inst_cache, ec); - CACHE_RETURN( &inst->inst_cache, &ec ); - } - else - { - backentry_free(&ec); - } - if (inst) { - if (e) { - cache_unlock_entry( &inst->inst_cache, e); - CACHE_RETURN( &inst->inst_cache, &e); + if (ec_in_cache) { + cache_unlock_entry( &inst->inst_cache, ec); + CACHE_RETURN( &inst->inst_cache, &ec ); + } else { + backentry_free(&ec); + /* if ec was not in cache, cache_replace was not done. + * i.e., e was not unlocked. */ + if (e) { + cache_unlock_entry( &inst->inst_cache, e); + CACHE_RETURN( &inst->inst_cache, &e); + } } if (inst->inst_ref_count) { slapi_counter_decrement(inst->inst_ref_count); diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c index f528b07..b1b6cc9 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c @@ -45,10 +45,10 @@ #include "back-ldbm.h" -static const char *moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn); +static const char *moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn, int is_tombstone); static void moddn_unlock_and_return_entry(backend *be,struct backentry **targetentry); static int moddn_newrdn_mods(Slapi_PBlock *pb, const char *olddn, struct backentry *ec, Slapi_Mods *smods_wsi, int is_repl_op); -static IDList *moddn_get_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, struct backentry *parententry, Slapi_DN *parentdn, struct backentry ***child_entries, struct backdn ***child_dns); +static IDList *moddn_get_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, struct backentry *parententry, Slapi_DN *parentdn, struct backentry ***child_entries, struct backdn ***child_dns, int is_resurect_operation); static int moddn_rename_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, IDList *children, Slapi_DN *dn_parentdn, Slapi_DN *dn_newsuperiordn, struct backentry *child_entries[]); static int modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbminfo *li, struct backentry *e, struct backentry **ec, Slapi_Mods *smods1, Slapi_Mods *smods2, Slapi_Mods *smods3, int *e_in_cache, int *ec_in_cache); static void mods_remove_nsuniqueid(Slapi_Mods *smods); @@ -110,6 +110,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) Slapi_Operation *operation; int is_replicated_operation= 0; int is_fixup_operation = 0; + int is_resurect_operation = 0; + int is_tombstone = 0; entry_address new_addr; entry_address *old_addr; entry_address oldparent_addr; @@ -137,6 +139,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_IS_REPLICATED_OPERATION, &is_replicated_operation ); is_ruv = operation_is_flag_set(operation, OP_FLAG_REPL_RUV); is_fixup_operation = operation_is_flag_set(operation, OP_FLAG_REPL_FIXUP); + is_resurect_operation = operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY); + is_tombstone = operation_is_flag_set(operation,OP_FLAG_TOMBSTONE_ENTRY); /* tombstone_to_glue on parent entry*/ if (NULL == sdn) { slapi_send_ldap_result( pb, LDAP_INVALID_DN_SYNTAX, NULL, @@ -175,7 +179,13 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, &dn_newsuperiordn ); slapi_sdn_init_dn_byref(&dn_newrdn, newrdn); /* slapi_sdn_init_normdn_byref(&dn_newsuperiordn, newsuperiordn); */ - slapi_sdn_get_parent(sdn, &dn_parentdn); + if (is_resurect_operation) { + /* no need to free this pdn. */ + const char *pdn = slapi_dn_find_parent_ext(slapi_sdn_get_dn(sdn), is_resurect_operation); + slapi_sdn_set_dn_byval(&dn_parentdn, pdn); + } else { + slapi_sdn_get_parent(sdn, &dn_parentdn); + } } /* if old and new superior are equals, newsuperior should not be set @@ -273,7 +283,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } /* make sure the original entry is back in the cache if it was removed */ if (!e_in_cache) { - CACHE_ADD(&inst->inst_cache, e, NULL); + if (CACHE_ADD(&inst->inst_cache, e, NULL)) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_back_modrdn: CACHE_ADD %s to cache failed\n", + slapi_entry_get_dn_const(e->ep_entry)); + } e_in_cache = 1; } slapi_pblock_get( pb, SLAPI_MODRDN_EXISTING_ENTRY, &ent ); @@ -316,11 +330,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; } - if (ruv_c_init) { - /* reset the ruv txn stuff */ - modify_term(&ruv_c, be); - ruv_c_init = 0; - } + if (ruv_c_init) { + /* reset the ruv txn stuff */ + modify_term(&ruv_c, be); + ruv_c_init = 0; + } /* We're re-trying */ LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "Modrdn Retrying Transaction\n"); @@ -371,7 +385,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) * it's passed to slapi_sdn_init_normdn_byref */ slapi_pblock_get(pb, SLAPI_MODRDN_NEWRDN, &newrdn); slapi_sdn_init_dn_byref(&dn_newrdn, newrdn); - newdn = moddn_get_newdn(pb,sdn, &dn_newrdn, dn_newsuperiordn); + newdn = moddn_get_newdn(pb,sdn, &dn_newrdn, dn_newsuperiordn, is_tombstone); slapi_sdn_set_dn_passin(&dn_newdn,newdn); new_addr.sdn = &dn_newdn; new_addr.udn = NULL; @@ -492,13 +506,15 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; /* error result sent by find_entry2modify() */ } e_in_cache = 1; /* e is in the cache and locked */ - if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) ) { - ldap_result_code = LDAP_UNWILLING_TO_PERFORM; + if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) && + !is_resurect_operation) { + ldap_result_code = LDAP_UNWILLING_TO_PERFORM; ldap_result_message = "Operation not allowed on tombstone entry."; - slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", - "Attempt to rename a tombstone entry %s\n", slapi_sdn_get_dn(slapi_entry_get_sdn_const( e->ep_entry ))); - goto error_return; - } + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "Attempt to rename a tombstone entry %s\n", + slapi_sdn_get_dn(slapi_entry_get_sdn_const( e->ep_entry ))); + goto error_return; + } /* Check that an entry with the same DN doesn't already exist. */ { Slapi_Entry *entry; @@ -515,7 +531,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* Fetch and lock the parent of the entry that is moving */ oldparent_addr.sdn = &dn_parentdn; - oldparent_addr.uniqueid = NULL; + if (is_resurect_operation) { + oldparent_addr.uniqueid = operation->o_params.p.p_modrdn.modrdn_newsuperior_address.uniqueid; + } else { + oldparent_addr.uniqueid = NULL; + } parententry = find_entry2modify_only( pb, be, &oldparent_addr, &txn ); modify_init(&parent_modify_context,parententry); @@ -523,7 +543,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) if(slapi_sdn_get_ndn(dn_newsuperiordn) != NULL) { slapi_pblock_get (pb, SLAPI_MODRDN_NEWSUPERIOR_ADDRESS, &newsuperior_addr); + if (is_resurect_operation) { + newsuperior_addr->uniqueid = slapi_entry_attr_get_charptr(e->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + } newparententry = find_entry2modify_only( pb, be, newsuperior_addr, &txn ); + slapi_ch_free_string(&newsuperior_addr->uniqueid); modify_init(&newparent_modify_context,newparententry); } @@ -566,8 +590,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } slapi_pblock_set( pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, dn_newsuperiordn ); } - slapi_sdn_set_dn_passin(&dn_newdn, - moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn)); + slapi_sdn_set_dn_passin(&dn_newdn, moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn, is_tombstone)); /* Check that we're allowed to add an entry below the new superior */ if ( newparententry == NULL ) @@ -695,6 +718,57 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_rdn_done(&srdn); } + if(is_resurect_operation) { + slapi_log_error(SLAPI_LOG_REPL, "ldbm_back_modrdn", + "Resurrecting an entry %s\n", slapi_entry_get_dn(ec->ep_entry)); + slapi_entry_attr_delete(ec->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + slapi_entry_delete_string(ec->ep_entry, SLAPI_ATTR_OBJECTCLASS, SLAPI_ATTR_VALUE_TOMBSTONE); + /* Now also remove the nscpEntryDN */ + if (slapi_entry_attr_delete(ec->ep_entry, SLAPI_ATTR_NSCP_ENTRYDN) != 0){ + LDAPDebug(LDAP_DEBUG_REPL, "Resurrection of %s - Couldn't remove %s\n", + slapi_entry_get_dn(ec->ep_entry), SLAPI_ATTR_NSCP_ENTRYDN, 0); + } + + /* Set the reason (this is only a reason why modrdn is needed for resurrection) */ + slapi_entry_add_string(ec->ep_entry, "nsds5ReplConflict", "deletedEntryHasChildren"); + + /* Clear the Tombstone Flag in the entry */ + slapi_entry_clear_flag(ec->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE); + + /* make sure the objectclass + - does not contain any duplicate values + - has CSNs for the new values we added + */ + { + Slapi_Attr *sa = NULL; + Slapi_Value sv; + const struct berval *svbv = NULL; + + /* add the extensibleobject objectclass with csn if not present */ + slapi_entry_attr_find(ec->ep_entry, SLAPI_ATTR_OBJECTCLASS, &sa); + slapi_value_init_string(&sv, "extensibleobject"); + svbv = slapi_value_get_berval(&sv); + if (slapi_attr_value_find(sa, svbv)) { /* not found, so add it */ + if (opcsn) { + value_update_csn(&sv, CSN_TYPE_VALUE_UPDATED, opcsn); + } + slapi_attr_add_value(sa, &sv); + } + value_done(&sv); + + /* add the glue objectclass with csn if not present */ + slapi_value_init_string(&sv, "glue"); + svbv = slapi_value_get_berval(&sv); + if (slapi_attr_value_find(sa, svbv)) { /* not found, so add it */ + if (opcsn) { + value_update_csn(&sv, CSN_TYPE_VALUE_UPDATED, opcsn); + } + slapi_attr_add_value(sa, &sv); + } + value_done(&sv); + } + } + /* create it in the cache - prevents others from creating it */ if (( cache_add_tentative( &inst->inst_cache, ec, NULL ) != 0 ) ) { ec_in_cache = 0; /* not in cache */ @@ -704,6 +778,10 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* somebody must've created it between dn2entry() and here */ /* JCMREPL - Hmm... we can't permit this to happen...? */ ldap_result_code= LDAP_ALREADY_EXISTS; + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "cache_add_tentative failed: %s\n", slapi_entry_get_dn(ec->ep_entry)); + } goto error_return; } /* so if the old dn is the same as the new dn, the entry will not be cached @@ -751,8 +829,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } } - slapi_pblock_get( pb, SLAPI_MODIFY_MODS, &mods ); - slapi_mods_init_byref(&smods_operation_wsi,mods); + slapi_pblock_get( pb, SLAPI_MODIFY_MODS, &mods ); + slapi_mods_init_byref(&smods_operation_wsi,mods); /* * We are about to pass the last abandon test, so from now on we are @@ -842,48 +920,81 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* * Update the subordinate count of the parents to reflect the moved child. */ - if ( parententry!=NULL ) - { + if (parententry) { retval = parent_update_on_childchange(&parent_modify_context, PARENTUPDATE_DEL, NULL); /* The parent modify context now contains info needed later */ - if (0 != retval) - { + if (retval) { goto error_return; } } - if ( newparententry!=NULL ) - { + if (newparententry) { retval = parent_update_on_childchange(&newparent_modify_context, PARENTUPDATE_ADD, NULL); /* The newparent modify context now contains info needed later */ - if (0 != retval) - { + if (retval) { goto error_return; } } } + /* is_resurect_operation case, there's no new superior. Just rename. */ + if (is_resurect_operation && parententry) { + retval = parent_update_on_childchange(&parent_modify_context, PARENTUPDATE_RESURECT, NULL); + if (retval) { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "parent_update_on_childchange parent %s of %s failed, rc=%d\n", + slapi_entry_get_dn_const(parent_modify_context.old_entry->ep_entry), + slapi_entry_get_dn_const(ec->ep_entry), retval); + goto error_return; + } + } /* - * If the entry has children then we're going to have to rename them all. + * If the entry has children including tombstones, + * then we're going to have to rename them all. */ - if (slapi_entry_has_children( e->ep_entry )) - { + if (slapi_entry_has_children_ext(e->ep_entry, 1)) { /* JCM - This is where the subtree lock will appear */ if (entryrdn_get_switch()) /* subtree-rename: on */ { + if (is_resurect_operation) { +#if defined(DEBUG) + /* Get the present value of the subcount attr, or 0 if not present */ + Slapi_Attr *read_attr = NULL; + int sub_count = -1; + if (0 == slapi_entry_attr_find(parent_modify_context.old_entry->ep_entry, + "numsubordinates", &read_attr)) { + /* decode the value */ + Slapi_Value *sval; + slapi_attr_first_value(read_attr, &sval); + if (sval) { + const struct berval *bval = slapi_value_get_berval(sval); + if(bval) { + sub_count = atol(bval->bv_val); + } + } + } + LDAPDebug(LDAP_DEBUG_ANY, "parent_update_on_childchange parent %s of %s numsub=%d\n", + slapi_entry_get_dn_const(parent_modify_context.old_entry->ep_entry), + slapi_entry_get_dn_const(e->ep_entry), sub_count); +#endif + slapi_log_error(SLAPI_LOG_BACKLDBM, "ldbm_back_modrdn", + "%s has children\n", slapi_entry_get_dn(e->ep_entry)); + } children = moddn_get_children(&txn, pb, be, e, sdn, - &child_entries, &child_dns); + &child_entries, &child_dns, is_resurect_operation); } else { children = moddn_get_children(&txn, pb, be, e, sdn, - &child_entries, NULL); + &child_entries, NULL, 0); } /* JCM - Shouldn't we perform an access control check on all the children. */ /* JCMREPL - But, the replication client has total rights over its subtree, so no access check needed. */ /* JCM - A subtree move could break ACIs, static groups, and dynamic groups. */ + } else if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "%s has NO children\n", slapi_entry_get_dn(e->ep_entry)); } /* @@ -937,10 +1048,10 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (retval != 0 ) + if (retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "modrdn_rename_entry_update_indexes failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + LDAPDebug(LDAP_DEBUG_ANY, "modrdn_rename_entry_update_indexes %s --> %s failed, err=%d\n", + slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn(ec->ep_entry), retval); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; @@ -1060,14 +1171,17 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_rdn_init_sdn(&newsrdn, (const Slapi_DN *)&dn_newdn); retval = entryrdn_rename_subtree(be, (const Slapi_DN *)sdn, &newsrdn, (const Slapi_DN *)dn_newsuperiordn, - e->ep_id, &txn); + e->ep_id, &txn, is_tombstone); slapi_rdn_done(&newsrdn); if (retval != 0) { if (retval == DB_LOCK_DEADLOCK) continue; if (retval == DB_RUNRECOVERY || LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - MOD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "entryrdn_rename_subtree failed (%d); dn: %s, newsrdn: %s, dn_newsuperiordn: %s\n", + retval, slapi_sdn_get_dn(sdn), slapi_rdn_get_rdn(&newsrdn), + slapi_sdn_get_dn(dn_newsuperiordn)); goto error_return; } } @@ -1093,7 +1207,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; } - if (!is_ruv && !is_fixup_operation && !NO_RUV_UPDATE(li)) { + if (!is_ruv && !is_fixup_operation && !NO_RUV_UPDATE(li)) { ruv_c_init = ldbm_txn_ruv_modify_context( pb, &ruv_c ); if (-1 == ruv_c_init) { LDAPDebug( LDAP_DEBUG_ANY, @@ -1193,6 +1307,12 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) * "renamed" dn is generated based upon the moved subtree. */ for (i = 0; child_entries[i] != NULL; i++) { + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "Calling cache remove & return %s (refcnt: %d)\n", + slapi_entry_get_dn(child_entries[i]->ep_entry), + child_entries[i]->ep_refcnt); + } CACHE_REMOVE( &inst->inst_cache, child_entries[i] ); cache_unlock_entry( &inst->inst_cache, child_entries[i] ); CACHE_RETURN( &inst->inst_cache, &child_entries[i] ); @@ -1229,21 +1349,13 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) error_return: /* result already sent above - just free stuff */ - if ( NULL != postentry ) - { + if (postentry) { slapi_entry_free( postentry ); postentry= NULL; /* make sure caller doesn't attempt to free this */ slapi_pblock_set( pb, SLAPI_ENTRY_POST_OP, postentry ); } - if (e && entryrdn_get_switch() && inst) - { - struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); - CACHE_REMOVE(&inst->inst_dncache, bdn); - CACHE_RETURN(&inst->inst_dncache, &bdn); - } - if(children) - { + if(children) { int i = 0; if (child_entries && *child_entries && inst) { @@ -1341,9 +1453,32 @@ common_return: /* Free up the resource we don't need any more */ if (ec) { + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_REPL, "ldbm_back_modrdn", + "Resurrecting an entry %s: result: %d, %d\n", + slapi_entry_get_dn(ec->ep_entry), ldap_result_code, retval); + } + if (inst && (0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, ec->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *ecsdn = slapi_sdn_dup(slapi_entry_get_sdn(ec->ep_entry)); + if (ecsdn) { + bdn = backdn_init(ecsdn, ec->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + CACHE_RETURN(&inst->inst_dncache, &bdn); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "set %s to dn cache\n", slapi_sdn_get_dn(sdn)); + } + } + } + } /* remove the new entry from the cache if the op failed - otherwise, leave it in */ - if (ec_in_cache && inst) { + if (ec_in_cache && ec && inst) { if (retval) { CACHE_REMOVE( &inst->inst_cache, ec ); } @@ -1355,16 +1490,16 @@ common_return: ec_in_cache = 0; } - /* put e back in the cache if the modrdn failed */ - if (e) { - if (!e_in_cache && retval && inst) { - CACHE_ADD(&inst->inst_cache, e, NULL); - e_in_cache = 1; + if (inst) { + if (e && entryrdn_get_switch() && (0 == retval)) { + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); } - } - if (inst && inst->inst_ref_count) { - slapi_counter_decrement(inst->inst_ref_count); - } + if (inst->inst_ref_count) { + slapi_counter_decrement(inst->inst_ref_count); + } + } moddn_unlock_and_return_entry(be,&e); @@ -1374,11 +1509,11 @@ common_return: if (ldap_result_code!=-1) { - if (not_an_error) { - /* This is mainly used by urp. Solved conflict is not an error. - * And we don't want the supplier to halt sending the updates. */ - ldap_result_code = LDAP_SUCCESS; - } + if (not_an_error) { + /* This is mainly used by urp. Solved conflict is not an error. + * And we don't want the supplier to halt sending the updates. */ + ldap_result_code = LDAP_SUCCESS; + } slapi_send_ldap_result( pb, ldap_result_code, ldap_result_matcheddn, ldap_result_message, 0,NULL ); } @@ -1427,7 +1562,7 @@ common_return: * Work out what the new DN of the entry will be. */ static const char * -moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn) +moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn, int is_tombstone) { char *newdn; const char *newrdn= slapi_sdn_get_dn(dn_newrdn); @@ -1448,18 +1583,18 @@ moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi else { /* construct the new dn */ - char *pdn; - const char *dn= slapi_sdn_get_dn(dn_olddn); - pdn = slapi_dn_beparent( pb, dn ); - if ( pdn != NULL ) - { - newdn= slapi_dn_plus_rdn(pdn, newrdn); /* JCM - Use Slapi_RDN */ - } - else - { - newdn= slapi_ch_strdup(newrdn); + const char *dn = slapi_sdn_get_dn((const Slapi_DN *)dn_olddn); + if (slapi_dn_isbesuffix(pb, dn)) { + newdn = slapi_ch_strdup(newrdn); + } else { + /* no need to free this pdn. */ + const char *pdn = slapi_dn_find_parent_ext(dn, is_tombstone); + if (pdn) { + newdn = slapi_dn_plus_rdn(pdn, newrdn); + } else { + newdn = slapi_ch_strdup(newrdn); + } } - slapi_ch_free( (void**)&pdn ); } return newdn; } @@ -1686,6 +1821,7 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm Slapi_Operation *operation; int is_ruv = 0; /* True if the current entry is RUV */ int orig_ec_in_cache = 0; + int cache_rc = 0; slapi_pblock_get( pb, SLAPI_BACKEND, &be ); slapi_pblock_get( pb, SLAPI_OPERATION, &operation ); @@ -1697,7 +1833,12 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm * Update the ID to Entry index. * Note that id2entry_add replaces the entry, so the Entry ID stays the same. */ - retval = id2entry_add( be, *ec, ptxn ); + retval = id2entry_add_ext(be, *ec, ptxn, 1, &cache_rc); + if (cache_rc) { + LDAPDebug(LDAP_DEBUG_CACHE, + "modrdn_rename_entry_update_indexes: adding %s failed to add to the cache (rc: %d, cache_rc: %d)\n", + slapi_entry_get_dn(e->ep_entry), retval, cache_rc); + } if (DB_LOCK_DEADLOCK == retval) { /* Retry txn */ @@ -1791,11 +1932,11 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm } } if (cache_replace( &inst->inst_cache, e, *ec ) != 0 ) { - LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "modrdn_rename_entry_update_indexes cache_replace failed\n"); + LDAPDebug2Args(LDAP_DEBUG_CACHE, + "modrdn_rename_entry_update_indexes cache_replace %s -> %s failed\n", + slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn((*ec)->ep_entry)); retval= -1; goto error_return; - } else { - *e_in_cache = 0; /* e un-cached */ } if (orig_ec_in_cache) { /* ec was already added to the cache via cache_add_tentative (to reserve its spot in the cache) @@ -2011,7 +2152,8 @@ moddn_get_children(back_txn *ptxn, struct backentry *parententry, Slapi_DN *dn_parentdn, struct backentry ***child_entries, - struct backdn ***child_dns) + struct backdn ***child_dns, + int is_resurect_operation) { ldbm_instance *inst = (ldbm_instance *) be->be_instance_info; int err= 0; @@ -2039,7 +2181,7 @@ moddn_get_children(back_txn *ptxn, { err = entryrdn_get_subordinates(be, slapi_entry_get_sdn_const(parententry->ep_entry), - parententry->ep_id, &candidates, ptxn); + parententry->ep_id, &candidates, ptxn, is_resurect_operation); if (err) { LDAPDebug1Arg( LDAP_DEBUG_ANY, "moddn_get_children: " "entryrdn_get_subordinates returned %d\n", err); @@ -2058,8 +2200,13 @@ moddn_get_children(back_txn *ptxn, slapi_filter_free(filter,1); } - if (candidates!=NULL) - { + if (candidates) { + Slapi_DN parentsdn = {0}; + if (is_resurect_operation) { + slapi_sdn_get_parent(dn_parentdn, &parentsdn); + dn_parentdn = &parentsdn; + } + sr_current = idl_iterator_init(candidates); result_idl= idl_alloc(candidates->b_nids); do @@ -2092,6 +2239,7 @@ moddn_get_children(back_txn *ptxn, } } while (id!=NOID); idl_free(candidates); + slapi_sdn_done(&parentsdn); } nids = result_idl ? result_idl->b_nids : 0; diff --git a/ldap/servers/slapd/back-ldbm/ldbm_search.c b/ldap/servers/slapd/back-ldbm/ldbm_search.c index 1dcf698..d0563bd 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_search.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_search.c @@ -1209,7 +1209,7 @@ subtree_candidates( /* subtree-rename: on && no ancestorid */ *err = entryrdn_get_subordinates(be, slapi_entry_get_sdn_const(e->ep_entry), - e->ep_id, &descendants, &txn); + e->ep_id, &descendants, &txn, 0); idl_insert(&descendants, e->ep_id); candidates = idl_intersection(be, candidates, descendants); idl_free(tmp); diff --git a/ldap/servers/slapd/back-ldbm/ldif2ldbm.c b/ldap/servers/slapd/back-ldbm/ldif2ldbm.c index d2308a7..c4c9414 100644 --- a/ldap/servers/slapd/back-ldbm/ldif2ldbm.c +++ b/ldap/servers/slapd/back-ldbm/ldif2ldbm.c @@ -923,7 +923,7 @@ static IDList *ldbm_fetch_subtrees(backend *be, char **include, int *err) */ if (entryrdn_get_noancestorid()) { /* subtree-rename: on && no ancestorid */ - *err = entryrdn_get_subordinates(be, &sdn, id, &idl, txn); + *err = entryrdn_get_subordinates(be, &sdn, id, &idl, txn, 0); } else { *err = ldbm_ancestorid_read(be, txn, id, &idl); } diff --git a/ldap/servers/slapd/back-ldbm/parents.c b/ldap/servers/slapd/back-ldbm/parents.c index 59e8763..d375519 100644 --- a/ldap/servers/slapd/back-ldbm/parents.c +++ b/ldap/servers/slapd/back-ldbm/parents.c @@ -58,11 +58,17 @@ char *tombstone_numsubordinates = LDBM_TOMBSTONE_NUMSUBORDINATES_STR; * The routine is allowed to modify the parent entry, and to return a set of * LDAPMods reflecting the changes it made. The LDAPMods array must be freed * by the called by calling ldap_free_mods(p,1) + * + * PARENTUPDATE_RESURECT == turning a tombstone into an entry + * tombstone_numsubordinates-- + * numsubordinates++ */ /* - * PARENTUPDATE_CREATE_TOMBSTONE: increment tombstone_numsubordinates + * PARENTUPDATE_CREATE_TOMBSTONE: turning an entry into a tombstone + * numsubordinates-- + * tombstone_numsubordinates++ * PARENTUPDATE_DELETE_TOMBSTONE: don't touch numsubordinates, and - * decrement tombstone_numsubordinates + * tombstone_numsubordinates-- */ int @@ -84,7 +90,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) op &= PARENTUPDATE_MASK; /* Check nobody is trying to use op == 3, it's not implemented yet */ - PR_ASSERT( (op == PARENTUPDATE_ADD) || (op == PARENTUPDATE_DEL)); + PR_ASSERT((op == PARENTUPDATE_ADD) || (op == PARENTUPDATE_DEL) || (op == PARENTUPDATE_RESURECT)); /* We want to invent a mods set to be passed to modify_apply_mods() */ @@ -114,16 +120,17 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) if (PARENTUPDATE_DELETE_TOMBSTONE != repl_op) { /* are we adding ? */ - if ( (PARENTUPDATE_ADD == op) && !already_present) { + if (((PARENTUPDATE_ADD == op) || (PARENTUPDATE_RESURECT == op)) && !already_present) { /* If so, and the parent entry does not already have a subcount * attribute, we need to add it */ mod_op = LDAP_MOD_ADD; } else if (PARENTUPDATE_DEL == op) { if (!already_present) { /* This means that something is wrong---deleting a child but no subcount present on parent */ - LDAPDebug0Args( LDAP_DEBUG_ANY, - "numsubordinates assertion failure\n" ); + LDAPDebug(LDAP_DEBUG_ANY, "Parent %s has no children. (op 0x%x, repl_op 0x%x)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), op, repl_op); slapi_mods_free(&smods); + PR_ASSERT(0); return -1; } else { if (current_sub_count == 1) { @@ -138,7 +145,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) } /* Now compute the new value */ - if (PARENTUPDATE_ADD == op) { + if ((PARENTUPDATE_ADD == op) || (PARENTUPDATE_RESURECT == op)) { current_sub_count++; } else { current_sub_count--; @@ -159,8 +166,8 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) /* tombstoneNumSubordinates is needed only when this is repl op * and a child is being deleted */ - if (repl_op && (PARENTUPDATE_DEL == op)) { - current_sub_count = LDAP_MAXINT; + current_sub_count = LDAP_MAXINT; + if ((repl_op && (PARENTUPDATE_DEL == op)) || (PARENTUPDATE_RESURECT == op)) { ret = slapi_entry_attr_find(mc->old_entry->ep_entry, tombstone_numsubordinates, &read_attr); if (0 == ret) { @@ -175,7 +182,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) } } - if (PARENTUPDATE_DELETE_TOMBSTONE == repl_op) { + if ((PARENTUPDATE_DELETE_TOMBSTONE == repl_op) || (PARENTUPDATE_RESURECT == op)) { /* deleting a tombstone entry: * reaping or manually deleting it */ if ((current_sub_count != LDAP_MAXINT) && @@ -187,9 +194,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) tombstone_numsubordinates, strlen(value_buffer), value_buffer); } - } - - if (PARENTUPDATE_CREATE_TOMBSTONE == repl_op) { + } else if (PARENTUPDATE_CREATE_TOMBSTONE == repl_op) { /* creating a tombstone entry */ if (current_sub_count != LDAP_MAXINT) { current_sub_count++; diff --git a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h index 95a28b0..3870ce5 100644 --- a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h @@ -201,6 +201,7 @@ struct backentry *dn2entry_ext(Slapi_Backend *be, const Slapi_DN *sdn, back_txn struct backentry *dn2entry_or_ancestor(Slapi_Backend *be, const Slapi_DN *sdn, Slapi_DN *ancestor, back_txn *txn, int *err); struct backentry *dn2ancestor(Slapi_Backend *be,const Slapi_DN *sdn,Slapi_DN *ancestordn,back_txn *txn,int *err); int get_copy_of_entry(Slapi_PBlock *pb, const entry_address *addr, back_txn *txn, int plock_parameter, int must_exist); +int get_copy_of_entry_ext(Slapi_PBlock *pb, ID id, const entry_address *addr, back_txn *txn, int plock_parameter, int must_exist); void done_with_pblock_entry(Slapi_PBlock *pb, int plock_parameter); /* @@ -715,8 +716,8 @@ int entryrdn_index_entry(backend *be, struct backentry *e, int flags, back_txn * int entryrdn_index_read(backend *be, const Slapi_DN *sdn, ID *id, back_txn *txn); int entryrdn_index_read_ext(backend *be, const Slapi_DN *sdn, ID *id, int flags, back_txn *txn); -int entryrdn_rename_subtree(backend *be, const Slapi_DN *oldsdn, Slapi_RDN *newsrdn, const Slapi_DN *newsupsdn, ID id, back_txn *txn); -int entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, back_txn *txn); +int entryrdn_rename_subtree(backend *be, const Slapi_DN *oldsdn, Slapi_RDN *newsrdn, const Slapi_DN *newsupsdn, ID id, back_txn *txn, int flags); +int entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, back_txn *txn, int flags); int entryrdn_lookup_dn(backend *be, const char *rdn, ID id, char **dn, Slapi_RDN **psrdn, back_txn *txn); int entryrdn_get_parent(backend *be, const char *rdn, ID id, char **prdn, ID *pid, back_txn *txn); #endif diff --git a/ldap/servers/slapd/entry.c b/ldap/servers/slapd/entry.c index adf7860..25549a7 100644 --- a/ldap/servers/slapd/entry.c +++ b/ldap/servers/slapd/entry.c @@ -3340,9 +3340,10 @@ slapi_entry_add_rdn_values( Slapi_Entry *e ) * Author/Modifier: RJP */ int -slapi_entry_has_children(const Slapi_Entry *entry) +slapi_entry_has_children_ext(const Slapi_Entry *entry, int include_tombstone) { Slapi_Attr *attr; + int count = 0; LDAPDebug( LDAP_DEBUG_TRACE, "=> slapi_has_children( %s )\n", slapi_entry_get_dn_const(entry), 0, 0); @@ -3352,15 +3353,34 @@ slapi_entry_has_children(const Slapi_Entry *entry) Slapi_Value *sval; slapi_attr_first_value( attr, &sval ); if(sval!=NULL) - { + { const struct berval *bval = slapi_value_get_berval( sval ); if(bval!=NULL) { /* The entry has the attribute, and it's non-zero */ - if (strcmp(bval->bv_val, "0") != 0) - { - LDAPDebug( LDAP_DEBUG_TRACE, "<= slapi_has_children 1\n", 0, 0, 0 ); - return(1); + count = strtol(bval->bv_val, (char **)NULL, 10); + if (count > 0) { + LDAPDebug1Arg( LDAP_DEBUG_TRACE, "<= slapi_has_children %d\n", count); + return count; + } + } + } + } + /*If the subordinatecount exists, and it's nonzero, then return 1.*/ + if (include_tombstone && (slapi_entry_attr_find( entry, "tombstonenumsubordinates", &attr) == 0)) + { + Slapi_Value *sval; + slapi_attr_first_value( attr, &sval ); + if(sval!=NULL) + { + const struct berval *bval = slapi_value_get_berval( sval ); + if(bval!=NULL) + { + /* The entry has the attribute, and it's non-zero */ + count = strtol(bval->bv_val, (char **)NULL, 10); + if (count > 0) { + LDAPDebug1Arg( LDAP_DEBUG_TRACE, "<= slapi_has_tombstone_children %d\n", count); + return count; } } } @@ -3369,6 +3389,12 @@ slapi_entry_has_children(const Slapi_Entry *entry) return(0); } +int +slapi_entry_has_children(const Slapi_Entry *entry) +{ + return slapi_entry_has_children_ext(entry, 0); +} + /* * Renames an entry to simulate a MODRDN operation */ diff --git a/ldap/servers/slapd/rdn.c b/ldap/servers/slapd/rdn.c index 32d783b..a77d086 100644 --- a/ldap/servers/slapd/rdn.c +++ b/ldap/servers/slapd/rdn.c @@ -124,7 +124,7 @@ slapi_rdn_init_dn(Slapi_RDN *rdn,const char *dn) * 1 -- "dn" does not belong to the database; could be "rdn" */ static int -_slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn) +_slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone) { const char *dn = NULL; const char *ndn= NULL; @@ -202,9 +202,16 @@ _slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn) } /* Get the last matched position */ - if(dns) - { - rdn->rdn = slapi_ch_strdup(dns[0]); + if (dns) { + if (is_tombstone && slapi_is_special_rdn(dns[0], RDN_IS_TOMBSTONE)) { + /* merge nsuniqueid=...,<rdn> into one rdn */ + rdn->rdn = slapi_ch_smprintf("%s,%s", dns[0], dns[1]); + slapi_ch_free_string(&dns[0]); + dns[0] = slapi_ch_strdup(rdn->rdn); + charray_remove(dns, dns[1], 1); + } else { + rdn->rdn = slapi_ch_strdup(dns[0]); + } rdn->all_rdns = dns; slapi_setbit_uchar(rdn->flag,FLAG_ALL_RDNS); } @@ -238,7 +245,7 @@ slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn) slapi_rdn_init(rdn); slapi_sdn_init(&sdn); slapi_sdn_set_dn_byval(&sdn, dn); - rc = _slapi_rdn_init_all_dn_ext(rdn, (const Slapi_DN *)&sdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, (const Slapi_DN *)&sdn, 0); slapi_sdn_done(&sdn); return rc; } @@ -257,6 +264,33 @@ slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn) * 1 -- dn does not belong to the database */ int +slapi_rdn_init_all_sdn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone) +{ + int rc = 0; /* success */ + + if (NULL == rdn || NULL == sdn) + { + return -1; + } + slapi_rdn_init(rdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, sdn, is_tombstone); + return rc; +} + +/* + * This function sets DN from sdn to Slapi_RDN. + * Note: The underlying function _slapi_rdn_init_all_dn_ext checks if the DN + * is in the root or sub suffix the server owns. If it is, the root or sub + * suffix is treated as one "rdn" (e.g., "dc=sub,dc=example,dc=com") and 0 is + * returned. If it is not, the DN is separated by ',' and each string is set + * to RDN array. (e.g., input: "uid=A,ou=does_not_exist" ==> "uid=A", "ou= + * does_not_exist") and 1 is returned. + * + * Return Value: 0 -- Success + * -1 -- Error + * 1 -- dn does not belong to the database + */ +int slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn) { int rc = 0; /* success */ @@ -266,7 +300,7 @@ slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn) return -1; } slapi_rdn_init(rdn); - rc = _slapi_rdn_init_all_dn_ext(rdn, sdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, sdn, 0); return rc; } @@ -291,6 +325,17 @@ slapi_rdn_init_rdn(Slapi_RDN *rdn,const Slapi_RDN *fromrdn) } void +slapi_rdn_set_dn_ext(Slapi_RDN *rdn,const char *dn, int skip_tombstone) +{ + const char *mydn = dn; + slapi_rdn_done(rdn); + if (skip_tombstone && slapi_is_special_rdn(dn, RDN_IS_TOMBSTONE)) { + mydn = dn + slapi_uniqueIDRdnSize() + 1/*,*/; + } + slapi_rdn_init_dn(rdn, mydn); +} + +void slapi_rdn_set_dn(Slapi_RDN *rdn,const char *dn) { slapi_rdn_done(rdn); @@ -473,6 +518,33 @@ slapi_rdn_contains(Slapi_RDN *rdn, const char *type, const char *value, size_t l } int +slapi_rdn_is_multivalued(Slapi_RDN *rdn) +{ + char *p = NULL; + if (rdn && rdn->rdn) { + p = PL_strchr(rdn->rdn, '+'); + } + if (p) { + return 1; + } else { + return 0; + } +} + +/* + * Return value 1: if rdn is a conflict rdn + * 0: otherwise + */ +int +slapi_rdn_is_conflict(Slapi_RDN *rdn) +{ + if (!rdn) { + return 0; + } + return slapi_is_special_rdn(slapi_rdn_get_nrdn(rdn), RDN_IS_CONFLICT); +} + +int slapi_rdn_add(Slapi_RDN *rdn, const char *type, const char *value) { PR_ASSERT(NULL != type); @@ -484,12 +556,32 @@ slapi_rdn_add(Slapi_RDN *rdn, const char *type, const char *value) } else { - /* type=value+rdn '\0' */ - char *newrdn = slapi_create_dn_string("%s=%s+%s", type, value, rdn->rdn); + char *newrdn = NULL; + char *rp = rdn->rdn; + PRUint32 uniqueidlen = slapi_uniqueIDRdnSize(); + + if (slapi_is_special_rdn(rp, RDN_IS_TOMBSTONE)) { + char *corerp = rp + uniqueidlen + 1; + *(rp + uniqueidlen) = '\0'; + newrdn = slapi_create_dn_string("%s,%s=%s+%s", rp, type, value, corerp); + } else { + /* type=value+rdn '\0' */ + newrdn = slapi_create_dn_string("%s=%s+%s", type, value, rp); + } slapi_ch_free_string(&rdn->rdn); rdn->rdn = newrdn; } slapi_unsetbit_uchar(rdn->flag,FLAG_RDNS); + + if (rdn->all_rdns && rdn->all_rdns[0]) { + slapi_ch_free_string(&rdn->all_rdns[0]); + rdn->all_rdns[0] = slapi_ch_strdup(rdn->rdn); + } + if (rdn->all_nrdns && rdn->all_nrdns[0]) { + slapi_ch_free_string(&rdn->all_nrdns[0]); + rdn->all_nrdns[0] = slapi_ch_strdup(rdn->rdn); + slapi_dn_ignore_case(rdn->all_nrdns[0]); + } return 1; } diff --git a/ldap/servers/slapd/slapi-plugin.h b/ldap/servers/slapd/slapi-plugin.h index c5a8dd3..c39d3b3 100644 --- a/ldap/servers/slapd/slapi-plugin.h +++ b/ldap/servers/slapd/slapi-plugin.h @@ -2005,6 +2005,17 @@ int slapi_entry_attr_has_syntax_value(const Slapi_Entry *e, const char *type, co int slapi_entry_has_children(const Slapi_Entry *e); /** + * This function determines if the specified entry has child entries + * including the tombstoned descendents. + * + * \param e Entry that you want to test for child entries. + * \param include_tombstone If non-zero, check tombstone_subordinates, too. + * \return \c 1 if the entry you supply has child entries. + * \return \c 0 if the entry you supply has child entries. + */ +int slapi_entry_has_children_ext(const Slapi_Entry *e, int include_tombstone); + +/** * This function determines if an entry is the root DSE. * * The root DSE is a special entry that contains information about the Directory @@ -3152,9 +3163,11 @@ void slapi_rdn_init_rdn(Slapi_RDN *rdn,const Slapi_RDN *fromrdn); * \see slapi_rdn_set_rdn() */ void slapi_rdn_set_dn(Slapi_RDN *rdn,const char *dn); +void slapi_rdn_set_dn_ext(Slapi_RDN *rdn,const char *dn, int skip_tombstone); Slapi_RDN *slapi_rdn_new_all_dn(const char *dn); int slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn); int slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn); +int slapi_rdn_init_all_sdn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone); /** * Sets the RDN value in a \c Slapi_RDN structure from a \c Slapi_DN. @@ -3597,6 +3610,24 @@ char * slapi_rdn_get_value(const char *rdn); */ char * slapi_rdn_get_value_by_ref(const char *rdn); +/** + * Check if the rdn is multivalued or not + * + * \param rdn A pointer to rdn to exam. + * \return 1, if the rdn is multi valued. + * 0, if the rdn is simgle valued. + */ +int slapi_rdn_is_multivalued(Slapi_RDN *rdn); + +/** + * Check if the rdn is a conflict rdn or not + * + * \param rdn A pointer to rdn to exam. + * \return 1, if the rdn is a conflict rdn + * 0, if the rdn is not a conflict rdn + */ +int slapi_rdn_is_conflict(Slapi_RDN *rdn); + /* * utility routines for dealing with DNs */ @@ -7887,6 +7918,20 @@ void slapi_plugin_op_started(void *arg); */ void slapi_plugin_op_finished(void *arg); +/** Check if rdn is a slecial rdn/dn or not. + * + * \param rdn rdn/dn to check + * \param flags specify the type: RDN_IS_TOMBSTONE or RDN_IS_CONFLICT or 0 + * + * \return 1 if rdn matches the flag. + * If flag is IS_TOMBSTONE, returns 1 if rdn is a tombstone rdn/dn. + * If flag is IS_CONFLICT, returns 1 if rdn is a conflict rdn/dn. + * \return 0 otherwise + */ +#define RDN_IS_TOMBSTONE 0x1 +#define RDN_IS_CONFLICT 0x2 +int slapi_is_special_rdn(const char *rdn, int flag); + #ifdef __cplusplus } #endif diff --git a/ldap/servers/slapd/slapi-private.h b/ldap/servers/slapd/slapi-private.h index 5c374f6..3fbdb44 100644 --- a/ldap/servers/slapd/slapi-private.h +++ b/ldap/servers/slapd/slapi-private.h @@ -1020,6 +1020,13 @@ int slapi_uniqueIDIsUUID(const Slapi_UniqueID *uId); */ int slapi_uniqueIDSize( void ); +/* Name: slapi_uniqueIDRdnSize + Description: returns size of SLAPI_ATTR_UNIQUEID=slapi_uniqueIDSize() + Parameters: none + Return: size of the string version of "SLAPI_ATTR_UNIQUEID=uniqueID" in bytes + */ +int slapi_uniqueIDRdnSize( void ); + /* Name: slapi_uniqueIDDup Description: duplicates an UniqueID object Parameters: uId - id to duplicate diff --git a/ldap/servers/slapd/util.c b/ldap/servers/slapd/util.c index c37f6c4..4c23af2 100644 --- a/ldap/servers/slapd/util.c +++ b/ldap/servers/slapd/util.c @@ -1444,3 +1444,70 @@ slapi_get_plugin_name(const char *path, const char *lib) return fullname; } + +/* + * Check if rdn is a slecial rdn/dn or not. + * If flag is IS_TOMBSTONE, returns 1 if rdn is a tombstone rdn/dn. + * If flag is IS_CONFLICT, returns 1 if rdn is a conflict rdn/dn. + * Otherwise returns 0. + */ +static int util_uniqueidlen = 0; +int +slapi_is_special_rdn(const char *rdn, int flag) +{ + char *rp; + int plus = 0; + if (!util_uniqueidlen) { + util_uniqueidlen = SLAPI_ATTR_UNIQUEID_LENGTH + slapi_uniqueIDSize() + 1/*=*/; + } + + if ((RDN_IS_TOMBSTONE != flag) && (RDN_IS_CONFLICT != flag)) { + LDAPDebug1Arg(LDAP_DEBUG_ANY, "slapi_is_special_rdn: invalid flag %d\n", flag); + return 0; /* not a special rdn/dn */ + } + if (!rdn) { + LDAPDebug0Args(LDAP_DEBUG_ANY, "slapi_is_special_rdn: NULL rdn\n"); + return 0; /* not a special rdn/dn */ + } + + if (strlen(rdn) < util_uniqueidlen) { + return 0; /* not a special rdn/dn */ + } + rp = (char *)rdn; + while (rp) { + char *comma = NULL; + if (!PL_strncasecmp(rp, SLAPI_ATTR_UNIQUEID, SLAPI_ATTR_UNIQUEID_LENGTH) && + (*(rp + SLAPI_ATTR_UNIQUEID_LENGTH) == '=')) { + if (RDN_IS_TOMBSTONE == flag) { + if ((*(rp + util_uniqueidlen) == ',') || + (*(rp + util_uniqueidlen) == '\0')) { + return 1; + } else { + return 0; + } + } else { + if ((*(rp + util_uniqueidlen) == '+') || + (plus && ((*(rp + util_uniqueidlen) == ',') || + (*(rp + util_uniqueidlen) == '\0')))) { + return 1; + } + } + } + comma = PL_strchr(rp, ','); + rp = PL_strchr(rp, '+'); + if (rp && (rp < comma)) { + plus = 1; + rp++; + } + } + return 0; +} + +int +slapi_uniqueIDRdnSize() +{ + if (!util_uniqueidlen) { + util_uniqueidlen = SLAPI_ATTR_UNIQUEID_LENGTH + slapi_uniqueIDSize() + 1/*=*/; + } + return util_uniqueidlen; +}
1
0
0
0
Branch '389-ds-base-1.2.11' - ldap/servers
by Noriko Hosoi
12 May '14
12 May '14
ldap/servers/plugins/replication/repl5.h | 1 ldap/servers/plugins/replication/urp.c | 79 +++-- ldap/servers/plugins/replication/urp.h | 5 ldap/servers/plugins/replication/urp_glue.c | 32 +- ldap/servers/plugins/replication/urp_tombstone.c | 55 +++ ldap/servers/slapd/add.c | 15 ldap/servers/slapd/back-ldbm/back-ldbm.h | 3 ldap/servers/slapd/back-ldbm/backentry.c | 1 ldap/servers/slapd/back-ldbm/cache.c | 103 ++++-- ldap/servers/slapd/back-ldbm/ldbm_add.c | 362 ++++++++++++++--------- ldap/servers/slapd/back-ldbm/ldbm_delete.c | 337 ++++++++++++++------- ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c | 12 ldap/servers/slapd/back-ldbm/ldbm_modify.c | 41 +- ldap/servers/slapd/back-ldbm/ldbm_modrdn.c | 281 +++++++++++++---- ldap/servers/slapd/back-ldbm/ldbm_search.c | 2 ldap/servers/slapd/back-ldbm/ldif2ldbm.c | 2 ldap/servers/slapd/back-ldbm/parents.c | 31 + ldap/servers/slapd/back-ldbm/proto-back-ldbm.h | 5 ldap/servers/slapd/entry.c | 38 ++ ldap/servers/slapd/rdn.c | 108 ++++++ ldap/servers/slapd/slapi-plugin.h | 45 ++ ldap/servers/slapd/slapi-private.h | 7 ldap/servers/slapd/util.c | 67 ++++ 23 files changed, 1184 insertions(+), 448 deletions(-) New commits: commit 9e5b3332cc33167837fce0477c8a4d0edc42f0db Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Fri May 9 09:53:36 2014 -0700 Ticket #47750 - Creating a glue fails if one above level is a conflict or missing Including the fix for Ticket #47696 - Large Searches Hang - Possibly entryrdn related Bug Descriptions: There were multiple issues to break the parent-child relationship when a resurrect occurred. When resurrecting a tombstone, there is a possibility that an entry having the same name was already added. In the case, the code consdered the entry is already there, which confused the numsubordinates and tombstone-num- subordinates count. If it is detected, it creates a conflict glue entry instead of a pure glue entry. The 2 numsubordinates counts were also not correctly updated in the resurrection case. When the entryrdn was introduced, Slapi_RDN was held in the entry, which was not correctly updated in the resurrect case. Fix Descriptions: [replication/urp.c] urp_fixup_rename_entry sets the parent uniqueid if it is a RESURRECT mode (called from tombstone_to_glue) as being done in urp_fixup_add_entry, which is necessary to retrieve the correct parent in ldbm_back_modrdn. [replication/urp_glue.c] In create_glue_entry, . there was a possibility to pass a tombstone DN (nsuniqueid=...,<DN>) is used to create a glue entry. If found, the part is removed before passing it to create a glue entry. . there is a possibility to create a glue entry fails due to the parent entry not found. This occurs when the parent entry has turned to be a conflict entry. In the case, instead of retrying the glue creation, it logs an error and returns. [replication/urp_tombstone.c] In tombstone_to_glue, . turning a tombstone to an ordinary glue entry could fail with the entry already exists. (Note: ldbm_back_add guarantees that the found entry has a different nsuniqueid from this tombstone has.) This situation occurs an entry having the same dn could have been added/replicated by this moment. And the entry cannot be used for this tombstone to resurrect since the subordinates/tombstonesubordinates relationships get corrupted. To maintain the relationship properly, the tombstone is resurrected as a conflict, glue entry using urp_fixup_rename_entry. If it still returns LDAP_ALREADY_EXISTS, it is assured the expected glue entry exists since the same nsuniqueid is shared. [slapd/back-ldbm/ldbm_add.c] In ldbm_back_add, . in is_resurect_operation case, a SLAPI_ADD_EXISTING_DN_ENTRY entry and a SLAPI_ADD_EXISTING_UNIQUEID_ENTRY entry are identical, it returns SUCCESS instead of LDAP_ALREADY_EXISTS. . in is_resurect_operation case, the DN is modified from the tombstone DN to the ordinary DN. The DN is stored in e_sdn as well as in e_srdn in the Slapi_Entry. The e_srdn part was not updated. This patch fixes it and e_sdn and e_srdn are in sync. . parent_update_on_childchange adjusts the count of numsubordinates and tombstonenumsubordinates. It only took PARENTUPDATE_ADD|_DELETE operations. This patch adds PARENTUPDATE_RESURECT operation to increment numsubordinates and decrement tombstonenumsubordinates at the same time. . in is_resurect_operation case, a tombstone to be replaced with a glue entry needs to be deleted from the entryrdn index, which was missing. [slapd/back-ldbm/ldbm_delete.c] . Check the target DN if it starts with "nsuniqueid=...,", if it is, log an error and returns. [slapd/back-ldbm/ldbm_modrdn.c] . If it is is_resurect_operation case, it allows to rename a tombstone entry. This is used to support to change a tombstone entry to a conflict glue entry in tombstone_to_glue. . ldbm_back_modify checks if the renamed entry has children or not and if there are, they are updated to point to the renamed entry. The check function slapi_entry_has_children was just checking the numsubordinates attribute type. If a positive value is returned, it took care all the children including the tombstones, but if the renamed entry had tombstoned children only, it considered there were no children and the tombstoned children were not updated to point the renamed entry. This patch fixes it. [slapd/back-ldbm/ldbm_entryrdn.c] . Added a flag to entryrdn_rename_subtree and entryrdn_get_subordinates to treat the given DN a tombstoned DN if it is. In addition, helper APIs such as slapi_rdn_is_multivalued and slapi_rdn_is_ conflict, and slapi_is_special_rdn are added.
https://fedorahosted.org/389/ticket/47750
Reviewed by mreynolds(a)redhat.com (Thank you, Mark!!) diff --git a/ldap/servers/plugins/replication/repl5.h b/ldap/servers/plugins/replication/repl5.h index b10b358..a5761be 100644 --- a/ldap/servers/plugins/replication/repl5.h +++ b/ldap/servers/plugins/replication/repl5.h @@ -60,6 +60,7 @@ #include "llist.h" #include "repl5_ruv.h" #include "cl4.h" +#include "plstr.h" #define REPLICA_TYPE_WINDOWS 1 #define REPLICA_TYPE_MULTIMASTER 0 diff --git a/ldap/servers/plugins/replication/urp.c b/ldap/servers/plugins/replication/urp.c index 9787ff7..0182af3 100644 --- a/ldap/servers/plugins/replication/urp.c +++ b/ldap/servers/plugins/replication/urp.c @@ -58,7 +58,6 @@ static int urp_naming_conflict_removal (Slapi_PBlock *pb, char *sessionid, CSN * static int mod_namingconflict_attr (const char *uniqueid, const Slapi_DN *entrysdn, const Slapi_DN *conflictsdn, CSN *opcsn); static int del_replconflict_attr (Slapi_Entry *entry, CSN *opcsn, int opflags); static char *get_dn_plus_uniqueid(char *sessionid,const Slapi_DN *oldsdn,const char *uniqueid); -static char *get_rdn_plus_uniqueid(char *sessionid,const char *olddn,const char *uniqueid); static int is_suffix_entry (Slapi_PBlock *pb, Slapi_Entry *entry, Slapi_DN **parenddn); /* @@ -372,6 +371,9 @@ urp_modrdn_operation( Slapi_PBlock *pb ) slapi_entry_get_sdn (target_entry), "renameTombstone", opcsn); */ op_result = LDAP_NO_SUCH_OBJECT; + slapi_log_error(SLAPI_LOG_REPL, sessionid, + "urp_modrdn: target_entry %s is a tombstone; returning NO_SUCH_OBJECT.\n", + slapi_entry_get_dn(target_entry)); slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); if (op_result == 0) { @@ -448,16 +450,16 @@ urp_modrdn_operation( Slapi_PBlock *pb ) slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc= -1; /* Ignore this Operation */ PROFILE_POINT; /* ModRDN Conflict; Entry with Target DN Exists; - Unique ID already in RDN - Change to Lost and Found entry */ + Unique ID already in RDN - Change to Lost and Found entry */ goto bailout; } mod_namingconflict_attr (op_uniqueid, target_sdn, existing_sdn, opcsn); slapi_pblock_set(pb, SLAPI_MODRDN_NEWRDN, newrdn_with_uniqueid); slapi_log_error(slapi_log_urp, sessionid, - "Naming conflict MODRDN. Rename target entry to %s\n", - newrdn_with_uniqueid ); + "urp_modrdn_operation: Naming conflict MODRDN. Rename target entry from %s to %s\n", + newrdn, newrdn_with_uniqueid ); - rc= slapi_setbit_int(rc,SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); + rc= slapi_setbit_int(rc, SLAPI_RTN_BIT_FETCH_EXISTING_DN_ENTRY); PROFILE_POINT; /* ModRDN Conflict; Entry with Target DN Exists; Rename Operation Entry */ goto bailout; } @@ -608,14 +610,20 @@ urp_delete_operation( Slapi_PBlock *pb ) op_result= LDAP_NO_SUCH_OBJECT; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc= -1; /* Don't apply the Delete */ + slapi_log_error(slapi_log_urp, "no sessionid", + "Entry %s does not exist; returning NO_SUCH_OBJECT.\n", + slapi_entry_get_dn(deleteentry)); PROFILE_POINT; /* Delete Operation; Entry not exist. */ } else if(is_tombstone_entry(deleteentry)) { /* The entry is already a Tombstone, ignore this delete. */ - op_result= LDAP_SUCCESS; + op_result= LDAP_ALREADY_EXISTS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc = -1; /* Don't apply the Delete */ + slapi_log_error(slapi_log_urp, "no sessionid", + "Entry %s is already a tombstone; returning ALREADY_EXISTS.\n", + slapi_entry_get_dn(deleteentry)); PROFILE_POINT; /* Delete Operation; Already a Tombstone. */ } else /* The entry to be deleted exists and is not a tombstone */ @@ -639,7 +647,10 @@ urp_delete_operation( Slapi_PBlock *pb ) op_result= LDAP_SUCCESS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &op_result); rc = -1; /* Don't apply the Delete */ - PROFILE_POINT; /* Delete Operation; Entry has children. */ + slapi_log_error(slapi_log_urp, sessionid, + "Entry %s was turned to a glue; returning SUCCESS.\n", + slapi_entry_get_dn(deleteentry) ); + PROFILE_POINT; /* Delete Operation; Entry has children. */ } } return rc; @@ -785,7 +796,7 @@ urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *pa } int -urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, int opflags) +urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags) { Slapi_PBlock *newpb; Slapi_Operation *op; @@ -813,7 +824,12 @@ urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, int opflags) opcsn = (CSN *)entry_get_dncsn (entry); slapi_pblock_get (newpb, SLAPI_OPERATION, &op); operation_set_csn (op, opcsn); - + if (parentuniqueid) + { + struct slapi_operation_parameters *op_params; + slapi_pblock_get( newpb, SLAPI_OPERATION_PARAMETERS, &op_params ); + op_params->p.p_modrdn.modrdn_newsuperior_address.uniqueid = (char*)parentuniqueid; /* Consumes parentuniqueid */ + } slapi_modrdn_internal_pb(newpb); slapi_pblock_get(newpb, SLAPI_PLUGIN_INTOP_RESULT, &op_result); @@ -1012,7 +1028,7 @@ urp_annotate_dn (char *sessionid, Slapi_Entry *entry, CSN *opcsn, const char *op if(newrdn!=NULL) { mod_namingconflict_attr (uniqueid, basesdn, basesdn, opcsn); - op_result = urp_fixup_rename_entry ( entry, newrdn, 0 ); + op_result = urp_fixup_rename_entry ( entry, newrdn, NULL, 0 ); switch(op_result) { case LDAP_SUCCESS: @@ -1191,7 +1207,7 @@ urp_naming_conflict_removal ( Slapi_PBlock *pb, char *sessionid, CSN *opcsn, con * is done after DB lock was released. The backend modrdn * will acquire the DB lock if it sees this flag. */ - op_result = urp_fixup_rename_entry (min_naming_conflict_entry, newrdnstr, OP_FLAG_ACTION_INVOKE_FOR_REPLOP); + op_result = urp_fixup_rename_entry (min_naming_conflict_entry, newrdnstr, NULL, OP_FLAG_ACTION_INVOKE_FOR_REPLOP); if ( op_result != LDAP_SUCCESS ) { slapi_log_error (slapi_log_urp, sessionid, @@ -1224,15 +1240,19 @@ bailout: static char * get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *uniqueid) { - Slapi_RDN *rdn= slapi_rdn_new(); - char *newdn; + Slapi_RDN *rdn = slapi_rdn_new(); + char *newdn = NULL; + int rc = slapi_rdn_init_all_sdn_ext(rdn, oldsdn, 1); + if (rc) { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to convert %s to RDN\n", slapi_sdn_get_dn(oldsdn)); + goto bail; + } PR_ASSERT(uniqueid!=NULL); /* Check if the RDN already contains the Unique ID */ - slapi_rdn_set_dn(rdn, slapi_sdn_get_dn(oldsdn)); - if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid))) - { + if (slapi_rdn_is_conflict(rdn)) { /* The Unique ID is already in the RDN. * This is a highly improbable collision. * It suggests that a duplicate UUID was generated. @@ -1241,7 +1261,6 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique */ slapi_log_error(SLAPI_LOG_FATAL, sessionid, "Annotated DN %s has naming conflict\n", slapi_sdn_get_dn(oldsdn) ); - newdn= NULL; } else { @@ -1250,21 +1269,27 @@ get_dn_plus_uniqueid(char *sessionid, const Slapi_DN *oldsdn, const char *unique newdn = slapi_ch_smprintf("%s,%s", slapi_rdn_get_rdn(rdn), parentdn); slapi_ch_free_string(&parentdn); } +bail: slapi_rdn_free(&rdn); return newdn; } -static char * +char * get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid) { - char *newrdn; + char *newrdn = NULL; /* Check if the RDN already contains the Unique ID */ - Slapi_DN *sdn= slapi_sdn_new_dn_byval(olddn); - Slapi_RDN *rdn= slapi_rdn_new(); - slapi_sdn_get_rdn(sdn,rdn); + Slapi_DN *sdn = slapi_sdn_new_dn_byval(olddn); + Slapi_RDN *rdn = slapi_rdn_new(); + int rc = slapi_rdn_init_all_sdn_ext(rdn, sdn, 1); + if (rc) { + slapi_log_error(SLAPI_LOG_FATAL, sessionid, + "Failed to convert %s to RDN\n", olddn); + goto bail; + } + PR_ASSERT(uniqueid!=NULL); - if(slapi_rdn_contains(rdn,SLAPI_ATTR_UNIQUEID,uniqueid,strlen(uniqueid))) - { + if (slapi_rdn_is_conflict(rdn)) { /* The Unique ID is already in the RDN. * This is a highly improbable collision. * It suggests that a duplicate UUID was generated. @@ -1272,14 +1297,14 @@ get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid) * require admin intercession */ slapi_log_error(SLAPI_LOG_FATAL, sessionid, - "Annotated DN %s has naming conflict\n", olddn ); - newrdn= NULL; + "Annotated RDN %s has naming conflict\n", olddn); } else { slapi_rdn_add(rdn,SLAPI_ATTR_UNIQUEID,uniqueid); - newrdn= slapi_ch_strdup(slapi_rdn_get_rdn(rdn)); + newrdn = slapi_ch_strdup(slapi_rdn_get_rdn(rdn)); } +bail: slapi_sdn_free(&sdn); slapi_rdn_free(&rdn); return newrdn; diff --git a/ldap/servers/plugins/replication/urp.h b/ldap/servers/plugins/replication/urp.h index ec7c94d..c7d8ca5 100644 --- a/ldap/servers/plugins/replication/urp.h +++ b/ldap/servers/plugins/replication/urp.h @@ -55,11 +55,12 @@ int urp_delete_operation( Slapi_PBlock *pb ); int urp_post_delete_operation( Slapi_PBlock *pb ); int urp_modrdn_operation( Slapi_PBlock *pb ); int urp_post_modrdn_operation( Slapi_PBlock *pb ); +char *get_rdn_plus_uniqueid(char *sessionid, const char *olddn, const char *uniqueid); /* urp internal ops */ int urp_fixup_add_entry (Slapi_Entry *e, const char *target_uniqueid, const char *parentuniqueid, CSN *opcsn, int opflags); int urp_fixup_delete_entry (const char *uniqueid, const char *dn, CSN *opcsn, int opflags); -int urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, int opflags); +int urp_fixup_rename_entry (Slapi_Entry *entry, const char *newrdn, const char *parentuniqueid, int opflags); int urp_fixup_modify_entry (const char *uniqueid, const Slapi_DN *sdn, CSN *opcsn, Slapi_Mods *smods, int opflags); int is_suffix_dn (Slapi_PBlock *pb, const Slapi_DN *dn, Slapi_DN **parenddn); @@ -78,6 +79,6 @@ PRBool get_glue_csn(const Slapi_Entry *entry, const CSN **gluecsn); * urp_tombstone.c */ int is_tombstone_entry(const Slapi_Entry* entry); -int tombstone_to_glue(Slapi_PBlock *pb, const char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn); +int tombstone_to_glue(Slapi_PBlock *pb, char *sessionid, Slapi_Entry *entry, const Slapi_DN *parentdn, const char *reason, CSN *opcsn); int entry_to_tombstone ( Slapi_PBlock *pb, Slapi_Entry *entry ); PRBool get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn); diff --git a/ldap/servers/plugins/replication/urp_glue.c b/ldap/servers/plugins/replication/urp_glue.c index 6eeb826..199d4de 100644 --- a/ldap/servers/plugins/replication/urp_glue.c +++ b/ldap/servers/plugins/replication/urp_glue.c @@ -153,7 +153,7 @@ static const char *glue_entry = "objectclass: top\n" "objectclass: extensibleObject\n" /* JCMREPL - To avoid schema checking. */ "objectclass: glue\n" - "nsuniqueid: %s\n" + "nsuniqueid: %s\n" /* this uniqueid is set to Slapi_Entry in slapi_str2entry. */ "%s: %s\n"; /* Add why it's been created */ static int @@ -166,15 +166,15 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha Slapi_DN *sdn = NULL; Slapi_RDN *newrdn = slapi_rdn_new_rdn(rdn); char *estr, *rdnstr, *rdntype, *rdnval, *rdnpair; - sdn = slapi_sdn_new_dn_byval(slapi_sdn_get_ndn(superiordn)); + sdn = slapi_sdn_new_ndn_byval(slapi_sdn_get_ndn(superiordn)); slapi_sdn_add_rdn(sdn,rdn); /* must take care of multi-valued rdn: split rdn into different lines introducing * '\n' between each type/value pair. - */ + */ alloc_len = RDNBUFSIZE; rdnstr = slapi_ch_malloc(alloc_len); - rdnpair = rdnstr; + rdnpair = rdnstr; *rdnpair = '\0'; /* so that strlen(rdnstr) may return 0 the first time it's called */ while ((rdnval_index = slapi_rdn_get_next(newrdn, rdnval_index, &rdntype, &rdnval)) != -1) { rdntype_len = strlen(rdntype); @@ -188,7 +188,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha } slapi_ldif_put_type_and_value_with_options(&rdnpair, rdntype, rdnval, rdnval_len, LDIF_OPT_NOWRAP); *rdnpair = '\0'; - } + } estr= slapi_ch_smprintf(glue_entry, slapi_sdn_get_ndn(sdn), rdnstr, uniqueid, ATTR_NSDS5_REPLCONFLICT, reason); slapi_ch_free((void**)&rdnstr); @@ -196,9 +196,7 @@ do_create_glue_entry(const Slapi_RDN *rdn, const Slapi_DN *superiordn, const cha slapi_ch_free((void**)&newrdn); e = slapi_str2entry( estr, 0 ); PR_ASSERT(e!=NULL); - if ( e!=NULL ) - { - slapi_entry_set_uniqueid (e, slapi_ch_strdup(uniqueid)); + if (e) { op_result = urp_fixup_add_entry (e, NULL, NULL, opcsn, 0); } slapi_ch_free_string(&estr); @@ -232,7 +230,7 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char slapi_pblock_get( pb, SLAPI_BACKEND, &backend ); slapi_sdn_get_backend_parent ( dn, superiordn, backend ); - slapi_sdn_get_rdn ( dn, rdn ); + slapi_rdn_set_dn_ext(rdn, slapi_sdn_get_dn(dn), 1/* skip nsuniqeid=..., in dn */); while(!done) { @@ -246,16 +244,30 @@ create_glue_entry ( Slapi_PBlock *pb, char *sessionid, Slapi_DN *dn, const char done= 1; break; case LDAP_ALREADY_EXISTS: + { + struct slapi_operation_parameters *op_params; + /* This is okay. While creating a glue, a real entry was added. */ slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name, "%s: Skipped creating glue entry %s uniqueid=%s reason Entry Already Exists\n", sessionid, dnstr, uniqueid); + op_result = LDAP_SUCCESS; + /* If we could not create a glue having the nsuniqueid, + * we have to abandon it. */ + slapi_pblock_get(pb, SLAPI_OPERATION_PARAMETERS, &op_params); + slapi_ch_free_string(&op_params->p.p_add.parentuniqueid); done= 1; break; + } case LDAP_NO_SUCH_OBJECT: /* The parent is missing */ { /* JCMREPL - Create the parent ... recursion?... but what's the uniqueid? */ - PR_ASSERT(0); /* JCMREPL */ + slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, + "%s: Can't created glue entry %s uniqueid=%s, error %d; " + "Possibly, parent entry is a conflict entry.\n", + sessionid, dnstr, uniqueid, op_result); + done= 1; + break; } default: slapi_log_error ( SLAPI_LOG_FATAL, repl_plugin_name, diff --git a/ldap/servers/plugins/replication/urp_tombstone.c b/ldap/servers/plugins/replication/urp_tombstone.c index a61f1e5..68cb269 100644 --- a/ldap/servers/plugins/replication/urp_tombstone.c +++ b/ldap/servers/plugins/replication/urp_tombstone.c @@ -84,7 +84,7 @@ get_tombstone_csn(const Slapi_Entry *entry, const CSN **delcsn) static int tombstone_to_glue_resolve_parent ( Slapi_PBlock *pb, - const char *sessionid, + char *sessionid, const Slapi_DN *parentdn, const char *parentuniqueid, CSN *opcsn) @@ -144,7 +144,7 @@ tombstone_to_glue_resolve_parent ( int tombstone_to_glue ( Slapi_PBlock *pb, - const char *sessionid, + char *sessionid, Slapi_Entry *tombstoneentry, const Slapi_DN *tombstonedn, const char *reason, @@ -156,6 +156,7 @@ tombstone_to_glue ( Slapi_Entry *addingentry; const char *addingdn; int op_result; + int rdn_is_conflict = 0; /* JCMREPL * Nothing logged to the 5.0 Change Log @@ -170,18 +171,19 @@ tombstone_to_glue ( * which won't help us identify the correct backend to search. */ is_suffix_dn_ext (pb, tombstonedn, &parentdn, 1 /* is_tombstone */); - parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, - SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */ + parentuniqueid= slapi_entry_attr_get_charptr (tombstoneentry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); /* Allocated */ tombstone_to_glue_resolve_parent (pb, sessionid, parentdn, parentuniqueid, opcsn); - slapi_sdn_free(&parentdn); - /* Submit an Add operation to turn the tombstone entry into glue. */ + /* Submit an Add operation to turn the tombstone entry into glue. */ /* * The tombstone is stored with an invalid DN, we must fix this. */ addingentry = slapi_entry_dup(tombstoneentry); addingdn = slapi_sdn_get_dn(tombstonedn); slapi_entry_set_sdn(addingentry, tombstonedn); + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry), slapi_entry_get_dn_const(addingentry)); + rdn_is_conflict = slapi_rdn_is_conflict(slapi_entry_get_srdn(addingentry)); if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) { @@ -189,18 +191,57 @@ tombstone_to_glue ( slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason); } tombstoneuniqueid= slapi_entry_get_uniqueid(tombstoneentry); - op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, parentuniqueid, opcsn, OP_FLAG_RESURECT_ENTRY); + /* + * addingentry and parentuniqueid are consumed in urp_fixup_add_entry, + * regardless of the result. + * Note: addingentry is not really consumed in ldbm_back_add. + * tombstoneentry from DB/entry cache is duplicated and turned to be a glue. + * This addingentry is freed in op_shared_add. + */ + op_result = urp_fixup_add_entry (addingentry, tombstoneuniqueid, slapi_ch_strdup(parentuniqueid), opcsn, OP_FLAG_RESURECT_ENTRY); + if ((LDAP_ALREADY_EXISTS == op_result) && !rdn_is_conflict) { + /* conflict -- there's already the same named entry added. + * But this to-be-glued entry needs to be added since this is a parent of child entries... + * So, rename this tombstone parententry a conflict, glue entry. + * Instead of "fixup_add", we have to "fixup_rename"... + * */ + char *conflictrdn = get_rdn_plus_uniqueid(sessionid, addingdn, tombstoneuniqueid); + if (conflictrdn) { + addingentry = slapi_entry_dup(tombstoneentry); + if (!slapi_entry_attr_hasvalue(addingentry, ATTR_NSDS5_REPLCONFLICT, reason)) { + /* Add the reason of turning it to glue - The backend code will use it*/ + slapi_entry_add_string(addingentry, ATTR_NSDS5_REPLCONFLICT, reason); + } + slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, + "%s: Can't resurrect tombstone to glue reason '%s'. " + "Try with conflict dn %s, error=%d\n", + sessionid, reason, addingdn, op_result); + op_result = urp_fixup_rename_entry(addingentry, (const char *)conflictrdn, parentuniqueid, + OP_FLAG_RESURECT_ENTRY|OP_FLAG_TOMBSTONE_ENTRY); + slapi_entry_free(addingentry); + addingentry = NULL; + } + } + slapi_ch_free_string(&parentuniqueid); if (op_result == LDAP_SUCCESS) { slapi_log_error (slapi_log_urp, repl_plugin_name, "%s: Resurrected tombstone %s to glue reason '%s'\n", sessionid, addingdn, reason); } + else if (LDAP_ALREADY_EXISTS == op_result) + { + slapi_log_error(slapi_log_urp, repl_plugin_name, + "%s: No need to turn tombstone %s to glue; it was already resurrected.\n", + sessionid, addingdn); + op_result = LDAP_SUCCESS; + } else { slapi_log_error (SLAPI_LOG_FATAL, repl_plugin_name, "%s: Can't resurrect tombstone %s to glue reason '%s', error=%d\n", sessionid, addingdn, reason, op_result); } + slapi_sdn_free(&parentdn); return op_result; } diff --git a/ldap/servers/slapd/add.c b/ldap/servers/slapd/add.c index 44a9aa2..1ce975a 100644 --- a/ldap/servers/slapd/add.c +++ b/ldap/servers/slapd/add.c @@ -699,10 +699,17 @@ static void op_shared_add (Slapi_PBlock *pb) slapi_pblock_get(pb, SLAPI_ENTRY_POST_OP, &pse); do_ps_service(pse, NULL, LDAP_CHANGETYPE_ADD, 0); - - /* If be_add succeeded, then e is consumed. We - * set e to NULL to prevent freeing it ourselves. */ - e = NULL; + /* + * If be_add succeeded, then e is consumed except the resurect case. + * If it is resurect, the corresponding tombstone entry is resurected + * and put into the cache. + * Otherwise, we set e to NULL to prevent freeing it ourselves. + */ + if (operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY) && save_e) { + e = save_e; + } else { + e = NULL; + } } else { diff --git a/ldap/servers/slapd/back-ldbm/back-ldbm.h b/ldap/servers/slapd/back-ldbm/back-ldbm.h index 8ad3c20..d27a664 100644 --- a/ldap/servers/slapd/back-ldbm/back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/back-ldbm.h @@ -893,7 +893,8 @@ typedef struct _back_search_result_set /* operation for parent_update_on_childchange */ #define PARENTUPDATE_ADD 0x1 #define PARENTUPDATE_DEL 0x2 -#define PARENTUPDATE_MASK (PARENTUPDATE_ADD|PARENTUPDATE_DEL) +#define PARENTUPDATE_RESURECT 0x4 +#define PARENTUPDATE_MASK (PARENTUPDATE_ADD|PARENTUPDATE_DEL|PARENTUPDATE_RESURECT) #define PARENTUPDATE_CREATE_TOMBSTONE 0x10 #define PARENTUPDATE_DELETE_TOMBSTONE 0x20 diff --git a/ldap/servers/slapd/back-ldbm/backentry.c b/ldap/servers/slapd/back-ldbm/backentry.c index 52468ad..114c2df 100644 --- a/ldap/servers/slapd/back-ldbm/backentry.c +++ b/ldap/servers/slapd/back-ldbm/backentry.c @@ -52,6 +52,7 @@ backentry_free( struct backentry **bep ) return; } ep = *bep; + PR_ASSERT(ep->ep_state & (ENTRY_STATE_DELETED|ENTRY_STATE_NOTINCACHE)); if ( ep->ep_entry != NULL ) { slapi_entry_free( ep->ep_entry ); } diff --git a/ldap/servers/slapd/back-ldbm/cache.c b/ldap/servers/slapd/back-ldbm/cache.c index 865f1ef..5510a9a 100644 --- a/ldap/servers/slapd/back-ldbm/cache.c +++ b/ldap/servers/slapd/back-ldbm/cache.c @@ -890,7 +890,7 @@ entrycache_remove_int(struct cache *cache, struct backentry *e) } else { - LOG("remove %d from id hash failed\n", e->ep_id, 0, 0); + LOG("remove %s (%d) from id hash failed\n", ndn, e->ep_id, 0); } } #ifdef UUIDCACHE_ON @@ -984,13 +984,19 @@ int cache_replace(struct cache *cache, void *oldptr, void *newptr) static int entrycache_replace(struct cache *cache, struct backentry *olde, struct backentry *newe) { - int found; + int found = 0; + int found_in_dn = 0; + int found_in_id = 0; +#ifdef UUIDCACHE_ON + int found_in_uuid = 0; +#endif const char *oldndn; const char *newndn; #ifdef UUIDCACHE_ON const char *olduuid; const char *newuuid; #endif + struct backentry *alte = NULL; LOG("=> entrycache_replace (%s) -> (%s)\n", backentry_get_ndn(olde), backentry_get_ndn(newe), 0); @@ -1013,31 +1019,21 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, * cache tables, operation error */ if ( (olde->ep_state & ENTRY_STATE_NOTINCACHE) == 0 ) { - int found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn)); - int found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID)); + found_in_dn = remove_hash(cache->c_dntable, (void *)oldndn, strlen(oldndn)); + found_in_id = remove_hash(cache->c_idtable, &(olde->ep_id), sizeof(ID)); #ifdef UUIDCACHE_ON - int found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid)); + found_in_uuid = remove_hash(cache->c_uuidtable, (void *)olduuid, strlen(olduuid)); #endif found = found_in_dn && found_in_id; #ifdef UUIDCACHE_ON found = found && found_in_uuid; #endif - if (!found) { -#ifdef UUIDCACHE_ON - LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n", - found_in_dn, found_in_id, found_in_uuid); -#else - LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d]\n", - found_in_dn, found_in_id, 0); -#endif - PR_Unlock(cache->c_mutex); - return 1; - } } - if (! entry_same_dn(newe, (void *)oldndn) && - (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) { - /* if we're doing a modrdn, the new entry can be in the dn table - * already, so we need to remove that too. + /* If fails, we have to make sure the both entires are removed from the cache, + * otherwise, we have no idea what's left in the cache or not... */ + if (!entry_same_dn(newe, (void *)oldndn) && (newe->ep_state & ENTRY_STATE_NOTINCACHE) == 0) { + /* if we're doing a modrdn or turning an entry to a tombstone, + * the new entry can be in the dn table already, so we need to remove that too. */ if (remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn))) { @@ -1046,20 +1042,44 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, LOG("entry cache replace remove entry size %lu\n", newe->ep_size, 0, 0); } } - + /* + * The old entry could have been "removed" between the add and this replace, + * The entry is NOT freed, but NOT in the dn hash. + * which could happen since the entry is not necessarily locked. + * This is ok. + */ + olde->ep_state = ENTRY_STATE_DELETED; /* olde is removed from the cache, so set DELETED here. */ + if (!found) { + if (olde->ep_state & ENTRY_STATE_DELETED) { + LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]; but the entry is alreay deleted.\n", + oldndn, found_in_dn, found_in_id); + } else { +#ifdef UUIDCACHE_ON + LOG("entry cache replace: cache index tables out of sync - found dn [%d] id [%d] uuid [%d]\n", + found_in_dn, found_in_id, found_in_uuid); +#else + LOG("entry cache replace (%s): cache index tables out of sync - found dn [%d] id [%d]\n", + oldndn, found_in_dn, found_in_id); +#endif + PR_Unlock(cache->c_mutex); + return 1; + } + } /* now, add the new entry to the hashtables */ /* (probably don't need such extensive error handling, once this has been * tested enough that we believe it works.) */ - if (!add_hash(cache->c_dntable, (void *)newndn, strlen(newndn), newe, NULL)) { - LOG("entry cache replace: can't add dn\n", 0, 0, 0); + if (!add_hash(cache->c_dntable, (void *)newndn, strlen(newndn), newe, (void **)&alte)) { + LOG("entry cache replace (%s): can't add to dn table (returned %s)\n", + newndn, alte?slapi_entry_get_dn(alte->ep_entry):"none", 0); PR_Unlock(cache->c_mutex); return 1; } - if (!add_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID), newe, NULL)) { - LOG("entry cache replace: can't add id\n", 0, 0, 0); + if (!add_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID), newe, (void **)&alte)) { + LOG("entry cache replace (%s): can't add to id table (returned %s)\n", + newndn, alte?slapi_entry_get_dn(alte->ep_entry):"none", 0); if(remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)) == 0){ - LOG("entry cache replace: failed to remove dn table\n", 0, 0, 0); + LOG("entry cache replace: failed to remove dn table\n", 0, 0, 0); } PR_Unlock(cache->c_mutex); return 1; @@ -1069,10 +1089,10 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, newe, NULL)) { LOG("entry cache replace: can't add uuid\n", 0, 0, 0); if(remove_hash(cache->c_dntable, (void *)newndn, strlen(newndn)) == 0){ - LOG("entry cache replace: failed to remove dn table(uuid cache)\n", 0, 0, 0); + LOG("entry cache replace: failed to remove dn table(uuid cache)\n", 0, 0, 0); } if(remove_hash(cache->c_idtable, &(newe->ep_id), sizeof(ID)) == 0){ - LOG("entry cache replace: failed to remove id table(uuid cache)\n", 0, 0, 0); + LOG("entry cache replace: failed to remove id table(uuid cache)\n", 0, 0, 0); } PR_Unlock(cache->c_mutex); return 1; @@ -1086,7 +1106,6 @@ static int entrycache_replace(struct cache *cache, struct backentry *olde, } else if (newe->ep_size < olde->ep_size) { slapi_counter_subtract(cache->c_cursize, olde->ep_size - newe->ep_size); } - olde->ep_state = ENTRY_STATE_DELETED; newe->ep_state = 0; PR_Unlock(cache->c_mutex); LOG("<= entrycache_replace OK, cache size now %lu cache count now %ld\n", @@ -1269,7 +1288,7 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, PR_Lock(cache->c_mutex); if (! add_hash(cache->c_dntable, (void *)ndn, strlen(ndn), e, (void **)&my_alt)) { - LOG("entry \"%s\" already in dn cache\n", backentry_get_ndn(e), 0, 0); + LOG("entry \"%s\" already in dn cache\n", ndn, 0, 0); /* add_hash filled in 'my_alt' if necessary */ if (my_alt == e) { @@ -1309,14 +1328,15 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, { if (my_alt->ep_state & ENTRY_STATE_CREATING) { - LOG("the entry is reserved\n", 0, 0, 0); + LOG("the entry %s is reserved (ep_state: 0x%x, state: 0x%x\n", ndn, e->ep_state, state); e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); return -1; } else if (state != 0) { - LOG("the entry already exists. cannot reserve it.\n", 0, 0, 0); + LOG("the entry %s already exists. cannot reserve it. (ep_state: 0x%x, state: 0x%x\n", + ndn, e->ep_state, state); e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); return -1; @@ -1328,6 +1348,11 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, if ((*alt)->ep_refcnt == 0) lru_delete(cache, (void *)*alt); (*alt)->ep_refcnt++; + LOG("the entry %s already exists. returning existing entry %s (state: 0x%x)\n", + ndn, backentry_get_ndn(my_alt), state); + } else { + LOG("the entry %s already exists. Not returning existing entry %s (state: 0x%x)\n", + ndn, backentry_get_ndn(my_alt), state); } PR_Unlock(cache->c_mutex); return 1; @@ -1342,7 +1367,7 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, if (state == 0) { /* neither of these should fail, or something is very wrong. */ if (! add_hash(cache->c_idtable, &(e->ep_id), sizeof(ID), e, NULL)) { - LOG("entry %s already in id cache!\n", backentry_get_ndn(e), 0, 0); + LOG("entry %s already in id cache!\n", ndn, 0, 0); if (already_in) { /* there's a bug in the implementatin of 'modify' and 'modrdn' * that i'm working around here. basically they do a @@ -1364,10 +1389,12 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, return 0; } if(remove_hash(cache->c_dntable, (void *)ndn, strlen(ndn)) == 0){ - LOG("entrycache_add_int: failed to remove dn table\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove %s from dn table\n", 0, 0, 0); } e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); + LOG("entrycache_add_int: failed to add %s to cache (ep_state: %x, already_in: %d)\n", + ndn, e->ep_state, already_in); return -1; } #ifdef UUIDCACHE_ON @@ -1378,10 +1405,10 @@ entrycache_add_int(struct cache *cache, struct backentry *e, int state, LOG("entry %s already in uuid cache!\n", backentry_get_ndn(e), 0, 0); if(remove_hash(cache->c_dntable, (void *)ndn, strlen(ndn)) == 0){ - LOG("entrycache_add_int: failed to remove dn table(uuid cache)\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove dn table(uuid cache)\n", 0, 0, 0); } if(remove_hash(cache->c_idtable, &(e->ep_id), sizeof(ID)) == 0){ - LOG("entrycache_add_int: failed to remove id table(uuid cache)\n", 0, 0, 0); + LOG("entrycache_add_int: failed to remove id table(uuid cache)\n", 0, 0, 0); } e->ep_state |= ENTRY_STATE_NOTINCACHE; PR_Unlock(cache->c_mutex); @@ -1487,8 +1514,8 @@ int cache_lock_entry(struct cache *cache, struct backentry *e) if (!e->ep_mutexp) { LOG("<= cache_lock_entry (DELETED)\n", 0, 0, 0); LDAPDebug1Arg(LDAP_DEBUG_ANY, - "cache_lock_entry: failed to create a lock for %s\n", - backentry_get_ndn(e)); + "cache_lock_entry: failed to create a lock for %s\n", + backentry_get_ndn(e)); LOG("<= cache_lock_entry (FAILED)\n", 0, 0, 0); return 1; } diff --git a/ldap/servers/slapd/back-ldbm/ldbm_add.c b/ldap/servers/slapd/back-ldbm/ldbm_add.c index 0ade23c..b129307 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_add.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_add.c @@ -79,7 +79,6 @@ ldbm_back_add( Slapi_PBlock *pb ) backend *be; struct ldbminfo *li; ldbm_instance *inst; - const char *dn = NULL; Slapi_Entry *e = NULL; struct backentry *tombstoneentry = NULL; struct backentry *addingentry = NULL; @@ -118,6 +117,7 @@ ldbm_back_add( Slapi_PBlock *pb ) CSN *opcsn = NULL; entry_address addr = {0}; int parent_switched = 0; + int noabort = 1; slapi_pblock_get( pb, SLAPI_PLUGIN_PRIVATE, &li ); slapi_pblock_get( pb, SLAPI_ADD_ENTRY, &e ); @@ -210,15 +210,14 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_ADD_TARGET_SDN, &sdn ); if (NULL == sdn) { - LDAPDebug0Args(LDAP_DEBUG_ANY, - "ldbm_back_add: Null target dn\n"); + LDAPDebug0Args(LDAP_DEBUG_ANY, "ldbm_back_add: FETCH_EXISTING_DN_ENTRY is set, but target dn is NULL.\n"); + ldap_result_code = LDAP_OPERATIONS_ERROR; goto error_return; } /* not need to check the dn syntax as this is a replicated op */ if(!is_replicated_operation){ - dn = slapi_sdn_get_dn(sdn); - ldap_result_code = slapi_dn_syntax_check(pb, dn, 1); + ldap_result_code = slapi_dn_syntax_check(pb, slapi_sdn_get_dn(sdn), 1); if (ldap_result_code) { ldap_result_code = LDAP_INVALID_DN_SYNTAX; @@ -226,7 +225,10 @@ ldbm_back_add( Slapi_PBlock *pb ) goto error_return; } } - + /* + * If the parent is conflict, slapi_sdn_get_backend_parent does not support it. + * That is, adding children to a conflict entry is not allowed. + */ slapi_sdn_get_backend_parent(sdn, &parentsdn, pb->pb_backend); /* Check if an entry with the intended DN already exists. */ done_with_pblock_entry(pb,SLAPI_ADD_EXISTING_DN_ENTRY); /* Could be through this multiple times */ @@ -234,30 +236,27 @@ ldbm_back_add( Slapi_PBlock *pb ) addr.udn = NULL; addr.uniqueid = NULL; ldap_result_code= get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_EXISTING_DN_ENTRY, !is_replicated_operation); - if(ldap_result_code==LDAP_OPERATIONS_ERROR || - ldap_result_code==LDAP_INVALID_DN_SYNTAX) - { - goto error_return; + if ((ldap_result_code == LDAP_OPERATIONS_ERROR) || (ldap_result_code==LDAP_INVALID_DN_SYNTAX)) { + goto error_return; } } /* if we can find the parent by dn or uniqueid, and the operation has requested the parent then get it */ - if(have_parent_address(&parentsdn, operation->o_params.p.p_add.parentuniqueid) && - slapi_isbitset_int(rc,SLAPI_RTN_BIT_FETCH_PARENT_ENTRY)) - { + if (have_parent_address(&parentsdn, operation->o_params.p.p_add.parentuniqueid) && + slapi_isbitset_int(rc,SLAPI_RTN_BIT_FETCH_PARENT_ENTRY)) { done_with_pblock_entry(pb,SLAPI_ADD_PARENT_ENTRY); /* Could be through this multiple times */ addr.sdn = &parentsdn; addr.udn = NULL; addr.uniqueid = operation->o_params.p.p_add.parentuniqueid; - ldap_result_code= get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_PARENT_ENTRY, !is_replicated_operation); - /* need to set parentsdn or parentuniqueid if either is not set? */ + ldap_result_code = get_copy_of_entry(pb, &addr, &txn, SLAPI_ADD_PARENT_ENTRY, + !is_replicated_operation); } /* Call the Backend Pre Add plugins */ + ldap_result_code = LDAP_SUCCESS; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); - rc= plugin_call_plugins(pb, SLAPI_PLUGIN_BE_PRE_ADD_FN); - if(rc==-1) - { + rc = plugin_call_plugins(pb, SLAPI_PLUGIN_BE_PRE_ADD_FN); + if (rc == -1) { int opreturn = 0; /* * Plugin indicated some kind of failure, @@ -284,7 +283,7 @@ ldbm_back_add( Slapi_PBlock *pb ) * the loop once again to get the new present state. */ /* JCMREPL - Warning: A Plugin could cause an infinite loop by always returning a result code that requires some action. */ - } + } /* * Originally (in the U-M LDAP 3.3 code), there was a comment near this @@ -309,10 +308,14 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Set the parentuniqueid now */ operation->o_params.p.p_add.parentuniqueid = slapi_ch_strdup(slapi_entry_get_uniqueid(parententry->ep_entry)); } - if (slapi_sdn_isempty(&parentsdn)) { + if (slapi_sdn_isempty(&parentsdn) || + slapi_sdn_compare(&parentsdn, slapi_entry_get_sdn(parententry->ep_entry))) { /* Set the parentsdn now */ slapi_sdn_set_dn_byval(&parentsdn, slapi_entry_get_dn_const(parententry->ep_entry)); } + } else { + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "find_entry2modify_only returned NULL parententry pdn: %s, uniqueid: %s\n", + slapi_sdn_get_dn(&parentsdn), addr.uniqueid?addr.uniqueid:"none"); } modify_init(&parent_modify_c,parententry); } @@ -320,11 +323,39 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Check if the entry we have been asked to add already exists */ { Slapi_Entry *entry; - slapi_pblock_get( pb, SLAPI_ADD_EXISTING_DN_ENTRY, &entry); - if ( entry != NULL ) - { - /* The entry already exists */ - ldap_result_code= LDAP_ALREADY_EXISTS; + slapi_pblock_get(pb, SLAPI_ADD_EXISTING_DN_ENTRY, &entry); + if (entry) { + if (is_resurect_operation) { + Slapi_Entry *uniqentry; + slapi_pblock_get(pb, SLAPI_ADD_EXISTING_UNIQUEID_ENTRY, &uniqentry); + if (uniqentry == entry) { + /* + * adding entry having the uniqueid exists. + * No need to resurrect. + */ + ldap_result_code = LDAP_SUCCESS; + } else { + /* + * The entry having the DN already exists. + * But the + */ + if (uniqentry) { + if (PL_strcmp(slapi_entry_get_uniqueid(entry), + slapi_entry_get_uniqueid(uniqentry))) { + /* Not match; conflict. */ + ldap_result_code = LDAP_ALREADY_EXISTS; + } else { + /* Same entry; no need to resurrect. */ + ldap_result_code = LDAP_SUCCESS; + } + } else { + ldap_result_code = LDAP_ALREADY_EXISTS; + } + } + } else { + /* The entry already exists */ + ldap_result_code = LDAP_ALREADY_EXISTS; + } goto error_return; } else @@ -335,7 +366,7 @@ ldbm_back_add( Slapi_PBlock *pb ) * entry we did match has a referral we should return * instead. we do this only if managedsait is not on. */ - if ( !managedsait && !is_tombstone_operation ) + if (!managedsait && !is_tombstone_operation && !is_resurect_operation) { int err= 0; Slapi_DN ancestorsdn; @@ -392,14 +423,14 @@ ldbm_back_add( Slapi_PBlock *pb ) if ( tombstoneentry==NULL ) { ldap_result_code= -1; - goto error_return; /* error result sent by find_entry2modify() */ + goto error_return; /* error result sent by find_entry2modify() */ } tombstone_in_cache = 1; addingentry = backentry_dup( tombstoneentry ); if ( addingentry==NULL ) { - ldap_result_code= LDAP_OPERATIONS_ERROR; + ldap_result_code = LDAP_OPERATIONS_ERROR; goto error_return; } /* @@ -411,17 +442,21 @@ ldbm_back_add( Slapi_PBlock *pb ) */ if (NULL == sdn) { LDAPDebug0Args(LDAP_DEBUG_ANY, "ldbm_back_add: Null target dn\n"); + ldap_result_code = LDAP_OPERATIONS_ERROR; goto error_return; } - dn = slapi_sdn_get_dn(sdn); slapi_entry_set_sdn(addingentry->ep_entry, sdn); /* The DN is passed into the entry. */ + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry->ep_entry), + slapi_entry_get_dn_const(addingentry->ep_entry)); /* LPREPL: the DN is normalized...Somehow who should get a not normalized one */ addingentry->ep_id = slapi_entry_attr_get_ulong(addingentry->ep_entry,"entryid"); slapi_entry_attr_delete(addingentry->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); slapi_entry_delete_string(addingentry->ep_entry, SLAPI_ATTR_OBJECTCLASS, SLAPI_ATTR_VALUE_TOMBSTONE); /* Now also remove the nscpEntryDN */ if (slapi_entry_attr_delete(addingentry->ep_entry, SLAPI_ATTR_NSCP_ENTRYDN) != 0){ - LDAPDebug(LDAP_DEBUG_REPL, "Resurrection of %s - Couldn't remove %s\n", dn, SLAPI_ATTR_NSCP_ENTRYDN, 0); + LDAPDebug(LDAP_DEBUG_REPL, "Resurrection of %s - Couldn't remove %s\n", + slapi_sdn_get_dn(sdn), SLAPI_ATTR_NSCP_ENTRYDN, 0); } /* And copy the reason from e */ @@ -429,7 +464,8 @@ ldbm_back_add( Slapi_PBlock *pb ) if (reason) { if (!slapi_entry_attr_hasvalue(addingentry->ep_entry, "nsds5ReplConflict", reason)) { slapi_entry_add_string(addingentry->ep_entry, "nsds5ReplConflict", reason); - LDAPDebug(LDAP_DEBUG_REPL, "Resurrection of %s - Added Conflict reason %s\n", dn, reason, 0); + LDAPDebug(LDAP_DEBUG_REPL, "Resurrection of %s - Added Conflict reason %s\n", + slapi_sdn_get_dn(sdn), reason, 0); } slapi_ch_free((void **)&reason); } @@ -578,9 +614,8 @@ ldbm_back_add( Slapi_PBlock *pb ) Slapi_DN ancestorsdn; struct backentry *ancestorentry; - LDAPDebug( LDAP_DEBUG_TRACE, - "parent does not exist, pdn = %s\n", - slapi_sdn_get_dn(&parentsdn), 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_BACKLDBM, "ldbm_add: Parent \"%s\" does not exist. " + "It might be a conflict entry.\n", slapi_sdn_get_dn(&parentsdn)); slapi_sdn_init(&ancestorsdn); ancestorentry = dn2ancestor(be, &parentsdn, &ancestorsdn, &txn, &err ); @@ -591,22 +626,38 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_sdn_done(&ancestorsdn); goto error_return; } - ldap_result_code = plugin_call_acl_plugin (pb, e, NULL, NULL, SLAPI_ACL_ADD, - ACLPLUGIN_ACCESS_DEFAULT, &errbuf ); + ldap_result_code = plugin_call_acl_plugin(pb, e, NULL, NULL, SLAPI_ACL_ADD, + ACLPLUGIN_ACCESS_DEFAULT, &errbuf ); if ( ldap_result_code != LDAP_SUCCESS ) { - LDAPDebug( LDAP_DEBUG_TRACE, "no access to parent\n", 0, 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "no access to parent, pdn = %s\n", + slapi_sdn_get_dn(&parentsdn)); ldap_result_message= errbuf; goto error_return; } pid = parententry->ep_id; + + /* We may need to adjust the DN since parent could be a resrected conflict entry... */ + if (!slapi_sdn_isparent(slapi_entry_get_sdn_const(parententry->ep_entry), + slapi_entry_get_sdn_const(addingentry->ep_entry))) { + Slapi_DN adjustedsdn = {0}; + char *adjusteddn = slapi_ch_smprintf("%s,%s", + slapi_entry_get_rdn_const(addingentry->ep_entry), + slapi_entry_get_dn_const(parententry->ep_entry)); + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "ldbm_add: adjusting dn: %s --> %s\n", + slapi_entry_get_dn(addingentry->ep_entry), adjusteddn); + slapi_sdn_set_normdn_passin(&adjustedsdn, adjusteddn); + slapi_entry_set_sdn(addingentry->ep_entry, &adjustedsdn); + /* not just e_sdn, e_rsdn needs to be updated. */ + slapi_rdn_set_all_dn(slapi_entry_get_srdn(addingentry->ep_entry), adjusteddn); + slapi_sdn_done(&adjustedsdn); + } } else { /* no parent */ - if ( !isroot && !is_replicated_operation) + if (!isroot && !is_replicated_operation) { - LDAPDebug( LDAP_DEBUG_TRACE, "no parent & not root\n", - 0, 0, 0 ); + LDAPDebug0Args(LDAP_DEBUG_TRACE, "no parent & not root\n"); ldap_result_code= LDAP_INSUFFICIENT_ACCESS; goto error_return; } @@ -635,11 +686,12 @@ ldbm_back_add( Slapi_PBlock *pb ) * operational attributes to ensure that the cache is sized correctly. */ if ( cache_add_tentative( &inst->inst_cache, addingentry, NULL )!= 0 ) { - LDAPDebug( LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected\n", 0, 0, 0 ); + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code= LDAP_ALREADY_EXISTS; goto error_return; } - addingentry_in_cache= 1; + addingentry_in_cache = 1; /* * Before we add the entry, find out if the syntax of the aci @@ -647,7 +699,8 @@ ldbm_back_add( Slapi_PBlock *pb ) * the entry if the syntax is incorrect. */ if ( plugin_call_acl_verify_syntax (pb, addingentry->ep_entry, &errbuf) != 0 ) { - LDAPDebug( LDAP_DEBUG_TRACE, "ACL syntax error\n", 0,0,0); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "ACL syntax error: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code= LDAP_INVALID_SYNTAX; ldap_result_message= errbuf; goto error_return; @@ -656,12 +709,15 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Having decided that we're really going to do the operation, let's modify the in-memory state of the parent to reflect the new child (update subordinate count specifically */ - if (NULL != parententry) + if (parententry) { retval = parent_update_on_childchange(&parent_modify_c, - PARENTUPDATE_ADD, NULL); + is_resurect_operation?PARENTUPDATE_RESURECT:PARENTUPDATE_ADD, + NULL); /* The modify context now contains info needed later */ - if (0 != retval) { + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_BACKLDBM, "parent_update_on_childchange: %s, rc=%d\n", + slapi_entry_get_dn_const(addingentry->ep_entry), retval); ldap_result_code= LDAP_OPERATIONS_ERROR; goto error_return; } @@ -681,11 +737,16 @@ ldbm_back_add( Slapi_PBlock *pb ) for (retry_count = 0; retry_count < RETRY_TIMES; retry_count++) { if (txn.back_txn_txn && (txn.back_txn_txn != parent_txn)) { dblayer_txn_abort(li,&txn); + noabort = 1; slapi_pblock_set(pb, SLAPI_TXN, parent_txn); if (addingentry_in_cache) { /* addingentry is in cache. Remove it once. */ - CACHE_REMOVE(&inst->inst_cache, addingentry); + retval = CACHE_REMOVE(&inst->inst_cache, addingentry); + if (retval) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "ldbm_add: cache_remove %s failed.\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); + } CACHE_RETURN(&inst->inst_cache, &addingentry); } else { backentry_free(&addingentry); @@ -698,11 +759,11 @@ ldbm_back_add( Slapi_PBlock *pb ) } if (addingentry_in_cache) { /* Adding the resetted addingentry to the cache. */ - if (cache_add_tentative(&inst->inst_cache, - addingentry, NULL) != 0) { - LDAPDebug0Args(LDAP_DEBUG_CACHE, - "cache_add_tentative concurrency detected\n"); + if (cache_add_tentative(&inst->inst_cache, addingentry, NULL) != 0) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, "cache_add_tentative concurrency detected: %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry)); ldap_result_code = LDAP_ALREADY_EXISTS; + addingentry_in_cache = 0; goto error_return; } } @@ -732,6 +793,7 @@ ldbm_back_add( Slapi_PBlock *pb ) ldap_result_code= LDAP_OPERATIONS_ERROR; goto error_return; } + noabort = 0; /* stash the transaction for plugins */ slapi_pblock_set(pb, SLAPI_TXN, txn.back_txn_txn); @@ -739,14 +801,12 @@ ldbm_back_add( Slapi_PBlock *pb ) /* call the transaction pre add plugins just after creating the transaction */ if ((retval = plugin_call_plugins(pb, SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN))) { int opreturn = 0; - LDAPDebug1Arg( LDAP_DEBUG_TRACE, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " - "returned error code %d\n", retval ); + LDAPDebug1Arg(LDAP_DEBUG_TRACE, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " + "returned error code %d\n", retval ); if (!ldap_result_code) { slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } if (!ldap_result_code) { - LDAPDebug0Args( LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin " - "returned error but did not setSLAPI_RESULT_CODE \n" ); ldap_result_code = LDAP_OPERATIONS_ERROR; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); } @@ -754,6 +814,8 @@ ldbm_back_add( Slapi_PBlock *pb ) if (!opreturn) { slapi_pblock_set(pb, SLAPI_PLUGIN_OPRETURN, ldap_result_code ? &ldap_result_code : &retval); } + LDAPDebug1Arg(LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_PRE_ADD_FN plugin failed: %d", + ldap_result_code ? ldap_result_code : retval); goto error_return; } @@ -764,9 +826,10 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_TRACE, "id2entry_add failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + if (retval) { + LDAPDebug(LDAP_DEBUG_TRACE, "id2entry_add(%s) failed, err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; @@ -782,11 +845,11 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 1 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string TOMBSTONE (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -800,10 +863,10 @@ ldbm_back_add( Slapi_PBlock *pb ) continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 2 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string UNIQUEID (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -821,17 +884,29 @@ ldbm_back_add( Slapi_PBlock *pb ) continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 3 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + LDAPDebug(LDAP_DEBUG_TRACE, "index_addordel_string ENTRYDN (%s), err=%d %s\n", + slapi_entry_get_dn_const(addingentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; } goto error_return; } - } + /* Need to delete the entryrdn index of the resurrected tombstone... */ + if (entryrdn_get_switch()) { /* subtree-rename: on */ + if (tombstoneentry) { + retval = entryrdn_index_entry(be, tombstoneentry, BE_INDEX_DEL, &txn); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, "Resurrecting %s: failed to remove entryrdn index, err=%d %s\n", + slapi_entry_get_dn_const(tombstoneentry->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + goto error_return; + } + } + } + } if (is_tombstone_operation) { retval = index_addordel_entry( be, addingentry, BE_INDEX_ADD | BE_INDEX_TOMBSTONE, &txn ); @@ -846,9 +921,9 @@ ldbm_back_add( Slapi_PBlock *pb ) /* retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_ANY, "add: attempt to index %lu failed\n", - (u_long)addingentry->ep_id, 0, 0 ); + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_ANY, "add: attempt to index %lu failed; rc=%d\n", + (u_long)addingentry->ep_id, retval); ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; @@ -865,11 +940,10 @@ ldbm_back_add( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "add 1 BAD, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "modify_update_all: %s (%lu) failed; rc=%d\n", + slapi_entry_get_dn(addingentry->ep_entry), (u_long)addingentry->ep_id, retval); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -884,17 +958,16 @@ ldbm_back_add( Slapi_PBlock *pb ) { retval= vlv_update_all_indexes(&txn, be, pb, NULL, addingentry); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, - "add DEADLOCK vlv_update_index\n", 0, 0, 0 ); + LDAPDebug(LDAP_DEBUG_ARGS, + "add DEADLOCK vlv_update_index\n", 0, 0, 0 ); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, - "vlv_update_index failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); - ADD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + if (retval) { + LDAPDebug2Args(LDAP_DEBUG_TRACE, + "vlv_update_index failed, err=%d %s\n", + retval, (msg = dblayer_strerror( retval )) ? msg : ""); + ADD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); if (LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; goto diskfull_return; @@ -940,7 +1013,7 @@ ldbm_back_add( Slapi_PBlock *pb ) if (retry_count == RETRY_TIMES) { /* Failed */ LDAPDebug( LDAP_DEBUG_ANY, "Retry count exceeded in add\n", 0, 0, 0 ); - ldap_result_code= LDAP_BUSY; + ldap_result_code= LDAP_BUSY; goto error_return; } @@ -956,33 +1029,49 @@ ldbm_back_add( Slapi_PBlock *pb ) /* * We can now switch the tombstone entry with the real entry. */ - if (cache_replace( &inst->inst_cache, tombstoneentry, addingentry ) != 0 ) - { + retval = cache_replace(&inst->inst_cache, tombstoneentry, addingentry); + if (retval) { /* This happens if the dn of addingentry already exists */ - cache_unlock_entry( &inst->inst_cache, tombstoneentry ); ADD_SET_ERROR(ldap_result_code, LDAP_ALREADY_EXISTS, retry_count); + LDAPDebug2Args(LDAP_DEBUG_CACHE, "ldap_add: cache_replace concurrency detected: %s (rc: %d)\n", + slapi_entry_get_dn_const(addingentry->ep_entry), retval); + retval = -1; goto error_return; } + if (addingentry_in_cache) { /* decrease the refcnt added by tentative */ + CACHE_RETURN( &inst->inst_cache, &addingentry ); + } + addingentry_in_cache = 1; /* reset it to make it sure... */ /* * The tombstone was locked down in the cache... we can * get rid of the entry in the cache now. + * We cannot expect tombstoneentry exists from now on. */ + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, delete the tombstone dn from the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, + tombstoneentry->ep_id); + if (bdn) { /* in the dncache, remove it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } cache_unlock_entry( &inst->inst_cache, tombstoneentry ); CACHE_RETURN( &inst->inst_cache, &tombstoneentry ); - tombstone_in_cache = 0; /* deleted */ + tombstone_in_cache = 0; } if (parent_found) { /* switch the parent entry copy into play */ - modify_switch_entries( &parent_modify_c,be); + modify_switch_entries(&parent_modify_c,be); parent_switched = 1; } if (ruv_c_init) { if (modify_switch_entries(&ruv_c, be) != 0 ) { ldap_result_code= LDAP_OPERATIONS_ERROR; - LDAPDebug( LDAP_DEBUG_ANY, - "ldbm_back_add: modify_switch_entries failed\n", 0, 0, 0); + LDAPDebug0Args(LDAP_DEBUG_ANY, + "ldbm_back_add: modify_switch_entries failed\n"); goto error_return; } } @@ -996,8 +1085,6 @@ ldbm_back_add( Slapi_PBlock *pb ) slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } if (!ldap_result_code) { - LDAPDebug0Args( LDAP_DEBUG_ANY, "SLAPI_PLUGIN_BE_TXN_POST_ADD_FN plugin " - "returned error but did not set SLAPI_RESULT_CODE\n" ); ldap_result_code = LDAP_OPERATIONS_ERROR; slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); } @@ -1020,6 +1107,7 @@ ldbm_back_add( Slapi_PBlock *pb ) } goto error_return; } + noabort = 1; rc= 0; goto common_return; @@ -1029,21 +1117,20 @@ error_return: { next_id_return( be, addingentry->ep_id ); } - if ( NULL != addingentry ) + if ( addingentry ) { if ( addingentry_in_cache ) { CACHE_REMOVE(&inst->inst_cache, addingentry); - addingentry_in_cache = 0; } - backentry_clear_entry(addingentry); /* e is released in the frontend */ - backentry_free( &addingentry ); /* release the backend wrapper, here */ - } - if(tombstone_in_cache) - { - CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + else + { + if (!is_resurect_operation) { /* if resurect, tombstoneentry is dupped. */ + backentry_clear_entry(addingentry); /* e is released in the frontend */ + } + backentry_free( &addingentry ); /* release the backend wrapper, here */ + } } - if (rc == DB_RUNRECOVERY) { dblayer_remember_disk_filled(li); ldbm_nasty("Add",80,rc); @@ -1063,7 +1150,7 @@ error_return: diskfull_return: if (disk_full) { - rc= return_on_disk_full(li); + rc = return_on_disk_full(li); } else { /* It is safer not to abort when the transaction is not started. */ if (txn.back_txn_txn && (txn.back_txn_txn != parent_txn)) { @@ -1096,39 +1183,44 @@ diskfull_return: } } - dblayer_txn_abort(li,&txn); /* abort crashes in case disk full */ + if (!noabort) { + dblayer_txn_abort(li,&txn); /* abort crashes in case disk full */ + } /* txn is no longer valid - reset the txn pointer to the parent */ slapi_pblock_set(pb, SLAPI_TXN, parent_txn); } - rc= SLAPI_FAIL_GENERAL; + rc = SLAPI_FAIL_GENERAL; } common_return: - if (addingentry_in_cache && addingentry) - { - if (entryrdn_get_switch()) { /* subtree-rename: on */ - /* since adding the entry to the entry cache was successful, - * let's add the dn to dncache, if not yet done. */ - struct backdn *bdn = dncache_find_id(&inst->inst_dncache, - addingentry->ep_id); - if (bdn) { /* already in the dncache */ - CACHE_RETURN(&inst->inst_dncache, &bdn); - } else { /* not in the dncache yet */ - Slapi_DN *addingsdn = - slapi_sdn_dup(slapi_entry_get_sdn(addingentry->ep_entry)); - if (addingsdn) { - bdn = backdn_init(addingsdn, addingentry->ep_id, 0); - if (bdn) { - CACHE_ADD( &inst->inst_dncache, bdn, NULL ); - CACHE_RETURN(&inst->inst_dncache, &bdn); - slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_add", - "set %s to dn cache\n", dn); - } - } - } - } - CACHE_RETURN( &inst->inst_cache, &addingentry ); - } + if (inst) { + if(tombstone_in_cache && tombstoneentry) { + cache_unlock_entry(&inst->inst_cache, tombstoneentry); + CACHE_RETURN(&inst->inst_cache, &tombstoneentry); + } + } + if (addingentry_in_cache && addingentry) { + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, + addingentry->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *addingsdn = slapi_sdn_dup(slapi_entry_get_sdn(addingentry->ep_entry)); + if (addingsdn) { + bdn = backdn_init(addingsdn, addingentry->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + CACHE_RETURN(&inst->inst_dncache, &bdn); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_add", + "set %s to dn cache\n", slapi_sdn_get_dn(sdn)); + } + } + } + } + CACHE_RETURN( &inst->inst_cache, &addingentry ); + } /* bepost op needs to know this result */ slapi_pblock_set(pb, SLAPI_RESULT_CODE, &ldap_result_code); /* JCMREPL - The bepostop is called even if the operation fails. */ diff --git a/ldap/servers/slapd/back-ldbm/ldbm_delete.c b/ldap/servers/slapd/back-ldbm/ldbm_delete.c index c9ce70f..32feeb1 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_delete.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_delete.c @@ -91,6 +91,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) /* replication, we must create a new tombstone entry */ int tombstone_in_cache = 0; int e_in_cache = 0; + int remove_e_from_cache = 0; entry_address *addr; int addordel_flags = 0; /* passed to index_addordel */ char *entryusn_str = NULL; @@ -178,18 +179,29 @@ ldbm_back_delete( Slapi_PBlock *pb ) } /* find and lock the entry we are about to modify */ + /* + * A corner case: + * If a conflict occurred in a MMR topology, a replicated delete + * op from another master could target a conflict entry; while the + * corresponding entry on this server could have been already + * deleted. That is, the entry 'e' found with "addr" is a tomb- + * stone. If it is the case, we need to back off. + */ if ( (e = find_entry2modify( pb, be, addr, &txn )) == NULL ) { ldap_result_code= LDAP_NO_SUCH_OBJECT; /* retval is -1 */ + LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "ldbm_back_delete: Deleting entry is already deleted.\n"); goto error_return; /* error result sent by find_entry2modify() */ } e_in_cache = 1; /* e is cached */ - if ( slapi_entry_has_children( e->ep_entry ) ) - { + retval = slapi_entry_has_children(e->ep_entry); + if (retval) { ldap_result_code= LDAP_NOT_ALLOWED_ON_NONLEAF; - /* retval is -1 */ + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_delete", "Deleting entry %s has %d children.\n", + slapi_entry_get_dn(e->ep_entry), retval); + retval = -1; goto error_return; } @@ -221,7 +233,17 @@ ldbm_back_delete( Slapi_PBlock *pb ) * Plugin indicated some kind of failure, * or that this Operation became a No-Op. */ + slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); if (!ldap_result_code) { + if (LDAP_ALREADY_EXISTS == ldap_result_code) { + /* + * The target entry is already a tombstone. + * We need to treat this as a success, + * but we need to remove the entry e from the entry cache. + */ + remove_e_from_cache = 1; + ldap_result_code = LDAP_SUCCESS; + } slapi_pblock_get(pb, SLAPI_RESULT_CODE, &ldap_result_code); } /* restore original entry so the front-end delete code can free it */ @@ -233,8 +255,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } /* the flag could be set in a preop plugin (e.g., USN) */ - delete_tombstone_entry = operation_is_flag_set(operation, - OP_FLAG_TOMBSTONE_ENTRY); + delete_tombstone_entry = operation_is_flag_set(operation, OP_FLAG_TOMBSTONE_ENTRY); } /* @@ -370,13 +391,15 @@ ldbm_back_delete( Slapi_PBlock *pb ) } if (NULL == parent) { entry_address parent_addr; - + if (is_tombstone_entry) { + parent_addr.uniqueid = slapi_entry_attr_get_charptr(e->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + } else { + parent_addr.uniqueid = NULL; + } parent_addr.sdn = &parentsdn; - parent_addr.uniqueid = NULL; - parent = find_entry2modify_only_ext(pb, be, &parent_addr, - TOMBSTONE_INCLUDED, &txn); + parent = find_entry2modify_only_ext(pb, be, &parent_addr, TOMBSTONE_INCLUDED, &txn); } - if (NULL != parent) { + if (parent) { int isglue; size_t haschildren = 0; int op = PARENTUPDATE_DEL; @@ -393,11 +416,11 @@ ldbm_back_delete( Slapi_PBlock *pb ) } else if (delete_tombstone_entry) { op |= PARENTUPDATE_DELETE_TOMBSTONE; } - retval = parent_update_on_childchange(&parent_modify_c, - op, &haschildren); + retval = parent_update_on_childchange(&parent_modify_c, op, &haschildren); /* The modify context now contains info needed later */ if (0 != retval) { ldap_result_code= LDAP_OPERATIONS_ERROR; + slapi_sdn_done(&parentsdn); goto error_return; } @@ -426,10 +449,18 @@ ldbm_back_delete( Slapi_PBlock *pb ) */ const char *childuniqueid= slapi_entry_get_uniqueid(e->ep_entry); const char *parentuniqueid= NULL; - char *tombstone_dn = compute_entry_tombstone_dn(slapi_entry_get_dn(e->ep_entry), - childuniqueid); + char *edn = slapi_entry_get_dn(e->ep_entry); + char *tombstone_dn; Slapi_Value *tomb_value; + if (slapi_is_special_rdn(edn, RDN_IS_TOMBSTONE)) { + LDAPDebug1Arg(LDAP_DEBUG_ANY, "Turning a tombstone into a tombstone! \"%s\"\n", edn); + ldap_result_code= LDAP_OPERATIONS_ERROR; + retval = -1; + goto error_return; + } + tombstone_dn = compute_entry_tombstone_dn(edn, childuniqueid); + slapi_sdn_set_ndn_byval(&nscpEntrySDN, slapi_sdn_get_ndn(slapi_entry_get_sdn(e->ep_entry))); /* Copy the entry unique_id for URP conflict checking */ @@ -471,6 +502,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if ( (original_tombstone = backentry_dup( tombstone )) == NULL ) { ldap_result_code= LDAP_OPERATIONS_ERROR; + retval = -1; goto error_return; } } @@ -505,6 +537,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (tombstone_in_cache) { CACHE_REMOVE(&inst->inst_cache, tombstone); CACHE_RETURN(&inst->inst_cache, &tombstone); + tombstone = NULL; tombstone_in_cache = 0; } else { backentry_free(&tombstone); @@ -515,13 +548,6 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } } - - /* reset original entry in cache */ - if (!e_in_cache) { - CACHE_ADD(&inst->inst_cache, e, NULL); - e_in_cache = 1; - } - if (ruv_c_init) { /* reset the ruv txn stuff */ modify_term(&ruv_c, be); @@ -615,41 +641,85 @@ ldbm_back_delete( Slapi_PBlock *pb ) * tentatively for now, then cache_add again when the original * entry is removed from the cache. */ - if (cache_add_tentative( &inst->inst_cache, tombstone, NULL) == 0) { + retval = cache_add_tentative(&inst->inst_cache, tombstone, NULL); + if (0 == retval) { tombstone_in_cache = 1; - } else if (!(tombstone->ep_state & ENTRY_STATE_NOTINCACHE)) { - LDAPDebug1Arg(LDAP_DEBUG_CACHE, - "id2entry_add tombstone (%s) is in cache\n", - slapi_entry_get_dn(tombstone->ep_entry)); - tombstone_in_cache = 1; + } else { + LDAPDebug2Args(LDAP_DEBUG_ANY, + "tombstone entry %s failed to add to the cache: %d\n", + slapi_entry_get_dn(tombstone->ep_entry), retval); + tombstone_in_cache = 0; + if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; + DEL_SET_ERROR(ldap_result_code, + LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; } retval = id2entry_add( be, tombstone, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 1 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete 1 DB_LOCK_DEADLOCK\n", 0, 0, 0 ); /* Abort and re-try */ continue; } - if (0 != retval) { + if (retval) { LDAPDebug( LDAP_DEBUG_ANY, "id2entry_add failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } - if (cache_replace( &inst->inst_cache, e, tombstone ) != 0 ) { - LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "ldbm_back_delete cache_replace failed\n"); - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); - retval= -1; - goto error_return; + if (tombstone_in_cache) { + retval = cache_replace(&inst->inst_cache, e, tombstone); + if (retval) { + LDAPDebug(LDAP_DEBUG_CACHE, "ldbm_back_delete: cache_replace failed (%d): %s --> %s\n", + retval, slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn(tombstone->ep_entry)); + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } } else { - e_in_cache = 0; /* e un-cached */ + struct backentry *imposter = NULL; + retval = CACHE_ADD(&inst->inst_cache, tombstone, &imposter); + if (retval > 0) { + if (imposter) { + /* + * The same tombstone entry (different Slapi_Entry) is already + * generated and set to cache. Back off. + */ + CACHE_RETURN(&inst->inst_cache, &imposter); + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: same DN tombstone in cache: %s\n", + slapi_entry_get_dn(tombstone->ep_entry)); + } else { + /* + * The same tombstone entry (same Slapi_Entry) is being created. + * Something is wrong. We should clean it up from the cache, + * and back off. + */ + tombstone_in_cache = 1; + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: same tombstone in cache: %s\n", + slapi_entry_get_dn(tombstone->ep_entry)); + } + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } else if (retval < 0) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_delete: cache add: Add %s failed.\n", + slapi_entry_get_dn(tombstone->ep_entry)); + /* Complete add error */ + retval= -1; + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + goto error_return; + } } - /* tombstone was already added to the cache via cache_add_tentative (to reserve its spot in the cache) - and/or id2entry_add - so it already had one refcount - cache_replace adds another refcount - - drop the extra ref added by cache_replace */ - CACHE_RETURN( &inst->inst_cache, &tombstone ); + if (tombstone_in_cache) { + /* tombstone was already added to the cache via cache_add_tentative (to reserve its spot in the cache) + and/or id2entry_add - so it already had one refcount - cache_replace adds another refcount - + drop the extra ref added by cache_replace */ + CACHE_RETURN( &inst->inst_cache, &tombstone ); + } + tombstone_in_cache = 1; } else { @@ -657,12 +727,12 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = id2entry_delete( be, e, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 2 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete 2 DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } if (retval != 0 ) { - if (retval == DB_RUNRECOVERY || + if (retval == DB_RUNRECOVERY || LDBM_OS_ERR_IS_DISKFULL(retval)) { disk_full = 1; } @@ -680,12 +750,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = index_addordel_entry( be, e, addordel_flags, &txn ); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete 1 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete 1 DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } - if (retval != 0) { - LDAPDebug( LDAP_DEBUG_TRACE, "index_del_entry failed\n", 0, 0, 0 ); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, "index_del_entry(%s, 0x%x) failed (%d)\n", + slapi_entry_get_dn(e->ep_entry), addordel_flags, retval); DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } @@ -700,14 +771,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) SLAPI_ATTR_VALUE_TOMBSTONE, tombstone->ep_id,BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_VALUE_TOMBSTONE, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_VALUE_TOMBSTONE, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -720,14 +791,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_entry_get_uniqueid(tombstone->ep_entry), tombstone->ep_id,BE_INDEX_ADD,&txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_UNIQUEID, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_UNIQUEID, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -740,14 +811,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_sdn_get_ndn(&nscpEntrySDN), tombstone->ep_id, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_NSCP_ENTRYDN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_NSCP_ENTRYDN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -764,14 +835,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) entryusn_str, tombstone->ep_id, BE_INDEX_ADD, &txn); slapi_ch_free_string(&entryusn_str); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (adding %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_ENTRYUSN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (adding %s) failed, err=%d %s\n", SLAPI_ATTR_ENTRYUSN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -795,13 +866,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) svals, NULL, e->ep_id, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (updating " LDBM_PARENTID_STR ") DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if ( retval ) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", LDBM_PARENTID_STR, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -811,15 +882,16 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto error_return; } } +#if 0 /* The entryrdn element is already deleted in the index_addordel_entry */ retval = entryrdn_index_entry(be, e, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (deleting entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, + if (retval) { + LDAPDebug2Args( LDAP_DEBUG_ANY, "delete (deleting entryrdn) failed, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -828,21 +900,22 @@ ldbm_back_delete( Slapi_PBlock *pb ) LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } +#endif retval = entryrdn_index_entry(be, tombstone, BE_INDEX_ADD, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "adding (adding tombstone entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } - if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, - "adding (adding tombstone entryrdn) failed, err=%d %s\n", - retval, - (msg = dblayer_strerror( retval )) ? msg : "" ); + if (retval) { + LDAPDebug(LDAP_DEBUG_ANY, + "adding (adding tombstone entryrdn %s) failed, err=%d %s\n", + slapi_entry_get_dn(tombstone->ep_entry), + retval, (msg = dblayer_strerror( retval )) ? msg : "" ); + if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - DEL_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + DEL_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } } @@ -859,14 +932,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) SLAPI_ATTR_VALUE_TOMBSTONE, e->ep_id, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_VALUE_TOMBSTONE, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_VALUE_TOMBSTONE, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -879,14 +952,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) slapi_entry_get_uniqueid(e->ep_entry), e->ep_id, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_UNIQUEID, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_UNIQUEID, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -903,14 +976,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) nscpedn, e->ep_id, BE_INDEX_DEL, &txn); slapi_ch_free((void **)&nscpedn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_NSCP_ENTRYDN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_NSCP_ENTRYDN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -929,14 +1002,14 @@ ldbm_back_delete( Slapi_PBlock *pb ) BE_INDEX_DEL|BE_INDEX_EQUALITY, &txn); slapi_ch_free_string(&entryusn_str); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete (deleting %s) DB_LOCK_DEADLOCK\n", SLAPI_ATTR_ENTRYUSN, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, + LDAPDebug( LDAP_DEBUG_ANY, "delete (deleting %s) failed, err=%d %s\n", SLAPI_ATTR_ENTRYUSN, retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -950,13 +1023,13 @@ ldbm_back_delete( Slapi_PBlock *pb ) { retval = entryrdn_index_entry(be, e, BE_INDEX_DEL, &txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug0Args( LDAP_DEBUG_ARGS, + LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "delete (deleting entryrdn) DB_LOCK_DEADLOCK\n"); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug2Args( LDAP_DEBUG_TRACE, + LDAPDebug2Args( LDAP_DEBUG_ANY, "delete (deleting entryrdn) failed, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "" ); @@ -973,12 +1046,12 @@ ldbm_back_delete( Slapi_PBlock *pb ) retval = modify_update_all(be,pb,&parent_modify_c,&txn); if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "del 4 DEADLOCK\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "del 4 DEADLOCK\n", 0, 0, 0 ); /* Retry txn */ continue; } if (0 != retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "delete 3 BAD, err=%d %s\n", + LDAPDebug( LDAP_DEBUG_ANY, "delete 3 BAD, err=%d %s\n", retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; DEL_SET_ERROR(ldap_result_code, @@ -996,7 +1069,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) if (DB_LOCK_DEADLOCK == retval) { - LDAPDebug( LDAP_DEBUG_ARGS, "delete DEADLOCK vlv_update_index\n", 0, 0, 0 ); + LDAPDebug( LDAP_DEBUG_BACKLDBM, "delete DEADLOCK vlv_update_index\n", 0, 0, 0 ); /* Retry txn */ continue; } @@ -1087,14 +1160,24 @@ ldbm_back_delete( Slapi_PBlock *pb ) /* delete from cache and clean up */ if (e) { + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, delete the tombstone dn from the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); + if (bdn) { /* in the dncache, remove it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } if (e_in_cache) { CACHE_REMOVE(&inst->inst_cache, e); + cache_unlock_entry(&inst->inst_cache, e); + CACHE_RETURN(&inst->inst_cache, &e); + } else { + cache_unlock_entry(&inst->inst_cache, e); } - cache_unlock_entry(&inst->inst_cache, e); - CACHE_RETURN(&inst->inst_cache, &e); e = NULL; } - + if (ruv_c_init) { if (modify_switch_entries(&ruv_c, be) != 0 ) { ldap_result_code= LDAP_OPERATIONS_ERROR; @@ -1115,16 +1198,37 @@ ldbm_back_delete( Slapi_PBlock *pb ) goto common_return; error_return: - if (tombstone_in_cache) - { - CACHE_REMOVE( &inst->inst_cache, tombstone ); - CACHE_RETURN( &inst->inst_cache, &tombstone ); - tombstone = NULL; - tombstone_in_cache = 0; + if (tombstone) { + if (entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, tombstone->ep_id); + if (bdn) { /* already in the dncache. Delete it. */ + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + if (tombstone_in_cache) { /* successfully replaced */ + CACHE_REMOVE( &inst->inst_cache, tombstone ); + CACHE_RETURN( &inst->inst_cache, &tombstone ); + tombstone = NULL; + tombstone_in_cache = 0; + } else { + backentry_free( &tombstone ); + } } - else - { - backentry_free( &tombstone ); + + /* Need to return to cache after post op plugins are called */ + if (e) { + if (e_in_cache) { + if (remove_e_from_cache) { + /* The entry is already transformed to a tombstone. */ + CACHE_REMOVE( &inst->inst_cache, e ); + } + cache_unlock_entry( &inst->inst_cache, e ); + CACHE_RETURN( &inst->inst_cache, &e ); + } else { + cache_unlock_entry( &inst->inst_cache, e ); + } } if (retval == DB_RUNRECOVERY) { @@ -1184,15 +1288,32 @@ common_return: for the post op plugins */ slapi_pblock_set( pb, SLAPI_DELETE_BEPREOP_ENTRY, orig_entry ); } - if (tombstone_in_cache) - { - CACHE_RETURN( &inst->inst_cache, &tombstone ); - tombstone = NULL; - tombstone_in_cache = 0; - } - else - { - backentry_free( &tombstone ); + if (tombstone) { + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, tombstone->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *tombstonesdn = slapi_sdn_dup(slapi_entry_get_sdn(tombstone->ep_entry)); + if (tombstonesdn) { + bdn = backdn_init(tombstonesdn, tombstone->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_delete", + "set %s to dn cache\n", slapi_sdn_get_dn(tombstonesdn)); + CACHE_RETURN(&inst->inst_dncache, &bdn); + } + } + } + } + if (tombstone_in_cache) { /* successfully replaced */ + CACHE_RETURN( &inst->inst_cache, &tombstone ); + tombstone = NULL; + tombstone_in_cache = 0; + } else { + backentry_free( &tombstone ); + } } /* result code could be used in the bepost plugin functions. */ @@ -1205,14 +1326,6 @@ common_return: plugin_call_plugins (pb, SLAPI_PLUGIN_BE_POST_DELETE_FN); } - /* Need to return to cache after post op plugins are called */ - if (retval) { /* error case */ - if (e) { - cache_unlock_entry( &inst->inst_cache, e ); - CACHE_RETURN( &inst->inst_cache, &e ); - } - } - if (ruv_c_init) { modify_term(&ruv_c, be); } @@ -1222,7 +1335,7 @@ diskfull_return: { slapi_send_ldap_result( pb, ldap_result_code, NULL, ldap_result_message, 0, NULL ); } - modify_term(&parent_modify_c,be); + modify_term(&parent_modify_c, be); if(dblock_acquired) { dblayer_unlock_backend(be); diff --git a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c index 887f74b..6426fb7 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_entryrdn.c @@ -387,7 +387,7 @@ entryrdn_index_read_ext(backend *be, rc = slapi_rdn_init_all_sdn(&srdn, sdn); if (rc < 0) { - slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, + slapi_log_error(SLAPI_LOG_BACKLDBM, ENTRYRDN_TAG, "entryrdn_index_read: Param error: Failed to convert " "%s to Slapi_RDN\n", slapi_sdn_get_dn(sdn)); rc = LDAP_INVALID_DN_SYNTAX; @@ -501,7 +501,8 @@ entryrdn_rename_subtree(backend *be, Slapi_RDN *newsrdn, /* new rdn */ const Slapi_DN *newsupsdn, /* new superior dn */ ID id, - back_txn *txn) + back_txn *txn, + int flags) { int rc = -1; struct attrinfo *ai = NULL; @@ -543,7 +544,7 @@ entryrdn_rename_subtree(backend *be, goto bail; } - rc = slapi_rdn_init_all_sdn(&oldsrdn, oldsdn); + rc = slapi_rdn_init_all_sdn_ext(&oldsrdn, oldsdn, flags); if (rc < 0) { slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, "entryrdn_rename_subtree: Failed to convert olddn " @@ -968,7 +969,8 @@ entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, - back_txn *txn) + back_txn *txn, + int flags) { int rc = -1; struct attrinfo *ai = NULL; @@ -1000,7 +1002,7 @@ entryrdn_get_subordinates(backend *be, goto bail; } - rc = slapi_rdn_init_all_sdn(&srdn, sdn); + rc = slapi_rdn_init_all_sdn_ext(&srdn, sdn, flags); if (rc) { if (rc < 0) { slapi_log_error(SLAPI_LOG_FATAL, ENTRYRDN_TAG, diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modify.c b/ldap/servers/slapd/back-ldbm/ldbm_modify.c index be47fcc..f7ebac1 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modify.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modify.c @@ -116,8 +116,14 @@ int modify_switch_entries(modify_context *mc,backend *be) ldbm_instance *inst = (ldbm_instance *) be->be_instance_info; int ret = 0; if (mc->old_entry!=NULL && mc->new_entry!=NULL) { - ret = cache_replace(&(inst->inst_cache), mc->old_entry, mc->new_entry); - if (ret == 0) mc->new_entry_in_cache = 1; + ret = cache_replace(&(inst->inst_cache), mc->old_entry, mc->new_entry); + if (ret == 0) { + mc->new_entry_in_cache = 1; + } else { + LDAPDebug(LDAP_DEBUG_CACHE, "modify_switch_entries: replacing %s with %s failed (%d)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), + slapi_entry_get_dn(mc->new_entry->ep_entry), ret); + } } return ret; } @@ -154,6 +160,10 @@ modify_unswitch_entries(modify_context *mc,backend *be) CACHE_RETURN( &(inst->inst_cache), &(mc->old_entry) ); mc->new_entry_in_cache = 1; mc->old_entry = NULL; + } else { + LDAPDebug(LDAP_DEBUG_CACHE, "modify_unswitch_entries: replacing %s with %s failed (%d)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), + slapi_entry_get_dn(mc->new_entry->ep_entry), ret); } } @@ -722,6 +732,12 @@ ldbm_back_modify( Slapi_PBlock *pb ) MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; } + /* e uncached */ + /* we must return both e (which has been deleted) and new entry ec */ + /* cache_replace removes e from the caches */ + cache_unlock_entry( &inst->inst_cache, e ); + CACHE_RETURN( &inst->inst_cache, &e ); + /* lock new entry in cache to prevent usage until we are complete */ cache_lock_entry( &inst->inst_cache, ec ); ec_in_cache = 1; @@ -732,10 +748,6 @@ ldbm_back_modify( Slapi_PBlock *pb ) /* invalidate virtual cache */ ec->ep_entry->e_virtual_watermark = 0; - /* we must return both e (which has been deleted) and new entry ec */ - /* cache_replace removes e from the caches */ - cache_unlock_entry( &inst->inst_cache, e ); - CACHE_RETURN( &inst->inst_cache, &e ); /* * LP Fix of crash when the commit will fail: * If the commit fail, the common error path will @@ -833,8 +845,10 @@ error_return: CACHE_REMOVE( &inst->inst_cache, ec ); /* if ec was in cache, e was not - add back e */ if (e) { - CACHE_ADD( &inst->inst_cache, e, NULL ); - cache_lock_entry( &inst->inst_cache, e ); + if (CACHE_ADD( &inst->inst_cache, e, NULL )) { + LDAPDebug1Arg( LDAP_DEBUG_CACHE, "ldbm_modify: CACHE_ADD %s failed\n", + slapi_entry_get_dn(e->ep_entry)); + } } } @@ -849,11 +863,12 @@ common_return: else { backentry_free(&ec); - } - - if (e!=NULL) { - cache_unlock_entry( &inst->inst_cache, e); - CACHE_RETURN( &inst->inst_cache, &e); + /* if ec was not in cache, cache_replace was not done. + * i.e., e was not unlocked. */ + if (e) { + cache_unlock_entry( &inst->inst_cache, e); + CACHE_RETURN( &inst->inst_cache, &e); + } } /* result code could be used in the bepost plugin functions. */ diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c index c539463..4e5ce45 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modrdn.c @@ -45,10 +45,10 @@ #include "back-ldbm.h" -static const char *moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn); +static const char *moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn, int is_tombstone); static void moddn_unlock_and_return_entry(backend *be,struct backentry **targetentry); static int moddn_newrdn_mods(Slapi_PBlock *pb, const char *olddn, struct backentry *ec, Slapi_Mods *smods_wsi, int is_repl_op); -static IDList *moddn_get_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, struct backentry *parententry, Slapi_DN *parentdn, struct backentry ***child_entries, struct backdn ***child_dns); +static IDList *moddn_get_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, struct backentry *parententry, Slapi_DN *parentdn, struct backentry ***child_entries, struct backdn ***child_dns, int is_resurect_operation); static int moddn_rename_children(back_txn *ptxn, Slapi_PBlock *pb, backend *be, IDList *children, Slapi_DN *dn_parentdn, Slapi_DN *dn_newsuperiordn, struct backentry *child_entries[]); static int modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbminfo *li, struct backentry *e, struct backentry **ec, Slapi_Mods *smods1, Slapi_Mods *smods2, Slapi_Mods *smods3, int *e_in_cache, int *ec_in_cache); static void mods_remove_nsuniqueid(Slapi_Mods *smods); @@ -111,6 +111,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) int dblock_acquired= 0; int is_replicated_operation= 0; int is_fixup_operation = 0; + int is_resurect_operation = 0; + int is_tombstone = 0; entry_address new_addr; entry_address *old_addr; entry_address oldparent_addr; @@ -136,6 +138,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_IS_REPLICATED_OPERATION, &is_replicated_operation ); is_ruv = operation_is_flag_set(operation, OP_FLAG_REPL_RUV); is_fixup_operation = operation_is_flag_set(operation, OP_FLAG_REPL_FIXUP); + is_resurect_operation = operation_is_flag_set(operation,OP_FLAG_RESURECT_ENTRY); + is_tombstone = operation_is_flag_set(operation,OP_FLAG_TOMBSTONE_ENTRY); /* tombstone_to_glue on parent entry*/ if (NULL == sdn) { slapi_send_ldap_result( pb, LDAP_INVALID_DN_SYNTAX, NULL, @@ -168,7 +172,13 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_pblock_get( pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, &dn_newsuperiordn ); slapi_sdn_init_normdn_byref(&dn_newrdn, newrdn); /* slapi_sdn_init_normdn_byref(&dn_newsuperiordn, newsuperiordn); */ - slapi_sdn_get_parent(sdn, &dn_parentdn); + if (is_resurect_operation) { + /* no need to free this pdn. */ + const char *pdn = slapi_dn_find_parent_ext(slapi_sdn_get_dn(sdn), is_resurect_operation); + slapi_sdn_set_dn_byval(&dn_parentdn, pdn); + } else { + slapi_sdn_get_parent(sdn, &dn_parentdn); + } } /* if old and new superior are equals, newsuperior should not be set @@ -235,7 +245,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) * it's passed to slapi_sdn_init_normdn_byref */ slapi_pblock_get(pb, SLAPI_MODRDN_NEWRDN, &newrdn); slapi_sdn_init_normdn_byref(&dn_newrdn, newrdn); - newdn= moddn_get_newdn(pb,sdn, &dn_newrdn, dn_newsuperiordn); + newdn = moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn, is_tombstone); slapi_sdn_set_dn_passin(&dn_newdn,newdn); new_addr.sdn = &dn_newdn; new_addr.udn = NULL; @@ -353,11 +363,14 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } e_in_cache = 1; /* e is in the cache and locked */ if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) ) { - ldap_result_code = LDAP_UNWILLING_TO_PERFORM; - ldap_result_message = "Operation not allowed on tombstone entry."; - slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", - "Attempt to rename a tombstone entry %s\n", slapi_sdn_get_dn(slapi_entry_get_sdn_const( e->ep_entry ))); - goto error_return; + if (!is_resurect_operation) { + ldap_result_code = LDAP_UNWILLING_TO_PERFORM; + ldap_result_message = "Operation not allowed on tombstone entry."; + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "Attempt to rename a tombstone entry %s\n", + slapi_sdn_get_dn(slapi_entry_get_sdn_const( e->ep_entry ))); + goto error_return; + } } /* Check that an entry with the same DN doesn't already exist. */ { @@ -375,7 +388,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* Fetch and lock the parent of the entry that is moving */ oldparent_addr.sdn = &dn_parentdn; - oldparent_addr.uniqueid = NULL; + if (is_resurect_operation) { + oldparent_addr.uniqueid = operation->o_params.p.p_modrdn.modrdn_newsuperior_address.uniqueid; + } else { + oldparent_addr.uniqueid = NULL; + } parententry = find_entry2modify_only( pb, be, &oldparent_addr, &txn ); modify_init(&parent_modify_context,parententry); @@ -383,7 +400,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) if(slapi_sdn_get_ndn(dn_newsuperiordn) != NULL) { slapi_pblock_get (pb, SLAPI_MODRDN_NEWSUPERIOR_ADDRESS, &newsuperior_addr); + if (is_resurect_operation) { + newsuperior_addr->uniqueid = slapi_entry_attr_get_charptr(e->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + } newparententry = find_entry2modify_only( pb, be, newsuperior_addr, &txn ); + slapi_ch_free_string(&newsuperior_addr->uniqueid); modify_init(&newparent_modify_context,newparententry); } @@ -426,8 +447,7 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } slapi_pblock_set( pb, SLAPI_MODRDN_NEWSUPERIOR_SDN, dn_newsuperiordn ); } - slapi_sdn_set_dn_passin(&dn_newdn, - moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn)); + slapi_sdn_set_dn_passin(&dn_newdn, moddn_get_newdn(pb, sdn, &dn_newrdn, dn_newsuperiordn, is_tombstone)); /* Check that we're allowed to add an entry below the new superior */ if ( newparententry == NULL ) @@ -528,6 +548,57 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_rdn_done(&srdn); } + if(is_resurect_operation) { + slapi_log_error(SLAPI_LOG_REPL, "ldbm_back_modrdn", + "Resurrecting an entry %s\n", slapi_entry_get_dn(ec->ep_entry)); + slapi_entry_attr_delete(ec->ep_entry, SLAPI_ATTR_VALUE_PARENT_UNIQUEID); + slapi_entry_delete_string(ec->ep_entry, SLAPI_ATTR_OBJECTCLASS, SLAPI_ATTR_VALUE_TOMBSTONE); + /* Now also remove the nscpEntryDN */ + if (slapi_entry_attr_delete(ec->ep_entry, SLAPI_ATTR_NSCP_ENTRYDN) != 0){ + LDAPDebug(LDAP_DEBUG_REPL, "Resurrection of %s - Couldn't remove %s\n", + slapi_entry_get_dn(ec->ep_entry), SLAPI_ATTR_NSCP_ENTRYDN, 0); + } + + /* Set the reason (this is only a reason why modrdn is needed for resurrection) */ + slapi_entry_add_string(ec->ep_entry, "nsds5ReplConflict", "deletedEntryHasChildren"); + + /* Clear the Tombstone Flag in the entry */ + slapi_entry_clear_flag(ec->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE); + + /* make sure the objectclass + - does not contain any duplicate values + - has CSNs for the new values we added + */ + { + Slapi_Attr *sa = NULL; + Slapi_Value sv; + const struct berval *svbv = NULL; + + /* add the extensibleobject objectclass with csn if not present */ + slapi_entry_attr_find(ec->ep_entry, SLAPI_ATTR_OBJECTCLASS, &sa); + slapi_value_init_string(&sv, "extensibleobject"); + svbv = slapi_value_get_berval(&sv); + if (slapi_attr_value_find(sa, svbv)) { /* not found, so add it */ + if (opcsn) { + value_update_csn(&sv, CSN_TYPE_VALUE_UPDATED, opcsn); + } + slapi_attr_add_value(sa, &sv); + } + value_done(&sv); + + /* add the glue objectclass with csn if not present */ + slapi_value_init_string(&sv, "glue"); + svbv = slapi_value_get_berval(&sv); + if (slapi_attr_value_find(sa, svbv)) { /* not found, so add it */ + if (opcsn) { + value_update_csn(&sv, CSN_TYPE_VALUE_UPDATED, opcsn); + } + slapi_attr_add_value(sa, &sv); + } + value_done(&sv); + } + } + /* create it in the cache - prevents others from creating it */ if (( cache_add_tentative( &inst->inst_cache, ec, NULL ) != 0 ) ) { ec_in_cache = 0; /* not in cache */ @@ -537,6 +608,10 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* somebody must've created it between dn2entry() and here */ /* JCMREPL - Hmm... we can't permit this to happen...? */ ldap_result_code= LDAP_ALREADY_EXISTS; + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "cache_add_tentative failed: %s\n", slapi_entry_get_dn(ec->ep_entry)); + } goto error_return; } /* so if the old dn is the same as the new dn, the entry will not be cached @@ -584,8 +659,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } } - slapi_pblock_get( pb, SLAPI_MODIFY_MODS, &mods ); - slapi_mods_init_byref(&smods_operation_wsi,mods); + slapi_pblock_get( pb, SLAPI_MODIFY_MODS, &mods ); + slapi_mods_init_byref(&smods_operation_wsi,mods); /* * We are about to pass the last abandon test, so from now on we are @@ -675,10 +750,8 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* * Update the subordinate count of the parents to reflect the moved child. */ - if ( parententry!=NULL ) - { - retval = parent_update_on_childchange(&parent_modify_context, - PARENTUPDATE_DEL, NULL); + if (parententry) { + retval = parent_update_on_childchange(&parent_modify_context, PARENTUPDATE_DEL, NULL); /* The parent modify context now contains info needed later */ if (0 != retval) { @@ -696,27 +769,64 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } } } + /* is_resurect_operation case, there's no new superior. Just rename. */ + if (is_resurect_operation && parententry) { + retval = parent_update_on_childchange(&parent_modify_context, PARENTUPDATE_RESURECT, NULL); + if (retval) { + LDAPDebug(LDAP_DEBUG_BACKLDBM, "parent_update_on_childchange parent %s of %s failed, rc=%d\n", + slapi_entry_get_dn_const(parent_modify_context.old_entry->ep_entry), + slapi_entry_get_dn_const(ec->ep_entry), retval); + goto error_return; + } + } /* - * If the entry has children then we're going to have to rename them all. + * If the entry has children including tombstones, + * then we're going to have to rename them all. */ - if (slapi_entry_has_children( e->ep_entry )) - { + if (slapi_entry_has_children_ext(e->ep_entry, 1)) { /* JCM - This is where the subtree lock will appear */ if (entryrdn_get_switch()) /* subtree-rename: on */ { + if (is_resurect_operation) { +#if defined(DEBUG) + /* Get the present value of the subcount attr, or 0 if not present */ + Slapi_Attr *read_attr = NULL; + int sub_count = -1; + if (0 == slapi_entry_attr_find(parent_modify_context.old_entry->ep_entry, + "numsubordinates", &read_attr)) { + /* decode the value */ + Slapi_Value *sval; + slapi_attr_first_value(read_attr, &sval); + if (sval) { + const struct berval *bval = slapi_value_get_berval(sval); + if(bval) { + sub_count = atol(bval->bv_val); + } + } + } + LDAPDebug(LDAP_DEBUG_ANY, "parent_update_on_childchange parent %s of %s numsub=%d\n", + slapi_entry_get_dn_const(parent_modify_context.old_entry->ep_entry), + slapi_entry_get_dn_const(e->ep_entry), sub_count); +#endif + slapi_log_error(SLAPI_LOG_BACKLDBM, "ldbm_back_modrdn", + "%s has children\n", slapi_entry_get_dn(e->ep_entry)); + } children = moddn_get_children(&txn, pb, be, e, sdn, - &child_entries, &child_dns); + &child_entries, &child_dns, is_resurect_operation); } else { children = moddn_get_children(&txn, pb, be, e, sdn, - &child_entries, NULL); + &child_entries, NULL, 0); } /* JCM - Shouldn't we perform an access control check on all the children. */ /* JCMREPL - But, the replication client has total rights over its subtree, so no access check needed. */ /* JCM - A subtree move could break ACIs, static groups, and dynamic groups. */ + } else if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "%s has NO children\n", slapi_entry_get_dn(e->ep_entry)); } /* @@ -777,7 +887,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) } /* make sure the original entry is back in the cache if it was removed */ if (!e_in_cache) { - CACHE_ADD(&inst->inst_cache, e, NULL); + if (CACHE_ADD(&inst->inst_cache, e, NULL)) { + LDAPDebug1Arg(LDAP_DEBUG_CACHE, + "ldbm_back_modrdn: CACHE_ADD %s to cache failed\n", + slapi_entry_get_dn_const(e->ep_entry)); + } e_in_cache = 1; } slapi_pblock_get( pb, SLAPI_MODRDN_EXISTING_ENTRY, &ent ); @@ -820,11 +934,11 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) goto error_return; } - if (ruv_c_init) { - /* reset the ruv txn stuff */ - modify_term(&ruv_c, be); - ruv_c_init = 0; - } + if (ruv_c_init) { + /* reset the ruv txn stuff */ + modify_term(&ruv_c, be); + ruv_c_init = 0; + } /* We're re-trying */ LDAPDebug0Args(LDAP_DEBUG_BACKLDBM, "Modrdn Retrying Transaction\n"); @@ -871,10 +985,10 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) /* Retry txn */ continue; } - if (retval != 0 ) + if (retval) { - LDAPDebug( LDAP_DEBUG_TRACE, "modrdn_rename_entry_update_indexes failed, err=%d %s\n", - retval, (msg = dblayer_strerror( retval )) ? msg : "", 0 ); + LDAPDebug(LDAP_DEBUG_ANY, "modrdn_rename_entry_update_indexes %s --> %s failed, err=%d\n", + slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn(ec->ep_entry), retval); if (LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); goto error_return; @@ -994,14 +1108,17 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) slapi_rdn_init_sdn(&newsrdn, (const Slapi_DN *)&dn_newdn); retval = entryrdn_rename_subtree(be, (const Slapi_DN *)sdn, &newsrdn, (const Slapi_DN *)dn_newsuperiordn, - e->ep_id, &txn); + e->ep_id, &txn, is_tombstone); slapi_rdn_done(&newsrdn); if (retval != 0) { if (retval == DB_LOCK_DEADLOCK) continue; if (retval == DB_RUNRECOVERY || LDBM_OS_ERR_IS_DISKFULL(retval)) disk_full = 1; - MOD_SET_ERROR(ldap_result_code, - LDAP_OPERATIONS_ERROR, retry_count); + MOD_SET_ERROR(ldap_result_code, LDAP_OPERATIONS_ERROR, retry_count); + slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modrdn", + "entryrdn_rename_subtree failed (%d); dn: %s, newsrdn: %s, dn_newsuperiordn: %s\n", + retval, slapi_sdn_get_dn(sdn), slapi_rdn_get_rdn(&newsrdn), + slapi_sdn_get_dn(dn_newsuperiordn)); goto error_return; } } @@ -1125,6 +1242,12 @@ ldbm_back_modrdn( Slapi_PBlock *pb ) * "renamed" dn is generated based upon the moved subtree. */ for (i = 0; child_entries[i] != NULL; i++) { + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "Calling cache remove & return %s (refcnt: %d)\n", + slapi_entry_get_dn(child_entries[i]->ep_entry), + child_entries[i]->ep_refcnt); + } CACHE_REMOVE( &inst->inst_cache, child_entries[i] ); cache_unlock_entry( &inst->inst_cache, child_entries[i] ); CACHE_RETURN( &inst->inst_cache, &child_entries[i] ); @@ -1168,12 +1291,6 @@ error_return: /* make sure caller doesn't attempt to free this */ slapi_pblock_set( pb, SLAPI_ENTRY_POST_OP, postentry ); } - if (e && entryrdn_get_switch()) - { - struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); - CACHE_REMOVE(&inst->inst_dncache, bdn); - CACHE_RETURN(&inst->inst_dncache, &bdn); - } if(children) { int i = 0; @@ -1270,9 +1387,32 @@ common_return: /* Free up the resource we don't need any more */ if (ec) { + if (is_resurect_operation) { + slapi_log_error(SLAPI_LOG_REPL, "ldbm_back_modrdn", + "Resurrecting an entry %s: result: %d, %d\n", + slapi_entry_get_dn(ec->ep_entry), ldap_result_code, retval); + } + if ((0 == retval) && entryrdn_get_switch()) { /* subtree-rename: on */ + /* since the op was successful, add the addingentry's dn to the dn cache */ + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, ec->ep_id); + if (bdn) { /* already in the dncache */ + CACHE_RETURN(&inst->inst_dncache, &bdn); + } else { /* not in the dncache yet */ + Slapi_DN *ecsdn = slapi_sdn_dup(slapi_entry_get_sdn(ec->ep_entry)); + if (ecsdn) { + bdn = backdn_init(ecsdn, ec->ep_id, 0); + if (bdn) { + CACHE_ADD( &inst->inst_dncache, bdn, NULL ); + CACHE_RETURN(&inst->inst_dncache, &bdn); + slapi_log_error(SLAPI_LOG_CACHE, "ldbm_back_modrdn", + "set %s to dn cache\n", slapi_sdn_get_dn(sdn)); + } + } + } + } /* remove the new entry from the cache if the op failed - otherwise, leave it in */ - if (ec_in_cache && retval) { + if (ec_in_cache && ec && retval) { CACHE_REMOVE( &inst->inst_cache, ec ); } if (ec_in_cache) { @@ -1286,9 +1426,10 @@ common_return: /* put e back in the cache if the modrdn failed */ if (e) { - if (!e_in_cache && retval) { - CACHE_ADD(&inst->inst_cache, e, NULL); - e_in_cache = 1; + if (entryrdn_get_switch() && (0 == retval)) { + struct backdn *bdn = dncache_find_id(&inst->inst_dncache, e->ep_id); + CACHE_REMOVE(&inst->inst_dncache, bdn); + CACHE_RETURN(&inst->inst_dncache, &bdn); } } @@ -1352,7 +1493,7 @@ common_return: * Work out what the new DN of the entry will be. */ static const char * -moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn) +moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi_DN *dn_newsuperiordn, int is_tombstone) { char *newdn; const char *newrdn= slapi_sdn_get_dn(dn_newrdn); @@ -1373,18 +1514,18 @@ moddn_get_newdn(Slapi_PBlock *pb, Slapi_DN *dn_olddn, Slapi_DN *dn_newrdn, Slapi else { /* construct the new dn */ - char *pdn; - const char *dn= slapi_sdn_get_dn(dn_olddn); - pdn = slapi_dn_beparent( pb, dn ); - if ( pdn != NULL ) - { - newdn= slapi_dn_plus_rdn(pdn, newrdn); /* JCM - Use Slapi_RDN */ - } - else - { - newdn= slapi_ch_strdup(newrdn); + const char *dn = slapi_sdn_get_dn((const Slapi_DN *)dn_olddn); + if (slapi_dn_isbesuffix(pb, dn)) { + newdn = slapi_ch_strdup(newrdn); + } else { + /* no need to free this pdn. */ + const char *pdn = slapi_dn_find_parent_ext(dn, is_tombstone); + if (pdn) { + newdn = slapi_dn_plus_rdn(pdn, newrdn); + } else { + newdn = slapi_ch_strdup(newrdn); + } } - slapi_ch_free( (void**)&pdn ); } return newdn; } @@ -1611,6 +1752,7 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm Slapi_Operation *operation; int is_ruv = 0; /* True if the current entry is RUV */ int orig_ec_in_cache = 0; + int cache_rc = 0; slapi_pblock_get( pb, SLAPI_BACKEND, &be ); slapi_pblock_get( pb, SLAPI_OPERATION, &operation ); @@ -1622,7 +1764,12 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm * Update the ID to Entry index. * Note that id2entry_add replaces the entry, so the Entry ID stays the same. */ - retval = id2entry_add( be, *ec, ptxn ); + retval = id2entry_add_ext(be, *ec, ptxn, 1, &cache_rc); + if (cache_rc) { + LDAPDebug(LDAP_DEBUG_CACHE, + "modrdn_rename_entry_update_indexes: adding %s failed to add to the cache (rc: %d, cache_rc: %d)\n", + slapi_entry_get_dn(e->ep_entry), retval, cache_rc); + } if (DB_LOCK_DEADLOCK == retval) { /* Retry txn */ @@ -1716,11 +1863,11 @@ modrdn_rename_entry_update_indexes(back_txn *ptxn, Slapi_PBlock *pb, struct ldbm } } if (cache_replace( &inst->inst_cache, e, *ec ) != 0 ) { - LDAPDebug0Args( LDAP_DEBUG_BACKLDBM, "modrdn_rename_entry_update_indexes cache_replace failed\n"); + LDAPDebug2Args(LDAP_DEBUG_CACHE, + "modrdn_rename_entry_update_indexes cache_replace %s -> %s failed\n", + slapi_entry_get_dn(e->ep_entry), slapi_entry_get_dn((*ec)->ep_entry)); retval= -1; goto error_return; - } else { - *e_in_cache = 0; /* e un-cached */ } if (orig_ec_in_cache) { /* ec was already added to the cache via cache_add_tentative (to reserve its spot in the cache) @@ -1936,7 +2083,8 @@ moddn_get_children(back_txn *ptxn, struct backentry *parententry, Slapi_DN *dn_parentdn, struct backentry ***child_entries, - struct backdn ***child_dns) + struct backdn ***child_dns, + int is_resurect_operation) { ldbm_instance *inst = (ldbm_instance *) be->be_instance_info; int err= 0; @@ -1964,7 +2112,7 @@ moddn_get_children(back_txn *ptxn, { err = entryrdn_get_subordinates(be, slapi_entry_get_sdn_const(parententry->ep_entry), - parententry->ep_id, &candidates, ptxn); + parententry->ep_id, &candidates, ptxn, is_resurect_operation); if (err) { LDAPDebug1Arg( LDAP_DEBUG_ANY, "moddn_get_children: " "entryrdn_get_subordinates returned %d\n", err); @@ -1985,6 +2133,12 @@ moddn_get_children(back_txn *ptxn, if (candidates!=NULL) { + Slapi_DN parentsdn = {0}; + if (is_resurect_operation) { + slapi_sdn_get_parent(dn_parentdn, &parentsdn); + dn_parentdn = &parentsdn; + } + sr_current = idl_iterator_init(candidates); result_idl= idl_alloc(candidates->b_nids); do @@ -2017,6 +2171,7 @@ moddn_get_children(back_txn *ptxn, } } while (id!=NOID); idl_free(candidates); + slapi_sdn_done(&parentsdn); } nids = result_idl ? result_idl->b_nids : 0; diff --git a/ldap/servers/slapd/back-ldbm/ldbm_search.c b/ldap/servers/slapd/back-ldbm/ldbm_search.c index a097307..746528e 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_search.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_search.c @@ -1164,7 +1164,7 @@ subtree_candidates( /* subtree-rename: on && no ancestorid */ *err = entryrdn_get_subordinates(be, slapi_entry_get_sdn_const(e->ep_entry), - e->ep_id, &descendants, &txn); + e->ep_id, &descendants, &txn, 0); idl_insert(&descendants, e->ep_id); candidates = idl_intersection(be, candidates, descendants); idl_free(tmp); diff --git a/ldap/servers/slapd/back-ldbm/ldif2ldbm.c b/ldap/servers/slapd/back-ldbm/ldif2ldbm.c index 47e0269..51ff62b 100644 --- a/ldap/servers/slapd/back-ldbm/ldif2ldbm.c +++ b/ldap/servers/slapd/back-ldbm/ldif2ldbm.c @@ -898,7 +898,7 @@ static IDList *ldbm_fetch_subtrees(backend *be, char **include, int *err) */ if (entryrdn_get_noancestorid()) { /* subtree-rename: on && no ancestorid */ - *err = entryrdn_get_subordinates(be, &sdn, id, &idl, txn); + *err = entryrdn_get_subordinates(be, &sdn, id, &idl, txn, 0); } else { *err = ldbm_ancestorid_read(be, txn, id, &idl); } diff --git a/ldap/servers/slapd/back-ldbm/parents.c b/ldap/servers/slapd/back-ldbm/parents.c index 7ca9f24..669eaef 100644 --- a/ldap/servers/slapd/back-ldbm/parents.c +++ b/ldap/servers/slapd/back-ldbm/parents.c @@ -58,11 +58,17 @@ char *tombstone_numsubordinates = LDBM_TOMBSTONE_NUMSUBORDINATES_STR; * The routine is allowed to modify the parent entry, and to return a set of * LDAPMods reflecting the changes it made. The LDAPMods array must be freed * by the called by calling ldap_free_mods(p,1) + * + * PARENTUPDATE_RESURECT == turning a tombstone into an entry + * tombstone_numsubordinates-- + * numsubordinates++ */ /* - * PARENTUPDATE_CREATE_TOMBSTONE: increment tombstone_numsubordinates + * PARENTUPDATE_CREATE_TOMBSTONE: turning an entry into a tombstone + * numsubordinates-- + * tombstone_numsubordinates++ * PARENTUPDATE_DELETE_TOMBSTONE: don't touch numsubordinates, and - * decrement tombstone_numsubordinates + * tombstone_numsubordinates-- */ int @@ -84,7 +90,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) op &= PARENTUPDATE_MASK; /* Check nobody is trying to use op == 3, it's not implemented yet */ - PR_ASSERT( (op == PARENTUPDATE_ADD) || (op == PARENTUPDATE_DEL)); + PR_ASSERT((op == PARENTUPDATE_ADD) || (op == PARENTUPDATE_DEL) || (op == PARENTUPDATE_RESURECT)); /* We want to invent a mods set to be passed to modify_apply_mods() */ @@ -114,16 +120,17 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) if (PARENTUPDATE_DELETE_TOMBSTONE != repl_op) { /* are we adding ? */ - if ( (PARENTUPDATE_ADD == op) && !already_present) { + if (((PARENTUPDATE_ADD == op) || (PARENTUPDATE_RESURECT == op)) && !already_present) { /* If so, and the parent entry does not already have a subcount * attribute, we need to add it */ mod_op = LDAP_MOD_ADD; } else if (PARENTUPDATE_DEL == op) { if (!already_present) { /* This means that something is wrong---deleting a child but no subcount present on parent */ - LDAPDebug0Args( LDAP_DEBUG_ANY, - "numsubordinates assertion failure\n" ); + LDAPDebug(LDAP_DEBUG_ANY, "Parent %s has no children. (op 0x%x, repl_op 0x%x)\n", + slapi_entry_get_dn(mc->old_entry->ep_entry), op, repl_op); slapi_mods_free(&smods); + PR_ASSERT(0); return -1; } else { if (current_sub_count == 1) { @@ -138,7 +145,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) } /* Now compute the new value */ - if (PARENTUPDATE_ADD == op) { + if ((PARENTUPDATE_ADD == op) || (PARENTUPDATE_RESURECT == op)) { current_sub_count++; } else { current_sub_count--; @@ -159,8 +166,8 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) /* tombstoneNumSubordinates is needed only when this is repl op * and a child is being deleted */ - if (repl_op && (PARENTUPDATE_DEL == op)) { - current_sub_count = LDAP_MAXINT; + current_sub_count = LDAP_MAXINT; + if ((repl_op && (PARENTUPDATE_DEL == op)) || (PARENTUPDATE_RESURECT == op)) { ret = slapi_entry_attr_find(mc->old_entry->ep_entry, tombstone_numsubordinates, &read_attr); if (0 == ret) { @@ -175,7 +182,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) } } - if (PARENTUPDATE_DELETE_TOMBSTONE == repl_op) { + if ((PARENTUPDATE_DELETE_TOMBSTONE == repl_op) || (PARENTUPDATE_RESURECT == op)) { /* deleting a tombstone entry: * reaping or manually deleting it */ if ((current_sub_count != LDAP_MAXINT) && @@ -187,9 +194,7 @@ parent_update_on_childchange(modify_context *mc,int op, size_t *new_sub_count ) tombstone_numsubordinates, strlen(value_buffer), value_buffer); } - } - - if (PARENTUPDATE_CREATE_TOMBSTONE == repl_op) { + } else if (PARENTUPDATE_CREATE_TOMBSTONE == repl_op) { /* creating a tombstone entry */ if (current_sub_count != LDAP_MAXINT) { current_sub_count++; diff --git a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h index 02b261b..f3c84a1 100644 --- a/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h +++ b/ldap/servers/slapd/back-ldbm/proto-back-ldbm.h @@ -190,6 +190,7 @@ struct backentry *dn2entry_ext(Slapi_Backend *be, const Slapi_DN *sdn, back_txn struct backentry *dn2entry_or_ancestor(Slapi_Backend *be, const Slapi_DN *sdn, Slapi_DN *ancestor, back_txn *txn, int *err); struct backentry *dn2ancestor(Slapi_Backend *be,const Slapi_DN *sdn,Slapi_DN *ancestordn,back_txn *txn,int *err); int get_copy_of_entry(Slapi_PBlock *pb, const entry_address *addr, back_txn *txn, int plock_parameter, int must_exist); +int get_copy_of_entry_ext(Slapi_PBlock *pb, ID id, const entry_address *addr, back_txn *txn, int plock_parameter, int must_exist); void done_with_pblock_entry(Slapi_PBlock *pb, int plock_parameter); /* @@ -699,8 +700,8 @@ int entryrdn_index_entry(backend *be, struct backentry *e, int flags, back_txn * int entryrdn_index_read(backend *be, const Slapi_DN *sdn, ID *id, back_txn *txn); int entryrdn_index_read_ext(backend *be, const Slapi_DN *sdn, ID *id, int flags, back_txn *txn); -int entryrdn_rename_subtree(backend *be, const Slapi_DN *oldsdn, Slapi_RDN *newsrdn, const Slapi_DN *newsupsdn, ID id, back_txn *txn); -int entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, back_txn *txn); +int entryrdn_rename_subtree(backend *be, const Slapi_DN *oldsdn, Slapi_RDN *newsrdn, const Slapi_DN *newsupsdn, ID id, back_txn *txn, int flags); +int entryrdn_get_subordinates(backend *be, const Slapi_DN *sdn, ID id, IDList **subordinates, back_txn *txn, int flags); int entryrdn_lookup_dn(backend *be, const char *rdn, ID id, char **dn, back_txn *txn); int entryrdn_get_parent(backend *be, const char *rdn, ID id, char **prdn, ID *pid, back_txn *txn); #endif diff --git a/ldap/servers/slapd/entry.c b/ldap/servers/slapd/entry.c index 20ac8ea..8b7766e 100644 --- a/ldap/servers/slapd/entry.c +++ b/ldap/servers/slapd/entry.c @@ -3113,9 +3113,10 @@ slapi_entry_add_rdn_values( Slapi_Entry *e ) * Author/Modifier: RJP */ int -slapi_entry_has_children(const Slapi_Entry *entry) +slapi_entry_has_children_ext(const Slapi_Entry *entry, int include_tombstone) { Slapi_Attr *attr; + int count = 0; LDAPDebug( LDAP_DEBUG_TRACE, "=> slapi_has_children( %s )\n", slapi_entry_get_dn_const(entry), 0, 0); @@ -3125,15 +3126,34 @@ slapi_entry_has_children(const Slapi_Entry *entry) Slapi_Value *sval; slapi_attr_first_value( attr, &sval ); if(sval!=NULL) - { + { const struct berval *bval = slapi_value_get_berval( sval ); if(bval!=NULL) { /* The entry has the attribute, and it's non-zero */ - if (strcmp(bval->bv_val, "0") != 0) - { - LDAPDebug( LDAP_DEBUG_TRACE, "<= slapi_has_children 1\n", 0, 0, 0 ); - return(1); + count = strtol(bval->bv_val, (char **)NULL, 10); + if (count > 0) { + LDAPDebug1Arg( LDAP_DEBUG_TRACE, "<= slapi_has_children %d\n", count); + return count; + } + } + } + } + /*If the subordinatecount exists, and it's nonzero, then return 1.*/ + if (include_tombstone && (slapi_entry_attr_find( entry, "tombstonenumsubordinates", &attr) == 0)) + { + Slapi_Value *sval; + slapi_attr_first_value( attr, &sval ); + if(sval!=NULL) + { + const struct berval *bval = slapi_value_get_berval( sval ); + if(bval!=NULL) + { + /* The entry has the attribute, and it's non-zero */ + count = strtol(bval->bv_val, (char **)NULL, 10); + if (count > 0) { + LDAPDebug1Arg( LDAP_DEBUG_TRACE, "<= slapi_has_tombstone_children %d\n", count); + return count; } } } @@ -3142,6 +3162,12 @@ slapi_entry_has_children(const Slapi_Entry *entry) return(0); } +int +slapi_entry_has_children(const Slapi_Entry *entry) +{ + return slapi_entry_has_children_ext(entry, 0); +} + /* * Renames an entry to simulate a MODRDN operation */ diff --git a/ldap/servers/slapd/rdn.c b/ldap/servers/slapd/rdn.c index fe2fae0..6e40ed8 100644 --- a/ldap/servers/slapd/rdn.c +++ b/ldap/servers/slapd/rdn.c @@ -124,7 +124,7 @@ slapi_rdn_init_dn(Slapi_RDN *rdn,const char *dn) * 1 -- "dn" does not belong to the database; could be "rdn" */ static int -_slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn) +_slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone) { const char *dn = NULL; const char *ndn= NULL; @@ -202,9 +202,16 @@ _slapi_rdn_init_all_dn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn) } /* Get the last matched position */ - if(dns) - { - rdn->rdn = slapi_ch_strdup(dns[0]); + if (dns) { + if (is_tombstone && slapi_is_special_rdn(dns[0], RDN_IS_TOMBSTONE)) { + /* merge nsuniqueid=...,<rdn> into one rdn */ + rdn->rdn = slapi_ch_smprintf("%s,%s", dns[0], dns[1]); + slapi_ch_free_string(&dns[0]); + dns[0] = slapi_ch_strdup(rdn->rdn); + charray_remove(dns, dns[1], 1); + } else { + rdn->rdn = slapi_ch_strdup(dns[0]); + } rdn->all_rdns = dns; slapi_setbit_uchar(rdn->flag,FLAG_ALL_RDNS); } @@ -238,7 +245,7 @@ slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn) slapi_rdn_init(rdn); slapi_sdn_init(&sdn); slapi_sdn_set_dn_byval(&sdn, dn); - rc = _slapi_rdn_init_all_dn_ext(rdn, (const Slapi_DN *)&sdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, (const Slapi_DN *)&sdn, 0); slapi_sdn_done(&sdn); return rc; } @@ -257,6 +264,33 @@ slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn) * 1 -- dn does not belong to the database */ int +slapi_rdn_init_all_sdn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone) +{ + int rc = 0; /* success */ + + if (NULL == rdn || NULL == sdn) + { + return -1; + } + slapi_rdn_init(rdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, sdn, is_tombstone); + return rc; +} + +/* + * This function sets DN from sdn to Slapi_RDN. + * Note: The underlying function _slapi_rdn_init_all_dn_ext checks if the DN + * is in the root or sub suffix the server owns. If it is, the root or sub + * suffix is treated as one "rdn" (e.g., "dc=sub,dc=example,dc=com") and 0 is + * returned. If it is not, the DN is separated by ',' and each string is set + * to RDN array. (e.g., input: "uid=A,ou=does_not_exist" ==> "uid=A", "ou= + * does_not_exist") and 1 is returned. + * + * Return Value: 0 -- Success + * -1 -- Error + * 1 -- dn does not belong to the database + */ +int slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn) { int rc = 0; /* success */ @@ -266,7 +300,7 @@ slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn) return -1; } slapi_rdn_init(rdn); - rc = _slapi_rdn_init_all_dn_ext(rdn, sdn); + rc = _slapi_rdn_init_all_dn_ext(rdn, sdn, 0); return rc; } @@ -291,6 +325,17 @@ slapi_rdn_init_rdn(Slapi_RDN *rdn,const Slapi_RDN *fromrdn) } void +slapi_rdn_set_dn_ext(Slapi_RDN *rdn,const char *dn, int skip_tombstone) +{ + const char *mydn = dn; + slapi_rdn_done(rdn); + if (skip_tombstone && slapi_is_special_rdn(dn, RDN_IS_TOMBSTONE)) { + mydn = dn + slapi_uniqueIDRdnSize() + 1/*,*/; + } + slapi_rdn_init_dn(rdn, mydn); +} + +void slapi_rdn_set_dn(Slapi_RDN *rdn,const char *dn) { slapi_rdn_done(rdn); @@ -473,6 +518,33 @@ slapi_rdn_contains(Slapi_RDN *rdn, const char *type, const char *value, size_t l } int +slapi_rdn_is_multivalued(Slapi_RDN *rdn) +{ + char *p = NULL; + if (rdn && rdn->rdn) { + p = PL_strchr(rdn->rdn, '+'); + } + if (p) { + return 1; + } else { + return 0; + } +} + +/* + * Return value 1: if rdn is a conflict rdn + * 0: otherwise + */ +int +slapi_rdn_is_conflict(Slapi_RDN *rdn) +{ + if (!rdn) { + return 0; + } + return slapi_is_special_rdn(slapi_rdn_get_nrdn(rdn), RDN_IS_CONFLICT); +} + +int slapi_rdn_add(Slapi_RDN *rdn, const char *type, const char *value) { PR_ASSERT(NULL != type); @@ -484,12 +556,32 @@ slapi_rdn_add(Slapi_RDN *rdn, const char *type, const char *value) } else { - /* type=value+rdn '\0' */ - char *newrdn = slapi_create_dn_string("%s=%s+%s", type, value, rdn->rdn); + char *newrdn = NULL; + char *rp = rdn->rdn; + PRUint32 uniqueidlen = slapi_uniqueIDRdnSize(); + + if (slapi_is_special_rdn(rp, RDN_IS_TOMBSTONE)) { + char *corerp = rp + uniqueidlen + 1; + *(rp + uniqueidlen) = '\0'; + newrdn = slapi_create_dn_string("%s,%s=%s+%s", rp, type, value, corerp); + } else { + /* type=value+rdn '\0' */ + newrdn = slapi_create_dn_string("%s=%s+%s", type, value, rp); + } slapi_ch_free_string(&rdn->rdn); rdn->rdn = newrdn; } slapi_unsetbit_uchar(rdn->flag,FLAG_RDNS); + + if (rdn->all_rdns && rdn->all_rdns[0]) { + slapi_ch_free_string(&rdn->all_rdns[0]); + rdn->all_rdns[0] = slapi_ch_strdup(rdn->rdn); + } + if (rdn->all_nrdns && rdn->all_nrdns[0]) { + slapi_ch_free_string(&rdn->all_nrdns[0]); + rdn->all_nrdns[0] = slapi_ch_strdup(rdn->rdn); + slapi_dn_ignore_case(rdn->all_nrdns[0]); + } return 1; } diff --git a/ldap/servers/slapd/slapi-plugin.h b/ldap/servers/slapd/slapi-plugin.h index 39c50a9..4e84060 100644 --- a/ldap/servers/slapd/slapi-plugin.h +++ b/ldap/servers/slapd/slapi-plugin.h @@ -1868,6 +1868,17 @@ int slapi_entry_attr_has_syntax_value(const Slapi_Entry *e, const char *type, co int slapi_entry_has_children(const Slapi_Entry *e); /** + * This function determines if the specified entry has child entries + * including the tombstoned descendents. + * + * \param e Entry that you want to test for child entries. + * \param include_tombstone If non-zero, check tombstone_subordinates, too. + * \return \c 1 if the entry you supply has child entries. + * \return \c 0 if the entry you supply has child entries. + */ +int slapi_entry_has_children_ext(const Slapi_Entry *e, int include_tombstone); + +/** * This function determines if an entry is the root DSE. * * The root DSE is a special entry that contains information about the Directory @@ -2966,9 +2977,11 @@ void slapi_rdn_init_rdn(Slapi_RDN *rdn,const Slapi_RDN *fromrdn); * \see slapi_rdn_set_rdn() */ void slapi_rdn_set_dn(Slapi_RDN *rdn,const char *dn); +void slapi_rdn_set_dn_ext(Slapi_RDN *rdn,const char *dn, int skip_tombstone); Slapi_RDN *slapi_rdn_new_all_dn(const char *dn); int slapi_rdn_init_all_dn(Slapi_RDN *rdn, const char *dn); int slapi_rdn_init_all_sdn(Slapi_RDN *rdn, const Slapi_DN *sdn); +int slapi_rdn_init_all_sdn_ext(Slapi_RDN *rdn, const Slapi_DN *sdn, int is_tombstone); /** * Sets the RDN value in a \c Slapi_RDN structure from a \c Slapi_DN. @@ -3384,6 +3397,24 @@ size_t slapi_rdn_get_size(Slapi_RDN *srdn); */ char * slapi_rdn_get_value(const char *rdn); +/** + * Check if the rdn is multivalued or not + * + * \param rdn A pointer to rdn to exam. + * \return 1, if the rdn is multi valued. + * 0, if the rdn is simgle valued. + */ +int slapi_rdn_is_multivalued(Slapi_RDN *rdn); + +/** + * Check if the rdn is a conflict rdn or not + * + * \param rdn A pointer to rdn to exam. + * \return 1, if the rdn is a conflict rdn + * 0, if the rdn is not a conflict rdn + */ +int slapi_rdn_is_conflict(Slapi_RDN *rdn); + /* * utility routines for dealing with DNs */ @@ -7337,6 +7368,20 @@ char *slapi_pr_strerror( const PRErrorCode prerrno ); */ const char *slapi_system_strerror( const int syserrno ); +/** Check if rdn is a slecial rdn/dn or not. + * + * \param rdn rdn/dn to check + * \param flags specify the type: RDN_IS_TOMBSTONE or RDN_IS_CONFLICT or 0 + * + * \return 1 if rdn matches the flag. + * If flag is IS_TOMBSTONE, returns 1 if rdn is a tombstone rdn/dn. + * If flag is IS_CONFLICT, returns 1 if rdn is a conflict rdn/dn. + * \return 0 otherwise + */ +#define RDN_IS_TOMBSTONE 0x1 +#define RDN_IS_CONFLICT 0x2 +int slapi_is_special_rdn(const char *rdn, int flag); + #ifdef __cplusplus } #endif diff --git a/ldap/servers/slapd/slapi-private.h b/ldap/servers/slapd/slapi-private.h index 5b940e9..940260f 100644 --- a/ldap/servers/slapd/slapi-private.h +++ b/ldap/servers/slapd/slapi-private.h @@ -998,6 +998,13 @@ int slapi_uniqueIDIsUUID(const Slapi_UniqueID *uId); */ int slapi_uniqueIDSize( void ); +/* Name: slapi_uniqueIDRdnSize + Description: returns size of SLAPI_ATTR_UNIQUEID=slapi_uniqueIDSize() + Parameters: none + Return: size of the string version of "SLAPI_ATTR_UNIQUEID=uniqueID" in bytes + */ +int slapi_uniqueIDRdnSize( void ); + /* Name: slapi_uniqueIDDup Description: duplicates an UniqueID object Parameters: uId - id to duplicate diff --git a/ldap/servers/slapd/util.c b/ldap/servers/slapd/util.c index 0a9d23d..c566512 100644 --- a/ldap/servers/slapd/util.c +++ b/ldap/servers/slapd/util.c @@ -1089,3 +1089,70 @@ slapi_str_to_u64(const char *s) (v6 << 36) | (v7 << 32) | (v8 << 28) | (v9 << 24) | (v10 << 20) | (v11 << 16) | (v12 << 12) | (v13 << 8) | (v14 << 4) | v15; } + +/* + * Check if rdn is a slecial rdn/dn or not. + * If flag is IS_TOMBSTONE, returns 1 if rdn is a tombstone rdn/dn. + * If flag is IS_CONFLICT, returns 1 if rdn is a conflict rdn/dn. + * Otherwise returns 0. + */ +static int util_uniqueidlen = 0; +int +slapi_is_special_rdn(const char *rdn, int flag) +{ + char *rp; + int plus = 0; + if (!util_uniqueidlen) { + util_uniqueidlen = SLAPI_ATTR_UNIQUEID_LENGTH + slapi_uniqueIDSize() + 1/*=*/; + } + + if ((RDN_IS_TOMBSTONE != flag) && (RDN_IS_CONFLICT != flag)) { + LDAPDebug1Arg(LDAP_DEBUG_ANY, "slapi_is_special_rdn: invalid flag %d\n", flag); + return 0; /* not a special rdn/dn */ + } + if (!rdn) { + LDAPDebug0Args(LDAP_DEBUG_ANY, "slapi_is_special_rdn: NULL rdn\n"); + return 0; /* not a special rdn/dn */ + } + + if (strlen(rdn) < util_uniqueidlen) { + return 0; /* not a special rdn/dn */ + } + rp = (char *)rdn; + while (rp) { + char *comma = NULL; + if (!PL_strncasecmp(rp, SLAPI_ATTR_UNIQUEID, SLAPI_ATTR_UNIQUEID_LENGTH) && + (*(rp + SLAPI_ATTR_UNIQUEID_LENGTH) == '=')) { + if (RDN_IS_TOMBSTONE == flag) { + if ((*(rp + util_uniqueidlen) == ',') || + (*(rp + util_uniqueidlen) == '\0')) { + return 1; + } else { + return 0; + } + } else { + if ((*(rp + util_uniqueidlen) == '+') || + (plus && ((*(rp + util_uniqueidlen) == ',') || + (*(rp + util_uniqueidlen) == '\0')))) { + return 1; + } + } + } + comma = PL_strchr(rp, ','); + rp = PL_strchr(rp, '+'); + if (rp && (rp < comma)) { + plus = 1; + rp++; + } + } + return 0; +} + +int +slapi_uniqueIDRdnSize() +{ + if (!util_uniqueidlen) { + util_uniqueidlen = SLAPI_ATTR_UNIQUEID_LENGTH + slapi_uniqueIDSize() + 1/*=*/; + } + return util_uniqueidlen; +}
1
0
0
0
dirsrvtests/tickets ldap/servers
by thierry bordaz
12 May '14
12 May '14
dirsrvtests/tickets/ticket47787_test.py | 630 +++++++++++++++++++++++++++++ ldap/servers/slapd/back-ldbm/ldbm_modify.c | 2 2 files changed, 631 insertions(+), 1 deletion(-) New commits: commit 804d923ea44a7d96e49e550ff8d1fc3ff128f5c2 Author: Thierry bordaz (tbordaz) <tbordaz(a)redhat.com> Date: Fri Apr 25 15:07:08 2014 +0200 Ticket 47787 - A replicated MOD fails (Unwilling to perform) if it targets a tombstone Bug Description: A fix for
https://fedorahosted.org/389/ticket/47396
prevents a MOD operation on a tombstone. A problem is that legacy version may allowed it or if a MOD entry and DEL entry occurs on different master, it may happen that a consumer receive a MOD on a tombstone. The server return 'Unwilling to perform' that may break replication. Fix Description: Allows a MOD on tombstone, only if it comes from a replication session
https://fedorahosted.org/389/ticket/47787
Reviewed by: Rich Megginson (thanks) Platforms tested: F17, F19 Flag Day: no Doc impact: no diff --git a/dirsrvtests/tickets/ticket47787_test.py b/dirsrvtests/tickets/ticket47787_test.py new file mode 100644 index 0000000..d9d3ca9 --- /dev/null +++ b/dirsrvtests/tickets/ticket47787_test.py @@ -0,0 +1,630 @@ +''' +Created on April 14, 2014 + +@author: tbordaz +''' +import os +import sys +import time +import ldap +import logging +import socket +import time +import logging +import pytest +import re +from lib389 import DirSrv, Entry, tools, NoSuchEntryError +from lib389.tools import DirSrvTools +from lib389._constants import * +from lib389.properties import * +from constants import * +from lib389._constants import REPLICAROLE_MASTER + +logging.getLogger(__name__).setLevel(logging.DEBUG) +log = logging.getLogger(__name__) + +# +# important part. We can deploy Master1 and Master2 on different versions +# +installation1_prefix = None +installation2_prefix = None + +# set this flag to False so that it will assert on failure _status_entry_both_server +DEBUG_FLAG = False + +TEST_REPL_DN = "cn=test_repl, %s" % SUFFIX + +STAGING_CN = "staged user" +PRODUCTION_CN = "accounts" +EXCEPT_CN = "excepts" + +STAGING_DN = "cn=%s,%s" % (STAGING_CN, SUFFIX) +PRODUCTION_DN = "cn=%s,%s" % (PRODUCTION_CN, SUFFIX) +PROD_EXCEPT_DN = "cn=%s,%s" % (EXCEPT_CN, PRODUCTION_DN) + +STAGING_PATTERN = "cn=%s*,%s" % (STAGING_CN[:2], SUFFIX) +PRODUCTION_PATTERN = "cn=%s*,%s" % (PRODUCTION_CN[:2], SUFFIX) +BAD_STAGING_PATTERN = "cn=bad*,%s" % (SUFFIX) +BAD_PRODUCTION_PATTERN = "cn=bad*,%s" % (SUFFIX) + +BIND_CN = "bind_entry" +BIND_DN = "cn=%s,%s" % (BIND_CN, SUFFIX) +BIND_PW = "password" + +NEW_ACCOUNT = "new_account" +MAX_ACCOUNTS = 20 + +CONFIG_MODDN_ACI_ATTR = "nsslapd-moddn-aci" + +class TopologyMaster1Master2(object): + def __init__(self, master1, master2): + master1.open() + self.master1 = master1 + + master2.open() + self.master2 = master2 + + +(a)pytest.fixture(scope="module") +def topology(request): + ''' + This fixture is used to create a replicated topology for the 'module'. + The replicated topology is MASTER1 <-> Master2. + At the beginning, It may exists a master2 instance and/or a master2 instance. + It may also exists a backup for the master1 and/or the master2. + + Principle: + If master1 instance exists: + restart it + If master2 instance exists: + restart it + If backup of master1 AND backup of master2 exists: + create or rebind to master1 + create or rebind to master2 + + restore master1 from backup + restore master2 from backup + else: + Cleanup everything + remove instances + remove backups + Create instances + Initialize replication + Create backups + ''' + global installation1_prefix + global installation2_prefix + + # allocate master1 on a given deployement + master1 = DirSrv(verbose=False) + if installation1_prefix: + args_instance[SER_DEPLOYED_DIR] = installation1_prefix + + # Args for the master1 instance + args_instance[SER_HOST] = HOST_MASTER_1 + args_instance[SER_PORT] = PORT_MASTER_1 + args_instance[SER_SERVERID_PROP] = SERVERID_MASTER_1 + args_master = args_instance.copy() + master1.allocate(args_master) + + # allocate master1 on a given deployement + master2 = DirSrv(verbose=False) + if installation2_prefix: + args_instance[SER_DEPLOYED_DIR] = installation2_prefix + + # Args for the consumer instance + args_instance[SER_HOST] = HOST_MASTER_2 + args_instance[SER_PORT] = PORT_MASTER_2 + args_instance[SER_SERVERID_PROP] = SERVERID_MASTER_2 + args_master = args_instance.copy() + master2.allocate(args_master) + + + # Get the status of the backups + backup_master1 = master1.checkBackupFS() + backup_master2 = master2.checkBackupFS() + + # Get the status of the instance and restart it if it exists + instance_master1 = master1.exists() + if instance_master1: + master1.stop(timeout=10) + master1.start(timeout=10) + + instance_master2 = master2.exists() + if instance_master2: + master2.stop(timeout=10) + master2.start(timeout=10) + + if backup_master1 and backup_master2: + # The backups exist, assuming they are correct + # we just re-init the instances with them + if not instance_master1: + master1.create() + # Used to retrieve configuration information (dbdir, confdir...) + master1.open() + + if not instance_master2: + master2.create() + # Used to retrieve configuration information (dbdir, confdir...) + master2.open() + + # restore master1 from backup + master1.stop(timeout=10) + master1.restoreFS(backup_master1) + master1.start(timeout=10) + + # restore master2 from backup + master2.stop(timeout=10) + master2.restoreFS(backup_master2) + master2.start(timeout=10) + else: + # We should be here only in two conditions + # - This is the first time a test involve master-consumer + # so we need to create everything + # - Something weird happened (instance/backup destroyed) + # so we discard everything and recreate all + + # Remove all the backups. So even if we have a specific backup file + # (e.g backup_master) we clear all backups that an instance my have created + if backup_master1: + master1.clearBackupFS() + if backup_master2: + master2.clearBackupFS() + + # Remove all the instances + if instance_master1: + master1.delete() + if instance_master2: + master2.delete() + + # Create the instances + master1.create() + master1.open() + master2.create() + master2.open() + + # + # Now prepare the Master-Consumer topology + # + # First Enable replication + master1.replica.enableReplication(suffix=SUFFIX, role=REPLICAROLE_MASTER, replicaId=REPLICAID_MASTER_1) + master2.replica.enableReplication(suffix=SUFFIX, role=REPLICAROLE_MASTER, replicaId=REPLICAID_MASTER_2) + + # Initialize the supplier->consumer + + properties = {RA_NAME: r'meTo_$host:$port', + RA_BINDDN: defaultProperties[REPLICATION_BIND_DN], + RA_BINDPW: defaultProperties[REPLICATION_BIND_PW], + RA_METHOD: defaultProperties[REPLICATION_BIND_METHOD], + RA_TRANSPORT_PROT: defaultProperties[REPLICATION_TRANSPORT]} + repl_agreement = master1.agreement.create(suffix=SUFFIX, host=master2.host, port=master2.port, properties=properties) + + if not repl_agreement: + log.fatal("Fail to create a replica agreement") + sys.exit(1) + + log.debug("%s created" % repl_agreement) + + properties = {RA_NAME: r'meTo_$host:$port', + RA_BINDDN: defaultProperties[REPLICATION_BIND_DN], + RA_BINDPW: defaultProperties[REPLICATION_BIND_PW], + RA_METHOD: defaultProperties[REPLICATION_BIND_METHOD], + RA_TRANSPORT_PROT: defaultProperties[REPLICATION_TRANSPORT]} + master2.agreement.create(suffix=SUFFIX, host=master1.host, port=master1.port, properties=properties) + + master1.agreement.init(SUFFIX, HOST_MASTER_2, PORT_MASTER_2) + master1.waitForReplInit(repl_agreement) + + # Check replication is working fine + master1.add_s(Entry((TEST_REPL_DN, { + 'objectclass': "top person".split(), + 'sn': 'test_repl', + 'cn': 'test_repl'}))) + loop = 0 + while loop <= 10: + try: + ent = master2.getEntry(TEST_REPL_DN, ldap.SCOPE_BASE, "(objectclass=*)") + break + except ldap.NO_SUCH_OBJECT: + time.sleep(1) + loop += 1 + + # Time to create the backups + master1.stop(timeout=10) + master1.backupfile = master1.backupFS() + master1.start(timeout=10) + + master2.stop(timeout=10) + master2.backupfile = master2.backupFS() + master2.start(timeout=10) + + # + # Here we have two instances master and consumer + # with replication working. Either coming from a backup recovery + # or from a fresh (re)init + # Time to return the topology + return TopologyMaster1Master2(master1, master2) + + + +def _bind_manager(server): + server.log.info("Bind as %s " % DN_DM) + server.simple_bind_s(DN_DM, PASSWORD) + +def _bind_normal(server): + server.log.info("Bind as %s " % BIND_DN) + server.simple_bind_s(BIND_DN, BIND_PW) + +def _header(topology, label): + topology.master1.log.info("\n\n###############################################") + topology.master1.log.info("#######") + topology.master1.log.info("####### %s" % label) + topology.master1.log.info("#######") + topology.master1.log.info("###############################################") + +def _status_entry_both_server(topology, name=None, desc=None, debug=True): + if not name: + return + topology.master1.log.info("\n\n######################### Tombstone on M1 ######################\n") + ent_m1 = _find_tombstone(topology.master1, SUFFIX, 'sn', name) + assert ent_m1 + + topology.master1.log.info("\n\n######################### Tombstone on M2 ######################\n") + ent_m2 = _find_tombstone(topology.master2, SUFFIX, 'sn', name) + assert ent_m2 + + topology.master1.log.info("\n\n######################### Description ######################\n%s\n" % desc) + topology.master1.log.info("M1 only\n") + for attr in ent_m1.getAttrs(): + + if not debug: + assert attr in ent_m2.getAttrs() + + if not attr in ent_m2.getAttrs(): + topology.master1.log.info(" %s" % attr) + for val in ent_m1.getValues(attr): + topology.master1.log.info(" %s" % val) + + topology.master1.log.info("M2 only\n") + for attr in ent_m2.getAttrs(): + + if not debug: + assert attr in ent_m1.getAttrs() + + if not attr in ent_m1.getAttrs(): + topology.master1.log.info(" %s" % attr) + for val in ent_m2.getValues(attr): + topology.master1.log.info(" %s" % val) + + topology.master1.log.info("M1 differs M2\n") + + if not debug: + assert ent_m1.dn == ent_m2.dn + + if ent_m1.dn != ent_m2.dn: + topology.master1.log.info(" M1[dn] = %s\n M2[dn] = %s" % (ent_m1.dn, ent_m2.dn)) + + for attr1 in ent_m1.getAttrs(): + if attr1 in ent_m2.getAttrs(): + for val1 in ent_m1.getValues(attr1): + found = False + for val2 in ent_m2.getValues(attr1): + if val1 == val2: + found = True + break + + if not debug: + assert found + + if not found: + topology.master1.log.info(" M1[%s] = %s" % (attr1, val1)) + + + for attr2 in ent_m2.getAttrs(): + if attr2 in ent_m1.getAttrs(): + for val2 in ent_m2.getValues(attr2): + found = False + for val1 in ent_m1.getValues(attr2): + if val2 == val1: + found = True + break + + if not debug: + assert found + + if not found: + topology.master1.log.info(" M2[%s] = %s" % (attr2, val2)) + +def _pause_RAs(topology): + topology.master1.log.info("\n\n######################### Pause RA M1<->M2 ######################\n") + ents = topology.master1.agreement.list(suffix=SUFFIX) + assert len(ents) == 1 + topology.master1.agreement.pause(ents[0].dn) + + ents = topology.master2.agreement.list(suffix=SUFFIX) + assert len(ents) == 1 + topology.master2.agreement.pause(ents[0].dn) + +def _resume_RAs(topology): + topology.master1.log.info("\n\n######################### resume RA M1<->M2 ######################\n") + ents = topology.master1.agreement.list(suffix=SUFFIX) + assert len(ents) == 1 + topology.master1.agreement.resume(ents[0].dn) + + ents = topology.master2.agreement.list(suffix=SUFFIX) + assert len(ents) == 1 + topology.master2.agreement.resume(ents[0].dn) + +def _find_tombstone(instance, base, attr, value): + # + # we can not use a filter with a (&(objeclass=nsTombstone)(sn=name)) because + # tombstone are not index in 'sn' so 'sn=name' will return NULL + # and even if tombstone are indexed for objectclass the '&' will set + # the candidate list to NULL + # + filt = '(objectclass=%s)' % REPLICA_OC_TOMBSTONE + ents = instance.search_s(base, ldap.SCOPE_SUBTREE, filt) + found = False + for ent in ents: + if ent.hasAttr(attr): + for val in ent.getValues(attr): + if val == value: + instance.log.debug("tombstone found: %r" % ent) + return ent + return None + + +def _delete_entry(instance, entry_dn, name): + instance.log.info("\n\n######################### DELETE %s (M1) ######################\n" % name) + + # delete the entry + instance.delete_s(entry_dn) + assert _find_tombstone(instance, SUFFIX, 'sn', name) != None + + + +def _mod_entry(instance, entry_dn, attr, value): + instance.log.info("\n\n######################### MOD %s (M2) ######################\n" % entry_dn) + mod = [(ldap.MOD_REPLACE, attr, value)] + instance.modify_s(entry_dn, mod) + +def _modrdn_entry(instance=None, entry_dn=None, new_rdn=None, del_old=0, new_superior=None): + assert instance != None + assert entry_dn != None + + if not new_rdn: + pattern = 'cn=(.*),(.*)' + rdnre = re.compile(pattern) + match = rdnre.match(entry_dn) + old_value = match.group(1) + new_rdn_val = "%s_modrdn" % old_value + new_rdn = "cn=%s" % new_rdn_val + + + instance.log.info("\n\n######################### MODRDN %s (M2) ######################\n" % new_rdn) + if new_superior: + instance.rename_s(entry_dn, new_rdn, newsuperior=new_superior, delold=del_old) + else: + instance.rename_s(entry_dn, new_rdn, delold=del_old) + +def _check_entry_exists(instance, entry_dn): + loop = 0 + while loop <= 10: + try: + ent = instance.getEntry(entry_dn, ldap.SCOPE_BASE, "(objectclass=*)") + break + except ldap.NO_SUCH_OBJECT: + time.sleep(1) + loop += 1 + assert loop <= 10 + +def _check_mod_received(instance, base, filt, attr, value): + instance.log.info("\n\n######################### Check MOD replicated on %s ######################\n" % instance.serverid) + loop = 0 + while loop <= 10: + ent = instance.getEntry(base, ldap.SCOPE_SUBTREE, filt) + if ent.hasAttr(attr) and ent.getValue(attr) == value: + break + time.sleep(1) + loop += 1 + assert loop <= 10 + +def _check_replication(topology, entry_dn): + # prepare the filter to retrieve the entry + filt = entry_dn.split(',')[0] + + topology.master1.log.info("\n######################### Check replicat M1->M2 ######################\n") + loop = 0 + while loop <= 10: + attr = 'description' + value = 'test_value_%d' % loop + mod = [(ldap.MOD_REPLACE, attr, value)] + topology.master1.modify_s(entry_dn, mod) + _check_mod_received(topology.master2, SUFFIX, filt, attr, value) + loop += 1 + + topology.master1.log.info("\n######################### Check replicat M2->M1 ######################\n") + loop = 0 + while loop <= 10: + attr = 'description' + value = 'test_value_%d' % loop + mod = [(ldap.MOD_REPLACE, attr, value)] + topology.master2.modify_s(entry_dn, mod) + _check_mod_received(topology.master1, SUFFIX, filt, attr, value) + loop += 1 + + +def test_ticket47787_init(topology): + """ + Creates + - a staging DIT + - a production DIT + - add accounts in staging DIT + + """ + + topology.master1.log.info("\n\n######################### INITIALIZATION ######################\n") + + # entry used to bind with + topology.master1.log.info("Add %s" % BIND_DN) + topology.master1.add_s(Entry((BIND_DN, { + 'objectclass': "top person".split(), + 'sn': BIND_CN, + 'cn': BIND_CN, + 'userpassword': BIND_PW}))) + + # DIT for staging + topology.master1.log.info("Add %s" % STAGING_DN) + topology.master1.add_s(Entry((STAGING_DN, { + 'objectclass': "top organizationalRole".split(), + 'cn': STAGING_CN, + 'description': "staging DIT"}))) + + # DIT for production + topology.master1.log.info("Add %s" % PRODUCTION_DN) + topology.master1.add_s(Entry((PRODUCTION_DN, { + 'objectclass': "top organizationalRole".split(), + 'cn': PRODUCTION_CN, + 'description': "production DIT"}))) + + + # enable replication error logging + mod = [(ldap.MOD_REPLACE, 'nsslapd-errorlog-level', '8192')] + topology.master1.modify_s(DN_CONFIG, mod) + topology.master2.modify_s(DN_CONFIG, mod) + + + + + + # add dummy entries in the staging DIT + for cpt in range(MAX_ACCOUNTS): + name = "%s%d" % (NEW_ACCOUNT, cpt) + topology.master1.add_s(Entry(("cn=%s,%s" % (name, STAGING_DN), { + 'objectclass': "top person".split(), + 'sn': name, + 'cn': name}))) + + +def test_ticket47787_2(topology): + ''' + Disable replication so that updates are not replicated + Delete an entry on M1. Modrdn it on M2 (chg rdn + delold=0 + same superior). + update a test entry on M2 + Reenable the RA. + checks that entry was deleted on M2 (with the modified RDN) + checks that test entry was replicated on M1 (replication M2->M1 not broken by modrdn) + ''' + + _header(topology, "test_ticket47787_2") + _bind_manager(topology.master1) + _bind_manager(topology.master2) + + #entry to test the replication is still working + name = "%s%d" % (NEW_ACCOUNT, MAX_ACCOUNTS -1) + test_rdn = "cn=%s" % (name) + testentry_dn = "%s,%s" % (test_rdn, STAGING_DN) + + name = "%s%d" % (NEW_ACCOUNT, MAX_ACCOUNTS - 2) + test2_rdn = "cn=%s" % (name) + testentry2_dn = "%s,%s" % (test2_rdn, STAGING_DN) + + # value of updates to test the replication both ways + attr = 'description' + value = 'test_ticket47787_2' + + # entry for the modrdn + name = "%s%d" % (NEW_ACCOUNT, 1) + rdn = "cn=%s" % (name) + entry_dn = "%s,%s" % (rdn, STAGING_DN) + + # created on M1, wait the entry exists on M2 + _check_entry_exists(topology.master2, entry_dn) + _check_entry_exists(topology.master2, testentry_dn) + + _pause_RAs(topology) + + # Delete 'entry_dn' on M1. + # dummy update is only have a first CSN before the DEL + # else the DEL will be in min_csn RUV and make diagnostic a bit more complex + _mod_entry(topology.master1, testentry2_dn, attr, 'dummy') + _delete_entry(topology.master1, entry_dn, name) + _mod_entry(topology.master1, testentry2_dn, attr, value) + + time.sleep(1) # important to have MOD.csn != DEL.csn + + # MOD 'entry_dn' on M1. + # dummy update is only have a first CSN before the MOD entry_dn + # else the DEL will be in min_csn RUV and make diagnostic a bit more complex + _mod_entry(topology.master2, testentry_dn, attr, 'dummy') + _mod_entry(topology.master2, entry_dn, attr, value) + _mod_entry(topology.master2, testentry_dn, attr, value) + + + _resume_RAs(topology) + + topology.master1.log.info("\n\n######################### Check DEL replicated on M2 ######################\n") + loop = 0 + while loop <= 10: + ent = _find_tombstone(topology.master2, SUFFIX, 'sn', name) + if ent: + break + time.sleep(1) + loop += 1 + assert loop <= 10 + assert ent + + # the following checks are not necessary + # as this bug is only for failing replicated MOD (entry_dn) on M1 + #_check_mod_received(topology.master1, SUFFIX, "(%s)" % (test_rdn), attr, value) + #_check_mod_received(topology.master2, SUFFIX, "(%s)" % (test2_rdn), attr, value) + # + #_check_replication(topology, testentry_dn) + + _status_entry_both_server(topology, name=name, desc="DEL M1 - MOD M2", debug=DEBUG_FLAG) + + topology.master1.log.info("\n\n######################### Check MOD replicated on M1 ######################\n") + loop = 0 + while loop <= 10: + ent = _find_tombstone(topology.master1, SUFFIX, 'sn', name) + if ent: + break + time.sleep(1) + loop += 1 + assert loop <= 10 + assert ent + assert ent.hasAttr(attr) + assert ent.getValue(attr) == value + + +def test_ticket47787_final(topology): + topology.master1.stop(timeout=10) + topology.master2.stop(timeout=10) + +def run_isolated(): + ''' + run_isolated is used to run these test cases independently of a test scheduler (xunit, py.test..) + To run isolated without py.test, you need to + - edit this file and comment '@pytest.fixture' line before 'topology' function. + - set the installation prefix + - run this program + ''' + global installation1_prefix + global installation2_prefix + installation1_prefix = None + installation2_prefix = None + + topo = topology(True) + topo.master1.log.info("\n\n######################### Ticket 47787 ######################\n") + test_ticket47787_init(topo) + + test_ticket47787_2(topo) + + test_ticket47787_final(topo) + + + + +if __name__ == '__main__': + run_isolated() + diff --git a/ldap/servers/slapd/back-ldbm/ldbm_modify.c b/ldap/servers/slapd/back-ldbm/ldbm_modify.c index f8fb168..e618c6b 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_modify.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_modify.c @@ -558,7 +558,7 @@ ldbm_back_modify( Slapi_PBlock *pb ) if ( !is_fixup_operation ) { - if (slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) ) { + if (!repl_op && slapi_entry_flag_is_set(e->ep_entry, SLAPI_ENTRY_FLAG_TOMBSTONE) ) { ldap_result_code = LDAP_UNWILLING_TO_PERFORM; ldap_result_message = "Operation not allowed on tombstone entry."; slapi_log_error(SLAPI_LOG_FATAL, "ldbm_back_modify",
1
0
0
0
← Newer
1
2
3
4
5
6
7
Older →
Jump to page:
1
2
3
4
5
6
7
Results per page:
10
25
50
100
200