[389-ds-base] branch 389-ds-base-1.4.3 updated: Ticket 51079 - container pid start and stop issues
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
firstyear pushed a commit to branch 389-ds-base-1.4.3
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.3 by this push:
new 3bb6b50 Ticket 51079 - container pid start and stop issues
3bb6b50 is described below
commit 3bb6b5032c4938c7a0cbf6ba333f7dfa74ff0373
Author: William Brown <william(a)blackhats.net.au>
AuthorDate: Tue May 12 13:16:54 2020 +1000
Ticket 51079 - container pid start and stop issues
Bug Description: During the container startup, we were incorrectly
checking for the pidfile as we started. We also were not properly
catching sigint, and dscontainer on keyboard int was not passing
some signals through.
Fix Description: Improve signal handling in dscontainer, add sigint
as a caught signal to ns-slapd, and remove the pid file from the container
instance as we do not require it.
https://pagure.io/389-ds-base/issue/51079
https://pagure.io/389-ds-base/issue/51080
Author: William Brown <william(a)blackhats.net.au>
Review by: ???
---
ldap/servers/slapd/daemon.c | 1 +
src/lib389/cli/dscontainer | 62 ++++++++++++++++++++++++++++++---------------
2 files changed, 43 insertions(+), 20 deletions(-)
diff --git a/ldap/servers/slapd/daemon.c b/ldap/servers/slapd/daemon.c
index 65f2336..8a24ccc 100644
--- a/ldap/servers/slapd/daemon.c
+++ b/ldap/servers/slapd/daemon.c
@@ -2189,6 +2189,7 @@ init_shutdown_detect(void)
(void)SIGNAL(SIGUSR2, set_shutdown);
#endif
(void)SIGNAL(SIGTERM, set_shutdown);
+ (void)SIGNAL(SIGINT, set_shutdown);
(void)SIGNAL(SIGHUP, set_shutdown);
#endif /* HPUX */
return 0;
diff --git a/src/lib389/cli/dscontainer b/src/lib389/cli/dscontainer
index 4977fe1..de74ae1 100755
--- a/src/lib389/cli/dscontainer
+++ b/src/lib389/cli/dscontainer
@@ -65,6 +65,10 @@ def _sigchild_handler(*args, **kwargs):
# log.debug("Received SIGCHLD ...")
os.waitpid(-1, os.WNOHANG)
+# Start the exit process.
+def _sigterm_handler(*args, **kwargs):
+ exit()
+
def _gen_instance():
inst = DirSrv(verbose=True)
inst.local_simple_allocate("localhost")
@@ -284,10 +288,12 @@ binddn = cn=Directory Manager
ds_proc = subprocess.Popen([
"%s/ns-slapd" % paths.sbin_dir,
"-D", paths.config_dir,
- "-i", "/data/run/slapd-localhost.pid",
+ # This container version doesn't actually use or need the pidfile to track
+ # the process.
+ # "-i", "/data/run/slapd-localhost.pid",
# See /ldap/servers/slapd/slap.h SLAPD_DEFAULT_ERRORLOG_LEVEL
"-d", "266354688",
- ], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ ], stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=os.environ.copy())
# Setup the process and shutdown handler in an init-esque fashion.
def kill_ds():
@@ -295,27 +301,29 @@ binddn = cn=Directory Manager
pass
else:
try:
- os.kill(ds_proc.pid, signal.SIGTERM)
- except ProcessLookupError:
+ ds_proc.terminate()
+ log.info("STOPPING: Sent SIGTERM to ns-slapd ...")
+ except:
# It's already gone ...
pass
log.info("STOPPING: Shutting down 389-ds-container ...")
# To make sure we really do shutdown, we actually re-block on the proc
# again here to be sure it's done.
ds_proc.wait()
+ log.info("STOPPED: Shut down 389-ds-container")
atexit.register(kill_ds)
# Wait on the health check to show we are ready for ldapi.
- failure_count = 0
- max_failure_count = 5
+ healthy = False
+ max_failure_count = 20
for i in range(0, max_failure_count):
- status = begin_healthcheck()
- if status is True:
+ (check_again, healthy) = begin_healthcheck(ds_proc)
+ if check_again is False:
break
- failure_count += 1
time.sleep(3)
- if failure_count == max_failure_count:
+ # Check again then ....
+ if not healthy:
log.error("389-ds-container failed to start")
sys.exit(1)
@@ -334,22 +342,36 @@ binddn = cn=Directory Manager
# THE LETTER OF THE DAY IS C AND THE NUMBER IS 10
-def begin_healthcheck():
- # Is there an ns-slapd pid?
- # Can we get ldapi response?
- inst = _gen_instance()
- if inst.status() is not True:
- return False
+def begin_healthcheck(ds_proc):
+ if ds_proc is None:
+ log.warning("ns-slapd pid has disappeared ...")
+ return (False, False)
+ if ds_proc.poll() is not None:
+ # Ruh-Roh
+ log.warning("ns-slapd pid has completed, you should check the error log ...")
+ return (False, False)
# Now do an ldapi check, make sure we are dm.
- inst.open()
- if "dn: cn=Directory Manager" == inst.whoami_s():
- return True
- return False
+ try:
+ inst = _gen_instance()
+ inst.open()
+ if "dn: cn=Directory Manager" == inst.whoami_s():
+ return (False, True)
+ else:
+ log.error("The instance may be misconfigured, unable to cn=Directory Manager autobind.")
+ return (False, False)
+ except:
+ log.debug("Instance LDAPI not functional (yet?)")
+ pass
+ return (True, False)
if __name__ == '__main__':
# Before all else, we are INIT so setup sigchild
signal.signal(signal.SIGCHLD, _sigchild_handler)
+ # Setup catching for term and int
+ signal.signal(signal.SIGTERM, _sigterm_handler)
+ signal.signal(signal.SIGINT, _sigterm_handler)
+ signal.signal(signal.SIGHUP, _sigterm_handler)
parser = argparse.ArgumentParser(allow_abbrev=True, description="""
dscontainer - this is a container entry point that will run a stateless
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] branch 389-ds-base-1.4.2 updated: Ticket 51079 - container pid start and stop issues
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
firstyear pushed a commit to branch 389-ds-base-1.4.2
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.2 by this push:
new a7430ed Ticket 51079 - container pid start and stop issues
a7430ed is described below
commit a7430ed5a496c79e7885e904ddfdedc093d5a406
Author: William Brown <william(a)blackhats.net.au>
AuthorDate: Tue May 12 13:16:54 2020 +1000
Ticket 51079 - container pid start and stop issues
Bug Description: During the container startup, we were incorrectly
checking for the pidfile as we started. We also were not properly
catching sigint, and dscontainer on keyboard int was not passing
some signals through.
Fix Description: Improve signal handling in dscontainer, add sigint
as a caught signal to ns-slapd, and remove the pid file from the container
instance as we do not require it.
https://pagure.io/389-ds-base/issue/51079
https://pagure.io/389-ds-base/issue/51080
Author: William Brown <william(a)blackhats.net.au>
Review by: ???
---
ldap/servers/slapd/daemon.c | 1 +
src/lib389/cli/dscontainer | 62 ++++++++++++++++++++++++++++++---------------
2 files changed, 43 insertions(+), 20 deletions(-)
diff --git a/ldap/servers/slapd/daemon.c b/ldap/servers/slapd/daemon.c
index e6aca97..5e7ac87 100644
--- a/ldap/servers/slapd/daemon.c
+++ b/ldap/servers/slapd/daemon.c
@@ -2099,6 +2099,7 @@ init_shutdown_detect(void)
(void)SIGNAL(SIGUSR2, set_shutdown);
#endif
(void)SIGNAL(SIGTERM, set_shutdown);
+ (void)SIGNAL(SIGINT, set_shutdown);
(void)SIGNAL(SIGHUP, set_shutdown);
#endif /* HPUX */
return 0;
diff --git a/src/lib389/cli/dscontainer b/src/lib389/cli/dscontainer
index 7bea9ba..55d79d2 100755
--- a/src/lib389/cli/dscontainer
+++ b/src/lib389/cli/dscontainer
@@ -58,6 +58,10 @@ def _sigchild_handler(*args, **kwargs):
# log.debug("Received SIGCHLD ...")
os.waitpid(-1, os.WNOHANG)
+# Start the exit process.
+def _sigterm_handler(*args, **kwargs):
+ exit()
+
def _gen_instance():
inst = DirSrv(verbose=True)
inst.local_simple_allocate("localhost")
@@ -224,10 +228,12 @@ binddn = cn=Directory Manager
ds_proc = subprocess.Popen([
"%s/ns-slapd" % paths.sbin_dir,
"-D", paths.config_dir,
- "-i", "/data/run/slapd-localhost.pid",
+ # This container version doesn't actually use or need the pidfile to track
+ # the process.
+ # "-i", "/data/run/slapd-localhost.pid",
# See /ldap/servers/slapd/slap.h SLAPD_DEFAULT_ERRORLOG_LEVEL
"-d", "266354688",
- ], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ ], stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=os.environ.copy())
# Setup the process and shutdown handler in an init-esque fashion.
def kill_ds():
@@ -235,27 +241,29 @@ binddn = cn=Directory Manager
pass
else:
try:
- os.kill(ds_proc.pid, signal.SIGTERM)
- except ProcessLookupError:
+ ds_proc.terminate()
+ log.info("STOPPING: Sent SIGTERM to ns-slapd ...")
+ except:
# It's already gone ...
pass
log.info("STOPPING: Shutting down 389-ds-container ...")
# To make sure we really do shutdown, we actually re-block on the proc
# again here to be sure it's done.
ds_proc.wait()
+ log.info("STOPPED: Shut down 389-ds-container")
atexit.register(kill_ds)
# Wait on the health check to show we are ready for ldapi.
- failure_count = 0
- max_failure_count = 5
+ healthy = False
+ max_failure_count = 20
for i in range(0, max_failure_count):
- status = begin_healthcheck()
- if status is True:
+ (check_again, healthy) = begin_healthcheck(ds_proc)
+ if check_again is False:
break
- failure_count += 1
time.sleep(3)
- if failure_count == max_failure_count:
+ # Check again then ....
+ if not healthy:
log.error("389-ds-container failed to start")
sys.exit(1)
@@ -274,22 +282,36 @@ binddn = cn=Directory Manager
# THE LETTER OF THE DAY IS C AND THE NUMBER IS 10
-def begin_healthcheck():
- # Is there an ns-slapd pid?
- # Can we get ldapi response?
- inst = _gen_instance()
- if inst.status() is not True:
- return False
+def begin_healthcheck(ds_proc):
+ if ds_proc is None:
+ log.warning("ns-slapd pid has disappeared ...")
+ return (False, False)
+ if ds_proc.poll() is not None:
+ # Ruh-Roh
+ log.warning("ns-slapd pid has completed, you should check the error log ...")
+ return (False, False)
# Now do an ldapi check, make sure we are dm.
- inst.open()
- if "dn: cn=Directory Manager" == inst.whoami_s():
- return True
- return False
+ try:
+ inst = _gen_instance()
+ inst.open()
+ if "dn: cn=Directory Manager" == inst.whoami_s():
+ return (False, True)
+ else:
+ log.error("The instance may be misconfigured, unable to cn=Directory Manager autobind.")
+ return (False, False)
+ except:
+ log.debug("Instance LDAPI not functional (yet?)")
+ pass
+ return (True, False)
if __name__ == '__main__':
# Before all else, we are INIT so setup sigchild
signal.signal(signal.SIGCHLD, _sigchild_handler)
+ # Setup catching for term and int
+ signal.signal(signal.SIGTERM, _sigterm_handler)
+ signal.signal(signal.SIGINT, _sigterm_handler)
+ signal.signal(signal.SIGHUP, _sigterm_handler)
parser = argparse.ArgumentParser(allow_abbrev=True, description="""
dscontainer - this is a container entry point that will run a stateless
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] branch master updated: Revert "Issue 51017 - Implement dynamic ds/bz pytest markers"
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mhonek pushed a commit to branch master
in repository 389-ds-base.
The following commit(s) were added to refs/heads/master by this push:
new b3dec42 Revert "Issue 51017 - Implement dynamic ds/bz pytest markers"
b3dec42 is described below
commit b3dec427f1ab79c731aaa079e2769aac6f002bbc
Author: Matus Honek <mhonek(a)redhat.com>
AuthorDate: Thu May 14 13:22:37 2020 +0200
Revert "Issue 51017 - Implement dynamic ds/bz pytest markers"
Apparently, in some situations, grepping over all the files gets very
slow due to filesystem implementation (e.g. docker on MacOS) specifics.
Instead of this implementation, we'll rather look into pre-commit hooks
adding new markings on the fly.
This reverts commit 6a7a154159583c09fcbba0578eaf576d577ccb11.
Relates: https://pagure.io/389-ds-base/issue/51017
---
dirsrvtests/conftest.py | 11 -----------
1 file changed, 11 deletions(-)
diff --git a/dirsrvtests/conftest.py b/dirsrvtests/conftest.py
index 1577bc9..1ac02fa 100644
--- a/dirsrvtests/conftest.py
+++ b/dirsrvtests/conftest.py
@@ -12,17 +12,6 @@ from enum import Enum
pkgs = ['389-ds-base', 'nss', 'nspr', 'openldap', 'cyrus-sasl']
p = Paths()
-
-def pytest_configure(config):
- """Auto add custom markers we use to reference upstream/bugzilla tickets."""
- prefix = '@pytest.mark.'
- cmd = ['grep', '-rh', f'^{prefix}\\(ds\\|bz\\)[0-9]\\+']
- p = subprocess.run(cmd, check=True, stdout=subprocess.PIPE)
- markers = {x[len(prefix):] for x in p.stdout.decode().splitlines()}
- for m in markers:
- config.addinivalue_line('markers', m)
-
-
class FIPSState(Enum):
ENABLED = 'enabled'
DISABLED = 'disabled'
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] branch 389-ds-base-1.4.1 updated: Ticket 51082 - abort when a empty valueset is freed
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
tbordaz pushed a commit to branch 389-ds-base-1.4.1
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.1 by this push:
new 0fb23f3 Ticket 51082 - abort when a empty valueset is freed
0fb23f3 is described below
commit 0fb23f3c5aff94149dfa3a2ae040394aa2f2f9f2
Author: Thierry Bordaz <tbordaz(a)redhat.com>
AuthorDate: Mon May 11 17:11:49 2020 +0200
Ticket 51082 - abort when a empty valueset is freed
Bug Description:
A large valueset (more than 10 values) manages a sorted array of values.
replication purges old values from a valueset (valueset_array_purge). If it purges all the values
the valueset is freed (slapi_valueset_done).
A problem is that the counter of values, in the valueset, is still reflecting the initial number
of values (before the purge). When the valueset is freed (because empty) a safety checking
detects incoherent values based on the wrong counter.
Fix Description:
When all the values have been purge reset the counter before freeing the valueset
https://pagure.io/389-ds-base/issue/51082
Reviewed by: Mark Reynolds
Platforms tested: F30
Flag Day: no
Doc impact: no
---
.../tests/suites/replication/acceptance_test.py | 57 ++++++++++++++++++++++
ldap/servers/slapd/valueset.c | 4 ++
2 files changed, 61 insertions(+)
diff --git a/dirsrvtests/tests/suites/replication/acceptance_test.py b/dirsrvtests/tests/suites/replication/acceptance_test.py
index 943e6ab..c5cee0c 100644
--- a/dirsrvtests/tests/suites/replication/acceptance_test.py
+++ b/dirsrvtests/tests/suites/replication/acceptance_test.py
@@ -499,6 +499,63 @@ def test_warining_for_invalid_replica(topo_m4):
assert topo_m4.ms["master1"].ds_error_log.match('.*nsds5ReplicaBackoffMax.*10.*invalid.*')
+(a)pytest.mark.ds51082
+def test_csnpurge_large_valueset(topo_m2):
+ """Test csn generator test
+
+ :id: 63e2bdb2-0a8f-4660-9465-7b80a9f72a74
+ :setup: MMR with 2 masters
+ :steps:
+ 1. Create a test_user
+ 2. add a large set of values (more than 10)
+ 3. delete all the values (more than 10)
+ 4. configure the replica to purge those values (purgedelay=5s)
+ 5. Waiting for 6 second
+ 6. do a series of update
+ :expectedresults:
+ 1. Should succeeds
+ 2. Should succeeds
+ 3. Should succeeds
+ 4. Should succeeds
+ 5. Should succeeds
+ 6. Should not crash
+ """
+ m1 = topo_m2.ms["master2"]
+
+ test_user = UserAccount(m1, TEST_ENTRY_DN)
+ if test_user.exists():
+ log.info('Deleting entry {}'.format(TEST_ENTRY_DN))
+ test_user.delete()
+ test_user.create(properties={
+ 'uid': TEST_ENTRY_NAME,
+ 'cn': TEST_ENTRY_NAME,
+ 'sn': TEST_ENTRY_NAME,
+ 'userPassword': TEST_ENTRY_NAME,
+ 'uidNumber' : '1000',
+ 'gidNumber' : '2000',
+ 'homeDirectory' : '/home/mmrepl_test',
+ })
+
+ # create a large value set so that it is sorted
+ for i in range(1,20):
+ test_user.add('description', 'value {}'.format(str(i)))
+
+ # delete all values of the valueset
+ for i in range(1,20):
+ test_user.remove('description', 'value {}'.format(str(i)))
+
+ # set purging delay to 5 second and wait more that 5second
+ replicas = Replicas(m1)
+ replica = replicas.list()[0]
+ log.info('nsds5ReplicaPurgeDelay to 5')
+ replica.set('nsds5ReplicaPurgeDelay', '5')
+ time.sleep(6)
+
+ # add some new values to the valueset containing entries that should be purged
+ for i in range(21,25):
+ test_user.add('description', 'value {}'.format(str(i)))
+
+
if __name__ == '__main__':
# Run isolated
# -s for DEBUG mode
diff --git a/ldap/servers/slapd/valueset.c b/ldap/servers/slapd/valueset.c
index 2af3ee1..12027ec 100644
--- a/ldap/servers/slapd/valueset.c
+++ b/ldap/servers/slapd/valueset.c
@@ -801,6 +801,10 @@ valueset_array_purge(const Slapi_Attr *a, Slapi_ValueSet *vs, const CSN *csn)
}
}
} else {
+ /* empty valueset - reset the vs->num so that further
+ * checking will not abort
+ */
+ vs->num = 0;
slapi_valueset_done(vs);
}
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] branch 389-ds-base-1.4.2 updated: Ticket 51082 - abort when a empty valueset is freed
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
tbordaz pushed a commit to branch 389-ds-base-1.4.2
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.2 by this push:
new 469e141 Ticket 51082 - abort when a empty valueset is freed
469e141 is described below
commit 469e14132a40a005f93791826a7dd8508dfcafe8
Author: Thierry Bordaz <tbordaz(a)redhat.com>
AuthorDate: Mon May 11 17:11:49 2020 +0200
Ticket 51082 - abort when a empty valueset is freed
Bug Description:
A large valueset (more than 10 values) manages a sorted array of values.
replication purges old values from a valueset (valueset_array_purge). If it purges all the values
the valueset is freed (slapi_valueset_done).
A problem is that the counter of values, in the valueset, is still reflecting the initial number
of values (before the purge). When the valueset is freed (because empty) a safety checking
detects incoherent values based on the wrong counter.
Fix Description:
When all the values have been purge reset the counter before freeing the valueset
https://pagure.io/389-ds-base/issue/51082
Reviewed by: Mark Reynolds
Platforms tested: F30
Flag Day: no
Doc impact: no
---
.../tests/suites/replication/acceptance_test.py | 57 ++++++++++++++++++++++
ldap/servers/slapd/valueset.c | 4 ++
2 files changed, 61 insertions(+)
diff --git a/dirsrvtests/tests/suites/replication/acceptance_test.py b/dirsrvtests/tests/suites/replication/acceptance_test.py
index c8e0a4c..5009f4e 100644
--- a/dirsrvtests/tests/suites/replication/acceptance_test.py
+++ b/dirsrvtests/tests/suites/replication/acceptance_test.py
@@ -500,6 +500,63 @@ def test_warining_for_invalid_replica(topo_m4):
assert topo_m4.ms["master1"].ds_error_log.match('.*nsds5ReplicaBackoffMax.*10.*invalid.*')
+(a)pytest.mark.ds51082
+def test_csnpurge_large_valueset(topo_m2):
+ """Test csn generator test
+
+ :id: 63e2bdb2-0a8f-4660-9465-7b80a9f72a74
+ :setup: MMR with 2 masters
+ :steps:
+ 1. Create a test_user
+ 2. add a large set of values (more than 10)
+ 3. delete all the values (more than 10)
+ 4. configure the replica to purge those values (purgedelay=5s)
+ 5. Waiting for 6 second
+ 6. do a series of update
+ :expectedresults:
+ 1. Should succeeds
+ 2. Should succeeds
+ 3. Should succeeds
+ 4. Should succeeds
+ 5. Should succeeds
+ 6. Should not crash
+ """
+ m1 = topo_m2.ms["master2"]
+
+ test_user = UserAccount(m1, TEST_ENTRY_DN)
+ if test_user.exists():
+ log.info('Deleting entry {}'.format(TEST_ENTRY_DN))
+ test_user.delete()
+ test_user.create(properties={
+ 'uid': TEST_ENTRY_NAME,
+ 'cn': TEST_ENTRY_NAME,
+ 'sn': TEST_ENTRY_NAME,
+ 'userPassword': TEST_ENTRY_NAME,
+ 'uidNumber' : '1000',
+ 'gidNumber' : '2000',
+ 'homeDirectory' : '/home/mmrepl_test',
+ })
+
+ # create a large value set so that it is sorted
+ for i in range(1,20):
+ test_user.add('description', 'value {}'.format(str(i)))
+
+ # delete all values of the valueset
+ for i in range(1,20):
+ test_user.remove('description', 'value {}'.format(str(i)))
+
+ # set purging delay to 5 second and wait more that 5second
+ replicas = Replicas(m1)
+ replica = replicas.list()[0]
+ log.info('nsds5ReplicaPurgeDelay to 5')
+ replica.set('nsds5ReplicaPurgeDelay', '5')
+ time.sleep(6)
+
+ # add some new values to the valueset containing entries that should be purged
+ for i in range(21,25):
+ test_user.add('description', 'value {}'.format(str(i)))
+
+
if __name__ == '__main__':
# Run isolated
# -s for DEBUG mode
diff --git a/ldap/servers/slapd/valueset.c b/ldap/servers/slapd/valueset.c
index 2af3ee1..12027ec 100644
--- a/ldap/servers/slapd/valueset.c
+++ b/ldap/servers/slapd/valueset.c
@@ -801,6 +801,10 @@ valueset_array_purge(const Slapi_Attr *a, Slapi_ValueSet *vs, const CSN *csn)
}
}
} else {
+ /* empty valueset - reset the vs->num so that further
+ * checking will not abort
+ */
+ vs->num = 0;
slapi_valueset_done(vs);
}
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] branch 389-ds-base-1.4.3 updated: Ticket 51082 - abort when a empty valueset is freed
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
tbordaz pushed a commit to branch 389-ds-base-1.4.3
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.3 by this push:
new 280ba7e Ticket 51082 - abort when a empty valueset is freed
280ba7e is described below
commit 280ba7e50231c968b0f903dcab5f412d13c144be
Author: Thierry Bordaz <tbordaz(a)redhat.com>
AuthorDate: Mon May 11 17:11:49 2020 +0200
Ticket 51082 - abort when a empty valueset is freed
Bug Description:
A large valueset (more than 10 values) manages a sorted array of values.
replication purges old values from a valueset (valueset_array_purge). If it purges all the values
the valueset is freed (slapi_valueset_done).
A problem is that the counter of values, in the valueset, is still reflecting the initial number
of values (before the purge). When the valueset is freed (because empty) a safety checking
detects incoherent values based on the wrong counter.
Fix Description:
When all the values have been purge reset the counter before freeing the valueset
https://pagure.io/389-ds-base/issue/51082
Reviewed by: Mark Reynolds
Platforms tested: F30
Flag Day: no
Doc impact: no
---
.../tests/suites/replication/acceptance_test.py | 57 ++++++++++++++++++++++
ldap/servers/slapd/valueset.c | 4 ++
2 files changed, 61 insertions(+)
diff --git a/dirsrvtests/tests/suites/replication/acceptance_test.py b/dirsrvtests/tests/suites/replication/acceptance_test.py
index c8e0a4c..5009f4e 100644
--- a/dirsrvtests/tests/suites/replication/acceptance_test.py
+++ b/dirsrvtests/tests/suites/replication/acceptance_test.py
@@ -500,6 +500,63 @@ def test_warining_for_invalid_replica(topo_m4):
assert topo_m4.ms["master1"].ds_error_log.match('.*nsds5ReplicaBackoffMax.*10.*invalid.*')
+(a)pytest.mark.ds51082
+def test_csnpurge_large_valueset(topo_m2):
+ """Test csn generator test
+
+ :id: 63e2bdb2-0a8f-4660-9465-7b80a9f72a74
+ :setup: MMR with 2 masters
+ :steps:
+ 1. Create a test_user
+ 2. add a large set of values (more than 10)
+ 3. delete all the values (more than 10)
+ 4. configure the replica to purge those values (purgedelay=5s)
+ 5. Waiting for 6 second
+ 6. do a series of update
+ :expectedresults:
+ 1. Should succeeds
+ 2. Should succeeds
+ 3. Should succeeds
+ 4. Should succeeds
+ 5. Should succeeds
+ 6. Should not crash
+ """
+ m1 = topo_m2.ms["master2"]
+
+ test_user = UserAccount(m1, TEST_ENTRY_DN)
+ if test_user.exists():
+ log.info('Deleting entry {}'.format(TEST_ENTRY_DN))
+ test_user.delete()
+ test_user.create(properties={
+ 'uid': TEST_ENTRY_NAME,
+ 'cn': TEST_ENTRY_NAME,
+ 'sn': TEST_ENTRY_NAME,
+ 'userPassword': TEST_ENTRY_NAME,
+ 'uidNumber' : '1000',
+ 'gidNumber' : '2000',
+ 'homeDirectory' : '/home/mmrepl_test',
+ })
+
+ # create a large value set so that it is sorted
+ for i in range(1,20):
+ test_user.add('description', 'value {}'.format(str(i)))
+
+ # delete all values of the valueset
+ for i in range(1,20):
+ test_user.remove('description', 'value {}'.format(str(i)))
+
+ # set purging delay to 5 second and wait more that 5second
+ replicas = Replicas(m1)
+ replica = replicas.list()[0]
+ log.info('nsds5ReplicaPurgeDelay to 5')
+ replica.set('nsds5ReplicaPurgeDelay', '5')
+ time.sleep(6)
+
+ # add some new values to the valueset containing entries that should be purged
+ for i in range(21,25):
+ test_user.add('description', 'value {}'.format(str(i)))
+
+
if __name__ == '__main__':
# Run isolated
# -s for DEBUG mode
diff --git a/ldap/servers/slapd/valueset.c b/ldap/servers/slapd/valueset.c
index 2af3ee1..12027ec 100644
--- a/ldap/servers/slapd/valueset.c
+++ b/ldap/servers/slapd/valueset.c
@@ -801,6 +801,10 @@ valueset_array_purge(const Slapi_Attr *a, Slapi_ValueSet *vs, const CSN *csn)
}
}
} else {
+ /* empty valueset - reset the vs->num so that further
+ * checking will not abort
+ */
+ vs->num = 0;
slapi_valueset_done(vs);
}
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] 01/01: Bump version to 1.4.1.19
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mreynolds pushed a commit to tag 389-ds-base-1.4.1.19
in repository 389-ds-base.
commit 1e5b255a40c0026ae0ea45957afe7b451c7caad5
Author: Mark Reynolds <mreynolds(a)redhat.com>
AuthorDate: Fri May 8 17:10:24 2020 -0400
Bump version to 1.4.1.19
---
VERSION.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/VERSION.sh b/VERSION.sh
index acf81f5..8f4be8a 100644
--- a/VERSION.sh
+++ b/VERSION.sh
@@ -10,7 +10,7 @@ vendor="389 Project"
# PACKAGE_VERSION is constructed from these
VERSION_MAJOR=1
VERSION_MINOR=4
-VERSION_MAINT=1.18
+VERSION_MAINT=1.19
# NOTE: VERSION_PREREL is automatically set for builds made out of a git tree
VERSION_PREREL=
VERSION_DATE=$(date -u +%Y%m%d)
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] branch 389-ds-base-1.4.1 updated: Bump version to 1.4.1.19
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mreynolds pushed a commit to branch 389-ds-base-1.4.1
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.1 by this push:
new b0b4a13 Bump version to 1.4.1.19
b0b4a13 is described below
commit b0b4a13c74cb4ca43c3fb32f0f54084999402f73
Author: Mark Reynolds <mreynolds(a)redhat.com>
AuthorDate: Tue May 12 08:19:41 2020 -0400
Bump version to 1.4.1.19
But this commit was not psuhed in time, so 1.4.1.19 really
starts after commit: 4cc6a99cf04444e6129ace1723a87769cc23a99a
---
VERSION.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/VERSION.sh b/VERSION.sh
index acf81f5..8f4be8a 100644
--- a/VERSION.sh
+++ b/VERSION.sh
@@ -10,7 +10,7 @@ vendor="389 Project"
# PACKAGE_VERSION is constructed from these
VERSION_MAJOR=1
VERSION_MINOR=4
-VERSION_MAINT=1.18
+VERSION_MAINT=1.19
# NOTE: VERSION_PREREL is automatically set for builds made out of a git tree
VERSION_PREREL=
VERSION_DATE=$(date -u +%Y%m%d)
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] branch 389-ds-base-1.4.1 updated: Issue 51076 - prevent unnecessarily duplication of the target entry
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mreynolds pushed a commit to branch 389-ds-base-1.4.1
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.1 by this push:
new 20b1e42 Issue 51076 - prevent unnecessarily duplication of the target entry
20b1e42 is described below
commit 20b1e42d3c6c3dafb8317be8adeee79d13713206
Author: Mark Reynolds <mreynolds(a)redhat.com>
AuthorDate: Fri May 8 10:52:43 2020 -0400
Issue 51076 - prevent unnecessarily duplication of the target entry
Bug Description: For any update operation the MEP plugin was calling
slapi_search_internal_get_entry() which duplicates
the entry it returns. In this case the entry is just
read from and discarded, but this entry is already
in the pblock (the PRE OP ENTRY).
Fix Description: Just grab the PRE OP ENTRY from the pblock and use
that to read the attribute values from. This saves
two entry duplications for every update operation
from MEP.
fixes: https://pagure.io/389-ds-base/issue/51076
Reviewed by: tbordaz & firstyear(Thanks!!)
---
ldap/servers/plugins/mep/mep.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/ldap/servers/plugins/mep/mep.c b/ldap/servers/plugins/mep/mep.c
index ca9a64b..401d95e 100644
--- a/ldap/servers/plugins/mep/mep.c
+++ b/ldap/servers/plugins/mep/mep.c
@@ -2165,9 +2165,8 @@ mep_pre_op(Slapi_PBlock *pb, int modop)
if (e && free_entry) {
slapi_entry_free(e);
}
-
- slapi_search_internal_get_entry(sdn, 0, &e, mep_get_plugin_id());
- free_entry = 1;
+ slapi_pblock_get(pb, SLAPI_ENTRY_PRE_OP, &e);
+ free_entry = 0;
}
if (e && mep_is_managed_entry(e)) {
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months
[389-ds-base] branch 389-ds-base-1.4.2 updated: Issue 51076 - prevent unnecessarily duplication of the target entry
by pagure@pagure.io
This is an automated email from the git hooks/post-receive script.
mreynolds pushed a commit to branch 389-ds-base-1.4.2
in repository 389-ds-base.
The following commit(s) were added to refs/heads/389-ds-base-1.4.2 by this push:
new 21aa9ab Issue 51076 - prevent unnecessarily duplication of the target entry
21aa9ab is described below
commit 21aa9ab5647e49aac248fbef427ee9295eeaea19
Author: Mark Reynolds <mreynolds(a)redhat.com>
AuthorDate: Fri May 8 10:52:43 2020 -0400
Issue 51076 - prevent unnecessarily duplication of the target entry
Bug Description: For any update operation the MEP plugin was calling
slapi_search_internal_get_entry() which duplicates
the entry it returns. In this case the entry is just
read from and discarded, but this entry is already
in the pblock (the PRE OP ENTRY).
Fix Description: Just grab the PRE OP ENTRY from the pblock and use
that to read the attribute values from. This saves
two entry duplications for every update operation
from MEP.
fixes: https://pagure.io/389-ds-base/issue/51076
Reviewed by: tbordaz & firstyear(Thanks!!)
---
ldap/servers/plugins/mep/mep.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/ldap/servers/plugins/mep/mep.c b/ldap/servers/plugins/mep/mep.c
index ca9a64b..401d95e 100644
--- a/ldap/servers/plugins/mep/mep.c
+++ b/ldap/servers/plugins/mep/mep.c
@@ -2165,9 +2165,8 @@ mep_pre_op(Slapi_PBlock *pb, int modop)
if (e && free_entry) {
slapi_entry_free(e);
}
-
- slapi_search_internal_get_entry(sdn, 0, &e, mep_get_plugin_id());
- free_entry = 1;
+ slapi_pblock_get(pb, SLAPI_ENTRY_PRE_OP, &e);
+ free_entry = 0;
}
if (e && mep_is_managed_entry(e)) {
--
To stop receiving notification emails like this one, please contact
the administrator of this repository.
3 years, 11 months