389-commits November 2015

389-commits@lists.fedoraproject.org

4 participants
48 discussions

Branch '389-ds-base-1.3.2' - ldap/servers
by Mark Reynolds 07 Nov '15

07 Nov '15

ldap/servers/plugins/replication/repl5_replica_config.c | 1 + 1 file changed, 1 insertion(+) New commits: commit 378de6aabc37e2efa1176b8b41e2dd661c750036 Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Fri Nov 6 14:41:36 2015 -0500 Ticket 48325 - Replica promotion leaves RUV out of order Bug Description: When promoting a consumer to a master the new RUV element is appended to the RUV. However, when trying to replicate from the newly promoted replica the remote replica checks the first element in the RUV and sees that its the same replica ID, and aborts the replication session. Essentailly this completely breaks replication between the two servers, and can actually corrupt other RUVs on other replicas. Fix Description: When promoting a replica to a master, reorder the RUV so that it is the first in the list. https://fedorahosted.org/389/ticket/48325 Reviewed by: nhosoi(Thanks!) (cherry picked from commit b896840d270a540698f35a4aac4f7a91742952b0) diff --git a/ldap/servers/plugins/replication/repl5_replica_config.c b/ldap/servers/plugins/replication/repl5_replica_config.c index 755097a..08fd9b3 100644 --- a/ldap/servers/plugins/replication/repl5_replica_config.c +++ b/ldap/servers/plugins/replication/repl5_replica_config.c @@ -983,6 +983,7 @@ replica_config_change_type_and_id (Replica *r, const char *new_type, csngen_rewrite_rid(gen, rid); if(purl && type == REPLICA_TYPE_UPDATABLE){ ruv_add_replica(ruv, rid, purl); + ruv_move_local_supplier_to_first(ruv, rid); replica_reset_csn_pl(r); } ruv_delete_replica(ruv, oldrid);

1 0

Branch '389-ds-base-1.3.3' - ldap/servers
by Mark Reynolds 07 Nov '15

07 Nov '15

ldap/servers/plugins/replication/repl5_replica_config.c | 1 + 1 file changed, 1 insertion(+) New commits: commit 2764bc455bef2e88713fc014d917c74df9c484ae Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Fri Nov 6 14:41:36 2015 -0500 Ticket 48325 - Replica promotion leaves RUV out of order Bug Description: When promoting a consumer to a master the new RUV element is appended to the RUV. However, when trying to replicate from the newly promoted replica the remote replica checks the first element in the RUV and sees that its the same replica ID, and aborts the replication session. Essentailly this completely breaks replication between the two servers, and can actually corrupt other RUVs on other replicas. Fix Description: When promoting a replica to a master, reorder the RUV so that it is the first in the list. https://fedorahosted.org/389/ticket/48325 Reviewed by: nhosoi(Thanks!) (cherry picked from commit b896840d270a540698f35a4aac4f7a91742952b0) diff --git a/ldap/servers/plugins/replication/repl5_replica_config.c b/ldap/servers/plugins/replication/repl5_replica_config.c index 8718f92..849184f 100644 --- a/ldap/servers/plugins/replication/repl5_replica_config.c +++ b/ldap/servers/plugins/replication/repl5_replica_config.c @@ -1032,6 +1032,7 @@ replica_config_change_type_and_id (Replica *r, const char *new_type, csngen_rewrite_rid(gen, rid); if(purl && type == REPLICA_TYPE_UPDATABLE){ ruv_add_replica(ruv, rid, purl); + ruv_move_local_supplier_to_first(ruv, rid); replica_reset_csn_pl(r); } ruv_delete_replica(ruv, oldrid);

1 0

Branch '389-ds-base-1.3.4' - ldap/servers
by Mark Reynolds 07 Nov '15

07 Nov '15

ldap/servers/plugins/replication/repl5_replica_config.c | 1 + 1 file changed, 1 insertion(+) New commits: commit 6180b91c3f65e9c5e375816a72baa95678458a0a Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Fri Nov 6 14:41:36 2015 -0500 Ticket 48325 - Replica promotion leaves RUV out of order Bug Description: When promoting a consumer to a master the new RUV element is appended to the RUV. However, when trying to replicate from the newly promoted replica the remote replica checks the first element in the RUV and sees that its the same replica ID, and aborts the replication session. Essentailly this completely breaks replication between the two servers, and can actually corrupt other RUVs on other replicas. Fix Description: When promoting a replica to a master, reorder the RUV so that it is the first in the list. https://fedorahosted.org/389/ticket/48325 Reviewed by: nhosoi(Thanks!) (cherry picked from commit b896840d270a540698f35a4aac4f7a91742952b0) diff --git a/ldap/servers/plugins/replication/repl5_replica_config.c b/ldap/servers/plugins/replication/repl5_replica_config.c index e85ae3e..4d7135c 100644 --- a/ldap/servers/plugins/replication/repl5_replica_config.c +++ b/ldap/servers/plugins/replication/repl5_replica_config.c @@ -1003,6 +1003,7 @@ replica_config_change_type_and_id (Replica *r, const char *new_type, csngen_rewrite_rid(gen, rid); if(purl && type == REPLICA_TYPE_UPDATABLE){ ruv_add_replica(ruv, rid, purl); + ruv_move_local_supplier_to_first(ruv, rid); replica_reset_csn_pl(r); } ruv_delete_replica(ruv, oldrid);

1 0

ldap/servers
by Mark Reynolds 07 Nov '15

07 Nov '15

ldap/servers/plugins/replication/repl5_replica_config.c | 1 + 1 file changed, 1 insertion(+) New commits: commit b896840d270a540698f35a4aac4f7a91742952b0 Author: Mark Reynolds <mreynolds(a)redhat.com> Date: Fri Nov 6 14:41:36 2015 -0500 Ticket 48325 - Replica promotion leaves RUV out of order Bug Description: When promoting a consumer to a master the new RUV element is appended to the RUV. However, when trying to replicate from the newly promoted replica the remote replica checks the first element in the RUV and sees that its the same replica ID, and aborts the replication session. Essentailly this completely breaks replication between the two servers, and can actually corrupt other RUVs on other replicas. Fix Description: When promoting a replica to a master, reorder the RUV so that it is the first in the list. https://fedorahosted.org/389/ticket/48325 Reviewed by: nhosoi(Thanks!) diff --git a/ldap/servers/plugins/replication/repl5_replica_config.c b/ldap/servers/plugins/replication/repl5_replica_config.c index e85ae3e..4d7135c 100644 --- a/ldap/servers/plugins/replication/repl5_replica_config.c +++ b/ldap/servers/plugins/replication/repl5_replica_config.c @@ -1003,6 +1003,7 @@ replica_config_change_type_and_id (Replica *r, const char *new_type, csngen_rewrite_rid(gen, rid); if(purl && type == REPLICA_TYPE_UPDATABLE){ ruv_add_replica(ruv, rid, purl); + ruv_move_local_supplier_to_first(ruv, rid); replica_reset_csn_pl(r); } ruv_delete_replica(ruv, oldrid);

1 0

ldap/servers
by Noriko Hosoi 05 Nov '15

05 Nov '15

ldap/servers/slapd/daemon.c | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) New commits: commit ab8ed9a5ebb0d15b55d7525ed1d5dbeebd8c7563 Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Thu Nov 5 13:08:56 2015 -0800 Ticket #48339 - Share nsslapd-threadnumber in the case nunc-stans is enabled, as well. Description: When nunc-stans is enabled, instead of getting the thread number from the environment variable MAX_THREADS, use the value of config parameter nsslapd-threadnumber. https://fedorahosted.org/389/ticket/48339 Reviewed by rmeggins(a)redhat.com (Thank you, Rich!!) diff --git a/ldap/servers/slapd/daemon.c b/ldap/servers/slapd/daemon.c index 82099bc..90f0523 100644 --- a/ldap/servers/slapd/daemon.c +++ b/ldap/servers/slapd/daemon.c @@ -1199,10 +1199,7 @@ void slapd_daemon( daemon_ports_t *ports ) #ifdef ENABLE_NUNC_STANS if (enable_nunc_stans && !g_get_shutdown()) { int ii; - PRInt32 maxthreads = 3; - if (getenv("MAX_THREADS")) { - maxthreads = atoi(getenv("MAX_THREADS")); - } + PRInt32 maxthreads = (PRInt32)config_get_threadnumber(); /* Set the nunc-stans thread pool config */ ns_thrpool_config_init(&tp_config);

1 0

Branch '389-ds-base-1.2.11' - ldap/servers
by Noriko Hosoi 05 Nov '15

05 Nov '15

ldap/servers/slapd/opshared.c | 13 ++++++++----- ldap/servers/slapd/pagedresults.c | 28 ++++++++++++++++++---------- ldap/servers/slapd/proto-slap.h | 5 ++--- 3 files changed, 28 insertions(+), 18 deletions(-) New commits: commit d5ccc06cefa6f423d80c6cc0e527b57c116b0cb8 Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Thu Nov 5 10:44:08 2015 -0800 Ticket #48338 - SimplePagedResults -- abandon could happen between the abandon check and sending results Description: An abandon request for a SimplePagedResults request could happened between the abandon check and the code for sending the search results. The abandon frees the search results although sending result code still refers it. Fix description: The code (from getting search results through sending the search results) in op_shared_search is protected by c_mutex locking. https://fedorahosted.org/389/ticket/48338 Reviewed by rmeggins(a)redhat.com (Thank you, Rich!!) (cherry picked from commit 390b8bd9076e8976facc0858e60985d6b4fac05c) (cherry picked from commit 8f49d33d30fade7b579062414250a0ddb1a66c62) (cherry picked from commit a7a8b2d3cb8452cc629ec5228bea6571d72e6538) diff --git a/ldap/servers/slapd/opshared.c b/ldap/servers/slapd/opshared.c index 31d41e8..391d1c3 100644 --- a/ldap/servers/slapd/opshared.c +++ b/ldap/servers/slapd/opshared.c @@ -530,7 +530,7 @@ op_shared_search (Slapi_PBlock *pb, int send_result) be = be_list[index]; } } - pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, pr_idx); + pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, 0/*not locked*/, pr_idx); estimate = pagedresults_get_search_result_set_size_estimate(pb->pb_conn, operation, pr_idx); if (pagedresults_get_unindexed(pb->pb_conn, operation, pr_idx)) { opnote |= SLAPI_OP_NOTE_UNINDEXED; @@ -700,13 +700,15 @@ op_shared_search (Slapi_PBlock *pb, int send_result) * In async paged result case, the search result might be released * by other theads. We need to double check it in the locked region. */ - pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, pr_idx); + PR_Lock(pb->pb_conn->c_mutex); + pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, 1/*locked*/, pr_idx); if (pr_search_result) { - if (pagedresults_is_abandoned_or_notavailable(pb->pb_conn, pr_idx)) { + if (pagedresults_is_abandoned_or_notavailable(pb->pb_conn, 1/*locked*/, pr_idx)) { pagedresults_unlock(pb->pb_conn, pr_idx); /* Previous operation was abandoned and the simplepaged object is not in use. */ send_ldap_result(pb, 0, NULL, "Simple Paged Results Search abandoned", 0, NULL); rc = LDAP_SUCCESS; + PR_Unlock(pb->pb_conn->c_mutex); goto free_and_return; } else { slapi_pblock_set( pb, SLAPI_SEARCH_RESULT_SET, pr_search_result ); @@ -714,7 +716,8 @@ op_shared_search (Slapi_PBlock *pb, int send_result) /* search result could be reset in the backend/dse */ slapi_pblock_get(pb, SLAPI_SEARCH_RESULT_SET, &sr); - pagedresults_set_search_result(pb->pb_conn, operation, sr, 0, pr_idx); + pagedresults_set_search_result(pb->pb_conn, operation, sr, 1/*locked*/, pr_idx); + PR_Unlock(pb->pb_conn->c_mutex); } } else { pr_stat = PAGEDRESULTS_SEARCH_END; @@ -745,7 +748,7 @@ op_shared_search (Slapi_PBlock *pb, int send_result) if (PAGEDRESULTS_SEARCH_END == pr_stat) { pagedresults_lock(pb->pb_conn, pr_idx); slapi_pblock_set(pb, SLAPI_SEARCH_RESULT_SET, NULL); - if (!pagedresults_is_abandoned_or_notavailable(pb->pb_conn, pr_idx)) { + if (!pagedresults_is_abandoned_or_notavailable(pb->pb_conn, 0/*not locked*/, pr_idx)) { pagedresults_free_one(pb->pb_conn, operation, pr_idx); } pagedresults_unlock(pb->pb_conn, pr_idx); diff --git a/ldap/servers/slapd/pagedresults.c b/ldap/servers/slapd/pagedresults.c index 434e48d..c1947f1 100644 --- a/ldap/servers/slapd/pagedresults.c +++ b/ldap/servers/slapd/pagedresults.c @@ -408,20 +408,25 @@ pagedresults_set_current_be(Connection *conn, Slapi_Backend *be, int index, int } void * -pagedresults_get_search_result(Connection *conn, Operation *op, int index) +pagedresults_get_search_result(Connection *conn, Operation *op, int locked, int index) { void *sr = NULL; if (!op_is_pagedresults(op)) { return sr; /* noop */ } - LDAPDebug1Arg(LDAP_DEBUG_TRACE, - "--> pagedresults_get_search_result: idx=%d\n", index); + LDAPDebug2Args(LDAP_DEBUG_TRACE, + "--> pagedresults_get_search_result(%s): idx=%d\n", + locked?"locked":"not locked", index); if (conn && (index > -1)) { - PR_Lock(conn->c_mutex); + if (!locked) { + PR_Lock(conn->c_mutex); + } if (index < conn->c_pagedresults.prl_maxlen) { sr = conn->c_pagedresults.prl_list[index].pr_search_result_set; } - PR_Unlock(conn->c_mutex); + if (!locked) { + PR_Unlock(conn->c_mutex); + } } LDAPDebug1Arg(LDAP_DEBUG_TRACE, "<-- pagedresults_get_search_result: %p\n", sr); @@ -429,8 +434,7 @@ pagedresults_get_search_result(Connection *conn, Operation *op, int index) } int -pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, - int locked, int index) +pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, int locked, int index) { int rc = -1; if (!op_is_pagedresults(op)) { @@ -1016,15 +1020,19 @@ pagedresults_unlock( Connection *conn, int index ) } int -pagedresults_is_abandoned_or_notavailable( Connection *conn, int index ) +pagedresults_is_abandoned_or_notavailable(Connection *conn, int locked, int index) { PagedResults *prp; if (!conn || (index < 0) || (index >= conn->c_pagedresults.prl_maxlen)) { return 1; /* not abandoned, but do not want to proceed paged results op. */ } - PR_Lock(conn->c_mutex); + if (!locked) { + PR_Lock(conn->c_mutex); + } prp = conn->c_pagedresults.prl_list + index; - PR_Unlock(conn->c_mutex); + if (!locked) { + PR_Unlock(conn->c_mutex); + } return prp->pr_flags & CONN_FLAG_PAGEDRESULTS_ABANDONED; } diff --git a/ldap/servers/slapd/proto-slap.h b/ldap/servers/slapd/proto-slap.h index 8f86eb7..decc29e 100644 --- a/ldap/servers/slapd/proto-slap.h +++ b/ldap/servers/slapd/proto-slap.h @@ -1438,8 +1438,7 @@ void pagedresults_set_response_control(Slapi_PBlock *pb, int iscritical, int curr_search_count, int index); Slapi_Backend *pagedresults_get_current_be(Connection *conn, int index); int pagedresults_set_current_be(Connection *conn, Slapi_Backend *be, int index, int nolock); -void *pagedresults_get_search_result(Connection *conn, Operation *op, - int index); +void *pagedresults_get_search_result(Connection *conn, Operation *op, int locked, int index); int pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, int locked, int index); int pagedresults_get_search_result_count(Connection *conn, Operation *op, @@ -1480,7 +1479,7 @@ int pagedresults_cleanup_all(Connection *conn, int needlock); void op_set_pagedresults(Operation *op); void pagedresults_lock(Connection *conn, int index); void pagedresults_unlock(Connection *conn, int index); -int pagedresults_is_abandoned_or_notavailable(Connection *conn, int index); +int pagedresults_is_abandoned_or_notavailable(Connection *conn, int locked, int index); int pagedresults_set_search_result_pb(Slapi_PBlock *pb, void *sr, int locked); /*

1 0

Branch '389-ds-base-1.3.3' - ldap/servers
by Noriko Hosoi 05 Nov '15

05 Nov '15

ldap/servers/slapd/opshared.c | 13 ++++++++----- ldap/servers/slapd/pagedresults.c | 28 ++++++++++++++++++---------- ldap/servers/slapd/proto-slap.h | 5 ++--- 3 files changed, 28 insertions(+), 18 deletions(-) New commits: commit a7a8b2d3cb8452cc629ec5228bea6571d72e6538 Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Thu Nov 5 10:44:08 2015 -0800 Ticket #48338 - SimplePagedResults -- abandon could happen between the abandon check and sending results Description: An abandon request for a SimplePagedResults request could happened between the abandon check and the code for sending the search results. The abandon frees the search results although sending result code still refers it. Fix description: The code (from getting search results through sending the search results) in op_shared_search is protected by c_mutex locking. https://fedorahosted.org/389/ticket/48338 Reviewed by rmeggins(a)redhat.com (Thank you, Rich!!) (cherry picked from commit 390b8bd9076e8976facc0858e60985d6b4fac05c) (cherry picked from commit 8f49d33d30fade7b579062414250a0ddb1a66c62) diff --git a/ldap/servers/slapd/opshared.c b/ldap/servers/slapd/opshared.c index fa506b3..873a78b 100644 --- a/ldap/servers/slapd/opshared.c +++ b/ldap/servers/slapd/opshared.c @@ -537,7 +537,7 @@ op_shared_search (Slapi_PBlock *pb, int send_result) be = be_list[index]; } } - pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, pr_idx); + pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, 0/*not locked*/, pr_idx); estimate = pagedresults_get_search_result_set_size_estimate(pb->pb_conn, operation, pr_idx); if (pagedresults_get_unindexed(pb->pb_conn, operation, pr_idx)) { opnote |= SLAPI_OP_NOTE_UNINDEXED; @@ -707,13 +707,15 @@ op_shared_search (Slapi_PBlock *pb, int send_result) * In async paged result case, the search result might be released * by other theads. We need to double check it in the locked region. */ - pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, pr_idx); + PR_Lock(pb->pb_conn->c_mutex); + pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, 1/*locked*/, pr_idx); if (pr_search_result) { - if (pagedresults_is_abandoned_or_notavailable(pb->pb_conn, pr_idx)) { + if (pagedresults_is_abandoned_or_notavailable(pb->pb_conn, 1/*locked*/, pr_idx)) { pagedresults_unlock(pb->pb_conn, pr_idx); /* Previous operation was abandoned and the simplepaged object is not in use. */ send_ldap_result(pb, 0, NULL, "Simple Paged Results Search abandoned", 0, NULL); rc = LDAP_SUCCESS; + PR_Unlock(pb->pb_conn->c_mutex); goto free_and_return; } else { slapi_pblock_set( pb, SLAPI_SEARCH_RESULT_SET, pr_search_result ); @@ -721,7 +723,8 @@ op_shared_search (Slapi_PBlock *pb, int send_result) /* search result could be reset in the backend/dse */ slapi_pblock_get(pb, SLAPI_SEARCH_RESULT_SET, &sr); - pagedresults_set_search_result(pb->pb_conn, operation, sr, 0, pr_idx); + pagedresults_set_search_result(pb->pb_conn, operation, sr, 1/*locked*/, pr_idx); + PR_Unlock(pb->pb_conn->c_mutex); } } else { pr_stat = PAGEDRESULTS_SEARCH_END; @@ -752,7 +755,7 @@ op_shared_search (Slapi_PBlock *pb, int send_result) if (PAGEDRESULTS_SEARCH_END == pr_stat) { pagedresults_lock(pb->pb_conn, pr_idx); slapi_pblock_set(pb, SLAPI_SEARCH_RESULT_SET, NULL); - if (!pagedresults_is_abandoned_or_notavailable(pb->pb_conn, pr_idx)) { + if (!pagedresults_is_abandoned_or_notavailable(pb->pb_conn, 0/*not locked*/, pr_idx)) { pagedresults_free_one(pb->pb_conn, operation, pr_idx); } pagedresults_unlock(pb->pb_conn, pr_idx); diff --git a/ldap/servers/slapd/pagedresults.c b/ldap/servers/slapd/pagedresults.c index 434e48d..c1947f1 100644 --- a/ldap/servers/slapd/pagedresults.c +++ b/ldap/servers/slapd/pagedresults.c @@ -408,20 +408,25 @@ pagedresults_set_current_be(Connection *conn, Slapi_Backend *be, int index, int } void * -pagedresults_get_search_result(Connection *conn, Operation *op, int index) +pagedresults_get_search_result(Connection *conn, Operation *op, int locked, int index) { void *sr = NULL; if (!op_is_pagedresults(op)) { return sr; /* noop */ } - LDAPDebug1Arg(LDAP_DEBUG_TRACE, - "--> pagedresults_get_search_result: idx=%d\n", index); + LDAPDebug2Args(LDAP_DEBUG_TRACE, + "--> pagedresults_get_search_result(%s): idx=%d\n", + locked?"locked":"not locked", index); if (conn && (index > -1)) { - PR_Lock(conn->c_mutex); + if (!locked) { + PR_Lock(conn->c_mutex); + } if (index < conn->c_pagedresults.prl_maxlen) { sr = conn->c_pagedresults.prl_list[index].pr_search_result_set; } - PR_Unlock(conn->c_mutex); + if (!locked) { + PR_Unlock(conn->c_mutex); + } } LDAPDebug1Arg(LDAP_DEBUG_TRACE, "<-- pagedresults_get_search_result: %p\n", sr); @@ -429,8 +434,7 @@ pagedresults_get_search_result(Connection *conn, Operation *op, int index) } int -pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, - int locked, int index) +pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, int locked, int index) { int rc = -1; if (!op_is_pagedresults(op)) { @@ -1016,15 +1020,19 @@ pagedresults_unlock( Connection *conn, int index ) } int -pagedresults_is_abandoned_or_notavailable( Connection *conn, int index ) +pagedresults_is_abandoned_or_notavailable(Connection *conn, int locked, int index) { PagedResults *prp; if (!conn || (index < 0) || (index >= conn->c_pagedresults.prl_maxlen)) { return 1; /* not abandoned, but do not want to proceed paged results op. */ } - PR_Lock(conn->c_mutex); + if (!locked) { + PR_Lock(conn->c_mutex); + } prp = conn->c_pagedresults.prl_list + index; - PR_Unlock(conn->c_mutex); + if (!locked) { + PR_Unlock(conn->c_mutex); + } return prp->pr_flags & CONN_FLAG_PAGEDRESULTS_ABANDONED; } diff --git a/ldap/servers/slapd/proto-slap.h b/ldap/servers/slapd/proto-slap.h index 5fa2310..b119546 100644 --- a/ldap/servers/slapd/proto-slap.h +++ b/ldap/servers/slapd/proto-slap.h @@ -1482,8 +1482,7 @@ void pagedresults_set_response_control(Slapi_PBlock *pb, int iscritical, int curr_search_count, int index); Slapi_Backend *pagedresults_get_current_be(Connection *conn, int index); int pagedresults_set_current_be(Connection *conn, Slapi_Backend *be, int index, int nolock); -void *pagedresults_get_search_result(Connection *conn, Operation *op, - int index); +void *pagedresults_get_search_result(Connection *conn, Operation *op, int locked, int index); int pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, int locked, int index); int pagedresults_get_search_result_count(Connection *conn, Operation *op, @@ -1524,7 +1523,7 @@ int pagedresults_cleanup_all(Connection *conn, int needlock); void op_set_pagedresults(Operation *op); void pagedresults_lock(Connection *conn, int index); void pagedresults_unlock(Connection *conn, int index); -int pagedresults_is_abandoned_or_notavailable(Connection *conn, int index); +int pagedresults_is_abandoned_or_notavailable(Connection *conn, int locked, int index); int pagedresults_set_search_result_pb(Slapi_PBlock *pb, void *sr, int locked); /*

1 0

Branch '389-ds-base-1.3.4' - ldap/servers
by Noriko Hosoi 05 Nov '15

05 Nov '15

ldap/servers/slapd/opshared.c | 13 ++++++++----- ldap/servers/slapd/pagedresults.c | 28 ++++++++++++++++++---------- ldap/servers/slapd/proto-slap.h | 5 ++--- 3 files changed, 28 insertions(+), 18 deletions(-) New commits: commit 8f49d33d30fade7b579062414250a0ddb1a66c62 Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Thu Nov 5 10:44:08 2015 -0800 Ticket #48338 - SimplePagedResults -- abandon could happen between the abandon check and sending results Description: An abandon request for a SimplePagedResults request could happened between the abandon check and the code for sending the search results. The abandon frees the search results although sending result code still refers it. Fix description: The code (from getting search results through sending the search results) in op_shared_search is protected by c_mutex locking. https://fedorahosted.org/389/ticket/48338 Reviewed by rmeggins(a)redhat.com (Thank you, Rich!!) (cherry picked from commit 390b8bd9076e8976facc0858e60985d6b4fac05c) diff --git a/ldap/servers/slapd/opshared.c b/ldap/servers/slapd/opshared.c index dcdbb04..586ca1f 100644 --- a/ldap/servers/slapd/opshared.c +++ b/ldap/servers/slapd/opshared.c @@ -500,7 +500,7 @@ op_shared_search (Slapi_PBlock *pb, int send_result) be = be_list[index]; } } - pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, pr_idx); + pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, 0/*not locked*/, pr_idx); estimate = pagedresults_get_search_result_set_size_estimate(pb->pb_conn, operation, pr_idx); if (pagedresults_get_unindexed(pb->pb_conn, operation, pr_idx)) { opnote |= SLAPI_OP_NOTE_UNINDEXED; @@ -675,13 +675,15 @@ op_shared_search (Slapi_PBlock *pb, int send_result) * In async paged result case, the search result might be released * by other theads. We need to double check it in the locked region. */ - pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, pr_idx); + PR_Lock(pb->pb_conn->c_mutex); + pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, 1/*locked*/, pr_idx); if (pr_search_result) { - if (pagedresults_is_abandoned_or_notavailable(pb->pb_conn, pr_idx)) { + if (pagedresults_is_abandoned_or_notavailable(pb->pb_conn, 1/*locked*/, pr_idx)) { pagedresults_unlock(pb->pb_conn, pr_idx); /* Previous operation was abandoned and the simplepaged object is not in use. */ send_ldap_result(pb, 0, NULL, "Simple Paged Results Search abandoned", 0, NULL); rc = LDAP_SUCCESS; + PR_Unlock(pb->pb_conn->c_mutex); goto free_and_return; } else { slapi_pblock_set( pb, SLAPI_SEARCH_RESULT_SET, pr_search_result ); @@ -689,7 +691,8 @@ op_shared_search (Slapi_PBlock *pb, int send_result) /* search result could be reset in the backend/dse */ slapi_pblock_get(pb, SLAPI_SEARCH_RESULT_SET, &sr); - pagedresults_set_search_result(pb->pb_conn, operation, sr, 0, pr_idx); + pagedresults_set_search_result(pb->pb_conn, operation, sr, 1/*locked*/, pr_idx); + PR_Unlock(pb->pb_conn->c_mutex); } } else { pr_stat = PAGEDRESULTS_SEARCH_END; @@ -720,7 +723,7 @@ op_shared_search (Slapi_PBlock *pb, int send_result) if (PAGEDRESULTS_SEARCH_END == pr_stat) { pagedresults_lock(pb->pb_conn, pr_idx); slapi_pblock_set(pb, SLAPI_SEARCH_RESULT_SET, NULL); - if (!pagedresults_is_abandoned_or_notavailable(pb->pb_conn, pr_idx)) { + if (!pagedresults_is_abandoned_or_notavailable(pb->pb_conn, 0/*not locked*/, pr_idx)) { pagedresults_free_one(pb->pb_conn, operation, pr_idx); } pagedresults_unlock(pb->pb_conn, pr_idx); diff --git a/ldap/servers/slapd/pagedresults.c b/ldap/servers/slapd/pagedresults.c index 87447c4..4458cfb 100644 --- a/ldap/servers/slapd/pagedresults.c +++ b/ldap/servers/slapd/pagedresults.c @@ -395,20 +395,25 @@ pagedresults_set_current_be(Connection *conn, Slapi_Backend *be, int index, int } void * -pagedresults_get_search_result(Connection *conn, Operation *op, int index) +pagedresults_get_search_result(Connection *conn, Operation *op, int locked, int index) { void *sr = NULL; if (!op_is_pagedresults(op)) { return sr; /* noop */ } - LDAPDebug1Arg(LDAP_DEBUG_TRACE, - "--> pagedresults_get_search_result: idx=%d\n", index); + LDAPDebug2Args(LDAP_DEBUG_TRACE, + "--> pagedresults_get_search_result(%s): idx=%d\n", + locked?"locked":"not locked", index); if (conn && (index > -1)) { - PR_Lock(conn->c_mutex); + if (!locked) { + PR_Lock(conn->c_mutex); + } if (index < conn->c_pagedresults.prl_maxlen) { sr = conn->c_pagedresults.prl_list[index].pr_search_result_set; } - PR_Unlock(conn->c_mutex); + if (!locked) { + PR_Unlock(conn->c_mutex); + } } LDAPDebug1Arg(LDAP_DEBUG_TRACE, "<-- pagedresults_get_search_result: %p\n", sr); @@ -416,8 +421,7 @@ pagedresults_get_search_result(Connection *conn, Operation *op, int index) } int -pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, - int locked, int index) +pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, int locked, int index) { int rc = -1; if (!op_is_pagedresults(op)) { @@ -1003,15 +1007,19 @@ pagedresults_unlock( Connection *conn, int index ) } int -pagedresults_is_abandoned_or_notavailable( Connection *conn, int index ) +pagedresults_is_abandoned_or_notavailable(Connection *conn, int locked, int index) { PagedResults *prp; if (!conn || (index < 0) || (index >= conn->c_pagedresults.prl_maxlen)) { return 1; /* not abandoned, but do not want to proceed paged results op. */ } - PR_Lock(conn->c_mutex); + if (!locked) { + PR_Lock(conn->c_mutex); + } prp = conn->c_pagedresults.prl_list + index; - PR_Unlock(conn->c_mutex); + if (!locked) { + PR_Unlock(conn->c_mutex); + } return prp->pr_flags & CONN_FLAG_PAGEDRESULTS_ABANDONED; } diff --git a/ldap/servers/slapd/proto-slap.h b/ldap/servers/slapd/proto-slap.h index b10c1eb..e1cb53e 100644 --- a/ldap/servers/slapd/proto-slap.h +++ b/ldap/servers/slapd/proto-slap.h @@ -1445,8 +1445,7 @@ void pagedresults_set_response_control(Slapi_PBlock *pb, int iscritical, int curr_search_count, int index); Slapi_Backend *pagedresults_get_current_be(Connection *conn, int index); int pagedresults_set_current_be(Connection *conn, Slapi_Backend *be, int index, int nolock); -void *pagedresults_get_search_result(Connection *conn, Operation *op, - int index); +void *pagedresults_get_search_result(Connection *conn, Operation *op, int locked, int index); int pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, int locked, int index); int pagedresults_get_search_result_count(Connection *conn, Operation *op, @@ -1487,7 +1486,7 @@ int pagedresults_cleanup_all(Connection *conn, int needlock); void op_set_pagedresults(Operation *op); void pagedresults_lock(Connection *conn, int index); void pagedresults_unlock(Connection *conn, int index); -int pagedresults_is_abandoned_or_notavailable(Connection *conn, int index); +int pagedresults_is_abandoned_or_notavailable(Connection *conn, int locked, int index); int pagedresults_set_search_result_pb(Slapi_PBlock *pb, void *sr, int locked); /*

1 0

ldap/servers
by Noriko Hosoi 05 Nov '15

05 Nov '15

ldap/servers/slapd/opshared.c | 13 ++++++++----- ldap/servers/slapd/pagedresults.c | 28 ++++++++++++++++++---------- ldap/servers/slapd/proto-slap.h | 5 ++--- 3 files changed, 28 insertions(+), 18 deletions(-) New commits: commit 390b8bd9076e8976facc0858e60985d6b4fac05c Author: Noriko Hosoi <nhosoi(a)redhat.com> Date: Thu Nov 5 10:44:08 2015 -0800 Ticket #48338 - SimplePagedResults -- abandon could happen between the abandon check and sending results Description: An abandon request for a SimplePagedResults request could happened between the abandon check and the code for sending the search results. The abandon frees the search results although sending result code still refers it. Fix description: The code (from getting search results through sending the search results) in op_shared_search is protected by c_mutex locking. https://fedorahosted.org/389/ticket/48338 Reviewed by rmeggins(a)redhat.com (Thank you, Rich!!) diff --git a/ldap/servers/slapd/opshared.c b/ldap/servers/slapd/opshared.c index 4fe7969..8a14431 100644 --- a/ldap/servers/slapd/opshared.c +++ b/ldap/servers/slapd/opshared.c @@ -500,7 +500,7 @@ op_shared_search (Slapi_PBlock *pb, int send_result) be = be_list[index]; } } - pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, pr_idx); + pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, 0/*not locked*/, pr_idx); estimate = pagedresults_get_search_result_set_size_estimate(pb->pb_conn, operation, pr_idx); if (pagedresults_get_unindexed(pb->pb_conn, operation, pr_idx)) { opnote |= SLAPI_OP_NOTE_UNINDEXED; @@ -675,13 +675,15 @@ op_shared_search (Slapi_PBlock *pb, int send_result) * In async paged result case, the search result might be released * by other theads. We need to double check it in the locked region. */ - pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, pr_idx); + PR_Lock(pb->pb_conn->c_mutex); + pr_search_result = pagedresults_get_search_result(pb->pb_conn, operation, 1/*locked*/, pr_idx); if (pr_search_result) { - if (pagedresults_is_abandoned_or_notavailable(pb->pb_conn, pr_idx)) { + if (pagedresults_is_abandoned_or_notavailable(pb->pb_conn, 1/*locked*/, pr_idx)) { pagedresults_unlock(pb->pb_conn, pr_idx); /* Previous operation was abandoned and the simplepaged object is not in use. */ send_ldap_result(pb, 0, NULL, "Simple Paged Results Search abandoned", 0, NULL); rc = LDAP_SUCCESS; + PR_Unlock(pb->pb_conn->c_mutex); goto free_and_return; } else { slapi_pblock_set( pb, SLAPI_SEARCH_RESULT_SET, pr_search_result ); @@ -689,7 +691,8 @@ op_shared_search (Slapi_PBlock *pb, int send_result) /* search result could be reset in the backend/dse */ slapi_pblock_get(pb, SLAPI_SEARCH_RESULT_SET, &sr); - pagedresults_set_search_result(pb->pb_conn, operation, sr, 0, pr_idx); + pagedresults_set_search_result(pb->pb_conn, operation, sr, 1/*locked*/, pr_idx); + PR_Unlock(pb->pb_conn->c_mutex); } } else { pr_stat = PAGEDRESULTS_SEARCH_END; @@ -720,7 +723,7 @@ op_shared_search (Slapi_PBlock *pb, int send_result) if (PAGEDRESULTS_SEARCH_END == pr_stat) { pagedresults_lock(pb->pb_conn, pr_idx); slapi_pblock_set(pb, SLAPI_SEARCH_RESULT_SET, NULL); - if (!pagedresults_is_abandoned_or_notavailable(pb->pb_conn, pr_idx)) { + if (!pagedresults_is_abandoned_or_notavailable(pb->pb_conn, 0/*not locked*/, pr_idx)) { pagedresults_free_one(pb->pb_conn, operation, pr_idx); } pagedresults_unlock(pb->pb_conn, pr_idx); diff --git a/ldap/servers/slapd/pagedresults.c b/ldap/servers/slapd/pagedresults.c index 87447c4..4458cfb 100644 --- a/ldap/servers/slapd/pagedresults.c +++ b/ldap/servers/slapd/pagedresults.c @@ -395,20 +395,25 @@ pagedresults_set_current_be(Connection *conn, Slapi_Backend *be, int index, int } void * -pagedresults_get_search_result(Connection *conn, Operation *op, int index) +pagedresults_get_search_result(Connection *conn, Operation *op, int locked, int index) { void *sr = NULL; if (!op_is_pagedresults(op)) { return sr; /* noop */ } - LDAPDebug1Arg(LDAP_DEBUG_TRACE, - "--> pagedresults_get_search_result: idx=%d\n", index); + LDAPDebug2Args(LDAP_DEBUG_TRACE, + "--> pagedresults_get_search_result(%s): idx=%d\n", + locked?"locked":"not locked", index); if (conn && (index > -1)) { - PR_Lock(conn->c_mutex); + if (!locked) { + PR_Lock(conn->c_mutex); + } if (index < conn->c_pagedresults.prl_maxlen) { sr = conn->c_pagedresults.prl_list[index].pr_search_result_set; } - PR_Unlock(conn->c_mutex); + if (!locked) { + PR_Unlock(conn->c_mutex); + } } LDAPDebug1Arg(LDAP_DEBUG_TRACE, "<-- pagedresults_get_search_result: %p\n", sr); @@ -416,8 +421,7 @@ pagedresults_get_search_result(Connection *conn, Operation *op, int index) } int -pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, - int locked, int index) +pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, int locked, int index) { int rc = -1; if (!op_is_pagedresults(op)) { @@ -1003,15 +1007,19 @@ pagedresults_unlock( Connection *conn, int index ) } int -pagedresults_is_abandoned_or_notavailable( Connection *conn, int index ) +pagedresults_is_abandoned_or_notavailable(Connection *conn, int locked, int index) { PagedResults *prp; if (!conn || (index < 0) || (index >= conn->c_pagedresults.prl_maxlen)) { return 1; /* not abandoned, but do not want to proceed paged results op. */ } - PR_Lock(conn->c_mutex); + if (!locked) { + PR_Lock(conn->c_mutex); + } prp = conn->c_pagedresults.prl_list + index; - PR_Unlock(conn->c_mutex); + if (!locked) { + PR_Unlock(conn->c_mutex); + } return prp->pr_flags & CONN_FLAG_PAGEDRESULTS_ABANDONED; } diff --git a/ldap/servers/slapd/proto-slap.h b/ldap/servers/slapd/proto-slap.h index c49f1d1..ff9dd09 100644 --- a/ldap/servers/slapd/proto-slap.h +++ b/ldap/servers/slapd/proto-slap.h @@ -1446,8 +1446,7 @@ void pagedresults_set_response_control(Slapi_PBlock *pb, int iscritical, int curr_search_count, int index); Slapi_Backend *pagedresults_get_current_be(Connection *conn, int index); int pagedresults_set_current_be(Connection *conn, Slapi_Backend *be, int index, int nolock); -void *pagedresults_get_search_result(Connection *conn, Operation *op, - int index); +void *pagedresults_get_search_result(Connection *conn, Operation *op, int locked, int index); int pagedresults_set_search_result(Connection *conn, Operation *op, void *sr, int locked, int index); int pagedresults_get_search_result_count(Connection *conn, Operation *op, @@ -1488,7 +1487,7 @@ int pagedresults_cleanup_all(Connection *conn, int needlock); void op_set_pagedresults(Operation *op); void pagedresults_lock(Connection *conn, int index); void pagedresults_unlock(Connection *conn, int index); -int pagedresults_is_abandoned_or_notavailable(Connection *conn, int index); +int pagedresults_is_abandoned_or_notavailable(Connection *conn, int locked, int index); int pagedresults_set_search_result_pb(Slapi_PBlock *pb, void *sr, int locked); /*

1 0

Branch '389-ds-base-1.3.4' - ldap/servers
by thierry bordaz 05 Nov '15

05 Nov '15

ldap/servers/slapd/back-ldbm/ldbm_delete.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) New commits: commit 55434d308b4e459ba3a169eff94568312dba767c Author: Thierry Bordaz <tbordaz(a)redhat.com> Date: Tue Nov 3 15:59:54 2015 +0100 Ticket 47976: deadlock in mep delete post op Bug Description: When deleting the original entry, some DB pages are acquired in write. The deadlock occurs because when reading the parent entry of the MEP entry MEP plugin requires read access to one of the page acquired by the deletion of the original entry. The read access can be granted if it is using the parent txn. This bug requires that the parent entry of the MEP entry is not found in the entry cache, so it requires database access Fix Description: Fix ldbm_delete, so that it reads id2entry db with parent txn https://fedorahosted.org/389/ticket/47976 Reviewed by: Ludwig Krispenz, Rich Megginson (Thanks you !!) Platforms tested: F17 Flag Day: no Doc impact: no diff --git a/ldap/servers/slapd/back-ldbm/ldbm_delete.c b/ldap/servers/slapd/back-ldbm/ldbm_delete.c index f31d545..100a71d 100644 --- a/ldap/servers/slapd/back-ldbm/ldbm_delete.c +++ b/ldap/servers/slapd/back-ldbm/ldbm_delete.c @@ -477,7 +477,7 @@ ldbm_back_delete( Slapi_PBlock *pb ) * the parent. If we fail to lock the entry, just try again. */ while(1){ - parent = id2entry(be, pid ,NULL, &retval); + parent = id2entry(be, pid ,&txn, &retval); if (parent && (cache_retry = cache_lock_entry(&inst->inst_cache, parent))) { /* Failed to obtain parent entry's entry lock */ if(cache_retry == RETRY_CACHE_LOCK &&

1 0

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

389-commits November 2015