trying to redump the stacktrace after 5 minutes,the stacktrace is complete match,could it be some kind of deadlock?

鐳鍶 <laysdra7265@gmail.com> 於 2020年3月10日 週二 下午5:44寫道:
Thanks for help,I encounter the same problem again today.
There is stacktrace,it is really really helpful to get the more detail of server
looks like server always hangs after request
   op=1 BIND dn="" method=sasl version=3 mech=GSSAPI  
Please take a look of  stacktrace.log  and access.log 

Thanks,
   Lays 

access.log
```
[10/Mar/2020:11:51:14.264988067 +0800] conn=10 op=8337 SRCH base="<masked>" scope=2 filter="(&(|(objectClass=krbprincipalaux)(objectClass=krbprincipal)(objectClass=ipakrbprincipal))(|(ipaKrbPrincipalAlias=krbtgt/<masked>@<masked>)(krbPrincipalName:caseIgnoreIA5Match:=krbtgt/<masked>@<masked>)))" attrs="krbPrincipalName krbCanonicalName krbUPEnabled krbPrincipalKey krbTicketPolicyReference krbPrincipalExpiration krbPasswordExpiration krbPwdPolicyReference krbPrincipalType krbPwdHistory krbLastPwdChange krbPrincipalAliases krbLastSuccessfulAuth krbLastFailedAuth krbLoginFailedCount krbPrincipalAuthInd krbExtraData krbLastAdminUnlock krbObjectReferences krbTicketFlags krbMaxTicketLife krbMaxRenewableAge nsAccountLock passwordHistory ipaKrbAuthzData ipaUserAuthType ipatokenRadiusConfigLink krbAuthIndMaxTicke..."
[10/Mar/2020:11:51:14.265362689 +0800] conn=10 op=8337 RESULT err=0 tag=101 nentries=1 etime=0.000487706
[10/Mar/2020:11:51:14.265475208 +0800] conn=10 op=8338 SRCH base="<masked>" scope=2 filter="(&(|(objectClass=krbprincipalaux)(objectClass=krbprincipal)(objectClass=ipakrbprincipal))(|(ipaKrbPrincipalAlias=ldap/ipa1.<masked>@<masked>)(krbPrincipalName:caseIgnoreIA5Match:=ldap/ipa1.<masked>@<masked>)))" attrs="krbPrincipalName krbCanonicalName krbUPEnabled krbPrincipalKey krbTicketPolicyReference krbPrincipalExpiration krbPasswordExpiration krbPwdPolicyReference krbPrincipalType krbPwdHistory krbLastPwdChange krbPrincipalAliases krbLastSuccessfulAuth krbLastFailedAuth krbLoginFailedCount krbPrincipalAuthInd krbExtraData krbLastAdminUnlock krbObjectReferences krbTicketFlags krbMaxTicketLife krbMaxRenewableAge nsAccountLock passwordHistory ipaKrbAuthzData ipaUserAuthType ipatokenRadiusConfigLink krbAuthIndMaxTicke..."
[10/Mar/2020:11:51:14.265713946 +0800] conn=10 op=8338 RESULT err=0 tag=101 nentries=1 etime=0.000385250
[10/Mar/2020:11:51:14.265799202 +0800] conn=10 op=8339 SRCH base="cn=<masked>,cn=kerberos,<masked>" scope=0 filter="(objectClass=krbticketpolicyaux)" attrs="krbMaxTicketLife krbMaxRenewableAge krbTicketFlags krbAuthIndMaxTicketLife krbAuthIndMaxRenewableAge"
[10/Mar/2020:11:51:14.265869305 +0800] conn=10 op=8339 RESULT err=0 tag=101 nentries=1 etime=0.000182813
[10/Mar/2020:11:51:14.266027805 +0800] conn=10 op=8340 SRCH base="<masked>" scope=2 filter="(&(|(objectClass=krbprincipalaux)(objectClass=krbprincipal))(krbPrincipalName=host/rancher1.<masked>@<masked>))" attrs="krbPrincipalName krbCanonicalName krbUPEnabled krbPrincipalKey krbTicketPolicyReference krbPrincipalExpiration krbPasswordExpiration krbPwdPolicyReference krbPrincipalType krbPwdHistory krbLastPwdChange krbPrincipalAliases krbLastSuccessfulAuth krbLastFailedAuth krbLoginFailedCount krbPrincipalAuthInd krbExtraData krbLastAdminUnlock krbObjectReferences krbTicketFlags krbMaxTicketLife krbMaxRenewableAge nsAccountLock passwordHistory ipaKrbAuthzData ipaUserAuthType ipatokenRadiusConfigLink krbAuthIndMaxTicke..."
[10/Mar/2020:11:51:14.266226284 +0800] conn=10 op=8340 RESULT err=0 tag=101 nentries=1 etime=0.000251517
[10/Mar/2020:11:51:14.266284747 +0800] conn=10 op=8341 SRCH base="cn=<masked>,cn=kerberos,<masked>" scope=0 filter="(objectClass=krbticketpolicyaux)" attrs="krbMaxTicketLife krbMaxRenewableAge krbTicketFlags krbAuthIndMaxTicketLife krbAuthIndMaxRenewableAge"
[10/Mar/2020:11:51:14.266578962 +0800] conn=10 op=8341 RESULT err=0 tag=101 nentries=1 etime=0.000326237
[10/Mar/2020:11:51:14.267100741 +0800] conn=5261 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI
[10/Mar/2020:11:51:15.331791547 +0800] conn=5262 fd=170 slot=170 connection from <masked>.154 to <masked>.165
[10/Mar/2020:11:51:20.009768876 +0800] conn=5263 fd=171 slot=171 connection from <masked>.156 to <masked>.165
[10/Mar/2020:11:51:20.719915656 +0800] conn=5264 fd=172 slot=172 connection from <masked>.151 to <masked>.165
[10/Mar/2020:11:51:21.334589697 +0800] conn=5265 fd=173 slot=173 connection from <masked>.154 to <masked>.165
[10/Mar/2020:11:51:25.881631700 +0800] conn=5266 fd=174 slot=174 connection from <masked>.153 to <masked>.165
[10/Mar/2020:11:51:26.028079666 +0800] conn=5267 fd=175 slot=175 connection from <masked>.156 to <masked>.165
[10/Mar/2020:11:51:26.291901685 +0800] conn=5268 fd=176 slot=176 connection from <masked>.152 to <masked>.165
[10/Mar/2020:11:51:27.275114819 +0800] conn=5269 fd=180 slot=180 connection from <masked>.150 to <masked>.165
[10/Mar/2020:11:51:33.403472677 +0800] conn=5270 fd=181 slot=181 connection from <masked>.154 to <masked>.165
```
stacktrace.1583832889.txt
```
GNU gdb (GDB) Fedora 8.3.50.20190824-30.fc31
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/ns-slapd...
Reading symbols from .gnu_debugdata for /usr/sbin/ns-slapd...
(No debugging symbols found in .gnu_debugdata for /usr/sbin/ns-slapd)
Attaching to program: /usr/sbin/ns-slapd, process 1939
[New LWP 1941]
[New LWP 1942]
[New LWP 1943]
[New LWP 1944]
[New LWP 1945]
[New LWP 1946]
[New LWP 1947]
[New LWP 1948]
[New LWP 1949]
[New LWP 1950]
[New LWP 1951]
[New LWP 1952]
[New LWP 1953]
[New LWP 1954]
[New LWP 1955]
[New LWP 1956]
[New LWP 1957]
[New LWP 1958]
[New LWP 1959]
[New LWP 1960]
[New LWP 1961]
[New LWP 1962]
[New LWP 1963]
[New LWP 1964]
[New LWP 1965]
[New LWP 1966]
[New LWP 1967]
[New LWP 1968]
[New LWP 1969]
[New LWP 1970]
[New LWP 1971]
[New LWP 1972]
[New LWP 1973]
[New LWP 1974]
[New LWP 1975]
[New LWP 1976]
[New LWP 1977]
[New LWP 1981]
[New LWP 1982]
[New LWP 2019]
[New LWP 2023]
[New LWP 44777]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
0x00007fae16f52a6f in poll () from /lib64/libc.so.6

Thread 43 (Thread 0x7fae06b38700 (LWP 44777)):
#0  0x00007fae170e8918 in pthread_rwlock_wrlock () at /lib64/libpthread.so.0
#1  0x00007fae12aabbb9 in backend_be_pre_write_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae12c9464d in ldbm_back_delete () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#5  0x00007fae173e460c in op_shared_delete () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x00007fae173e481e in delete_internal_pb () at /usr/lib64/dirsrv/libslapd.so.0
#7  0x00007fae12aed83f in delete_changerecord () at /usr/lib64/dirsrv/plugins/libretrocl-plugin.so
#8  0x00007fae12aeda05 in changelog_trim_thread_fn () at /usr/lib64/dirsrv/plugins/libretrocl-plugin.so
#9  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#10 0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#11 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 42 (Thread 0x7fadea4fe700 (LWP 2023)):
#0  0x00007fae170ea07a in pthread_cond_timedwait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148803 in pt_TimedWait () at /lib64/libnspr4.so
#2  0x00007fae17148cc0 in PR_WaitCondVar () at /lib64/libnspr4.so
#3  0x00007fae12df4e44 in sync_send_results () at /usr/lib64/dirsrv/plugins/libcontentsync-plugin.so
#4  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#5  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#6  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 41 (Thread 0x7fadeb6ff700 (LWP 2019)):
#0  0x00007fae170ea07a in pthread_cond_timedwait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148803 in pt_TimedWait () at /lib64/libnspr4.so
#2  0x00007fae17148cc0 in PR_WaitCondVar () at /lib64/libnspr4.so
#3  0x00007fae12df4e44 in sync_send_results () at /usr/lib64/dirsrv/plugins/libcontentsync-plugin.so
#4  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#5  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#6  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 40 (Thread 0x7fadef9f9700 (LWP 1982)):
#0  0x00007fae170e9d45 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148d28 in PR_WaitCondVar () at /lib64/libnspr4.so
#2  0x000055b92c60e94d in ps_send_results ()
#3  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#4  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#5  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 39 (Thread 0x7fadf01fa700 (LWP 1981)):
#0  0x00007fae170e9d45 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148d28 in PR_WaitCondVar () at /lib64/libnspr4.so
#2  0x000055b92c60e94d in ps_send_results ()
#3  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#4  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#5  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 38 (Thread 0x7fadf11fc700 (LWP 1977)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 37 (Thread 0x7fadf1bfd700 (LWP 1976)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 36 (Thread 0x7fadf23fe700 (LWP 1975)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aadb8c in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae1742a6ad in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x000055b92c614452 in do_search ()
#7  0x000055b92c60258d in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 35 (Thread 0x7fadf2dff700 (LWP 1974)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aadb8c in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae1742a6ad in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x000055b92c614452 in do_search ()
#7  0x000055b92c60258d in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 34 (Thread 0x7fadf39fe700 (LWP 1973)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 33 (Thread 0x7fadf41ff700 (LWP 1972)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 32 (Thread 0x7fadf4fff700 (LWP 1971)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 31 (Thread 0x7fadf5bfe700 (LWP 1970)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 30 (Thread 0x7fadf63ff700 (LWP 1969)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 29 (Thread 0x7fadf73fe700 (LWP 1968)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 28 (Thread 0x7fadf7bff700 (LWP 1967)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aadb8c in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae1742a6ad in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x000055b92c614452 in do_search ()
#7  0x000055b92c60258d in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 27 (Thread 0x7fadf89ff700 (LWP 1966)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aadb8c in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae1742a6ad in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x000055b92c614452 in do_search ()
#7  0x000055b92c60258d in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 26 (Thread 0x7fadf95fe700 (LWP 1965)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 25 (Thread 0x7fadf9fff700 (LWP 1964)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 24 (Thread 0x7fadfabff700 (LWP 1963)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aadb8c in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae1742a6ad in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x000055b92c614452 in do_search ()
#7  0x000055b92c60258d in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 23 (Thread 0x7fadfb7fe700 (LWP 1962)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aadb8c in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae1742a6ad in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x000055b92c614452 in do_search ()
#7  0x000055b92c60258d in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 22 (Thread 0x7fadfc1ff700 (LWP 1961)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 21 (Thread 0x7fadfcdfe700 (LWP 1960)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aadb8c in backend_search_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae1742a6ad in op_shared_search () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x000055b92c614452 in do_search ()
#7  0x000055b92c60258d in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 20 (Thread 0x7fadfd7ff700 (LWP 1959)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 19 (Thread 0x7fadfe9ff700 (LWP 1958)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 18 (Thread 0x7fadff5fc700 (LWP 1957)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 17 (Thread 0x7fadffdfd700 (LWP 1956)):
#0  0x00007fae170e9d45 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148fcb in PR_EnterMonitor () at /lib64/libnspr4.so
#2  0x00007fae12c745fe in dblayer_txn_begin () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#3  0x00007fae12c85a03 in ldbm_back_add () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#4  0x00007fae173d4fb5 in op_shared_add () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae173d542f in add_internal_pb () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x00007fae173d61c7 in slapi_add_internal_pb () at /usr/lib64/dirsrv/libslapd.so.0
#7  0x00007fae12aecfc5 in retrocl_postob () at /usr/lib64/dirsrv/plugins/libretrocl-plugin.so
#8  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#9  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#10 0x00007fae12ca23d0 in ldbm_back_modify () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#11 0x00007fae17423ac7 in op_shared_modify () at /usr/lib64/dirsrv/libslapd.so.0
#12 0x00007fae17424b0f in do_modify () at /usr/lib64/dirsrv/libslapd.so.0
#13 0x000055b92c6024ed in connection_threadmain ()
#14 0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#15 0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#16 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 16 (Thread 0x7fae005fe700 (LWP 1955)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 15 (Thread 0x7fae00dff700 (LWP 1954)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aac450 in backend_bind_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#4  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x000055b92c613415 in ids_sasl_check_bind ()
#6  0x000055b92c5fc917 in do_bind ()
#7  0x000055b92c60249f in connection_threadmain ()
#8  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#9  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#10 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 14 (Thread 0x7fae01bfb700 (LWP 1953)):
#0  0x00007fae170e81ec in pthread_rwlock_rdlock () at /lib64/libpthread.so.0
#1  0x00007fae12ac0b01 in map_rdlock () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#2  0x00007fae12aae277 in backend_write_cb.isra.0.part () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#3  0x00007fae12aae35c in backend_betxn_pre_write_cb () at /usr/lib64/dirsrv/plugins/schemacompat-plugin.so
#4  0x00007fae17437b09 in plugin_call_func () at /usr/lib64/dirsrv/libslapd.so.0
#5  0x00007fae17437d4b in plugin_call_plugins () at /usr/lib64/dirsrv/libslapd.so.0
#6  0x00007fae173ee64b in dse_modify () at /usr/lib64/dirsrv/libslapd.so.0
#7  0x00007fae17423ac7 in op_shared_modify () at /usr/lib64/dirsrv/libslapd.so.0
#8  0x00007fae174240d3 in modify_internal_pb () at /usr/lib64/dirsrv/libslapd.so.0
#9  0x00007fae12bbb7d8 in replica_update_state () at /usr/lib64/dirsrv/plugins/libreplication-plugin.so
#10 0x00007fae173fa154 in eq_loop () at /usr/lib64/dirsrv/libslapd.so.0
#11 0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#12 0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#13 0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 13 (Thread 0x7fae023fc700 (LWP 1952)):
#0  0x00007fae170ea07a in pthread_cond_timedwait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148803 in pt_TimedWait () at /lib64/libnspr4.so
#2  0x00007fae17148cc0 in PR_WaitCondVar () at /lib64/libnspr4.so
#3  0x000055b92c60a1f0 in housecleaning ()
#4  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#5  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#6  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 12 (Thread 0x7fae02bfd700 (LWP 1951)):
#0  0x00007fae170e9d45 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148d28 in PR_WaitCondVar () at /lib64/libnspr4.so
#2  0x00007fae1745cadc in slapi_wait_condvar () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007fae12ade44d in roles_cache_wait_on_change () at /usr/lib64/dirsrv/plugins/libroles-plugin.so
#4  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#5  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#6  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 11 (Thread 0x7fae033fe700 (LWP 1950)):
#0  0x00007fae170e9d45 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148d28 in PR_WaitCondVar () at /lib64/libnspr4.so
#2  0x00007fae1745cadc in slapi_wait_condvar () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007fae12ade44d in roles_cache_wait_on_change () at /usr/lib64/dirsrv/plugins/libroles-plugin.so
#4  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#5  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#6  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 10 (Thread 0x7fae03bff700 (LWP 1949)):
#0  0x00007fae170e9d45 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148d28 in PR_WaitCondVar () at /lib64/libnspr4.so
#2  0x00007fae1745cadc in slapi_wait_condvar () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007fae12ade44d in roles_cache_wait_on_change () at /usr/lib64/dirsrv/plugins/libroles-plugin.so
#4  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#5  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#6  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 9 (Thread 0x7fae049ff700 (LWP 1948)):
#0  0x00007fae170ea07a in pthread_cond_timedwait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148803 in pt_TimedWait () at /lib64/libnspr4.so
#2  0x00007fae17148cc0 in PR_WaitCondVar () at /lib64/libnspr4.so
#3  0x00007fae12bac346 in protocol_sleep () at /usr/lib64/dirsrv/plugins/libreplication-plugin.so
#4  0x00007fae12bafa72 in repl5_inc_run () at /usr/lib64/dirsrv/plugins/libreplication-plugin.so
#5  0x00007fae12bb4701 in prot_thread_main () at /usr/lib64/dirsrv/plugins/libreplication-plugin.so
#6  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#7  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#8  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 8 (Thread 0x7fae057ff700 (LWP 1947)):
#0  0x00007fae170e9d45 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148d28 in PR_WaitCondVar () at /lib64/libnspr4.so
#2  0x00007fae12bac346 in protocol_sleep () at /usr/lib64/dirsrv/plugins/libreplication-plugin.so
#3  0x00007fae12bafcbb in repl5_inc_run () at /usr/lib64/dirsrv/plugins/libreplication-plugin.so
#4  0x00007fae12bb4701 in prot_thread_main () at /usr/lib64/dirsrv/plugins/libreplication-plugin.so
#5  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#6  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#7  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 7 (Thread 0x7fae06bf9700 (LWP 1946)):
#0  0x00007fae170ea07a in pthread_cond_timedwait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148803 in pt_TimedWait () at /lib64/libnspr4.so
#2  0x00007fae17148cc0 in PR_WaitCondVar () at /lib64/libnspr4.so
#3  0x00007fae12b965dc in _cl5TrimMain () at /usr/lib64/dirsrv/plugins/libreplication-plugin.so
#4  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#5  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#6  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 6 (Thread 0x7fae065ff700 (LWP 1945)):
#0  0x00007fae170e9d45 in pthread_cond_wait@@GLIBC_2.3.2 () at /lib64/libpthread.so.0
#1  0x00007fae17148d28 in PR_WaitCondVar () at /lib64/libnspr4.so
#2  0x00007fae1745cadc in slapi_wait_condvar () at /usr/lib64/dirsrv/libslapd.so.0
#3  0x00007fae12e023be in cos_cache_wait_on_change () at /usr/lib64/dirsrv/plugins/libcos-plugin.so
#4  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#5  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#6  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 5 (Thread 0x7fae073fa700 (LWP 1944)):
#0  0x00007fae16f54f6b in select () at /lib64/libc.so.6
#1  0x00007fae17470224 in DS_Sleep () at /usr/lib64/dirsrv/libslapd.so.0
#2  0x00007fae12cb9c44 in perf_threadmain () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#3  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#4  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#5  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 4 (Thread 0x7fae07bfb700 (LWP 1943)):
#0  0x00007fae16f54f6b in select () at /lib64/libc.so.6
#1  0x00007fae17470224 in DS_Sleep () at /usr/lib64/dirsrv/libslapd.so.0
#2  0x00007fae12cb9abf in trickle_threadmain () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#3  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#4  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#5  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 3 (Thread 0x7fae083fc700 (LWP 1942)):
#0  0x00007fae16f54f6b in select () at /lib64/libc.so.6
#1  0x00007fae17470224 in DS_Sleep () at /usr/lib64/dirsrv/libslapd.so.0
#2  0x00007fae12cbc71f in checkpoint_threadmain () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#3  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#4  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#5  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 2 (Thread 0x7fae08bfd700 (LWP 1941)):
#0  0x00007fae16f54f6b in select () at /lib64/libc.so.6
#1  0x00007fae17470224 in DS_Sleep () at /usr/lib64/dirsrv/libslapd.so.0
#2  0x00007fae12cb960f in deadlock_threadmain () at /usr/lib64/dirsrv/plugins/libback-ldbm.so
#3  0x00007fae1714eb34 in _pt_root () at /lib64/libnspr4.so
#4  0x00007fae170e34e2 in start_thread () at /lib64/libpthread.so.0
#5  0x00007fae16f5d693 in clone () at /lib64/libc.so.6

Thread 1 (Thread 0x7fae16244140 (LWP 1939)):
#0  0x00007fae16f52a6f in poll () at /lib64/libc.so.6
#1  0x00007fae1714a7be in _pr_poll_with_poll () at /lib64/libnspr4.so
#2  0x000055b92c605c48 in slapd_daemon ()
#3  0x000055b92c5f99c5 in main ()
Detaching from program: /usr/sbin/ns-slapd, process 1939
[Inferior 1 (process 1939) detached]
```

Mark Reynolds <mreynolds@redhat.com> 於 2020年3月9日 週一 上午7:48寫道:
A stack trace would be very useful in determining why the Directory
Server is misbehaving.  You can grab stack traces following these steps:

http://www.port389.org/docs/389ds/FAQ/faq.html#sts=Debugging%C2%A0Hangs

Thanks,
Mark


On 3/7/20 11:48 PM, Lays Dragon via FreeIPA-users wrote:
> I deployed a two replica FreeIPA Servers,it woks well until this month,it start at the service report the LDAP is Timeout,I try to restart the server,even reinstall two IPA server and maintain the data via replica from another server. And it still happen after several days. The 389ds server just simply stop to response to any connection ,the wierd thing is the connection is established but no response after the connection.
> LDAP server seems to blocked on something,even replica is dead because the ldap is blocked.simply restart not slove the problem,the ldap server will blocked really soon caused other service like IPA Web service or kinit dead too.
> I guess the blocked is caused via replica function somehow,since I figure out I have to close the ldap port on blocked server firewall to make it isolate,and restart the server,waiting for about 10 min after the server is start,reopen the ldap port on firewall to let replica recover,and everything will be fine...And I notice there some connection stuck at CLOSE_WAIT of ns-slapd may be related.
> Need some help . I not so familiar with of freeipa,and trying to deal this problem over the week but nothing works.
>
> FreeIPA server version:4.8.4
> Server System: Fedora 31 (Cloud Edition)
>
> server1 access log
> ```
> krbLastFailedAuth krbLoginFailedCount krbPrincipalAuthInd krbExtraData krbLastAdminUnlock krbObjectReferences krbTicketFlags krbMaxTicketLife krbMaxRenewableAge nsAccountLock passwordHistory
> ipaKrbAuthzData ipaUserAuthType ipatokenRadiusConfigLink krbAuthIndMaxTicke..."
> [08/Mar/2020:10:01:23.390837315 +0800] conn=4 op=6091 RESULT err=0 tag=101 nentries=1 etime=0.000276689
> [08/Mar/2020:10:01:23.390906790 +0800] conn=4 op=6092 SRCH base="cn=ENMD.NET,cn=kerberos,dc=enmd,dc=net" scope=0 filter="(objectClass=krbticketpolicyaux)" attrs="krbMaxTicketLife krbMaxRenewableAge krbTicketFlags krbAuthIndMaxTicketLife krbAuthIndMaxRenewableAge"
> [08/Mar/2020:10:01:23.391302403 +0800] conn=4 op=6092 RESULT err=0 tag=101 nentries=1 etime=0.000432879
> [08/Mar/2020:10:01:23.392418974 +0800] conn=3351 op=1 BIND dn="" method=sasl version=3 mech=GSSAPI
> [08/Mar/2020:10:01:25.953517485 +0800] conn=3352 fd=161 slot=161 connection from <masked>.152 to <masked>.165
> [08/Mar/2020:10:01:27.007620375 +0800] conn=3353 fd=162 slot=162 connection from <masked>.154 to <masked>.165
> [08/Mar/2020:10:01:27.151656148 +0800] conn=3354 fd=163 slot=163 connection from <masked>.150 to <masked>.165
> [08/Mar/2020:10:01:27.559750675 +0800] conn=3355 fd=164 slot=164 connection from <masked>.153 to <masked>.165
> [08/Mar/2020:10:01:39.015400434 +0800] conn=3356 fd=165 slot=165 connection from <masked>.154 to <masked>.165
> [08/Mar/2020:10:01:51.582586229 +0800] conn=3357 fd=166 slot=166 connection from <masked>.153 to <masked>.165
> [08/Mar/2020:10:01:52.513047687 +0800] conn=3358 fd=167 slot=167 connection from <masked>.150 to <masked>.165
> [08/Mar/2020:10:01:53.573811317 +0800] conn=3359 fd=168 slot=168 connection from <masked>.152 to <masked>.165
> [08/Mar/2020:10:02:44.012371005 +0800] conn=3360 fd=169 slot=169 connection from <masked>.160 to <masked>.165
> [08/Mar/2020:10:02:44.419580574 +0800] conn=3361 fd=170 slot=170 connection from <masked>.151 to <masked>.165
> [08/Mar/2020:10:02:45.548493596 +0800] conn=3362 fd=171 slot=171 connection from <masked>.153 to <masked>.165
> [08/Mar/2020:10:02:50.018712852 +0800] conn=3363 fd=172 slot=172 connection from <masked>.160 to <masked>.165
> [08/Mar/2020:10:02:51.081867407 +0800] conn=3364 fd=173 slot=173 connection from <masked>.152 to <masked>.165
> [08/Mar/2020:10:03:04.062925765 +0800] conn=3365 fd=174 slot=174 connection from <masked>.154 to <masked>.165
> [08/Mar/2020:10:03:06.223438080 +0800] conn=3366 fd=175 slot=175 connection from <masked>.150 to <masked>.165
> [08/Mar/2020:10:03:10.063982993 +0800] conn=3367 fd=176 slot=176 connection from <masked>.154 to <masked>.165
> [08/Mar/2020:10:03:52.027006125 +0800] conn=3368 fd=177 slot=177 connection from <masked>.153 to <masked>.165
> [08/Mar/2020:10:03:57.005297121 +0800] conn=3369 fd=178 slot=178 connection from <masked>.152 to <masked>.165
> [08/Mar/2020:10:04:01.001767909 +0800] conn=3370 fd=179 slot=179 connection from <masked>.150 to <masked>.165
> [08/Mar/2020:10:04:08.003082421 +0800] conn=3371 fd=180 slot=180 connection from <masked>.154 to <masked>.165
> [08/Mar/2020:10:04:12.014090964 +0800] conn=3372 fd=181 slot=181 connection from <masked>.151 to <masked>.165
> [08/Mar/2020:10:04:18.140192092 +0800] conn=3373 fd=182 slot=182 connection from <masked>.166 to <masked>.165
> [08/Mar/2020:10:04:20.007046774 +0800] conn=3374 fd=183 slot=183 connection from <masked>.154 to <masked>.165
> [08/Mar/2020:10:04:24.040348027 +0800] conn=3375 fd=184 slot=184 connection from <masked>.160 to <masked>.165
> [08/Mar/2020:10:04:30.139898749 +0800] conn=3376 fd=185 slot=185 connection from <masked>.160 to <masked>.165
> [08/Mar/2020:10:05:22.043556910 +0800] conn=3377 fd=186 slot=186 connection from <masked>.160 to <masked>.165
> [08/Mar/2020:10:05:34.140357676 +0800] conn=3378 fd=187 slot=187 connection from <masked>.160 to <masked>.165
> [08/Mar/2020:10:05:36.006033007 +0800] conn=3379 fd=188 slot=188 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:06:07.002808000 +0800] conn=3380 fd=189 slot=189 connection from <masked>.150 to <masked>.165
> [08/Mar/2020:10:06:12.043478717 +0800] conn=3381 fd=190 slot=190 connection from <masked>.152 to <masked>.165
> [08/Mar/2020:10:06:15.007914045 +0800] conn=3382 fd=191 slot=191 connection from <masked>.153 to <masked>.165
> [08/Mar/2020:10:06:17.005632290 +0800] conn=3383 fd=192 slot=192 connection from <masked>.154 to <masked>.165
> [08/Mar/2020:10:06:19.016341572 +0800] conn=3384 fd=193 slot=193 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:06:23.007594584 +0800] conn=3385 fd=194 slot=194 connection from <masked>.154 to <masked>.165
> [08/Mar/2020:10:06:27.026262632 +0800] conn=3386 fd=195 slot=195 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:06:30.031700186 +0800] conn=3387 fd=196 slot=196 SSL connection from <masked>.159 to <masked>.180
> [08/Mar/2020:10:06:37.009611536 +0800] conn=3388 fd=197 slot=197 connection from <masked>.151 to <masked>.165
> [08/Mar/2020:10:06:37.033108567 +0800] conn=3389 fd=198 slot=198 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:07:23.002813545 +0800] conn=3390 fd=199 slot=199 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:07:31.011795943 +0800] conn=3391 fd=200 slot=200 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:07:36.011894960 +0800] conn=3392 fd=201 slot=201 connection from <masked>.160 to <masked>.165
> [08/Mar/2020:10:07:41.021108836 +0800] conn=3393 fd=202 slot=202 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:07:42.014874690 +0800] conn=3394 fd=203 slot=203 connection from <masked>.160 to <masked>.165
> [08/Mar/2020:10:09:16.005883198 +0800] conn=3395 fd=204 slot=204 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:09:24.009940147 +0800] conn=3396 fd=205 slot=205 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:09:34.015154400 +0800] conn=3397 fd=206 slot=206 connection from <masked>.165 to <masked>.165
> [08/Mar/2020:10:10:24.040398249 +0800] conn=3398 fd=207 slot=207 connection from <masked>.153 to <masked>.165
> [08/Mar/2020:10:10:27.003675219 +0800] conn=3399 fd=208 slot=208 connection from <masked>.152 to <masked>.165
> [08/Mar/2020:10:10:28.005336766 +0800] conn=3400 fd=209 slot=209 connection from <masked>.150 to <masked>.165
> ```
> server1 error log
> ```
> [08/Mar/2020:09:30:52.966764268 +0800] - ERR - NSMMReplicationPlugin - repl5_inc_waitfor_async_results  - Timed out waiting for responses: 0 3074
> [08/Mar/2020:09:32:53.684831136 +0800] - ERR - NSMMReplicationPlugin - release_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Attempting to release replica, but unable to receive endReplication extended operation response from the replica. Error -5 (Timed out)
> [08/Mar/2020:09:34:53.625806166 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:36:56.570809366 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:38:56.509924342 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:40:59.458123866 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:42:59.402931124 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:45:02.343312876 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:47:02.282487714 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:49:05.220734403 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:51:05.160565112 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:53:08.105641621 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:55:08.040503542 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:57:11.997307120 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:09:59:12.965695447 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:10:01:15.903578926 +0800] - WARN - NSMMReplicationPlugin - acquire_replica - agmt="cn=meToipa2.enmd.net" (ipa2:389): Unable to receive the response for a startReplication extended operation to consumer (Timed out). Will retry later.
> [08/Mar/2020:11:26:42.560167019 +0800] - INFO - slapd_extract_cert - CA CERT NAME: ENMD.NET IPA CA
> [08/Mar/2020:11:26:42.567890161 +0800] - WARN - Security Initialization - SSL alert: Sending pin request to SVRCore. You may need to run systemd-tty-ask-password-agent to provide the password.
> [08/Mar/2020:11:26:42.647668764 +0800] - INFO - slapd_extract_cert - SERVER CERT NAME: Server-Cert
> [08/Mar/2020:11:26:42.722748631 +0800] - INFO - Security Initialization - SSL info: Enabling default cipher set.
> [08/Mar/2020:11:26:42.726554182 +0800] - INFO - Security Initialization - SSL info: Configured NSS Ciphers
> [08/Mar/2020:11:26:42.730304776 +0800] - INFO - Security Initialization - SSL info:   TLS_AES_128_GCM_SHA256: enabled
> [08/Mar/2020:11:26:42.733614343 +0800] - INFO - Security Initialization - SSL info:   TLS_CHACHA20_POLY1305_SHA256: enabled
> [08/Mar/2020:11:26:42.740389595 +0800] - INFO - Security Initialization - SSL info:   TLS_AES_256_GCM_SHA384: enabled
> [08/Mar/2020:11:26:42.743830864 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256: enabled
> [08/Mar/2020:11:26:42.748868878 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256: enabled
> [08/Mar/2020:11:26:42.762016895 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256: enabled
> [08/Mar/2020:11:26:42.766962209 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256: enabled
> [08/Mar/2020:11:26:42.779721887 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384: enabled
> [08/Mar/2020:11:26:42.787619421 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384: enabled
> [08/Mar/2020:11:26:42.795024632 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA: enabled
> [08/Mar/2020:11:26:42.799027752 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA: enabled
> [08/Mar/2020:11:26:42.802532993 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA: enabled
> [08/Mar/2020:11:26:42.806279559 +0800] - INFO - Security Initialization - SSL info:   TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256: enabled
> ```
> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org

--

389 Directory Server Development Team