Hmm that's very strange, try this:Hi Mark,
I’ve afraid adding nsIndexIDListScanLimit to cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config doesn’t seem to have any effect. Did this:
dn: cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: modify
add: nsIndexIDListScanLimit
nsIndexIDListScanLimit: limit=-1 type=eq values=inetOrgPerson
CPU was still in the 70’s. Logconv.pl output looks much the same. Follows.
Thanks a lot,Trev
Total Log Lines Analysed: 424930
----------- Access Log Output ------------
Start of Logs: 09/Dec/2014:11:20:00End of Logs: 09/Dec/2014:14:09:20
Processed Log Time: 2 Hours, 49 Minutes, 20 Seconds
Restarts: 0Total Connections: 3909- StartTLS Connections: 0- LDAPS Connections: 3534- LDAPI Connections: 0Peak Concurrent Connections: 6Total Operations: 61321Total Results: 61321Overall Performance: 100.0%
Searches: 58178 (5.73/sec) (343.57/min)Modifications: 266 (0.03/sec) (1.57/min)Adds: 93 (0.01/sec) (0.55/min)Deletes: 0 (0.00/sec) (0.00/min)Mod RDNs: 0 (0.00/sec) (0.00/min)Compares: 0 (0.00/sec) (0.00/min)Binds: 2570 (0.25/sec) (15.18/min)
Proxied Auth Operations: 0Persistent Searches: 0Internal Operations: 0Entry Operations: 0Extended Operations: 214Abandoned Requests: 0Smart Referrals Received: 0
VLV Operations: 0VLV Unindexed Searches: 0VLV Unindexed Components: 0SORT Operations: 1
Entire Search Base Queries: 0Paged Searches: 546Unindexed Searches: 1Unindexed Components: 545
Unindexed Search #1 (notes=A)- Date/Time: 09/Dec/2014:13:19:05- Connection Number: 348121- Operation Number: 2- Etime: 0- Nentries: 100- IP Address: 10.7.8.105- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))
Unindexed Component #1 (notes=U)- Date/Time: 09/Dec/2014:11:30:16- Connection Number: 344719- Operation Number: 101- Etime: 0- Nentries: 100- IP Address: Unknown_Host- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))
Unindexed Component #2 (notes=U)- Date/Time: 09/Dec/2014:11:31:47- Connection Number: 344719- Operation Number: 105- Etime: 0- Nentries: 100- IP Address: Unknown_Host- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))…<snip>Unindexed Component #545 (notes=U)- Date/Time: 09/Dec/2014:13:19:21- Connection Number: 348121- Operation Number: 9- Etime: 1- Nentries: 100- IP Address: 10.7.8.105- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))
FDs Taken: 3909FDs Returned: 3910Highest FD Taken: 257
Broken Pipes: 0Connections Reset By Peer: 0Resource Unavailable: 0Max BER Size Exceeded: 0
Binds: 2570Unbinds: 1470- LDAP v2 Binds: 0- LDAP v3 Binds: 2570- AUTOBINDs: 0- SSL Client Binds: 0- Failed SSL Client Binds: 0- SASL Binds: 0- Directory Manager Binds: 1170- Anonymous Binds: 0- Other Binds: 1400
----- Connection Latency Details -----
(in seconds) <=1 2 3 4-5 6-10 11-15 >15--------------------------------------------------------------------------(# of connections) 3783 2 2 22 10 2 89
----- Current Open Connection IDs -----
Conn Number: 348500 (10.7.127.250)Conn Number: 348440 (10.7.127.250)
----- Errors -----
err=0 59285 Successful Operationserr=32 1017 No Such Objecterr=53 1017 Unwilling To Performerr=49 2 Invalid Credentials (Bad Password)
----- Top 20 Failed Logins ------
2 uid=elainez,ou=employees,dc=<our dc>
From the IP address(s) :
2 10.7.8.104
----- Total Connection Codes -----
U1 1470 Cleanly Closed ConnectionsB1 1089 Bad Ber Tag Encountered
----- Top 20 Clients -----
Number of Clients: 15
[1] Client: 10.20.220.161170 - Connections1170 - U1 (Cleanly Closed Connections)
[2] Client: 137.82.5.971017 - Connections1017 - B1 (Bad Ber Tag Encountered)
[3] Client: 142.103.218.107674 - Connections
[4] Client: 142.103.218.106674 - Connections
[5] Client: 10.7.127.25058 - Connections54 - U1 (Cleanly Closed Connections)
[6] Client: 10.7.0.5253 - Connections53 - U1 (Cleanly Closed Connections)
[7] Client: 142.103.106.14240 - Connections40 - U1 (Cleanly Closed Connections)
[8] Client: 142.103.106.13538 - Connections38 - U1 (Cleanly Closed Connections)
[9] Client: 127.0.0.136 - Connections35 - B1 (Bad Ber Tag Encountered)1 - U1 (Cleanly Closed Connections)
[10] Client: 142.103.149.13634 - Connections33 - B1 (Bad Ber Tag Encountered)1 - U1 (Cleanly Closed Connections)
[11] Client: 142.103.106.14333 - Connections33 - U1 (Cleanly Closed Connections)
[12] Client: 142.103.106.14531 - Connections31 - U1 (Cleanly Closed Connections)
[13] Client: 142.103.106.14128 - Connections28 - U1 (Cleanly Closed Connections)
[14] Client: 10.7.8.10412 - Connections8 - U1 (Cleanly Closed Connections)4 - B1 (Bad Ber Tag Encountered)
[15] Client: 10.7.8.10511 - Connections11 - U1 (Cleanly Closed Connections)
----- Top 20 Bind DN's -----
Number of Unique Bind DN's: 11
1240 cn=directory manager1017 cn=bill fisher,ou=people,dc=dartware,dc=com206 uid=<service accoount 1>53 cn=replication manager,cn=config34 uid=<service account 2>6 uid=rogermo,ou=employees,dc=<our dc>6 uid=elainez,ou=employees,dc=<our dc>2 uid=rwong,ou=employees,dc=<our dc>2 uid=steve0d6,ou=employees,dc=<our dc>2 uid=huynhh,ou=employees,dc=<our dc>2 uid=robg,ou=employees,dc=<our dc>
----- Top 20 Search Bases -----
Number of Unique Search Bases: 9
55737 ou=groups,dc=<our dc>1017 ou=people,dc=dartware,dc=com766 ou=employees,dc=<our dc>391 ou=people,dc=<our dc>140 cn=mapping tree,cn=config121 root dse2 uid=ark826,ou=employees,dc=<our dc>2 uid=emilygib,ou=employees,dc=<our dc>2 uid=kgg39500,ou=employees,dc=<our dc>
----- Top 20 Search Filters -----
Number of Unique Search Filters: 31129
1017 (cn=bill fisher)546 (&(objectclass=inetorgperson))124 (objectclass=*)35 (cn=eldap3*)35 (cn=eldap5*)35 (cn=eldap4*)35 (cn=eldap2*)5 (&(objectclass=groupofnames)(member=uid=elainez,ou=employees,dc=<our dc>))4 (|(cn=elainez)(uid=elainez)(mail=elainez))4 (&(objectclass=inetorgperson)(ubceducwlloginkey=<login key 1>))4 (&(uid=dkavoura))4 (&(uid=umashank))4 (&(objectclass=groupofnames)(member=uid=rogermo,ou=employees,dc=<our dc>))4 (&(uid=joshb11))4 (&(uid=joaweg))4 (&(objectclass=inetorgperson)(ubceducwlloginkey=<login key 2>))3 (&(objectclass=groupofnames)(member=uid=dentonel,ou=employees,dc=<our dc>))3 (&(objectclass=groupofnames)(member=uid=wexler,ou=employees,dc=<our dc>))3 (&(objectclass=groupofnames)(member=uid=marizulu,ou=employees,dc=<our dc>))3 (&(objectclass=groupofnames)(member=uid=agurjal,ou=employees,dc=<our dc>))
----- Top 20 Most Frequent etimes -----
53984 etime=07319 etime=111 etime=24 etime=81 etime=1421 etime=1591 etime=9
----- Top 20 Longest etimes -----
etime=159 1etime=142 1etime=9 1etime=8 4etime=2 11etime=1 7319etime=0 53984
----- Top 20 Largest nentries -----
nentries=100 544nentries=36 2nentries=7 2nentries=6 6nentries=5 34nentries=4 153nentries=3 1182nentries=2 4697nentries=1 41333nentries=0 10225
----- Top 20 Most returned nentries -----
41333 nentries=110225 nentries=04697 nentries=21182 nentries=3544 nentries=100153 nentries=434 nentries=56 nentries=62 nentries=362 nentries=7
----- Extended Operations -----
107 2.16.840.1.113730.3.5.12 DS90 Start Replication Request107 2.16.840.1.113730.3.5.5 End Replication Request (incremental update)
----- Top 20 Most Requested Attributes -----
54570 1.12498 All Attributes1126 o954 description954 givenName954 ubcEduCwlPUID954 uid954 cn954 displayName954 employeeNumber954 mail954 nsAccountLock954 objectClass954 sn563 businessCategory563 employeeType563 facsimileTelephoneNumber563 groups563 homePostalAddress563 l
----- Recommendations -----
1. You have unindexed searches, this can be caused from a search on an unindexed attribute, or your returned results exceeded the allidsthreshold. Unindexed searches are not recommended. To refuse unindexed searches, switch 'nsslapd-require-index' to 'on' under your database entry (e.g. cn=UserRoot,cn=ldbm database,cn=plugins,cn=config).
2. You have unindexed components, this can be caused from a search on an unindexed attribute, or your returned results exceeded the allidsthreshold. Unindexed components are not recommended. To refuse unindexed searches, switch 'nsslapd-require-index' to 'on' under your database entry (e.g. cn=UserRoot,cn=ldbm database,cn=plugins,cn=config).
3. You have a significant difference between binds and unbinds. You may want to investigate this difference.
4. You have a high number of Directory Manager binds. The Directory Manager account should only be used under certain circumstances. Avoid using this account for client applications.
Cleaning up temp files...Done.
From: Mark Reynolds <mareynol@redhat.com>
Reply-To: "mreynolds@redhat.com" <mreynolds@redhat.com>
Date: Tuesday, December 9, 2014 at 9:33 AM
To: Trevor Fong <trevor.fong@ubc.ca>, "mreynolds@redhat.com" <mreynolds@redhat.com>, "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>
Subject: Re: [389-users] 389-ds and Multi CPU's
On 12/09/2014 12:14 PM, Fong, Trevor wrote:
Hi Mark,
Oops – sorry about the Mike thing! I sit beside a Mike :pNo worries :-)
So your high CPU is most likely coming from your unindexed searches.
The version you are on should support the new per index idlistscanlimit, so try this:
Note, this this is just an example, your objectclass entry might have a different DN (userRoot might be something else in your deployment)
ldapmodify ...
dn: cn=objectclass,cn=index,cn=userRoot,cn=ldbm database,cn=plugins,cn=config
changetype: modify
replace: nsIndexIDListScanLimit
nsIndexIDListScanLimit: limit=-1 type=eq values=inetOrgPerson
Depending on your results we might tweak the nsIndexIDListScanLimit value again. This change should take effect immediately - no need to restart.
Let me know how it goes.
Thanks,
Mark
Oh – no notes in that search, but there are notes in the preceding and subsequent searches. It looks like it returns the first 100 results then searches for groupOfNames for each of those users, and then repeats until all 30,000 users is done:
08/Dec/2014:07:08:04 -0800] conn=130261 fd=64 slot=64 SSL connection from <ip addr 1> to <ip addr 2>[08/Dec/2014:07:08:04 -0800] conn=130261 SSL 256-bit AES[08/Dec/2014:07:08:04 -0800] conn=130261 op=0 BIND dn="<service account>" method=128 version=3[08/Dec/2014:07:08:04 -0800] conn=130261 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="<service account>"[08/Dec/2014:07:08:04 -0800] conn=130261 op=1 SRCH base="" scope=0 filter="(objectClass=*)" attrs="supportedControl"[08/Dec/2014:07:08:04 -0800] conn=130261 op=1 RESULT err=0 tag=101 nentries=1 etime=0[08/Dec/2014:07:08:04 -0800] conn=130261 op=2 SRCH base="ou=EMPLOYEES,dc=<our dc>" scope=2 filter="(&(objectClass=inetOrgPerson))" attrs="businessCategory carLicense cn departmentNumber description destinationIndicator displayName distinguishedName employeeNumber employeeType facsimileTelephoneNumber givenName groups homePhone homePostalAddress initials internationalISDNNumber l mail manager mobile o objectClass o ou pager physicalDeliveryOfficeName postOfficeBox postalAddress postalCode preferredDeliveryMethod preferredLanguage registeredAddress roomNumber secretary seeAlso sn st street telephoneNumber teletexTerminalIdentifier telexNumber title ubcEduCwlPUID uid nsAccountLock passwordRetryCount"[08/Dec/2014:07:08:04 -0800] conn=130261 op=2 SORT cn (*)[08/Dec/2014:07:08:04 -0800] conn=130261 op=2 RESULT err=0 tag=101 nentries=100 etime=0 notes=P,A[08/Dec/2014:07:08:04 -0800] conn=130262 fd=83 slot=83 SSL connection from <ip addr 1> to <ip addr 2>[08/Dec/2014:07:08:04 -0800] conn=130262 SSL 256-bit AES[08/Dec/2014:07:08:04 -0800] conn=130262 op=0 BIND dn="<service account>" method=128 version=3[08/Dec/2014:07:08:04 -0800] conn=130262 op=0 RESULT err=0 tag=97 nentries=0 etime=0 dn="<service account>"[08/Dec/2014:07:08:04 -0800] conn=130262 op=1 SRCH base="ou=GROUPS,dc=<our dc>" scope=2 filter="(&(objectClass=groupOfNames)(member=uid=<login_x>,ou=EMPLOYEES,dc=<our dc>))" attrs="1.1"[08/Dec/2014:07:08:04 -0800] conn=130262 op=1 RESULT err=0 tag=101 nentries=5 etime=0… <snip>[08/Dec/2014:07:08:05 -0800] conn=130262 op=100 SRCH base="ou=GROUPS,dc=<our dc>" scope=2 filter="(&(objectClass=groupOfNames)(member=uid=<login_x+100>,ou=EMPLOYEES,dc=<our dc>))" attrs="1.1"[08/Dec/2014:07:08:05 -0800] conn=130262 op=100 RESULT err=0 tag=101 nentries=3 etime=0[08/Dec/2014:07:08:05 -0800] conn=130261 op=3 SRCH base="ou=EMPLOYEES,dc=<our dc>" scope=2 filter="(&(objectClass=inetOrgPerson))" attrs="businessCategory carLicense cn departmentNumber description destinationIndicator displayName distinguishedName employeeNumber employeeType facsimileTelephoneNumber givenName groups homePhone homePostalAddress initials internationalISDNNumber l mail manager mobile o objectClass o ou pager physicalDeliveryOfficeName postOfficeBox postalAddress postalCode preferredDeliveryMethod preferredLanguage registeredAddress roomNumber secretary seeAlso sn st street telephoneNumber teletexTerminalIdentifier telexNumber title ubcEduCwlPUID uid nsAccountLock passwordRetryCount"… <snip>[08/Dec/2014:07:08:05 -0800] conn=130261 op=3 RESULT err=0 tag=101 nentries=100 etime=0 notes=U,P
Oh cool – I didn’t know about logconv.pl! Output follows.
Thanks again,Trev
Total Log Lines Analysed: 799338
----------- Access Log Output ------------
Start of Logs: 08/Dec/2014:05:56:33End of Logs: 08/Dec/2014:10:31:46
Processed Log Time: 4 Hours, 35 Minutes, 13 Seconds
Restarts: 0Total Connections: 87873- StartTLS Connections: 0- LDAPS Connections: 87265- LDAPI Connections: 0Peak Concurrent Connections: 34Total Operations: 227313Total Results: 227314Overall Performance: 100.0%
Searches: 109051 (6.60/sec) (396.24/min)Modifications: 28121 (1.70/sec) (102.18/min)Adds: 69 (0.00/sec) (0.25/min)Deletes: 0 (0.00/sec) (0.00/min)Mod RDNs: 0 (0.00/sec) (0.00/min)Compares: 0 (0.00/sec) (0.00/min)Binds: 85702 (5.19/sec) (311.40/min)
Proxied Auth Operations: 0Persistent Searches: 0Internal Operations: 0Entry Operations: 0Extended Operations: 4370Abandoned Requests: 0Smart Referrals Received: 0
VLV Operations: 0VLV Unindexed Searches: 0VLV Unindexed Components: 0SORT Operations: 3
Entire Search Base Queries: 0Paged Searches: 505Unindexed Searches: 2Unindexed Components: 502
Unindexed Search #1 (notes=A)- Date/Time: 08/Dec/2014:10:30:12- Connection Number: 215422- Operation Number: 2- Etime: 1- Nentries: 100- IP Address: 10.7.8.105- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))
Unindexed Search #2 (notes=A)- Date/Time: 08/Dec/2014:07:08:04- Connection Number: 130261- Operation Number: 2- Etime: 0- Nentries: 100- IP Address: 10.7.8.105- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))
Unindexed Component #1 (notes=U)- Date/Time: 08/Dec/2014:05:56:45- Connection Number: 129083- Operation Number: 117- Etime: 0- Nentries: 100- IP Address: Unknown_Host- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))
Unindexed Component #2 (notes=U)- Date/Time: 08/Dec/2014:05:57:10- Connection Number: 129083- Operation Number: 122- Etime: 0- Nentries: 100- IP Address: Unknown_Host- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))…<snip>Unindexed Component #502 (notes=U)- Date/Time: 08/Dec/2014:10:30:47- Connection Number: 215422- Operation Number: 3- Etime: 0- Nentries: 100- IP Address: 10.7.8.105- Search Base: ou=employees,dc=<our dc>- Search Scope: 2 (subtree)- Search Filter: (&(objectclass=inetorgperson))
FDs Taken: 87873FDs Returned: 87863Highest FD Taken: 275
Broken Pipes: 0Connections Reset By Peer: 0Resource Unavailable: 0Max BER Size Exceeded: 0
Binds: 85702Unbinds: 83924- LDAP v2 Binds: 0- LDAP v3 Binds: 85702- AUTOBINDs: 0- SSL Client Binds: 0- Failed SSL Client Binds: 0- SASL Binds: 0- Directory Manager Binds: 0- Anonymous Binds: 0- Other Binds: 85702
----- Connection Latency Details -----
(in seconds) <=1 2 3 4-5 6-10 11-15 >15--------------------------------------------------------------------------(# of connections) 87553 169 27 18 5 2 89
----- Current Open Connection IDs -----
Conn Number: 201302 (10.7.127.250)Conn Number: 217063 (10.7.8.105)Conn Number: 217064 (10.7.8.105)Conn Number: 217061 (10.7.8.105)Conn Number: 132242 (10.7.0.52)Conn Number: 196007 (10.7.127.250)Conn Number: 196107 (10.7.127.250)Conn Number: 215425 (10.7.8.105)Conn Number: 217060 (10.7.8.105)Conn Number: 215422 (10.7.8.105)Conn Number: 207139 (10.7.127.250)Conn Number: 217062 (10.7.8.105)
----- Errors -----
err=0 224008 Successful Operationserr=32 1652 No Such Objecterr=53 1652 Unwilling To Performerr=49 1 Invalid Credentials (Bad Password)err=65 1 Objectclass Violation
----- Top 20 Failed Logins ------
1 uid=ehannah,ou=employees,dc=<our dc>
From the IP address(s) :
1 10.7.127.250
----- Total Connection Codes -----
U1 83924 Cleanly Closed ConnectionsB1 1749 Bad Ber Tag Encountered
----- Top 20 Clients -----
Number of Clients: 13
[1] Client: 10.7.8.10583638 - Connections83631 - U1 (Cleanly Closed Connections)
[2] Client: 137.82.5.971652 - Connections1652 - B1 (Bad Ber Tag Encountered)
[3] Client: 142.103.218.1071095 - Connections
[4] Client: 142.103.218.1061094 - Connections
[5] Client: 10.7.127.250112 - Connections106 - U1 (Cleanly Closed Connections)1 - B1 (Bad Ber Tag Encountered)
[6] Client: 127.0.0.157 - Connections55 - B1 (Bad Ber Tag Encountered)2 - U1 (Cleanly Closed Connections)
[7] Client: 142.103.149.13655 - Connections41 - B1 (Bad Ber Tag Encountered)14 - U1 (Cleanly Closed Connections)
[8] Client: 10.7.0.5251 - Connections50 - U1 (Cleanly Closed Connections)
[9] Client: 142.103.106.14129 - Connections29 - U1 (Cleanly Closed Connections)
[10] Client: 142.103.106.14328 - Connections28 - U1 (Cleanly Closed Connections)
[11] Client: 142.103.106.14523 - Connections23 - U1 (Cleanly Closed Connections)
[12] Client: 142.103.106.13523 - Connections23 - U1 (Cleanly Closed Connections)
[13] Client: 142.103.106.14216 - Connections16 - U1 (Cleanly Closed Connections)
----- Top 20 Bind DN's -----
Number of Unique Bind DN's: 18
83795 uid=eldap-svciiq,ou=service accounts,ou=eldap,ou=services,dc=id,dc=ubc,dc=ca1652 cn=bill fisher,ou=people,dc=dartware,dc=com112 cn=directory manager55 uid=eldap-svcvpnsync,ou=service accounts,ou=eldap,ou=services,dc=id,dc=ubc,dc=ca51 cn=replication manager,cn=config6 uid=alexnam,ou=employees,dc=<our dc>4 uid=priyanth,ou=employees,dc=<our dc>4 uid=arsuther,ou=employees,dc=<our dc>4 uid=huynhh,ou=employees,dc=<our dc>3 uid=ehannah,ou=employees,dc=<our dc>2 uid=andyfehr,ou=employees,dc=<our dc>2 uid=bullfrog,ou=employees,dc=<our dc>2 uid=eddiefok,ou=employees,dc=<our dc>2 uid=elyoung,ou=employees,dc=<our dc>2 uid=elainez,ou=employees,dc=<our dc>2 uid=flokam,ou=employees,dc=<our dc>2 uid=mailvaga,ou=employees,dc=<our dc>2 uid=sjones,ou=employees,dc=<our dc>
----- Top 20 Search Bases -----
Number of Unique Search Bases: 27876
78174 ou=groups,dc=<our dc>1652 ou=people,dc=dartware,dc=com676 ou=employees,dc=<our dc>312 ou=people,dc=<our dc>220 cn=mapping tree,cn=config124 root dse13 cn=idm-status-activeemployees,ou=idm,ou=role,ou=groups,dc=<our dc>3 uid=jbartnik,ou=employees,dc=<our dc>2 uid=rodsprus,ou=employees,dc=<our dc>2 uid=zanottic,ou=employees,dc=<our dc>2 uid=ttucker,ou=employees,dc=<our dc>2 uid=dbilczuk,ou=employees,dc=<our dc>2 uid=phys2012,ou=employees,dc=<our dc>2 uid=ydeng,ou=employees,dc=<our dc>2 uid=natcrang,ou=employees,dc=<our dc>2 uid=lillaroy,ou=employees,dc=<our dc>2 uid=wpr8847,ou=employees,dc=<our dc>1 uid=ap8040,ou=employees,dc=<our dc>1 uid=asr5,ou=employees,dc=<our dc>1 uid=belzberg,ou=employees,dc=<our dc>
----- Top 20 Search Filters -----
Number of Unique Search Filters: 30968
28016 (objectclass=*)1652 (cn=bill fisher)504 (&(objectclass=inetorgperson))55 (cn=eldap3*)55 (cn=eldap5*)55 (cn=eldap4*)55 (cn=eldap2*)6 (&(objectclass=groupofnames)(member=uid=alexnam,ou=employees,dc=<our dc>))5 (&(objectclass=groupofnames)(member=uid=ehannah,ou=employees,dc=<our dc>))5 (&(objectclass=groupofnames)(member=uid=priyanth,ou=employees,dc=<our dc>))5 (&(objectclass=groupofnames)(member=uid=arsuther,ou=employees,dc=<our dc>))4 (&(objectclass=groupofnames)(member=uid=ydeng,ou=employees,dc=<our dc>))4 (&(objectclass=inetorgperson)(ubceducwlloginkey=<login key 1>))4 (&(objectclass=groupofnames)(member=uid=elyoung,ou=employees,dc=<our dc>))4 (&(objectclass=groupofnames)(member=uid=zanottic,ou=employees,dc=<our dc>))4 (&(uid=devlin11))4 (&(objectclass=groupofnames)(member=uid=huynhh,ou=employees,dc=<our dc>))4 (&(objectclass=inetorgperson)(ubceducwlloginkey=<login key 2>))4 (&(objectclass=groupofnames)(member=uid=lillaroy,ou=employees,dc=<our dc>))4 (&(objectclass=groupofnames)(member=uid=rodsprus,ou=employees,dc=<our dc>))
----- Top 20 Most Frequent etimes -----
219090 etime=08202 etime=120 etime=21 etime=1471 etime=135
----- Top 20 Longest etimes -----
etime=147 1etime=135 1etime=2 20etime=1 8202etime=0 219090
----- Top 20 Largest nentries -----
nentries=100 502nentries=34 1nentries=27 1nentries=26 1nentries=7 3nentries=6 9nentries=5 85nentries=4 334nentries=3 1957nentries=2 6701nentries=1 85038nentries=0 14420
----- Top 20 Most returned nentries -----
85038 nentries=114420 nentries=06701 nentries=21957 nentries=3502 nentries=100334 nentries=485 nentries=59 nentries=63 nentries=71 nentries=261 nentries=271 nentries=34
----- Extended Operations -----
2185 2.16.840.1.113730.3.5.12 DS90 Start Replication Request2185 2.16.840.1.113730.3.5.5 End Replication Request (incremental update)
----- Top 20 Most Requested Attributes -----
78174 1.156799 o28712 description28712 cn28712 nsAccountLock28711 givenName28711 ubcEduCwlPUID28711 uid28711 displayName28711 employeeNumber28711 mail28711 objectClass28711 sn28400 ou28400 passwordRetryCount28400 distinguishedName28399 businessCategory28399 employeeType28399 facsimileTelephoneNumber28399 groups
----- Recommendations -----
1. You have unindexed searches, this can be caused from a search on an unindexed attribute, or your returned results exceeded the allidsthreshold. Unindexed searches are not recommended. To refuse unindexed searches, switch 'nsslapd-require-index' to 'on' under your database entry (e.g. cn=UserRoot,cn=ldbm database,cn=plugins,cn=config).
2. You have unindexed components, this can be caused from a search on an unindexed attribute, or your returned results exceeded the allidsthreshold. Unindexed components are not recommended. To refuse unindexed searches, switch 'nsslapd-require-index' to 'on' under your database entry (e.g. cn=UserRoot,cn=ldbm database,cn=plugins,cn=config).
Cleaning up temp files...Done.
From: Mark Reynolds <mareynol@redhat.com>
Reply-To: "mreynolds@redhat.com" <mreynolds@redhat.com>
Date: Tuesday, December 9, 2014 at 7:25 AM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>, Trevor Fong <trevor.fong@ubc.ca>
Subject: Re: [389-users] 389-ds and Multi CPU's
On 12/08/2014 05:43 PM, Fong, Trevor wrote:
It's Mark :-) I get that a lot for some reason.Hi Mike,
Yeah this looks fine.
Thanks for the reply. The typical result of the result is:
[08/Dec/2014:07:08:04 -0800] conn=130262 op=1 RESULT err=0 tag=101 nentries=5 etime=0
Do you mean in the entire access log, or just for that search?
There are no notes=A/notes=U in the results.
Can you run logconv.pl and post the results? "logconv.pl -V <access logs>"
Thanks Trevor,
Mark
Objectclass and member are both indexed.There were 30,000-odd searches on conn=130262, which took 34 mins.
Thanks,Trev
From: Mark Reynolds <mareynol@redhat.com>
Reply-To: "mreynolds@redhat.com" <mreynolds@redhat.com>
Date: Monday, December 8, 2014 at 11:29 AM
To: "389-users@lists.fedoraproject.org" <389-users@lists.fedoraproject.org>, Trevor Fong <trevor.fong@ubc.ca>
Subject: Re: [389-users] 389-ds and Multi CPU's
On 12/08/2014 02:08 PM, Fong, Trevor wrote:
Trevor,Hi Everyone,
We’ve inherited a 389-ds system (1.2.11.15-48.el6_6) that is running on a VM provisioned with a single CPU. We have been experiencing high CPU with a client that connects with a single connection, and then runs large amounts of queries of the form:
SRCH base="ou=GROUPS,dc=<our dc>" scope=2 filter="(&(objectClass=groupOfNames)(member=uid=<loginname>,ou=EMPLOYEES,<our dc>))" attrs=“1.1"
From the access log, what is the result of this search? etime? It there a notes=U/notes=A in the result? It could be an unindexed search which would cause the high CPU.
Thanks,
Mark
We’re wondering if adding extra CPU’s to the vm will make things better. The original engineer noted that at the time of implementation, he had come across some notes that indicated that the underlying process was single threaded and adding extra CPU’s would not make any improvement; in fact, on heavily loaded vm infrastructure like ours, may make things worse as the the vm would have to wait for the CPU to become available. Is this still true?
Thanks a lot,Trev
-- 389 users mailing list 389-users@lists.fedoraproject.orghttps://admin.fedoraproject.org/mailman/listinfo/389-users
-- 389 users mailing list 389-users@lists.fedoraproject.orghttps://admin.fedoraproject.org/mailman/listinfo/389-users