https://fedorahosted.org/389/ticket/412

https://fedorahosted.org/389/attachment/ticket/412/0001-Trac-Ticket-412-memberof-performance-enhancement.patch

Fix description:
memberof.c: replaced DN strings with Slapi_DN and set the
normalized info to Slapi_Value flags. It reduces the number
of slaip_dn_normalize_ext call by ~25%.

attr.c, slapi-plugin.h: introduced a new API slapi_attr_
value_cmp_ext which takes Slapi_Value instead of struct
berval. By replacing with Slapi_Value, the value flag
(e.g., normalized info) can be passed to the syntax plugin.

value.c: changed slapi_value_compare to call slapi_attr_
value_cmp_ext instead of slapi_attr_value_cmp.

Replying to nkinder:

What sort of performance increase do these changes give you in your testing?

Unfortunately, there was no observable performance gain. Let me explain it with the test env next...

As replied to the Nathan's question, I could not see any performance gain in the elapsed time, although it reduced the count of slapi_dn_normalize_ext by ~25%.

But the server's behaviour observed in the test was interesting.  The dominant time of the elapsed time spent in the memberof operation is likely in libdb.  The time difference 14 sec vs. 27 sec between "Straight ldapmodify ...; sh memof_script.sh" and "ldapmodify ...; restart the server; [ldapsearch <all>; ]sh memof_script.sh" cannot be explained in the server level.   Callgrind outputs almost the same graph except the libdb internal.  And once we put all memberof operations in one transaction, the response time is extremely short.  We have not enabled betxn by default yet.  Can we make a plan to do so for brushing up the code especially on Fedora?

Result:

Note: With/Without? the attached patch: 0001-Trac-Ticket-412-memberof-performance-enhancement.patch​, there was no difference in the elapsed time.

This test adds 1000 user entries first (ldapmodify...).
Then memof_script.sh adds a group entry which contains the 1000 members; the script waits for all the user entries have memberof attribute value.
(The test files are attached to the trac #412 with the instructions.)

Straight ldapmodify ...; sh memof_script.sh: 14 seconds
ldapmodify ...; restart the server; sh memof_script.sh: 27 seconds
ldapmodify ...; restart the server; ldapsearch <all>; sh memof_script.sh: 27 seconds
(I.e, the difference is not due to the entries in the entry cache or not)

Enable betxn of the Memberof Plugin
dn: cn=MemberOf? Plugin,cn=plugins,cn=config
nsslapd-pluginType: betxnpostoperation
Straight ldapmodify ...; sh memof_script.sh: < 1 second
ldapmodify ...; restart the server; ldapsearch <all>; sh memof_script.sh: < 1 second

Thanks,
--noriko