I've been running some more tests before setting up the ticket, but I think I have
enough information now. The uniqueMember attribute has extra processing overhead, but the
necessary optimization might apply across the board for all attributes. I found also that
adding large sets of values for other attributes also increases modification times
heavily, though not quite as much as uniqueMember. Luckily, the modification delay is
based on the size of the modification rather than the size of the entry, so even if the
modification is done to a 100K-value attribute, if the modification is only to remove a
few members and add a few others, then the change is still relatively quick. The delay is
noticed most when first setting up a group, for instance, adding 100K members to an empty
group takes 2.5 hours on 389 as opposed to 1 minute on Sun DS.
Also during this testing I have noticed a memory leak when running large quantities of
ldapmodify operations. When I set up a loop to delete and then re-add the
eduPersonEntitlement attribute across 100K entries, I found that memory consumption
continuously increased and the server crashed after the fifth iteration of this loop.
(And this one really is with ldapmodify and is not related to my earlier issues with
excessively creating tombstones by deleting and adding entire entries). Before digging
into this too deeply and making another ticket, I wanted to ask if this had been noticed
and fixed in the 1.2.10 release? I am using the default 22.214.171.124 release. I'm
guessing it hasn't since I didn't see it in the release notes.
I am starting up the server with the valgrind command you recommended a few messages back
to see if I can spot the leak, though of course with valgrind in the mix, the overhead and
runtimes are, as might be expected, much increased.
On Apr 19, 2012, at 1:42 PM, Rich Megginson wrote:
OK. If you've ruled out the possibility that some plugin is
interfering with the processing, then it must be something we will have to fix in the core
server. Please file a ticket at https://fedorahosted.org/389