On Tue, 14 Sep 2010 15:13:19 +0200
Ralf Haferkamp <rhafer(a)suse.de> wrote:
On Friday 10 September 2010 13:48:31 Simo Sorce wrote:
> On Fri, 10 Sep 2010 12:06:25 +0200
> Ralf Haferkamp <rhafer(a)suse.de> wrote:
> > On Thursday 09 September 2010 18:33:26 Simo Sorce wrote:
> > > So we have the following scenarios:
> > >
> > > 1) If we use rfc2307 classic with memberUid attributes, we can
> > > just create the fake/expired users and be done with it.
> > Agreed.
> > > 2) If we use rfc2307bis w/o nested groups we can count the
> > > number of members on the group entry and switch to a full user
> > > enumeration if the number of members is greater than a defined
> > > (potentially user defined) threshold. Assuming a threshold value
> > > of 100, if we have 10 members we just do 10 lookups, while, if
> > > we have > 100 member we do a full enumeration (it's a single
> > > ldap search, and if
> > > modifyTimestamp is used also highly optimized after the first
> > > search) so we are sure all users are there with the right name.
> > I think that's an optimization that could be worth exploring
> > later. In a first iteration I'd just go for implementing the
> > complete group lookup by looking up every single user regardless
> > of how large the group is. (plus the possiblity to completely
> > disable member unrolling through the configuration)
> Ok, although performances will suck so much I bet we will have to
> implement the optimization pretty soon.
After doing some test here, I really wonder if the LDAP lookups are
going to be the bottleneck. I just tested a slightly improved version
of my previous patch with a group with 500 members (not really big, I
know). Doing "getent group" on that group takes about 2sec on my
testsystem, which is really slow (nss_ldap needs 0.5sec). But looking
closer at that, the LDAP searches and the processing of the search
results accounts only for about 160ms of that two seconds. Much more
time (more than 1sec) is then spent in saving the user objects and
the group in sysdb. Probably it would be worth looking for
optimizations in that area first. Other calls (initgroups?) could
benefit from that as well.
Not that easy unfortunately.
Note, I already save all users in one transaction instead of
a new transaction for every user. Also note that, opening the sysdb
with LDB_FLG_NOSYNC brings down the time to about 1.3sec.
Yes, we have a bit of a bottlneck issue with LDB, because every
transaction requires at least to fsyncs.
Unfortunately we can't run with LDB_FLG_NOSYNC as that would make it
too easy to corrupt the db in case of power loss.
Simo Sorce * Red Hat, Inc * New York