[SSSD] Thoughts...

Simo Sorce simo at redhat.com
Wed Oct 5 11:56:09 UTC 2011


On Wed, 2011-10-05 at 07:30 -0400, Stephen Gallagher wrote:
> CCing sssd-devel
> 
> On Tue, 2011-10-04 at 22:41 +0000, JR Aquino wrote:
> > So I have been looking through the results of ldbsearch and have been
> > wondering...
> > 
> > Why is it that I see cached entries for objects (hbac/ hosts/ etc) that
> > have no correlation to the host itself?
> > 
> > Might it be more optimized to search / cache only the data relevant to
> > the client system in question?
> > 
> > Surely that would cut down on the exponential costs against an ldap
> > database with grown hosts/users/hbacrules/sudorules ? 
> 
> Well, the issue is this: rules can specify hostgroups as well as hosts.
> So in order to decide which rules to download, we need to know which
> hostgroups the current machine is a member of. This results in a minimum
> of two ldap searches (one for the hosts, and then one for the
> hostgroups) to get the machine-specific information. We then need to
> perform a rule search for just the rules that apply to that machine.
> 
> After we have the rule, we'll need to perform an iterative series of
> lookups for every host and hostgroup that the rules we just acquired
> list as a source host. Now, suppose that one or more of the hostgroups
> specified in source host contains thousands of host entries?
> 
> So in other words, doing it this way will result in a net increase in
> the number of back-and-forth LDAP exchanges. In theory this is much
> slower than just grabbing everything in two lookups up front (getting
> all hosts and hostgroups) and doing the processing locally.
> 
> In practice, we're hitting a performance issue because of the way we're
> updating the data. So it's better all-around to resolve the issue here,
> rather than deferring the problem by changing the lookup order around
> and potentially increasing the LDAP lookups. And since we still might
> end up with rules that have to handle a lot of hosts, we'll still have
> the same update issue.
> 
> The problem in the source is this: I tried to simplify the update
> process by simply wiping out the hosts and hostgroups subtrees in the
> cache and then writing the new values in. The problem with this is that
> deleting entries with memberOf attributes is an expensive operation,
> which I did not take into account.
> 
> There are two approaches to resolving this that I'm taking into
> consideration.
> 
> 1) Instead of doing a full purge followed by a full write, make only
> changes. This will be a somewhat more complex logic to follow, but if
> little or nothing has changed on the server, then it will save us a LOT
> of memberOf operations.
> 
> 2) Add a new feature to the memberOf plugin to allow us to defer
> processing memberOf until all entries have first been written (keeping
> an index of the attributes that will need processing). This is *very*
> complicated and potentially dangerous (though also has the potential for
> the highest performance gain), so I'd prefer to try option 1 first.

3) speed up deletion of entire subtrees using a special control in the
memberof plugin, relatively simple and safe if you ask me.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York




More information about the sssd-devel mailing list