[SSSD] SSSD MemberOf / HBAC Process

JR Aquino JR.Aquino at citrix.com
Wed Oct 5 14:11:16 UTC 2011


On Oct 5, 2011, at 4:30 AM, "Stephen Gallagher" <sgallagh at redhat.com> wrote:

> CCing sssd-devel
> 
> On Tue, 2011-10-04 at 22:41 +0000, JR Aquino wrote:
>> So I have been looking through the results of ldbsearch and have been
>> wondering...
>> 
>> Why is it that I see cached entries for objects (hbac/ hosts/ etc) that
>> have no correlation to the host itself?
>> 
>> Might it be more optimized to search / cache only the data relevant to
>> the client system in question?
>> 
>> Surely that would cut down on the exponential costs against an ldap
>> database with grown hosts/users/hbacrules/sudorules ? 
> 
> Well, the issue is this: rules can specify hostgroups as well as hosts.
> So in order to decide which rules to download, we need to know which
> hostgroups the current machine is a member of. This results in a minimum
> of two ldap searches (one for the hosts, and then one for the
> hostgroups) to get the machine-specific information. We then need to
> perform a rule search for just the rules that apply to that machine.
> 
> After we have the rule, we'll need to perform an iterative series of
> lookups for every host and hostgroup that the rules we just acquired
> list as a source host. Now, suppose that one or more of the hostgroups
> specified in source host contains thousands of host entries?
> 
> So in other words, doing it this way will result in a net increase in
> the number of back-and-forth LDAP exchanges. In theory this is much
> slower than just grabbing everything in two lookups up front (getting
> all hosts and hostgroups) and doing the processing locally.
> 
> In practice, we're hitting a performance issue because of the way we're
> updating the data. So it's better all-around to resolve the issue here,
> rather than deferring the problem by changing the lookup order around
> and potentially increasing the LDAP lookups. And since we still might
> end up with rules that have to handle a lot of hosts, we'll still have
> the same update issue.
> 
> The problem in the source is this: I tried to simplify the update
> process by simply wiping out the hosts and hostgroups subtrees in the
> cache and then writing the new values in. The problem with this is that
> deleting entries with memberOf attributes is an expensive operation,
> which I did not take into account.
> 
> There are two approaches to resolving this that I'm taking into
> consideration.
> 
> 1) Instead of doing a full purge followed by a full write, make only
> changes. This will be a somewhat more complex logic to follow, but if
> little or nothing has changed on the server, then it will save us a LOT
> of memberOf operations.
> 
> 2) Add a new feature to the memberOf plugin to allow us to defer
> processing memberOf until all entries have first been written (keeping
> an index of the attributes that will need processing). This is *very*
> complicated and potentially dangerous (though also has the potential for
> the highest performance gain), so I'd prefer to try option 1 first.

I have a use case that I would like to share so that it is measured in optimizations:

Given the "cloud" world we live in today, please consider a network with thousands of servers who have both frequent addition / deletion, as well as a lot of activity in bursts.  So, given a set of 1000 servers who's cache has timed out, a batch ssh job to all of them would cause 1000 requests for All of the hosts/host groups in the ldap directory.  It's not clear to me how the alternative methods affect the ldap load comparatively in a situation like this. 


More information about the sssd-devel mailing list