[SSSD] Thoughts...

Jan Zelený jzeleny at redhat.com
Wed Oct 5 11:54:24 UTC 2011


> CCing sssd-devel
> 
> On Tue, 2011-10-04 at 22:41 +0000, JR Aquino wrote:
> > So I have been looking through the results of ldbsearch and have been
> > wondering...
> > 
> > Why is it that I see cached entries for objects (hbac/ hosts/ etc) that
> > have no correlation to the host itself?
> > 
> > Might it be more optimized to search / cache only the data relevant to
> > the client system in question?
> > 
> > Surely that would cut down on the exponential costs against an ldap
> > database with grown hosts/users/hbacrules/sudorules ?
> 
> Well, the issue is this: rules can specify hostgroups as well as hosts.
> So in order to decide which rules to download, we need to know which
> hostgroups the current machine is a member of. This results in a minimum
> of two ldap searches (one for the hosts, and then one for the
> hostgroups) to get the machine-specific information. We then need to
> perform a rule search for just the rules that apply to that machine.
> 
> After we have the rule, we'll need to perform an iterative series of
> lookups for every host and hostgroup that the rules we just acquired
> list as a source host. Now, suppose that one or more of the hostgroups
> specified in source host contains thousands of host entries?
> 
> So in other words, doing it this way will result in a net increase in
> the number of back-and-forth LDAP exchanges. In theory this is much
> slower than just grabbing everything in two lookups up front (getting
> all hosts and hostgroups) and doing the processing locally.
> 
> In practice, we're hitting a performance issue because of the way we're
> updating the data. So it's better all-around to resolve the issue here,
> rather than deferring the problem by changing the lookup order around
> and potentially increasing the LDAP lookups. And since we still might
> end up with rules that have to handle a lot of hosts, we'll still have
> the same update issue.
> 
> The problem in the source is this: I tried to simplify the update
> process by simply wiping out the hosts and hostgroups subtrees in the
> cache and then writing the new values in. The problem with this is that
> deleting entries with memberOf attributes is an expensive operation,
> which I did not take into account.
> 
> There are two approaches to resolving this that I'm taking into
> consideration.
> 
> 1) Instead of doing a full purge followed by a full write, make only
> changes. This will be a somewhat more complex logic to follow, but if
> little or nothing has changed on the server, then it will save us a LOT
> of memberOf operations.
> 
> 2) Add a new feature to the memberOf plugin to allow us to defer
> processing memberOf until all entries have first been written (keeping
> an index of the attributes that will need processing). This is *very*
> complicated and potentially dangerous (though also has the potential for
> the highest performance gain), so I'd prefer to try option 1 first.

I was thinking about the same thing when I was in the middle of the first run 
of rewriting the memberof plugin. But then I completely changed the design to 
the current one and stopped thinking about this.

In fact in my new design I count with the fact that we add only one group to 
the membership structure at a time. To support the proposed recalculation 
would take a major change in the pugin design. I'd wait for results of the 
profiling I'm planning to do (I guess next week) and see if the concept needs 
to be changed. If so, I would definitely consider this approach and let you 
know.


Jan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: This is a digitally signed message part.
URL: <https://lists.fedorahosted.org/pipermail/sssd-devel/attachments/20111005/795c583a/attachment.sig>


More information about the sssd-devel mailing list