[SSSD] Thoughts...

Simo Sorce simo at redhat.com
Wed Oct 5 12:26:46 UTC 2011


On Wed, 2011-10-05 at 07:58 -0400, Stephen Gallagher wrote:
> On Wed, 2011-10-05 at 07:56 -0400, Simo Sorce wrote:
> > On Wed, 2011-10-05 at 07:30 -0400, Stephen Gallagher wrote:
> > > CCing sssd-devel
> > > 
> > > On Tue, 2011-10-04 at 22:41 +0000, JR Aquino wrote:
> > > > So I have been looking through the results of ldbsearch and have been
> > > > wondering...
> > > > 
> > > > Why is it that I see cached entries for objects (hbac/ hosts/ etc) that
> > > > have no correlation to the host itself?
> > > > 
> > > > Might it be more optimized to search / cache only the data relevant to
> > > > the client system in question?
> > > > 
> > > > Surely that would cut down on the exponential costs against an ldap
> > > > database with grown hosts/users/hbacrules/sudorules ? 
> > > 
> > > Well, the issue is this: rules can specify hostgroups as well as hosts.
> > > So in order to decide which rules to download, we need to know which
> > > hostgroups the current machine is a member of. This results in a minimum
> > > of two ldap searches (one for the hosts, and then one for the
> > > hostgroups) to get the machine-specific information. We then need to
> > > perform a rule search for just the rules that apply to that machine.
> > > 
> > > After we have the rule, we'll need to perform an iterative series of
> > > lookups for every host and hostgroup that the rules we just acquired
> > > list as a source host. Now, suppose that one or more of the hostgroups
> > > specified in source host contains thousands of host entries?
> > > 
> > > So in other words, doing it this way will result in a net increase in
> > > the number of back-and-forth LDAP exchanges. In theory this is much
> > > slower than just grabbing everything in two lookups up front (getting
> > > all hosts and hostgroups) and doing the processing locally.
> > > 
> > > In practice, we're hitting a performance issue because of the way we're
> > > updating the data. So it's better all-around to resolve the issue here,
> > > rather than deferring the problem by changing the lookup order around
> > > and potentially increasing the LDAP lookups. And since we still might
> > > end up with rules that have to handle a lot of hosts, we'll still have
> > > the same update issue.
> > > 
> > > The problem in the source is this: I tried to simplify the update
> > > process by simply wiping out the hosts and hostgroups subtrees in the
> > > cache and then writing the new values in. The problem with this is that
> > > deleting entries with memberOf attributes is an expensive operation,
> > > which I did not take into account.
> > > 
> > > There are two approaches to resolving this that I'm taking into
> > > consideration.
> > > 
> > > 1) Instead of doing a full purge followed by a full write, make only
> > > changes. This will be a somewhat more complex logic to follow, but if
> > > little or nothing has changed on the server, then it will save us a LOT
> > > of memberOf operations.
> > > 
> > > 2) Add a new feature to the memberOf plugin to allow us to defer
> > > processing memberOf until all entries have first been written (keeping
> > > an index of the attributes that will need processing). This is *very*
> > > complicated and potentially dangerous (though also has the potential for
> > > the highest performance gain), so I'd prefer to try option 1 first.
> > 
> > 3) speed up deletion of entire subtrees using a special control in the
> > memberof plugin, relatively simple and safe if you ask me.
> 
> Could you describe how this would work? Because I was thinking about
> this and I couldn't come up with a way to make it work.

Right now if you are deleting a subtree you will execute a full memberof
delete for each entry. That means doing expensive searches all over the
tree and changes to each and every entry repeatedly.

Assume you delete 10 groups and you have 100 users each member of each
group. It mean 1000 operations as you change each user in turn to remove
the memberof of the group you are currently handling, 10 times, one for
each group.

The idea is to add a new control to memberof to tell it: deleting this
whole subtree. The control will be passed down and will tell memberof
that it should not do the normal delte operation until the final delete
of the top entry of the subtree. When that is doen we do a single search
for all member/memberof attributes of the form *,<subtree suffix>, and
for each result we remove all member/memberof attribute once.
If the tree is self-contained wrt members this turns out to be a search
that returns 0 entries so no modify operations at all.

If there are members outside of this tree we do one mod operation for
each regardless of how many attribute match per entry.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York




More information about the sssd-devel mailing list