[SSSD] [PATCH] Use dereference to speed up RFC2307bis group processing

Stephen Gallagher sgallagh at redhat.com
Mon May 9 14:20:41 UTC 2011


On Wed, 2011-05-04 at 13:19 +0200, Jakub Hrozek wrote:
> The attached patch set uses special dereference LDAP controls to speed
> up RFC2307bis group processing. In my setup, with the LDAP server being
> in Boston and client in Brno, the time for "getent group ipausers"
> consisting of 2000 entries went down from 1.5 minute to 4 seconds.
> 
> One part of the patchset I'd like to draw attention to is the heuristics
> in patch #9. Dereference can potentially provide huge savings, but it's
> an all-or-nothing operation. As far as I know, you can't only say "only
> dereference entries matching this filter", at least not using the
> OpenLDAP dereference.
> 
> This could actually slow down the processing - consider if a single IPA
> user being created triggered a full dereference of "ipausers" group. In
> order to overcome this, the code includes a simple heuristics that
> either issues a full dereference or a series of single lookups.
> 
> The heuristics itself was pulled out of thin air, really. If there are
> any suggestions on a better one, I'll be glad to hear them.
> 
> And now the patches:
> 
> [PATCH 1/9] sdap_get_generic_ext
> Add a private sdap_get_generic_ext_send()/_recv() request that
> exposes more of ldap_search_ext options, in particular the server
> contols. The existing sdap_generic_search_send()/_recv() request
> is now a thin wrapper around the new _ext request.
> 
> The other important change is that an entry parsing is a callback now.
> That was done in order to allow custom parsing for results such as
> OpenLDAP deref or Attribute Scoped Queries.

Ack

> 
> [PATCH 2/9] Add support for Attribute Scoped Queries
> For more details on ASQ, see:
> http://msdn.microsoft.com/en-us/library/aa366976%28VS.85%29.aspx
> http://msdn.microsoft.com/en-us/library/aa746418%28v=VS.85%29.aspx
> 

Ack

> [PATCH 3/9] OpenLDAP dereference searches
> For more details, see:
> http://tools.ietf.org/html/draft-masarati-ldap-deref-00

Nack

What's the point of derefcount in sdap_x_deref_parse_entry()? It's only
ever assigned and never read.

sdap_parse_deref() assumes that it can determine the mapping to use
based on the objectClass, but this falls down in situations like Magic
Private Groups, where user entries have both the posixAccount and
posixGroup attribute.



> 
> [PATCH 4/9] Generic dereference search
> A generic wrapper around ASQ and OpenLDAP dereference searches.
> https://fedorahosted.org/sssd/ticket/635
> 

Nack.
It would be preferable to have a single sdap_deref_search_done()
function instead of a separate one for each deref type. See
sdap_get_initgr_done() for an example of what I mean. There should be
separate _recv() functions, but only one common _done() function to make
it simpler to follow execution.


> [PATCH 5/9] append_attrs_to_array allocs on a memory context
> A convenience function
> 

Nack.
We discussed this on IRC. This function serves no real purpose (it was
never used prior to patch 9 in this set) and this modification has zero
net effect, since the mem_ctx attribute has no effect on a
talloc_realloc() for previously-allocated memory.

We decided to remove this function entirely, since it just causes
confusion.

> [PATCH 6/9] Change sysdb_add_fake_user to add OriginalDN
> RFC2307bis code relies heavily on originalDN, so the fake users need to
> have an option to store it, too.
> 

Ack.



Review still pending for the RFC2307bis results.

> [PATCH 7/9] Use fake users during RFC2307bis nested group processing
> Instead of downloading complete user data which is potentionally very
> slow, only download the necessary minimum information and store the
> users as dummy entries.
> 
> [PATCH 8/9] Refactor RFC2307bis nested group processing
> This patch splits checking cache and hash tables into standalone
> functions. This will make it easy to reuse the code in a new branch that
> uses dereferencing.
> 
> [PATCH 9/9] Use dereference in processing RFC2307bis nested groups
> Instead of issuing N LDAP requests when processing a group with N users,
> utilize the dereference functionality to pull down all the members in a
> single LDAP request.



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <https://lists.fedorahosted.org/pipermail/sssd-devel/attachments/20110509/a758bd94/attachment.sig>


More information about the sssd-devel mailing list