[SSSD] [PATCHES][PRELIMINARY] Add support for AD's chain matching filter

Stephen Gallagher sgallagh at redhat.com
Wed Jun 13 01:16:13 UTC 2012


On Tue, 2012-06-12 at 09:33 -0400, Stephen Gallagher wrote:
> On Tue, 2012-06-12 at 15:29 +0200, Jan Zelený wrote:
> > > On Tue, 2012-06-12 at 08:28 -0400, Simo Sorce wrote:
> > > > On Tue, 2012-06-12 at 07:37 -0400, Stephen Gallagher wrote:
> > > > > On Tue, 2012-06-12 at 10:50 +0200, Jan Zelený wrote:
> > > > > > > On Mon, 2012-06-11 at 21:19 -0400, Stephen Gallagher wrote:
> > > > > > > > New patches attached, along with the results of my (limited)
> > > > > > > > performance
> > > > > > > > testing.
> > > > > > > > 
> > > > > > > > These patches split the option into two, so it can be enabled for
> > > > > > > > initgroups or group lookups separately. The testing I did on
> > > > > > > > group lookups seems to suggest that it's a distinct performance
> > > > > > > > hit.
> > > > > > > 
> > > > > > > I'm wondering if we shouldn't try at least the initgroups by
> > > > > > > default. What's the error for servers that do not recognize the
> > > > > > > syntax ? Can we 'probe' the syntax at connection time (like we
> > > > > > > check for the rootdse) and set a flag about whether it work or not
> > > > > > > ?
> > > > > > > 
> > > > > > > That way we can set at least
> > > > > > > ldap_initgroups_use_matching_rule_in_chain = auto or similar by
> > > > > > > default and have the benefit any time the option is available w/o
> > > > > > > forcing the admins to find out.
> > > > > > > It would be a pretty obscure toggle anyway, very few would take
> > > > > > > benefit if we do not find a way to auto-discover it.
> > > > > > > 
> > > > > > > Simo.
> > > > > > 
> > > > > > The auto-discovery is possible to some degree but a bit ugly. You can
> > > > > > do following search [1]:
> > > > > > 
> > > > > > base: CN=<domain_name>,OU=Domain Controllers,<suffix>
> > > > > > scope: base
> > > > > > attributes: operatingSystemVersion, operatingSystemServicePack
> > > > > > 
> > > > > > And then use following condition to detect if server supports the
> > > > > > feature (the code is not accurate, it only demonstrates the idea.
> > > > > > Some processing of both attributes would be necessary beforehand):
> > > > > > 
> > > > > > if ((operatingSystemVersion == 3790 &&
> > > > > > 
> > > > > >      operatingSystemServicePack >= 2) ||
> > > > > >      operatingSystemVersion > 3790)
> > > > > >      
> > > > > >          return true;
> > > > > > 
> > > > > > else
> > > > > > 
> > > > > >          return false;
> > > > > > 
> > > > > > Of course the code only detects if the AD server supports chaining.
> > > > > > If there is possibilty to turn it off somehow, I don't know how to
> > > > > > detect it. I checked both supportedControl and supportedCapabilities
> > > > > > attributes in rootDSE of Stephen's testing AD server, there is no
> > > > > > mention about this one.
> > > > > 
> > > > > This is an interesting idea, but there are still problems with it.
> > > > > First, you need to identify the domain name (which is not retrievable
> > > > > from the RootDSE, so you'd need to configure that). Second, this only
> > > > > reports what the OS is installed on the server, but Windows Server can
> > > > > be configured to operate in compatibility mode with older versions.
> > > > > (i.e. Windows 2008 R2 servers operating in Windows 2003 compatibility
> > > > > mode during a phased migration).
> > > > > 
> > > > > So this is not sufficient information to make this determination.
> > > > > 
> > > > > However! It looks like we should be able to autodetect this by
> > > > > performing a request using the match rule OID. When I tried to use it
> > > > > against an openldap server, I got back:
> > > > > 
> > > > > search: 4
> > > > > result: 12 Critical extension is unavailable
> > > > > text: Bad search filter
> > > > > 
> > > > > So this is probably sufficient to enable-disable this feature. We need
> > > > > to figure out what would be a reasonable search request to make,
> > > > > however.
> > > > 
> > > > The actual search doesn't really matter, we care only if we get back a
> > > > result or an error. We could search a random user name or any other info
> > > > we normally search at rootdse discovery time.
> > > 
> > > Well, at RootDSE discovery time, we search only the RootDSE with a base
> > > search. Base searches ignore the filter, so we need to make a separate
> > > request. I'm currently investigating just doing a onelevel search of the
> > > user_search_base for an attribute unlikely to exist. This should be
> > > sufficient.
> > 
> > Quoting the documentation you sent earlier:
> > 
> > "Note that when using LDAP_MATCHING_RULE_IN_CHAIN, scope is not limited—it can 
> > be base, one-level, or subtree."
> 
> Ah, you know what. I could have sworn I got a different result the first
> time, but I just re-tested against an openldap server and did in fact
> get the 
> result: 12 Critical extension is unavailable
> when trying to use this filter.
> 
> So I suppose we probably can just do a second base search against the
> rootdse with this specified.


Ok, I've added one new patch to the set, patch 0009. This patch makes
the following additions:
1) Support for LDAP_MATCHING_RULE_IN_CHAIN is now auto-detected during
the RootDSE check phase if either groups or initgroups are supported
2) The meaning of "True" for either option becomes "auto".
3) The default for initgroups is now True/Auto.

I'm also attaching a new set of performance tests I ran. I realized that
my initial run was slightly flawed, in that each test was run from a
fresh startup of SSSD (meaning it incurred the costs of the initial
connection as well). With this second round, I first performed a 'getent
passwd notarealuserinldap' before each timed test, to ensure that the
SSSD was working from a live connection.

The results for groups were largely unchanged, but for initgroups it
reveals something closer to a 3x performance boost in the example data
set, as compared to the 2x boost seen in my first shot at this.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0006-LDAP-Add-ldap_-_use_matching_rule_in_chain-options.patch
Type: text/x-patch
Size: 8551 bytes
Desc: not available
URL: <https://lists.fedorahosted.org/pipermail/sssd-devel/attachments/20120612/1bf9e15f/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0007-LDAP-Add-support-for-AD-chain-matching-extension-in-.patch
Type: text/x-patch
Size: 18846 bytes
Desc: not available
URL: <https://lists.fedorahosted.org/pipermail/sssd-devel/attachments/20120612/1bf9e15f/attachment-0001.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0008-LDAP-Add-support-for-AD-chain-matching-extension-in-.patch
Type: text/x-patch
Size: 15130 bytes
Desc: not available
URL: <https://lists.fedorahosted.org/pipermail/sssd-devel/attachments/20120612/1bf9e15f/attachment-0002.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0009-LDAP-Auto-detect-support-for-the-ldap-match-rule.patch
Type: text/x-patch
Size: 12443 bytes
Desc: not available
URL: <https://lists.fedorahosted.org/pipermail/sssd-devel/attachments/20120612/1bf9e15f/attachment-0003.bin>
-------------- next part --------------
Tests were performed against a single-CPU VM running AD 2008 R2 with 1GB RAM. Monitoring with Windows Task Manager did not reveal any perod during which the CPU went above 35% or the RAM above 850MB.

The server was located across the internet (VPN) to simulate real-world remote network conditions.

The server has one group (topgroup) that contains 3000 test users, as well as a nesting chain of nestedgroup, topgroup, abovetopgroup, aboveabovetop and aboveeventhat. I set ldap_group_nesting_level = 10 in sssd.conf to ensure both approaches showed complete results.

The user kurashima is a direct member of nestedgroup (part of the nesting chain) and sssdgroup (no parents).

My analysis of the performance degradation with the complete id lookup is that there are three pieces to it.
1) Lookups take longer againt the AD server, as the match rule lookups are more CPU intensive. The time on the wire for each LDAP round-trip was approximately double. This is most highly-visible across the internet, as these tests were performed.
2) Switching from the ASQ lookups meant that we lost its (possibly unintended) side-effect that it can return more values than the configured paging limit. This meant more trips back across the network, made worse by 1).
3) 'id' loops through the groups. With the nested approach, if we processed a parent group in that loop before its children, we'd have a reduction in the number of round-trips to the server (since the nested processing with ASQ is relatively fast). With this match rule enabled, we don't process the nests, so we go to the LDAP server for every group we need to look up. Which because of 2) and 3) may mean a significantly longer time to respond.

===================================
== Baseline (no extensibleMatch) ==
===================================

[sgallagh at testmachine ~]# time id -G kurashima at adtest

100514 101108 101109 104696 104698 104700 104699 101110

real	0m1.722s


[sgallagh at testmachine ~]# time id  kurashima at adtest
uid=101146(kurashima) gid=100514(domain users) groups=100514

(domain users),101108(sssdgroup),101109(topgroup),104696(uppergroup),104698(abovetopgroup),104700(aboveeventhat),104699(aboveabovetop),101110(nestedgroup)

real	0m8.656s


[sgallagh at testmachine ~]# time getent group aboveeventhat at adtest
...
real	0m5.070s

=================================================
== With ldap_groups_use_matching_rule_in_chain ==
=================================================

[sgallagh at testmachine ~]# time id -G kurashima at adtest

100514 101108 101109 104696 104698 104700 104699 101110

real	0m1.730s


[sgallagh at testmachine ~]# time id  kurashima at adtest

uid=101146(kurashima) gid=100514 groups=100514,101108(sssdgroup),101109(topgroup),104696(uppergroup),104698(abovetopgroup),104700(aboveeventhat),104699(aboveabovetop),101110(nestedgroup)

real	0m23.265s

[sgallagh at testmachine ~]# time getent group aboveeventhat at adtest
...
real	0m3.903s


=====================================================
== With ldap_initgroups_use_matching_rule_in_chain ==
=====================================================

[sgallagh at testmachine ~]# time id -G kurashima at adtest
100514 104698 101108 101109 101110 104699 104700 104696

real	0m0.601s


[sgallagh at testmachine ~]# time id  kurashima at adtest
uid=101146(kurashima) gid=100514(domain users) groups=100514(domain users),104698(abovetopgroup),101108(sssdgroup),101109(topgroup),101110(nestedgroup),104699(aboveabovetop),104700(aboveeventhat),104696(uppergroup)

real	0m7.846s

[sgallagh at testmachine ~]# time getent group aboveeventhat at adtest
...
real	0m4.770s


===================================
==      With both                ==
===================================
[sgallagh at testmachine ~]# time id -G kurashima at adtest
100514 104698 101108 101109 101110 104699 104700 104696

real	0m0.611s

[sgallagh at testmachine ~]# time id  kurashima at adtest
uid=101146(kurashima) gid=100514 groups=100514,104698(abovetopgroup),101108(sssdgroup),101109(topgroup),101110(nestedgroup),104699(aboveabovetop),104700(aboveeventhat),104696(uppergroup)

real	0m21.454s

[sgallagh at testmachine ~]# time getent group aboveeventhat at adtest
...
real	0m4.343s

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <https://lists.fedorahosted.org/pipermail/sssd-devel/attachments/20120612/1bf9e15f/attachment.sig>


More information about the sssd-devel mailing list