On Fri, 10 Sep 2010 11:31:48 +0200
Ralf Haferkamp <rhafer(a)suse.de> wrote:
> On Thursday 09 September 2010 17:48:09 Simo Sorce wrote:
>> On Thu, 9 Sep 2010 16:46:35 +0200
>> It is true you may not see other members unless they log in, but why
>> would that be a problem ? If they do not log in they are not
>> affected by group memberships anyway.
> The user currently logged in might need to see who else is member of
> the groups he is member of.
Ok, I know of no application that needs it, but if users request it its
>>> BTW, I just found out that the behaviour of
>>> getgrouplist()/initgroups() is similar currently. It will only
>>> return the groups that are already present in the cache. That
>>> means in many cases it will return nothing. (Or just the gid you
>>> supplied via the group argument).
>> Initgroups will always call the LDAP server, and will make sure all
>> groups the user is member of are available.
> Ok, there something else seems broken here:
> 1. When initgroups is called for a user that is currently not cached
> no LDAP groups are returned. (I guess that's inline with your idea
> about how sssd should behave)
No, initgroups should always fetch the user's groups, please open a
> 2. When the user is already cached and rfc2307bis+memberof is used on
> the Serverside, nothing is returned upon the first initgroups call,
> but the cache is correctly populated with the groups, a second
> initgroups call then returns the correct results. So I guess there's
> a bug here somewhere. Haven't investigated this deeper yet.
Yes, and is probably connected with the above issue.
> 3. When using rfc2307 or rfc2307bis (without memberof Attributes)
> initgroups() seems completely broken.
Sigh, please open bug ....
> I'll create bug reports after I was able to take a deeper look. (It
> seems some of the issues are already reported. See e.g.
>> In order to save a user you have to look it up. so if you have a
>> group with 50k users in it you will have to do 50k lookups.
>> Multiply this for N groups.
> Users that were already looked up when looking up the first group
> will not be looked up again (they are already cached) when resolving
> the second group. If the groups really have 50k distinct members you
> are of course right that a lot of LDAP lookups will need to be done.
> But I wonder how common such a scenario is and if sssd's default
> configuration should be optimized for such a usecases. I'd guess most
> enviroments that sssd operates in are much smaller. And there you'll
> confuse the users/admins when groups show up incomplete.
SSSD was initially born as IPA's client. In IPA (at least in v1) every
user you create is automatically made member of the ipausers groups.
So that group always is as big as the whole userbase ...
>> If you need enumeration completeness then you should just have
>> enumerations turned on IMO.
>> Now thinking a bit more there is probably a way to avoid the
>> lookups. You can avoid the 50k lookups by actually *not* fetching
>> each member from LDAP but create fake entries that are marked as
>> expired so they are immediately refreshed with the right data if
>> they are ever requested from the user space.
> Right, this how I was thinking about implementing it for rfc2307
> groups, but I haven't come to that yet.
>> But this technique has some pretty sever limits when using
>> rfc2307bis and nested groups. (should work just fine for classic
>> One limit is that you don't know whether a DN is a user or a Group.
>> It is true that we can adopt some heuristics like examining the DN
>> components to get a clue, but they would be pretty much schema
>> dependent and would fail consistently in some scenarios. So you may
>> fail to properly unroll nested groups.
>> The other is that the actual user name may not be contained within
>> the DN. if the DN includes the uid attribute as part of it does,
>> but if the CN is used (like against AD where the samAccoutnName
>> attribute is used as the OS user name) then you do not have the
>> user name you need to present in the group as member. So, again, it
>> would work only in some setups.
> I think it is not worth to go that path for rfc2307bis groups. It
> wrong to rely too much on such specific setups.
Yes, I am not really suggesting we do that, but I thought it was
important to explain limitations in this area.
>> Another concern is about how to properly do caching.
>> When we did the initgroups call I used to fully resolve groups, we
>> removed that code because it would make initgroups unbearably slow
>> as it would end up doing way too many lookups.
> Agreed. For initgroups the groups should not be completely resolved.
> OTOH when dealing with nested groups (in a scenario without
> "memberof") you'd have to look up all group members anyways. (IMO a
> reason why nested groups should be disabled by default when using a
> standard rfc2307bis setup.)
Yes, a good reason. We should probably have a config option to
enable/disable nested groups rolling at this point.
>> So we need to make sure
>> initgroups can create/manipulate groups and know when a group can be
>> considered up to date, and yet at the same time mark it as
>> "incomplete" or something like that for getent group <group>
>> (and vice versa ?)
> Yes. Probably something like this: After initgroups has finished
> successfully for a specific user. The sysdb object of that user
> should be "marked" that the initgroups result for that user in
> completely cached. All groups that needed to be loaded from LDAP
> during that initgroups call need to be marked as incomplete. When
> getgr(gid|nam) is called for such a group the group will be resolved
> and the "imcomplete" mark can be removed.
Right, requires a few modifications but shouldn't be too difficult.
should we put this into a bug/rfe ?
We should. Steve is not in today but I can do it.
Is this the right summary:
1a) Initgroups do not fetch groups from LDAP - bug
1b) When the user is already cached and rfc2307bis+memberof is used on
the Serverside, nothing is returned upon the first initgroups call (bug
above), but the cache is correctly populated with the groups, a second
initgroups call then returns the correct results (task to test related
to the bug above).
2) When using rfc2307 or rfc2307bis (without memberof Attributes)
initgroups() seems completely broken. (this is probably related to
3) Have a config option to enable/disable nested groups rolling with
default enabled. We should document the scenarios when it makes sense to
enable or disable it.
4) Add marking to the objects. "Complete" mark is put on the user object
when all groups he is a member of are fetched. The groups that were
fetched and were not in the cache are marked as "incomplete". When the
group members are enumerated for a group all users for a group should be
fetched and "group" should be marked as "complete", users fetched by
this lookup that were not in the cache are marked as "incomplete". In
the implementation there is probably no difference between the case when
entry does not exist or or has "incomplete" status. In both cases same
action needs to be taken. In both cases we need to fetch an entry. So I
guess we really need just the "complete" mark.
Is this the correct summary of all the issues?