On (28/07/16 16:37), thierry bordaz wrote:
On 07/28/2016 12:58 PM, Lukas Slebodnik wrote:
> On (28/07/16 12:44), thierry bordaz wrote:
> >
> > On 07/28/2016 11:17 AM, Lukas Slebodnik wrote:
> > > On (28/07/16 10:24), thierry bordaz wrote:
> > > > On 07/28/2016 09:39 AM, Jakub Hrozek wrote:
> > > > > On Wed, Jul 27, 2016 at 04:09:07PM +0200, thierry bordaz wrote:
> > > > > > On 07/27/2016 03:36 PM, Jakub Hrozek wrote:
> > > > > > > On Wed, Jul 27, 2016 at 02:55:37PM +0200, thierry
bordaz wrote:
> > > > > > > > On 07/27/2016 01:56 PM, Jakub Hrozek wrote:
> > > > > > > > > On Wed, Jul 27, 2016 at 01:03:59PM +0200,
Jakub Hrozek wrote:
> > > > > > > > > > On Wed, Jul 27, 2016 at 12:22:46PM
+0200, Lukas Slebodnik wrote:
> > > > > > > > > > > On (27/07/16 12:08), Jakub Hrozek
wrote:
> > > > > > > > > > > > On Wed, Jul 27, 2016 at
12:02:24PM +0200, Jakub Hrozek wrote:
> > > > > > > > > > > > > On Wed, Jul 27, 2016 at
11:54:16AM +0200, Lukas Slebodnik wrote:
> > > > > > > > > > > > > > ehlo,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > attached patch fixes
acces denied after activating user in 389ds.
> > > > > > > > > > > > > > Jakub had some
comments/ideas in ticket but I think it's better to discuss
> > > > > > > > > > > > > > about virtual
attributes and timestamp cache on mailing list.
> > > > > > > > > > > > > Yes, so the comment I
have is that while this works, it might break some
> > > > > > > > > > > > > strange LDAP servers.
> > > > > > > > > > > > >
> > > > > > > > > > > > > We use modifyTimestamp as
a 'positive' indicator that the entry has not
> > > > > > > > > > > > > changed -- if the
modifyTimestamp didn't change, we consider the cached
> > > > > > > > > > > > > entry the same as what is
on the server and only bump the timestamp
> > > > > > > > > > > > > cache. If the timestamp
is different, we do a deep-comparison of cached
> > > > > > > > > > > > > attribute values with
what is on the LDAP server and write the sysdb
> > > > > > > > > > > > > cache entry only if the
attributes differ.
> > > > > > > > > > > > >
> > > > > > > > > > > > > I was wondering if we can
use the modifyTimestamp at all, then, because
> > > > > > > > > > > > > even if it's the
same, we might want to check the attributes to see if
> > > > > > > > > > > > > some of the values are
different because some of the attributes might be
> > > > > > > > > > > > > this operational/virtual
attribute..
> > > > > > > > > > > > Sorry, sent too soon.
> > > > > > > > > > > >
> > > > > > > > > > > > I think the questions are --
1) can we enumerate the virtual attributes?
> > > > > > > > > > > That might be a question for 389-ds
developers.
> > > > > > > > > > > But it's very likely it will be
different on other LDAP servers.
> > > > > > > > > > >
> > > > > > > > > > > > 2) Would different LDAP
servers have different virtual attributes.
> > > > > > > > > > > >
> > > > > > > > > > > > For 2) maybe a possible
solution might be to set a non-existing
> > > > > > > > > > > > modifyTimestamp attribute
value, but I would consider that only a
> > > > > > > > > > > > kludge, we shouldn't break
existing setups..
> > > > > > > > > > > I am not satisfied with this POC
solution either.
> > > > > > > > > > >
> > > > > > > > > > > So should we remove usage of
modifyTimestamp for detecting changes?
> > > > > > > > > > I would prefer to ask the DS developers
before removing it completely.
> > > > > > > > > >
> > > > > > > > > > At least for large groups it might take
a long time to compare all attribute
> > > > > > > > > > values and IIRC we don't depend on
any virtual attributes for groups. Maybe
> > > > > > > > > > we could parametrize that part of the
code and enable the fast way with
> > > > > > > > > > modifyTimestamps for 'known'
server types, that is for setups with AD and
> > > > > > > > > > IPA providers.
> > > > > > > > > >
> > > > > > > > > > For users, there is typically not as
many attributes so we should be
> > > > > > > > > > fine deep-comparing all attributes.
> > > > > > > > > I'm adding Thierry (so please
reply-to-all to keep him in the thread).
> > > > > > > > >
> > > > > > > > > Thierry, in the latest sssd version we tried
to add a performance
> > > > > > > > > improvement related to how we store SSSD
entries in the cache. The short
> > > > > > > > > version is that we store the modifyTimestamp
attribute in the cache and
> > > > > > > > > when we fetch an entry, we compare the entry
modifyTimestamp with what
> > > > > > > > > is on the server. When the two are the same,
we say that the entry did
> > > > > > > > > not change and don't update the cache.
> > > > > > > > >
> > > > > > > > > This works fine for most attributes, but not
for attributes like
> > > > > > > > > nsAccountLock which do not change
modifyTimestamp when they are
> > > > > > > > > modified. So when an entry was already cached
but then nsAccountLock
> > > > > > > > > changed, we treated the entry as the same and
never read the new
> > > > > > > > > nsAccountLock value.
> > > > > > > > >
> > > > > > > > > To fix this, I think we have several
options:
> > > > > > > > > 1) special-case the nsAccountLock.
This seems a bit dangerous,
> > > > > > > > > because I'm not sure we can say
that some other attribute we are
> > > > > > > > > interested in behaves the same as
nsAccountLock.
> > > > > > > > > 2) drop the modifyTimestamp
optimization completely. Then we fall
> > > > > > > > > back to comparing the attribute
values, which might work, but for
> > > > > > > > > huge objects like groups with
thousands of members, this might be
> > > > > > > > > too expensive.
> > > > > > > > > 3) only use the modifyTimestamp
optimization for cases where we know
> > > > > > > > > we don't read any virtual
attributes.
> > > > > > > > >
> > > > > > > > > And my question is -- can we, in general,
know if the modifyTimestamp
> > > > > > > > > way of detecting changes is realiable for all
LDAP servers? Or do you
> > > > > > > > > think it should only be used for cases where
we know we are not
> > > > > > > > > interested in any virtual attributes (that
would mostly be storing
> > > > > > > > > groups from servers where we know exactly
what is on the server side,
> > > > > > > > > like IPA or AD).
> > > > > > > > Hello,
> > > > > > > >
> > > > > > > > Relying on modifytimestamp looks a good idea. Any
MOD/MODRDN will update it,
> > > > > > > > except I think it is unchanged when updating some
password policy
> > > > > > > > attributes.
> > > > > > > >
> > > > > > > > Regarding virtual attribute, the only one I know
in IPA is nsaccountlock.
> > > > > > > > nsaccountlock is an operational attribute (you
need to request it to see it)
> > > > > > > > and is also a virtual attribute BUT only for
'staged' and 'deleted' users.
> > > > > > > > It is a stored attribute for regular users and we
should update
> > > > > > > > modifytimestamp when it is set.
> > > > > > > >
> > > > > > > > thanks
> > > > > > > > thierry
> > > > > > > OK, in that case it seems like we can special-case it.
But do you know
> > > > > > > about any other attributes in any other LDAP servers?
> > > > > > Any LDAP server following standard should provide
modifytimestamp that
> > > > > > reflect the last update of the entry. Now virtual attribute
values may be
> > > > > > "attached" to the entry and its value change
without modification of
> > > > > > modifytimestamp.
> > > > > > For 389-ds and IPA it is fine as virtual value of
nsaccountlock is changed
> > > > > > only when the DN change.
> > > > > > For others LDAP servers I suppose it exists the same ability
to define
> > > > > > service providers that return virtual attribute values. The
difficulty is
> > > > > > that the schema may not give any hint if the retrieved
attributes values
> > > > > > were stored or computed and consequently trust
modifytimestamp to know if
> > > > > > the values changed or not.
> > > > > > For example in ODSEE, memberof is a virtual attribute.
> > > > > Thank you, for the explanation Thierry.
> > > > >
> > > > > Then to be on the safe side I propose:
> > > > > 1) We add an (probably undocumented?) flag that says
whether to use
> > > > > modifyTimestamp to detect entry changes or not
> > > > > 2) for the generic LDAP provider we always really compare
the
> > > > > attribute values, in other words the option would be
set to
> > > > > false. If there is anyone with performance issues with
a generic
> > > > > setup, we tell them to flip the option.
> > > > > 3) For the IPA and AD providers, we set this option to
true and use
> > > > > the modifyTimestamp value to detect changes
> > > > > 4) We special case nsAccountLock
> > > > I am not sure I understand why nsAccountLock is a special case ?
> > > > In IPA/389:
> > > >
> > > > * Stage user, it is always 'nsaccountlock: True'
> > > > When stage entry is created or updated, 'modifytimestamp'
is also
> > > > updated. So you can rely on modifytimestamp to detect a change in
a
> > > > stage user.
> > > > There is no way to update nsaccountlock for a stage entry
> > > > * Deleted user. Idem Stage user
> > > We haven't tested stage users yet. I ran just sssd related regression
tests.
> > >
> > > > * Regular user. 'nsaccountlock' is *not* a virtual attribute,
so if it
> > > > is enable/disable you can rely on modifytimestamp to detect a
change
> > > > of 'nsaccountlock' for a regular user.
> > > > Also any change on regular user will update
'modifytimestamp' so you
> > > > can rely on it to detect a change.
> > > >
> > > My experience with ns-inactivate.pl and ns-activate.pl is different.
> > > modifytimestamp is not changed even though nsaccountlock was changed.
> > >
> > > It was a plain 389-ds with id_provider ldap in sssd
> > >
> > > LS
> > Hi Lukas,
> >
> >
> > It may have change recently because I can not reproduce. What versions are
> > you running ?
> >
> > #
> > #initial entry
> > #
> > ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx
-b
> > "cn=accounts, <suffix>" 'uid=user0' nsaccountlock
modifytimestamp
> > createtimestamp
> > dn: uid=user0,cn=users,cn=accounts,<suffix>
> > modifytimestamp: *20160728103441Z*
> > createtimestamp: 20160726160436Z
> >
> > #
> > # Inactivate: modifytimestamp changed
> > #
> > /usr/sbin/ns-inactivate.pl -Z <instance> -D "cn=directory
manager"
> > -w xxx -I uid=user0,cn=users,cn=accounts,<suffix>
> > uid=user0,cn=users,cn=accounts,<suffix> inactivated.
> >
> > ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx
-b
> > "cn=accounts, <suffix>" 'uid=user0' nsaccountlock
modifytimestamp
> > createtimestamp
> > dn: uid=user0,cn=users,cn=accounts,<suffix>
> > nsaccountlock: true
> > modifytimestamp: *20160728103642Z*
> > createtimestamp: 20160726160436Z
> >
> > #
> > # activate: modifytimestamp changed
> > #
> > /usr/sbin/ns-activate.pl -Z <instance> -D "cn=directory
manager" -w
> > xxx -I uid=user0,cn=users,cn=accounts,<suffix>
> > uid=user0,cn=users,cn=accounts,<suffix> activated.
> >
> > ldapsearch -LLL -o ldif-wrap=no -D "cn=directory manager" -w xxx
-b
> > "cn=accounts, <suffix>" 'uid=user0' nsaccountlock
modifytimestamp
> > createtimestamp
> > dn: uid=user0,cn=users,cn=accounts,<suffix>
> > modifytimestamp: *20160728103711Z*
> > createtimestamp: 20160726160436Z
> >
> rhel7.3
> 389-ds-base-1.3.5.10-5.el7.x86_64
>
> But I didn't test directly with user but indirectly
>
> *Add Managed role
> dn: cn=managed,ou=people,dc=example,dc=com
> objectclass: top
> objectclass: LdapSubEntry
> objectclass: nsRoleDefinition
> objectclass: nsSimpleRoleDefinition
> objectclass: nsManagedRoleDefinition
> cn: Managed
>
> *Authenticate with lockuser
>
> *Add user to managed role
> dn: uid=lockuser,ou=people,dc=example,dc=com
> changetype: modify
> add: nsRoleDN
> nsRoleDN: cn=managed,ou=people,dc=example,dc=com
>
> ns-inactivate.pl -D cn=Manager,dc=example,dc=com" -E -p 389 \
> -h $SERVER -I cn=managed,ou=people,dc=example,dc=com
>
> *Authenticate with lockuser
>
> ns-activate.pl -D cn=Manager,dc=example,dc=com" -E -p 389 \
> -h $SERVER -I cn=managed,ou=people,dc=example,dc=com
>
> *Authenticate with lockuser
>
> Should I test with rhel7.2 or is it an expected behaviour ?
>
> LS
That is correct and this is the expected behavior.
Using ns-inactivate.pl with a role, it inactivates all the entries in that
role adding nsaccountlock virtual attibute.
You are right, update (add of nsaccountlock) of regular user can be done
without update of its modifytimestamp.
Thank you very much for confirmation and for info that plugin
is not used on IPA. So we needn't special case nsaccountlock for IPA.
We had a discussion on sssd devel meeting. And we agreed that we will
do some performace measurements. And if there will be significant
difference then we will check modifytimestamp only with IPA and AD.
and it will be disabled by default with generic LDAP.
LS