Alexander,
The only other oddity I'd like to report is that we were seeing
some
strange caching issues on the IPA masters, post upgrade. Despite the
error messages that I reported, users would be added to groups without
a problem. But, over the course of ~48 hours, we were getting reports
that some trusted users weren't able to login (HBAC rule, permission
denied) for random intervals, and I noticed that group memberships
were "lost"; a trusted user with 31 group memberships would randomly
report 27 memberships, while other hosts reported 31. I traced it
back to the IPA servers' SSSD caches. Despite multiple cache clearing
(stop SSSD, rm -rf /var/lib/sss/db/*; start SSSD), the problem would
arise again within an hour. It appears that downgrading SSSD from
1.16.5-10.el7_9.7 to 1.16.5-10.el7_9.6 on the IPA servers has
stabilized the situation.
Does this sound familiar to you, or anyone else on the FreeIPA team?
Well, I found the answer to my own question it seems:
https://github.com/SSSD/sssd/issues/5534
Coffee is a wonderful thing if it's drank before any real attempt to
use your brain!
John DeSantis
Il giorno sab 10 apr 2021 alle ore 09:59 John Desantis
<desantis(a)mail.usf.edu> ha scritto:
>
> Alexander,
>
> > Correct. There was also another use case where netgroups access was
> > emulated with the compat tree. It was used mainly by sudoers LDAP
> > integration. Since SSSD provides own sudoers plugin, I believe recent
> > SSSD should already have support for pulling netgroups from the primary
> > IPA LDAP tree even in RHEL 7. So if you don't have RHEL 6 clients, you
> > should be OK.
>
> Thanks for the extra information.
>
> We do have some older EL6 clients that are online but those don't
> directly utilize the trust except for basic DNS and NTP functions;
> only IDM users can login if need be and there are no sudo rules
> assigned.
>
> > >The only query that I utilize to identify AD users who have
> > >transitioned into an inactive status is by checking the
> > >ipaExternalMember attribute in the external group tree, i.e.
> > >cn=posixgroup_external,cn=groups,cn=accounts,dc=ipa,dc=domain.
> >
> > This is not using compat tree (e.g. your DN is not in
> > cn=compat,dc=ipa,dc=domain).
>
> Perfect! This means we can continue to utilize that method to
> identify AD users who have transitioned into an inactive status.
>
> > >I'm definitely uninitiated, but from the information I've provided
> > >above does it seem that I'm using the compatibility tree?
> >
> > It is easy to check by looking at the LDAP server access log and seeing
> > whether there are any requests to cn=compat,dc=ipa,dc=domain subtree.
>
> Haha! This is exactly what I did at first, but since I wasn't 100%
> certain it was necessary to hear someone else's expertise.
>
The only other oddity I'd like to report is that we were seeing
some
strange caching issues on the IPA masters, post upgrade. Despite the
error messages that I reported, users would be added to groups without
a problem. But, over the course of ~48 hours, we were getting reports
that some trusted users weren't able to login (HBAC rule, permission
denied) for random intervals, and I noticed that group memberships
were "lost"; a trusted user with 31 group memberships would randomly
report 27 memberships, while other hosts reported 31. I traced it
back to the IPA servers' SSSD caches. Despite multiple cache clearing
(stop SSSD, rm -rf /var/lib/sss/db/*; start SSSD), the problem would
arise again within an hour. It appears that downgrading SSSD from
1.16.5-10.el7_9.7 to 1.16.5-10.el7_9.6 on the IPA servers has
stabilized the situation.
Does this sound familiar to you, or anyone else on the FreeIPA team?
>
> Anyways, as long as the stability remains over the weekend I'll
> disable the compatibility plugin since it's not being used.
>
> Thanks again for your replies and insight.
>
> John DeSantis
>
> Il giorno sab 10 apr 2021 alle ore 09:03 Alexander Bokovoy via
> FreeIPA-users <freeipa-users(a)lists.fedorahosted.org> ha scritto:
> >
> > On pe, 09 huhti 2021, John Desantis wrote:
> > >Alexander,
> > >
> > >> Schema compatibility plugin populates them automatically based on the
> > >> content of the primary tree. Presence of the entries does not mean you
> > >> are using them.
> > >
> > >I appreciate this reply!
> > >
> > >I don't think we're using this plugin. We have a fairly
"standard"
> > >FreeIPA deployment with an AD trust. We only use IDM users to manage
> > >resources via the cli and web GUI. We do use FreeRadius to
> > >authenticate against IDM users on our switches, and we use direct LDAP
> > >calls to authenticate IDM users against the IPA master(s). The compat
> > >tree isn't used in the LDAP connection strings.
> > >
> > >As I understand it, the compatibility plugin is primarily used for the
> > >following items:
> > >
> > >1.) Applications that cannot utilize RFC 2307bis
> > >2.) Older SSSD clients (< 1.9) that need to resolve trusted AD users
> >
> > Correct. There was also another use case where netgroups access was
> > emulated with the compat tree. It was used mainly by sudoers LDAP
> > integration. Since SSSD provides own sudoers plugin, I believe recent
> > SSSD should already have support for pulling netgroups from the primary
> > IPA LDAP tree even in RHEL 7. So if you don't have RHEL 6 clients, you
> > should be OK.
> >
> > >During the installation I don't recall specifically enabling the
> > >plugin, and the following logs (ipaserver-install.log and
> > >ipareplica-install.log) seem to confirm:
> > >
> > >ipaserver-install.log:2019-05-08T17:15:17Z DEBUG ipa-server-install
> > >was invoked with arguments ... 'enable_compat': False,
> > >ipareplica-install.log:2019-05-08T17:35:20Z DEBUG ipa-replica-install
> > >was invoked with arguments ... 'enable_compat': False,
> > >
> > >The only query that I utilize to identify AD users who have
> > >transitioned into an inactive status is by checking the
> > >ipaExternalMember attribute in the external group tree, i.e.
> > >cn=posixgroup_external,cn=groups,cn=accounts,dc=ipa,dc=domain.
> >
> > This is not using compat tree (e.g. your DN is not in
> > cn=compat,dc=ipa,dc=domain).
> >
> > >I'm definitely uninitiated, but from the information I've provided
> > >above does it seem that I'm using the compatibility tree?
> >
> > It is easy to check by looking at the LDAP server access log and seeing
> > whether there are any requests to cn=compat,dc=ipa,dc=domain subtree.
> >
> > >> If SSSD cannot recover from offline state. If it does recover, fine.
> > >
> > >Perfect! SSSD does recover and the trusted AD users have been added
> > >to the posix groups without an issue. I was just alarmed initially
> > >since it was an error message I hadn't seen logged before.
> > >
> > >Thanks again!
> > >
> > >John DeSantis
> > >
> > >Il giorno ven 9 apr 2021 alle ore 04:40 Alexander Bokovoy
> > ><abokovoy(a)redhat.com> ha scritto:
> > >>
> > >> On to, 08 huhti 2021, John Desantis via FreeIPA-users wrote:
> > >> >Alexander,
> > >> >
> > >> >Thanks for your reply!
> > >> >
> > >> >> If you are not using cn=compat tree anywhere, it is probably
better to
> > >> >> disable it.
> > >> >
> > >> >We do appear to be using it, so this doesn't seem to be an
option;
> > >> >there were entries listed as
> > >> >cn=groupname,cn=groups,cn=compat,dc=ipa,dc=domain.
> > >>
> > >> Schema compatibility plugin populates them automatically based on the
> > >> content of the primary tree. Presence of the entries does not mean you
> > >> are using them.
> > >>
> > >> >> What helps is to get the same groups looked up locally on IPA
master to
> > >> >> cache them in SSSD cache. Then on restart of LDAP server and
Schema
> > >> >> Compatibility tree requests, SSSD would serve most of those
from cache.
> > >> >>
> > >> >> >Has anyone else run into this issue? Can the messages be
ignored? Are
> > >> >> >there any steps to take to resolve the issue?
> > >> >>
> > >> >> These can be ignored, mostly. It is something we cannot
currently
> > >> >> address well due to chicken/egg problem at startup of LDAP
server.
> > >> >
> > >> >So, these entries did get logged well after dirsrv had started.
> > >> >
> > >> >If they can mostly be ignored, when should they not be ignored?
> > >>
> > >> If SSSD cannot recover from offline state. If it does recover, fine.
> > >>
> > >> --
> > >> / Alexander Bokovoy
> > >> Sr. Principal Software Engineer
> > >> Security / Identity Management Engineering
> > >> Red Hat Limited, Finland
> > >>
> > >
> >
> >
> >
> > --
> > / Alexander Bokovoy
> > Sr. Principal Software Engineer
> > Security / Identity Management Engineering
> > Red Hat Limited, Finland
> > _______________________________________________
> > FreeIPA-users mailing list -- freeipa-users(a)lists.fedorahosted.org
> > To unsubscribe send an email to freeipa-users-leave(a)lists.fedorahosted.org
> > Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedoraho...
> > Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure