Hi,

The 'id' command and server login for an AD user is failing in some IPA clients joined to the server recently. For other clients, the 'id' command as well as server login for  the AD user, is working fine. For clients where AD login is working, we are also seeing recently, some amount of slowness. Not sure what is causing these issues.

I can see in the client sssd domain logs, it is able to pull around 20 groups from the ipa master, but then it goes to timeout while getting membership of a group.

(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [sss_domain_get_state] (0x1000): Domain ipa.domain.com is Active
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): Received [20] groups in group list from IPA Server
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [g_ad_user@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group1@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group2@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group3@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group4@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group5@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [g_ad_user@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group6@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group7@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group8@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group9@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group10@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group11@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group12@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group13@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group14@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group15@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ad_group16@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ipa_group1@ipa.domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0400): [ipa_group2@ipa.domain.com].
.......
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_step] (0x0400): Sending request_type: [REQ_FULL_WITH_MEMBERS] for object [ad_group1@domain.com].
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_exop_send] (0x0400): Executing extended operation
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_exop_send] (0x2000): ldap_extended_operation sent, msgid = 6
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [sdap_op_add] (0x2000): New operation 6 timeout 6
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [sdap_process_result] (0x2000): Trace: sh[0x55f5df2f2a10], connected[1], ops[0x55f5df2838f0], ldap[0x55f5df315c90]
(Tue Jan  5 12:56:08 2021) [sssd[be[ipa.domain.com]]] [sdap_process_result] (0x2000): Trace: end of ldap_result list
(Tue Jan  5 12:56:12 2021) [sssd[be[ipa.domain.com]]] [sdap_process_result] (0x2000): Trace: sh[0x55f5df2bdf20], connected[1], ops[(nil)], ldap[0x55f5df2bda90]
(Tue Jan  5 12:56:12 2021) [sssd[be[ipa.domain.com]]] [sdap_process_result] (0x2000): Trace: end of ldap_result list
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [sdap_op_timeout] (0x1000): Issuing timeout for 6
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [sdap_op_destructor] (0x1000): Abandoning operation 6
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_next] (0x0040): s2n exop request failed.
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_list_done] (0x0040): s2n get_fqlist request failed.


I am also seeing timeout while getting the group details from the ipa replica.

(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [fo_set_port_status] (0x0100): Marking port 0 of server 'ipa-replica.ipa.domain.com' as 'working'
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [set_server_common_status] (0x0100): Marking server 'ipa-replica.ipa.domain.com' as 'working'
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [fo_set_port_status] (0x0400): Marking port 0 of duplicate server 'ipa-replica.ipa.domain.com' as 'working'
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [sdap_id_op_connect_done] (0x2000): Old USN: 4739522, New USN: 4855028
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_acct_info_send] (0x0400): Sending request_type: [REQ_FULL_WITH_MEMBERS] for trust user [ad_user] to IPA server
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_exop_send] (0x0400): Executing extended operation
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_exop_send] (0x2000): ldap_extended_operation sent, msgid = 5
(Tue Jan  5 12:56:14 2021) [sssd[be[ipa.domain.com]]] [sdap_op_add] (0x2000): New operation 5 timeout 6
(Tue Jan  5 12:56:20 2021) [sssd[be[ipa.domain.com]]] [sdap_op_timeout] (0x1000): Issuing timeout for 5
(Tue Jan  5 12:56:20 2021) [sssd[be[ipa.domain.com]]] [sdap_op_destructor] (0x1000): Abandoning operation 5
(Tue Jan  5 12:56:20 2021) [sssd[be[ipa.domain.com]]] [ipa_s2n_get_user_done] (0x0040): s2n exop request failed.
(Tue Jan  5 12:56:20 2021) [sssd[be[ipa.domain.com]]] [sdap_id_op_done] (0x0200): communication error on cached connection, moving to next server



Below my ipa master sssd.conf

[domain/ipa.domain.com]
cache_credentials = True-krb5_store_password_if_offline = True
ipa_domain = ipa.domain.com
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname = ipa_master.ipa.fhcrc.org
chpass_provider = ipa
ipa_server = ipa-master.ipa.fhcrc.org
ipa_server_mode = True
ldap_tls_cacert = /etc/ipa/ca.crt
[sssd]
services = sudo, nss, ifp, pam, ssh

domains = ipa.domain.com
[nss]
memcache_timeout = 600
homedir_substring = /home

[pam]

[sudo]

[autofs]

[ssh]

[pac]

[ifp]
allowed_uids = ipaapi, root

[secrets]

[session_recording]


Below my replica sssd.conf

[domain/ipa.domain.com]

cache_credentials = True
krb5_store_password_if_offline = True
ipa_domain = ipa.fhcrc.org
id_provider = ipa
auth_provider = ipa
access_provider = ipa
ipa_hostname = ipa-replica.ipa.domain.com
chpass_provider = ipa
ipa_server = ipa-replica.ipa.domain.com
ldap_tls_cacert = /etc/ipa/ca.crt
ipa_server_mode = True
[sssd]
services = nss, sudo, ifp, pam, ssh

domains = ipa.domain.com
[nss]
homedir_substring = /home

[pam]

[sudo]

[autofs]

[ssh]

[pac]

[ifp]
allowed_uids = ipaapi, root

[secrets]

[session_recording]



Below the client sssd.conf

[domain/ipa.domain.com]
debug_level=8
id_provider = ipa
ipa_server = _srv_, ipa-replica.ipa.domain.com
ipa_domain = ipa.domain.com
ipa_hostname = ipa-client.ipa.domain.com
auth_provider = ipa
chpass_provider = ipa
access_provider = ipa
cache_credentials = True
ldap_tls_cacert = /etc/ipa/ca.crt
krb5_store_password_if_offline = True
[sssd]
debug_level=10
services = nss, pam, ssh, sudo

domains = ipa.domain.com
[nss]
debug_level=10
homedir_substring = /home

[pam]

[sudo]

[autofs]

[ssh]

[pac]

[ifp]

[secrets]

[session_recording]


Any help is appreciated.
TIA
Suchi.

On Mon, Jan 4, 2021 at 11:19 PM Sumit Bose via FreeIPA-users <freeipa-users@lists.fedorahosted.org> wrote:
On Mon, Jan 04, 2021 at 09:48:54AM -0800, Suchismita Panda via FreeIPA-users wrote:
> Hi,
>
> Thanks for the reply.
>
> Yes the replica has been configured with AD Trust Agent. Any other pointer
> would be really helpful.

Hi,

please add more log context, 's2n exop request failed.' might have
different reasons, e.g. timeouts, object wan not found etc.

Does the 'id' for an AD user command fail on all clients? In this case
please check the output of the same 'id' command on the master or
replica if all groups can be resolved. If there is a GID in the output
without a matching group-name you should add a matching group so that
all group can be resolved.

bye,
Sumit

>
> Thanks
> Suchi
>
> On Mon, Jan 4, 2021 at 12:47 AM Florence Blanc-Renaud via FreeIPA-users <
> freeipa-users@lists.fedorahosted.org> wrote:
>
> > On 12/31/20 12:51 AM, Suchismita Panda via FreeIPA-users wrote:
> > > Hi,
> > >
> > > We have a pair of FreeIPA servers (1 master and 1 replica)
> > > Freeipa server version 4.6.8
> > >
> > > Recently when we are trying to enroll any new freeipa client to the
> > > server, the installation goes successful, but AD user login does
> > > not work. Even the client fails to retrieve AD user information using id
> > > command. This works fine on the FreeIPA server.
> > >
> > Hi,
> >
> > Is the IdM replica configured as trust controller / trust agent or not
> > configured with any trust role? If the replica is neither controller not
> > agent, this may explain the behavior that you are seeing. For more
> > information please refer to the "Trust Controllers and Trust Agents"
> > chapter [1].
> >
> > HTH,
> > flo
> >
> > [1]
> >
> > https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/windows_integration_guide/active-directory-trust#trust-controller-agent
> > > Freeipa local user login is working fine on the client.
> > >
> > > There are other FreeIPA clients, where the AD user login is working
> > > fine. We generally use Ansible to join FreeIPA. So the installation
> > > process is also the same for all servers. Not sure why, recently it does
> > > not work. Any advice would be really helpful.
> > >
> > > Freeipa client version 4.8.6
> > >
> > > In the logs mostly I am seeing below error -
> > >
> > > [ipa_s2n_get_user_done] (0x0040): s2n exop request failed.
> > >
> > > Thanks
> > > Suchi
> > >
> > > _______________________________________________
> > > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
> > > To unsubscribe send an email to
> > freeipa-users-leave@lists.fedorahosted.org
> > > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > > List Archives:
> > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
> > >
> > _______________________________________________
> > FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
> > To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
> > Fedora Code of Conduct:
> > https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> > List Archives:
> > https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
> >

> _______________________________________________
> FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
> To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
> Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
> List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org
_______________________________________________
FreeIPA-users mailing list -- freeipa-users@lists.fedorahosted.org
To unsubscribe send an email to freeipa-users-leave@lists.fedorahosted.org
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedorahosted.org/archives/list/freeipa-users@lists.fedorahosted.org