[SSSD-users] sssd troubles under load

Jakub Hrozek jhrozek at redhat.com
Wed Jan 7 06:32:32 UTC 2015


On Tue, Jan 06, 2015 at 05:06:39PM -0700, Orion Poplawski wrote:
> We're having some trouble with sssd on centos 7 under load on a VPS. 389ds
> ldap server for id/auth.  Part may be an issue with the VPS, but I'm trying
> to track down all possible issues.
> 
> Also, we realized that we were running in a bit of a bad state - the primary
> ldap server was not available, but the backup was.
> 
>  Some logs:
> 
> General question, is this bad?:
> (Tue Jan  6 23:17:43 2015) [sssd[be[default]]] [sdap_get_users_done]
> (0x0040): Failed to retrieve users

This can be caused by many things, we neer more context..in general, the
LDAP search has failed and SSSD would fall back to cached entries.

> 
> see that fairly frequently.
> 
> Trouble:
> (Tue Jan  6 22:30:31 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:30:31 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:30:36 2015) [sssd[be[default]]] [resolv_gethostbyname_done]
> (0x0040): querying hosts database failed [5]: Input/output error
> (Tue Jan  6 22:30:36 2015) [sssd[be[default]]] [fo_resolve_service_done]
> (0x0020): Failed to resolve server 'server.com': Timeout while contacting
> DNS servers
> (Tue Jan  6 22:30:36 2015) [sssd[be[default]]] [be_resolve_server_process]
> (0x0080): Couldn't resolve server (server.com), resolver returned (5)

This seems like the core issue, can you resolve server.com from outside
SSSD (with dig, maybe) ?

> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]] [fo_resolve_service_send]
> (0x0020): No available servers for service 'LDAP'
> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]] [sdap_id_op_connect_done]
> (0x0020): Failed to connect, going offline (5 [Input/output error])
> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]] [be_run_offline_cb] (0x0080):
> Going offline. Running callbacks.

Here SSSD goes offline and the front end would switch to using the
cache.

> (Tue Jan  6 22:31:52 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:31:52 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:31:52 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:31:52 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:32:00 2015) [sssd[be[default]]] [fo_resolve_service_send]
> (0x0020): No available servers for service 'LDAP'
> (Tue Jan  6 22:32:00 2015) [sssd[be[default]]] [sdap_id_op_connect_done]
> (0x0020): Failed to connect, going offline (5 [Input/output error])
> (Tue Jan  6 22:32:00 2015) [sssd[be[default]]] [be_run_offline_cb] (0x0080):
> Going offline. Running callbacks.
> (Tue Jan  6 22:33:07 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:33:07 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:33:07 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:33:07 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:33:08 2015) [sssd[be[default]]] [get_single_value_as_string]
> (0x0080): More than one value found.
> (Tue Jan  6 22:33:08 2015) [sssd[be[default]]]
> [sdap_set_config_options_with_rootdse] (0x0020): get_naming_context failed.
> (Tue Jan  6 22:33:14 2015) [sssd[be[default]]] [get_single_value_as_string]
> (0x0080): More than one value found.

This is weird as well, here SSSD is complaining that defaultNamingContext
attribute of rootDSE contains multiple values. But I don't see SSSD grabbing
the rootDSE anywhere at all..what log level did you use?

You can read the rootDSE manually using:
ldapsearch -x -H ldap://server.com -s base -b "" defaultNamingContext

> (Tue Jan  6 22:33:14 2015) [sssd[be[default]]]
> [sdap_set_config_options_with_rootdse] (0x0020): get_naming_context failed.
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]] [be_resolve_server_process]
> (0x0040): The fail over cycled through all available servers
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]] [be_run_offline_cb] (0x0080):
> Going offline. Running callbacks.
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]] [be_resolve_server_process]
> (0x0040): The fail over cycled through all available servers
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]] [be_run_offline_cb] (0x0080):
> Going offline. Running callbacks.
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]] [be_resolve_server_process]
> (0x0040): The fail over cycled through all available servers
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]] [be_run_offline_cb] (0x0080):
> Going offline. Running callbacks.
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:34:06 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:34:16 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:34:16 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan  6 22:34:16 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan  6 22:34:16 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> 
> don't know why it wasn't able to reconnect to the backup, or perhaps it did,
> but just not logged.


More information about the sssd-users mailing list