[SSSD-users] sssd troubles under load

Orion Poplawski orion at cora.nwra.com
Thu Jan 8 16:31:16 UTC 2015


On 01/06/2015 11:32 PM, Jakub Hrozek wrote:
> On Tue, Jan 06, 2015 at 05:06:39PM -0700, Orion Poplawski wrote:
>> We're having some trouble with sssd on centos 7 under load on a VPS. 389ds
>> ldap server for id/auth.  Part may be an issue with the VPS, but I'm trying
>> to track down all possible issues.
>>
>> Also, we realized that we were running in a bit of a bad state - the primary
>> ldap server was not available, but the backup was.
>>
>>  Some logs:
>>
>> General question, is this bad?:
>> (Tue Jan  6 23:17:43 2015) [sssd[be[default]]] [sdap_get_users_done]
>> (0x0040): Failed to retrieve users
> 
> This can be caused by many things, we neer more context..in general, the
> LDAP search has failed and SSSD would fall back to cached entries.

Okay, appears to be a simple "unknown user" type of response:

(Thu Jan  8 15:50:47 2015) [sssd[be[default]]] [be_get_account_info] (0x0100):
Got request for [4097][1][name=contracts-grants]
(Thu Jan  8 15:50:47 2015) [sssd[be[default]]] [sdap_get_users_done] (0x0040):
Failed to retrieve users

contracts-grants is not in LDAP.
> 
>>
>> see that fairly frequently.
>>
>> Trouble:
>> (Tue Jan  6 22:30:31 2015) [sssd[be[default]]]
>> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
>> failed.
>> (Tue Jan  6 22:30:31 2015) [sssd[be[default]]] [sdap_sys_connect_done]
>> (0x0020): sdap_async_connect_call request failed.
>> (Tue Jan  6 22:30:36 2015) [sssd[be[default]]] [resolv_gethostbyname_done]
>> (0x0040): querying hosts database failed [5]: Input/output error
>> (Tue Jan  6 22:30:36 2015) [sssd[be[default]]] [fo_resolve_service_done]
>> (0x0020): Failed to resolve server 'server.com': Timeout while contacting
>> DNS servers
>> (Tue Jan  6 22:30:36 2015) [sssd[be[default]]] [be_resolve_server_process]
>> (0x0080): Couldn't resolve server (server.com), resolver returned (5)
> 
> This seems like the core issue, can you resolve server.com from outside
> SSSD (with dig, maybe) ?

In general, yes.  But it seems that some kind of load/network issue caused a
temporary failure.  We've since added entries to /etc/hosts to try to prevent
this from happening again.

> 
>> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]]
>> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
>> failed.
>> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]] [sdap_sys_connect_done]
>> (0x0020): sdap_async_connect_call request failed.
>> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]] [fo_resolve_service_send]
>> (0x0020): No available servers for service 'LDAP'
>> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]] [sdap_id_op_connect_done]
>> (0x0020): Failed to connect, going offline (5 [Input/output error])
>> (Tue Jan  6 22:30:45 2015) [sssd[be[default]]] [be_run_offline_cb] (0x0080):
>> Going offline. Running callbacks.
> 
> Here SSSD goes offline and the front end would switch to using the
> cache.


>> (Tue Jan  6 22:33:08 2015) [sssd[be[default]]] [get_single_value_as_string]
>> (0x0080): More than one value found.
>> (Tue Jan  6 22:33:08 2015) [sssd[be[default]]]
>> [sdap_set_config_options_with_rootdse] (0x0020): get_naming_context failed.
>> (Tue Jan  6 22:33:14 2015) [sssd[be[default]]] [get_single_value_as_string]
>> (0x0080): More than one value found.
> 
> This is weird as well, here SSSD is complaining that defaultNamingContext
> attribute of rootDSE contains multiple values. But I don't see SSSD grabbing
> the rootDSE anywhere at all..what log level did you use?

Log level 3

> 
> You can read the rootDSE manually using:
> ldapsearch -x -H ldap://server.com -s base -b "" defaultNamingContext

# extended LDIF
#
# LDAPv3
# base <> with scope baseObject
# filter: (objectclass=*)
# requesting: defaultNamingContext
#

#
dn:

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

I've now added a defaultnamingcontext.


-- 
Orion Poplawski
Technical Manager                     303-415-9701 x222
NWRA, Boulder/CoRA Office             FAX: 303-415-9702
3380 Mitchell Lane                       orion at nwra.com
Boulder, CO 80301                   http://www.nwra.com


More information about the sssd-users mailing list