On 01/06/2015 11:32 PM, Jakub Hrozek wrote:
On Tue, Jan 06, 2015 at 05:06:39PM -0700, Orion Poplawski wrote:
> We're having some trouble with sssd on centos 7 under load on a VPS. 389ds
> ldap server for id/auth. Part may be an issue with the VPS, but I'm trying
> to track down all possible issues.
>
> Also, we realized that we were running in a bit of a bad state - the primary
> ldap server was not available, but the backup was.
>
> Some logs:
>
> General question, is this bad?:
> (Tue Jan 6 23:17:43 2015) [sssd[be[default]]] [sdap_get_users_done]
> (0x0040): Failed to retrieve users
This can be caused by many things, we neer more context..in general, the
LDAP search has failed and SSSD would fall back to cached entries.
Okay, appears to be a simple "unknown user" type of response:
(Thu Jan 8 15:50:47 2015) [sssd[be[default]]] [be_get_account_info] (0x0100):
Got request for [4097][1][name=contracts-grants]
(Thu Jan 8 15:50:47 2015) [sssd[be[default]]] [sdap_get_users_done] (0x0040):
Failed to retrieve users
contracts-grants is not in LDAP.
>
> see that fairly frequently.
>
> Trouble:
> (Tue Jan 6 22:30:31 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan 6 22:30:31 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan 6 22:30:36 2015) [sssd[be[default]]] [resolv_gethostbyname_done]
> (0x0040): querying hosts database failed [5]: Input/output error
> (Tue Jan 6 22:30:36 2015) [sssd[be[default]]] [fo_resolve_service_done]
> (0x0020): Failed to resolve server 'server.com': Timeout while contacting
> DNS servers
> (Tue Jan 6 22:30:36 2015) [sssd[be[default]]] [be_resolve_server_process]
> (0x0080): Couldn't resolve server (
server.com), resolver returned (5)
This seems like the core issue, can you resolve
server.com from outside
SSSD (with dig, maybe) ?
In general, yes. But it seems that some kind of load/network issue caused a
temporary failure. We've since added entries to /etc/hosts to try to prevent
this from happening again.
> (Tue Jan 6 22:30:45 2015) [sssd[be[default]]]
> [sss_ldap_init_sys_connect_done] (0x0020): sdap_async_sys_connect request
> failed.
> (Tue Jan 6 22:30:45 2015) [sssd[be[default]]] [sdap_sys_connect_done]
> (0x0020): sdap_async_connect_call request failed.
> (Tue Jan 6 22:30:45 2015) [sssd[be[default]]] [fo_resolve_service_send]
> (0x0020): No available servers for service 'LDAP'
> (Tue Jan 6 22:30:45 2015) [sssd[be[default]]] [sdap_id_op_connect_done]
> (0x0020): Failed to connect, going offline (5 [Input/output error])
> (Tue Jan 6 22:30:45 2015) [sssd[be[default]]] [be_run_offline_cb] (0x0080):
> Going offline. Running callbacks.
Here SSSD goes offline and the front end would switch to using the
cache.
> (Tue Jan 6 22:33:08 2015) [sssd[be[default]]]
[get_single_value_as_string]
> (0x0080): More than one value found.
> (Tue Jan 6 22:33:08 2015) [sssd[be[default]]]
> [sdap_set_config_options_with_rootdse] (0x0020): get_naming_context failed.
> (Tue Jan 6 22:33:14 2015) [sssd[be[default]]] [get_single_value_as_string]
> (0x0080): More than one value found.
This is weird as well, here SSSD is complaining that defaultNamingContext
attribute of rootDSE contains multiple values. But I don't see SSSD grabbing
the rootDSE anywhere at all..what log level did you use?
Log level 3
You can read the rootDSE manually using:
ldapsearch -x -H
ldap://server.com -s base -b "" defaultNamingContext
# extended LDIF
#
# LDAPv3
# base <> with scope baseObject
# filter: (objectclass=*)
# requesting: defaultNamingContext
#
#
dn:
# search result
search: 2
result: 0 Success
# numResponses: 2
# numEntries: 1
I've now added a defaultnamingcontext.
--
Orion Poplawski
Technical Manager 303-415-9701 x222
NWRA, Boulder/CoRA Office FAX: 303-415-9702
3380 Mitchell Lane orion(a)nwra.com
Boulder, CO 80301
http://www.nwra.com