[SSSD] [PATCH 1/2] Set default DNS resolution timeout to 15 seconds.

Jakub Hrozek jhrozek at redhat.com
Fri Jul 19 10:15:14 UTC 2013


On Mon, Jul 08, 2013 at 09:43:42PM +0200, Michal Židek wrote:
> On 07/04/2013 07:45 PM, Simo Sorce wrote:
> >On Thu, 2013-07-04 at 16:23 +0200, Michal Židek wrote:
> >>https://fedorahosted.org/sssd/ticket/1966 (SSSD failover doesn't work if
> >>the first DNS server in resolv.conf is unavailable).
> >>
> >>The problem here is, that if first nameserver in resolv.conf is down,
> >>the resolution is too slow and SSSD will not wait for the result of
> >>ares_search and go offline. In my case the resolution was sometimes more
> >>than a minute, because all search domains in resolv.conf were searched
> >>inside ares_search() call using the first (not working) nameserver in
> >>the first place and then with the working nameserver (and before that,
> >>SSSD tried to figure out the domain name from my incorrectly set
> >>hostname, which added more unnecessary DNS lookups).
> >>
> >>To avoid this problem, the option dns_discovery_domain must be set
> >>properly, so that only the correct domain is searched, but even that is
> >>not enough, because the default timeout for dns resolver operation in
> >>sssd is too low. This patch rises the default value to 15 seconds
> >>(instead of 5 seconds).
> >>
> >>Another option might be to lower the amount of time ares waits for a
> >>nameserver to respond (currently it is 5 seconds, that is why 5 second
> >>for the entire dns resolution is not sufficient), but I do not want to
> >>do this.
> >
> >Why not ?
> >
> >5 seconds these days is quite awfully high all in all.
> >I think 1 second should be plenty in most cases, so maybe a default of 2
> >seconds should be considered.
> >
> 
> I also think that 5 seconds is high value, but in general, I have bad
> feeling when lowering not configurable timeouts (theoretically it could
> cause some DNS servers that are reachable now to become unreachable).
> 

Right, but in practice I can't think of a valid network configuration
where the name server would take so long to reply. I mean, if the
network was so slow that just resolving name would take 5 seconds, then
chances are we wouldn't be able to download the entry from LDAP at all.
And if the nameserver is slow, then let's rather skip it and fail over
to another one.

> Another possibility would be to make this value configurable. Then I
> would have no problem lowering the default value. But, should this be
> configurable? We already have one option to configure timeout for the
> entire DNS resolution, adding another timeout for subtaks of this
> operation does not sound nice to me. But I do not have strong opinion
> on this.
> 

Here is what we discussed with Michal today:
    * make the per-nameserver timeout lower a little. Now it's 5
      seconds, which is too long especially considering the failover
      timeout is 5 seconds as well. We could change it to 2 or 3
      seconds.
    * the glibc allows 3 nameservers. The failover request timeut could
      then default to 3*nameserver_timeout to give the resolver a chance
      to iterate over all the configured name servers.

> Btw, once the correct data is returned from DNS servers, we no longer
> require any DNS operation, so it only happens first time the service
> is required (for example the first 'getent' call will be slow, but
> after that the service is fast). This is not so bad IMO.
> 
> >>These patches also change man pages, so probably master only (string
> >>freeze)? Even if this is a really small change.

We accept changes again now that we've released 1.10.1.

But this change would be OK even for a string-frozen release, because it
only changes the number, not translatable strings.

> >>
> >>I was also thinking, would it make sense to write a warning to the logs
> >>if the dns_discovery_domain option is not set? It seem to be important
> >>to set it properly for cases like this one.
> >
> >I wonder, should we use the machine name to automatically set the
> >discovery domain, and only if that fails fall back to the ones defined
> >in resolv.conf ?
> 
> Actually, we do this. Wireshark shows that the first srv lookup was
> constructed from my hostname. But the whole process is unclear to me,
> for example I do not know why we call the resolv_gethostbyname_send()
> inside of resolv_get_domain_send(). It adds another DNS lookups
> for A records (which causes another slow-down in the case of bad
> nameserver searched first). In wireshark I also see searches for A/AAA
> records for names like:
> mzidek.example.com
> mzidek.example.com.searchdom1.com
> mzidek.example.com.searchdom2.com
> mzidek.example.com.searchdom3.com

IIRC we try to canonicalize the host name before chopping of the first
part and using the rest as domain name. 

Would using gethostname() be safe in real world environments?
I know that host names are often set to complete garbage and can't be
relied on..but is it OK to just say "fix your environment" in that case?

Personally I would argue the current canonicalization doesn't add much,
because if the host name is set to complete garbage then the canonicalization
wouldn't magically fix it..

> 
> where mzidek.example.com is my hostname and searchdom1/2/3 are search
> domains specified in resolv.conf. Why do we even try to resolve these
> names? It seems like waste of time to me. If the dns_discovery_domain is
> properly set, these lookups do not appear.
> 
> Thanks
> Michal
> 
> PS: Patches are the same as before.
> 
> >
> >Simo.
> >
> 

> From 1bb9dc2d17ec10c48860578ebec4dbe19cb166e6 Mon Sep 17 00:00:00 2001
> From: Michal Zidek <mzidek at redhat.com>
> Date: Thu, 4 Jul 2013 13:32:07 +0200
> Subject: [PATCH 1/2] Set default DNS resolution timeout to 15 seconds.
> 
> Partially solves ticket:
> https://fedorahosted.org/sssd/ticket/1966
> 
> To avoid the problem mentioned in the ticket above, option
> dns_discovery_domain must be set properly.
> ---
>  src/providers/data_provider_fo.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/src/providers/data_provider_fo.c b/src/providers/data_provider_fo.c
> index 83294b5..3d1de26 100644
> --- a/src/providers/data_provider_fo.c
> +++ b/src/providers/data_provider_fo.c
> @@ -774,8 +774,8 @@ void be_fo_set_port_status(struct be_ctx *ctx,
>  /* Resolver back end interface */
>  static struct dp_option dp_res_default_opts[] = {
>      { "lookup_family_order", DP_OPT_STRING, { "ipv4_first" }, NULL_STRING },
> -    { "dns_resolver_timeout", DP_OPT_NUMBER, { .number = 5 }, NULL_NUMBER },
> -    { "dns_resolver_op_timeout", DP_OPT_NUMBER, { .number = 5 }, NULL_NUMBER },
> +    { "dns_resolver_timeout", DP_OPT_NUMBER, { .number = 15 }, NULL_NUMBER },
> +    { "dns_resolver_op_timeout", DP_OPT_NUMBER, { .number = 15 }, NULL_NUMBER },
>      { "dns_discovery_domain", DP_OPT_STRING, NULL_STRING, NULL_STRING },
>      DP_OPTION_TERMINATOR
>  };
> -- 
> 1.7.11.2
> 

> From 97cf4e830d59ee04c1bd77cb73241d846144c681 Mon Sep 17 00:00:00 2001
> From: Michal Zidek <mzidek at redhat.com>
> Date: Thu, 4 Jul 2013 14:51:52 +0200
> Subject: [PATCH 2/2] MAN: Changed default for dns_resolver_timeout
> 
> Changed default value of dns_resolver_timeout
> from 5 to 15 seconds.
> ---
>  src/man/sssd.conf.5.xml | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/src/man/sssd.conf.5.xml b/src/man/sssd.conf.5.xml
> index 8df2bd9..a8b71ae 100644
> --- a/src/man/sssd.conf.5.xml
> +++ b/src/man/sssd.conf.5.xml
> @@ -1656,7 +1656,7 @@ override_homedir = /home/%u
>                              offline mode.
>                          </para>
>                          <para>
> -                            Default: 5
> +                            Default: 15
>                          </para>
>                      </listitem>
>                  </varlistentry>
> -- 
> 1.7.11.2
> 

> _______________________________________________
> sssd-devel mailing list
> sssd-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/sssd-devel




More information about the sssd-devel mailing list