[SSSD] [PATCH] Temporarily mark subdomain as inactive instead marking the whole back end offline

Jakub Hrozek jhrozek at redhat.com
Fri Sep 18 13:03:29 UTC 2015


On Fri, Sep 18, 2015 at 11:04:36AM +0200, Jakub Hrozek wrote:
> On Thu, Sep 17, 2015 at 10:41:07PM +0200, Pavel Březina wrote:
> > On 09/04/2015 02:35 PM, Jakub Hrozek wrote:
> > >Hi,
> > >
> > >the attached patches implement https://fedorahosted.org/sssd/ticket/2637
> > >
> > >There are two main use-cases:
> > >     1) If AD DCs are not reachable on the IPA server itself, we should
> > >     avoid going offline completely, at least the IPA domain should be
> > >     still reachable.
> > >     2) If SSSD is connected to a non-root AD DC, we still try to contact
> > >     the forest root because only the forest root normally knows all the
> > >     subdomains. But in many setups, the forest root is not reachable
> > >     due to network restrictions.
> > >
> > >The full design is described here:
> > >https://fedorahosted.org/sssd/wiki/DesignDocs/OneWayTrusts#Subdomainofflinestatuschanges
> > >
> > >There is one area that I'm not sure about myself -- in the
> > >ad_subdomains.c changes, I always set ignore_mark_offline to true for
> > >the root DC. I think it's safe because in this case the root domain is a
> > >subdomain and we don't want a subdomain to allow marking the be as
> > >offline, but I would like to ask that this change is double-checked.
> > 
> > I think it is ok as well.
> > 
> > >
> > >I mostly checked by pausing AD DC VMs in my test setups and making sure
> > >that the backend stays offline after lookup error, a subsequent lookup
> > >is answered as if we were online on the backend side and that the
> > >inactive status is reset. Also, main domain failures still must mark
> > >sssd as offline.
> > 
> > Hi, I'm about to code-ack those patches. But I would like to make more clear
> > about the tri-state in the following function.
> > 
> > >+static void be_mark_subdom_offline(struct sss_domain_info *subdom,
> > >+                                   struct be_ctx *be_ctx)
> > >+{
> > >+    struct timeval tv;
> > >+    struct tevent_timer *timeout = NULL;
> > >+    int reset_status_timeout;
> > >+
> > >+    reset_status_timeout = get_offline_timeout(be_ctx);
> > >+    tv = tevent_timeval_current_ofs(reset_status_timeout, 0);
> > >+
> > >+    switch (subdom->state) {
> > >+    case DOM_DISABLED:
> > >+        DEBUG(SSSDBG_MINOR_FAILURE, "Won't touch disabled subdomain\n");
> > >+        return;
> > >+    case DOM_INACTIVE:
> > >+        DEBUG(SSSDBG_TRACE_ALL, "Subdomain already disabled\n");
> > 
> > Subdomain is inactive not disabled.
> > 
> > >+        return;
> > >+    case DOM_ENABLED:
> > >+        DEBUG(SSSDBG_TRACE_LIBS, "Disabling subdomain %s\n", subdom->name);
> > 
> > You are marking it as inactive not as disabled.
> > 
> > >+        break;
> > >+    }
> > >+
> > >+    timeout = tevent_add_timer(be_ctx->ev, be_ctx, tv,
> > >+                               be_subdom_reset_status, subdom);
> > >+    if (timeout == NULL) {
> > >+        DEBUG(SSSDBG_OP_FAILURE, "Cannot create timer\n");
> > >+        return;
> > >+    }
> > >+
> > >+    subdom->state = DOM_INACTIVE;
> >                        ^^
> > 
> > >+}
> > 
> > So are we marking it as disabled, inactive or offline?
> > 
> > Have you considered to rename DOM_INACTIVE -> DOM_OFFLINE maybe? I think it
> > more complies to the situation.
> > 
> > Also DOM_ENABLED should be DOM_ACTIVE (according to design document and it
> > makes more sense) or DOM_ONLINE (if you decide to rename DOM_INACTIVE to
> > DOM_OFFLINE).
> > 
> > I will test the patches tomorrow.
> 
> The only reason I didn't name the constants as online/offline is that I
> still think this is a bit of a hack and we should have a proper failover
> support in the future.
> 
> But it's also only an internal change, so I'm fine changing the patches
> as well. (I can do that in-tree and then resubmit if there are more
> changes required).

I changed the states to be active, disabled and inactive and reworded
the debug messages.

No logic changes were done in the patchset.
-------------- next part --------------
>From d0ace3f56c14032676934d880661cee1965301d9 Mon Sep 17 00:00:00 2001
From: Jakub Hrozek <jhrozek at redhat.com>
Date: Tue, 18 Aug 2015 15:15:44 +0000
Subject: [PATCH 1/8] UTIL: Convert domain->disabled into tri-state with domain
 states

Required for:
https://fedorahosted.org/sssd/ticket/2637

This is a first step towards making it possible for domain to be around,
but not contacted by Data Provider.

Also explicitly create domains as active, previously we only relied on
talloc_zero marking dom->disabled as false.
---
 src/confdb/confdb.c                      |  2 ++
 src/confdb/confdb.h                      | 19 ++++++++++++++++++-
 src/db/sysdb_subdomains.c                |  7 +++++--
 src/providers/ad/ad_subdomains.c         |  2 +-
 src/providers/ipa/ipa_subdomains.c       |  2 +-
 src/responder/common/responder_common.c  |  5 +++--
 src/tests/cmocka/test_sysdb_subdomains.c |  6 +++++-
 src/tests/cmocka/test_utils.c            |  6 +++---
 src/util/domain_info_utils.c             | 20 +++++++++++++++++---
 src/util/util.h                          |  3 +++
 src/util/util_errors.c                   |  1 +
 src/util/util_errors.h                   |  1 +
 12 files changed, 60 insertions(+), 14 deletions(-)

diff --git a/src/confdb/confdb.c b/src/confdb/confdb.c
index 3a8a1c01b92e62302ac4f787ccd085be9d8f05c3..c097aad7745eda4fff051c7da027776f95db0f03 100644
--- a/src/confdb/confdb.c
+++ b/src/confdb/confdb.c
@@ -1342,6 +1342,8 @@ static int confdb_get_domain_internal(struct confdb_ctx *cdb,
     domain->has_views = false;
     domain->view_name = NULL;
 
+    domain->state = DOM_ACTIVE;
+
     *_domain = domain;
     ret = EOK;
 done:
diff --git a/src/confdb/confdb.h b/src/confdb/confdb.h
index 427c309a290dc9b02270311dacfcf308bd78430c..eca3d4475541b9c18917413f3491f4f93c553e2c 100644
--- a/src/confdb/confdb.h
+++ b/src/confdb/confdb.h
@@ -216,6 +216,23 @@
 struct confdb_ctx;
 struct config_file_ctx;
 
+/** sssd domain state */
+enum sss_domain_state {
+    /** Domain is usable by both responders and providers. This
+     * is the default state after creating a new domain
+     */
+    DOM_ACTIVE,
+    /** Domain was removed, should not be used be neither responders
+     * not providers.
+     */
+    DOM_DISABLED,
+    /** Domain cannot be contacted. Providers return an offline error code
+     * when receiving request for inactive domain, but responders should
+     * return cached data
+     */
+    DOM_INACTIVE,
+};
+
 /**
  * Data structure storing all of the basic features
  * of a domain.
@@ -278,7 +295,7 @@ struct sss_domain_info {
     struct sss_domain_info *prev;
     struct sss_domain_info *next;
 
-    bool disabled;
+    enum sss_domain_state state;
     char **sd_inherit;
 
     /* Do not use the forest pointer directly in new code, but rather the
diff --git a/src/db/sysdb_subdomains.c b/src/db/sysdb_subdomains.c
index 142520c1836d74ef7bc5c5269487b8971f261b88..546dc1c8d7e5e30ce9e0b56b097894d24d8c94a7 100644
--- a/src/db/sysdb_subdomains.c
+++ b/src/db/sysdb_subdomains.c
@@ -111,6 +111,8 @@ struct sss_domain_info *new_subdomain(TALLOC_CTX *mem_ctx,
     dom->enumerate = enumerate;
     dom->fqnames = true;
     dom->mpg = mpg;
+    dom->state = DOM_ACTIVE;
+
     /* If the parent domain filters out group members, the subdomain should
      * as well if configured */
     inherit_option = string_in_list(CONFDB_DOMAIN_IGNORE_GROUP_MEMBERS,
@@ -268,7 +270,7 @@ errno_t sysdb_update_subdomains(struct sss_domain_info *domain)
     /* disable all domains,
      * let the search result refresh any that are still valid */
     for (dom = domain->subdomains; dom; dom = get_next_domain(dom, false)) {
-        dom->disabled = true;
+        sss_domain_set_state(dom, DOM_DISABLED);
     }
 
     if (res->count == 0) {
@@ -312,7 +314,8 @@ errno_t sysdb_update_subdomains(struct sss_domain_info *domain)
         /* explicitly use dom->next as we need to check 'disabled' domains */
         for (dom = domain->subdomains; dom; dom = dom->next) {
             if (strcasecmp(dom->name, name) == 0) {
-                dom->disabled = false;
+                sss_domain_set_state(dom, DOM_ACTIVE);
+
                 /* in theory these may change, but it should never happen */
                 if (strcasecmp(dom->realm, realm) != 0) {
                     DEBUG(SSSDBG_TRACE_INTERNAL,
diff --git a/src/providers/ad/ad_subdomains.c b/src/providers/ad/ad_subdomains.c
index 9b42f03a0067ab5844432a0f19dd2930dcc200c9..d1d468043410c80e6bf7f0f48a13bd9e962552af 100644
--- a/src/providers/ad/ad_subdomains.c
+++ b/src/providers/ad/ad_subdomains.c
@@ -376,7 +376,7 @@ static errno_t ad_subdomains_refresh(struct ad_subdomains_ctx *ctx,
 
         if (c >= count) {
             /* ok this subdomain does not exist anymore, let's clean up */
-            dom->disabled = true;
+            sss_domain_set_state(dom, DOM_DISABLED);
             ret = sysdb_subdomain_delete(dom->sysdb, dom->name);
             if (ret != EOK) {
                 goto done;
diff --git a/src/providers/ipa/ipa_subdomains.c b/src/providers/ipa/ipa_subdomains.c
index b2e2fec353f7b168d28a880cb0f1b6181abb1ccb..089736b47d8f384a8024682dd203d324292df9ce 100644
--- a/src/providers/ipa/ipa_subdomains.c
+++ b/src/providers/ipa/ipa_subdomains.c
@@ -528,7 +528,7 @@ static errno_t ipa_subdomains_refresh(struct ipa_subdomains_ctx *ctx,
 
         if (c >= count) {
             /* ok this subdomain does not exist anymore, let's clean up */
-            dom->disabled = true;
+            sss_domain_set_state(dom, DOM_DISABLED);
             ret = sysdb_subdomain_delete(dom->sysdb, dom->name);
             if (ret != EOK) {
                 goto done;
diff --git a/src/responder/common/responder_common.c b/src/responder/common/responder_common.c
index 36e7f15948632e9c637886dee259b494e46ceecb..2097004cb0fc24d8b356f9d924243f948227ef58 100644
--- a/src/responder/common/responder_common.c
+++ b/src/responder/common/responder_common.c
@@ -923,7 +923,7 @@ responder_get_domain(struct resp_ctx *rctx, const char *name)
     struct sss_domain_info *ret_dom = NULL;
 
     for (dom = rctx->domains; dom; dom = get_next_domain(dom, true)) {
-        if (dom->disabled) {
+        if (sss_domain_get_state(dom) == DOM_DISABLED) {
             continue;
         }
 
@@ -958,7 +958,8 @@ errno_t responder_get_domain_by_id(struct resp_ctx *rctx, const char *id,
     id_len = strlen(id);
 
     for (dom = rctx->domains; dom; dom = get_next_domain(dom, true)) {
-        if (dom->disabled || dom->domain_id == NULL) {
+        if (sss_domain_get_state(dom) == DOM_DISABLED ||
+                dom->domain_id == NULL) {
             continue;
         }
 
diff --git a/src/tests/cmocka/test_sysdb_subdomains.c b/src/tests/cmocka/test_sysdb_subdomains.c
index 15dcf709fe1231beb5a55fd128b994dae2a14f46..0fb1d6eed4efa3bc05f97f5f169ada10d242ea4c 100644
--- a/src/tests/cmocka/test_sysdb_subdomains.c
+++ b/src/tests/cmocka/test_sysdb_subdomains.c
@@ -151,7 +151,11 @@ static void test_sysdb_subdomain_create(void **state)
     ret = sysdb_update_subdomains(test_ctx->tctx->dom);
     assert_int_equal(ret, EOK);
 
-    assert_true(test_ctx->tctx->dom->subdomains->disabled);
+    assert_int_equal(sss_domain_get_state(test_ctx->tctx->dom->subdomains),
+                     DOM_DISABLED);
+    assert_int_equal(
+            sss_domain_get_state(test_ctx->tctx->dom->subdomains->next),
+            DOM_DISABLED);
 }
 
 static void test_sysdb_master_domain_ops(void **state)
diff --git a/src/tests/cmocka/test_utils.c b/src/tests/cmocka/test_utils.c
index 2e413c87d069549ddd0ca0ee5a8dec0b663f481f..a22c39c4709f5b33f02b26ece791d82c1c8b2d64 100644
--- a/src/tests/cmocka/test_utils.c
+++ b/src/tests/cmocka/test_utils.c
@@ -259,7 +259,7 @@ void test_find_domain_by_name_disabled(void **state)
         dom = dom->next;
     }
     assert_non_null(dom);
-    dom->disabled = true;
+    sss_domain_set_state(dom, DOM_DISABLED);
 
     for (c = 0; c < test_ctx->dom_count; c++) {
         name = talloc_asprintf(global_talloc_context, DOMNAME_TMPL, c);
@@ -426,7 +426,7 @@ void test_find_domain_by_sid_disabled(void **state)
         dom = dom->next;
     }
     assert_non_null(dom);
-    dom->disabled = true;
+    sss_domain_set_state(dom, DOM_DISABLED);
 
     for (c = 0; c < test_ctx->dom_count; c++) {
         name = talloc_asprintf(global_talloc_context, DOMNAME_TMPL, c);
@@ -578,7 +578,7 @@ static void test_get_next_domain_disabled(void **state)
     struct sss_domain_info *dom = NULL;
 
     for (dom = test_ctx->dom_list; dom; dom = get_next_domain(dom, true)) {
-        dom->disabled = true;
+        sss_domain_set_state(dom, DOM_DISABLED);
     }
 
     dom = get_next_domain(test_ctx->dom_list, true);
diff --git a/src/util/domain_info_utils.c b/src/util/domain_info_utils.c
index 4eabcff7a0e0af342ec3833d24da26ede0cb5148..ffbb9475b27a45c07e2e0936464c6e68ed682052 100644
--- a/src/util/domain_info_utils.c
+++ b/src/util/domain_info_utils.c
@@ -50,7 +50,10 @@ struct sss_domain_info *get_next_domain(struct sss_domain_info *domain,
         } else {
             dom = NULL;
         }
-        if (dom && !dom->disabled) break;
+
+        if (dom && sss_domain_get_state(dom) != DOM_DISABLED) {
+            break;
+        }
     }
 
     return dom;
@@ -91,7 +94,7 @@ struct sss_domain_info *find_domain_by_name(struct sss_domain_info *domain,
         return NULL;
     }
 
-    while (dom && dom->disabled) {
+    while (dom && sss_domain_get_state(dom) == DOM_DISABLED) {
         dom = get_next_domain(dom, true);
     }
     while (dom) {
@@ -119,7 +122,7 @@ struct sss_domain_info *find_domain_by_sid(struct sss_domain_info *domain,
 
     sid_len = strlen(sid);
 
-    while (dom && dom->disabled) {
+    while (dom && sss_domain_get_state(dom) == DOM_DISABLED) {
         dom = get_next_domain(dom, true);
     }
 
@@ -730,3 +733,14 @@ done:
 
     return ret;
 }
+
+enum sss_domain_state sss_domain_get_state(struct sss_domain_info *dom)
+{
+    return dom->state;
+}
+
+void sss_domain_set_state(struct sss_domain_info *dom,
+                          enum sss_domain_state state)
+{
+    dom->state = state;
+}
diff --git a/src/util/util.h b/src/util/util.h
index 3e29e748768fc2cac547d16d61b449f1b555a56f..f9fe1ca7189c6b2cdcb29f143005b20a2d969fee 100644
--- a/src/util/util.h
+++ b/src/util/util.h
@@ -566,6 +566,9 @@ struct sss_domain_info *find_domain_by_name(struct sss_domain_info *domain,
                                             bool match_any);
 struct sss_domain_info *find_domain_by_sid(struct sss_domain_info *domain,
                                            const char *sid);
+enum sss_domain_state sss_domain_get_state(struct sss_domain_info *dom);
+void sss_domain_set_state(struct sss_domain_info *dom,
+                          enum sss_domain_state state);
 
 struct sss_domain_info*
 sss_get_domain_by_sid_ldap_fallback(struct sss_domain_info *domain,
diff --git a/src/util/util_errors.c b/src/util/util_errors.c
index fd6b9fbfe32eb8bb456e42cacacfebe439091754..ed19346d9b588a711367af4c891b1298cd4f067e 100644
--- a/src/util/util_errors.c
+++ b/src/util/util_errors.c
@@ -81,6 +81,7 @@ struct err_string error_to_str[] = {
     { "p11_child failed" }, /* ERR_P11_CHILD */
     { "Address family not supported" }, /* ERR_ADDR_FAMILY_NOT_SUPPORTED */
     { "Message sender is the bus" }, /* ERR_SBUS_SENDER_BUS */
+    { "Subdomain is inactive" }, /* ERR_SUBDOM_INACTIVE */
     { "ERR_LAST" } /* ERR_LAST */
 };
 
diff --git a/src/util/util_errors.h b/src/util/util_errors.h
index bda0c9b7d330d8e7a15bc460fc8ed97c80510ea8..c1d081912a382d645c27809a3ac336ff90047cdf 100644
--- a/src/util/util_errors.h
+++ b/src/util/util_errors.h
@@ -103,6 +103,7 @@ enum sssd_errors {
     ERR_P11_CHILD,
     ERR_ADDR_FAMILY_NOT_SUPPORTED,
     ERR_SBUS_SENDER_BUS,
+    ERR_SUBDOM_INACTIVE,
     ERR_LAST            /* ALWAYS LAST */
 };
 
-- 
2.4.3

-------------- next part --------------
>From 8e108816cfd4bf83166415590f9b9a997d56bf5e Mon Sep 17 00:00:00 2001
From: Jakub Hrozek <jhrozek at redhat.com>
Date: Fri, 4 Sep 2015 09:27:17 +0200
Subject: [PATCH 2/8] DP: Provide a way to mark subdomain as disabled and
 auto-enable it later with offline_timeout

https://fedorahosted.org/sssd/ticket/2637

Adds a new Data Provider function be_mark_dom_offline() that is a
replacement for be_mark_offline(). When called, the function would
either set the whole back end offline, just like be_mark_offline or just
set the subdomain status to inactive.

When a subdomain is inactive, there is a singleton timed task that would
re-set the subdomin after offline_timeout seconds.
---
 Makefile.am                              |  26 +++
 src/providers/data_provider_be.c         | 102 +++++++++++-
 src/providers/dp_backend.h               |   1 +
 src/tests/cmocka/test_data_provider_be.c | 275 +++++++++++++++++++++++++++++++
 4 files changed, 395 insertions(+), 9 deletions(-)
 create mode 100644 src/tests/cmocka/test_data_provider_be.c

diff --git a/Makefile.am b/Makefile.am
index 3eaf578a8c7c3b1e13a9516f96886c7a8f18c4d3..18e9867d21cd88017e78757267cd77e31f994915 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -237,6 +237,7 @@ if HAVE_CMOCKA
         test_ipa_subdom_server \
         test_krb5_wait_queue \
         test_cert_utils \
+        test_data_provider_be \
         $(NULL)
 
 if HAVE_LIBRESOLV
@@ -2573,6 +2574,31 @@ test_cert_utils_LDADD = \
     libsss_cert.la \
     libsss_crypt.la \
     $(NULL)
+
+test_data_provider_be_SOURCES = \
+    $(sssd_be_SOURCES) \
+    src/tests/cmocka/test_data_provider_be.c \
+    src/tests/cmocka/common_mock_be.c \
+    $(NULL)
+test_data_provider_be_CFLAGS = \
+    $(AM_CFLAGS) \
+    -DUNIT_TESTING \
+    $(CRYPTO_CFLAGS) \
+    $(NULL)
+test_data_provider_be_LDFLAGS = \
+    -Wl,-wrap,_tevent_add_timer \
+    $(NULL)
+test_data_provider_be_LDADD = \
+    $(CMOCKA_LIBS) \
+    $(CARES_LIBS) \
+    $(POPT_LIBS) \
+    $(PAM_LIBS) \
+    $(TALLOC_LIBS) \
+    $(SSSD_INTERNAL_LTLIBS) \
+    libsss_debug.la \
+    libsss_test_common.la \
+    $(NULL)
+
 endif # HAVE_CMOCKA
 
 noinst_PROGRAMS = pam_test_client
diff --git a/src/providers/data_provider_be.c b/src/providers/data_provider_be.c
index d71a69cb8e2997975828236998ec0b0e3f353f07..effa185f905097e0be1e03596b5fc6da75a7e382 100644
--- a/src/providers/data_provider_be.c
+++ b/src/providers/data_provider_be.c
@@ -478,6 +478,24 @@ try_to_go_online(TALLOC_CTX *mem_ctx,
     return EOK;
 }
 
+static int get_offline_timeout(struct be_ctx *ctx)
+{
+    errno_t ret;
+    int offline_timeout;
+
+    ret = confdb_get_int(ctx->cdb, ctx->conf_path,
+                         CONFDB_DOMAIN_OFFLINE_TIMEOUT, 60,
+                         &offline_timeout);
+    if (ret != EOK) {
+        DEBUG(SSSDBG_CRIT_FAILURE,
+              "Failed to get offline_timeout from confdb. "
+              "Will use 60 seconds.\n");
+        offline_timeout = 60;
+    }
+
+    return offline_timeout;
+}
+
 void be_mark_offline(struct be_ctx *ctx)
 {
     int offline_timeout;
@@ -493,15 +511,9 @@ void be_mark_offline(struct be_ctx *ctx)
         /* This is the first time we go offline - create a periodic task
          * to check if we can switch to online. */
         DEBUG(SSSDBG_TRACE_INTERNAL, "Initialize check_if_online_ptask.\n");
-        ret = confdb_get_int(ctx->cdb, ctx->conf_path,
-                             CONFDB_DOMAIN_OFFLINE_TIMEOUT, 60,
-                             &offline_timeout);
-        if (ret != EOK) {
-            DEBUG(SSSDBG_CRIT_FAILURE,
-                  "Failed to get offline_timeout from confdb. "
-                  "Will use 60 seconds.\n");
-            offline_timeout = 60;
-        }
+
+        offline_timeout = get_offline_timeout(ctx);
+
         ret = be_ptask_create_sync(ctx, ctx,
                                    offline_timeout, offline_timeout,
                                    offline_timeout, 30, offline_timeout,
@@ -524,10 +536,82 @@ void be_mark_offline(struct be_ctx *ctx)
     be_run_offline_cb(ctx);
 }
 
+static void be_subdom_reset_status(struct tevent_context *ev,
+                                  struct tevent_timer *te,
+                                  struct timeval current_time,
+                                  void *pvt)
+{
+    struct sss_domain_info *subdom = talloc_get_type(pvt,
+                                                     struct sss_domain_info);
+
+    DEBUG(SSSDBG_TRACE_LIBS, "Resetting subdomain %s\n", subdom->name);
+    subdom->state = DOM_ACTIVE;
+}
+
+static void be_mark_subdom_offline(struct sss_domain_info *subdom,
+                                   struct be_ctx *be_ctx)
+{
+    struct timeval tv;
+    struct tevent_timer *timeout = NULL;
+    int reset_status_timeout;
+
+    reset_status_timeout = get_offline_timeout(be_ctx);
+    tv = tevent_timeval_current_ofs(reset_status_timeout, 0);
+
+    switch (subdom->state) {
+    case DOM_DISABLED:
+        DEBUG(SSSDBG_MINOR_FAILURE, "Won't touch disabled subdomain\n");
+        return;
+    case DOM_INACTIVE:
+        DEBUG(SSSDBG_TRACE_ALL, "Subdomain already inactive\n");
+        return;
+    case DOM_ACTIVE:
+        DEBUG(SSSDBG_TRACE_LIBS,
+              "Marking subdomain %s as inactive\n", subdom->name);
+        break;
+    }
+
+    timeout = tevent_add_timer(be_ctx->ev, be_ctx, tv,
+                               be_subdom_reset_status, subdom);
+    if (timeout == NULL) {
+        DEBUG(SSSDBG_OP_FAILURE, "Cannot create timer\n");
+        return;
+    }
+
+    subdom->state = DOM_INACTIVE;
+}
+
+void be_mark_dom_offline(struct sss_domain_info *dom, struct be_ctx *ctx)
+{
+    if (IS_SUBDOMAIN(dom) == false) {
+        DEBUG(SSSDBG_TRACE_LIBS, "Marking back end offline\n");
+        be_mark_offline(ctx);
+    } else {
+        DEBUG(SSSDBG_TRACE_LIBS, "Marking subdomain %s offline\n", dom->name);
+        be_mark_subdom_offline(dom, ctx);
+    }
+}
+
+static void reactivate_subdoms(struct sss_domain_info *head)
+{
+    struct sss_domain_info *dom;
+
+    DEBUG(SSSDBG_TRACE_LIBS, "Resetting all subdomains");
+
+    for (dom = head; dom; dom = get_next_domain(dom, true)) {
+        if (sss_domain_get_state(dom) == DOM_INACTIVE) {
+            sss_domain_set_state(dom, DOM_ACTIVE);
+        }
+    }
+}
+
 static void be_reset_offline(struct be_ctx *ctx)
 {
     ctx->offstat.went_offline = 0;
     ctx->offstat.offline = false;
+
+    reactivate_subdoms(ctx->domain);
+
     be_ptask_disable(ctx->check_if_online_ptask);
     be_run_online_cb(ctx);
 }
diff --git a/src/providers/dp_backend.h b/src/providers/dp_backend.h
index bca0c2f97691a96086555482ce181dbdf684466b..4bffcee9e8739302343b7813d3540eb43e966581 100644
--- a/src/providers/dp_backend.h
+++ b/src/providers/dp_backend.h
@@ -189,6 +189,7 @@ struct be_host_req {
 
 bool be_is_offline(struct be_ctx *ctx);
 void be_mark_offline(struct be_ctx *ctx);
+void be_mark_dom_offline(struct sss_domain_info *dom, struct be_ctx *ctx);
 
 int be_add_reconnect_cb(TALLOC_CTX *mem_ctx,
                         struct be_ctx *ctx,
diff --git a/src/tests/cmocka/test_data_provider_be.c b/src/tests/cmocka/test_data_provider_be.c
new file mode 100644
index 0000000000000000000000000000000000000000..68eb5841bee568bef64cb62461403036b1320bec
--- /dev/null
+++ b/src/tests/cmocka/test_data_provider_be.c
@@ -0,0 +1,275 @@
+/*
+    Copyright (C) 2015 Red Hat
+
+    This program is free software; you can redistribute it and/or modify
+    it under the terms of the GNU General Public License as published by
+    the Free Software Foundation; either version 3 of the License, or
+    (at your option) any later version.
+
+    This program is distributed in the hope that it will be useful,
+    but WITHOUT ANY WARRANTY; without even the implied warranty of
+    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+    GNU General Public License for more details.
+
+    You should have received a copy of the GNU General Public License
+    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include <talloc.h>
+#include <tevent.h>
+#include <errno.h>
+#include <popt.h>
+#include <time.h>
+
+#include "providers/dp_backend.h"
+#include "tests/cmocka/common_mock.h"
+#include "tests/cmocka/common_mock_be.h"
+#include "tests/common.h"
+
+#define TESTS_PATH "tests_dp_be"
+#define TEST_CONF_DB "test_dp_be_conf.ldb"
+#define TEST_DOM_NAME "dp_be_test"
+#define TEST_ID_PROVIDER "ldap"
+
+#define OFFLINE_TIMEOUT 2
+#define AS_STR(param) (#param)
+
+static TALLOC_CTX *global_mock_context = NULL;
+static bool global_timer_added;
+
+struct tevent_timer *__real__tevent_add_timer(struct tevent_context *ev,
+                                              TALLOC_CTX *mem_ctx,
+                                              struct timeval next_event,
+                                              tevent_timer_handler_t handler,
+                                              void *private_data,
+                                              const char *handler_name,
+                                              const char *location);
+
+struct tevent_timer *__wrap__tevent_add_timer(struct tevent_context *ev,
+                                              TALLOC_CTX *mem_ctx,
+                                              struct timeval next_event,
+                                              tevent_timer_handler_t handler,
+                                              void *private_data,
+                                              const char *handler_name,
+                                              const char *location)
+{
+    global_timer_added = true;
+
+    return __real__tevent_add_timer(ev, mem_ctx, next_event,
+                                    handler, private_data, handler_name,
+                                    location);
+}
+
+
+struct test_ctx {
+    struct sss_test_ctx *tctx;
+    struct be_ctx *be_ctx;
+};
+
+static struct sss_domain_info *named_domain(TALLOC_CTX *mem_ctx,
+                                            const char *name,
+                                            struct sss_domain_info *parent)
+{
+    struct sss_domain_info *dom = NULL;
+
+    dom = talloc_zero(mem_ctx, struct sss_domain_info);
+    assert_non_null(dom);
+
+    dom->name = talloc_strdup(dom, name);
+    assert_non_null(dom->name);
+
+    dom->parent = parent;
+
+    return dom;
+}
+
+static int test_setup(void **state)
+{
+    struct test_ctx *test_ctx = NULL;
+    struct sss_test_conf_param params[] = {
+        { "offline_timeout", AS_STR(OFFLINE_TIMEOUT) },
+        { NULL, NULL },             /* Sentinel */
+    };
+
+    assert_true(leak_check_setup());
+    global_mock_context = talloc_new(global_talloc_context);
+    assert_non_null(global_mock_context);
+
+    test_ctx = talloc_zero(global_talloc_context, struct test_ctx);
+    assert_non_null(test_ctx);
+
+    test_ctx->tctx = create_dom_test_ctx(test_ctx, TESTS_PATH,
+                                         TEST_CONF_DB, TEST_DOM_NAME,
+                                         TEST_ID_PROVIDER, params);
+    assert_non_null(test_ctx->tctx);
+
+    test_ctx->be_ctx = mock_be_ctx(test_ctx, test_ctx->tctx);
+    assert_non_null(test_ctx->be_ctx);
+
+    test_ctx->be_ctx->domain->subdomains = named_domain(test_ctx,
+                                                        "subdomains",
+                                                        test_ctx->be_ctx->domain);
+    assert_non_null(test_ctx->be_ctx->domain->subdomains);
+
+    *state = test_ctx;
+
+    return 0;
+}
+
+static int test_teardown(void **state)
+{
+    talloc_zfree(*state);
+    assert_true(leak_check_teardown());
+    return 0;
+}
+
+static void assert_domain_state(struct sss_domain_info *dom,
+                                enum sss_domain_state expected_state)
+{
+    enum sss_domain_state dom_state;
+
+    dom_state = sss_domain_get_state(dom);
+    assert_int_equal(dom_state, expected_state);
+}
+
+static void test_mark_subdom_offline_check(struct tevent_context *ev,
+                                           struct tevent_timer *te,
+                                           struct timeval current_time,
+                                           void *pvt)
+{
+    struct test_ctx *test_ctx = talloc_get_type(pvt, struct test_ctx);
+
+    assert_domain_state(test_ctx->be_ctx->domain->subdomains,
+                        DOM_ACTIVE);
+
+    test_ctx->tctx->done = true;
+    test_ctx->tctx->error = EOK;
+}
+
+static void test_mark_dom_offline(void **state)
+{
+    struct test_ctx *test_ctx = talloc_get_type(*state, struct test_ctx);
+
+    assert_domain_state(test_ctx->be_ctx->domain, DOM_ACTIVE);
+    assert_false(be_is_offline(test_ctx->be_ctx));
+
+    be_mark_dom_offline(test_ctx->be_ctx->domain, test_ctx->be_ctx);
+
+    assert_true(be_is_offline(test_ctx->be_ctx));
+    assert_domain_state(test_ctx->be_ctx->domain, DOM_ACTIVE);
+}
+
+static void test_mark_subdom_offline(void **state)
+{
+    struct timeval tv;
+    struct tevent_timer *check_ev = NULL;
+    struct test_ctx *test_ctx = talloc_get_type(*state, struct test_ctx);
+    errno_t ret;
+
+    assert_domain_state(test_ctx->be_ctx->domain->subdomains,
+                        DOM_ACTIVE);
+    assert_false(be_is_offline(test_ctx->be_ctx));
+
+    global_timer_added = false;
+    be_mark_dom_offline(test_ctx->be_ctx->domain->subdomains, test_ctx->be_ctx);
+    assert_domain_state(test_ctx->be_ctx->domain->subdomains,
+                        DOM_INACTIVE);
+
+    /* A timer must be added that resets the state back */
+    assert_true(global_timer_added);
+
+    /* Global offline state must not change */
+    assert_false(be_is_offline(test_ctx->be_ctx));
+
+    /* Make sure we don't add a second timer */
+    global_timer_added = false;
+    be_mark_dom_offline(test_ctx->be_ctx->domain->subdomains, test_ctx->be_ctx);
+    assert_domain_state(test_ctx->be_ctx->domain->subdomains,
+                        DOM_INACTIVE);
+    assert_false(global_timer_added);
+
+    /* Wait for the internal timer to reset our subdomain back */
+    tv = tevent_timeval_current_ofs(OFFLINE_TIMEOUT + 1, 0);
+
+    check_ev = tevent_add_timer(test_ctx->tctx->ev, test_ctx, tv,
+                                test_mark_subdom_offline_check,
+                                test_ctx);
+    if (check_ev == NULL) {
+        DEBUG(SSSDBG_OP_FAILURE, "Cannot create timer\n");
+        return;
+    }
+
+    ret = test_ev_loop(test_ctx->tctx);
+    assert_int_equal(ret, EOK);
+}
+
+static void test_mark_subdom_offline_disabled(void **state)
+{
+    struct test_ctx *test_ctx = talloc_get_type(*state, struct test_ctx);
+
+    sss_domain_set_state(test_ctx->be_ctx->domain->subdomains, DOM_DISABLED);
+    assert_domain_state(test_ctx->be_ctx->domain->subdomains,
+                        DOM_DISABLED);
+
+    be_mark_dom_offline(test_ctx->be_ctx->domain->subdomains, test_ctx->be_ctx);
+    assert_domain_state(test_ctx->be_ctx->domain->subdomains,
+                        DOM_DISABLED);
+}
+
+int main(int argc, const char *argv[])
+{
+    poptContext pc;
+    int opt;
+    int rv;
+    int no_cleanup = 0;
+    struct poptOption long_options[] = {
+        POPT_AUTOHELP
+        SSSD_DEBUG_OPTS
+        {"no-cleanup", 'n', POPT_ARG_NONE, &no_cleanup, 0,
+         _("Do not delete the test database after a test run"), NULL },
+        POPT_TABLEEND
+    };
+
+    const struct CMUnitTest tests[] = {
+        cmocka_unit_test_setup_teardown(test_mark_dom_offline,
+                                        test_setup,
+                                        test_teardown),
+        cmocka_unit_test_setup_teardown(test_mark_subdom_offline,
+                                        test_setup,
+                                        test_teardown),
+        cmocka_unit_test_setup_teardown(test_mark_subdom_offline_disabled,
+                                        test_setup,
+                                        test_teardown),
+    };
+
+    /* Set debug level to invalid value so we can deside if -d 0 was used. */
+    debug_level = SSSDBG_INVALID;
+
+    pc = poptGetContext(argv[0], argc, argv, long_options, 0);
+    while((opt = poptGetNextOpt(pc)) != -1) {
+        switch(opt) {
+        default:
+            fprintf(stderr, "\nInvalid option %s: %s\n\n",
+                    poptBadOption(pc, 0), poptStrerror(opt));
+            poptPrintUsage(pc, stderr, 0);
+            return 1;
+        }
+    }
+    poptFreeContext(pc);
+
+    DEBUG_CLI_INIT(debug_level);
+
+    /* Even though normally the tests should clean up after themselves
+     * they might not after a failed run. Remove the old db to be sure */
+    tests_set_cwd();
+    test_dom_suite_cleanup(TESTS_PATH, TEST_CONF_DB, TEST_DOM_NAME);
+    test_dom_suite_setup(TESTS_PATH);
+
+    rv = cmocka_run_group_tests(tests, NULL, NULL);
+    if (rv == 0 && !no_cleanup) {
+        test_dom_suite_cleanup(TESTS_PATH, TEST_CONF_DB, TEST_DOM_NAME);
+    }
+    return rv;
+
+    return cmocka_run_group_tests(tests, NULL, NULL);
+}
-- 
2.4.3

-------------- next part --------------
>From 6569e6358a829e18d44268b8d5ad628fc358a8f5 Mon Sep 17 00:00:00 2001
From: Jakub Hrozek <jhrozek at redhat.com>
Date: Wed, 2 Sep 2015 13:40:48 +0200
Subject: [PATCH 3/8] SDAP: Do not set is_offline if ignore_mark_offline is set

Required for:
https://fedorahosted.org/sssd/ticket/2637

The caller of the sdap_id_op requests can set the ignore_mark_offline
flag to avoid the sdap_id_op from marking the whole back end as offline.

However, there was a small bug - the is_offline internal sdap_id_op flag
was still being set. As a consequence, the error code from the
connection was ignored and EAGAIN was always returned.
---
 src/providers/ldap/sdap_id_op.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/providers/ldap/sdap_id_op.c b/src/providers/ldap/sdap_id_op.c
index 508bbd2ad3ebb3f9159a8b7f48258ff31a7cd6e2..0474a9cb7828ef43ef76cf6bebac077315296875 100644
--- a/src/providers/ldap/sdap_id_op.c
+++ b/src/providers/ldap/sdap_id_op.c
@@ -567,9 +567,9 @@ static void sdap_id_op_connect_done(struct tevent_req *subreq)
             DEBUG(SSSDBG_CRIT_FAILURE,
                   "Failed to connect, going offline (%d [%s])\n",
                    ret, strerror(ret));
+            is_offline = true;
             be_mark_offline(conn_cache->id_conn->id_ctx->be);
         }
-        is_offline = true;
     }
 
     if (ret == EOK) {
-- 
2.4.3

-------------- next part --------------
>From 85729433e3dc094a97e08e1d81f6f9ce32f21807 Mon Sep 17 00:00:00 2001
From: Jakub Hrozek <jhrozek at redhat.com>
Date: Wed, 2 Sep 2015 13:42:06 +0200
Subject: [PATCH 4/8] AD: Only ignore errors from SDAP lookups if there's
 another connection to fallback to

Required for:
https://fedorahosted.org/sssd/ticket/2637

The AD lookup code honors the ignore_mark_offline flag in the sense that
if it's set, the sdap return code is not reported to the upper layer,
but EOK is returned as request status and the sdap return code is
returned separately.

This patch modifies the behaviour further to only apply if there is
another connection to fall back to.
---
 src/providers/ad/ad_id.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/providers/ad/ad_id.c b/src/providers/ad/ad_id.c
index 7a0c6eccd2d2f0d4f8a545a9d4873a9447179a00..4f327f823173eb113153a556322dae4cc4b42f3e 100644
--- a/src/providers/ad/ad_id.c
+++ b/src/providers/ad/ad_id.c
@@ -146,6 +146,7 @@ ad_handle_acct_info_done(struct tevent_req *subreq)
 
     ret = sdap_handle_acct_req_recv(subreq, &dp_error, &err, &sdap_err);
     if (dp_error == DP_ERR_OFFLINE
+        && state->conn[state->cindex+1] != NULL
         && state->conn[state->cindex]->ignore_mark_offline) {
          /* This is a special case: GC does not work.
           *  We need to Fall back to ldap
-- 
2.4.3

-------------- next part --------------
>From e2fefd695ffce51e111666fb7bf5a654fdf65f55 Mon Sep 17 00:00:00 2001
From: Jakub Hrozek <jhrozek at redhat.com>
Date: Wed, 2 Sep 2015 15:53:34 +0200
Subject: [PATCH 5/8] KRB5: Offline operation with disabled domain

https://fedorahosted.org/sssd/ticket/2637

If a subdomain is in the disabled state, switch krb5_child operation
into offline mode.

Similarly, instead of marking the whole back end as offline, mark just
the domain as offline -- depending on the domain type, this would mark
the whole back end or just inactivate subdomain.
---
 src/providers/krb5/krb5_auth.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/src/providers/krb5/krb5_auth.c b/src/providers/krb5/krb5_auth.c
index d35df13994a3e16feff90592bec16d7a8f30b70a..e3e9601b356efd72e50ab86e8b7cdd048e4e70d4 100644
--- a/src/providers/krb5/krb5_auth.c
+++ b/src/providers/krb5/krb5_auth.c
@@ -720,7 +720,7 @@ static void krb5_auth_resolve_done(struct tevent_req *subreq)
              * was found good, setting offline,
              * but we still have to call the child to setup
              * the ccache file if we are performing auth */
-            be_mark_offline(state->be_ctx);
+            be_mark_dom_offline(state->domain, state->be_ctx);
             kr->is_offline = true;
 
             if (kr->pd->cmd == SSS_PAM_CHAUTHTOK ||
@@ -754,9 +754,19 @@ static void krb5_auth_resolve_done(struct tevent_req *subreq)
         kr->is_offline = be_is_offline(state->be_ctx);
     }
 
+    if (!kr->is_offline
+            && sss_domain_get_state(state->domain) == DOM_INACTIVE) {
+        DEBUG(SSSDBG_TRACE_INTERNAL,
+              "Subdomain %s is inactive, will proceed offline\n",
+              state->domain->name);
+        kr->is_offline = true;
+    }
+
     if (kr->is_offline
             && sss_krb5_realm_has_proxy(dp_opt_get_cstring(kr->krb5_ctx->opts,
                                         KRB5_REALM))) {
+        DEBUG(SSSDBG_TRACE_FUNC,
+              "Resetting offline status, KDC proxy is in use\n");
         kr->is_offline = false;
     }
 
-- 
2.4.3

-------------- next part --------------
>From 3e56b7ec8a5d4295cb52c16ef4f12cdd20ee9ec0 Mon Sep 17 00:00:00 2001
From: Jakub Hrozek <jhrozek at redhat.com>
Date: Wed, 2 Sep 2015 15:52:51 +0200
Subject: [PATCH 6/8] AD: Do not mark the whole back end as offline if
 subdomain lookup fails

Required for:
https://fedorahosted.org/sssd/ticket/2637

Rather mark the domain as inactive. It will be marked as active later,
in the meantime the main domain can continue to work online and
subdomain requests will be answered from cache.

The lookup request itself just returns a special error code and lets the
caller handle the error code as appropriate (normally by disabling the
subdomain temporarily).
---
 src/providers/ad/ad_id.c | 81 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 67 insertions(+), 14 deletions(-)

diff --git a/src/providers/ad/ad_id.c b/src/providers/ad/ad_id.c
index 4f327f823173eb113153a556322dae4cc4b42f3e..ecaf6c993bf7ddb7ba565d40ef0ad250114f5536 100644
--- a/src/providers/ad/ad_id.c
+++ b/src/providers/ad/ad_id.c
@@ -91,17 +91,27 @@ ad_handle_acct_info_send(TALLOC_CTX *mem_ctx,
     state->ad_options = ad_options;
     state->cindex = 0;
 
+    if (sss_domain_get_state(sdom->dom) == DOM_INACTIVE) {
+        ret = ERR_SUBDOM_INACTIVE;
+        goto immediate;
+    }
+
     ret = ad_handle_acct_info_step(req);
-    if (ret == EOK) {
-        tevent_req_done(req);
-        tevent_req_post(req, be_ctx->ev);
-    } else if (ret != EAGAIN) {
-        tevent_req_error(req, ret);
-        tevent_req_post(req, be_ctx->ev);
+    if (ret != EAGAIN) {
+        goto immediate;
     }
 
     /* Lookup in progress */
     return req;
+
+immediate:
+    if (ret != EOK) {
+        tevent_req_error(req, ret);
+    } else {
+        tevent_req_done(req);
+    }
+    tevent_req_post(req, be_ctx->ev);
+    return req;
 }
 
 static errno_t
@@ -160,8 +170,7 @@ ad_handle_acct_info_done(struct tevent_req *subreq)
         state->dp_error = dp_error;
         state->err = err;
 
-        tevent_req_error(req, ret);
-        return;
+        goto fail;
     }
 
     if (sdap_err == EOK) {
@@ -170,8 +179,8 @@ ad_handle_acct_info_done(struct tevent_req *subreq)
     } else if (sdap_err == ERR_NO_POSIX) {
         disable_gc(state->ad_options);
     } else if (sdap_err != ENOENT) {
-        tevent_req_error(req, EIO);
-        return;
+        ret = EIO;
+        goto fail;
     }
 
     /* Ret is only ENOENT or ERR_NO_POSIX now. Try the next connection */
@@ -188,12 +197,27 @@ ad_handle_acct_info_done(struct tevent_req *subreq)
             /* No more connections */
             tevent_req_done(req);
         } else {
-            tevent_req_error(req, ret);
+            goto fail;
         }
         return;
     }
 
     /* Another lookup in progress */
+    return;
+
+fail:
+    if (IS_SUBDOMAIN(state->sdom->dom)) {
+        /* Deactivate subdomain on lookup errors instead of going
+         * offline completely.
+         * This is a stopgap, until our failover is per-domain,
+         * not per-backend. Unfortunately, we can't rewrite the error
+         * code on some reported codes only, because sdap_id_op code
+         * encapsulated the failover as well..
+         */
+        ret = ERR_SUBDOM_INACTIVE;
+    }
+    tevent_req_error(req, ret);
+    return;
 }
 
 errno_t
@@ -258,6 +282,16 @@ get_conn_list(struct be_req *breq, struct ad_id_ctx *ad_ctx,
         break;
     }
 
+    /* Regardless of connection types, a subdomain error must not be allowed
+     * to set the whole back end offline, rather report an error and let the
+     * caller deal with it (normally disable the subdomain
+     */
+    if (IS_SUBDOMAIN(dom)) {
+        for (cindex = 0; clist[cindex] != NULL; cindex++) {
+            clist[cindex]->ignore_mark_offline = true;
+        }
+    }
+
     return clist;
 }
 
@@ -328,6 +362,11 @@ done:
 
 static void ad_account_info_complete(struct tevent_req *req);
 
+struct ad_account_info_state {
+    struct be_req *be_req;
+    struct sss_domain_info *dom;
+};
+
 void
 ad_account_info_handler(struct be_req *be_req)
 {
@@ -341,6 +380,7 @@ ad_account_info_handler(struct be_req *be_req)
     struct sdap_id_conn_ctx **clist;
     bool shortcut;
     errno_t ret;
+    struct ad_account_info_state *state;
 
     ad_ctx = talloc_get_type(be_ctx->bet_info[BET_ID].pvt_bet_data,
                              struct ad_id_ctx);
@@ -391,13 +431,21 @@ ad_account_info_handler(struct be_req *be_req)
         goto fail;
     }
 
+    state = talloc(be_req, struct ad_account_info_state);
+    if (state == NULL) {
+        ret = ENOMEM;
+        goto fail;
+    }
+    state->dom = sdom->dom;
+    state->be_req = be_req;
+
     req = ad_handle_acct_info_send(be_req, be_req, ar, sdap_id_ctx,
                                    ad_ctx->ad_options, sdom, clist);
     if (req == NULL) {
         ret = ENOMEM;
         goto fail;
     }
-    tevent_req_set_callback(req, ad_account_info_complete, be_req);
+    tevent_req_set_callback(req, ad_account_info_complete, state);
     return;
 
 fail:
@@ -412,12 +460,17 @@ ad_account_info_complete(struct tevent_req *req)
     int dp_error;
     const char *error_text = "Internal error";
     const char *req_error_text;
+    struct ad_account_info_state *state;
 
-    be_req = tevent_req_callback_data(req, struct be_req);
+    state = tevent_req_callback_data(req, struct ad_account_info_state);
+    be_req = state->be_req;
 
     ret = ad_handle_acct_info_recv(req, &dp_error, &req_error_text);
     talloc_zfree(req);
-    if (dp_error == DP_ERR_OK) {
+    if (ret == ERR_SUBDOM_INACTIVE) {
+        be_mark_dom_offline(state->dom, be_req_get_be_ctx(be_req));
+        return be_req_terminate(be_req, DP_ERR_OFFLINE, EAGAIN, "Offline");
+    } else if (dp_error == DP_ERR_OK) {
         if (ret == EOK) {
             error_text = NULL;
         } else {
-- 
2.4.3

-------------- next part --------------
>From e29f4a6810d78261788bb061571f6a84e020b36d Mon Sep 17 00:00:00 2001
From: Jakub Hrozek <jhrozek at redhat.com>
Date: Wed, 2 Sep 2015 12:10:03 +0000
Subject: [PATCH 7/8] AD: Set ignore_mark_offline=false when resolving AD root
 domain

https://fedorahosted.org/sssd/ticket/2637

Avoid going offline in cases where SSSD is connected to a child domain
but the root domain is not accessible.
---
 src/providers/ad/ad_subdomains.c | 56 +++++++++++++++++++++++-----------------
 1 file changed, 33 insertions(+), 23 deletions(-)

diff --git a/src/providers/ad/ad_subdomains.c b/src/providers/ad/ad_subdomains.c
index d1d468043410c80e6bf7f0f48a13bd9e962552af..8ed3dab0995f78a16f4a7df2e729ea88a39a782c 100644
--- a/src/providers/ad/ad_subdomains.c
+++ b/src/providers/ad/ad_subdomains.c
@@ -80,7 +80,8 @@ struct ad_subdomains_req_ctx {
     struct ad_id_ctx *root_id_ctx;
     struct sdap_id_op *root_op;
     size_t root_base_iter;
-    struct sysdb_attrs *root_domain;
+    struct sysdb_attrs *root_domain_attrs;
+    struct sss_domain_info *root_domain;
 
     size_t reply_count;
     struct sysdb_attrs **reply;
@@ -689,6 +690,7 @@ static errno_t ad_subdomains_get_root(struct ad_subdomains_req_ctx *ctx)
     return EAGAIN;
 }
 
+static struct sss_domain_info *ads_get_root_domain(struct ad_subdomains_req_ctx *ctx);
 static struct ad_id_ctx *ads_get_root_id_ctx(struct ad_subdomains_req_ctx *ctx);
 static void ad_subdomains_root_conn_done(struct tevent_req *req);
 
@@ -769,7 +771,14 @@ static void ad_subdomains_get_root_domain_done(struct tevent_req *req)
         }
     }
 
-    ctx->root_domain = reply[0];
+    ctx->root_domain_attrs = reply[0];
+    ctx->root_domain = ads_get_root_domain(ctx);
+    if (ctx->root_domain == NULL) {
+        DEBUG(SSSDBG_OP_FAILURE, "Could not find the root domain\n");
+        ret = EFAULT;
+        goto fail;
+    }
+
     ctx->root_id_ctx = ads_get_root_id_ctx(ctx);
     if (ctx->root_id_ctx == NULL) {
         DEBUG(SSSDBG_OP_FAILURE, "Cannot create id ctx for the root domain\n");
@@ -803,15 +812,13 @@ fail:
     be_req_terminate(ctx->be_req, dp_error, ret, NULL);
 }
 
-static struct ad_id_ctx *ads_get_root_id_ctx(struct ad_subdomains_req_ctx *ctx)
+static struct sss_domain_info *ads_get_root_domain(struct ad_subdomains_req_ctx *ctx)
 {
     errno_t ret;
     const char *name;
     struct sss_domain_info *root;
-    struct sdap_domain *sdom;
-    struct ad_id_ctx *root_id_ctx;
 
-    ret = sysdb_attrs_get_string(ctx->root_domain, AD_AT_TRUST_PARTNER, &name);
+    ret = sysdb_attrs_get_string(ctx->root_domain_attrs, AD_AT_TRUST_PARTNER, &name);
     if (ret != EOK) {
         DEBUG(SSSDBG_OP_FAILURE, "sysdb_attrs_get_string failed.\n");
         return NULL;
@@ -820,32 +827,40 @@ static struct ad_id_ctx *ads_get_root_id_ctx(struct ad_subdomains_req_ctx *ctx)
     /* With a subsequent run, the root should already be known */
     root = find_domain_by_name(ctx->sd_ctx->be_ctx->domain,
                                name, false);
-    if (root == NULL) {
-        DEBUG(SSSDBG_OP_FAILURE, "Could not find the root domain\n");
-        return NULL;
-    }
 
-    sdom = sdap_domain_get(ctx->sd_ctx->ad_id_ctx->sdap_id_ctx->opts, root);
+    return root;
+}
+
+static struct ad_id_ctx *ads_get_root_id_ctx(struct ad_subdomains_req_ctx *ctx)
+{
+    errno_t ret;
+    struct sdap_domain *sdom;
+    struct ad_id_ctx *root_id_ctx;
+
+    sdom = sdap_domain_get(ctx->sd_ctx->ad_id_ctx->sdap_id_ctx->opts,
+                           ctx->root_domain);
     if (sdom == NULL) {
         DEBUG(SSSDBG_OP_FAILURE,
-              "Cannot get the sdom for %s!\n", root->name);
+              "Cannot get the sdom for %s!\n", ctx->root_domain->name);
         return NULL;
     }
 
     if (sdom->pvt == NULL) {
         ret = ad_subdom_ad_ctx_new(ctx->sd_ctx->be_ctx,
                                    ctx->sd_ctx->ad_id_ctx,
-                                   root,
+                                   ctx->root_domain,
                                    &root_id_ctx);
         if (ret != EOK) {
             DEBUG(SSSDBG_OP_FAILURE, "ad_subdom_ad_ctx_new failed.\n");
             return NULL;
         }
+
         sdom->pvt = root_id_ctx;
     } else {
         root_id_ctx = sdom->pvt;
     }
 
+    root_id_ctx->ldap_ctx->ignore_mark_offline = true;
     return root_id_ctx;
 }
 
@@ -860,16 +875,11 @@ static void ad_subdomains_root_conn_done(struct tevent_req *req)
     ret = sdap_id_op_connect_recv(req, &dp_error);
     talloc_zfree(req);
     if (ret) {
-        if (dp_error == DP_ERR_OFFLINE) {
-            DEBUG(SSSDBG_MINOR_FAILURE,
-                  "No AD server is available, cannot get the "
-                  "subdomain list while offline\n");
-        } else {
-            DEBUG(SSSDBG_OP_FAILURE,
-                  "Failed to connect to AD server: [%d](%s)\n",
-                  ret, strerror(ret));
-        }
+        be_mark_dom_offline(ctx->root_domain, be_req_get_be_ctx(ctx->be_req));
 
+        DEBUG(SSSDBG_OP_FAILURE,
+              "Failed to connect to AD server: [%d](%s)\n",
+              ret, strerror(ret));
         goto fail;
     }
 
@@ -1040,7 +1050,7 @@ static void ad_subdomains_get_slave_domain_done(struct tevent_req *req)
      */
     ret = ad_subdomains_process(ctx, ctx->sd_ctx->be_ctx->domain,
                                 ctx->reply_count, ctx->reply,
-                                ctx->root_domain, &nsubdoms, &subdoms);
+                                ctx->root_domain_attrs, &nsubdoms, &subdoms);
     if (ret != EOK) {
         DEBUG(SSSDBG_OP_FAILURE, ("Cannot process subdomain list\n"));
         tevent_req_error(req, ret);
-- 
2.4.3

-------------- next part --------------
>From 443c90d567270e2aa54f577a706c96e1afc74cd1 Mon Sep 17 00:00:00 2001
From: Jakub Hrozek <jhrozek at redhat.com>
Date: Wed, 2 Sep 2015 13:41:26 +0200
Subject: [PATCH 8/8] IPA: Do not allow the AD lookup code to set backend as
 offline in server mode

https://fedorahosted.org/sssd/ticket/2637

In server mode, we should not allow the AD lookups to set the backend
offline. Rather just let them report an error and deal with the error
separately.
---
 src/providers/ipa/ipa_subdomains_id.c | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/src/providers/ipa/ipa_subdomains_id.c b/src/providers/ipa/ipa_subdomains_id.c
index ad1743ae5fe7ff80518e7bc58e4df04143732719..7c609ab6e69d6f23c4de7c1c9569d73074d4e2dd 100644
--- a/src/providers/ipa/ipa_subdomains_id.c
+++ b/src/providers/ipa/ipa_subdomains_id.c
@@ -634,6 +634,7 @@ ipa_get_ad_acct_send(TALLOC_CTX *mem_ctx,
             ret = ENOMEM;
             goto fail;
         }
+        clist[1]->ignore_mark_offline = true;
         break;
     default:
         clist = talloc_zero_array(req, struct sdap_id_conn_ctx *, 2);
@@ -642,6 +643,7 @@ ipa_get_ad_acct_send(TALLOC_CTX *mem_ctx,
             goto fail;
         }
         clist[0] = ad_id_ctx->ldap_ctx;
+        clist[0]->ignore_mark_offline = true;
         clist[1] = NULL;
     }
 
@@ -1037,7 +1039,11 @@ ipa_get_ad_acct_ad_part_done(struct tevent_req *subreq)
 
     ret = ad_handle_acct_info_recv(subreq, &state->dp_error, NULL);
     talloc_zfree(subreq);
-    if (ret != EOK) {
+    if (ret == ERR_SUBDOM_INACTIVE) {
+        be_mark_dom_offline(state->obj_dom, be_req_get_be_ctx(state->be_req));
+        tevent_req_error(req, ret);
+        return;
+    } else if (ret != EOK) {
         DEBUG(SSSDBG_OP_FAILURE, "AD lookup failed: %d\n", ret);
         tevent_req_error(req, ret);
         return;
-- 
2.4.3



More information about the sssd-devel mailing list