[SSSD] Design discussion - Changes required to support one-way trusts

Jakub Hrozek jhrozek at redhat.com
Mon Apr 27 08:32:03 UTC 2015


On Sun, Apr 26, 2015 at 06:17:21PM -0400, Simo Sorce wrote:
> Very nice writeup!
> A few comments..
> 
> On Sun, 2015-04-26 at 21:22 +0200, Jakub Hrozek wrote:
> > Hi,
> > 
> > I prepared design page for the one-way trusts feature:
> >     https://fedorahosted.org/sssd/wiki/DesignDocs/OneWayTrusts
> > 
> > The text is copied below as well for your convenience. I have a couple
> > of questions to discuss:
> >     - Is it worth it to put the keytabs into a new subdirectory so that
> >       a special SELinux label can be applied? I think yes, since then
> >       we'd have additional layer of protection of the trust secrets
> 
> It is not clear to me what would be the advantage, who are you trying to
> protect against ?
> Adding different labels makes sense only if sssd_be has a different
> label from the other sssd processes, but then you can simply have
> selinux policies that label accordingly to the parent process in the
> same directory, no?

Yes, I tried to future-proof the directory structure against different
sssd processes that need access to the database (responders) but not the
keytabs, because I suspect we won't think about the directory structure
again once we set it up :-). And a single separate directory just seemed
easier to label with a new context than several files.

In the end, I don't care that much as long as the keytabs have different
contexts that the responders can't access.

> 
> >     - Is there any reasonable fallback beyond re-fetching the keytab we
> >       can do? I can't think of any..
> 
> Be careful with refetches and so we do not want to end up in infinite
> loops, where the keytab can be fetched but still operation fails for
> whatever reason.

Sure, I will add a more explicit note once we agree what the mechanism
would be.

> One thing that should be checked is if the new keytab and old keytab
> have the same keys in them, and, if so just stop re-fetching and raising
> a fatal error (at list for a reasonable period of time).

I thought about a flag in the ipa_ad_server_ctx structure, because after
the trust is deleted and set up again, the kvno is still 1. When you say
'keys', are you proposing to compare the secret key? (krb5_keytab_entry.key)

> 
> >     - Should the IPA tools signal SSSD either via UNIX signals or via
> >       the DBus API once a trust has been added or removed to minimize
> >       the race condition and improve user experience? Currently I hope
> >       not, the proposed mechanism should be enough but testing is still
> >       required.
> > 
> > When the design page is agreed on, I'll turn the three subsections of
> > the implementation phase into smaller tickets and close the tracker.
> > 
> > = Feature Name =
> > 
> > Related ticket(s):
> >  * https://fedorahosted.org/sssd/ticket/2579
> > 
> > === Problem statement ===
> > The next IPA release will support one-way trusts. SSSD needs to add support
> > for this feature in its server mode.
> > 
> > === Use cases ===
> > One-way trust to Active Directory where FreeIPA realm trusts Active Directory
> > forest using cross-forest trust feature of AD but the AD forest does not trust
> > FreeIPA realm. Users from AD forest can access resources in FreeIPA realm.
> > 
> > === Overview of the solution ===
> > At a high level, SSSD needs to examine the trust objects whether they
> > are one-way or two way trusts. For each one-way trust, SSSD needs to fetch
> > and store the keytab and use the keytab to secure the connection. For
> > two-way trusts, we can keep using the existing code that reuses the IPA
> > realm and the system keytab for both IPA and AD connectins. Care must be
> > taken to remove keytabs of trusts that were removed as well.
> > 
> > Fetching the keytab would be done by calling the `ipa-getkeytab` utility
> > for every one-way trust. The keytab would only be (re)fetched if it's
> > missing or if attempting to use the keytab failed. On the IPA server,
> > we must make sure that the IPA server identity is allowed to read the keytab.
> > 
> > Because handling multiple keytabs increases the risk of failing connections in
> > case the trust wasn't setup correctly, we need to modify the failover code to
> > not set the whole back end offline in case connecting to a subdomain AD server
> > fails. Instead, the subdomain will be marked as inactive for a short period of
> > time, during which it would act as offline. The proper way of solving this
> > problem would be to rework the failover module so that it can act per domain,
> > not only per back end, however, that change is out of scope for this release.
> > 
> > === Implementation details ===
> > This section describes all the required changes in detail.
> > 
> > ==== Reading the subdomains in the IPA subdomain handler ====
> > The IPA subdomain handler will include the attribute `ipaNTTrustDirection`
> > when reading the trust objects. Currently this attribute is not readable
> > by the host principal, so the IPA ACIs must be relaxed (ipa ticket?). If
> > the trust direction is set to an OR of `lsa.LSA_TRUST_DIRECTION_OUTBOUND`
> > and `lsa.LSA_TRUST_DIRECTION_INBOUND`, then it's a two-way trust and we'll
> > just use the existing code that re-uses the IPA keytab for the AD trusted
> > domain as well. If the attribute is only `lsa.LSA_TRUST_DIRECTION_OUTBOUND`,
> > we handle the trust as a one-way trust. The trust type can be stored in
> > `ipa_ad_server_ctx`.
> 
> What do you do if you have only lsa.LSA_TRUST_DIRECTION_INBOUND ?

Log this object loudly and ignore. Is there another reasonable thing to
do other than skip such trust relationship ? (Yes, skipping the object
should be spelled out explicitly in the design page)

> 
> > Each `sss_domain_info` structure will be created as `inactive` in the
> > subdomain code. After enumerating the trusted domains, the subdomain
> > handler will check if a keytab already exists for every one-way trusted
> > domain. If yes, the domain is ready to use and can be enabled. If there is
> > no keytab, the subdomain handler will fork out a call to `ipa-getkeytab`,
> > fetch the keytab and store it under `/var/lib/sss/db`. The `ipa-getkeytab`
> > call will be done using Kerberos credentials the host has. IPA ACIs must
> > be modified accordingly to allow the IPA server principals to fetch the
> > trust keytabs, but noone else.
> > 
> > The directory `/var/lib/sss/db` is already only accessible to the sssd
> > user only. As an additional security measure, we might want to store the
> > keytabs under a new directory so that the keytabs also receive a SELinux
> > context stricter than the default `sssd_var_lib_t`. If fetching the keytab
> > succeeds, the domain would be enabled.  The SELinux policy must also be
> > adjusted to allow calling `ipa-getkeytab` by the `sssd_be` process.
> 
> Getting a keytab is an LDAP extended operation, do you want to fork off
> ipa-getkeytab to avoid coding the LDAP operatin in SSSD ? Or for some
> other reason ?

No, I just wanted to use code that is already there. Retrieving the
keytab should be rare enough to justify the overhead of forking an
executable and would let us to keep up with any changes IPA might do.

Or let me turn that question around -- do you see any advantage of
searching the keytab directly from SSSD?

> 
> > If any trust relationships were removed, the corresponding keytabs must
> > be removed from the disk as well.
> 
> How do you do this ?
> Just regularly enumerate the keytab files on disk and remove the ones
> that have no object in the cache ?

Yes, after the subdomain handler finishes. We alrady have a return boolean
'refresh_has_changes' and re-initialize domain-realm mapping for
example.

> One advantage of using a separate subdir for keytabs, as you propose,
> would be to wipe it clean every time sssd restart (at stop and at start
> too for good measure), then enumerate the trusts on the server and
> re-fetch any keytab.

I thought about this, but it seemed a bit wasteful. But now that I think
of it, this would present a 'known control point' to start fresh..so it
might come handy.

> 
> > ==== Changes to the AD id_ctx instantiation ====
> > With two-way trust, we can keep using the default IPA principal and keytab.
> > 
> > With one-way trust, the keytab retrieved from the IPA server must be
> > used. Also, the principal must be passed into the `ad_create_default_options`
> > function. The custom values must be set before we proceed to instantiate
> > LDAP provider options. The only AD provider option we need to set is
> > AD_KRB5_REALM.
> > 
> > In the LDAP provider, we must take care that the following sdap options
> > are set correctly:
> >     * SDAP_SASL_AUTHID - must be set to the NetBIOS name of the IPA domain. (A
> >       domain `TRUST.COM` would set this value to `TRUST$`).
> 
> The NetBIOS domain can be completely unrelated to the DNS domain, so
> make sure you do not just try to truncate the realm but actually fetch
> the info from IPA/AD.

Sure, sorry, that was meant as an example only. I added a more explicit
sentence.

> 
> >     * SDAP_SASL_REALM - must be set to the AD realm
> >     * SDAP_KRB5_KEYTAB - must be set to the per-domain keytab retrieved from IPA
> > 
> > The AD provider eventually calls `sdap_set_sasl_options()` from the
> > LDAP provider, we need to make sure this function receives the correct
> > values. During experimentation we were able to show that using multiple
> > different SASL users and different realms doesn't cause any problems in
> > SASL or LDAP libraries.
> > 
> > The only place that will keep using the IPA realm is the failover
> > instantiation. We need to keep using this hack until failover is per-backend.
> > 
> > ==== Subdomain offline status changes ====
> > At the moment, the whole back end can be either online or offline and
> > the status applies to both the main domain and the subdomains alike. As
> > an effect, a failure to connect to a subdomain server would also make the
> > main domain operate offline. In many subdomain setups, it's actually more
> > convenient not to, because the subdomain server might be on a different
> > network segment, behind a different firewall etc. Instead, the domain
> > should only be made inactive.
> 
> Does this cover accurately the Forest vs AD subdomain cases ?
> We will have a keytab only for the AD root domain (the forest) and we
> need to use those credentials for any other domain in the AD forest.

Well, both subdomains would be made inactive, then. The point is to
avoid marking the IPA domain as offline if the trusts break.

btw this would be also useful for AD-AD trusts because what I learned is
that AD subdomains tend to be isolated from one another on the network
level as well and then failure to reach other domains is currently fatal
which brings the whole sssd_be process offline.

> 
> > The `sss_domain_info` structure would convert the 'bool disabled' parameter
> > into an `enum sss_domain_state`. The supported values would be:
> >     * ''disabled'' - the domain should not be used by either responder or
> >       provider. It was removed or disabled on the server.
> >     * ''active'' - the domain can be used by a responder and the data provider
> >       would forward its request to the backend
> >     * ''inactive'' - the domain can be used by a responder, but the data
> >       provider would just shortcut as if the domain was offline. For now, this
> >       option will be used by subdomains only.
> > 
> > The implementation would include renaming the existing `be_mark_offline()`
> > function to be called `be_mark_dom_offline()` and modifying its behaviour. The
> > existing code that sets the offline status and runs the offline callbacks
> > would be called for parent domains only. For subdomains, we would mark the
> > subdomain as inactive and schedule a tevent request that would unconditionally
> > reset inactive domain to active. The request would be scheduled after
> > `offline_timeout` seconds to be consistent with main domains from user's
> > perspective. Likewise, the `be_reset_offline()` function will be extended to
> > reset inactive domains to active as well as the SIGUSR1 and SIGUSR2 signal
> > handlers. Finally, all calls to the `be_is_offline()` function should be
> > inspected and the invocations that are per-domain should be converted to a new
> > function `be_dom_is_offline()` that would check the offline status for parent
> > domains and the offline state for subdomains. We should also make sure the
> > backend offline status structure is opaque as currently its internals are
> > readable by external users as well. Making the offline status opaque would
> > make it safer to perform modifications to the offline code.
> > 
> > In both offline and inactive cases, the ID handlers would reply with
> > `DP_ERR_OFFLINE`. The crucial difference between offline and inactive at this
> > point would be that inactive domains are re-activated undonditionally. When we
> > modify the failover code to handle domains separately, we'll be able to
> > leverage per-domain online checks or online/offline callbacks as well.
> > 
> > ==== Detecting re-established trusts ====
> > In cases the `sdap_kinit_send()` request fails, the sdap code would return a
> > special error code instead of blindly returning `EIO` as it does at the
> > moment. When the `ipa_get_ad_acct` request receives this error code, it would
> > re-run the subdomain request in order to check if the trust relationship still
> > exists and in order to re-fetch the keytab again. In case the trust is still
> > established, we would re-set the inactive domain status and re-run the account
> > request. In case the trust went away, the subdomain code should remove the
> > trusted domain (however, this must be tested). If the account request returns
> > any error after reinitializing the ketab, the subdomain would be marked
> > inactive in order to avoid fetching the keytab again and again. In order to be
> > able to run the subdomain request separately from the subdomain back end
> > handler, the subdomain code must be wrapped into a subdomain request as the
> > code currently assumes it's being called from the subdomain backend handler
> > only.
> > 
> > === Future work ===
> > Handling failover and offline status on per-domain basis instead of
> > per-backend basis should be done in the next release.
> > 
> > === Configuration changes ===
> > none
> > 
> > === How To Test ===
> > Establish a one-way trust relationship with an AD domain. Make sure both
> > IPA and AD users are resolvable. It's prudent to test combinations of
> > one-way and two-way trusts with different forests. Make sure removing a
> > trust relationship removes the keytab from the filesystem. Make sure that
> > SSSD handles re-establishing a trust relationship.
> > 
> > === Authors ===
> > * Jakub Hrozek <jhrozek at redhat.com>
> 
> Good job.

Thank you for taking the time to review the design.

I made some changes to the design page, here is the diff:
https://fedorahosted.org/sssd/wiki/DesignDocs/OneWayTrusts?action=diff&version=9&old_version=8

I'll do one more change after we agree on how should we protect against
loops in fetching keytabs (flag vs. check keytab contents).



More information about the sssd-devel mailing list