[SSSD] Design discussion - Changes required to support one-way trusts

Alexander Bokovoy abokovoy at redhat.com
Mon Apr 27 12:04:15 UTC 2015


On Mon, 27 Apr 2015, Jakub Hrozek wrote:
>On Sun, Apr 26, 2015 at 06:17:21PM -0400, Simo Sorce wrote:
>> Very nice writeup!
>> A few comments..
>>
>> On Sun, 2015-04-26 at 21:22 +0200, Jakub Hrozek wrote:
>> > Hi,
>> >
>> > I prepared design page for the one-way trusts feature:
>> >     https://fedorahosted.org/sssd/wiki/DesignDocs/OneWayTrusts
>> >
>> > The text is copied below as well for your convenience. I have a couple
>> > of questions to discuss:
>> >     - Is it worth it to put the keytabs into a new subdirectory so that
>> >       a special SELinux label can be applied? I think yes, since then
>> >       we'd have additional layer of protection of the trust secrets
>>
>> It is not clear to me what would be the advantage, who are you trying to
>> protect against ?
>> Adding different labels makes sense only if sssd_be has a different
>> label from the other sssd processes, but then you can simply have
>> selinux policies that label accordingly to the parent process in the
>> same directory, no?
>
>Yes, I tried to future-proof the directory structure against different
>sssd processes that need access to the database (responders) but not the
>keytabs, because I suspect we won't think about the directory structure
>again once we set it up :-). And a single separate directory just seemed
>easier to label with a new context than several files.
>
>In the end, I don't care that much as long as the keytabs have different
>contexts that the responders can't access.
Given that we want to have one keytab per forest trust and naming
keytabs after forest trust name, we can live with a single directory. It
would be good, indeed, to have it separated from the rest of sssd.


>>
>> >     - Is there any reasonable fallback beyond re-fetching the keytab we
>> >       can do? I can't think of any..
>>
>> Be careful with refetches and so we do not want to end up in infinite
>> loops, where the keytab can be fetched but still operation fails for
>> whatever reason.
>
>Sure, I will add a more explicit note once we agree what the mechanism
>would be.
Another thing to remember is a potential need to limit enctypes you'd be
requesting because camellia ciphers are not know to AD and might cause
issues at some point.


>> One thing that should be checked is if the new keytab and old keytab
>> have the same keys in them, and, if so just stop re-fetching and raising
>> a fatal error (at list for a reasonable period of time).
>
>I thought about a flag in the ipa_ad_server_ctx structure, because after
>the trust is deleted and set up again, the kvno is still 1. When you say
>'keys', are you proposing to compare the secret key? (krb5_keytab_entry.key)
Yes, don't use KVNO value to compare, verify actual keys.

>>
>> >     - Should the IPA tools signal SSSD either via UNIX signals or via
>> >       the DBus API once a trust has been added or removed to minimize
>> >       the race condition and improve user experience? Currently I hope
>> >       not, the proposed mechanism should be enough but testing is still
>> >       required.
>> >
>> > When the design page is agreed on, I'll turn the three subsections of
>> > the implementation phase into smaller tickets and close the tracker.
>> >
>> > = Feature Name =
>> >
>> > Related ticket(s):
>> >  * https://fedorahosted.org/sssd/ticket/2579
>> >
>> > === Problem statement ===
>> > The next IPA release will support one-way trusts. SSSD needs to add support
>> > for this feature in its server mode.
>> >
>> > === Use cases ===
>> > One-way trust to Active Directory where FreeIPA realm trusts Active Directory
>> > forest using cross-forest trust feature of AD but the AD forest does not trust
>> > FreeIPA realm. Users from AD forest can access resources in FreeIPA realm.
>> >
>> > === Overview of the solution ===
>> > At a high level, SSSD needs to examine the trust objects whether they
>> > are one-way or two way trusts. For each one-way trust, SSSD needs to fetch
>> > and store the keytab and use the keytab to secure the connection. For
>> > two-way trusts, we can keep using the existing code that reuses the IPA
>> > realm and the system keytab for both IPA and AD connectins. Care must be
>> > taken to remove keytabs of trusts that were removed as well.
>> >
>> > Fetching the keytab would be done by calling the `ipa-getkeytab` utility
>> > for every one-way trust. The keytab would only be (re)fetched if it's
>> > missing or if attempting to use the keytab failed. On the IPA server,
>> > we must make sure that the IPA server identity is allowed to read the keytab.
>> >
>> > Because handling multiple keytabs increases the risk of failing connections in
>> > case the trust wasn't setup correctly, we need to modify the failover code to
>> > not set the whole back end offline in case connecting to a subdomain AD server
>> > fails. Instead, the subdomain will be marked as inactive for a short period of
>> > time, during which it would act as offline. The proper way of solving this
>> > problem would be to rework the failover module so that it can act per domain,
>> > not only per back end, however, that change is out of scope for this release.
>> >
>> > === Implementation details ===
>> > This section describes all the required changes in detail.
>> >
>> > ==== Reading the subdomains in the IPA subdomain handler ====
>> > The IPA subdomain handler will include the attribute `ipaNTTrustDirection`
>> > when reading the trust objects. Currently this attribute is not readable
>> > by the host principal, so the IPA ACIs must be relaxed (ipa ticket?). If
>> > the trust direction is set to an OR of `lsa.LSA_TRUST_DIRECTION_OUTBOUND`
>> > and `lsa.LSA_TRUST_DIRECTION_INBOUND`, then it's a two-way trust and we'll
>> > just use the existing code that re-uses the IPA keytab for the AD trusted
>> > domain as well. If the attribute is only `lsa.LSA_TRUST_DIRECTION_OUTBOUND`,
>> > we handle the trust as a one-way trust. The trust type can be stored in
>> > `ipa_ad_server_ctx`.
>>
>> What do you do if you have only lsa.LSA_TRUST_DIRECTION_INBOUND ?
>
>Log this object loudly and ignore. Is there another reasonable thing to
>do other than skip such trust relationship ? (Yes, skipping the object
>should be spelled out explicitly in the design page)
Yes, nothing else you can do. I'm not sure at which point we'll get to
INBOUND-only case.


>> > Each `sss_domain_info` structure will be created as `inactive` in the
>> > subdomain code. After enumerating the trusted domains, the subdomain
>> > handler will check if a keytab already exists for every one-way trusted
>> > domain. If yes, the domain is ready to use and can be enabled. If there is
>> > no keytab, the subdomain handler will fork out a call to `ipa-getkeytab`,
>> > fetch the keytab and store it under `/var/lib/sss/db`. The `ipa-getkeytab`
>> > call will be done using Kerberos credentials the host has. IPA ACIs must
>> > be modified accordingly to allow the IPA server principals to fetch the
>> > trust keytabs, but noone else.
>> >
>> > The directory `/var/lib/sss/db` is already only accessible to the sssd
>> > user only. As an additional security measure, we might want to store the
>> > keytabs under a new directory so that the keytabs also receive a SELinux
>> > context stricter than the default `sssd_var_lib_t`. If fetching the keytab
>> > succeeds, the domain would be enabled.  The SELinux policy must also be
>> > adjusted to allow calling `ipa-getkeytab` by the `sssd_be` process.
>>
>> Getting a keytab is an LDAP extended operation, do you want to fork off
>> ipa-getkeytab to avoid coding the LDAP operatin in SSSD ? Or for some
>> other reason ?
>
>No, I just wanted to use code that is already there. Retrieving the
>keytab should be rare enough to justify the overhead of forking an
>executable and would let us to keep up with any changes IPA might do.
>
>Or let me turn that question around -- do you see any advantage of
>searching the keytab directly from SSSD?
I can see a downside in the fact that you'd need to handle ASN.1
encoding/decoding to fetch the keytab and if we ever change its
structure, SSSD would be another point of failure. Calling out to
ipa-getkeytab is giving us assurance to keep things working.


>> > If any trust relationships were removed, the corresponding keytabs must
>> > be removed from the disk as well.
>>
>> How do you do this ?
>> Just regularly enumerate the keytab files on disk and remove the ones
>> that have no object in the cache ?
>
>Yes, after the subdomain handler finishes. We alrady have a return boolean
>'refresh_has_changes' and re-initialize domain-realm mapping for
>example.
>
>> One advantage of using a separate subdir for keytabs, as you propose,
>> would be to wipe it clean every time sssd restart (at stop and at start
>> too for good measure), then enumerate the trusts on the server and
>> re-fetch any keytab.
>
>I thought about this, but it seemed a bit wasteful. But now that I think
>of it, this would present a 'known control point' to start fresh..so it
>might come handy.
If you do so, please make it triggered via some explicit mechanism, not
as a default.
 
>> > ==== Changes to the AD id_ctx instantiation ====
>> > With two-way trust, we can keep using the default IPA principal and keytab.
>> >
>> > With one-way trust, the keytab retrieved from the IPA server must be
>> > used. Also, the principal must be passed into the `ad_create_default_options`
>> > function. The custom values must be set before we proceed to instantiate
>> > LDAP provider options. The only AD provider option we need to set is
>> > AD_KRB5_REALM.
>> >
>> > In the LDAP provider, we must take care that the following sdap options
>> > are set correctly:
>> >     * SDAP_SASL_AUTHID - must be set to the NetBIOS name of the IPA domain. (A
>> >       domain `TRUST.COM` would set this value to `TRUST$`).
>>
>> The NetBIOS domain can be completely unrelated to the DNS domain, so
>> make sure you do not just try to truncate the realm but actually fetch
>> the info from IPA/AD.
>
>Sure, sorry, that was meant as an example only. I added a more explicit
>sentence.
>
>>
>> >     * SDAP_SASL_REALM - must be set to the AD realm
>> >     * SDAP_KRB5_KEYTAB - must be set to the per-domain keytab retrieved from IPA
>> >
>> > The AD provider eventually calls `sdap_set_sasl_options()` from the
>> > LDAP provider, we need to make sure this function receives the correct
>> > values. During experimentation we were able to show that using multiple
>> > different SASL users and different realms doesn't cause any problems in
>> > SASL or LDAP libraries.
>> >
>> > The only place that will keep using the IPA realm is the failover
>> > instantiation. We need to keep using this hack until failover is per-backend.
>> >
>> > ==== Subdomain offline status changes ====
>> > At the moment, the whole back end can be either online or offline and
>> > the status applies to both the main domain and the subdomains alike. As
>> > an effect, a failure to connect to a subdomain server would also make the
>> > main domain operate offline. In many subdomain setups, it's actually more
>> > convenient not to, because the subdomain server might be on a different
>> > network segment, behind a different firewall etc. Instead, the domain
>> > should only be made inactive.
>>
>> Does this cover accurately the Forest vs AD subdomain cases ?
>> We will have a keytab only for the AD root domain (the forest) and we
>> need to use those credentials for any other domain in the AD forest.
>
>Well, both subdomains would be made inactive, then. The point is to
>avoid marking the IPA domain as offline if the trusts break.
Yes, if you don't have keytab with keys to TDO object, you cannot
authenticate against any of the forest's AD DCs, in any domain of the
forest, so all of them would be offline in this case. Our current
problem is that we offline IPA domain here instead of the subdomains.


>btw this would be also useful for AD-AD trusts because what I learned is
>that AD subdomains tend to be isolated from one another on the network
>level as well and then failure to reach other domains is currently fatal
>which brings the whole sssd_be process offline.
>
>>
>> > The `sss_domain_info` structure would convert the 'bool disabled' parameter
>> > into an `enum sss_domain_state`. The supported values would be:
>> >     * ''disabled'' - the domain should not be used by either responder or
>> >       provider. It was removed or disabled on the server.
>> >     * ''active'' - the domain can be used by a responder and the data provider
>> >       would forward its request to the backend
>> >     * ''inactive'' - the domain can be used by a responder, but the data
>> >       provider would just shortcut as if the domain was offline. For now, this
>> >       option will be used by subdomains only.
>> >
>> > The implementation would include renaming the existing `be_mark_offline()`
>> > function to be called `be_mark_dom_offline()` and modifying its behaviour. The
>> > existing code that sets the offline status and runs the offline callbacks
>> > would be called for parent domains only. For subdomains, we would mark the
>> > subdomain as inactive and schedule a tevent request that would unconditionally
>> > reset inactive domain to active. The request would be scheduled after
>> > `offline_timeout` seconds to be consistent with main domains from user's
>> > perspective. Likewise, the `be_reset_offline()` function will be extended to
>> > reset inactive domains to active as well as the SIGUSR1 and SIGUSR2 signal
>> > handlers. Finally, all calls to the `be_is_offline()` function should be
>> > inspected and the invocations that are per-domain should be converted to a new
>> > function `be_dom_is_offline()` that would check the offline status for parent
>> > domains and the offline state for subdomains. We should also make sure the
>> > backend offline status structure is opaque as currently its internals are
>> > readable by external users as well. Making the offline status opaque would
>> > make it safer to perform modifications to the offline code.
>> >
>> > In both offline and inactive cases, the ID handlers would reply with
>> > `DP_ERR_OFFLINE`. The crucial difference between offline and inactive at this
>> > point would be that inactive domains are re-activated undonditionally. When we
>> > modify the failover code to handle domains separately, we'll be able to
>> > leverage per-domain online checks or online/offline callbacks as well.
>> >
>> > ==== Detecting re-established trusts ====
>> > In cases the `sdap_kinit_send()` request fails, the sdap code would return a
>> > special error code instead of blindly returning `EIO` as it does at the
>> > moment. When the `ipa_get_ad_acct` request receives this error code, it would
>> > re-run the subdomain request in order to check if the trust relationship still
>> > exists and in order to re-fetch the keytab again. In case the trust is still
>> > established, we would re-set the inactive domain status and re-run the account
>> > request. In case the trust went away, the subdomain code should remove the
>> > trusted domain (however, this must be tested). If the account request returns
>> > any error after reinitializing the ketab, the subdomain would be marked
>> > inactive in order to avoid fetching the keytab again and again. In order to be
>> > able to run the subdomain request separately from the subdomain back end
>> > handler, the subdomain code must be wrapped into a subdomain request as the
>> > code currently assumes it's being called from the subdomain backend handler
>> > only.
>> >
>> > === Future work ===
>> > Handling failover and offline status on per-domain basis instead of
>> > per-backend basis should be done in the next release.
>> >
>> > === Configuration changes ===
>> > none
>> >
>> > === How To Test ===
>> > Establish a one-way trust relationship with an AD domain. Make sure both
>> > IPA and AD users are resolvable. It's prudent to test combinations of
>> > one-way and two-way trusts with different forests. Make sure removing a
>> > trust relationship removes the keytab from the filesystem. Make sure that
>> > SSSD handles re-establishing a trust relationship.
>> >
>> > === Authors ===
>> > * Jakub Hrozek <jhrozek at redhat.com>
>>
>> Good job.
>
>Thank you for taking the time to review the design.
>
>I made some changes to the design page, here is the diff:
>https://fedorahosted.org/sssd/wiki/DesignDocs/OneWayTrusts?action=diff&version=9&old_version=8
>
>I'll do one more change after we agree on how should we protect against
>loops in fetching keytabs (flag vs. check keytab contents).
I think the design is sensible.
-- 
/ Alexander Bokovoy



More information about the sssd-devel mailing list