[SSSD] [DESIGN DISCUSSION] Configuring SSSD

Dmitri Pal dpal at redhat.com
Fri Jun 6 18:13:51 UTC 2014


On 06/06/2014 01:23 PM, Petr Viktorin wrote:
> On 06/06/2014 05:34 PM, Dmitri Pal wrote:
>> On 06/06/2014 06:46 AM, Petr Viktorin wrote:
>>> On 06/06/2014 04:09 AM, Dmitri Pal wrote:
>>>> Hello,
>>>>
>>>> Over the time multiple tools have been developed and used to address
>>>> different use cases around SSSD configuration.
>>>> We now have:
>>>> 1) SSSD that needs to read a configuration. It uses ding-libs
>>>> libini_config for that purpose.
>>>> libini_config was historically developed to read configuration from C
>>>> services like SSSD.
>>>> It is currently used by SSSD and GSS proxy
>>>> It is not capable of modifying and saving the configuration. To be 
>>>> fair
>>>> it is capable but the interfaces are not exposed at the high level 
>>>> API.
>>>> It will take some (couple days or so to close this gap).
>>>> 2) SSSD also needed to create sssd.conf at the installation time. 
>>>> It is
>>>> ueasy to do it from Python this is why an SSSD config python script 
>>>> have
>>>> been created. It is good at modifying the configuration and setting
>>>> specific values at the beginning of the sssd.conf's life. It is not
>>>> suited well (at least yet) for long term maintenance of the
>>>> configuration file. It is also SSSD specific.
>>>> 3) Over the time there were RFEs that suggested that sssd 
>>>> configuration
>>>> should be extensible or overwritable.
>>>> The following ticket cover that:
>>>> https://fedorahosted.org/sssd/ticket/2247
>>>> The design has been published here
>>>> https://fedorahosted.org/sssd/wiki/DesignDocs/ding-libs/INIConfigMerge
>>>> 4) There is a desire to manage SSSD over OpenLMI a cockpit in an 
>>>> object
>>>> oriented way. To do that (following the standard LMI model) the
>>>> following layring has been suggested and implemented:
>>>>
>>>> Consumer (OpenLMI/Cockpit) -> D-Bus interface -> ifp service 
>>>> (responder)
>>>> -> augeas c api
>>>>
>>>> More details are here:
>>>> https://fedorahosted.org/sssd/wiki/DesignDocs/OpenLMIProvider
>>>>
>>>> 5) In the absence of the good configuration solution for the 
>>>> integration
>>>> purposes, i.e. to integrate applications running on top of the system
>>>> with SSSD as a short term solution a puppet-augeas based way of
>>>> configuring sssd.conf was introduced the other day. The main
>>>> requirements are to be able to modify existing configuration and more
>>>> specifically change the values that represent a list of values.
>>>>
>>>> 6) From the very beginning we were hoping that there would be a way to
>>>> be able to define the grammar of the configuration file and check 
>>>> it to
>>>> be able to diagnose the config files and prevent their corruption. The
>>>> idea was to add this capability to libini as it provides a set of nice
>>>> convenience capabilities in comparison to Augeas. Also Augeas until 
>>>> not
>>>> long ago was not included in some distors.
>>>
>>> Could you elaborate on the convenience capabilities?
>>
>> Last time I looked at Augeas (which was some time ago) it did not:
>> - allow multiple key instances like this
>>   key = val1
>>   key = val2
>>   and ability to iterate a section and access them.
>>   This feature is used by GSS proxy
>> - allow processing the config and handling the sections in different
>> ways detecting collisions, erroring on them or merging them on the fly
>> as one requests
>> - merging two configs the way you want
>> - support of BOMs - encodings at the beginning of the file
>> - adding and or modifying comments
>> - allowing to wrap values
>> - processing lists of values in a convenient way
>
> Some of these are explicit non-goals of Augeas; some are features they 
> might have or want to have.
> In other words, yes, we will need to do some work if we adopt Augeas. 
> Some of that work would need to be be on top of Augeas.
>
>> May be some of these feature already available and I am just not aware.
>
> I guess some research is in order.
>
>> Also with libini there is a lot of control that we have to add features
>> on the pace we need.
>>
>>> Is it more work to complete libini, or add convenience capabilities
>>> to/on top of Augeas?
>>
>> libini has continence capabilities. It will be more work and AFAIU
>> feature degradation if we replace one with another but may be augeas is
>> more mature than I remember it to be.
>>
>>>
>>> Another factor is how many projects will benefit from SSSD devs
>>> working on libini vs. on Augeas.
>>
>> This is hard to say. We already did this evaluation once and said Augest
>> is not good enough for us so we will go with libini.
>> But I started this discussion to make a conscious choice moving forward.
>
> Obviously the choice is yours, I'm not even a SSSD developer. I'm just 
> providing some arguments to weigh.
>
>>>> The list above presents a spectrum of use cases and solution to those
>>>> use cases.
>>>> The goal of this email is to try to enumerate the use cases and
>>>> constraints and understand how the solution for those use cases can be
>>>> consolidated and unified and where it makes sense todo and where 
>>>> not. It
>>>> is a goal to define a reasonable alignment between the technologies 
>>>> and
>>>> provide a path forward to a reusable and maintanable stack.
>>>>
>>>> So we have the needs:
>>>>
>>>> 1) Read config files at run time from C (daemons and D-BUS provider)
>>>> 2) Write config files from C (configuration interface over LMI)
>>>> 3) Create and modify files at installation and upgade using some 
>>>> simple
>>>> mean/API which can be C, python or anything else
>>>> 4) Be able to manage configuration files in a object oriented manner
>>>> with or without D-BUS running
>>>> 5) Be able to augment the code file without touching it
>>>
>>> I don't understand this sentence, could you clarify?
>>
>> s/code/conf
>>
>>>
>>>> 6) Provide a  way to validate the file
>>>>
>>>>
>>>> So I see the following layering
>>>>
>>>> +------------------+
>>>> | High level tools |
>>>> +------------------+
>>>>       |
>>>>      \/
>>>> +---------------------------+
>>>> | Object oriented interface |
>>>> +---------------------------+
>>>>       |                    |
>>>>       \/                   \/
>>>> +-----------------+  +------------------+
>>>> | D-BUS responder |  | D-BUS shim layer |
>>>> +-----------------+  +------------------+
>>>>       |   --------------------|
>>>>       |   |
>>>>       \/ \/
>>>> +-------+              +----------------+
>>>> | C API |<-------------| File validator |
>>>> +-------+              +----------------+
>>>>     /\  \
>>>>      |   \               +--------------+
>>>>      |    -------------->| Config file  |
>>>>      |                   +--------------+
>>>>      |                        /\
>>>> +----------------+          |
>>>> | SSSD/GSS proxy |          |
>>>> +----------------+          |
>>>>                               |
>>>>                      +------------------+
>>>>                      | SSSD Coinfig API |
>>>>                      +------------------+
>>>>                              /\
>>>>                              |
>>>>                      +--------------+
>>>>                      | Simple tools |
>>>>                      +--------------+
>>>>
>>>>
>>>> This layout shows three distinct paths towards config file:
>>>> 1) Old python API and tools on top (does authconfig use this path?)
>>>> 2) Top level D-BUS api and advanced object oriented tools
>>>> 3) C API
>>>>
>>>>
>>>> We need to make several decisions at this point:
>>>> a) How much we enhance-update-improve existing python API instead of
>>>> using new D-BUS interface.
>>>> IMO we should leave python API alone for foreseeable future but any 
>>>> new
>>>> changes should go via the new D-BUS interface
>>>>
>>>> b) How we make sure that the D-BUS interface is really usable. It 
>>>> is not
>>>> that light weight.
>>>> Would be nice to be able to have a shim layer for the cases like 
>>>> docker
>>>> where there might not be D-BUS or systemd. This however might not be a
>>>> problem for us to solve. It might be a generic problem for all LMI 
>>>> style
>>>> providers. A lot in D-BUS world is autogenerated so I suggest that we
>>>> ask to be able to generate a mini D-BUS interface that would go 
>>>> directly
>>>> from the interface to the logic implemented in responder. I do not 
>>>> know
>>>> what is best. But I think it is a discussion to have with D-BUS 
>>>> guys and
>>>> should be more general.
>>>> For now we can assume that D-BUS is there and docker use case will be
>>>> solved in some generic way.
>>>>
>>>> c) If we assume D-BUS and systemd it really makes sense to separate 
>>>> the
>>>> responder in charge of configuration from other responders and make it
>>>> started on demand when there is a real call over the D-BUS to make a
>>>> config change. IMO this work for splitting ifp into data ifp and 
>>>> config
>>>> ifp should be filed as a ticket and done as a part of 1.13 release.
>>>>
>>>> d) What is going to be the low level C library? We have two candidates
>>>> libini and augeas.
>>>> I definitely have a preference towards libini but this does not mean I
>>>> am right. I do not have a lot of time to maintain libini. I do not 
>>>> see a
>>>> reason to rip it out and replace with augeas yet. But I also do not 
>>>> see
>>>> a reason why we should rely on augeas. Right now IMO it would be more
>>>> work to rip libini and switch to augeas but over time if we do more 
>>>> and
>>>> more augeas work it might be much less. I feel people are more
>>>> comfortable with augeas than using libini and adding to it as they 
>>>> go. I
>>>> do not know why but suspect it is because augeas has appeal of a more
>>>> commonly used tool/interface. It is all about what we think is the 
>>>> right
>>>> path. If we decide that augeas is the way to go then we should not
>>>> invest into enhancing libini and start building our improvements in
>>>> augeas or around augeas. If we think that libini is the right path
>>>> because it fill give us more features, flexibility and control then 
>>>> let
>>>> us make it so that I am not the only contributor to it and people 
>>>> would
>>>> not have problems throwing patches at it on as needed basis.
>>>> For me this is a very important decision. Right now I feel I am on 
>>>> hook
>>>> to continue investing my scares time into libini. I think I am holding
>>>> people back with this so I would rather come to terms and move forward
>>>> whatever we think is best for SSSD, GSS proxy and potential other
>>>> daemons that need to read and deal with config files.
>>>
>>> It seems the SSSD devs here in Red Hat's Brno office don't have the
>>> courage to say this, so let me convey the general idea floating around
>>> here: continuing with libini is a bad idea.
>>>
>>> We're in the business of collaboration, not writing our own little
>>> libraries where the main advantage is having control over the project.
>>
>> I would agree with you if we were at the decision point several years 
>> ago.
>> libini now is mature and can do what we need. It is stable.
>> Adding to it is simple and not much effort. I can prove it and turn back
>> pretty much all the code we need for it (except grammar validation) to
>> address writes, merging etc. But I can't support it. It is not
>> sustainable in a long run.
>
> That's the point.
> Augeas is sustainable in the long run, because it's, AFAICS, a healthy 
> community project.
> And anything useful built on top of Augeas has a better chance to get 
> a community. (Or be replaced with a more popular Augeas-based 
> implementation of the same feature, of course.) That's the way things go.
>
>> The grammar validation IMO should be a generic feature and can be
>> potentially another layer that processes config and check if it is OK.
>> It can be independent of augeas/libini but making it generic would be
>> more work than really needed so it should rather pick one and target it.
>
> If you want a generic feature, you'll need to base it on a tool that 
> other people use, not on a tool that only caters to the needs of two 
> projects. Otherwise the feature will never be generic, because it will 
> also only cater to those two projects. And if the two projects happen 
> to not have many outside contributions, neither will the feature.
>
> Of course, having a non-generic feature is possible, but you'll have 
> to write it and support the little tower of code forever. From the 
> ground up it'd be incompatible with what other people use, and there'd 
> be no integration at the higher levels because of the "making things 
> generic is more work" philosophy.
>
>> Now here is the situation - we have to pick one. I look at how much work
>> there is for me to address everything that needs to be addressed in
>> libini to meet the requirements of the stack above and I say days
>> (though I do not have those days). I look at augeas and I say weeks or
>> months. But this is my take because I know libini and invested into it.
>
> Well, you don't have the time, so ask the person who'll need to do the 
> changes for an opinion.
> Or actually don't ask the developer; developers' estimates always fall 
> short. Especially when they know the code :)
>
>> We already have libini. We already support it and it has a lot of useful
>> features. I am fine replacing it if we are sure that augeas would not
>> prevent us from doing what we are doing now. If that would require a
>> major refactoring and it would require us to contribute to augeas
>> significant chunks of code I do not think it is justified.
>>
>>
>>
>>> Contributing to Augeas will help many more people than playing in our
>>> own sandbox.
>>
>> True in general case. But we are not altruists here. Everything has cost
>> and merit.
>> So far I have not seen any real technical argument why augeas is a
>> better choice. I see a generic statement about general value to open
>> source that is not measurable. Let us be practical. Give me the value
>> and the cost and convince me that this is the right cost to have.
>
> Look, you're the manager here. Cost and benefit analysis is your turf; 
> I don't have the capability of measuring ideas by money.
>
> I am, however, an employee of a company that prides itself on 
> collaboration. I am used to working with others to solve common 
> problems. Augeas clearly solves many of the same problems libini does, 
> it's fairly popular and has some people who clearly know what they are 
> doing.
>
> We are not altruists. We are collaborators, because working with 
> others is better for us in the long run.
>
>> I can be convinced otherwise I would not have had this conversation but
>> not with the argument above.
>>
>>> And it'll also be better for us -- I don't think libini is getting
>>> Ruby bindings or a big tool ecosystem any time soon.
>>
>>
>> Do we need ruby bindings? Is it really a use case we care?
>
> Well, that's for the manager to decide. I can see two possible answers:
>
> * No, it is not.
> We don't care about anything that doesn't benefit us directly.
> This is exactly why there will never be a community around libini.
>
> * Yes, it is.
> There's a module for Puppet that uses these Ruby bindings to configure 
> stuff.
> AFAIK, neither Puppet nor Augeas authors needed to write the 
> integration. It came from the community.
> By joining the Augeas community, we'd get Puppet integration for SSSD 
> configs, right now.
> And later, we'd automatically get similar integration for whatever 
> tools are popular enough for someone to write a generic Augeas plugin 
> for.
>
>>> If there is anything missing in Augeas then we should extend Augeas,
>>> instead of trying to make another implementation.
>>>
>>>
>>
>> We already have implementation this is the point and the question is: is
>> it worth dropping it or not.
>> Convince me that it is worth.
>
>
>
>
> On 06/06/2014 05:47 PM, Dmitri Pal wrote:
>> So let me summarize pros and cons:
>>
>> Pros of libini
>> - already working and stable parser for INI files
>> - already integrated with SSSD and GSS proxy
>> - has advanced convenience features?
>> - can be turn around and delivered very fast because of control
>>
>> Cons of libini
>> - zero community
>> - INI files only
>> - main developer does not have time to work on it
>>
>> Pros of augeas
>> - has parser
>> - has community
>> - contributing to it will have general benefit
>>
>> Cons of augeas
>> - not integrated into SSSD and GSS proxy
>
> I'm sure you heard more than once that we need to integrate things, 
> rather than build separate stacks.
>
>> - does not have advanced features (to be verified)
>> - less control and thus slower turn around
>
> Yes, unfortunately, collaboration means having to do politics. It's 
> the way it works in open source, except maybe in our little island of 
> open-source projects with no community.
>
> Anyway, control is primarily based on the time you invest into the 
> project.
> If push comes to shove, you can always fork it, or carry your own 
> stack of patches for absolute control.
>
>> So when I look at it for me it makes perfect sense to continue 
>> investing into libini to address its cons, i.e. if we contribute to 
>> it we will foster a community for it.
>
> Umm, how good are we at fostering developer communities?
> Can you create a sustainable community around libini? Is it actually 
> useful for use cases we don't care about?
>
> If you want a *community* around the config parsing library we'll use, 
> I can tell you without doubt that porting to Augeas will be WAY easier 
> than attracting a community to libini.
>
> A healthy community would also mean that *we* would not really be in 
> control. One of libini's big advantages would be lost.
>
>> Then we already leverage the functionality we have, work for greater 
>> good, do not waste cycles and show leadership. It also allows us to 
>> be focused on the use cases we really care about.
>
>

This is a technical and a merit discussion so let us leave "manager" 
related sentiments out of it.
I am not against contributing to communities but I am against wasting 
anyone's time and doing unnecessary work.

Your note about Augeas's design non goals is exactly my point. libini 
was built with specific use cases and situations in mind. It fits SSSD 
and GSS proxy well.
It is a different tool with its advantages and disadvantage. You do not 
have just a hummer in your toolbox do you? There are different 
languages: java, ruby, python etc. Different languages are better for 
different use cases.
In the current situation I do not see why Augeas is better for SSSD and 
GSS proxy. I see why it is better for augeas if we stop doing anything 
with libini and start investing into augeas but I do not see a benefit 
of augeas to SSSD other then claims of a bigger community.
Regardless of the size of the community someone would need to be in 
charge of one or another. It really boils do to someone stepping up and 
saying: I will make augeas a viable replacement for libini for SSSD and 
GSS proxy or I will enhance libini for it to do what we need and keep up 
with demand. And I mean somone who would go extra mile to make things 
happen. Either would work for me.
But so far I do not see any takers.

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IdM portfolio
Red Hat, Inc.




More information about the sssd-devel mailing list