[SSSD] New version of the libini_config section merge design

Dmitri Pal dpal at redhat.com
Tue Jun 3 16:22:28 UTC 2014


On 06/02/2014 09:00 PM, Dmitri Pal wrote:
> On 06/02/2014 09:37 AM, Nikolai Kondrashov wrote:
>> On 05/31/2014 04:45 AM, Dmitri Pal wrote:
>>> I reworked the design based on the feedback provided.
>>> It is a completely rewritten version.
>>> Please take a look.
>>> https://fedorahosted.org/sssd/wiki/DesignDocs/ding-libs/INIConfigMerge
>>
>> I'm likely awfully under-informed (I haven't seen any prior 
>> discussion or
>> requirements, never used the library), but I'd like to give my 
>> opinion anyway,
>> if only as an exercise.
>>
>> I wouldn't like to steal much of anyone's time so please ignore me if 
>> I'm
>> entirely missing the point. I've likely missed some requirements and 
>> the below
>> needs details figured out, but I think this general approach would be 
>> better
>> for both end-users and developers.
>>
>> First of all, the proposal sounds terribly complicated. 
>
> Hm. I think it is significantly simplified in comparison to the 
> previous version.
> The function is not SSSD specific. It is a part of the generic 
> library, it is up to application developer to use different aspects of 
> it.
> I would rather implement a complex function and have a simplified use 
> of it than have a simplified function and then several month later 
> realize that we need to have more flexibility and a break ABI or have 
> function augment2, augment3 etc - ugly.
>
>> Both in merging logic
>> and in permission requirements. What's worse, this complexity is 
>> hidden from
>> system administrators (users) in the code. It means that all the 
>> parameters
>> passed to "ini_config_augment" will need to be documented somewhere.
>
> The function is for developers not for admins. It is a tool. It is up 
> to the developer to use it in such a way that admin can understand.
>
>>
>> As in: "you can put these and these sections in these files and the 
>> files will
>> need to have these and these permissions and be owned by such and 
>> such user
>> and group, etc.". 
>
> This is an application choice. SSSD might not required to use it (at 
> first) up until it realizes that overwriting some sections is not a 
> good idea.
>
>
>> This will be needed for every application using this feature
>> and it will need to be updated with the code. And all hell will break 
>> loose
>> when someone decides to adjust ini_config_augment arguments based on
>> previously read keys/sections. Just because it can be done, you know.
>
> I do not think it is a valid concern. You can shoot yourself into the 
> head in so many different ways.
>
>
>>
>> Assigning special treatment to "augmenting" configuration, and adding 
>> ability
>> to ignore erros there specifically, also doesn't seem right. Given the
>> overwriting feature is provided, the *whole* of the actual 
>> configuration can
>> be placed there, and thus it seems better to give equal treatment 
>> both to the
>> "main" and to the "augmented" configuration. I.e. either provide a 
>> way to
>> ignore/enforce errors in both or in none.
>
> I disagree. Overwriting is much more dangerous than augmenting.
>
>
>>
>> From my, once again, entirely uninformed view as a potential user and an
>> administrator I would have liked to see the following instead.
>>
>> Allow both sections and keys appear more than once in a configuration 
>> file.
>
>
> Ini library support this but it is application's choice to allow it or 
> not.
> SSSD does not allow it while GSS proxy does.
>
>> Repeated keys would overwrite previous value. New keys in repeated 
>> sections
>> would add keys. Missing keys in repeated sections wouldn't change 
>> anything.
>> I assume this is the way it works currently, as some QA tests use it.
>
> You are not testing the library itself. SSSD reads the config with a 
> special flag indicating what you described but there are all sorts of 
> different modes that can be specified that would treat duplicates as 
> errors or preserve the original value.
> Again library provides flexibility, application decides on the best 
> option to use.
>
>>
>> Adjust the configuration language to add syntax to remove sections 
>> and keys.
>
> NO. No language. That adds much more complexity then needed.
>
>> Something which can be understood with little or no documentation. For
>> example:
>>
>>     [section_a]
>>     key_a = 1
>>     key_b = 2
>>
>>     [section_b]
>>     key_c = 3
>>
>>     ; Add "key_d" to "section_b"
>>     [section_b]
>>     key_d = 4
>>
>>     ; Remove "section_a"
>>     -[section_a]
>>     ; Start "section_a" anew
>>     [section_a]
>>     key_d = 42
>>
>>     ; Remove "key_c" from "section_b" and overwrite "key_c"
>>     [section_b]
>>     -key_d
>>     key_c = 2
>>
>> Add configuration language syntax to include files/directories of files,
>> optionally specifying file permission requirements and section 
>> modification
>> permissions. With this amount of features the need for documentation 
>> to read
>> them would be inevitable, but we can at least try:
>>
>>     [section_a]
>>     key_a = 1
>>     key_b = 2
>>
>>     [section_b]
>>     key_c = 3
>>
>>     ; Include an application configuration file, only allowed to add 
>> some
>>     ; sections
>>     <application.conf add section_c_*,
>>                       user application,
>>                       group application,
>>                       mode 640>
>>
>>     ; Include a subdirectory of configuration files, allowed to add any
>>     ; and modify/remove some sections
>>     <service.d add *,
>>                modify section_a_*,
>>                user root,
>>                group service,
>>                mode 660>
>>
>>     ; Include another configuration file keeping the permissions intact
>>     <additional.conf>
>
> I am strongly apposed any kind of the include directives.
> Having include directives means someone needs to insert them.
> The use case we are trying to solve is to be able to augment config 
> files without changing the main config file.
>
>>
>> Included files would be processed as if read at the spot of the 
>> directive
>> without any other requirements.
>
> This is a significant rewrite of the parser because of the language of 
> the INI file change. Now any config file is a config file. As soon as 
> you introduce language you have a "proper" ini file and one that needs 
> to be merged and processed. It is just way more complex than suggested.
>
>> I.e. they can continue existing or start new
>> sections. The usual ordering of files in directories would apply. Nested
>> includes should be possible allowing applications to sub-delegate 
>> some of
>> their configuration. Includes should not be allowed to relax section
>> addition/modification permissions, only to make them stricter. File 
>> permission
>> requirements should probably be changeable, though.
>
> The ideas is to avoid the includes and have any config file be a valid 
> config file.
>
>>
>> The support for the extended syntax may be configurable via the API.
>
> Which API?
>
>>
>> This way there might be no need to add another function, although a 
>> feature
>> for merging many files from the code would be useful.
>
> There is already a way to merge snippets in the library. The function 
> is proposed to have a one tool instead of implementing the same 
> functionality in different places.
>
>
>>
>> This way everything is visible to and controllable by the 
>> administrator and
>> doesn't need application-specific documentation, only the configuration
>> language documentation, which may be borrowed from the library and would
>> rarely change. More complicated include statements would need some 
>> comments,
>> but at least those would be close to the actual logic.
>
> If someone want to implement it be my guest. Having spent some time in 
> the area I would say it is power of magnitude more complex than the 
> original "more complex" design that was proposed (you can read it in 
> the page history).
>
>
>>
>> Nick
>
>

I have read your suggestion again and realized that we might be talking 
about different use cases.

1) Admin changing the config file is not the use case.
If admin wants to change the file he just modifies it.
If he needs to change it remotely he would use OpenLMI to so see what is 
there and tweak things. Pavel already did that work. I think it is 
equivalent to the "language" you proposed but much more object oriented.
Admin also can use puppet if he wants because he has the control.

2) Here we are talking about the cases that will be highly automated by 
deployment and provisioning systems.
There are different parts of the stack that need to work together. Some 
part of the stack installs the client, some other package might bring 
some overwride because of the specific purpose of the particular server, 
the virtualization/containerization can require some new sections to be 
dropped in and/or removed dynamically. Merging these portions that 
different parts of the stack deliver would be very valuable. This is 
what the functionality is targeting.

3) What I also realized is that I should probably allow passing in a 
logging callback.
Since the function is complex it would be really nice to have a track 
record of the steps it took and the results that have been accomplished, 
not just errors.
Would that make sense?

-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IdM portfolio
Red Hat, Inc.




More information about the sssd-devel mailing list