On 04/08/2014 11:23 AM, Dmitri Pal wrote:
> On 04/08/2014 10:32 AM, Simo Sorce wrote:
>> On Tue, 2014-04-08 at 09:54 -0400, Dmitri Pal wrote:
>>> On 04/07/2014 10:13 PM, Simo Sorce wrote:
>>>> On Mon, 2014-04-07 at 19:20 -0400, Dmitri Pal wrote:
>>>>> Hello,
>>>>>
>>>>> This is the continuation of the discussion of the following
>>>>> design page:
>>>>>
https://fedorahosted.org/sssd/wiki/DesignDocs/ding-libs/INIConfigMerge
>>>>>
>>>>>
>>>>> I see three controversial items in this design based on the previous
>>>>> discussion:
>>>>>
>>>>> a) Merging values
>>>>> This horse has been beaten in the other thread.
>>>>> The answer is simple. If we can make SSSD to make a change to not
>>>>> use
>>>>> lists or domains or providers as Simo suggest we can remove this
>>>>> consideration from the merge design. If we can't then we need to
>>>>> make it
>>>>> optionally available in the merge logic. To make a final decision
>>>>> here
>>>>> we need Jakub's input on how hard it is to implement Simo's
idea.
>>>>>
>>>>> c) Snippets in a directory vs. snippets in a subdirectory per
>>>>> application.
>>>>> The design is built around the fact that it is better to put
>>>>> snippets
>>>>> into different subdirectories per source/application that updated
>>>>> them
>>>>> so that we can make sure that different sources of config info do
>>>>> not
>>>>> step on each other. We also can jail the config files with
>>>>> SELinux and
>>>>> allow different access to different subdirectories.
>>>> Keep in mind that selinux labels can be applied to single files,
>>>> they do
>>>> not need to be in subdirectories.
>>>>
>>>>> This is a deviation
>>>>> from the current approaches where all snippets are supposed to go
>>>>> into a
>>>>> single directory.
>>>>> IMO this is quite limiting and I see an opportunity to change it for
>>>>> better. However if people agree with Simo that this is too over
>>>>> engineered and complicated I can understand. It is in fact more
>>>>> complex.
>>>>> So chime in. If you think single directory is good enough so be it.
>>>>>
>>>>> d) .merge files
>>>>> I do not like them either.
>>>>> If we:
>>>>> a) Can avoid value merges
>>>>> b) Will require each snippet to have a section header ( the
>>>>> problem is
>>>>> really the fact that INI automatically generates one called
>>>>> "default" if
>>>>> there is none so it is hard to say whether the parse filled the
>>>>> blank or
>>>>> the snippet section really is called "[default]"). I think
I can
>>>>> work
>>>>> around it but it would require a change that I wanted to avoid
>>>>> with the
>>>>> design.
>>>>> c) Do not allow overwrites
>>>>> we can drop the .merge files.
>>>>>
>>>>> They are needed to support any of the three use cases above.
>>>>> I am mostly concerned about the last one because allowing
>>>>> overwrites on
>>>>> per section basis was something was requested in one of the RFEs.
>>>>> However here is how we can work around this because IMO it is a
>>>>> corner case.
>>>>>
>>>>> - The merge code will read snippets from a single directory.
>>>>> - It will inspect the name. If the name is <xxx>.over.conf then
>>>>> it will
>>>>> treat this file as the snippet that the author of the snippet
>>>>> wants to
>>>>> use to overwrite sections in the original config file. If the
>>>>> name of
>>>>> the file is <yyyy>.conf it will user it to do additive merge.
If
>>>>> file
>>>>> has any other extension it will be ignored.
>>>> This is not a bad idea, I need to think about it.
>>> Sure. I am open to alternatives.
>>> I do not like naming conventions that much but if we have to do it
>>> here
>>> so be it.
>> The other option is to add a directive within the file.
>> Like the first key must be merge = yes or override = yes without
>> section ?
>
> Or better:
>
> mode = overwrite or mode = merge
>
> with
>
> mode = merge
>
> being default if not there.
>
> So it 80% of the cases there will be no extra keys but if you want
> overwrite then you specify mode = overwrite
After some more thinking I would prefer the file naming convention.
The reason is that if the parse will create a section called "default"
for the KVPs without section.
That would mean that if there is a section in the real file called
"default" and I would want to overwrite it I will create a file
mode = over
[default]
k1=v1
k2=v2
The parser will read and see this as a dup section which will violate
the roles we are defining.
I am not sure I want to change the parser and its handling of the KVPs
outside the section. That would be a too much of a change.
File naming convention works around this issue nicely.
Also less processing and validation inside the file.
Any other feedback or I will rewrite the page then?
>
>>
>>>> Another option I can see is that he application tells you (via
>>>> library
>>>> call) what sections are overridable, but that is quite limiting
>>>> and may
>>>> not satisfy the original requester.
>>>>
>>>>> Common rules about snippets:
>>>>> - Snippets must have a section defined - absence will cause an
>>>>> error and
>>>>> snippet will be ignored
>>>> +1
>>>>
>>>>> - Snippets can't have the same section defined in more than one
>>>>> .conf
>>>>> file. If same name is detected it will be treated as a collision
>>>>> and all
>>>>> snippets will be ignored
>>>> I would rather say, only the first will be considered and all
>>>> conflicting one after that ignored.
>>> Which one is the first?
>>> It becomes implementation specific and depends on either dates or
>>> lexical order of the file names.
>> A lot of programs depend on ordering, so we need to explicitly know how
>> INI will order snippets.
>>
>> It is conventional to do it in lexicographic order.
>
> It can be done this way. The question is this the right way?
> I am just not sure.
> I envision a use case:
> You have some application dropping a snippet with name X while there
> is another application dropping a snippet with name Y. They are not
> aware of each other so they can't pick the names to define the right
> order in advance. You do not want admin involvement too because this
> would introduce manual step. The result is a bit unpredictable.
>
> Not sure how it can avoided though.
>
>
>>
>> SSSD also scans domains in order in the list (when more than one domain
>> is configured to not use fully qualified names, that is important), so
>> we need to be able to tell admin what the order will be.
>>
>>> I think this is not predictable and would be hard for admins to
>>> troubleshoot. I also think that just discarding the two colliding
>>> snippets would be a mistake since there can be a third one that then
>>> might go through because other two have been discarded.
>>> IMO any collision should be treated as a violation of the rules and
>>> abort the whole snippet stack.
>> We need an order (just decide on a rule that stay predictable over
>> multiple runs doesn't really matter which) and then discard anything
>> but
>> the first instance where the section appears :)
>
>
> This is OK too.
> So we boil down to the question of order.
>
> Any other bright ideas except lexical order?
>> Simo.
>>
>
>
OK since there are no other comments.
I will redo the page with what we have here and will start prototyping
if I have time over the weekend.
--
Thank you,
Dmitri Pal
Sr. Engineering Manager IdM portfolio
Red Hat, Inc.