[SSSD] Multi-line values in INI

Dmitri Pal dpal at redhat.com
Wed Apr 14 04:05:07 UTC 2010


[snip]
> None of the complexity discussed above is necessary if you support
> quoted strings. A double quote introduces a quoted string. You read
> until you find the closing quote (skipping any escaped quotes, e.g.
> \"). The entire quoted string (including the leading and trailing
> quotes) are preserved as part of the attribute's value. Every
> character is captured verbatim (including the delimiting quotes).
>
> Thus in the case of PEM data the application would only need to strip
> the leading and trailing quotes and it would have the exact textual
> data needed. This would also allow for easy cut-n-paste.
>
> The leading and trailing quotes need to be preserved so that later
> when the attribute values read from the ini file need to be
> interpreted by the application it can see the quoted string and
> optionally decide how to parse the string (possibly escaping backslash
> sequences).
>
> Thus for example:
>
> [silly section]
> my_attr = 1 "
> some text
> " true
>
> would see the value of my_attr exactly as written above, it would
> likely parse that into 3 tokens (integer, string, boolean), but
> tokenizing would be done by the application, not the ini parser.
>
I had a long discussion with John about this at the end of the day.
I think that we managed to come to some other consensus.
If I got it right the rules of thumb are the following:
a) Do not invent new meaning for the  characters that are already used
to have some other special meaning. Be consistent with use and meaning
of the characters.
b) Do not invent new syntax. Pick one of the exiting ones.
c) It is Ok to define rules about the syntax of the INI file values as
long as they do not contradict the first two rules
d) It is Ok to not support something if it is commented and declared in
the documentation (with intent or without intent to support it in future)

I think I captured our discussion reasonably well.
Now about John's proposal. I like it mostly with couple tweaks.
 
Here is what I suggest should be implemented (note: <NL> is a new line
character read from the file at the end of the line):
1) \<NL> should be special sequence that is always used to indicate line
wrapping. All \<NL> sequences are processed at the time the data is read
from the file.
Example:

my_val = Many words \
do not fit on one line!

Will be read into the memory and will be stored in the memory of the
config object as  ->Many words do not fit on one line!<-
This is the syntax that people use in make files and in C and expect the
\<NL> to be thrown away and sequential strings concatenated.
The following value

my_val = This value has \ in the \
middle of the line.

Will be read into the memory and will be stored in the memory of the
config object as  ->This value has \ in the middle of the line.<-

2) If someone wants to preserve the formatting i.e. spaces tabs or new
lines he should (as it was suggested by John) put the value into the
double quotes. But I had a dilemma. To read the quoted string verbatim
and interpret when the application asks for the value or do
preprocessing of the value right at the moment the data is read from the
file. After long mental exercise and comparing pros and cons of
different approaches I would think that the following logic would be best:
a) Use the quotes to define a sequence of the characters that needs to
be read verbatim. This sequence can cross multiple lines.
b) The quotes inside this sequence should be escaped with the back slash
c) \<NL> substitution rule ignores works inside the quoted string too
d) Non escaped <NL>s are copied verbatim
e) No support for any other escape sequence as \n \r \t \\ at least at
the moment.

Example:

my_val = " Preserve \"leading\" spaces
and embed new lines in
the middle of the line."

This will be read and stored in the memory verbatim:
->" Preserve \"leading\" spaces<NL>and embed new lines in<NL>the middle
of the line."<-

When the interpreting function is asked to translate it as an "escaped"
string
the following string will be returned to the application:
->  Preserve "leading" spaces<NL>and embed new lines in<NL>the middle of
the line.<-

Here is a more interesting example:

my_val = First split \
sentence, " \"Second\" \
split
 sentence" , Th" i "rd \
split \
sentence

This whole sequence will be interpreted as one value and will be stored
in the memory as:
->First split sentence, " \"Second\" split<NL> sentence" , Th" i "rd
split sentence<-

When the application would ask the INI interface to give it the value as
one escaped string
the result will be:
->First split sentence,  "Second" split<NL> sentence , Th i rd split
sentence<-
(note that there are two spaces after the first comma)

If instead application would try to interpret the value as an array
using comma as a separator
the result will consist of the array of the tree strings:
->First split sentence<-
-> "Second" split<NL> sentence<-   (note a leading space)
->Th i rd split sentence<-

I think this is sufficient functionality at the moment.
In future we might decide that we will support other escape characters.
To preserve backward future compatibility the interpreting functions
would have a special flag that would indicate the level of the escaping
the application is expecting the library to support.
Let me illustrate it on the example.
Say that we implement the rules described above (this is version or level 1)
but then later add capability to substitute \n (this is version or level 2).

Say the config file contained the following line:

my_val = Line with \n in the middle.

In version 1 the \n is ignored.
In version 2 we can substitute \n with <NL>.
But if we just update the library under application from version 1 to
version 2
and start interpreting \n ourselves that would not be what the
application would want.

So application will pass a flag that would indicate the level of the
parsing it expects.
So if the application was built using v1 escaping rules it would pass v1
flag to
the interpretation function: get_esc_str_value(..., USE_V1_ESCAPING);
Then if the library is updated to later version the application would
not be
affected by the new escaping rules until application actually
decides to support it. In this case a new version would use other, later,
flag and would migrate its config values to a new format.

But IMO I do not think the INI should be in the business of dealing with
escape symbols beyond what is currently proposed.
However the approach allows gradual future extension without loosing
backward compatibility.

And by the way no it is not too complex.
The complex thing is one you do not know how to do.
This one is not trivial but at least I understand how it can be
implemented.

-- 
Thank you,
Dmitri Pal

Engineering Manager IPA project,
Red Hat Inc.


-------------------------------
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/




More information about the sssd-devel mailing list