[lumberjack] New Syntax Spec and updated Field list

Dmitri Pal dpal at redhat.com
Wed May 2 20:26:36 UTC 2012


On 05/02/2012 04:18 PM, david at lang.hm wrote:
> On Wed, 2 May 2012, Gergely Nagy wrote:
>
>> William Heinbockel <wheinbockel at gmail.com> writes:
>>
>>> What I did was take the JSON field types and add datetime, ipv4, and
>>> ipv6 types.
>>> Then made a basic conversion from JSON --> XML and tried to align the
>>> libumberlog syntaxes with Keith's XML.
>>> I also added some notes about the "structured" vs. "inline" field
>>> naming.
>>
>> Sorry for the late reply, I've been swamped as of late. Nevertheless, my
>> former opinion that the inline representation is a very, very bad thing
>> still stands:
>>
>> First of all, JSON is perfectly capable of representing structured data
>> (and the syntax format already describes that too!), bolting something
>> like the inline representation on it is both counter-intuitive for those
>> who worked with JSON, it's incompatible with anything that is not
>> explicitly aware of it: you can't just forward it to a database that
>> happens to accept JSON as-is, you'll have to translate the inline
>> representation into proper JSON structure. (For example, if I had a
>> producer that spew out CEE-JSON on one end, and an ElasticSearch indexer
>> on the other end, I'd need to convert inline-syntax, which would pretty
>> much destroy the whole idea of having a thin layer.)
>>
>> It adds quite a bit of burden on the implementation side, and I still
>> can't see what benefit it would bring, apart from saving a few bytes on
>> the wire. I'm not entirely sure it would save bytes, either, by the
>> way.. For example:
>>
>> {"foo!field1":1,"foo!field2":2,"foo!field3":3} vs
>> {"foo":{"field1":1,"field2":2,"field3":3}}
>>
>> Proper structuring is shorteer pretty much the moment you have more than
>> two fields with the same 'object' prefix.
>>
>> Furthermore, as mentioned above already, anything that speaks JSON,
>> understands the structures. None of them understand the inline
>> syntax. It's alien to JSON as much as trying to bolt on new types were.
>>
>> It complicates both the sender side (do I want to send stuff as proper
>> JSON, or with inline representation?), it complicates things on the
>> consumer side too (do I need to expand inline syntax? if so, it will
>> likely turn into dotted-notation, and then how do I reassemble it back
>> if it needs to be forwarded?), and adds extra work on every level.
>
> I think that the only good reason for doing having the inline stuff is
> for a template based API. Over the wire it would be far better for
> everything to be nested JSON so that all recipients only need to know
> JSON.
>
> But when creating an API to generate the logs, figuring out how to
> specify structure is non-trivial and I can see the desire to be able
> to specify the fully qualified name of a field.

I see no problem in having "foo!field1" at the API layer and proposed
nested structure below and on the wire. Seems very reasonable.

>
> David Lang
> _______________________________________________
> lumberjack-developers mailing list
> lumberjack-developers at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/lumberjack-developers


-- 
Thank you,
Dmitri Pal

Sr. Engineering Manager IPA project,
Red Hat Inc.


-------------------------------
Looking to carve out IT costs?
www.redhat.com/carveoutcosts/





More information about the lumberjack-developers mailing list