[lumberjack] New Syntax Spec and updated Field list

Balazs Scheidler bazsi77 at gmail.com
Wed May 2 21:04:53 UTC 2012


----- Original message -----
> On 05/02/2012 04:18 PM, david at lang.hm wrote:
> > On Wed, 2 May 2012, Gergely Nagy wrote:
> > 
> > > William Heinbockel <wheinbockel at gmail.com> writes:
> > > 
> > > > What I did was take the JSON field types and add datetime, ipv4,
> > > > and ipv6 types.
> > > > Then made a basic conversion from JSON --> XML and tried to align
> > > > the libumberlog syntaxes with Keith's XML.
> > > > I also added some notes about the "structured" vs. "inline" field
> > > > naming.
> > > 
> > > Sorry for the late reply, I've been swamped as of late.
> > > Nevertheless, my former opinion that the inline representation is a
> > > very, very bad thing still stands:
> > > 
> > > First of all, JSON is perfectly capable of representing structured
> > > data (and the syntax format already describes that too!), bolting
> > > something like the inline representation on it is both
> > > counter-intuitive for those who worked with JSON, it's incompatible
> > > with anything that is not explicitly aware of it: you can't just
> > > forward it to a database that happens to accept JSON as-is, you'll
> > > have to translate the inline representation into proper JSON
> > > structure. (For example, if I had a producer that spew out CEE-JSON
> > > on one end, and an ElasticSearch indexer on the other end, I'd need
> > > to convert inline-syntax, which would pretty much destroy the whole
> > > idea of having a thin layer.)
> > > 
> > > It adds quite a bit of burden on the implementation side, and I still
> > > can't see what benefit it would bring, apart from saving a few bytes
> > > on the wire. I'm not entirely sure it would save bytes, either, by
> > > the way.. For example:
> > > 
> > > {"foo!field1":1,"foo!field2":2,"foo!field3":3} vs
> > > {"foo":{"field1":1,"field2":2,"field3":3}}
> > > 
> > > Proper structuring is shorteer pretty much the moment you have more
> > > than two fields with the same 'object' prefix.
> > > 
> > > Furthermore, as mentioned above already, anything that speaks JSON,
> > > understands the structures. None of them understand the inline
> > > syntax. It's alien to JSON as much as trying to bolt on new types
> > > were.
> > > 
> > > It complicates both the sender side (do I want to send stuff as
> > > proper JSON, or with inline representation?), it complicates things
> > > on the consumer side too (do I need to expand inline syntax? if so,
> > > it will likely turn into dotted-notation, and then how do I
> > > reassemble it back if it needs to be forwarded?), and adds extra
> > > work on every level.
> > 
> > I think that the only good reason for doing having the inline stuff is
> > for a template based API. Over the wire it would be far better for
> > everything to be nested JSON so that all recipients only need to know
> > JSON.
> > 
> > But when creating an API to generate the logs, figuring out how to
> > specify structure is non-trivial and I can see the desire to be able
> > to specify the fully qualified name of a field.
> 
> I see no problem in having "foo!field1" at the API layer and proposed
> nested structure below and on the wire. Seems very reasonable.
> 

cool, we have a conclusion then?

I'd recommend to use json dotted notation for fields though. It's more standard than the exclamation mark stuff. but this is certainly an API territory.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://fedorahosted.org/pipermail/lumberjack-developers/attachments/20120502/33d20320/attachment.html>


More information about the lumberjack-developers mailing list