On Mon, 2 Apr 2012 14:37:20 -0700 (PDT)
david(a)lang.hm wrote:
> On Mon, 2 Apr 2012, Botond Botyanszki wrote:
>
> > On Mon, 2 Apr 2012 14:34:22 -0400
> > William Heinbockel <wheinbockel(a)gmail.com> wrote:
> >
> >> Before we dive too deep into which fields to support and how to name
> >> them, we should first discuss the structure and organization of the
> >> names.
> >>
> >> I propose we roughly follow the proposal from Balazs:
> >>
> >> 1. Core Fields -- flat names that are defined by lumberjack/CEE. These
> >> should encompass the minimal set of fields that are commonly used.
> >>
> >> 2. Extended Fields -- additional fields are either tied to a specific
> >> product/application or a functional product group (anti-virus,
> >> firewalls, etc.)
> >>
> >>
> >> We should focus on scoping and defining #1, probably combining the
> >> libumberlog, patterndb, liblognorm, RFC5424/IANA fields. Maybe the
> >> best place to start is with the `auditd` list
> >> <
http://people.redhat.com/sgrubb/audit/audit-parse.txt> and abstract
> >> linux-specific names to be more generally applicable.
> >>
> >>
> >> For #2, we need to figure out how these work. I vote that the
> >> structure should infer the namespace and the namespace should either
> >> be unique to the vendor/product or developed by the community.
> >>
{"red_hat":{"auditd":{"time":"..."}}} would work,
or collapsed to
> >> {"red_hat|auditd|time":"..."}
> >> (feel free to substitute divisive char: !/.:|)
> >>
> >> The functional product categories should be related to an
> >> organizational entity -- lumberjack, cee, IANA, etc.
> >>
> >> We need some way to distinguish vendor/product namespaces from
> >> organization/functional ones.
> >
> > If you specify the name of the product in the field name then there is no
> > standardization.
> > The point is that you should be able to handle audit subsystem events
> > generically. If we have {"red_hat|auditd|filename":"..."}
and
> > {"solaris|auditing|filename":"..."}, then it is really hard
to use the
> > "filename" field across all product/application groups.
> > Also one event would never contain both as in the following:
> > {"red_hat|auditd|filename":"...",
"solaris|auditing|filename":"..."}
> > So I fail to see where this schema is useful.
> >
> > Instead, the event should contain the product/app name in a separate
> > field, i.e.:
> > {"SourceOS":"solaris",
"SourceName":"auditing", "filename":"..."}
> > and
> > {"SourceOS":"red-hat",
"SourceName":"auditd", "filename":"..."}
>
> Agreed. In a different thread I tried to make a similar point about the
> field definitions.
>
> For example, we have the severity concept, but that will be different for
> things that natively think syslog than for things that nativly think
> window (and I understand the cee standard is different from either)
>
> the severity tag needs to be the standards-defined tag, even if the
> conversion from native to cee is lossy
>
> but there needs to also be a native_severity tag and SourceOS tag so that
> if something cares about the full details of the priginator, the data is
> available (but not in a Unix_severity, Windows_severity, etc proliferation
> of tags)
Ok, this can work also. Though it is not SourceOS which will make the
distinction because Syslog exists on various OSes for example. The
SyslogSeverity would be then NativeSeverity and LogSourceName or
similar.
Though I'm not really convinced that this is a good approach because
there always are a couple other fields which are source/protocol
specific. For example in syslog, there is also SyslogFacility. If we keep
this as is, then it is confusing to have some fields with the
source/protocol name incuded the field and yet others are named
generically.
Trying to map all these source/protocol specific fields to a generic name
in our field list might not be the best idea because this will make it a
lot harder for the user to identify and/or map back to the original
format. Another reason for keeping SyslogSeverity is that this is easier
to validate and can possibly have a custom type because the value space
is known.
I was trying to take a hybrid approach and map only the common fields to
generic ones and leave the names of the rest (uncommon and very
source/protocol specific) as is.
I think this is the best approach.
Make things generic when the concept can be generalized, use more
specific names when it cannot.
That is what I was trying to get at with the syslog priority/facility
discussion in the other thread. These values only make sense within
the context of syslog, therefore they should be named as such.
What I am cautious of is going too far down these routes as a project;
it is very easy to start adding the fields/data from all of the
individual protocols.
What we should focus on is the most commonly used fields -- users,
files, src/dst, time, etc.