On Mon, 2 Apr 2012 14:37:20 -0700 (PDT) david@lang.hm wrote:
On Mon, 2 Apr 2012, Botond Botyanszki wrote:
On Mon, 2 Apr 2012 14:34:22 -0400 William Heinbockel wheinbockel@gmail.com wrote:
Before we dive too deep into which fields to support and how to name them, we should first discuss the structure and organization of the names.
I propose we roughly follow the proposal from Balazs:
- Core Fields -- flat names that are defined by lumberjack/CEE. These
should encompass the minimal set of fields that are commonly used.
- Extended Fields -- additional fields are either tied to a specific
product/application or a functional product group (anti-virus, firewalls, etc.)
We should focus on scoping and defining #1, probably combining the libumberlog, patterndb, liblognorm, RFC5424/IANA fields. Maybe the best place to start is with the `auditd` list http://people.redhat.com/sgrubb/audit/audit-parse.txt and abstract linux-specific names to be more generally applicable.
For #2, we need to figure out how these work. I vote that the structure should infer the namespace and the namespace should either be unique to the vendor/product or developed by the community. {"red_hat":{"auditd":{"time":"..."}}} would work, or collapsed to {"red_hat|auditd|time":"..."} (feel free to substitute divisive char: !/.:|)
The functional product categories should be related to an organizational entity -- lumberjack, cee, IANA, etc.
We need some way to distinguish vendor/product namespaces from organization/functional ones.
If you specify the name of the product in the field name then there is no standardization. The point is that you should be able to handle audit subsystem events generically. If we have {"red_hat|auditd|filename":"..."} and {"solaris|auditing|filename":"..."}, then it is really hard to use the "filename" field across all product/application groups. Also one event would never contain both as in the following: {"red_hat|auditd|filename":"...", "solaris|auditing|filename":"..."} So I fail to see where this schema is useful.
Instead, the event should contain the product/app name in a separate field, i.e.: {"SourceOS":"solaris", "SourceName":"auditing", "filename":"..."} and {"SourceOS":"red-hat", "SourceName":"auditd", "filename":"..."}
Agreed. In a different thread I tried to make a similar point about the field definitions.
For example, we have the severity concept, but that will be different for things that natively think syslog than for things that nativly think window (and I understand the cee standard is different from either)
the severity tag needs to be the standards-defined tag, even if the conversion from native to cee is lossy
but there needs to also be a native_severity tag and SourceOS tag so that if something cares about the full details of the priginator, the data is available (but not in a Unix_severity, Windows_severity, etc proliferation of tags)
Ok, this can work also. Though it is not SourceOS which will make the distinction because Syslog exists on various OSes for example. The SyslogSeverity would be then NativeSeverity and LogSourceName or similar.
Though I'm not really convinced that this is a good approach because there always are a couple other fields which are source/protocol specific. For example in syslog, there is also SyslogFacility. If we keep this as is, then it is confusing to have some fields with the source/protocol name incuded the field and yet others are named generically. Trying to map all these source/protocol specific fields to a generic name in our field list might not be the best idea because this will make it a lot harder for the user to identify and/or map back to the original format. Another reason for keeping SyslogSeverity is that this is easier to validate and can possibly have a custom type because the value space is known. I was trying to take a hybrid approach and map only the common fields to generic ones and leave the names of the rest (uncommon and very source/protocol specific) as is.
Regards, Botond