Hello, I have just published a 0.1 of ceelog/libceelog at https://fedorahosted.org/ceelog/ : Tarball at https://fedorahosted.org/releases/c/e/ceelog/ceelog-0.1.tar.xz git http://git.fedorahosted.org/cgit/ceelog.git/. A full README is appended below.
Comments (on project name, API design, command-line tool wishlist), testing, review, contributions of any kind are very welcome - on the list, in trac, to me privately, in any way you find convenient. Mirek
About ===== The ceelog project provides libceelog, a library for receiving, filtering and searching a stream or log of CEE/Lumberjack syslog records, and an associated command-line tool, named ceelog.
The goal is to abstract the user from the backend storage (files, some kind of local indexed storage, a remote database) and to provide efficient log processing tools that can be used in applications and scripts for automated log processing.
The project's home page is at https://fedorahosted.org/ceelog/ .
To get you started ================== The ceelog(1) tool reads the "default" event source and outputs events matching a filter. Currently, the "default" event source is hardcoded to /var/log/messages.
Example filters: * A regexp (matches the unstructured event text, or the "msg" field for CEE/Lumberjack structured events) ceelog '/DHCP/' * A field comparison (matches a CEE/Lumberjack field) ceelog 'uid == 0' ceelog 'uid != 0' ceelog 'trusted!uid == 0' ceelog 'username ~ /^guest-/' ceelog 'username !~ /^guest-/' * A combination of the above ceelog 'trusted!uid == 0 && username ~ /^guest-/'
See the source code in src/ceelog.c for an example of a subset of the API.
Roadmap ======= * Add "Live log file" input source that can handle messages being appended and log rotation. * Document the filter expression format. * Document ceelog(1). * Get as close to 100% test coverage as possible. * Support best-effort saving/restoring the current position in a source. * Add better support for JSON types. * Implement MongoDB input source. * Support searching directly in the input source (e.g. to evaluate the filter server-side). * ceelog(1) improvements: - Input processing (e.g. output only the last N recent events, block for more incoming events) - Output formatting (e.g. only output some structured fields) - Statistics/table output (group matching events by one field, output counts)
Bugs ==== Please consider reporting the bug to your distribution's bug tracking system.
Otherwise, please report bugs at https://fedorahosted.org/volume_key/ . Bug reports with patches are especially welcome.