On 01/13/2011 01:25 PM, Denys Vlasenko wrote:
On Fri, 2011-01-07 at 15:29 +0100, Jiri Moskovcak wrote:
> = ABRT 2.0 Features (what we need) =
> == need to be done features ==
> plugins free daemon
> -> add a generic log scanner
I would like to work on this one. Here's my
"How to implement log scanning?" brainstorm.
Proposal (1): daemon watches log files, whenever one of them changes,
it runs a program which parses the log file. It can give the offset
in the log from where parsing should start.
Table of files to watch and helpers to run should be somewhere in config
file:
# Specialized log watcher programs
/var/log/messages abrt-scan-oops -f $FNAME -o $OFF
/var/log/Xorg.0.log abrt-scan-xserver -f $FNAME -o $OFF
# generic log watcher program
/some/other.log abrt-scan-log -f $FNAME -o $OFF -start-regex='ALERT'
-end-regex='END' -maxlines=50
...
I see one problem with this setup.
We need to consider a case when watched logfile gets flooded
(growing continuously and quickly). This means daemon
will be starting the parser all the time, and will have
hard time determining where exactly parser stopped parsing
last time: say, /var/log/messages was 100M long. We started parser.
Parser finished. We look at /var/log/messages and it's 110M long now.
How can we be sure that parser read until the very end of 110th meg,
that it did not hit EOF at, say, 109M? IOW: where to restart
next parsing?
- parsers can return some status message saying where they stopped
- btw, what happens now if we parse /var/log/messages, find the oops,
but the oops is not fully written there - the line ---- end ----- is not
there?
Seems like this problem can be solved if we keep an open fd
to the log file in question, and use that fd to read the file.
This way, we always know where we stopped.
- smells like leaking fd, which selinux won't like...
Smaller problems of proposal (1) are increasing complexity
in the daemon - it needs to have many (and of additional kind)
inotify watches, needs to suspend watching while parser child
is running, and may need some mechanism for adding/deleting
new log files to watch.
- or we can use the current logic which parser logs every X seconds and
checks if the file was changed since the last check (but I like the
inotify watches better :))
This leads me to proposal (2): every log file is watched by
a separate "log watching daemon". Basically, we just start:
# Specialized log watcher programs
abrt-scan-oops -f /var/log/messages&
abrt-scan-xserver -f /var/log/Xorg.0.log&
# generic log watcher program
abrt-scan-log -f /some/other.log -start-regex='ALERT' -end-regex='END'
-maxlines=50&
...
in background, and every one of these watches and processes
one log file.
- here we would also need some way of adding/removing the scanner which
is the same problem as adding/removing the log files to scan..
- one solution might be to turn all scanners into a separate services
(with their own init files) but now when the trend is to lower the
number of services started on boot this doesn't sound as a good idea..
Which proposal do you like more? Do you have other ideas
or see other problems to watch for?
-don't have a strong opinion here, but I like the (1) a bit more,
because there is only one long running process involved...