On Thu, 2011-01-13 at 14:46 +0100, Jiri Moskovcak wrote:
On 01/13/2011 01:25 PM, Denys Vlasenko wrote:
> On Fri, 2011-01-07 at 15:29 +0100, Jiri Moskovcak wrote:
>> = ABRT 2.0 Features (what we need) =
>> == need to be done features ==
>> plugins free daemon
>> -> add a generic log scanner
>
> I would like to work on this one. Here's my
> "How to implement log scanning?" brainstorm.
>
>
> Proposal (1): daemon watches log files, whenever one of them changes,
> it runs a program which parses the log file. It can give the offset
> in the log from where parsing should start.
> Table of files to watch and helpers to run should be somewhere in config
> file:
>
> # Specialized log watcher programs
> /var/log/messages abrt-scan-oops -f $FNAME -o $OFF
> /var/log/Xorg.0.log abrt-scan-xserver -f $FNAME -o $OFF
> # generic log watcher program
> /some/other.log abrt-scan-log -f $FNAME -o $OFF -start-regex='ALERT'
-end-regex='END' -maxlines=50
> ...
>
> I see one problem with this setup.
> We need to consider a case when watched logfile gets flooded
> (growing continuously and quickly). This means daemon
> will be starting the parser all the time, and will have
> hard time determining where exactly parser stopped parsing
> last time: say, /var/log/messages was 100M long. We started parser.
> Parser finished. We look at /var/log/messages and it's 110M long now.
> How can we be sure that parser read until the very end of 110th meg,
> that it did not hit EOF at, say, 109M? IOW: where to restart
> next parsing?
>
- parsers can return some status message saying where they stopped
- btw, what happens now if we parse /var/log/messages, find the oops,
but the oops is not fully written there - the line ---- end ----- is not
there?
We drop it if it's too short. If it isn't, we submit truncated one.
> Seems like this problem can be solved if we keep an open fd
> to the log file in question, and use that fd to read the file.
> This way, we always know where we stopped.
- smells like leaking fd, which selinux won't like...
Not if the fd isn't in daemon but in log watcher process.
> This leads me to proposal (2): every log file is watched by
> a separate "log watching daemon". Basically, we just start:
>
> # Specialized log watcher programs
> abrt-scan-oops -f /var/log/messages&
> abrt-scan-xserver -f /var/log/Xorg.0.log&
> # generic log watcher program
> abrt-scan-log -f /some/other.log -start-regex='ALERT'
-end-regex='END' -maxlines=50&
> ...
>
> in background, and every one of these watches and processes
> one log file.
- here we would also need some way of adding/removing the scanner which
is the same problem as adding/removing the log files to scan..
Unix already has this mechanism: "killall PROG" and "PROG ARGS &"
- one solution might be to turn all scanners into a separate services
(with their own init files) but now when the trend is to lower the
number of services started on boot this doesn't sound as a good idea..
Yes. I decided to make them children of abrtd for now, to not have
many more services.
> Which proposal do you like more? Do you have other ideas
> or see other problems to watch for?
-don't have a strong opinion here, but I like the (1) a bit more,
because there is only one long running process involved...
I decided to try a mixed approach: log watcher is a separate process,
but it is started/stopped by abrtd:
$ abrt-dump-oops
Usage: abrt-dump-oops [-vsrdow] FILE
Extract oops from syslog/dmesg file
-v, --verbose Be verbose
-s Log to syslog
-r Parse kernel's message buffer before parsing FILE
-d Create ABRT dump for every oops found
-o Print found oopses on standard output
-w Do not exit, watch the file for new oopses
abrtd spawns "abrt-dump-oops -drw [-v[v[v]]] [-s] /var/log/messages".
By the way, it also replaces dumpoops.
Sending the patch in a minute.
--
vda