On 01/14/2011 02:10 PM, Denys Vlasenko wrote:
On Thu, 2011-01-13 at 14:46 +0100, Jiri Moskovcak wrote:
> On 01/13/2011 01:25 PM, Denys Vlasenko wrote:
>> On Fri, 2011-01-07 at 15:29 +0100, Jiri Moskovcak wrote:
>>> = ABRT 2.0 Features (what we need) =
>>> == need to be done features ==
>>> plugins free daemon
>>> -> add a generic log scanner
>>
>> I would like to work on this one. Here's my
>> "How to implement log scanning?" brainstorm.
>>
>>
>> Proposal (1): daemon watches log files, whenever one of them changes,
>> it runs a program which parses the log file. It can give the offset
>> in the log from where parsing should start.
>> Table of files to watch and helpers to run should be somewhere in config
>> file:
>>
>> # Specialized log watcher programs
>> /var/log/messages abrt-scan-oops -f $FNAME -o $OFF
>> /var/log/Xorg.0.log abrt-scan-xserver -f $FNAME -o $OFF
>> # generic log watcher program
>> /some/other.log abrt-scan-log -f $FNAME -o $OFF -start-regex='ALERT'
-end-regex='END' -maxlines=50
>> ...
>>
>> I see one problem with this setup.
>> We need to consider a case when watched logfile gets flooded
>> (growing continuously and quickly). This means daemon
>> will be starting the parser all the time, and will have
>> hard time determining where exactly parser stopped parsing
>> last time: say, /var/log/messages was 100M long. We started parser.
>> Parser finished. We look at /var/log/messages and it's 110M long now.
>> How can we be sure that parser read until the very end of 110th meg,
>> that it did not hit EOF at, say, 109M? IOW: where to restart
>> next parsing?
>>
>
> - parsers can return some status message saying where they stopped
>
> - btw, what happens now if we parse /var/log/messages, find the oops,
> but the oops is not fully written there - the line ---- end ----- is not
> there?
We drop it if it's too short. If it isn't, we submit truncated one.
>> Seems like this problem can be solved if we keep an open fd
>> to the log file in question, and use that fd to read the file.
>> This way, we always know where we stopped.
>
> - smells like leaking fd, which selinux won't like...
Not if the fd isn't in daemon but in log watcher process.
>> This leads me to proposal (2): every log file is watched by
>> a separate "log watching daemon". Basically, we just start:
>>
>> # Specialized log watcher programs
>> abrt-scan-oops -f /var/log/messages&
>> abrt-scan-xserver -f /var/log/Xorg.0.log&
>> # generic log watcher program
>> abrt-scan-log -f /some/other.log -start-regex='ALERT'
-end-regex='END' -maxlines=50&
>> ...
>>
>> in background, and every one of these watches and processes
>> one log file.
>
> - here we would also need some way of adding/removing the scanner which
> is the same problem as adding/removing the log files to scan..
Unix already has this mechanism: "killall PROG" and "PROG ARGS&"
- we didn't understand each other, I meant it as a problem
adding/removing it in abrt config file
> - one solution might be to turn all scanners into a separate services
> (with their own init files) but now when the trend is to lower the
> number of services started on boot this doesn't sound as a good idea..
Yes. I decided to make them children of abrtd for now, to not have
many more services.
>> Which proposal do you like more? Do you have other ideas
>> or see other problems to watch for?
>
> -don't have a strong opinion here, but I like the (1) a bit more,
> because there is only one long running process involved...
I decided to try a mixed approach: log watcher is a separate process,
but it is started/stopped by abrtd:
$ abrt-dump-oops
Usage: abrt-dump-oops [-vsrdow] FILE
Extract oops from syslog/dmesg file
-v, --verbose Be verbose
-s Log to syslog
-r Parse kernel's message buffer before parsing FILE
-d Create ABRT dump for every oops found
-o Print found oopses on standard output
-w Do not exit, watch the file for new oopses
abrtd spawns "abrt-dump-oops -drw [-v[v[v]]] [-s] /var/log/messages".
By the way, it also replaces dumpoops.
Sending the patch in a minute.