On Mon, 07.05.12 17:50, Jan Kratochvil (jan.kratochvil(a)redhat.com) wrote:
On Mon, 07 May 2012 17:40:18 +0200, Lennart Poettering wrote:
> on the individual machine,
There is no backing reason for this requirement. It does not matter
That's my requirement, and actually of many others, too.
Everybody who builds OSes or appliances, and who needs to supervise a
large number of systems, and hosts, wants stacktraces that just work,
and don't require network access or any complex infrastructure.
> without having to keep coredumps,
Core dump currently always have to be shortly stored on the disk anyway, even
for using minidebuginfo. Backtracing crashed program without dumping its core
would be a different project.
Temporarily and briefly storing things on disk is not a problem. Whether
something is in a temporary file or in memory is pretty much an
implementation detail. What matters it that we don't have to keep them
all the time.
> And currently we can't do this.
Unfortunately this is not possible with all your requirements due to:
* Even the optimal full debuginfo size (Jakub's dwz) is still only IIRC ~50%
of the current debuginfo size, which is still not suitable to install for
every package on every machine.
We don't need the full backtrace in all cases. There's a lot of room
between "no backtrace" and "best backtrace ever". For the client-side
backtraces in "low quality" the way Alex suggests are perfectly OK. It's
a reasonable tradeoff between disk usage and usefulness.
* Opinions to this item significantly differ but minidebuginfo-only
backtraces are in many (IMO most) cases not usable for problem
Well that's somewhere where we simply don't agree. The kernel folks only
have these low-quality backtraces, and they mostly are OK with it, never
asked for more. At least I couldn't find any mention on Google that
people were complaining loudly about the kernel backtraces, that they
were too limited.
Also note that a couple of projects over the years have been patched to do
in-process backtraces with backtrace(3). For example D-Bus did. The fact
that people did that makes clear that people want client side backtraces
that just work. (even though these are not too useful, since they only
use exported symbols names, instead of real debuginfo)
The Intel folks are actually leaving full debug info around for their
mobile distros, because they want client side backtraces that just
work, and their systems are big enough to not care too much about the
I mean, people all around of us go for client side backtraces, and it
has shown to be a valuable tool for very little price (if done properly
they way Alex suggests).
> You don't want to centralize dispatching of something that
> a million times a second all around the world.
Unique crashes do not happen so often.
Well, but how do you figure out that a crash is unique? You extract a
backtrace in some form and match it against some central database of
some form. That's kinda hard to do without, well, centralization.
And anyway: so I want my backtrace resolved, and by your suggestions I'd
hence have to talk to your server. But the server then tells me: nah, no
can do, yours has been seen before, go away! That makes little sense. I
just wanted my backtrace, nothing else resolved.
I mean, there are certain things that should just work, without any
complex centralized infrastructure, without having Fedora even know
about it. And one of those things is getting a frickin' stacktrace for
the crashes on your systems.
> Right now, it is easier to make sense of a kernel oops without
> special tools,
Kernel is a very specific package and its behavior and coding style cannot be
automatically generalized to other packages.
The kernel is a C program, with a stack and symbols, and is in this
regard very much the same as any other C program.
I mean, I can tell you: I want client-side backtraces that just work,
and I know a lot of people and companies who also do (see two examples
above). You tell me: "nah, nobody wants that". I mean, it's obvious that
you aren't right, are you?
Lennart Poettering - Red Hat, Inc.