Hi Lianbo, thanks for the work, the log infrastruct will be improved
by a log with this patch series.
On Tue, Oct 20, 2020 at 10:45 AM Lianbo Jiang <lijiang(a)redhat.com> wrote:
Sometimes, debugging the kdump service failure becomes very challenging
because there is no complete debugging information, which requires
modification of the options or the scripts like kdumpctl, mkdumprd, etc
to collect the information for troubleshooting.
That means users have to wait for the next failure so that they can
capture the additional information, which could waste valuable time.
This patch series will improve kdump debugging messages and have a
chance to save them to journald or a file. It includes the following
patches:
[1] [PATCH 1/5] introduce the kdump logger from the dracut
[2] [PATCH 2/5] enable the logger for kdump
[3] [PATCH 3/5] kdump.sysconfig: add the kdump logger configurations
[4] [PATCH 4/5] kdumpctl: add the '-d' option to enable the kexec loading
debugging messages
[5] [PATCH 5/5] Improve debugging in the kdump kernel
[6] [PATCH 6/6] Doc: add a documentation for the usage of logger
In addition, this patch series won't cause performance degradation,
basically this effect can be ignored. I simply checked the result of
executing the loading command as follow:
[1] before
# time kdumpctl start
Detected change(s) in the following file(s):
/etc/kdump.conf
Rebuilding /boot/initramfs-4.18.0-232.el8.x86_64kdump.img
Tips: If early kdump is enabled, also require rebuilding the system initramfs to make the
changes take effect for early kdump.
kexec: loaded kdump kernel
Starting kdump: [OK]
real 0m9.145s
user 0m20.733s
sys 0m1.765s
[2] after
# time kdumpctl start
kdump: Detected change(s) in the following file(s): /etc/kdump.conf
kdump: Rebuilding /boot/initramfs-4.18.0-232.el8.x86_64kdump.img
kdump: Tips: If early kdump is enabled, also require rebuilding the system initramfs to
make the changes take effect for early kdump.
kdump: kexec: loaded kdump kernel
kdump: Starting kdump: [OK]
real 0m9.177s
user 0m20.784s
sys 0m1.901s
Changes since v5:
[1] Only added a documentation for the usage of logger, there's
no code change.
Note:
Currently, kexec-tools disable storing logs to journald in order
to save memory, please refer to the following changes:
+++ b/dracut-module-setup.sh
@@ -788,6 +788,8 @@ kdump_install_random_seed() {
}
...
kdump_install_systemd_conf() {
if ! is_fadump_capable && [ "$failure_action" != "shell"
]; then
mkdir -p ${initdir}/etc/systemd/journald.conf.d
echo "[Journal]" >
${initdir}/etc/systemd/journald.conf.d/kdump.conf
echo "Storage=none" >>
${initdir}/etc/systemd/journald.conf.d/kdump.conf
^^^^
If we want to use the journald in kdump kernel, we have to change it to
"Storage=auto" or "volatile".
For the log storage in kdump kernel, I'm thinking maybe we have
Storage=none and ForwardToKmsg=yes, so kmsg will be used as the sole
log storage in kdump kernel, which looks a good balance of keeping the
logs and reduce memory usage. How do you think? If you want to keep
more log, just increase log_bufsize, simple and easy.
Maybe we can have a is_kdump_kernel helper in kdump-libs.sh, to judge
if it's kdump kernel.
Also, how about have a standalong log level like kdump_dumploglvl,
aside from "kdump_stdloglvl kdump_sysloglvl kdump_kmsgloglvl".
kdump_dumploglvl could be used as the only loglevel in kdump kernel,
and all log content just goes to logger (journalctl, it will forward
log to console, and store log in kmsg), as I think ideally, there is
only one log output/storage in kdump kernel anyway (log is always
printed to console, and stored in kmsg). Then to save the log in kdump
kernel, just use dmsg.
I've also added some comment in other individual patches.
Lianbo Jiang (6):
introduce the kdump logger from the dracut
enable the logger for kdump
kdump.sysconfig: add the kdump logger configurations
kdumpctl: add the '-d' option to enable the kexec loading debugging
messages
Improve debugging in the kdump kernel
Doc: add a documentation for the usage of logger
dracut-early-kdump-module-setup.sh | 3 +
dracut-early-kdump.sh | 24 +-
dracut-kdump.sh | 68 +++---
dracut-module-setup.sh | 4 +
kdump-lib-initramfs.sh | 67 ++++--
kdump-lib.sh | 23 +-
kdump-logger.sh | 344 +++++++++++++++++++++++++++++
kdump.sysconfig | 16 ++
kdump.sysconfig.aarch64 | 16 ++
kdump.sysconfig.i386 | 16 ++
kdump.sysconfig.ppc64 | 16 ++
kdump.sysconfig.ppc64le | 16 ++
kdump.sysconfig.s390x | 16 ++
kdump.sysconfig.x86_64 | 16 ++
kdumpctl | 241 +++++++++++---------
kexec-kdump-howto.txt | 35 +++
kexec-tools.spec | 2 +
mkdumprd | 12 +-
18 files changed, 766 insertions(+), 169 deletions(-)
create mode 100755 kdump-logger.sh
--
2.17.1
--
Best Regards,
Kairui Song