[PATCH v2] mkdumprd: use 300s as the default systemd unit timeout for kdump mount
by Xunlei Pang
Currently, systemd uses 90s as the default mount unit timeout,
in some cases, although it works well in 1st kernel, it's not
enough under kdump and results in mount timeout, further results
in kdump dumping failure.
We've met several such issues, we decided to enlarge this default
value a little for kdump.
We finalize "x-systemd.device-timeout=300" as the default timeout
to the mount options if there is no "x-systemd.device-timeout=X"
specified. It can be overridden by /etc/fstab mount options,etc,
so that users can specify other timeout values if they want to.
Note: this is different from rd.timeout which was introduced by
dracut initqueue.
Signed-off-by: Xunlei Pang <xlpang(a)redhat.com>
---
v1->v2:
-Use 300s other than 600s as the default value.
mkdumprd | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/mkdumprd b/mkdumprd
index d3ecbd6..29e51a9 100644
--- a/mkdumprd
+++ b/mkdumprd
@@ -104,6 +104,12 @@ to_mount() {
_options=$(echo $_options | sed 's/noauto//')
_options=${_options/#ro/rw} #mount fs target as rw in 2nd kernel
+ # kdump turns out to require longer default systemd mount timeout
+ # than 1st kernel(90s by default), we choose 300s for kdump.
+ if ! strstr $_options "x-systemd.device-timeout"; then
+ _options="$_options,x-systemd.device-timeout=300"
+ fi
+
_mntopts="$_target $_fstype $_options"
#for non-nfs _dev converting to use udev persistent name
if [ -b "$_source" ]; then
--
1.8.3.1
6 years, 4 months
[PATCH] mkdumprd: use 600s as the default systemd unit timeout for kdump mount
by Xunlei Pang
Currently, systemd uses 90s as the default mount unit timeout,
in some cases, it's not enough and results in mount timeout
further results in kdump dumping failure, but the device can
actually be ready after a while. We've met several such issues.
So, we add a "x-systemd.device-timeout=600"(600s should be long
enough) as the default timeout to the mount options if there is
no "x-systemd.device-timeout=X" specified. It can be overridden
by /etc/fstab mount options, so that users can specify other
timeout values if they want to.
Note: this is different from rd.timeout which was introduced by
dracut initqueue.
Signed-off-by: Xunlei Pang <xlpang(a)redhat.com>
---
mkdumprd | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/mkdumprd b/mkdumprd
index d3ecbd6..30f8ba6 100644
--- a/mkdumprd
+++ b/mkdumprd
@@ -104,6 +104,11 @@ to_mount() {
_options=$(echo $_options | sed 's/noauto//')
_options=${_options/#ro/rw} #mount fs target as rw in 2nd kernel
+ # use 600s as default systemd mount timeout if none
+ if ! strstr $_options "x-systemd.device-timeout"; then
+ _options="$_options,x-systemd.device-timeout=600"
+ fi
+
_mntopts="$_target $_fstype $_options"
#for non-nfs _dev converting to use udev persistent name
if [ -b "$_source" ]; then
--
1.8.3.1
6 years, 4 months
[PATCH V2] Improve 'cpu add' udev rules
by Pratyush Anand
Currently kdump service is restarted even when any new file is added in cpu
subsystem. So, it can be restarted multiple times in the cases like loading
of acpi_cpufreq module or online/offline of any cpu.
However, we should see kdump service restart only once in case a new CPU
is added or removed. cpu crash notes buffer is created when a new CPU is
added. It's location does not change when a CPU is onlined/offlined or
acpi_cpufreq driver is loaded. Therefore, no need to restart kdump
service in such cases.
Thus, we need to introduce an extra filter for the kernel name of the
directory created by cpu_add which is KERNEL=="cpu[0-9]*". This will ensure
that kdump service is not restarted when any new file is added in /removed
from cpu subsystem.
Signed-off-by: Pratyush Anand <panand(a)redhat.com>
---
- Although, I have not reproduced this issue on fedora,rather it was in
RHEL6, but it looks like that we should better have this modified rules,
which is more justified.
Changes since v1:
- v1 used to filter with subsystem as 'acpi',which might not work on
playform like powerpc where they have non-acpi method of cpu addition.
98-kexec.rules | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/98-kexec.rules b/98-kexec.rules
index e32ee13cb2e8..5dcf9fae419a 100644
--- a/98-kexec.rules
+++ b/98-kexec.rules
@@ -1,4 +1,4 @@
-SUBSYSTEM=="cpu", ACTION=="add", PROGRAM="/bin/systemctl try-restart kdump.service"
-SUBSYSTEM=="cpu", ACTION=="remove", PROGRAM="/bin/systemctl try-restart kdump.service"
+SUBSYSTEM=="cpu", ACTION=="add", KERNEL=="cpu[0-9]*", PROGRAM="/bin/systemctl try-restart kdump.service"
+SUBSYSTEM=="cpu", ACTION=="remove", KERNEL=="cpu[0-9]*", PROGRAM="/bin/systemctl try-restart kdump.service"
SUBSYSTEM=="memory", ACTION=="online", PROGRAM="/bin/systemctl try-restart kdump.service"
SUBSYSTEM=="memory", ACTION=="offline", PROGRAM="/bin/systemctl try-restart kdump.service"
--
2.9.4
6 years, 4 months
[PATCH] kdump-lib.sh: support dump ssh via local ipv6 address
by Ziyue Yang
When dumping to ssh via local ipv6 address, the ssh parameter
in kdump.conf is supposed to have the form like
ssh user@fe80::cc1:8bff:fe90:b95f%eth0
where "%eth0" is an existing network interface supporting ipv6.
However, the get_remote_host function in kdump-lib.sh currently
doesn't remove the network interface in the ipv6 address,
causing the ip command in kdump_install_net function to fail,
leading to a "Bad kdump location" message.
This patch supports dumping ssh via local ipv6 address by enabling
get_remote_host function to remove network interface in ipv6 addresses
if there are any.
Signed-off-by: Ziyue Yang <ziyang(a)redhat.com>
---
kdump-lib.sh | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/kdump-lib.sh b/kdump-lib.sh
index 3f0af91..cca68e5 100755
--- a/kdump-lib.sh
+++ b/kdump-lib.sh
@@ -323,6 +323,10 @@ get_remote_host()
_config_val=${_config_val%:/*}
_config_val=${_config_val#[}
_config_val=${_config_val%]}
+ # factor out network inteface segment in local ipv6 address
+ if is_ipv6_address $_config_val; then
+ _config_val=${_config_val%%%*}
+ fi
echo $_config_val
}
--
2.9.3
6 years, 4 months
[PATCH] module-setup: suppress the early iscsi error messages
by Xunlei Pang
Currently, we throw the error message at the very beginning, as
a result on a pure-hardware(all-offload) iscsi machine with many
iscsi partitions, we suffered from too much noise as follows:
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.1/host0/session1
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.3/host1/session2
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.1/host0/session1
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.3/host1/session2
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.1/host0/session1
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.3/host1/session2
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.1/host0/session1
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.3/host1/session2
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.1/host0/session1
iscsiadm: No records found
Unable to find iscsi record for
/sys/devices/pci0000:00/0000:00:06.0/0000:07:00.0/0000:08:01.3/host1/session2
kexec: loaded kdump kernel
Starting kdump: [OK]
There's no need to know the very early error messages, we can
remove the error output which is actually normal for the pure
hardware iscsi.
As for unexpected errors, we kept the error outputs in the
succeeding kdump_iscsi_get_rec_val() calls by not appending
"2>/dev/null".
Signed-off-by: Xunlei Pang <xlpang(a)redhat.com>
---
dracut-module-setup.sh | 8 +++-----
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh
index 59b4f04..43e3f2a 100755
--- a/dracut-module-setup.sh
+++ b/dracut-module-setup.sh
@@ -545,11 +545,9 @@ kdump_setup_iscsi_device() {
dinfo "Found iscsi component $1"
- # Check once before getting explicit values, so we can output a decent
- # error message.
-
- if ! /sbin/iscsiadm -m session -r ${path} >/dev/null ; then
- derror "Unable to find iscsi record for $path"
+ # Check once before getting explicit values, so we can bail out early,
+ # e.g. in case of pure-hardware(all-offload) iscsi.
+ if ! /sbin/iscsiadm -m session -r ${path} &>/dev/null ; then
return 1
fi
--
1.8.3.1
6 years, 4 months
[PATCH] Improve 'cpu add' udev rules
by Pratyush Anand
Currently kdump service is restarted even when any new file is added in cpu
subsystem. So, it can be restarted multiple times in the cases like loading
of acpi_cpufreq module or online/offline of any cpu.
However, we should see kdump service restart only once in case a new CPU
is added or removed. cpu crash notes buffer is created when a new CPU is
added. It's location does not change when a CPU is onlined/offlined or
acpi_cpufreq driver is loaded. Therefore, no need to restart kdump
service in such cases.
IMO, there is only one processor driver (drivers/acpi/processor_core.c) which
can enumerate a hot added cpu. An `udevadm info` enquiry to acpi device created
by this driver gives:
# udevadm info -a -p /sys/bus/acpi/devices/LNXCPU\:00/
looking at device
'/devices/LNXSYSTM:00/LNXSYBUS:00/ACPI0010:00/LNXCPU:00':
KERNEL=="LNXCPU:00"
SUBSYSTEM=="acpi"
Therefore, this patch modifies the rule to filter such devices.
Signed-off-by: Pratyush Anand <panand(a)redhat.com>
---
98-kexec.rules | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/98-kexec.rules b/98-kexec.rules
index e32ee13cb2e8..d43760e38f8f 100644
--- a/98-kexec.rules
+++ b/98-kexec.rules
@@ -1,4 +1,4 @@
-SUBSYSTEM=="cpu", ACTION=="add", PROGRAM="/bin/systemctl try-restart kdump.service"
-SUBSYSTEM=="cpu", ACTION=="remove", PROGRAM="/bin/systemctl try-restart kdump.service"
+SUBSYSTEM=="acpi", ACTION=="add", KERNEL=="LNXCPU*", PROGRAM="/etc/init.d/kdump condrestart"
+SUBSYSTEM=="acpi", ACTION=="remove", KERNEL=="LNXCPU*", PROGRAM="/etc/init.d/kdump condrestart"
SUBSYSTEM=="memory", ACTION=="online", PROGRAM="/bin/systemctl try-restart kdump.service"
SUBSYSTEM=="memory", ACTION=="offline", PROGRAM="/bin/systemctl try-restart kdump.service"
--
2.9.4
6 years, 4 months
[PATCH] mkdumprd: remove useless "x-initrd.mount"
by Xunlei Pang
After the following systemd commit, "x-initrd.mount"
option became useless actually, we can safely remove
it now.
commit ce3f6d82b003f365f718f24e48f55b8a0372b924
Author: nmartensen <nis.martensen(a)web.de>
Date: Fri Jan 15 07:55:25 2016 +0100
fstab-generator: remove bogus condition
The sysroot mount is already taken care of by the
add_sysroot_mount function. With this condition
left in, we can we can get something like this:
initrd-root-fs.target.requires
`-- usr.mount -> /run/systemd/generator/usr.mount
in the main system (i.e., not in the initramfs). In
the initramfs, the previous condition already kicks in.
diff --git a/src/fstab-generator/fstab-generator.c
<snip>
if (initrd)
post = SPECIAL_INITRD_FS_TARGET;
- else if (mount_in_initrd(me))
- post = SPECIAL_INITRD_ROOT_FS_TARGET;
else if (mount_is_network(me))
post = SPECIAL_REMOTE_FS_TARGET;
else
"mount_in_initrd(me)" is true with "x-initrd.mount" option,
the behaviour of systemd fstab generator changed after the
above-mentioned patch, it always generates local mount units
required by local-fs.target regardless of "x-initrd.mount".
After failure, it enters dracut emergency, further triggers
kdump emergency service, thus there is no problem.
Signed-off-by: Xunlei Pang <xlpang(a)redhat.com>
---
mkdumprd | 15 +--------------
1 file changed, 1 insertion(+), 14 deletions(-)
diff --git a/mkdumprd b/mkdumprd
index 5a25853..468b564 100644
--- a/mkdumprd
+++ b/mkdumprd
@@ -103,20 +103,7 @@ to_mount() {
# kernel, filter it out here.
_options=$(echo $_options | sed 's/noauto//')
_options=${_options/#ro/rw} #mount fs target as rw in 2nd kernel
- # "x-initrd.mount" mount failure will trigger isolate emergency service
- # W/o this, systemd won't isolate, thus we won't get to emergency.
- # This is not applicable to remote fs mount, because if we use
- # "x-initrd.mount", remote mount will become required by
- # "initrd-root-fs.target", instead of "remote-fs.target". That's how it is
- # handled within systemd internal. We need remote mount to be required
- # "remote-fs.target", because we need to bring up network before any remote
- # mount and "remote-fs.target" can be a checkpoint of that.
- # If remote mount fails, dracut-initqueue will still start and once
- # dracut-initqueue finishes, kdump service will start. Because remote mount
- # failed, kdump service will fail and it will lead to kdump error handler.
- if ! is_nfs_dump_target; then
- _options="$_options,x-initrd.mount"
- fi
+
_mntopts="$_target $_fstype $_options"
#for non-nfs _dev converting to use udev persistent name
if [ -b "$_source" ]; then
--
1.8.3.1
6 years, 4 months
[PATCH] kdumpctl: use "apicid" other than "initial apicid"
by Xunlei Pang
We met a problem on AMD machines, when using "nr_cpus=4" for
kdump, and crash happens on cpus other than cpu0, kdump kernel
will fail to boot and eventually reset.
After some debugging, we found that it stuck at the kernel path
do_boot_cpu()-> ... ->wakeup_secondary_cpu_via_init():
apic_icr_write(APIC_INT_LEVELTRIG|APIC_INT_ASSERT|APIC_DM_INIT,
phys_apicid);
that is, it stuck at sending INIT from AP to BP and reset, which
is actually what "disable_cpu_apicid=X" tries to solve. Printing
the value of @phys_apicid showed that it was the value of "apicid"
other that of "initial apicid" showed by /proc/cpuinfo.
As described in x86 specification:
"In MP systems, the local APIC ID is also used as a processor ID by the
BIOS and the operating system. Some processors permit software to modify
the APIC ID. However, the ability of software to modify the APIC ID is
processor model specific. Because of this, operating system software
should avoid writing to the local APIC ID register. The value returned by
bits 31-24 of the EBX register (when the CPUID instruction is executed with a
source operand value of 1 in the EAX register) is always the Initial APIC ID
(determined by the platform initialization). This is true even if software
has changed the value in the Local APIC ID register."
From kernel commit 151e0c7de("x86, apic, kexec: Add disable_cpu_apicid
kernel parameter"), we can see in generic_processor_info(), it uses
a)read_apic_id() and b)@apicid to compare with @disabled_cpu_apicid.
a)@apicid which is actually @phys_apicid above-mentioned is from the
following calltrace(on the problematic AMD machine):
generic_processor_info+0x37/0x300
acpi_register_lapic+0x30/0x90
acpi_parse_lapic+0x40/0x50
acpi_table_parse_entries_array+0x171/0x1de
acpi_boot_init+0xed/0x50f
The value of @apicid(from acpi MADT) is equal to the value of "apicid"
showed by /proc/cpuinfo as proved by our debug printk.
b)read_apic_id() gets the value from LAPIC ID register which is "apicid"
as well.
While the value of "initial apicid" is from cpuid instruction.
One example of "apicid" and "initial apicid" of cpu0 from /proc/cpuinfo
on AMD machine:
apicid : 32
initial apicid : 0
Therefore, we should assign /proc/cpuifo "apicid" to "disable_cpu_apicid=X".
We've never met such issue before, because we usually tested "nr_cpus=1",
and mostly on Intel machines, and "apicid" and "initial apicid" have the
same value in most cases on Intel machines.
Signed-off-by: Xunlei Pang <xlpang(a)redhat.com>
---
kdumpctl | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/kdumpctl b/kdumpctl
index 4d6b3e8..46b65d2 100755
--- a/kdumpctl
+++ b/kdumpctl
@@ -77,15 +77,15 @@ remove_cmdline_param()
}
#
-# This function returns the "initial apicid" of the
-# boot cpu (cpu 0) if present.
+# This function returns the "apicid" of the boot
+# cpu (cpu 0) if present.
#
-get_bootcpu_initial_apicid()
+get_bootcpu_apicid()
{
awk ' \
BEGIN { CPU = "-1"; } \
$1=="processor" && $2==":" { CPU = $NF; } \
- CPU=="0" && /initial apicid/ { print $NF; } \
+ CPU=="0" && /^apicid/ { print $NF; } \
' \
/proc/cpuinfo
}
@@ -206,7 +206,7 @@ prepare_cmdline()
cmdline="${cmdline} ${KDUMP_COMMANDLINE_APPEND}"
- id=`get_bootcpu_initial_apicid`
+ id=`get_bootcpu_apicid`
if [ ! -z ${id} ] ; then
cmdline=`append_cmdline "${cmdline}" disable_cpu_apicid ${id}`
fi
--
1.8.3.1
6 years, 4 months
[PATCH v3] kdumpctl: fix infinite loop caused by running under bash
by Ziyue Yang
Description of problem
(https://bugzilla.redhat.com/show_bug.cgi?id=1465735):
Run `kdumpctl status` as normal user, get below error messages:
Another app is currently holding the kdump lock; waiting for it to exit...
flock: 9: Bad file descriptor
Another app is currently holding the kdump lock; waiting for it to exit...
flock: 9: Bad file descriptor
...
The bug is caused by behavior difference between bash
and sh (bash in posix).
In the function single_instance_lock in kdumpctl script,
there is
exec 9>/var/lock/kdump
which will fail in user mode. However, this fail will cause
script exiting under bash but not exiting under sh, causing
infinite loop because the flock will always fail.
According to the 16th item in
ftp://ftp.gnu.org/old-gnu/Manuals/bash-2.02/html_node/bashref_66.html
If a POSIX.2 special builtin returns an error status, a non-
interactive shell exits.
And according to
https://www.gnu.org/software/bash/manual/html_node/Special-Builtins.html
exec is one of the POSIX.2 special builtin's.
This patch fixes the bug by checking exec return value.
Signed-off-by: Ziyue Yang <ziyang(a)redhat.com>
---
kdumpctl | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/kdumpctl b/kdumpctl
index e440bbb..1b50966 100755
--- a/kdumpctl
+++ b/kdumpctl
@@ -37,6 +37,10 @@ single_instance_lock()
local rc timeout=5
exec 9>/var/lock/kdump
+ if [ $? -ne 0 ]; then
+ echo "Create file lock failed"
+ exit 1
+ fi
flock -n 9
rc=$?
--
2.9.3
6 years, 4 months
[PATCH v4 0/8] collect kdump targets and improve kdump per these information
by Xunlei Pang
We collect all the kdump targets i.e. devices recognized under kdump,
then improve kdump according to the type of the target. Currently we
have two: dump target and root target in case of "dump_to_rootfs".
If we know there is no crypt target, we can remove dracut "crypt"
module, we only add rd.lvm.lv=X regarding the lvm target to kdump.
If we know there is only network dumping(ssh/nfs), we can omit
some non-network modules like dm/lvm/multipath/crypt/iscsi/fcoe/etc.
If we know ..., we can omit ...
This series does two major things:
1)Patch 1~5 solve https://bugzilla.redhat.com/1451717
2)Patch 6~8 reduce the initramfs size by omitting needless dracut modules.
Xunlei Pang (8):
kdump-lib.sh: fix improper get_block_dump_target()
kdump-lib.sh: introduce get_kdump_targets()
mkdumprd: change for_each_block_target() to use get_kdump_targets()
kdumpctl: use generated rd.lvm.lv=X
mkdumprd: omit crypt when there is no crypt kdump target
mkdumprd: omit dracut modules in case of no dm target
mkdumprd: omit dracut modules in case of network dumping
module-setup: fix 99kdumpbase network dependency
dracut-module-setup.sh | 2 +-
kdump-lib.sh | 111 +++++++++++++++++++++++++++++++++++++++++++------
kdumpctl | 69 ++++++++++++++++++------------
mkdumprd | 103 +++++++++++++++++++++++++--------------------
4 files changed, 200 insertions(+), 85 deletions(-)
--
1.8.3.1
6 years, 4 months