kexec November 2018

kexec@lists.fedoraproject.org

4 participants
6 discussions

[PATCH] Make udev reload rules quiet during bootup
by Kairui Song 05 Dec '18

05 Dec '18

In commit 1c97aee and commit 227c185 udev rules was rewritten to use systemd-run to run in a non-blocking mode. The problem is that it's a bit noise, especially on machine bootup, systemd will always generate extra logs for service start, you might see your journal full of lines like these if you have many CPUs (each CPU generates a udev event on boot): ... Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. Nov 22 22:23:05 localhost systemd[1]: Started /usr/lib/udev/kdump-udev-throttler. ... While system is still booting up, kdump service is not started yet, so systemd-run calls will end up doing nothing, the throttler being called by systemd-run will just exit if kdump is not loaded. This patch avoid systemd-run from being called at first place if kdump service is not running, so there won't be unnecessary logs. As udev rules don't support shell expressions, this patch reuse the kdump-udev-throttler script, adding a --no-block option. kdump-udev-throttler will call it self with systemd-run to run in non blocking mode if --no-block is given, and exit before doing anything if kdump is not started. Udev just call "kdump-udev-throttler --no-block" directly. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- 98-kexec.rules | 2 +- kdump-udev-throttler | 13 ++++++++++--- 2 files changed, 11 insertions(+), 4 deletions(-) diff --git a/98-kexec.rules b/98-kexec.rules index 2f88c77..eca909e 100644 --- a/98-kexec.rules +++ b/98-kexec.rules @@ -7,6 +7,6 @@ GOTO="kdump_reload_end" LABEL="kdump_reload" -RUN+="/usr/bin/systemd-run --no-block /usr/lib/udev/kdump-udev-throttler" +RUN+="/usr/lib/udev/kdump-udev-throttler --no-block" LABEL="kdump_reload_end" diff --git a/kdump-udev-throttler b/kdump-udev-throttler index 6cbb99a..6899379 100755 --- a/kdump-udev-throttler +++ b/kdump-udev-throttler @@ -13,13 +13,20 @@ # In this way, we can make sure kdump service is restarted immediately # and for exactly once after udev events are settled. - throttle_lock="/var/lock/kdump-udev-throttle" -interval=2 -# Don't reload kdump service if kdump service is not started by systemd +# Don't do anything if kdump service is not started by systemd systemctl is-active kdump.service &>/dev/null || exit 0 +if [[ $1 == '--no-block' ]]; then + systemd-run --no-block --quiet $0 + if [[ $? -ne 0 ]]; then + echo "kdump-udev-throttler: Failed to call systemd-run" + exit 1 + fi + exit 0 +fi + exec 9>$throttle_lock if [ $? -ne 0 ]; then echo "Failed to create the lock file! Fallback to non-throttled kdump service restart" -- 2.19.1

4 5

[PATCH] dracut-module-setup: Fix routing failure on multipath route
by Kairui Song 26 Nov '18

26 Nov '18

Currently we still don't support multipath route, when parsing multipath route kdumpctl will wrong consider 'nexthop' as the destination address, and raise errors in second kernel. When multipath route is in use, ip route output should be like this: $ /sbin/ip route show default via 192.168.122.1 dev ens1 proto dhcp metric 100 192.168.122.0/24 dev ens1 proto kernel scope link src 192.168.122.161 metric 100 192.168.122.8 nexthop via 192.168.122.1 dev ens1 weight 50 nexthop via 192.168.122.2 dev ens1 weight 5 As we won't care about HA/performance, simply use the rule with highest weight and ignore the rest. --- dracut-module-setup.sh | 41 ++++++++++++++++++++++++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 2f9d762..7499678 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -126,7 +126,8 @@ kdump_static_ip() { echo -n "${_srcaddr}::${_gateway}:${_netmask}::" fi - /sbin/ip $_ipv6_flag route show | grep -v default | grep ".*via.* $_netdev " |\ + /sbin/ip $_ipv6_flag route show | grep -v default |\ + grep ".*via.* $_netdev " | grep -v "^[[:space:]]*nexthop" |\ while read _route; do _target=`echo $_route | cut -d ' ' -f1` _nexthop=`echo $_route | cut -d ' ' -f3` @@ -136,6 +137,44 @@ kdump_static_ip() { fi echo "rd.route=$_target:$_nexthop:$_netdev" done >> ${initdir}/etc/cmdline.d/45route-static.conf + + kdump_handle_mulitpath_route $_netdev $_srcaddr +} + +kdump_handle_mulitpath_route() { + local _netdev="$1" _srcaddr="$2" _ipv6_flag + local _target _nexthop _route _weight _max_weight _rule + + if is_ipv6_address $_srcaddr; then + _ipv6_flag="-6" + fi + + while IFS="" read _route; do + if [[ "$_route" =~ [[:space:]]+nexthop ]]; then + _route=$(echo "$_route" | sed -e 's/^[[:space:]]*//') + # Parse multipath route, using previous _target + [[ "$_target" == 'default' ]] && continue + [[ "$_route" =~ .*via.*\ $_netdev ]] || continue + + _weight=`echo "$_route" | cut -d ' ' -f7` + if [[ "$_weight" -gt "$_max_weight" ]]; then + _nexthop=`echo "$_route" | cut -d ' ' -f3` + _max_weight=$_weight + if [ "x" != "x"$_ipv6_flag ]; then + _rule="rd.route=[$_target]:[$_nexthop]:$_netdev" + else + _rule="rd.route=$_target:$_nexthop:$_netdev" + fi + fi + else + [[ -n "$_rule" ]] && echo "$_rule" + _target=`echo "$_route" | cut -d ' ' -f1` + _rule="" _max_weight=0 _weight=0 + fi + done >> ${initdir}/etc/cmdline.d/45route-static.conf\ + < <(/sbin/ip $_ipv6_flag route show) + + [[ -n $_rule ]] && echo $_rule >> ${initdir}/etc/cmdline.d/45route-static.conf } kdump_get_mac_addr() { -- 2.17.1

3 5

[PATCH] mkdumprd: drop some nfs mount options when reading from kernel
by Kairui Song 23 Nov '18

23 Nov '18

nfs service will append extra mount options to kernel mount options. Those extra options represent current mounting details, but they may not suitable for the second kernel. IP address may change, and we only enable a single network stack (v4/v6), if nfs prefered another network stack, inheriting the options will force nfs service to use previous network stack and disable nfs's fallback mechanic and fail. As nfs service have the capability to negotiate required protocols and detect proper IP address, just drop those options and let nfs automatically adapt the possible change in the second kernel. Signed-off-by: Kairui Song <kasong(a)redhat.com> --- mkdumprd | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/mkdumprd b/mkdumprd index 078f988..a6f7fe8 100644 --- a/mkdumprd +++ b/mkdumprd @@ -98,7 +98,14 @@ to_mount() { _fstype=$(findmnt -k -f -n -r -o FSTYPE $_dev) [[ -e /etc/fstab ]] && _options=$(findmnt --fstab -f -n -r -o OPTIONS $_dev) - [ -z "$_options" ] && _options=$(findmnt -k -f -n -r -o OPTIONS $_dev) + if [ -z "$_options" ]; then + _options=$(findmnt -k -f -n -r -o OPTIONS $_dev) + if [[ $_fstype == "nfs"* ]]; then + _options=$(echo $_options | sed 's/,addr=[^,]*//') + _options=$(echo $_options | sed 's/,proto=[^,]*//') + _options=$(echo $_options | sed 's/,clientaddr=[^,]*//') + fi + fi # with 'noauto' in fstab nfs and non-root disk mount will fail in 2nd # kernel, filter it out here. _options=$(echo $_options | sed 's/\bnoauto\b//') -- 2.19.1

2 1

[PATCH] kdumpctl: respect address selected by nfs service
by Kairui Song 19 Nov '18

19 Nov '18

When using nfs target, kdumpctl will manually resolve nfs server's hostname. But the problem is nfs service may select another address if the server have multiple address. eg. nfs will try IPv6 first, then failover to IPv4. It's hard to tell if the NFS server is serving nfs service on it's IPv6 or IPv4 interface or both, things may get even more complex if the host have multiple address or there are some special routing rules. Blindly use IPv4 / IPv6 may have poteintial risk. So use the working address reported by nfs service, which should always work. --- dracut-module-setup.sh | 12 ++++++++++-- kdump-lib.sh | 10 +++++++++- mkdumprd | 5 ++--- 3 files changed, 21 insertions(+), 6 deletions(-) diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 7499678..2637c29 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -479,7 +479,7 @@ adjust_bind_mount_path() #install kdump.conf and what user specifies in kdump.conf kdump_install_conf() { - local _opt _val _pdev + local _opt _val _pdev _addr sed -ne '/^#/!p' /etc/kdump.conf > ${initdir}/tmp/$$-kdump.conf while read _opt _val; @@ -498,7 +498,15 @@ kdump_install_conf() { adjust_bind_mount_path "$_val" fi ;; - ssh|nfs) + nfs) + _addr=$(get_mount_options_from_target "$_val" | sed 's/.*,addr=$[0-9a-f:\.]*$.*/\1/') + if [[ -n "$_addr" ]]; then + kdump_install_net "$_addr" + else + kdump_install_net "$_val" + fi + ;; + ssh) kdump_install_net "$_val" ;; dracut_args) diff --git a/kdump-lib.sh b/kdump-lib.sh index 6acab8c..8b4e360 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -236,11 +236,19 @@ get_target_from_path() echo $_target } -get_fs_type_from_target() +get_fs_type_from_target() { echo $(findmnt -k -f -n -r -o FSTYPE $1) } +get_mount_options_from_target() +{ + local _options + [[ -e /etc/fstab ]] && _options=$(findmnt --fstab -f -n -r -o OPTIONS $1) + [ -z "$_options" ] && _options=$(findmnt -k -f -n -r -o OPTIONS $1) + echo $_options +} + # input: device path # output: the general mount point # find the general mount point, not the bind mounted point in atomic diff --git a/mkdumprd b/mkdumprd index 078f988..9c59fef 100644 --- a/mkdumprd +++ b/mkdumprd @@ -96,9 +96,8 @@ to_mount() { _target="/kdumproot/$_target" fi - _fstype=$(findmnt -k -f -n -r -o FSTYPE $_dev) - [[ -e /etc/fstab ]] && _options=$(findmnt --fstab -f -n -r -o OPTIONS $_dev) - [ -z "$_options" ] && _options=$(findmnt -k -f -n -r -o OPTIONS $_dev) + _fstype=$(get_fs_type_from_target $_dev) + _options=$(get_mount_options_from_target $_dev) # with 'noauto' in fstab nfs and non-root disk mount will fail in 2nd # kernel, filter it out here. _options=$(echo $_options | sed 's/\bnoauto\b//') -- 2.17.1

2 2

[PATCH] doc/kdump.conf: Local dump path should be <mnt>/<path>/%HOST_IP-%DATE
by Bhupesh Sharma 15 Nov '18

15 Nov '18

Resolves: bz1561837 Bugzilla: https://bugzilla.redhat.com/show_bug.cgi?id=1561837 Currently 'kdump.conf' and 'kdump.conf MAN page' entries state that the local dump path should be: <fs type> <partition> - Will mount -t <fs type> <partition> <mnt>, and copy /proc/vmcore to <mnt>/<path>/%DATE/. The correct vmcore path instead should be: <mnt>/<path>/%HOST_IP-%DATE/ Signed-off-by: Bhupesh Sharma <bhsharma(a)redhat.com> --- kdump.conf | 2 +- kdump.conf.5 | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/kdump.conf b/kdump.conf index 57af7b6ca52a..286ad275ce02 100644 --- a/kdump.conf +++ b/kdump.conf @@ -32,7 +32,7 @@ # # <fs type> <partition> # - Will mount -t <fs type> <partition> <mnt>, and copy -# /proc/vmcore to <mnt>/<path>/%DATE/. +# /proc/vmcore to <mnt>/<path>/%HOST_IP-%DATE/. # NOTE: <partition> can be a device node, label or uuid. # It's recommended to use persistent device names # such as /dev/vg/<devname>. diff --git a/kdump.conf.5 b/kdump.conf.5 index 11b1fad559b4..990076e9da6e 100644 --- a/kdump.conf.5 +++ b/kdump.conf.5 @@ -55,7 +55,7 @@ The default value is /root/.ssh/kdump_id_rsa. .B <fs type> <partition> .RS Will mount -t <fs type> <partition> <mnt>, and copy /proc/vmcore to -<mnt>/<path>/%DATE/. NOTE: <partition> can be a device node, label +<mnt>/<path>/%HOST_IP-%DATE/. NOTE: <partition> can be a device node, label or uuid. It's recommended to use persistent device names such as /dev/vg/<devname>. Otherwise it's suggested to use label or uuid. .RE -- 2.7.4

3 3

[PATCH v3 0/3] Fix serveral problems with kdump reloading on udev event
by Kairui Song 01 Nov '18

01 Nov '18

Currently kdump will restart for multiple times and may trigger a initramfs rebuild on memory / CPU hotplug if any configuration is modified. Besides, currently simply restart the service via systemctl directly will take a period of time as well, as kdumpctl will do a lot of config checking, which is not appreciated by udev, udev want everything to get done in a very short time. And kdumpctl may get kill for several times because systemd will kill the previous service starting process for later restart request. This may introduce extra risk too. Simply add reload support and use systemdctl reload will not work well either because systemd will ignore later reload request, so kdump may get reload earlier than the hotplug is finished. This patch series should solve the problem by applying following fixes: - Adding reload support as a fast path to reload kdump resources, bypassing most checking. So the reload progress will be faster and never trigger a initrmafs rebuild. - Let the reload request run in async mode, so udev will never get blocked. - Throttle udev events to avoid unnecessary kdump reloading and make sure kdump will reload after the udev settled. Update from V2: - Use "-ne" instead of "!=" for comparing return values of functions in kdumpctl. "!=" is used more for pattern matching. - Removed duplicated "is-system-running" check in kdump-udev-throttler - Add more detailed commit message about reason to use --no-block in udev rules Update from V1: - Add cover letter - Add --no-block option to systemd-run, starting from systemd-v220, systemd-run will run in synchronous mode by default, need to give --no-block explicitly to run in async mode. Kairui Song (3): kdumpctl: Add reload support Rewrite kdump's udev rules Throttle kdump reload request triggered by udev event 98-kexec.rules | 16 +++++++++--- kdump-udev-throttler | 47 +++++++++++++++++++++++++++++++++++ kdump.service | 1 + kdumpctl | 59 +++++++++++++++++++++++++++++++++++++------- kexec-tools.spec | 3 +++ 5 files changed, 113 insertions(+), 13 deletions(-) create mode 100755 kdump-udev-throttler -- 2.17.1

2 4

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

kexec November 2018