Currently, kexec-tools parses legacy ifcfg-* configuration files or NetworkManager .nmconnection connection profiles to build up dracut command line parameters like ip=. Then dracut parses these parameters and runs nm-initrd-generator to generate NetworkManager connection profiles. Taking a bonding network as an example, nm-initrd-generator generates two connections as follows, $ /usr/libexec/nm-initrd-generator -s -- bootdev=mybond0 rd.neednet kdumpnic=mybond0 bond=mybond0:kdump-eth0
*** Connection 'mybond0' ***
[connection] id=mybond0 uuid=ed87d02b-dd44-4f0e-8b11-37db7e89bb48 interface-name=mybond0
[ipv4] dhcp-timeout=90 ...
*** Connection 'kdump-eth0' ***
[connection] id=kdump-eth0 uuid=ed6a7448-b5f8-4f8a-b718-3e24ea1c924b type=ethernet interface-name=kdump-eth0 master=ed87d02b-dd44-4f0e-8b11-37db7e89bb48 slave-type=bond wait-device-timeout=60000 ...
Later in kdump initrd, dracut starts NetworkManager to activate these profiles to bring up the network connections. This way of setting up kdump network is tedious, error-prone and unnecessary. A better way is to directly copy the needed connection profiles to initrd. A potential benefit of this approach for the users is they can simply edit the connection profile directly like changing ipv4.dhcp-timeout instead of being forced to use the hard-coded value enforced by nm-initrd-generator.
This patch set reuses NetworkManager connection profiles to setup kdump network. It also reduces the memory consumption of network drivers and fix other issues at the same time. A machine could have multiple network drivers and a network driver may manage multiple NICs. It's possible kdump may only need one NIC. In this case, there is no need to install unneeded network drivers or bring up unneeded NICs.
There are cases where a NIC may have a different name in the kdump kernel. Currently, kexec-tools creates udev rules to rename NIC to have a consistent name. With the new approach, we can simply ask NM to match the connection profile by MAC address instead.
Here are the bug list that addressed by this patch set, - Bug 1962421 - [RHEL-9]"eth0: Failed to rename network interface 3 from 'eth0' to 'kdump-eth0': File exists" - Bug 2064708 - kdump: mkdumprd: failed to make kdump initrd for bridge network on z15 z/vm - bugs related to OOM caused by network driver - Bug 1950282 - shutdown those unneeded network interfaces to save memory for kdump - Bug 1958587 - the kdump initramfs includes unnecessary NIC drivers for SSH/NFS dumping target - Bug 1890021 - be2net is using too much memory during kdump - Bug 1662202 - [RHEL-8.1] aarch64: hpe-apache crashkernel OOM when dump to network targe
Note this patch set is available for review on https://github.com/coiby/kexec-tools/pull/9
v2: - removes useless cat which also avoids SC2013 - fix code format issue and shellcheck warning - squash four commits "set up kdump network {bridge,vlan,bridge,team} by directly copying NM connection profile to initrd" to switch to the new approach all at once to avoid confusion and causing trouble for git bisection - fix several issues found by Philipp - add missing reference - use an array instead of supress shellchek SC2086 warning - use $_netifs instead of $_netif_allowlist to intruct NM to only manage specified NICs - multiple typos - use nmcli --get-values to get type first then construct MAC field name - get rid of the extra indentation - s/uniq_name/unique_name/
v1: - refactor complex kdump_copy_nmconnection_file [Philipp] - clean up temporary file in a trap [Philipp] - improve sed [Philipp] - use long name consistently [Philipp] - use nmcli -t, --terse option to suppress printing of the header [Philipp] - don't pass kdumpip to initrd [Kairui] - address the case where kdump_install_net is called multiple times like fence kdump - call kdump_install_nm_netif_allowlist after all kdump_install_net calls have been finished - ask NM to match a connection profile to a physical NIC by MAC address - stop treating vlan specially when its parent interface is a physical NIC - don't add unused NIC drivers to kdump initrd - fix the error of "/etc/NetworkManager/system-connections/*.nmconnection" not exist grep: /etc/NetworkManager/system-connections/*.nmconnection: No such file or directory - redirect the messages of nmcli to ddebug - use "grep -s" to get rid of the following warnings grep: /var/tmp/dracut.6tqUm0/initramfs//etc/NetworkManager/system-connections/*.nmconnection: No such file or directory grep: /etc/sysconfig/network-scripts/ifcfg-*: No such file or directory grep: /etc/NetworkManager/system-connections/*.nmconnection: No such file or directory
Coiby Xu (11): add function to copy NetworkManage connection profile to the initramfs clone NM connection profiles to support legacy ifcfg automatically ask NM to wait the network device to be available stop dracut 35network-manager from running nm-initrd-generator fix error for vlan over team network interface set up kdump network by directly copying NM connection profile to initrd clean up unneeded code after copying .nmconnection to initrd reduce kdump memory consumption by not letting NetworkManager manage unneeded network interfaces reduce kdump memory consumption by only installing needed NIC drivers address the cases where a NIC has a different name in kdump kernel simplify setup_znet by copying connection profile to initrd
dracut-kdump.sh | 39 +- dracut-module-setup.sh | 483 ++++++++---------- dracut-module-setup.sh.fix | 994 +++++++++++++++++++++++++++++++++++++ kdump-lib-initramfs.sh | 14 + mkdumprd | 2 +- 5 files changed, 1243 insertions(+), 289 deletions(-) create mode 100755 dracut-module-setup.sh.fix
Each network interface is manged by a NM connection. Given a network interface name, this function will be used to copy a NetworkManager (NM) connection profile i.e. a .nmconnection file to the kdump initramfs.
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 17 +++++++++++++++++ 1 file changed, 17 insertions(+)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 85ec1aa..40ac07e 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -349,6 +349,17 @@ kdump_setup_ifname() { echo "$_ifname" }
+kdump_copy_nmconnection_file() { + local _dev _nmconnection_file_path _nmconnection_name _initrd_nmconnection_file_path + + _dev=$1 + + _nmconnection_file_path=${nmconnection_map[$_dev]} + _nmconnection_name=$(basename "$_nmconnection_file_path") + _initrd_nmconnection_file_path="/etc/NetworkManager/system-connections/$_nmconnection_name" + inst "$_nmconnection_file_path" "$_initrd_nmconnection_file_path" +} + kdump_setup_bridge() { local _netdev=$1 local _brif _dev _mac _kdumpdev @@ -537,6 +548,12 @@ kdump_install_net() { local _destaddr _srcaddr _route _netdev _conpath kdumpnic local _static _proto _ip_conf _ip_opts _ifname_opts local _znet_netdev _znet_conpath + # each netowrk interface is managed by a NM connection profile + declare -A nmconnection_map + + while IFS=: read -r _ifname _nmconn; do + nmconnection_map[$_ifname]="$_nmconn" + done <<< "$(nmcli -t -f device,filename connection show --active)"
_destaddr=$(kdump_get_remote_ip "$1") _route=$(kdump_get_ip_route "$_destaddr")
NetworkManager defaults to use keyfile i.e. .nmconnnection file but still supports reading ifcfg-* file [1]. Make use of "nmcli connection clone --temporary" to automatically convert a ifcfg-* file to a .nmconnection file.
[1] https://fedoraproject.org/wiki/Changes/NetworkManager_keyfile_instead_of_ifc...
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 54 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 40ac07e..ab7fe54 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -1,11 +1,22 @@ #!/bin/bash
+_DRACUT_KDUMP_NM_TMP_DIR="/tmp/$$-DRACUT_KDUMP_NM" + +cleanup() { + rm -rf "$_DRACUT_KDUMP_NM_TMP_DIR" +} + +# shellcheck disable=SC2154 # known issue of shellcheck https://github.com/koalaman/shellcheck/issues/1299 +trap 'ret=$?; cleanup; exit $ret;' EXIT + kdump_module_init() { # shellcheck disable=SC2154 # $initdir is a dracut variable if ! [[ -d "${initdir}/tmp" ]]; then mkdir -p "${initdir}/tmp" fi
+ mkdir -p "$_DRACUT_KDUMP_NM_TMP_DIR" + . /lib/kdump/kdump-lib.sh }
@@ -349,6 +360,47 @@ kdump_setup_ifname() { echo "$_ifname" }
+_clone_nmconnection() { + local _clone_output _name _unique_id + + _unique_id=$1 + _name=$(nmcli --get-values connection.id connection show "$_unique_id") + if _clone_output=$(nmcli connection clone --temporary "$_unique_id" "$_name"); then + sed -E -n "s/.* (.*) cloned as.*((.*)).$/\1/p" <<< "$_clone_output" + return 0 + fi + + return 1 +} + +# Clone NM connection profiles +# +# This function makes use of "nmcli clone" to automatically convert ifcfg-* +# files to Networkmanager .nmconnection connection profiles. +clone_nmconnections() { + local _dev _cloned_nmconnection_file_path _tmp_nmconnection_file_path _old_uuid _uuid + + for _dev in "${!nmconnection_map[@]}"; do + _nmconnection_file_path=${nmconnection_map[$_dev]} + _old_uuid=$(nmcli --get-values connection.uuid connection show "$_nmconnection_file_path") + + if ! _uuid=$(_clone_nmconnection "$_old_uuid"); then + derror "Failed to clone $_old_uuid" + exit 1 + fi + + _cloned_nmconnection_file_path=$(nmcli --get-values UUID,FILENAME connection show | sed -n "s/^${_uuid}://p") + _tmp_nmconnection_file_path=$_DRACUT_KDUMP_NM_TMP_DIR/$(basename "$_nmconnection_file_path") + cp "$_cloned_nmconnection_file_path" "$_tmp_nmconnection_file_path" + # change uuid back to old value in case it's refered by other connection + # profile e.g. connection.master could be interface name of the master + # device or UUID of the master connection. + sed -i -E "s/(^uuid=).*$/\1${_old_uuid}/g" "$_tmp_nmconnection_file_path" + nmconnection_map[$_dev]=$_tmp_nmconnection_file_path + nmcli connection del "$_uuid" &> >(ddebug) + done +} + kdump_copy_nmconnection_file() { local _dev _nmconnection_file_path _nmconnection_name _initrd_nmconnection_file_path
@@ -554,6 +606,7 @@ kdump_install_net() { while IFS=: read -r _ifname _nmconn; do nmconnection_map[$_ifname]="$_nmconn" done <<< "$(nmcli -t -f device,filename connection show --active)" + clone_nmconnections
_destaddr=$(kdump_get_remote_ip "$1") _route=$(kdump_get_ip_route "$_destaddr") @@ -625,6 +678,7 @@ kdump_install_net() { echo "kdumpnic=$kdumpnic" > "${initdir}/etc/cmdline.d/60kdumpnic.conf" echo "bootdev=$kdumpnic" > "${initdir}/etc/cmdline.d/70bootdev.conf" fi + }
# install etc/kdump/pre.d and /etc/kdump/post.d
During boot, it may take some time for a device driver to detect the its devices. By default, NetworkManager doesn't wait for the device to be available (this is not an issue for the 1st kernel because NM keeps running in the 1st kernel). So we need to set connection.wait-device-timeout=60000 [1] as the same to nm-initrd-generator to tell NM to wait for 60s. Otherwise dracut-kdump-capture.service would fail because network isn't ready. Note the user's custom value for wait-device-timeout is respected.
[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/issues/803#no...
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index ab7fe54..1707b82 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -410,6 +410,11 @@ kdump_copy_nmconnection_file() { _nmconnection_name=$(basename "$_nmconnection_file_path") _initrd_nmconnection_file_path="/etc/NetworkManager/system-connections/$_nmconnection_name" inst "$_nmconnection_file_path" "$_initrd_nmconnection_file_path" + + # Ask NM to wait 60s for the network device to be available + # wait-device-timeout=60000 is inserted after "[connection]" so the + # user's custom value could overwrite it + sed -i '/[connection]/a wait-device-timeout=60000' "${initdir}/$_initrd_nmconnection_file_path" }
kdump_setup_bridge() {
kexec-tools depends on dracut's 35network-manager module which will call nm-initrd-generator. We don't want nm-initrd-generator to generate connection profiles since we will copy them from 1st kernel to kdump kernel initramfs. NetworkManager >= 1.35.2 won't generate connection profiles if there's a connection dir with rd.neednet. For Fedora/RHEL, this connection dir is /etc/NetworkManager/system-connections. For the details, please refer to the NetworkManager commit 79885656d3 ("initrd: don't add a connection if there's a connection dir with rd.neednet") [1]. Before the release of NetworkManager >= 1.35.2, we need to mask /usr/libexec/nm-initrd-generator.
[1] https://gitlab.freedesktop.org/NetworkManager/NetworkManager/-/merge_request...
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 5 +++++ 1 file changed, 5 insertions(+)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 1707b82..e3b87db 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -671,6 +671,11 @@ kdump_install_net() { echo "rd.neednet" >> "${initdir}/etc/cmdline.d/50neednet.conf" fi
+ # Stop dracut 35network-manger to calling nm-initrd-generator. + # Note this line of code can be removed after NetworkManager >= 1.35.2 + # gets released. + echo > "${initdir}/usr/libexec/nm-initrd-generator" + # Save netdev used for kdump as cmdline # Whoever calling kdump_install_net() is setting up the default gateway, # ie. bootdev/kdumpnic. So don't override the setting if calling
6f9235887f7817085aabfcc67bf4a6d68e474264 ("module-setup.sh: enable vlan on team interface") skips establishing teaming network by mistake. Although it could use one of slave netifs to establish connection to transfer vmcore to remote fs, it breaks the implicit assumption of creating an identical network topology to the 1st kernel.
Fixes: 6f92358 ("module-setup.sh: enable vlan on team interface") Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index e3b87db..d5e71c3 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -508,6 +508,9 @@ kdump_setup_vlan() { elif kdump_is_bond "$_phydev"; then (kdump_setup_bond "$_phydev" "$(get_nmcli_connection_apath_by_ifname "$_phydev")") || exit 1 echo " vlan=$(kdump_setup_ifname "$_netdev"):$_phydev" > "${initdir}/etc/cmdline.d/43vlan.conf" + elif kdump_is_team "$_phydev"; then + (kdump_setup_team "$_phydev") || exit 1 + echo " vlan=$(kdump_setup_ifname "$_netdev"):$_phydev" > "${initdir}/etc/cmdline.d/43vlan.conf" else _kdumpdev="$(kdump_setup_ifname "$_phydev")" echo " vlan=$(kdump_setup_ifname "$_netdev"):$_kdumpdev ifname=$_kdumpdev:$_netmac" > "${initdir}/etc/cmdline.d/43vlan.conf"
This patch setup kdump network by directly copying NM connection profile(s) for different network setup including bond, bridge, vlan, and team. For vlan network, rename phydev to parent_netif to improve code readability.
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 90 ++++++++++-------------------------------- 1 file changed, 20 insertions(+), 70 deletions(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index d5e71c3..3e2eb9f 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -419,102 +419,55 @@ kdump_copy_nmconnection_file() {
kdump_setup_bridge() { local _netdev=$1 - local _brif _dev _mac _kdumpdev + local _dev for _dev in "/sys/class/net/$_netdev/brif/"*; do [[ -e $_dev ]] || continue _dev=${_dev##*/} - _kdumpdev=$_dev if kdump_is_bond "$_dev"; then - (kdump_setup_bond "$_dev" "$(get_nmcli_connection_apath_by_ifname "$_dev")") || exit 1 + (kdump_setup_bond "$_dev") || exit 1 elif kdump_is_team "$_dev"; then kdump_setup_team "$_dev" elif kdump_is_vlan "$_dev"; then kdump_setup_vlan "$_dev" - else - _mac=$(kdump_get_mac_addr "$_dev") - _kdumpdev=$(kdump_setup_ifname "$_dev") - echo -n " ifname=$_kdumpdev:$_mac" >> "${initdir}/etc/cmdline.d/41bridge.conf" fi - _brif+="$_kdumpdev," + kdump_copy_nmconnection_file "$_dev" done - echo " bridge=$_netdev:${_brif%,}" >> "${initdir}/etc/cmdline.d/41bridge.conf" }
-# drauct takes bond=<bondname>[:<bondslaves>:[:<options>]] syntax to parse -# bond. For example: -# bond=bond0:eth0,eth1:mode=balance-rr kdump_setup_bond() { local _netdev="$1" - local _conpath="$2" - local _dev _mac _slaves _kdumpdev _bondoptions + local _dev + for _dev in $(< "/sys/class/net/$_netdev/bonding/slaves"); do - _mac=$(kdump_get_perm_addr "$_dev") - _kdumpdev=$(kdump_setup_ifname "$_dev") - echo -n " ifname=$_kdumpdev:$_mac" >> "${initdir}/etc/cmdline.d/42bond.conf" - _slaves+="$_kdumpdev," + kdump_copy_nmconnection_file "$_dev" done - echo -n " bond=$_netdev:${_slaves%,}" >> "${initdir}/etc/cmdline.d/42bond.conf" - - _bondoptions=$(get_nmcli_field_by_conpath "bond.options" "$_conpath") - - if [[ -z $_bondoptions ]]; then - dwarning "Failed to get bond configuration via nmlci output. Now try sourcing ifcfg script." - source_ifcfg_file "$_netdev" - _bondoptions="$(echo "$BONDING_OPTS" | xargs echo | tr " " ",")" - fi - - if [[ -z $_bondoptions ]]; then - derror "Get empty bond options" - exit 1 - fi - - echo ":$_bondoptions" >> "${initdir}/etc/cmdline.d/42bond.conf" }
kdump_setup_team() { local _netdev=$1 - local _dev _mac _slaves _kdumpdev + local _dev for _dev in $(teamnl "$_netdev" ports | awk -F':' '{print $2}'); do - _mac=$(kdump_get_perm_addr "$_dev") - _kdumpdev=$(kdump_setup_ifname "$_dev") - echo -n " ifname=$_kdumpdev:$_mac" >> "${initdir}/etc/cmdline.d/44team.conf" - _slaves+="$_kdumpdev," + kdump_copy_nmconnection_file "$_dev" done - echo " team=$_netdev:${_slaves%,}" >> "${initdir}/etc/cmdline.d/44team.conf" - #Buggy version teamdctl outputs to stderr! - #Try to use the latest version of teamd. - if ! teamdctl "$_netdev" config dump > "${initdir}/tmp/$$-$_netdev.conf"; then - derror "teamdctl failed." - exit 1 - fi - inst_dir /etc/teamd - inst_simple "${initdir}/tmp/$$-$_netdev.conf" "/etc/teamd/$_netdev.conf" - rm -f "${initdir}/tmp/$$-$_netdev.conf" }
kdump_setup_vlan() { local _netdev=$1 - local _phydev - local _netmac - local _kdumpdev + local _parent_netif
- _phydev="$(awk '/^Device:/{print $2}' /proc/net/vlan/"$_netdev")" - _netmac="$(kdump_get_mac_addr "$_phydev")" + _parent_netif="$(awk '/^Device:/{print $2}' /proc/net/vlan/"$_netdev")"
#Just support vlan over bond and team - if kdump_is_bridge "$_phydev"; then + if kdump_is_bridge "$_parent_netif"; then derror "Vlan over bridge is not supported!" exit 1 - elif kdump_is_bond "$_phydev"; then - (kdump_setup_bond "$_phydev" "$(get_nmcli_connection_apath_by_ifname "$_phydev")") || exit 1 - echo " vlan=$(kdump_setup_ifname "$_netdev"):$_phydev" > "${initdir}/etc/cmdline.d/43vlan.conf" - elif kdump_is_team "$_phydev"; then - (kdump_setup_team "$_phydev") || exit 1 - echo " vlan=$(kdump_setup_ifname "$_netdev"):$_phydev" > "${initdir}/etc/cmdline.d/43vlan.conf" - else - _kdumpdev="$(kdump_setup_ifname "$_phydev")" - echo " vlan=$(kdump_setup_ifname "$_netdev"):$_kdumpdev ifname=$_kdumpdev:$_netmac" > "${initdir}/etc/cmdline.d/43vlan.conf" + elif kdump_is_bond "$_parent_netif"; then + (kdump_setup_bond "$_parent_netif") || exit 1 + elif kdump_is_team "$_parent_netif"; then + (kdump_setup_team "$_parent_netif") || exit 1 fi + + kdump_copy_nmconnection_file "$_parent_netif" }
# find online znet device @@ -606,7 +559,7 @@ kdump_get_remote_ip() { # $1: destination host kdump_install_net() { local _destaddr _srcaddr _route _netdev _conpath kdumpnic - local _static _proto _ip_conf _ip_opts _ifname_opts + local _static _proto _ip_conf _ip_opts local _znet_netdev _znet_conpath # each netowrk interface is managed by a NM connection profile declare -A nmconnection_map @@ -621,7 +574,6 @@ kdump_install_net() { _srcaddr=$(kdump_get_ip_route_field "$_route" "src") _netdev=$(kdump_get_ip_route_field "$_route" "dev") _conpath=$(get_nmcli_connection_apath_by_ifname "$_netdev") - _netmac=$(kdump_get_mac_addr "$_netdev") kdumpnic=$(kdump_setup_ifname "$_netdev")
_znet_netdev=$(find_online_znet_device) @@ -657,15 +609,13 @@ kdump_install_net() { if kdump_is_bridge "$_netdev"; then kdump_setup_bridge "$_netdev" elif kdump_is_bond "$_netdev"; then - (kdump_setup_bond "$_netdev" "$_conpath") || exit 1 + (kdump_setup_bond "$_netdev") || exit 1 elif kdump_is_team "$_netdev"; then kdump_setup_team "$_netdev" elif kdump_is_vlan "$_netdev"; then kdump_setup_vlan "$_netdev" - else - _ifname_opts=" ifname=$kdumpnic:$_netmac" - echo "$_ifname_opts" >> "$_ip_conf" fi + kdump_copy_nmconnection_file "$_netdev"
kdump_setup_dns "$_netdev" "$_conpath"
Clean up the related code to build up dracut cmdline parameter such rd.route, ip and etc. And there is no need to to setup dns when copying .nmconnection directly to initrd either.
Note the bootdev dracut command line parameter is only used by dracut's 35network-legacy and network-manager doesn't use it, remove related code as well.
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 148 +---------------------------------------- 1 file changed, 2 insertions(+), 146 deletions(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 3e2eb9f..b72a8e1 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -98,39 +98,6 @@ source_ifcfg_file() { fi }
-kdump_setup_dns() { - local _netdev="$1" - local _conpath="$2" - local _nameserver _dns _tmp array - local _dnsfile=${initdir}/etc/cmdline.d/42dns.conf - - _tmp=$(get_nmcli_field_by_conpath "IP4.DNS" "$_conpath") - # shellcheck disable=SC2206 - array=(${_tmp//|/ }) - if [[ ${array[*]} ]]; then - for _dns in "${array[@]}"; do - echo "nameserver=$_dns" >> "$_dnsfile" - done - else - dwarning "Failed to get DNS info via nmcli output. Now try sourcing ifcfg script" - source_ifcfg_file "$_netdev" - [[ -n $DNS1 ]] && echo "nameserver=$DNS1" > "$_dnsfile" - [[ -n $DNS2 ]] && echo "nameserver=$DNS2" >> "$_dnsfile" - fi - - while read -r content; do - _nameserver=$(echo "$content" | grep ^nameserver) - [[ -z $_nameserver ]] && continue - - _dns=$(echo "$_nameserver" | awk '{print $2}') - [[ -z $_dns ]] && continue - - if [[ ! -f $_dnsfile ]] || ! grep -q "$_dns" "$_dnsfile"; then - echo "nameserver=$_dns" >> "$_dnsfile" - fi - done < "/etc/resolv.conf" -} - # $1: repeat times # $2: string to be repeated # $3: separator @@ -241,89 +208,6 @@ cal_netmask_by_prefix() { fi }
-#$1: netdev name -#$2: srcaddr -#if it use static ip echo it, or echo null -kdump_static_ip() { - local _netdev="$1" _srcaddr="$2" kdumpnic="$3" _ipv6_flag - local _netmask _gateway _ipaddr _target _nexthop _prefix - - _ipaddr=$(ip addr show dev "$_netdev" permanent | awk "/ $_srcaddr/.* /{print $2}") - - if is_ipv6_address "$_srcaddr"; then - _ipv6_flag="-6" - fi - - if [[ -n $_ipaddr ]]; then - _gateway=$(ip $_ipv6_flag route list dev "$_netdev" \ - | awk '/^default /{print $3}' | head -n 1) - - if [[ "x" != "x"$_ipv6_flag ]]; then - # _ipaddr="2002::56ff:feb6:56d5/64", _netmask is the number after "/" - _netmask=${_ipaddr#*/} - _srcaddr="[$_srcaddr]" - _gateway="[$_gateway]" - else - _prefix=$(cut -d'/' -f2 <<< "$_ipaddr") - if ! _netmask=$(cal_netmask_by_prefix "$_prefix" "$_ipv6_flag"); then - derror "Failed to calculate netmask for $_ipaddr" - exit 1 - fi - fi - echo -n "${_srcaddr}::${_gateway}:${_netmask}::" - fi - - while read -r _route; do - _target=$(echo "$_route" | awk '{print $1}') - _nexthop=$(echo "$_route" | awk '{print $3}') - if [[ "x" != "x"$_ipv6_flag ]]; then - _target="[$_target]" - _nexthop="[$_nexthop]" - fi - echo "rd.route=$_target:$_nexthop:$kdumpnic" - done >> "${initdir}/etc/cmdline.d/45route-static.conf" \ - < <(/sbin/ip $_ipv6_flag route show | grep -v default \ - | grep ".*via.* $_netdev " | grep -v "^[[:space:]]*nexthop") - - kdump_handle_mulitpath_route "$_netdev" "$_srcaddr" "$kdumpnic" -} - -kdump_handle_mulitpath_route() { - local _netdev="$1" _srcaddr="$2" kdumpnic="$3" _ipv6_flag - local _target _nexthop _route _weight _max_weight _rule - - if is_ipv6_address "$_srcaddr"; then - _ipv6_flag="-6" - fi - - while IFS="" read -r _route; do - if [[ $_route =~ [[:space:]]+nexthop ]]; then - _route=${_route##[[:space:]]} - # Parse multipath route, using previous _target - [[ $_target == 'default' ]] && continue - [[ $_route =~ .*via.*\ $_netdev ]] || continue - - _weight=$(echo "$_route" | cut -d ' ' -f7) - if [[ $_weight -gt $_max_weight ]]; then - _nexthop=$(echo "$_route" | cut -d ' ' -f3) - _max_weight=$_weight - if [[ "x" != "x"$_ipv6_flag ]]; then - _rule="rd.route=[$_target]:[$_nexthop]:$kdumpnic" - else - _rule="rd.route=$_target:$_nexthop:$kdumpnic" - fi - fi - else - [[ -n $_rule ]] && echo "$_rule" - _target=$(echo "$_route" | cut -d ' ' -f1) - _rule="" _max_weight=0 _weight=0 - fi - done >> "${initdir}/etc/cmdline.d/45route-static.conf" \ - <<< "$(/sbin/ip $_ipv6_flag route show)" - - [[ -n $_rule ]] && echo "$_rule" >> "${initdir}/etc/cmdline.d/45route-static.conf" -} - kdump_get_mac_addr() { cat "/sys/class/net/$1/address" } @@ -558,8 +442,7 @@ kdump_get_remote_ip() { # initramfs accessing giving destination # $1: destination host kdump_install_net() { - local _destaddr _srcaddr _route _netdev _conpath kdumpnic - local _static _proto _ip_conf _ip_opts + local _destaddr _route _netdev kdumpnic local _znet_netdev _znet_conpath # each netowrk interface is managed by a NM connection profile declare -A nmconnection_map @@ -571,9 +454,7 @@ kdump_install_net() {
_destaddr=$(kdump_get_remote_ip "$1") _route=$(kdump_get_ip_route "$_destaddr") - _srcaddr=$(kdump_get_ip_route_field "$_route" "src") _netdev=$(kdump_get_ip_route_field "$_route" "dev") - _conpath=$(get_nmcli_connection_apath_by_ifname "$_netdev") kdumpnic=$(kdump_setup_ifname "$_netdev")
_znet_netdev=$(find_online_znet_device) @@ -585,27 +466,6 @@ kdump_install_net() { fi fi
- _static=$(kdump_static_ip "$_netdev" "$_srcaddr" "$kdumpnic") - if [[ -n $_static ]]; then - _proto=none - elif is_ipv6_address "$_srcaddr"; then - _proto=auto6 - else - _proto=dhcp - fi - - _ip_conf="${initdir}/etc/cmdline.d/40ip.conf" - _ip_opts=" ip=${_static}$kdumpnic:${_proto}" - - # dracut doesn't allow duplicated configuration for same NIC, even they're exactly the same. - # so we have to avoid adding duplicates - # We should also check /proc/cmdline for existing ip=xx arg. - # For example, iscsi boot will specify ip=xxx arg in cmdline. - if [[ ! -f $_ip_conf ]] || ! grep -q "$_ip_opts" "$_ip_conf" \ - && ! grep -q "ip=[^[:space:]]*$_netdev" /proc/cmdline; then - echo "$_ip_opts" >> "$_ip_conf" - fi - if kdump_is_bridge "$_netdev"; then kdump_setup_bridge "$_netdev" elif kdump_is_bond "$_netdev"; then @@ -617,8 +477,6 @@ kdump_install_net() { fi kdump_copy_nmconnection_file "$_netdev"
- kdump_setup_dns "$_netdev" "$_conpath" - if [[ ! -f ${initdir}/etc/cmdline.d/50neednet.conf ]]; then # network-manager module needs this parameter echo "rd.neednet" >> "${initdir}/etc/cmdline.d/50neednet.conf" @@ -636,10 +494,8 @@ kdump_install_net() { # the default gate way for network dump, eth1 in the fence kdump path will # call kdump_install_net again and we don't want eth1 to be the default # gateway. - if [[ ! -f ${initdir}/etc/cmdline.d/60kdumpnic.conf ]] \ - && [[ ! -f ${initdir}/etc/cmdline.d/70bootdev.conf ]]; then + if [[ ! -f ${initdir}/etc/cmdline.d/60kdumpnic.conf ]]; then echo "kdumpnic=$kdumpnic" > "${initdir}/etc/cmdline.d/60kdumpnic.conf" - echo "bootdev=$kdumpnic" > "${initdir}/etc/cmdline.d/70bootdev.conf" fi
}
By default, NetworkManger will manage all the network interfaces and try to set interface IFF_UP to get carrier state. Regardless of whether the network interface is connected to a cable or not, the NIC driver will allocate memory resources for e.g. ring buffers when setting IFF_UP. This could be a waste of memory. For example it's found i40e consumes ~15GB on a power machine. On this machine, i40e manages four interfaces but only one interface is valid. This patch use "managed=false" to tell NetworkManager to not manage network interfaces that are not needed by kdump by putting /etc/NetworkManager/conf.d/10-kdump-netif.conf in the initramfs.
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 34 +++++++++++++++++++++++++++++++++- 1 file changed, 33 insertions(+), 1 deletion(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index b72a8e1..f861d3b 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -1,6 +1,7 @@ #!/bin/bash
_DRACUT_KDUMP_NM_TMP_DIR="/tmp/$$-DRACUT_KDUMP_NM" +_TMP_KDUMP_NETIFS="$_DRACUT_KDUMP_NM_TMP_DIR/kdump_netifs"
cleanup() { rm -rf "$_DRACUT_KDUMP_NM_TMP_DIR" @@ -16,6 +17,7 @@ kdump_module_init() { fi
mkdir -p "$_DRACUT_KDUMP_NM_TMP_DIR" + echo -n > "$_TMP_KDUMP_NETIFS"
. /lib/kdump/kdump-lib.sh } @@ -299,6 +301,31 @@ kdump_copy_nmconnection_file() { # wait-device-timeout=60000 is inserted after "[connection]" so the # user's custom value could overwrite it sed -i '/[connection]/a wait-device-timeout=60000' "${initdir}/$_initrd_nmconnection_file_path" + + echo -n " $_dev" >> "$_TMP_KDUMP_NETIFS" +} + +kdump_install_nm_netif_allowlist() { + local _netif _except_netif _netif_allowlist _netif_allowlist_nm_conf + + for _netif in $1; do + _per_mac=$(kdump_get_perm_addr "$_netif") + if [[ "$_per_mac" != 'not set' ]]; then + _except_netif="mac:$_per_mac" + else + _except_netif="interface-name:$_netif" + fi + _netif_allowlist="${_netif_allowlist}except:${_except_netif};" + done + + _netif_allowlist_nm_conf=$_DRACUT_KDUMP_NM_TMP_DIR/netif_allowlist_nm_conf + cat <<- EOF > "$_netif_allowlist_nm_conf" + [device-others] + match-device=${_netif_allowlist} + managed=false + EOF + + inst "$_netif_allowlist_nm_conf" "/etc/NetworkManager/conf.d/10-kdump-netif.conf" }
kdump_setup_bridge() { @@ -915,7 +942,7 @@ remove_cpu_online_rule() { }
install() { - local arch + local arch _netifs
kdump_module_init kdump_install_conf @@ -976,6 +1003,11 @@ install() { inst "ip" fi
+ _netifs=$(cat "$_TMP_KDUMP_NETIFS") + if [[ -n "$_netifs" ]]; then + kdump_install_nm_netif_allowlist "$_netifs" + fi + # For the lvm type target under kdump, in /etc/lvm/lvm.conf we can # safely replace "reserved_memory=XXXX"(default value is 8192) with # "reserved_memory=1024" to lower memory pressure under kdump. We do
Even after having asked NM to stop managing a unneeded NIC, a NIC driver may still waste memory. For example, mlx5_core uses a substantial amount of memory during driver initialization,
======== Report format module_summary: ======== Module mlx5_core using 350.2MB (89650 pages), peak allocation 367.4MB (94056 pages) Module squashfs using 13.1MB (3360 pages), peak allocation 13.1MB (3360 pages) Module overlay using 2.1MB (550 pages), peak allocation 2.2MB (555 pages) Module dns_resolver using 0.9MB (219 pages), peak allocation 5.2MB (1338 pages) Module mlxfw using 0.7MB (172 pages), peak allocation 5.3MB (1349 pages) ======== Report format module_summary END ========
======== Report format module_top: ======== Top stack usage of module mlx5_core: (null) Pages: 89650 (peak: 94056) ret_from_fork (0xffffda088b4165f8) Pages: 60007 (peak: 60007) kthread (0xffffda088b4bd7e4) Pages: 60007 (peak: 60007) worker_thread (0xffffda088b4b48d0) Pages: 60007 (peak: 60007) process_one_work (0xffffda088b4b3f40) Pages: 60007 (peak: 60007) work_for_cpu_fn (0xffffda088b4aef00) Pages: 53906 (peak: 53906) local_pci_probe (0xffffda088b9e1e44) Pages: 53906 (peak: 53906) probe_one mlx5_core (0xffffda084f899cc8) Pages: 53518 (peak: 53518) mlx5_init_one mlx5_core (0xffffda084f8994ac) Pages: 49756 (peak: 49756) mlx5_function_setup.constprop.0 mlx5_core (0xffffda084f899100) Pages: 44434 (eak: 44434) mlx5_satisfy_startup_pages mlx5_core (0xffffda084f8a4f24) Pages: 44434 (peak: 44434) mlx5_function_setup.constprop.0 mlx5_core (0xffffda084f899078) Pages: 5285 (peak: 5285) mlx5_cmd_init mlx5_core (0xffffda084f89e414) Pages: 4818 (peak: 4818) mlx5_alloc_cmd_msg mlx5_core (0xffffda084f89aaa0) Pages: 4403 (peak: 4403)
This memory consumption is completely unnecessary when kdump doesn't need this NIC. Only install needed NIC drivers to prevent this kind of waste.
Note 1. this patch depends on [1] to ask dracut to not install NIC drivers. 2. "ethtool -i" somehow fails to get the vlan driver 3. team.ko doesn't depend on the team mode drivers so we need to install the team mode drivers manually.
[1] https://github.com/dracutdevs/dracut/pull/1789
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 31 +++++++++++++++++++++++++++++++ mkdumprd | 2 +- 2 files changed, 32 insertions(+), 1 deletion(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index f861d3b..01df7b7 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -328,6 +328,36 @@ kdump_install_nm_netif_allowlist() { inst "$_netif_allowlist_nm_conf" "/etc/NetworkManager/conf.d/10-kdump-netif.conf" }
+_get_nic_driver() { + ethtool -i "$1" | sed -n -E "s/driver: (.*)/\1/p" +} + +kdump_install_nic_driver() { + local _netif _driver _drivers + + _drivers=() + + for _netif in $1; do + _driver=$(_get_nic_driver "$_netif") + if [[ -z $_driver ]]; then + derror "Failed to get the driver of $_netif" + exit 1 + fi + + if [[ $_driver == "802.1Q VLAN Support" ]]; then + # ethtool somehow doesn't return the driver name for a VLAN NIC + _driver=8021q + elif [[ $_driver == "team" ]]; then + # install the team mode drivers like team_mode_roundrobin.ko as well + _driver='=drivers/net/team' + fi + + _drivers+=("$_driver") + done + + instmods "${_drivers[@]}" +} + kdump_setup_bridge() { local _netdev=$1 local _dev @@ -1006,6 +1036,7 @@ install() { _netifs=$(cat "$_TMP_KDUMP_NETIFS") if [[ -n "$_netifs" ]]; then kdump_install_nm_netif_allowlist "$_netifs" + kdump_install_nic_driver "$_netifs" fi
# For the lvm type target under kdump, in /etc/lvm/lvm.conf we can diff --git a/mkdumprd b/mkdumprd index bcbf47b..1a3747b 100644 --- a/mkdumprd +++ b/mkdumprd @@ -27,7 +27,7 @@ SAVE_PATH=$(get_save_path) OVERRIDE_RESETTABLE=0
extra_modules="" -dracut_args=(--add kdumpbase --quiet --hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode strict -o "plymouth resume ifcfg earlykdump") +dracut_args=(--add kdumpbase --quiet --hostonly --hostonly-cmdline --hostonly-i18n --hostonly-mode strict --hostonly-nics '' -o "plymouth resume ifcfg earlykdump")
MKDUMPRD_TMPDIR="$(mktemp -d -t mkdumprd.XXXXXX)" [ -d "$MKDUMPRD_TMPDIR" ] || perror_exit "dracut: mktemp -p -d -t dracut.XXXXXX failed."
A NIC may get a different name in the kdump kernel from 1st kernel in cases like, - kernel assigned network interface names are not persistent e.g. [1] - there is an udev rule to rename the NIC in the 1st kernel but the kdump initrd may not have that rule e.g. [2]
If NM tries to match a NIC with a connection profile based on NIC name i.e. connection.interface-name, it will fail the above bases. A simple solution is to ask NM to match a connection profile by MAC address. Note we don't need to do this for user-created NICs like vlan, bridge and bond.
An remaining issue is passing the name of a NIC via the kdumpnic dracut command line parameter which requires passing ifname=<interface>:<MAC> to have fixed NIC name. But we can simply drop this requirement. kdumpnic is needed because kdump needs to get the IP by NIC name and use the IP to created a dumping folder named "{IP}-{DATE}". We can simply pass the IP to the kdump kernel directly via a new dracut command line parameter kdumpip instead. In addition to the benefit of simplifying the code, there are other three benefits brought by this approach, - make use of whatever network to transfer the vmcore. Because as long as we have the network to we don't care which NIC is active. - if obtained IP in the kdump kernel is different from the one in the 1st kernel. "{IP}-{DATE}" would better tell where the dumped vmcore comes from. - without passing ifname=<interface>:<MAC> to kdump initrd, the issue of there are two interfaces with the same MAC address for Azure Hyper-V NIC SR-IOV [3] is resolved automatically.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1121778 [2] https://bugzilla.redhat.com/show_bug.cgi?id=810107 [3] https://bugzilla.redhat.com/show_bug.cgi?id=1962421
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-kdump.sh | 39 ++++++++++++++------------ dracut-module-setup.sh | 62 ++++++++++++++++-------------------------- kdump-lib-initramfs.sh | 14 ++++++++++ 3 files changed, 59 insertions(+), 56 deletions(-)
diff --git a/dracut-kdump.sh b/dracut-kdump.sh index 259a010..3259650 100755 --- a/dracut-kdump.sh +++ b/dracut-kdump.sh @@ -485,25 +485,28 @@ save_vmcore_dmesg_ssh()
get_host_ip() { - if is_nfs_dump_target || is_ssh_dump_target; then - kdumpnic=$(getarg kdumpnic=) - if [ -z "$kdumpnic" ]; then - derror "failed to get kdumpnic!" - return 1 - fi - if ! kdumphost=$(ip addr show dev "$kdumpnic" | grep '[ ]*inet'); then - derror "wrong kdumpnic: $kdumpnic" - return 1 - fi - kdumphost=$(echo "$kdumphost" | head -n 1 | awk '{print $2}') - kdumphost="${kdumphost%%/*}" - if [ -z "$kdumphost" ]; then - derror "wrong kdumpnic: $kdumpnic" - return 1 - fi - HOST_IP=$kdumphost + + if ! is_nfs_dump_target && ! is_ssh_dump_target; then + return 0 fi - return 0 + + _kdump_remote_ip=$(getarg kdump_remote_ip=) + + if [ -z "$_kdump_remote_ip" ]; then + derror "failed to get remote IP address!" + return 1 + fi + _route=$(kdump_get_ip_route "$_kdump_remote_ip") + _netdev=$(kdump_get_ip_route_field "$_route" "dev") + + if ! _kdumpip=$(ip addr show dev "$_netdev" | grep '[ ]*inet'); then + derror "Failed to get IP of $_netdev" + return 1 + fi + + _kdumpip=$(echo "$_kdumpip" | head -n 1 | awk '{print $2}') + _kdumpip="${_kdumpip%%/*}" + HOST_IP=$_kdumpip }
read_kdump_confs() diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 01df7b7..217bed1 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -226,26 +226,6 @@ kdump_get_perm_addr() { fi }
-# Prefix kernel assigned names with "kdump-". EX: eth0 -> kdump-eth0 -# Because kernel assigned names are not persistent between 1st and 2nd -# kernel. We could probably end up with eth0 being eth1, eth0 being -# eth1, and naming conflict happens. -kdump_setup_ifname() { - local _ifname - - # If ifname already has 'kdump-' prefix, we must be switching from - # fadump to kdump. Skip prefixing 'kdump-' in this case as adding - # another prefix may truncate the ifname. Since an ifname with - # 'kdump-' is already persistent, this should be fine. - if [[ $1 =~ eth* ]] && [[ ! $1 =~ ^kdump-* ]]; then - _ifname="kdump-$1" - else - _ifname="$1" - fi - - echo "$_ifname" -} - _clone_nmconnection() { local _clone_output _name _unique_id
@@ -259,10 +239,25 @@ _clone_nmconnection() { return 1 }
+_match_nmconnection_by_mac() { + local _unique_id _dev _mac _mac_field + + _unique_id=$1 + _dev=$2 + + _mac=$(kdump_get_perm_addr "$_dev") + if [[ $_mac != 'not set' ]]; then + _mac_field=$(nmcli --get-values connection.type connection show "$_unique_id").mac-address + nmcli connection modify --temporary "$_unique_id" "$_mac_field" "$_mac" + nmcli connection modify --temporary "$_unique_id" "connection.interface-name" "" + fi +} + # Clone NM connection profiles # # This function makes use of "nmcli clone" to automatically convert ifcfg-* -# files to Networkmanager .nmconnection connection profiles. +# files to Networkmanager .nmconnection connection profiles. It also uses +# "nmcli modify" to modify or remove properties clone_nmconnections() { local _dev _cloned_nmconnection_file_path _tmp_nmconnection_file_path _old_uuid _uuid
@@ -275,6 +270,10 @@ clone_nmconnections() { exit 1 fi
+ # For physical NIC i.e. non-user created NIC, ask NM to match a + # connection profile based on MAC address + _match_nmconnection_by_mac "$_uuid" "$_dev" + _cloned_nmconnection_file_path=$(nmcli --get-values UUID,FILENAME connection show | sed -n "s/^${_uuid}://p") _tmp_nmconnection_file_path=$_DRACUT_KDUMP_NM_TMP_DIR/$(basename "$_nmconnection_file_path") cp "$_cloned_nmconnection_file_path" "$_tmp_nmconnection_file_path" @@ -469,19 +468,6 @@ kdump_setup_znet() { echo "rd.znet=${NETTYPE},${SUBCHANNELS},${_options} rd.znet_ifname=$_netdev:${SUBCHANNELS}" > "${initdir}/etc/cmdline.d/30znet.conf" }
-kdump_get_ip_route() { - local _route - if ! _route=$(/sbin/ip -o route get to "$1" 2>&1); then - derror "Bad kdump network destination: $1" - exit 1 - fi - echo "$_route" -} - -kdump_get_ip_route_field() { - echo "$1" | sed -n -e "s/^.*<$2>\s+(\S+).*$/\1/p" -} - kdump_get_remote_ip() { local _remote _remote_temp _remote=$(get_remote_host "$1") @@ -499,7 +485,7 @@ kdump_get_remote_ip() { # initramfs accessing giving destination # $1: destination host kdump_install_net() { - local _destaddr _route _netdev kdumpnic + local _destaddr _route _netdev local _znet_netdev _znet_conpath # each netowrk interface is managed by a NM connection profile declare -A nmconnection_map @@ -512,7 +498,6 @@ kdump_install_net() { _destaddr=$(kdump_get_remote_ip "$1") _route=$(kdump_get_ip_route "$_destaddr") _netdev=$(kdump_get_ip_route_field "$_route" "dev") - kdumpnic=$(kdump_setup_ifname "$_netdev")
_znet_netdev=$(find_online_znet_device) if [[ -n $_znet_netdev ]]; then @@ -532,6 +517,7 @@ kdump_install_net() { elif kdump_is_vlan "$_netdev"; then kdump_setup_vlan "$_netdev" fi + kdump_copy_nmconnection_file "$_netdev"
if [[ ! -f ${initdir}/etc/cmdline.d/50neednet.conf ]]; then @@ -551,8 +537,8 @@ kdump_install_net() { # the default gate way for network dump, eth1 in the fence kdump path will # call kdump_install_net again and we don't want eth1 to be the default # gateway. - if [[ ! -f ${initdir}/etc/cmdline.d/60kdumpnic.conf ]]; then - echo "kdumpnic=$kdumpnic" > "${initdir}/etc/cmdline.d/60kdumpnic.conf" + if [[ ! -f ${initdir}/etc/cmdline.d/60kdumpip.conf ]]; then + echo "kdump_remote_ip=$_destaddr" > "${initdir}/etc/cmdline.d/60kdumpip.conf" fi
} diff --git a/kdump-lib-initramfs.sh b/kdump-lib-initramfs.sh index 2d37156..b39d41e 100755 --- a/kdump-lib-initramfs.sh +++ b/kdump-lib-initramfs.sh @@ -133,3 +133,17 @@ is_fs_dump_target() { [ -n "$(kdump_get_conf_val "ext[234]|xfs|btrfs|minix")" ] } + +kdump_get_ip_route() +{ + if ! _route=$(/sbin/ip -o route get to "$1" 2>&1); then + derror "Bad kdump network destination: $1" + exit 1 + fi + echo "$_route" +} + +kdump_get_ip_route_field() +{ + echo "$1" | sed -n -e "s/^.*<$2>\s+(\S+).*$/\1/p" +}
/usr/lib/udev/ccw_init [1] shipped by s390utils extracts the values of SUBCHANNELS, NETTYPE and LAYER2 from /etc/sysconfig/network-scripts/ifcfg-* or /etc/NetworkManager/system-connections/*.nmconnection to activate znet network device. If the connection profile is copied to initrd, there is no need to set up the "rd.znet" dracut cmdline parameter.
There are two cases addressed by this commit, 1. znet network interface is a slave of bonding/teaming/vlan/bridging network. The connection profile has been copied to initrd by kdump_copy_nmconnection_file and it contains the info needed by ccw_init. 2. znet network interface is a slave of bonding/teaming/vlan/bridging network. The corresponding ifcfg-*/*.nmconnection file may not contain info like SUBCHANNELS [2]. In this case, copy the ifcfg-*/*.nmconnection file that has this info to the kdump initrd. Also to prevent the copied connection profile from being chosen by NM, set connection.autoconnect=false for this connection profile.
Note 1. ccw_init doesn't care if SUBCHANNELS, NETTYPE and LAYER2 comes from an active NM profile or not. If an inactive NM profile contains this info, it needs to be copied to the kdump initrd as well. 2. "rd.znet_ifname=$_netdev:${SUBCHANNELS}" is no needed because now there is no renaming of s390x network interfaces when reusing NetworkManager profiles. rd.znet_ifname was introduced in commit ce0305d ("Add a new option 'rd.znet_ifname' in order to use it in udev rules") to address the special case of non-persistent MAC address by renaming a network interface by SUBCHANNELS.
[1] https://src.fedoraproject.org/rpms/s390utils/blob/rawhide/f/ccw_init [2] https://bugzilla.redhat.com/show_bug.cgi?id=2064708
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 70 +-- dracut-module-setup.sh.fix | 994 +++++++++++++++++++++++++++++++++++++ 2 files changed, 1031 insertions(+), 33 deletions(-) create mode 100755 dracut-module-setup.sh.fix
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 217bed1..f8f4c53 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -438,34 +438,46 @@ find_online_znet_device() { echo -n "$ifname" }
-# setup s390 znet cmdline -# $1: netdev (ifname) -# $2: nmcli connection path +_find_znet_nmconnection() { + LANG=C grep -s -E -i -l \ + "^s390-subchannels=([0-9].[0-9].[a-f0-9]+;){0,2}" \ + "$1"/*.nmconnection | LC_ALL=C sed -e "$2" +} + +# setup s390 znet +# +# Note part of code is extracted from ccw_init provided by s390utils kdump_setup_znet() { - local _netdev="$1" - local _conpath="$2" - local s390_prefix="802-3-ethernet.s390-" - local _options="" - local NETTYPE - local SUBCHANNELS - - NETTYPE=$(get_nmcli_field_by_conpath "${s390_prefix}nettype" "$_conpath") - SUBCHANNELS=$(get_nmcli_field_by_conpath "${s390_prefix}subchannels" "$_conpath") - _options=$(get_nmcli_field_by_conpath "${s390_prefix}options" "$_conpath") - - if [[ -z $NETTYPE || -z $SUBCHANNELS || -z $_options ]]; then - dwarning "Failed to get znet configuration via nmlci output. Now try sourcing ifcfg script." - source_ifcfg_file "$_netdev" - for i in $OPTIONS; do - _options=${_options},$i - done + local _config_file _unique_name _NM_conf_dir + local __sed_discard_ignored_files='/(~|.bak|.old|.orig|.rpmnew|.rpmorig|.rpmsave)$/d' + + _NM_conf_dir="/etc/NetworkManager/system-connections" + + _config_file=$(_find_znet_nmconnection "$initdir/$_NM_conf_dir" "$__sed_discard_ignored_files") + if [[ -n "$_config_file" ]]; then + ddebug "$_config_file has already contained the znet config" + return fi
- if [[ -z $NETTYPE || -z $SUBCHANNELS || -z $_options ]]; then - exit 1 + _config_file=$(LANG=C grep -s -E -i -l \ + "^[[:space:]]*SUBCHANNELS=['"]?([0-9].[0-9].[a-f0-9]+,){0,2}" \ + /etc/sysconfig/network-scripts/ifcfg-* \ + | LC_ALL=C sed -e "$__sed_discard_ignored_files") + + if [[ -n "$_config_file" ]]; then + LANG=c grep 'SUBCHANNELS|NETTYPE|OPTIONS' "$_config_file" > "$initdir/$_config_file" + else + _config_file=$(_find_znet_nmconnection "$_NM_conf_dir" "$__sed_discard_ignored_files") + fi + + if [[ -n "$_config_file" ]]; then + _unique_name=$(cat /proc/sys/kernel/random/uuid) + nmcli connection clone --temporary "$_config_file" "$_unique_name" &> >(ddebug) + nmcli connection modify --temporary "$_unique_name" connection.autoconnect false + inst "/run/NetworkManager/system-connections/${_unique_name}.nmconnection" "${_NM_conf_dir}/${_unique_name}.nmconnection" + nmcli connection del "$_unique_name" &> >(ddebug) fi
- echo "rd.znet=${NETTYPE},${SUBCHANNELS},${_options} rd.znet_ifname=$_netdev:${SUBCHANNELS}" > "${initdir}/etc/cmdline.d/30znet.conf" }
kdump_get_remote_ip() { @@ -486,7 +498,6 @@ kdump_get_remote_ip() { # $1: destination host kdump_install_net() { local _destaddr _route _netdev - local _znet_netdev _znet_conpath # each netowrk interface is managed by a NM connection profile declare -A nmconnection_map
@@ -499,15 +510,6 @@ kdump_install_net() { _route=$(kdump_get_ip_route "$_destaddr") _netdev=$(kdump_get_ip_route_field "$_route" "dev")
- _znet_netdev=$(find_online_znet_device) - if [[ -n $_znet_netdev ]]; then - _znet_conpath=$(get_nmcli_connection_apath_by_ifname "$_znet_netdev") - if ! (kdump_setup_znet "$_znet_netdev" "$_znet_conpath"); then - derror "Failed to set up znet" - exit 1 - fi - fi - if kdump_is_bridge "$_netdev"; then kdump_setup_bridge "$_netdev" elif kdump_is_bond "$_netdev"; then @@ -520,6 +522,8 @@ kdump_install_net() {
kdump_copy_nmconnection_file "$_netdev"
+ [[ -n "$(find_online_znet_device)" ]] && kdump_setup_znet + if [[ ! -f ${initdir}/etc/cmdline.d/50neednet.conf ]]; then # network-manager module needs this parameter echo "rd.neednet" >> "${initdir}/etc/cmdline.d/50neednet.conf" diff --git a/dracut-module-setup.sh.fix b/dracut-module-setup.sh.fix new file mode 100755 index 0000000..de6650c --- /dev/null +++ b/dracut-module-setup.sh.fix @@ -0,0 +1,994 @@ +#!/bin/bash + +_DRACUT_KDUMP_NM_TMP_DIR="/tmp/$$-DRACUT_KDUMP_NM" +_TMP_KDUMP_NETIFS="$_DRACUT_KDUMP_NM_TMP_DIR/kdump_netifs" + +cleanup() { + rm -rf $_DRACUT_KDUMP_NM_TMP_DIR +} + +trap 'ret=$?; cleanup; exit $ret;' EXIT + +kdump_module_init() { + if ! [[ -d "${initdir}/tmp" ]]; then + mkdir -p "${initdir}/tmp" + fi + + mkdir -p "$_DRACUT_KDUMP_NM_TMP_DIR" + echo -n > "$_TMP_KDUMP_NETIFS" + + . /lib/kdump/kdump-lib.sh +} + +check() { + [[ $debug ]] && set -x + #kdumpctl sets this explicitly + if [[ -z $IN_KDUMP ]] || [[ ! -f /etc/kdump.conf ]]; then + return 1 + fi + return 0 +} + +depends() { + local _dep="base shutdown" + + kdump_module_init + + add_opt_module() { + [[ " $omit_dracutmodules " != *\ $1\ * ]] && _dep="$_dep $1" + } + + if is_squash_available; then + add_opt_module squash + else + dwarning "Required modules to build a squashed kdump image is missing!" + fi + + if is_wdt_active; then + add_opt_module watchdog + fi + + if is_ssh_dump_target; then + _dep="$_dep ssh-client" + fi + + if [[ "$(uname -m)" == "s390x" ]]; then + _dep="$_dep znet" + fi + + if [[ -n "$(ls -A /sys/class/drm 2> /dev/null)" ]] || [[ -d /sys/module/hyperv_fb ]]; then + add_opt_module drm + fi + + if is_generic_fence_kdump || is_pcs_fence_kdump; then + _dep="$_dep network" + fi + + echo "$_dep" +} + +kdump_is_bridge() { + [[ -d /sys/class/net/"$1"/bridge ]] +} + +kdump_is_bond() { + [[ -d /sys/class/net/"$1"/bonding ]] +} + +kdump_is_team() { + [[ -f /usr/bin/teamnl ]] && teamnl "$1" ports &> /dev/null +} + +kdump_is_vlan() { + [[ -f /proc/net/vlan/"$1" ]] +} + +# $1: netdev name +source_ifcfg_file() { + local ifcfg_file + + dwarning "Network Scripts are deprecated. You are encouraged to set up network by NetworkManager." + ifcfg_file=$(get_ifcfg_filename "$1") + if [[ -f ${ifcfg_file} ]]; then + . "${ifcfg_file}" + else + dwarning "The ifcfg file of $1 is not found!" + fi +} + +# $1: repeat times +# $2: string to be repeated +# $3: separator +repeatedly_join_str() { + local _count="$1" + local _str="$2" + local _separator="$3" + local i _res + + if [[ $_count -le 0 ]]; then + echo -n "" + return + fi + + i=0 + _res="$_str" + ((_count--)) + + while [[ $i -lt $_count ]]; do + ((i++)) + _res="${_res}${_separator}${_str}" + done + echo -n "$_res" +} + +# $1: prefix +# $2: ipv6_flag="-6" indicates it's IPv6 +# Given a prefix, calculate the netmask (equivalent of "ipcalc -m") +# by concatenating three parts, +# 1) the groups with all bits set 1 +# 2) a group with partial bits set to 0 +# 3) the groups with all bits set to 0 +cal_netmask_by_prefix() { + local _prefix="$1" + local _ipv6_flag="$2" _ipv6 + local _bits_per_octet=8 + local _count _res _octets_per_group _octets_total _seperator _total_groups + local _max_group_value _max_group_value_repr _bits_per_group _tmp _zero_bits + + if [[ $_ipv6_flag == "-6" ]]; then + _ipv6=1 + else + _ipv6=0 + fi + + if [[ $_prefix -lt 0 || $_prefix -gt 128 ]] \ + || ( ((!_ipv6)) && [[ $_prefix -gt 32 ]]); then + derror "Bad prefix:$_prefix for calculating netmask" + exit 1 + fi + + if ((_ipv6)); then + _octets_per_group=2 + _octets_total=16 + _seperator=":" + else + _octets_per_group=1 + _octets_total=4 + _seperator="." + fi + + _total_groups=$((_octets_total / _octets_per_group)) + _bits_per_group=$((_octets_per_group * _bits_per_octet)) + _max_group_value=$(((1 << _bits_per_group) - 1)) + + if ((_ipv6)); then + _max_group_value_repr=$(printf "%x" $_max_group_value) + else + _max_group_value_repr="$_max_group_value" + fi + + _count=$((_prefix / _octets_per_group / _bits_per_octet)) + _first_part=$(repeatedly_join_str "$_count" "$_max_group_value_repr" "$_seperator") + _res="$_first_part" + + _tmp=$((_octets_total * _bits_per_octet - _prefix)) + _zero_bits=$((_tmp % _bits_per_group)) + if [[ $_zero_bits -ne 0 ]]; then + _second_part=$((_max_group_value >> _zero_bits << _zero_bits)) + if ((_ipv6)); then + _second_part=$(printf "%x" $_second_part) + fi + ((_count++)) + if [[ -z $_first_part ]]; then + _res="$_second_part" + else + _res="${_first_part}${_seperator}${_second_part}" + fi + fi + + _count=$((_total_groups - _count)) + if [[ $_count -eq 0 ]]; then + echo -n "$_res" + return + fi + + if ((_ipv6)) && [[ $_count -gt 1 ]]; then + # use condensed notion for IPv6 + _third_part=":" + else + _third_part=$(repeatedly_join_str "$_count" "0" "$_seperator") + fi + + if [[ -z $_res ]] && ((!_ipv6)); then + echo -n "${_third_part}" + else + echo -n "${_res}${_seperator}${_third_part}" + fi +} + +kdump_get_mac_addr() { + cat "/sys/class/net/$1/address" +} + +#Bonding or team master modifies the mac address +#of its slaves, we should use perm address +kdump_get_perm_addr() { + local addr + addr=$(ethtool -P "$1" | sed -e 's/Permanent address: //') + if [[ -z $addr ]] || [[ $addr == "00:00:00:00:00:00" ]]; then + derror "Can't get the permanent address of $1" + else + echo "$addr" + fi +} + +# Convert ifcfg-* files to Networkmanager .nmconnection connection profiles. +# +# This function makes use of "nmcli clone" to automatically convert ifcfg-* +# files to Networkmanager .nmconnection connection profiles. +# Note these temporory file should be cleaned up after being used. +ifcfg_to_nmconnection() { + local _dev _cloned_nmconnection_file_path _tmp_nmconnection_file_path _unique_name _old_uuid _old_name _uuid + + for _dev in "${!nmconnection_map[@]}"; do + _nmconnection_file_path=${nmconnection_map[$_dev]} + _old_uuid=$(nmcli --get-values connection.uuid connection show "$_nmconnection_file_path") + _old_name=$(nmcli --get-values connection.id connection show "$_nmconnection_file_path") + _unique_name=$(cat /proc/sys/kernel/random/uuid) + + nmcli connection clone --temporary "$_nmconnection_file_path" "$_unique_name" &> >(ddebug) + _uuid=$(nmcli --get-values connection.uuid connection show "$_unique_name") + # change back connection.id for the sake of readibility + nmcli connection modify --temporary "$_uuid" connection.id "$_old_name" &> >(ddebug) + + _cloned_nmconnection_file_path=$(nmcli --get-values UUID,FILENAME connection show | sed -n "s/^${_uuid}://p") + _tmp_nmconnection_file_path="$_DRACUT_KDUMP_NM_TMP_DIR/$(basename "$_nmconnection_file_path")" + cp "$_cloned_nmconnection_file_path" "$_tmp_nmconnection_file_path" + # change uuid back to old value in case it's refered by other connection + # profile e.g. connection.master could be tnterface name of the master + # device or UUID of the master connection. + sed -i -r "s/(^uuid=).*$/\1${_old_uuid}/g" "$_tmp_nmconnection_file_path" + nmconnection_map[$_dev]=$_tmp_nmconnection_file_path + nmcli connection del "$_uuid" + done +} + +kdump_copy_nmconnection_file() { + local _dev _nmconnection_file_path _nmconnection_name _initrd_nmconnection_file_path + _dev="$1" + + _nmconnection_file_path=${nmconnection_map[$_dev]} + _nmconnection_name=$(basename "$_nmconnection_file_path") + _initrd_nmconnection_file_path="/etc/NetworkManager/system-connections/$_nmconnection_name" + inst "$_nmconnection_file_path" "$_initrd_nmconnection_file_path" + + # Ask NM to wait 60s for the network device to be available + # wait-device-timeout=60000 is inserted after "[connection]" so the + # user's custom value could overwrite it + sed -i '/[connection]/a wait-device-timeout=60000' "${initdir}/$_initrd_nmconnection_file_path" + + echo -n " $_dev" >> "$_TMP_KDUMP_NETIFS" +} + +kdump_install_nm_netif_allowlist() { + local _netif _except_netif _netif_allowlist _netif_allowlist_nm_conf + + for _netif in $1; do + _per_mac=$(kdump_get_perm_addr "$_netif") + if [[ "$_per_mac" != 'not set' ]]; then + _except_netif="mac:$_per_mac" + else + _except_netif="interface-name:$_netif" + fi + _netif_allowlist="${_netif_allowlist}except:${_except_netif};" + done + + _netif_allowlist_nm_conf=$_DRACUT_KDUMP_NM_TMP_DIR/netif_allowlist_nm_conf + cat <<- EOF > "$_netif_allowlist_nm_conf" + [device-others] + match-device=${_netif_allowlist} + managed=false + EOF + + inst "$_netif_allowlist_nm_conf" "/etc/NetworkManager/conf.d/10-kdump-netif.conf" +} + +_get_nic_driver() { + ethtool -i "$1" | /usr/bin/grep driver | cut -d ":" -f2 +} + +kdump_install_nic_driver() { + local _netif _driver _drivers + + for _netif in $1; do + _driver=$(_get_nic_driver "$_netif") + if [[ -z $_driver ]]; then + derror "Failed to get the driver of $_netif" + exit 1 + fi + _drivers="$_drivers $_driver" + done + + # shellcheck disable=SC2086 # split the argument to have a list of drivers + instmods $_drivers +} + +kdump_setup_bridge() { + local _netdev=$1 + local _dev + for _dev in "/sys/class/net/$_netdev/brif/"*; do + [[ -e $_dev ]] || continue + _dev=${_dev##*/} + if kdump_is_bond "$_dev"; then + (kdump_setup_bond "$_dev") || exit 1 + elif kdump_is_team "$_dev"; then + kdump_setup_team "$_dev" + elif kdump_is_vlan "$_dev"; then + kdump_setup_vlan "$_dev" + fi + kdump_copy_nmconnection_file "$_dev" + done +} + +kdump_setup_bond() { + local _netdev="$1" + local _dev + + for _dev in $(<"/sys/class/net/$_netdev/bonding/slaves"); do + kdump_copy_nmconnection_file "$_dev" + done +} + +kdump_setup_team() { + local _netdev=$1 + local _dev + for _dev in $(teamnl "$_netdev" ports | awk -F':' '{print $2}'); do + kdump_copy_nmconnection_file "$_dev" + done +} + +kdump_setup_vlan() { + local _netdev=$1 + local _parent_netif + + _parent_netif="$(awk '/^Device:/{print $2}' /proc/net/vlan/"$_netdev")" + + #Just support vlan over bond and team + if kdump_is_bridge "$_parent_netif"; then + derror "Vlan over bridge is not supported!" + exit 1 + elif kdump_is_bond "$_parent_netif"; then + (kdump_setup_bond "$_parent_netif") || exit 1 + kdump_copy_nmconnection_file "$_parent_netif" + elif kdump_is_team "$_parent_netif"; then + (kdump_setup_team "$_parent_netif") || exit 1 + kdump_copy_nmconnection_file "$_parent_netif" + else + echo -n "except:interface-name:$_parent_netif," >> "/tmp/$$-netif_allowlist" + fi +} + +# find online znet device +# return ifname (_netdev) +# code reaped from the list_configured function of +# https://github.com/hreinecke/s390-tools/blob/master/zconf/znetconf +find_online_znet_device() { + local CCWGROUPBUS_DEVICEDIR="/sys/bus/ccwgroup/devices" + local NETWORK_DEVICES d ifname ONLINE + + [[ ! -d $CCWGROUPBUS_DEVICEDIR ]] && return + NETWORK_DEVICES=$(find $CCWGROUPBUS_DEVICEDIR) + for d in $NETWORK_DEVICES; do + [[ ! -f "$d/online" ]] && continue + read -r ONLINE < "$d/online" + if [[ $ONLINE -ne 1 ]]; then + continue + fi + # determine interface name, if there (only for qeth and if + # device is online) + if [[ -f $d/if_name ]]; then + read -r ifname < "$d/if_name" + elif [[ -d $d/net ]]; then + ifname=$(ls "$d/net/") + fi + [[ -n $ifname ]] && break + done + echo -n "$ifname" +} + +_find_znet_nmconnection() { + LANG=C grep -s -E -i -l \ + "^s390-subchannels=([0-9].[0-9].[a-f0-9]+;){0,2}" \ + "$1"/*.nmconnection | LC_ALL=C sed -e "$2" +} + +# setup s390 znet +# +# Note part of code is extracted from ccw_init provided by s390utils +kdump_setup_znet() { + local _config_file _uniq_name _NM_conf_dir + local __sed_discard_ignored_files='/(~|.bak|.old|.orig|.rpmnew|.rpmorig|.rpmsave)$/d' + + _NM_conf_dir="/etc/NetworkManager/system-connections" + + _config_file=$(_find_znet_nmconnection "$initdir/$_NM_conf_dir" "$__sed_discard_ignored_files") + if [[ -n "$_config_file" ]]; then + ddebug "$_config_file has already contained the znet config" + return + fi + + _config_file=$(LANG=C grep -s -E -i -l \ + "^[[:space:]]*SUBCHANNELS=['"]?([0-9].[0-9].[a-f0-9]+,){0,2}" \ + /etc/sysconfig/network-scripts/ifcfg-* \ + | LC_ALL=C sed -e "$__sed_discard_ignored_files") + + if [[ -n "$_config_file" ]]; then + LANG=c grep 'SUBCHANNELS|NETTYPE|OPTIONS' "$_config_file" > "$initdir/$_config_file" + else + _config_file=$(_find_znet_nmconnection "$_NM_conf_dir" "$__sed_discard_ignored_files") + fi + + if [[ -n "$_config_file" ]]; then + _uniq_name=$(cat /proc/sys/kernel/random/uuid) + nmcli connection clone --temporary "$_config_file" "$_uniq_name" &> >(ddebug) + nmcli connection modify --temporary "$_uniq_name" connection.autoconnect false + inst "/run/NetworkManager/system-connections/${_uniq_name}.nmconnection" "${_NM_conf_dir}/${_uniq_name}.nmconnection" + nmcli connection del "$_uniq_name" &> >(ddebug) + fi + +} + +kdump_get_remote_ip() { + local _remote _remote_temp + _remote=$(get_remote_host "$1") + if is_hostname "$_remote"; then + _remote_temp=$(getent ahosts "$_remote" | grep -v : | head -n 1) + if [[ -z $_remote_temp ]]; then + _remote_temp=$(getent ahosts "$_remote" | head -n 1) + fi + _remote=$(echo "$_remote_temp" | awk '{print $1}') + fi + echo "$_remote" +} + +# Setup dracut to bring up network interface that enable +# initramfs accessing giving destination +# $1: destination host +kdump_install_net() { + local _destaddr _route _netdev + # each netowrk interface is managed by a NM connection profile + declare -A nmconnection_map + + while IFS=: read -r _ifname _nmconn; do + nmconnection_map[$_ifname]="$_nmconn" + done <<< "$(nmcli -t -f device,filename connection show --active)" + ifcfg_to_nmconnection + + _destaddr=$(kdump_get_remote_ip "$1") + _route=$(kdump_get_ip_route "$_destaddr") + _netdev=$(kdump_get_ip_route_field "$_route" "dev") + + if kdump_is_bridge "$_netdev"; then + kdump_setup_bridge "$_netdev" + elif kdump_is_bond "$_netdev"; then + (kdump_setup_bond "$_netdev") || exit 1 + elif kdump_is_team "$_netdev"; then + kdump_setup_team "$_netdev" + elif kdump_is_vlan "$_netdev"; then + kdump_setup_vlan "$_netdev" + fi + + kdump_copy_nmconnection_file "$_netdev" + + [[ -n "$(find_online_znet_device)" ]] && kdump_setup_znet + + if [[ ! -f ${initdir}/etc/cmdline.d/50neednet.conf ]]; then + # network-manager module needs this parameter + echo "rd.neednet" >> "${initdir}/etc/cmdline.d/50neednet.conf" + fi + + # Stop dracut 35network-manger to calling nm-initrd-generator. + # Note this line of code can be removed after NetworkManager >= 1.35.2 + # gets released. + echo > "${initdir}/usr/libexec/nm-initrd-generator" + + # Save netdev used for kdump as cmdline + # Whoever calling kdump_install_net() is setting up the default gateway, + # ie. bootdev/kdumpnic. So don't override the setting if calling + # kdump_install_net() for another time. For example, after setting eth0 as + # the default gate way for network dump, eth1 in the fence kdump path will + # call kdump_install_net again and we don't want eth1 to be the default + # gateway. + if [[ ! -f ${initdir}/etc/cmdline.d/60kdumpip.conf ]]; then + echo "kdump_remote_ip=$_destaddr" > "${initdir}/etc/cmdline.d/60kdumpip.conf" + fi + +} + +# install etc/kdump/pre.d and /etc/kdump/post.d +kdump_install_pre_post_conf() { + if [[ -d /etc/kdump/pre.d ]]; then + for file in /etc/kdump/pre.d/*; do + if [[ -x $file ]]; then + dracut_install "$file" + elif [[ $file != "/etc/kdump/pre.d/*" ]]; then + echo "$file is not executable" + fi + done + fi + + if [[ -d /etc/kdump/post.d ]]; then + for file in /etc/kdump/post.d/*; do + if [[ -x $file ]]; then + dracut_install "$file" + elif [[ $file != "/etc/kdump/post.d/*" ]]; then + echo "$file is not executable" + fi + done + fi +} + +default_dump_target_install_conf() { + local _target _fstype + local _mntpoint _save_path + + is_user_configured_dump_target && return + + _save_path=$(get_bind_mount_source "$(get_save_path)") + _target=$(get_target_from_path "$_save_path") + _mntpoint=$(get_mntpoint_from_target "$_target") + + _fstype=$(get_fs_type_from_target "$_target") + if is_fs_type_nfs "$_fstype"; then + kdump_install_net "$_target" + _fstype="nfs" + else + _target=$(kdump_get_persistent_dev "$_target") + fi + + echo "$_fstype $_target" >> "${initdir}/tmp/$$-kdump.conf" + + # don't touch the path under root mount + if [[ $_mntpoint != "/" ]]; then + _save_path=${_save_path##"$_mntpoint"} + fi + + #erase the old path line, then insert the parsed path + sed -i "/^path/d" "${initdir}/tmp/$$-kdump.conf" + echo "path $_save_path" >> "${initdir}/tmp/$$-kdump.conf" +} + +#install kdump.conf and what user specifies in kdump.conf +kdump_install_conf() { + local _opt _val _pdev + + kdump_read_conf > "${initdir}/tmp/$$-kdump.conf" + + while read -r _opt _val; do + # remove inline comments after the end of a directive. + case "$_opt" in + raw) + _pdev=$(persistent_policy="by-id" kdump_get_persistent_dev "$_val") + sed -i -e "s#^${_opt}[[:space:]]+$_val#$_opt $_pdev#" "${initdir}/tmp/$$-kdump.conf" + ;; + ext[234] | xfs | btrfs | minix) + _pdev=$(kdump_get_persistent_dev "$_val") + sed -i -e "s#^${_opt}[[:space:]]+$_val#$_opt $_pdev#" "${initdir}/tmp/$$-kdump.conf" + ;; + ssh | nfs) + kdump_install_net "$_val" + ;; + dracut_args) + if [[ $(get_dracut_args_fstype "$_val") == nfs* ]]; then + kdump_install_net "$(get_dracut_args_target "$_val")" + fi + ;; + kdump_pre | kdump_post | extra_bins) + # shellcheck disable=SC2086 + dracut_install $_val + ;; + core_collector) + dracut_install "${_val%%[[:blank:]]*}" + ;; + esac + done <<< "$(kdump_read_conf)" + + kdump_install_pre_post_conf + + default_dump_target_install_conf + + kdump_configure_fence_kdump "${initdir}/tmp/$$-kdump.conf" + inst "${initdir}/tmp/$$-kdump.conf" "/etc/kdump.conf" + rm -f "${initdir}/tmp/$$-kdump.conf" +} + +# Default sysctl parameters should suffice for kdump kernel. +# Remove custom configurations sysctl.conf & sysctl.d/* +remove_sysctl_conf() { + + # As custom configurations like vm.min_free_kbytes can lead + # to OOM issues in kdump kernel, avoid them + rm -f "${initdir}/etc/sysctl.conf" + rm -rf "${initdir}/etc/sysctl.d" + rm -rf "${initdir}/run/sysctl.d" + rm -rf "${initdir}/usr/lib/sysctl.d" +} + +kdump_iscsi_get_rec_val() { + + local result + + # The open-iscsi 742 release changed to using flat files in + # /var/lib/iscsi. + + result=$(/sbin/iscsiadm --show -m session -r "$1" | grep "^${2} = ") + result=${result##* = } + echo "$result" +} + +kdump_get_iscsi_initiator() { + local _initiator + local initiator_conf="/etc/iscsi/initiatorname.iscsi" + + [[ -f $initiator_conf ]] || return 1 + + while read -r _initiator; do + [[ -z ${_initiator%%#*} ]] && continue # Skip comment lines + + case $_initiator in + InitiatorName=*) + initiator=${_initiator#InitiatorName=} + echo "rd.iscsi.initiator=${initiator}" + return 0 + ;; + *) ;; + esac + done < ${initiator_conf} + + return 1 +} + +# Figure out iBFT session according to session type +is_ibft() { + [[ "$(kdump_iscsi_get_rec_val "$1" "node.discovery_type")" == fw ]] +} + +kdump_setup_iscsi_device() { + local path=$1 + local tgt_name + local tgt_ipaddr + local username + local password + local userpwd_str + local username_in + local password_in + local userpwd_in_str + local netroot_str + local initiator_str + local netroot_conf="${initdir}/etc/cmdline.d/50iscsi.conf" + local initiator_conf="/etc/iscsi/initiatorname.iscsi" + + dinfo "Found iscsi component $1" + + # Check once before getting explicit values, so we can bail out early, + # e.g. in case of pure-hardware(all-offload) iscsi. + if ! /sbin/iscsiadm -m session -r "$path" &> /dev/null; then + return 1 + fi + + if is_ibft "$path"; then + return + fi + + # Remove software iscsi cmdline generated by 95iscsi, + # and let kdump regenerate here. + rm -f "${initdir}/etc/cmdline.d/95iscsi.conf" + + tgt_name=$(kdump_iscsi_get_rec_val "$path" "node.name") + tgt_ipaddr=$(kdump_iscsi_get_rec_val "$path" "node.conn[0].address") + + # get and set username and password details + username=$(kdump_iscsi_get_rec_val "$path" "node.session.auth.username") + [[ $username == "<empty>" ]] && username="" + password=$(kdump_iscsi_get_rec_val "$path" "node.session.auth.password") + [[ $password == "<empty>" ]] && password="" + username_in=$(kdump_iscsi_get_rec_val "$path" "node.session.auth.username_in") + [[ -n $username ]] && userpwd_str="$username:$password" + + # get and set incoming username and password details + [[ $username_in == "<empty>" ]] && username_in="" + password_in=$(kdump_iscsi_get_rec_val "$path" "node.session.auth.password_in") + [[ $password_in == "<empty>" ]] && password_in="" + + [[ -n $username_in ]] && userpwd_in_str=":$username_in:$password_in" + + kdump_install_net "$tgt_ipaddr" + + # prepare netroot= command line + # FIXME: Do we need to parse and set other parameters like protocol, port + # iscsi_iface_name, netdev_name, LUN etc. + + if is_ipv6_address "$tgt_ipaddr"; then + tgt_ipaddr="[$tgt_ipaddr]" + fi + netroot_str="netroot=iscsi:${userpwd_str}${userpwd_in_str}@$tgt_ipaddr::::$tgt_name" + + [[ -f $netroot_conf ]] || touch "$netroot_conf" + + # If netroot target does not exist already, append. + if ! grep -q "$netroot_str" "$netroot_conf"; then + echo "$netroot_str" >> "$netroot_conf" + dinfo "Appended $netroot_str to $netroot_conf" + fi + + # Setup initator + if ! initiator_str=$(kdump_get_iscsi_initiator); then + derror "Failed to get initiator name" + return 1 + fi + + # If initiator details do not exist already, append. + if ! grep -q "$initiator_str" "$netroot_conf"; then + echo "$initiator_str" >> "$netroot_conf" + dinfo "Appended $initiator_str to $netroot_conf" + fi +} + +kdump_check_iscsi_targets() { + # If our prerequisites are not met, fail anyways. + type -P iscsistart > /dev/null || return 1 + + kdump_check_setup_iscsi() ( + local _dev + _dev=$1 + + [[ -L /sys/dev/block/$_dev ]] || return + cd "$(readlink -f "/sys/dev/block/$_dev")" || return 1 + until [[ -d sys || -d iscsi_session ]]; do + cd .. + done + [[ -d iscsi_session ]] && kdump_setup_iscsi_device "$PWD" + ) + + [[ $hostonly ]] || [[ $mount_needs ]] && { + for_each_host_dev_and_slaves_all kdump_check_setup_iscsi + } +} + +# hostname -a is deprecated, do it by ourself +get_alias() { + local ips + local entries + local alias_set + + ips=$(hostname -I) + for ip in $ips; do + # in /etc/hosts, alias can come at the 2nd column + if entries=$(grep "$ip" /etc/hosts | awk '{ $1=""; print $0 }'); then + alias_set="$alias_set $entries" + fi + done + + echo "$alias_set" +} + +is_localhost() { + local hostnames + local shortnames + local aliasname + local nodename=$1 + + hostnames=$(hostname -A) + shortnames=$(hostname -A -s) + aliasname=$(get_alias) + hostnames="$hostnames $shortnames $aliasname" + + for name in ${hostnames}; do + if [[ $name == "$nodename" ]]; then + return 0 + fi + done + return 1 +} + +# retrieves fence_kdump nodes from Pacemaker cluster configuration +get_pcs_fence_kdump_nodes() { + local nodes + + pcs cluster sync > /dev/null 2>&1 && pcs cluster cib-upgrade > /dev/null 2>&1 + # get cluster nodes from cluster cib, get interface and ip address + nodelist=$(pcs cluster cib | xmllint --xpath "/cib/status/node_state/@uname" -) + + # nodelist is formed as 'uname="node1" uname="node2" ... uname="nodeX"' + # we need to convert each to node1, node2 ... nodeX in each iteration + for node in ${nodelist}; do + # convert $node from 'uname="nodeX"' to 'nodeX' + eval "$node" + nodename="$uname" + # Skip its own node name + if is_localhost "$nodename"; then + continue + fi + nodes="$nodes $nodename" + done + + echo "$nodes" +} + +# retrieves fence_kdump args from config file +get_pcs_fence_kdump_args() { + if [[ -f $FENCE_KDUMP_CONFIG_FILE ]]; then + . "$FENCE_KDUMP_CONFIG_FILE" + echo "$FENCE_KDUMP_OPTS" + fi +} + +get_generic_fence_kdump_nodes() { + local filtered + local nodes + + nodes=$(kdump_get_conf_val "fence_kdump_nodes") + for node in ${nodes}; do + # Skip its own node name + if is_localhost "$node"; then + continue + fi + filtered="$filtered $node" + done + echo "$filtered" +} + +# setup fence_kdump in cluster +# setup proper network and install needed files +kdump_configure_fence_kdump() { + local kdump_cfg_file=$1 + local nodes + local args + + if is_generic_fence_kdump; then + nodes=$(get_generic_fence_kdump_nodes) + + elif is_pcs_fence_kdump; then + nodes=$(get_pcs_fence_kdump_nodes) + + # set appropriate options in kdump.conf + echo "fence_kdump_nodes $nodes" >> "${kdump_cfg_file}" + + args=$(get_pcs_fence_kdump_args) + if [[ -n $args ]]; then + echo "fence_kdump_args $args" >> "${kdump_cfg_file}" + fi + + else + # fence_kdump not configured + return 1 + fi + + # setup network for each node + for node in ${nodes}; do + kdump_install_net "$node" + done + + dracut_install /etc/hosts + dracut_install /etc/nsswitch.conf + dracut_install "$FENCE_KDUMP_SEND" +} + +# Install a random seed used to feed /dev/urandom +# By the time kdump service starts, /dev/uramdom is already fed by systemd +kdump_install_random_seed() { + local poolsize + + poolsize=$(< /proc/sys/kernel/random/poolsize) + + if [[ ! -d "${initdir}/var/lib/" ]]; then + mkdir -p "${initdir}/var/lib/" + fi + + dd if=/dev/urandom of="${initdir}/var/lib/random-seed" \ + bs="$poolsize" count=1 2> /dev/null +} + +kdump_install_systemd_conf() { + # Kdump turns out to require longer default systemd mount timeout + # than 1st kernel(90s by default), we use default 300s for kdump. + if ! grep -q -r "^[[:space:]]*DefaultTimeoutStartSec=" "${initdir}/etc/systemd/system.conf"*; then + mkdir -p "${initdir}/etc/systemd/system.conf.d" + echo "[Manager]" > "${initdir}/etc/systemd/system.conf.d/kdump.conf" + echo "DefaultTimeoutStartSec=300s" >> "${initdir}/etc/systemd/system.conf.d/kdump.conf" + fi + + # Forward logs to console directly, and don't read Kmsg, this avoids + # unneccessary memory consumption and make console output more useful. + # Only do so for non fadump image. + mkdir -p "${initdir}/etc/systemd/journald.conf.d" + echo "[Journal]" > "${initdir}/etc/systemd/journald.conf.d/kdump.conf" + echo "Storage=volatile" >> "${initdir}/etc/systemd/journald.conf.d/kdump.conf" + echo "ReadKMsg=no" >> "${initdir}/etc/systemd/journald.conf.d/kdump.conf" + echo "ForwardToConsole=yes" >> "${initdir}/etc/systemd/journald.conf.d/kdump.conf" +} + +remove_cpu_online_rule() { + local file=${initdir}/usr/lib/udev/rules.d/40-redhat.rules + + sed -i '/SUBSYSTEM=="cpu"/d' "$file" +} + +install() { + local arch _netifs + + kdump_module_init + kdump_install_conf + remove_sysctl_conf + + # Onlining secondary cpus breaks kdump completely on KVM on Power hosts + # Though we use maxcpus=1 by default but 40-redhat.rules will bring up all + # possible cpus by default. (rhbz1270174 rhbz1266322) + # Thus before we get the kernel fix and the systemd rule fix let's remove + # the cpu online rule in kdump initramfs. + arch=$(uname -m) + if [[ "$arch" = "ppc64le" ]] || [[ "$arch" = "ppc64" ]]; then + remove_cpu_online_rule + fi + + if is_ssh_dump_target; then + kdump_install_random_seed + fi + dracut_install -o /etc/adjtime /etc/localtime + inst "$moddir/monitor_dd_progress" "/kdumpscripts/monitor_dd_progress" + chmod +x "${initdir}/kdumpscripts/monitor_dd_progress" + inst "/bin/dd" "/bin/dd" + inst "/bin/tail" "/bin/tail" + inst "/bin/date" "/bin/date" + inst "/bin/sync" "/bin/sync" + inst "/bin/cut" "/bin/cut" + inst "/bin/head" "/bin/head" + inst "/bin/awk" "/bin/awk" + inst "/bin/sed" "/bin/sed" + inst "/bin/stat" "/bin/stat" + inst "/sbin/makedumpfile" "/sbin/makedumpfile" + inst "/sbin/vmcore-dmesg" "/sbin/vmcore-dmesg" + inst "/usr/bin/printf" "/sbin/printf" + inst "/usr/bin/logger" "/sbin/logger" + inst "/usr/bin/chmod" "/sbin/chmod" + inst "/lib/kdump/kdump-lib-initramfs.sh" "/lib/kdump-lib-initramfs.sh" + inst "/lib/kdump/kdump-logger.sh" "/lib/kdump-logger.sh" + inst "$moddir/kdump.sh" "/usr/bin/kdump.sh" + inst "$moddir/kdump-capture.service" "$systemdsystemunitdir/kdump-capture.service" + systemctl -q --root "$initdir" add-wants initrd.target kdump-capture.service + # Replace existing emergency service and emergency target + cp "$moddir/kdump-emergency.service" "$initdir/$systemdsystemunitdir/emergency.service" + cp "$moddir/kdump-emergency.target" "$initdir/$systemdsystemunitdir/emergency.target" + # Also redirect dracut-emergency to kdump error handler + ln_r "$systemdsystemunitdir/emergency.service" "$systemdsystemunitdir/dracut-emergency.service" + + # Check for all the devices and if any device is iscsi, bring up iscsi + # target. Ideally all this should be pushed into dracut iscsi module + # at some point of time. + kdump_check_iscsi_targets + + kdump_install_systemd_conf + + # nfs/ssh dump will need to get host ip in second kernel and need to call 'ip' tool, see get_host_ip for more detail + if is_nfs_dump_target || is_ssh_dump_target; then + inst "ip" + fi + + _netifs=$(cat "$_TMP_KDUMP_NETIFS") + if [[ -n "$_netifs" ]]; then + # kdump_install_nm_netif_allowlist "$_netifs" + kdump_install_nic_driver "$_netifs" + fi + + # For the lvm type target under kdump, in /etc/lvm/lvm.conf we can + # safely replace "reserved_memory=XXXX"(default value is 8192) with + # "reserved_memory=1024" to lower memory pressure under kdump. We do + # it unconditionally here, if "/etc/lvm/lvm.conf" doesn't exist, it + # actually does nothing. + sed -i -e \ + 's/(^[[:space:]]*reserved_memory[[:space:]]*=)[[:space:]]*[[:digit:]]*/\1 1024/' \ + "${initdir}/etc/lvm/lvm.conf" &> /dev/null + + # Save more memory by dropping switch root capability + dracut_no_switch_root +}
On Thu, Aug 25, 2022 at 08:58:35AM +0800, Coiby Xu wrote:
/usr/lib/udev/ccw_init [1] shipped by s390utils extracts the values of SUBCHANNELS, NETTYPE and LAYER2 from /etc/sysconfig/network-scripts/ifcfg-* or /etc/NetworkManager/system-connections/*.nmconnection to activate znet network device. If the connection profile is copied to initrd, there is no need to set up the "rd.znet" dracut cmdline parameter.
There are two cases addressed by this commit,
- znet network interface is a slave of bonding/teaming/vlan/bridging network. The connection profile has been copied to initrd by kdump_copy_nmconnection_file and it contains the info needed by ccw_init.
- znet network interface is a slave of bonding/teaming/vlan/bridging network. The corresponding ifcfg-*/*.nmconnection file may not contain info like SUBCHANNELS [2]. In this case, copy the ifcfg-*/*.nmconnection file that has this info to the kdump initrd. Also to prevent the copied connection profile from being chosen by NM, set connection.autoconnect=false for this connection profile.
Note
- ccw_init doesn't care if SUBCHANNELS, NETTYPE and LAYER2 comes from
an active NM profile or not. If an inactive NM profile contains this info, it needs to be copied to the kdump initrd as well. 2. "rd.znet_ifname=$_netdev:${SUBCHANNELS}" is no needed because now there is no renaming of s390x network interfaces when reusing NetworkManager profiles. rd.znet_ifname was introduced in commit ce0305d ("Add a new option 'rd.znet_ifname' in order to use it in udev rules") to address the special case of non-persistent MAC address by renaming a network interface by SUBCHANNELS.
[1] https://src.fedoraproject.org/rpms/s390utils/blob/rawhide/f/ccw_init [2] https://bugzilla.redhat.com/show_bug.cgi?id=2064708
Signed-off-by: Coiby Xu coxu@redhat.com
dracut-module-setup.sh | 70 +-- dracut-module-setup.sh.fix | 994 +++++++++++++++++++++++++++++++++++++
Oops, I added dracut-module-setup.sh.fix by mistake. Please ignore it.
Hi Coiby,
On Thu, 25 Aug 2022 08:58:24 +0800 Coiby Xu coxu@redhat.com wrote:
[...]
Note this patch set is available for review on https://github.com/coiby/kexec-tools/pull/9
stupid question. On the pull request you mention a v3. But I couldn't find one on the list. Have I miss it or haven't you send it, yet?
Furthermore, I had problems applying this series to both the current rawhide as well as your github_action branch. What's the base you used when sending this version?
Anyway, I'm looking at your direct_nm branch now as it seems to be the latest version to me.
Thanks Philipp
Hi Philipp,
On Tue, Oct 25, 2022 at 06:35:11PM +0200, Philipp Rudo wrote:
Hi Coiby,
On Thu, 25 Aug 2022 08:58:24 +0800 Coiby Xu coxu@redhat.com wrote:
[...]
Note this patch set is available for review on https://github.com/coiby/kexec-tools/pull/9
stupid question. On the pull request you mention a v3. But I couldn't find one on the list. Have I miss it or haven't you send it, yet?
Sorry for the confusion. Yes, I haven't sent the subsequent changes to the list.
Furthermore, I had problems applying this series to both the current rawhide as well as your github_action branch. What's the base you used when sending this version?
Anyway, I'm looking at your direct_nm branch now as it seems to be the latest version to me.
Yeah, the direct_nm branch is the latest version.
Thanks Philipp