ifcfg scripts are deprecated. kexec-tools still set up network based on ifcfg scripts which lead to the issues like [1] [2].
We can get network configuration including dns, bond and znet by parsing nmcli output instead. Another benefit is we could potentially avoid subtle bugs caused by namespace pollution because of sourcing ifcfg scripts.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1919052 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1933679
Coiby Xu (8): Parse option value from nmcli output Get nmcli connection id by ifname get nmcli connection show output by ifname setup s390 znet cmdline by parsing nmcli connection show output setup dns by parsing nmcli connection show output setup bond by parsing nmcli connection show output clean up codes related to ifcfg scripts clean up unused is_nm_* functions
dracut-module-setup.sh | 72 +++++++++++++--------------- kdump-lib.sh | 106 ++++++++++++----------------------------- 2 files changed, 65 insertions(+), 113 deletions(-)
The nmcli output has the format of "OPTION_NAME: VALUE", e.g. IP4.ROUTE[4]: dst = 192.168.43.0/24, nh = 0.0.0.0, mt = 600 IP4.DNS[1]: 192.168.43.1 802-3-ethernet.s390-options: layer2=1,portname=z-104,portno=0 802-3-ethernet.s390-subchannels: --
Signed-off-by: Coiby Xu coxu@redhat.com --- kdump-lib.sh | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+)
diff --git a/kdump-lib.sh b/kdump-lib.sh index 21271cf..333591c 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -372,6 +372,30 @@ get_hwaddr() fi }
+ +# Parse option value from the output of nmcli connection/device show +# +# The output has the format of "OPTION_NAME: VALUE", e.g. +# IP4.ROUTE[4]: dst = 192.168.43.0/24, nh = 0.0.0.0, mt = 600 +# IP4.DNS[1]: 192.168.43.1 +# 802-3-ethernet.s390-options: layer2=1,portname=z-104,portno=0 +# 802-3-ethernet.s390-subchannels: -- +# +# Use -F (interept patterns as fixed string) for grep. +get_nmcli_value_by_option() +{ + local _nmcli_out=$1 + local _option=$2 + + local val=$(grep -F $_option <<< "$_nmcli_out" | sed "s/.*:\s*//g") + + if [ "$val" = "--" ]; then + val="" + fi + + echo -n "$val" +} + get_ifcfg_by_device() { grep -E -i -l "^[[:space:]]*DEVICE="*${1}"*[[:space:]]*$" \
ID is used for nmcli connection operations, e.g. $ nmcli connection show $id
Signed-off-by: Coiby Xu coxu@redhat.com --- kdump-lib.sh | 13 +++++++++++++ 1 file changed, 13 insertions(+)
diff --git a/kdump-lib.sh b/kdump-lib.sh index 333591c..e13729c 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -396,6 +396,19 @@ get_nmcli_value_by_option() echo -n "$val" }
+# Get nmcli connection id by ifname +# ID is used for nmcli connection operations, e.g. +# $ nmcli connection show $id +get_nmcli_connection_id_by_ifname() +{ + local _ifname=$1 + local _nmcli_out=$(LANG=C nmcli device show $_ifname) + + local _nmcli_id=$(get_nmcli_value_by_option "$_nmcli_out" "GENERAL.CONNECTION") + + echo -n "$_nmcli_id" +} + get_ifcfg_by_device() { grep -E -i -l "^[[:space:]]*DEVICE="*${1}"*[[:space:]]*$" \
Signed-off-by: Coiby Xu coxu@redhat.com --- kdump-lib.sh | 11 +++++++++++ 1 file changed, 11 insertions(+)
diff --git a/kdump-lib.sh b/kdump-lib.sh index e13729c..1b1fc85 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -409,6 +409,17 @@ get_nmcli_connection_id_by_ifname() echo -n "$_nmcli_id" }
+# Get nmcli connection show output by ifname +get_nmcli_connection_show_by_ifname() +{ + local _ifname="$1" + local _conn_id=$(get_nmcli_connection_id_by_ifname "$_ifname") + local _nmcli_out=$(LANG=C nmcli connection show "$_conn_id") + + echo -n "$_nmcli_out" +} + + get_ifcfg_by_device() { grep -E -i -l "^[[:space:]]*DEVICE="*${1}"*[[:space:]]*$" \
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 24 +++++++++++++++--------- 1 file changed, 15 insertions(+), 9 deletions(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 21143b4..7034abf 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -318,17 +318,22 @@ kdump_setup_vlan() { }
# setup s390 znet cmdline -# $1: netdev name +# $1: netdev (ifname) +# $2: nmcli connection show output kdump_setup_znet() { + local _netdev="$1" + local _nmcli_out="$2" + local s390_prefix="s390-" local _options="" - local _netdev=$1 + local NETTYPE + local SUBCHANNELS
- source_ifcfg_file $_netdev + NETTYPE=$(get_nmcli_value_by_option "$_nmcli_out" "${s390_prefix}nettype") + SUBCHANNELS=$(get_nmcli_value_by_option "$_nmcli_out" "${s390_prefix}subchannels") + _options=$(get_nmcli_value_by_option "$_nmcli_out" "${s390_prefix}options")
- for i in $OPTIONS; do - _options=${_options},$i - done - echo rd.znet=${NETTYPE},${SUBCHANNELS}${_options} rd.znet_ifname=$_netdev:${SUBCHANNELS} > ${initdir}/etc/cmdline.d/30znet.conf + [ -z "$NETTYPE" ] || [ -z "$SUBCHANNELS" ] || [ -z "$_options" ] && die "Failed to setup s390 znet cmdline" + echo rd.znet=${NETTYPE},${SUBCHANNELS},${_options} rd.znet_ifname=$_netdev:${SUBCHANNELS} > ${initdir}/etc/cmdline.d/30znet.conf }
kdump_get_ip_route() @@ -362,17 +367,18 @@ kdump_get_remote_ip() # initramfs accessing giving destination # $1: destination host kdump_install_net() { - local _destaddr _srcaddr _route _netdev + local _destaddr _srcaddr _route _netdev _nmcli_out local _static _proto _ip_conf _ip_opts _ifname_opts
_destaddr=$(kdump_get_remote_ip $1) _route=$(kdump_get_ip_route $_destaddr) _srcaddr=$(kdump_get_ip_route_field "$_route" "src") _netdev=$(kdump_get_ip_route_field "$_route" "dev") + _nmcli_out=$(get_nmcli_connection_show_by_ifname "$_netdev") _netmac=$(kdump_get_mac_addr $_netdev)
if [ "$(uname -m)" = "s390x" ]; then - kdump_setup_znet $_netdev + kdump_setup_znet "$_netdev" "$_nmcli_out" fi
_static=$(kdump_static_ip $_netdev $_srcaddr)
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 7034abf..0e7f046 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -97,15 +97,16 @@ source_ifcfg_file() { fi }
-# $1: netdev name +# $1: nmcli connection show output kdump_setup_dns() { - local _nameserver _dns + local _nmcli_out="$1" + local _nameserver _dns _dns1 _dns2 local _dnsfile=${initdir}/etc/cmdline.d/42dns.conf
- source_ifcfg_file $1 - - [ -n "$DNS1" ] && echo "nameserver=$DNS1" > "$_dnsfile" - [ -n "$DNS2" ] && echo "nameserver=$DNS2" >> "$_dnsfile" + _dns1=$(get_nmcli_value_by_option "$_nmcli_out" "IP4.DNS[1]") + [ -n "$_dns1" ] && echo "nameserver=$_dns1" > "$_dnsfile" + _dns2=$(get_nmcli_value_by_option "$_nmcli_out" "IP4.DNS[2]") + [ -n "$_dns2" ] && echo "nameserver=$_dn2" >> "$_dnsfile"
while read content; do @@ -415,7 +416,7 @@ kdump_install_net() { echo "$_ifname_opts" >> $_ip_conf fi
- kdump_setup_dns "$_netdev" + kdump_setup_dns "$_nmcli_out"
if [ ! -f ${initdir}/etc/cmdline.d/50neednet.conf ]; then # network-manager module needs this parameter
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 21 +++++++++++---------- 1 file changed, 11 insertions(+), 10 deletions(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 0e7f046..1351743 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -243,7 +243,7 @@ kdump_setup_bridge() { for _dev in `ls /sys/class/net/$_netdev/brif/`; do _kdumpdev=$_dev if kdump_is_bond "$_dev"; then - kdump_setup_bond "$_dev" + kdump_setup_bond "$_dev" "$(get_nmcli_connection_show_by_ifname "$_dev")" elif kdump_is_team "$_dev"; then kdump_setup_team "$_dev" elif kdump_is_vlan "$_dev"; then @@ -258,9 +258,13 @@ kdump_setup_bridge() { echo " bridge=$_netdev:$(echo $_brif | sed -e 's/,$//')" >> ${initdir}/etc/cmdline.d/41bridge.conf }
+# drauct takes bond=<bondname>[:<bondslaves>:[:<options>]] syntax to parse +# bond. For example: +# bond=bond0:eth0,eth1:mode=balance-rr kdump_setup_bond() { - local _netdev=$1 - local _dev _mac _slaves _kdumpdev + local _netdev="$1" + local _nmclid_out="$2" + local _dev _mac _slaves _kdumpdev _bondoptions for _dev in `cat /sys/class/net/$_netdev/bonding/slaves`; do _mac=$(kdump_get_perm_addr $_dev) _kdumpdev=$(kdump_setup_ifname $_dev) @@ -268,12 +272,9 @@ kdump_setup_bond() { _slaves+="$_kdumpdev," done echo -n " bond=$_netdev:$(echo $_slaves | sed 's/,$//')" >> ${initdir}/etc/cmdline.d/42bond.conf - # Get bond options specified in ifcfg - - source_ifcfg_file $_netdev
- bondoptions=":$(echo $BONDING_OPTS | xargs echo | tr " " ",")" - echo "$bondoptions" >> ${initdir}/etc/cmdline.d/42bond.conf + _bondoptions=":$(get_nmcli_value_by_option "$_nmcli_out" "bond.options")" + echo "$_bondoptions" >> ${initdir}/etc/cmdline.d/42bond.conf }
kdump_setup_team() { @@ -310,7 +311,7 @@ kdump_setup_vlan() { derror "Vlan over bridge is not supported!" exit 1 elif kdump_is_bond "$_phydev"; then - kdump_setup_bond "$_phydev" + kdump_setup_bond "$_phydev" "$(get_nmcli_connection_show_by_ifname "$_phydev")" echo " vlan=$(kdump_setup_ifname $_netdev):$_phydev" > ${initdir}/etc/cmdline.d/43vlan.conf else _kdumpdev="$(kdump_setup_ifname $_phydev)" @@ -406,7 +407,7 @@ kdump_install_net() { if kdump_is_bridge "$_netdev"; then kdump_setup_bridge "$_netdev" elif kdump_is_bond "$_netdev"; then - kdump_setup_bond "$_netdev" + kdump_setup_bond "$_netdev" "$_nmcli_out" elif kdump_is_team "$_netdev"; then kdump_setup_team "$_netdev" elif kdump_is_vlan "$_netdev"; then
Signed-off-by: Coiby Xu coxu@redhat.com --- dracut-module-setup.sh | 12 ------- kdump-lib.sh | 81 ------------------------------------------ 2 files changed, 93 deletions(-)
diff --git a/dracut-module-setup.sh b/dracut-module-setup.sh index 1351743..82a0e7d 100755 --- a/dracut-module-setup.sh +++ b/dracut-module-setup.sh @@ -85,18 +85,6 @@ kdump_is_vlan() { [ -f /proc/net/vlan/"$1" ] }
-# $1: netdev name -source_ifcfg_file() { - local ifcfg_file - - ifcfg_file=$(get_ifcfg_filename $1) - if [ -f "${ifcfg_file}" ]; then - . ${ifcfg_file} - else - dwarning "The ifcfg file of $1 is not found!" - fi -} - # $1: nmcli connection show output kdump_setup_dns() { local _nmcli_out="$1" diff --git a/kdump-lib.sh b/kdump-lib.sh index 1b1fc85..1a8247c 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -419,31 +419,6 @@ get_nmcli_connection_show_by_ifname() echo -n "$_nmcli_out" }
- -get_ifcfg_by_device() -{ - grep -E -i -l "^[[:space:]]*DEVICE="*${1}"*[[:space:]]*$" \ - /etc/sysconfig/network-scripts/ifcfg-* 2>/dev/null | head -1 -} - -get_ifcfg_by_hwaddr() -{ - grep -E -i -l "^[[:space:]]*HWADDR="*${1}"*[[:space:]]*$" \ - /etc/sysconfig/network-scripts/ifcfg-* 2>/dev/null | head -1 -} - -get_ifcfg_by_uuid() -{ - grep -E -i -l "^[[:space:]]*UUID="*${1}"*[[:space:]]*$" \ - /etc/sysconfig/network-scripts/ifcfg-* 2>/dev/null | head -1 -} - -get_ifcfg_by_name() -{ - grep -E -i -l "^[[:space:]]*NAME="*${1}"*[[:space:]]*$" \ - /etc/sysconfig/network-scripts/ifcfg-* 2>/dev/null | head -1 -} - is_nm_running() { [ "$(LANG=C nmcli -t --fields running general status 2>/dev/null)" = "running" ] @@ -455,62 +430,6 @@ is_nm_handling() | grep -q "^(${1}:connected)|(${1}:connecting.*)$" }
-# $1: netdev name -get_ifcfg_nmcli() -{ - local nm_uuid nm_name - local ifcfg_file - - # Get the active nmcli config name of $1 - if is_nm_running && is_nm_handling "${1}" ; then - # The configuration "uuid" and "name" generated by nm is wrote to - # the ifcfg file as "UUID=<nm_uuid>" and "NAME=<nm_name>". - nm_uuid=$(LANG=C nmcli -t --fields uuid,device c show --active 2>/dev/null \ - | grep "${1}" | head -1 | cut -d':' -f1) - nm_name=$(LANG=C nmcli -t --fields name,device c show --active 2>/dev/null \ - | grep "${1}" | head -1 | cut -d':' -f1) - ifcfg_file=$(get_ifcfg_by_uuid "${nm_uuid}") - [ -z "${ifcfg_file}" ] && ifcfg_file=$(get_ifcfg_by_name "${nm_name}") - fi - - echo -n "${ifcfg_file}" -} - -# $1: netdev name -get_ifcfg_legacy() -{ - local ifcfg_file - - ifcfg_file="/etc/sysconfig/network-scripts/ifcfg-${1}" - [ -f "${ifcfg_file}" ] && echo -n "${ifcfg_file}" && return - - ifcfg_file=$(get_ifcfg_by_name "${1}") - [ -f "${ifcfg_file}" ] && echo -n "${ifcfg_file}" && return - - local hwaddr=$(get_hwaddr "${1}") - if [ -n "$hwaddr" ]; then - ifcfg_file=$(get_ifcfg_by_hwaddr "${hwaddr}") - [ -f "${ifcfg_file}" ] && echo -n "${ifcfg_file}" && return - fi - - ifcfg_file=$(get_ifcfg_by_device "${1}") - - echo -n "${ifcfg_file}" -} - -# $1: netdev name -# Return the ifcfg file whole name(including the path) of $1 if any. -get_ifcfg_filename() { - local ifcfg_file - - ifcfg_file=$(get_ifcfg_nmcli "${1}") - if [ -z "${ifcfg_file}" ]; then - ifcfg_file=$(get_ifcfg_legacy "${1}") - fi - - echo -n "${ifcfg_file}" -} - # returns 0 when omission of a module is desired in dracut_args # returns 1 otherwise is_dracut_mod_omitted() {
Signed-off-by: Coiby Xu coxu@redhat.com --- kdump-lib.sh | 11 ----------- 1 file changed, 11 deletions(-)
diff --git a/kdump-lib.sh b/kdump-lib.sh index 1a8247c..b58a5e6 100755 --- a/kdump-lib.sh +++ b/kdump-lib.sh @@ -419,17 +419,6 @@ get_nmcli_connection_show_by_ifname() echo -n "$_nmcli_out" }
-is_nm_running() -{ - [ "$(LANG=C nmcli -t --fields running general status 2>/dev/null)" = "running" ] -} - -is_nm_handling() -{ - LANG=C nmcli -t --fields device,state dev status 2>/dev/null \ - | grep -q "^(${1}:connected)|(${1}:connecting.*)$" -} - # returns 0 when omission of a module is desired in dracut_args # returns 1 otherwise is_dracut_mod_omitted() {
Hi Coiby, thanks for the patch.
On Thu, Mar 18, 2021 at 10:33 AM Coiby Xu coxu@redhat.com wrote:
ifcfg scripts are deprecated. kexec-tools still set up network based on ifcfg scripts which lead to the issues like [1] [2].
We can get network configuration including dns, bond and znet by parsing nmcli output instead. Another benefit is we could potentially avoid subtle bugs caused by namespace pollution because of sourcing ifcfg scripts.
After checking nmcli doc, it has some params just for using withing scripts, like: nmcli -g GENERAL.DEVICE -c no device show enp0s31f6
`-g` can help extract specified value, so maybe we can avoid the awk/sed calls `-c no` disables the coloring of output, which should improve the performance. Also, like `-t` can remove these redundant spacing and make it easier to parse.
Maybe we can improve this patch series with these params.
And also we can keep the old ifcfg compatibility for now, and drop it later, in case of any user sticking with old-style configurations.
Another concern is that `nmcli` is about 20x times slower than `ip`, can you help measure the performance impact of this change?
And also, please also add [PATCH] prefix for the cover letter.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1919052 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1933679
Coiby Xu (8): Parse option value from nmcli output Get nmcli connection id by ifname get nmcli connection show output by ifname setup s390 znet cmdline by parsing nmcli connection show output setup dns by parsing nmcli connection show output setup bond by parsing nmcli connection show output clean up codes related to ifcfg scripts clean up unused is_nm_* functions
dracut-module-setup.sh | 72 +++++++++++++--------------- kdump-lib.sh | 106 ++++++++++++----------------------------- 2 files changed, 65 insertions(+), 113 deletions(-)
-- 2.30.1 _______________________________________________ kexec mailing list -- kexec@lists.fedoraproject.org To unsubscribe send an email to kexec-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/kexec@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- Best Regards, Kairui Song
Hi Kairui,
Thanks for reviewing this patch set and the advice!
On Wed, Mar 24, 2021 at 03:22:43PM +0800, Kairui Song wrote:
Hi Coiby, thanks for the patch.
On Thu, Mar 18, 2021 at 10:33 AM Coiby Xu coxu@redhat.com wrote:
ifcfg scripts are deprecated. kexec-tools still set up network based on ifcfg scripts which lead to the issues like [1] [2].
We can get network configuration including dns, bond and znet by parsing nmcli output instead. Another benefit is we could potentially avoid subtle bugs caused by namespace pollution because of sourcing ifcfg scripts.
After checking nmcli doc, it has some params just for using withing scripts, like: nmcli -g GENERAL.DEVICE -c no device show enp0s31f6
`-g` can help extract specified value, so maybe we can avoid the awk/sed calls `-c no` disables the coloring of output, which should improve the performance. Also, like `-t` can remove these redundant spacing and make it easier to parse.
Maybe we can improve this patch series with these params.
This issue is related to the third one.
And also we can keep the old ifcfg compatibility for now, and drop it later, in case of any user sticking with old-style configurations.
I'll keep the old ifcfg compatibility in next version. Thanks for the reminder.
Another concern is that `nmcli` is about 20x times slower than `ip`, can you help measure the performance impact of this change?
Although `nmcli` is much slower than `ip`, this patch set actually improves the performance compared with "source ifcfg-script".
There are two data sets comparing the execution time (second) of kdump_install_net obtained on ibm-z-110.rhts.eng.bos.redhat.com and hpe-dl585g5-01.hpe2.lab.eng.bos.redhat.com for the following approaches, - a: source ifcfg scripts - b: get nmcli output once and extract the values if needed - c: use "nmclig -g <field>"
a b c
ibm 0.20 0.11 0.14
hpe 0.31 0.25 0.24
So the results are determined by the times of "nmcli -g" being called but the nmcli approaches should be always faster than the old approach.
Btw "time kdumpctl rebuild" shows 24.062s on ibm-z-110.rhts.eng.bos.redhat.com. So kdump_install_net only takes 1% of the building the kdump initramfs. So I'll take your "nmcli -g" approach in the next version.
Here's the script to measure the execution time of kdump_install_net,
. /usr/lib/dracut/modules.d/99kdumpbase/module-setup.sh . /lib/kdump/kdump-lib.sh
START=$(date +%s.%N) # kdump_setup_netdev enc8000 kdump_install_net 10.73.194.36 END=$(date +%s.%N) DIFF=$(echo "$END - $START" | bc) echo $DIFF
And also, please also add [PATCH] prefix for the cover letter.
Sorry, "git format-patch" have added the prefix automatically but I removed it when editing the subject of the cover letter.
[1] https://bugzilla.redhat.com/show_bug.cgi?id=1919052 [2] https://bugzilla.redhat.com/show_bug.cgi?id=1933679
Coiby Xu (8): Parse option value from nmcli output Get nmcli connection id by ifname get nmcli connection show output by ifname setup s390 znet cmdline by parsing nmcli connection show output setup dns by parsing nmcli connection show output setup bond by parsing nmcli connection show output clean up codes related to ifcfg scripts clean up unused is_nm_* functions
dracut-module-setup.sh | 72 +++++++++++++--------------- kdump-lib.sh | 106 ++++++++++++----------------------------- 2 files changed, 65 insertions(+), 113 deletions(-)
-- 2.30.1 _______________________________________________ kexec mailing list -- kexec@lists.fedoraproject.org To unsubscribe send an email to kexec-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/kexec@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
-- Best Regards, Kairui Song
-
Coiby Xu -
Kairui Song