New subject: [kubevirt-dev] NMPolicy - a cmd/lib tool to generate dynamic NMState configurations

16 Dec 2021


      Hi all,
The kubevirt networking team is working on a new cmd/lib tool to generate
dynamic NMState configurations.
When networking configuration for a cluster is needed and all the details
are common between the nodes in the cluster a NMState yaml configuration is
enough.
Problems arise when some of the network configuration details are different
between nodes and depend on the current node network state.
For that a different NMState yaml configuration needs to be generated per
node and that's not convenient for big clusters and also at scale up
scenarios.
The NMPolicy goal is to solve this problem.
Given a node network state and a network configuration policy (common to
the cluster), the NMPolicy tool will generate a node specific desired
network state.
NMPolicy was already integrated to kubernetes-nmstate and a usage example
can be found here
https://github.com/nmstate/kubernetes-nmstate/blob/main/test/e2e/handler/default_bridged_network_with_nmpolicy_test.go
.
*CLI* Usage example -
Simulated --help output
./nmpolicyctl -h
This tool helps you to generate dynamic NMState configurations.
The generated NMState configuration is written to *STDOUT*.
Usage of ./*nmpolicyctl*:
Commands:
*gen* Generates NMState by policy filename
Flags:
-s, --current-state=current-state.yaml: input file path to current
NMState. If not specified, stdin is used.
-c, --captured-state-output=captured-state-output.yaml: output file
path to the emitted captured states. If not specified,
~/.cache/nmpolicy/cache.yaml will be used.
*Creating a new bridge with the base interface as its port. The bridge will
get the mac address of the base interface.*
# echo """
capture:
*default-gw: routes.running.destination=="0.0.0.0/0 http://0.0.0.0/0"*
*  base-iface: interfaces.name
http://interfaces.name==capture.default-gw.routes.running.0.next-hop-interface*
desiredState:
interfaces:
- name: br1
description: Linux bridge with base interface as a port
type: linux-bridge
state: up
mac-address: "{{ *capture.base-iface.interfaces.**0.mac-address* }}"
ipv4:
dhcp: true
enabled: true
bridge:
options:
stp:
enabled: false
port:
- name: "{{ *capture.base-iface.interfaces.0.name
http://capture.base-iface.interfaces.0.name* }}"
""" > *policy.yaml*
*# nmstatectl show | nmpolicyctl gen policy.yaml | nmstatectl apply*
*Output of - nmstatectl show | nmpolicyctl gen policy.yaml*
~/.cache/nmpolicy/*cache*.yaml (the default captured states output file)
base-iface:
metaInfo:
time: "2021-12-15T13:45:40Z"
state:
interfaces:
- accept-all-mac-addresses: false
ethernet:
auto-negotiation: false
ethtool:
feature:
rx-gro: true
rx-gro-list: false
rx-udp-gro-forwarding: false
tx-checksum-ip-generic: true
tx-generic-segmentation: true
tx-nocache-copy: false
tx-tcp-ecn-segmentation: true
tx-tcp-mangleid-segmentation: false
tx-tcp-segmentation: true
tx-tcp6-segmentation: true
ring:
rx: 256
tx: 256
ipv4:
address:
- ip: 192.168.66.101
prefix-length: 24
auto-dns: true
auto-gateway: true
auto-route-table-id: 0
auto-routes: true
dhcp: true
enabled: true
ipv6:
address:
- ip: fd00::101
prefix-length: 128
- ip: fe80::2d55:7c37:8090:7261
prefix-length: 64
auto-dns: true
auto-gateway: true
auto-route-table-id: 0
auto-routes: true
autoconf: true
dhcp: true
enabled: true
lldp:
enabled: false
*mac-address: 52:55:00:D1:55:01*
mtu: 1500
*name: eth0*
state: up
type: ethernet
default-gw:
metaInfo:
time: "2021-12-15T13:45:40Z"
state:
routes:
running:
- *destination: 0.0.0.0/0 http://0.0.0.0/0*
metric: 100
next-hop-address: 192.168.66.2
*next-hop-interface: eth0*
table-id: 254
*Stdout* (the *desired state* to be applied by nmstatectl)
interfaces:
- bridge:
options:
stp:
enabled: false
port:
- *name: eth0*
vlan:
mode: trunk
trunk-tags:
- id-range:
max: 4094
min: 2
description: Linux bridge with base interface as a port
ipv4:
dhcp: true
enabled: true
*mac-address: 52:55:00:D1:55:01*
*name: br1*
state: up
type: linux-bridge
For more details, please review the design doc
https://docs.google.com/document/d/1k8gljRJoGtCGXoQ3YkTpDmekLAIeUWZ4P3-oPfjn3kk
and share your feedback about it.
Thanks,
Alona.