[vdsm] Fedora, udev and nic renaming
Alon Bar-Lev
alonbl at redhat.com
Tue Dec 4 10:25:48 UTC 2012
Thanks for this verbose description.
I don't think using libguestfs is the solution for this.
Fixing qemu to accept BIOS interface name at -net parameter is preferable. I don't think we should expose the interface a PCI device as it will have some drawbacks, but attempt to use the onboard convention.
Alon
----- Original Message -----
> From: "Antoni Segura Puimedon" <asegurap at redhat.com>
> To: vdsm-devel at lists.fedorahosted.org
> Sent: Tuesday, December 4, 2012 11:08:31 AM
> Subject: [vdsm] Fedora, udev and nic renaming
>
> Hi,
>
> We are currently working on stabilizing the networking part of vdsm
> in Fedora
> 18 and, to achieve that purpose, we decided to test in in both
> physical hosts
> and, for extra convenience and better support, also in VMs.
>
> Due to the move of Fedora 17 and 18 to systemd and newer udev
> versions, we
> encountered some issues that should be noted and worked on to provide
> our users
> with a hassle-free experience.
>
> First of all, let me state what happens (in renaming) in RHEL-6.x
> when a new
> ethernet device is handled by udev:
> a) One or more udev rules match the characteristics of the interface:
> The last
> matching rule is applied.
> b) No rule is matching: /lib/udev/write-net-rules writes a permanent
> rule using the MAC address of the interface in a udev rules file, so
> the
> interface name will be permanent and in the ethX namespace.
>
> In Fedora 17 (but even more so in F18), with the move to a newer
> version of
> udev and, specially, with the change from sysV init to systemd, the
> mechanism
> changed. Since systemd is making the boot happen in a parallelized
> way, some
> changes had to be enforced in udev to keep the renaming working:
>
> - To avoid naming collisions, it was decided to use Dell's
> biosdevname software
> to retrieve a device name for the network interfaces. Typically emX
> for
> onboard nics and pXpY for pci connected nics.
> - For devices which biosdevname could not provide any information, it
> was
> agreed to assign them a name in the ethX space in a first-come,
> first-served
> fashion.
> - Optionally, one could define the interace MAC addr in an ifcfg file
> and /lib/udev/rename-device would look into the ifcfg file and
> assign
> the device name there set (I have not yet succeeded in that part, I
> have to
> investigate more, I guess).
>
> As you can see, biosdevname, never reports names in the eth space to
> avoid
> collision with a potential parallel discovery of an interface not
> recognizable
> by it, to which the kernel could have assigned already a bios
> reported name.
>
> For physical machines this approach works fine. However, for Virtual
> machines
> with more than one nic, the automatic process described above
> presents some
> issues. Biosdevname, due to the different ways the virtualization
> hypervisors
> report the vnics, dropped support for VMs in 0.3.7 (F18 uses 0.4.1-2)
> and
> decided that on VMs, it would just return "4" to indicate to udev to
> use
> kernel first-come, first-served for those interfaces (ethX
> namespace).
>
> The issue with using first-come first-served, is that due to the
> highly
> parallelized boot there is now, it is very common to encounter that
> the names
> of your devices (as identified by MAC address) suffer a permutation
> upon each
> reboot. Here you can see an example:
>
> NOTE: The libvirt dump of the VM reports the same PCI address for
> each
> interface across reboots.
>
> Boot 0 (Nov 13th 14:59)
> eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:54:85:57 txqueuelen 1000 (Ethernet)
> eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:77:45:6b txqueuelen 1000 (Ethernet)
> eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:ca:41:c7 txqueuelen 1000 (Ethernet)
> eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:f5:3d:c8 txqueuelen 1000 (Ethernet)
> eth4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:5e:10:76 txqueuelen 1000 (Ethernet)
> eth5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:95:00:93 txqueuelen 1000 (Ethernet)
>
> Boot 1 (Nov 13th 15:01)
> eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:ca:41:c7 txqueuelen 1000 (Ethernet)
> eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:54:85:57 txqueuelen 1000 (Ethernet)
> eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:77:45:6b txqueuelen 1000 (Ethernet)
> eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:f5:3d:c8 txqueuelen 1000 (Ethernet)
> eth4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:5e:10:76 txqueuelen 1000 (Ethernet)
> eth5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
> ether 52:54:00:95:00:93 txqueuelen 1000 (Ethernet)
>
> As you can see, after rebooting:
> eth0 -> eth1
> eth1 -> eth2
> eth2 -> eth0
>
> This is an issue if different vnics are connected to different
> networks or for
> whichever reason require distinct configuration. To solve this issue,
> on the
> guest there are three options:
>
> - Assign somebody with BIOS knowledge to add KVM guest support to
> biosdevname
> so we can use the emX/pXpY namespace and maintain a native like
> experience in
> the VMs. My intuition is that it could just report pXpY where X is
> bus and Y
> is slot. This is the preferred option.
> - Use libguestfs for setting udev rules using the MAC addresses we
> know from
> the VM definition in the netX namespace (I have been told that it
> is not
> entirely safe to use emX namespace for this, but hopefully for VMs
> this could
> be done safely or, at least, pXpY).
> - Use libguestfs to provide ifcfg files that specify the vnics' MAC
> addresses
> so they are picked up by udev's rename process (Have to find out
> how to make
> that work, as I have not managed to do it yet).
>
> Best,
>
> Toni
>
>
> PS: For physical machines, beware that some usb network devices don't
> have very
> original MAC addresses or don't have a mac address burned in
> (randomized at
> each boot) and, for this reason, they can create quite a lot of
> issues and more
> complicated (as in using bus and device and/or ID) udev rules should
> be used.
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
>
More information about the vdsm-devel
mailing list