[vdsm] Fedora, udev and nic renaming

Alon Bar-Lev alonbl at redhat.com
Tue Dec 4 10:25:48 UTC 2012


Thanks for this verbose description.

I don't think using libguestfs is the solution for this.

Fixing qemu to accept BIOS interface name at -net parameter is preferable. I don't think we should expose the interface a PCI device as it will have some drawbacks, but attempt to use the onboard convention.

Alon

----- Original Message -----
> From: "Antoni Segura Puimedon" <asegurap at redhat.com>
> To: vdsm-devel at lists.fedorahosted.org
> Sent: Tuesday, December 4, 2012 11:08:31 AM
> Subject: [vdsm] Fedora, udev and nic renaming
> 
> Hi,
> 
> We are currently working on stabilizing the networking part of vdsm
> in Fedora
> 18 and, to achieve that purpose, we decided to test in in both
> physical hosts
> and, for extra convenience and better support, also in VMs.
> 
> Due to the move of Fedora 17 and 18 to systemd and newer udev
> versions, we
> encountered some issues that should be noted and worked on to provide
> our users
> with a hassle-free experience.
> 
> First of all, let me state what happens (in renaming) in RHEL-6.x
> when a new
> ethernet device is handled by udev:
> a) One or more udev rules match the characteristics of the interface:
> The last
> matching rule is applied.
> b) No rule is matching: /lib/udev/write-net-rules writes a permanent
> rule using the MAC address of the interface in a udev rules file, so
> the
> interface name will be permanent and in the ethX namespace.
> 
> In Fedora 17 (but even more so in F18), with the move to a newer
> version of
> udev and, specially, with the change from sysV init to systemd, the
> mechanism
> changed. Since systemd is making the boot happen in a parallelized
> way, some
> changes had to be enforced in udev to keep the renaming working:
> 
> - To avoid naming collisions, it was decided to use Dell's
> biosdevname software
>   to retrieve a device name for the network interfaces. Typically emX
>   for
>   onboard nics and pXpY for pci connected nics.
> - For devices which biosdevname could not provide any information, it
> was
>   agreed to assign them a name in the ethX space in a first-come,
>   first-served
>   fashion.
> - Optionally, one could define the interace MAC addr in an ifcfg file
>   and /lib/udev/rename-device would look into the ifcfg file and
>   assign
>   the device name there set (I have not yet succeeded in that part, I
>   have to
>   investigate more, I guess).
> 
> As you can see, biosdevname, never reports names in the eth space to
> avoid
> collision with a potential parallel discovery of an interface not
> recognizable
> by it, to which the kernel could have assigned already a bios
> reported name.
> 
> For physical machines this approach works fine. However, for Virtual
> machines
> with more than one nic, the automatic process described above
> presents some
> issues. Biosdevname, due to the different ways the virtualization
> hypervisors
> report the vnics, dropped support for VMs in 0.3.7 (F18 uses 0.4.1-2)
> and
> decided that on VMs, it would just return "4" to indicate to udev to
> use
> kernel first-come, first-served for those interfaces (ethX
> namespace).
> 
> The issue with using first-come first-served, is that due to the
> highly
> parallelized boot there is now, it is very common to encounter that
> the names
> of your devices (as identified by MAC address) suffer a permutation
> upon each
> reboot. Here you can see an example:
> 
> NOTE: The libvirt dump of the VM reports the same PCI address for
> each
> interface across reboots.
> 
> Boot 0 (Nov 13th 14:59)
>     eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:54:85:57  txqueuelen 1000  (Ethernet)
>     eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:77:45:6b  txqueuelen 1000  (Ethernet)
>     eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:ca:41:c7  txqueuelen 1000  (Ethernet)
>     eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:f5:3d:c8  txqueuelen 1000  (Ethernet)
>     eth4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:5e:10:76  txqueuelen 1000  (Ethernet)
>     eth5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:95:00:93  txqueuelen 1000  (Ethernet)
> 
> Boot 1 (Nov 13th 15:01)
>     eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:ca:41:c7  txqueuelen 1000  (Ethernet)
>     eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:54:85:57  txqueuelen 1000  (Ethernet)
>     eth2: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:77:45:6b  txqueuelen 1000  (Ethernet)
>     eth3: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:f5:3d:c8  txqueuelen 1000  (Ethernet)
>     eth4: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:5e:10:76  txqueuelen 1000  (Ethernet)
>     eth5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
>         ether 52:54:00:95:00:93  txqueuelen 1000  (Ethernet)
> 
> As you can see, after rebooting:
> eth0 -> eth1
> eth1 -> eth2
> eth2 -> eth0
> 
> This is an issue if different vnics are connected to different
> networks or for
> whichever reason require distinct configuration. To solve this issue,
> on the
> guest there are three options:
> 
> - Assign somebody with BIOS knowledge to add KVM guest support to
> biosdevname
>   so we can use the emX/pXpY namespace and maintain a native like
>   experience in
>   the VMs. My intuition is that it could just report pXpY where X is
>   bus and Y
>   is slot. This is the preferred option.
> - Use libguestfs for setting udev rules using the MAC addresses we
> know from
>   the VM definition in the netX namespace (I have been told that it
>   is not
>   entirely safe to use emX namespace for this, but hopefully for VMs
>   this could
>   be done safely or, at least, pXpY).
> - Use libguestfs to provide ifcfg files that specify the vnics' MAC
> addresses
>   so they are picked up by udev's rename process (Have to find out
>   how to make
>   that work, as I have not managed to do it yet).
> 
> Best,
>  
> Toni
> 
> 
> PS: For physical machines, beware that some usb network devices don't
> have very
> original MAC addresses or don't have a mac address burned in
> (randomized at
> each boot) and, for this reason, they can create quite a lot of
> issues and more
> complicated (as in using bus and device and/or ID) udev rules should
> be used.
> _______________________________________________
> vdsm-devel mailing list
> vdsm-devel at lists.fedorahosted.org
> https://lists.fedorahosted.org/mailman/listinfo/vdsm-devel
> 


More information about the vdsm-devel mailing list