Change in vdsm[master]: multipath: Disable queuing on all multipath devices

nsoffer at redhat.com nsoffer at redhat.com
Mon Sep 1 23:41:41 UTC 2014


Nir Soffer has uploaded a new change for review.

Change subject: multipath: Disable queuing on all multipath devices
......................................................................

multipath: Disable queuing on all multipath devices

We specify "no_path_retry fail" in multipath configuration, but this
setting is used only for devices which do not specify their own settings
in multipath built-in device database.

For example, as we can see in "multipath -ll" output, this device is
using the queue_if_no_path feature:

    # multipath -ll
    360060160f4a030003268ab211002e411 dm-1 DGC     ,VRAID
    size=30G features='1 queue_if_no_path' hwhandler='1 emc' wp=rw
    |-+- policy='service-time 0' prio=4 status=active
    | `- 4:0:3:1 sde 8:64 active ready  running
    `-+- policy='service-time 0' prio=1 status=enabled
      `- 4:0:2:1 sdc 8:32 active ready  running

This particular device is using "no_path_retry 60" in multipath built-in
database. Using 5 seconds timeout, this means that any read or write to
this device may cause the process to block in D state for 5 minutes.

This issue causes lvm to block for many minutes when a device which is
*not* used by vdsm is unmapped on the storage server. On the next lvm
refresh, when lvm look for available pvs, it will block for 5 minutes on
such device. This is the root cause of many bugs since the first
versions of vdsm.

Unfortunately, multipath does not provide way to override the built-in
device database in multipath configuration. I tried several partly
working solutions:

- Implement the "no_path_retry fail" in vdsm, by not adding devices
  without active path to lvm filter. This should work for most cases,
  but is not the right place to fix this problem.
  http://gerrit.ovirt.org/31875

- Add a special (.*) device configuration matching all devices and
  overriding the no_path_retry option. This works - but also overrides
  other device configuration.

- Use disablequeuing multipathd command line option. This works - until
  multipathd service is reloaded.

This patch adds a udev rule for multipath devices, invoked after a
device is added or changed, disabling queuing for the device. This is
the same solution we use for configuring lvm volumes and setting
permissions on volumes. To disable queuing, I'm using dmsetup, as
documented in multipath administration guide.

Change-Id: I6bfe68f4a8d233afe1bb787ce028ffb4c167469b
Bug-Url: https://bugzilla.redhat.com/880738
Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
M .gitignore
M configure.ac
M debian/rules
M debian/vdsm.install
M vdsm.spec.in
A vdsm/storage/vdsm-multipath.rules.in
6 files changed, 31 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/77/32277/1

diff --git a/.gitignore b/.gitignore
index 5890806..fca4f2e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -61,6 +61,7 @@
 vdsm/storage/protect/safelease
 vdsm/storage/lvm.env
 vdsm/storage/vdsm-lvm.rules
+vdsm/storage/vdsm-multipath.rules
 vdsm/sudoers.vdsm
 vdsm/svdsm.logger.conf
 vdsm/vdscli.py
diff --git a/configure.ac b/configure.ac
index 6f00217..f40d57a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -338,6 +338,7 @@
 	vdsm/storage/imageRepository/Makefile
 	vdsm/storage/protect/Makefile
 	vdsm/storage/vdsm-lvm.rules
+	vdsm/storage/vdsm-multipath.rules
 	vdsm/virt/Makefile
 	vdsm_hooks/Makefile
 	vdsm_hooks/checkimages/Makefile
diff --git a/debian/rules b/debian/rules
index b32b160..6e37104 100755
--- a/debian/rules
+++ b/debian/rules
@@ -40,6 +40,9 @@
 	# Install the lvm rules
 	install -Dm 0644 vdsm/storage/vdsm-lvm.rules \
 		$(destdir)/lib/udev/rules.d/61-vdsm-lvm.rules
+	# Install the multipath rules
+	install -Dm 0644 vdsm/storage/vdsm-multipath.rules \
+		$(destdir)/lib/udev/rules.d/99-vdsm-multipath.rules
 	install -Dm 0644 vdsm/limits.conf \
 		$(destdir)/etc/security/limits.d/99-vdsm.conf
 	install -Dm 0644 init/upstart/vdsmd.upstart \
diff --git a/debian/vdsm.install b/debian/vdsm.install
index b1840ec..ae807fc 100644
--- a/debian/vdsm.install
+++ b/debian/vdsm.install
@@ -21,6 +21,7 @@
 ./etc/vdsm/svdsm.logger.conf
 ./etc/vdsm/vdsm-tmpfiles.conf
 ./lib/udev/rules.d/61-vdsm-lvm.rules
+./lib/udev/rules.d/99-vdsm-multipath.rules
 ./usr/lib/python2.7/dist-packages/sos/plugins/vdsm.py
 ./usr/lib/python2.7/dist-packages/vdsmapi.py
 ./usr/libexec/vdsm/curl-img-wrap
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 323b586..3c20f01 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -657,6 +657,10 @@
 install -Dm 0644 vdsm/storage/vdsm-lvm.rules \
                  %{buildroot}%{_udevrulesdir}/12-vdsm-lvm.rules
 
+# Install the multipath rules
+install -Dm 0644 vdsm/storage/vdsm-multipath.rules \
+                 %{buildroot}%{_udevrulesdir}/99-vdsm-multipath.rules
+
 install -Dm 0644 vdsm/limits.conf \
                  %{buildroot}/etc/security/limits.d/99-vdsm.conf
 
@@ -1133,6 +1137,7 @@
 %endif
 %{python_sitelib}/sos/plugins/vdsm.py*
 %{_udevrulesdir}/12-vdsm-lvm.rules
+%{_udevrulesdir}/99-vdsm-multipath.rules
 /etc/security/limits.d/99-vdsm.conf
 %{_mandir}/man8/vdsmd.8*
 %if 0%{?rhel}
diff --git a/vdsm/storage/vdsm-multipath.rules.in b/vdsm/storage/vdsm-multipath.rules.in
new file mode 100644
index 0000000..8fdd503
--- /dev/null
+++ b/vdsm/storage/vdsm-multipath.rules.in
@@ -0,0 +1,20 @@
+#
+# Copyright 2014 Red Hat, Inc. and/or its affiliates.
+#
+# Licensed to you under the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.  See the files README and
+# LICENSE_GPL_v2 which accompany this distribution.
+#
+
+# Vdsm udev rules for multipath devices.
+#
+# Ensure that multipath devices use no_path_retry fail, instead of
+# no_path_retry queue, which is hardcoded in multipath configuration for some
+# devices.
+#
+# See https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html-single/DM_Multipath/index.html
+# section 5.6. ISSUES WITH QUEUE_IF_NO_PATH FEATUR.
+#
+
+ACTION=="add|change", ENV{DM_UUID}=="mpath-?*", RUN+="@DMSETUP_PATH@ message $env{DM_NAME} 0 fail_if_no_path"


-- 
To view, visit http://gerrit.ovirt.org/32277
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I6bfe68f4a8d233afe1bb787ce028ffb4c167469b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer at redhat.com>


More information about the vdsm-patches mailing list