Change in vdsm[master]: vdsm-infra: zombie-reaper refactor

smizrahi at redhat.com smizrahi at redhat.com
Sun Aug 10 16:03:06 UTC 2014


Saggi Mizrahi has uploaded a new change for review.

Change subject: vdsm-infra: zombie-reaper refactor
......................................................................

vdsm-infra: zombie-reaper refactor

In a general intent to better organize how infra libraries and utilities
are organized.

It uses zombie-reaper as a sample use case.
Also removes the deprecated documentation in /doc/ and adds
documentation for zombie-reaper.

Change-Id: I86a7d8d2f512395c2f50e21430b9e939f553e91b
Signed-off-by: Saggi Mizrahi <smizrahi at redhat.com>
---
M configure.ac
M doc/conf.py
M doc/index.rst
A doc/infra/index.rst
A doc/infra/zombiereaper.rst
D doc/misc/utils.rst
D doc/storage/HSM.rst
D doc/storage/overview.rst
D doc/vdsm/clientIF.rst
D doc/vdsm/vm.rst
M lib/Makefile.am
M lib/vdsm/Makefile.am
C lib/vdsm/infra/Makefile.am
A lib/vdsm/infra/__init__.py
R lib/vdsm/infra/zombiereaper/Makefile.am
A lib/vdsm/infra/zombiereaper/__init__.py
R lib/vdsm/infra/zombiereaper/tests.py
M lib/vdsm/utils.py
D lib/zombiereaper/__init__.py
M tests/testrunner.py
M vdsm.spec.in
M vdsm/storage/remoteFileHandler.py
M vdsm/vdsm
23 files changed, 92 insertions(+), 165 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/01/31301/1

diff --git a/configure.ac b/configure.ac
index b7f5108..e67a070 100644
--- a/configure.ac
+++ b/configure.ac
@@ -132,6 +132,8 @@
 AC_SUBST([vdsmtsdir], ['${sysconfdir}/pki/vdsm'])
 AC_SUBST([vdsmrepo], ['/rhev/data-center'])
 AC_SUBST([vdsmpylibdir], ['${pyexecdir}/vdsm'])
+AC_SUBST([vdsminfradir], ['${vdsmpylibdir}/infra'])
+AC_SUBST([zombiereaperdir], ['${vdsminfradir}/zombiereaper'])
 AC_SUBST([vdsmtooldir], ['${vdsmpylibdir}/tool'])
 AC_SUBST([vdsmtestsdir], ['${datarootdir}/vdsm/tests'])
 
@@ -320,7 +322,8 @@
 	lib/vdsm/netlink/Makefile
 	lib/vdsm/tool/Makefile
 	lib/yajsonrpc/Makefile
-	lib/zombiereaper/Makefile
+	lib/vdsm/infra/Makefile
+	lib/vdsm/infra/zombiereaper/Makefile
 	tests/Makefile
 	tests/functional/Makefile
 	vds_bootstrap/Makefile
diff --git a/doc/conf.py b/doc/conf.py
index 266f7e5..eef8611 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -17,6 +17,7 @@
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
+sys.path.append(os.path.abspath('../lib'))
 sys.path.append(os.path.abspath('..'))
 
 # -- General configuration ----------------------------------------------------
diff --git a/doc/index.rst b/doc/index.rst
index c1bbca8..c4156ec 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -9,13 +9,10 @@
 Contents:
 
 .. toctree::
-   :glob:
    :maxdepth: 2
 
-   vdsm/*
-   storage/*
-   misc/*
-   
+   infra/index.rst
+
 Indices and tables
 ==================
 
diff --git a/doc/infra/index.rst b/doc/infra/index.rst
new file mode 100644
index 0000000..fa010c9
--- /dev/null
+++ b/doc/infra/index.rst
@@ -0,0 +1,7 @@
+VDSM Infrastrature
+==================
+
+Contents:
+
+.. toctree::
+    zombiereaper
diff --git a/doc/infra/zombiereaper.rst b/doc/infra/zombiereaper.rst
new file mode 100644
index 0000000..e39f372
--- /dev/null
+++ b/doc/infra/zombiereaper.rst
@@ -0,0 +1,5 @@
+Zombie Reaper
+=============
+
+.. automodule:: vdsm.infra.zombiereaper
+        :members:
diff --git a/doc/misc/utils.rst b/doc/misc/utils.rst
deleted file mode 100644
index f1c856d..0000000
--- a/doc/misc/utils.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-Documentation for the infamous Utils module
-*******************************************
-
-.. automodule:: vdsm.utils
-        :members:
diff --git a/doc/storage/HSM.rst b/doc/storage/HSM.rst
deleted file mode 100644
index de402cd..0000000
--- a/doc/storage/HSM.rst
+++ /dev/null
@@ -1,6 +0,0 @@
-Documentation for the Host Storage Manager
-******************************************
-
-.. automodule:: vdsm.storage.hsm
-        :members:
-        :noindex:
diff --git a/doc/storage/overview.rst b/doc/storage/overview.rst
deleted file mode 100644
index 8077b4d..0000000
--- a/doc/storage/overview.rst
+++ /dev/null
@@ -1,82 +0,0 @@
-Documentation for the Storage API Overview
-******************************************
-.. autoclass:: vdsm.storage.hsm.HSM
-
-File Based
-==========
-Device
-------
-.. automethod:: vdsm.storage.hsm.HSM.getDeviceList
-
-Volume
-------
-.. automethod:: vdsm.storage.hsm.HSM.getVolumePath
-.. automethod:: vdsm.storage.hsm.HSM.getVolumeSize
-.. automethod:: vdsm.storage.hsm.HSM.prepareVolume
-.. automethod:: vdsm.storage.hsm.HSM.teardownVolume
-
-Structure
-=========
-Storage Domain
---------------
-.. automethod:: vdsm.storage.hsm.HSM.createStorageDomain
-.. automethod:: vdsm.storage.hsm.HSM.validateStorageDomain
-.. automethod:: vdsm.storage.hsm.HSM.formatStorageDomain
-.. automethod:: vdsm.storage.hsm.HSM.forcedDetachStorageDomain
-.. automethod:: vdsm.storage.hsm.HSM.setStorageDomainDescription
-.. automethod:: vdsm.storage.hsm.HSM.getStorageDomainInfo
-.. automethod:: vdsm.storage.hsm.HSM.getStorageDomainStats
-.. automethod:: vdsm.storage.hsm.HSM.getStorageDomainsList
-
-Storage Pool
-------------
-.. automethod:: vdsm.storage.hsm.HSM.createStoragePool
-.. automethod:: vdsm.storage.hsm.HSM.destroyStoragePool
-.. automethod:: vdsm.storage.hsm.HSM.disconnectStoragePool
-.. automethod:: vdsm.storage.hsm.HSM.getStoragePoolInfo
-.. automethod:: vdsm.storage.hsm.HSM.getConnectedStoragePoolsList
-.. automethod:: vdsm.storage.hsm.HSM.refreshStoragePool
-
-Volume Group
-------------
-.. automethod:: vdsm.storage.hsm.HSM.createVG
-.. automethod:: vdsm.storage.hsm.HSM.removeVG
-.. automethod:: vdsm.storage.hsm.HSM.getVGInfo
-.. automethod:: vdsm.storage.hsm.HSM.getVGList
-
-
-Volumes / Images
-----------------
-.. automethod:: vdsm.storage.hsm.HSM.getFloppyList
-.. automethod:: vdsm.storage.hsm.HSM.getImageDomainsList
-.. automethod:: vdsm.storage.hsm.HSM.getImagesList
-.. automethod:: vdsm.storage.hsm.HSM.getIsoList
-.. automethod:: vdsm.storage.hsm.HSM.getVolumeInfo
-.. automethod:: vdsm.storage.hsm.HSM.getVolumesList
-.. automethod:: vdsm.storage.hsm.HSM.refreshVolume
-
-Cluster
-=======
-.. automethod:: vdsm.storage.hsm.HSM.prepareForShutdown
-
-Task Managment
---------------
-.. automethod:: vdsm.storage.hsm.HSM.getAllTasksInfo
-.. automethod:: vdsm.storage.hsm.HSM.getAllTasksStatuses
-.. automethod:: vdsm.storage.hsm.HSM.getTaskInfo
-.. automethod:: vdsm.storage.hsm.HSM.getTaskStatus
-.. automethod:: vdsm.storage.hsm.HSM.stopTask
-
-Storage Pool Managment
-----------------------
-.. automethod:: vdsm.storage.hsm.HSM.connectStoragePool
-.. automethod:: vdsm.storage.hsm.HSM.spmStart
-.. automethod:: vdsm.storage.hsm.HSM.reconstructMaster
-
-Storage Server Managment
-------------------------
-.. automethod:: vdsm.storage.hsm.HSM.connectStorageServer
-.. automethod:: vdsm.storage.hsm.HSM.disconnectStorageServer
-.. automethod:: vdsm.storage.hsm.HSM.discoverSendTargets
-.. automethod:: vdsm.storage.hsm.HSM.validateStorageServerConnection
-.. automethod:: vdsm.storage.hsm.HSM.repoStats
diff --git a/doc/vdsm/clientIF.rst b/doc/vdsm/clientIF.rst
deleted file mode 100644
index d8ebf22..0000000
--- a/doc/vdsm/clientIF.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-Documentation for the clientIF module
-******************************************
-
-.. automodule:: vdsm.clientIF
-        :members:
diff --git a/doc/vdsm/vm.rst b/doc/vdsm/vm.rst
deleted file mode 100644
index 39c3984..0000000
--- a/doc/vdsm/vm.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-Documentation for the vm module
-******************************************
-
-.. automodule:: vdsm.vm
-        :members:
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 493edc8..ff46328 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -19,4 +19,4 @@
 #
 include $(top_srcdir)/build-aux/Makefile.subs
 
-SUBDIRS = vdsm yajsonrpc zombiereaper
+SUBDIRS = vdsm yajsonrpc
diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am
index 8e90e6e..241b0d6 100644
--- a/lib/vdsm/Makefile.am
+++ b/lib/vdsm/Makefile.am
@@ -19,7 +19,7 @@
 #
 include $(top_srcdir)/build-aux/Makefile.subs
 
-SUBDIRS=netlink tool
+SUBDIRS=netlink tool infra
 
 dist_vdsmpylib_PYTHON = \
 	__init__.py \
diff --git a/lib/zombiereaper/Makefile.am b/lib/vdsm/infra/Makefile.am
similarity index 80%
copy from lib/zombiereaper/Makefile.am
copy to lib/vdsm/infra/Makefile.am
index ec01a0d..475bffb 100644
--- a/lib/zombiereaper/Makefile.am
+++ b/lib/vdsm/infra/Makefile.am
@@ -1,5 +1,5 @@
 #
-# Copyright 2008-2011 Red Hat, Inc.
+# Copyright 2013 Hat, Inc.
 #
 # This program is free software; you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -17,16 +17,10 @@
 #
 # Refer to the README and COPYING files for full details of the license
 #
+include $(top_srcdir)/build-aux/Makefile.subs
 
-zombiereaperdir = $(pythondir)/zombiereaper
+SUBDIRS = zombiereaper
 
-dist_zombiereaper_PYTHON = \
+dist_vdsminfra_PYTHON = \
 	__init__.py \
 	$(NULL)
-
-dist_noinst_PYTHON = \
-	tests.py \
-	$(NULL)
-
-check-local:
-	nosetests tests.py
diff --git a/lib/vdsm/infra/__init__.py b/lib/vdsm/infra/__init__.py
new file mode 100644
index 0000000..c499485
--- /dev/null
+++ b/lib/vdsm/infra/__init__.py
@@ -0,0 +1 @@
+__all__ = ["zombiereaper"]
diff --git a/lib/zombiereaper/Makefile.am b/lib/vdsm/infra/zombiereaper/Makefile.am
similarity index 95%
rename from lib/zombiereaper/Makefile.am
rename to lib/vdsm/infra/zombiereaper/Makefile.am
index ec01a0d..6707514 100644
--- a/lib/zombiereaper/Makefile.am
+++ b/lib/vdsm/infra/zombiereaper/Makefile.am
@@ -18,8 +18,6 @@
 # Refer to the README and COPYING files for full details of the license
 #
 
-zombiereaperdir = $(pythondir)/zombiereaper
-
 dist_zombiereaper_PYTHON = \
 	__init__.py \
 	$(NULL)
diff --git a/lib/vdsm/infra/zombiereaper/__init__.py b/lib/vdsm/infra/zombiereaper/__init__.py
new file mode 100644
index 0000000..5c113ec
--- /dev/null
+++ b/lib/vdsm/infra/zombiereaper/__init__.py
@@ -0,0 +1,52 @@
+"""
+ZombieReaper is a module to handle the annoying problem of cleaning up child
+processes. Specifically, for cases where the result of the child process is not
+needed.
+
+This replaces the common idiom of running a thread which only does
+:py:meth:`subprocess.Popen.wait()`, saving precious threads.
+"""
+
+import os
+import signal
+
+_trackedPids = set()
+
+
+def autoReapPID(pid):
+    """
+    Register a PID to be auto-cleaned.
+    """
+    _trackedPids.add(pid)
+    # SIGCHLD happend before we added the pid to the set
+    _tryReap(pid)
+
+
+def _tryReap(pid):
+        try:
+            pid, rv = os.waitpid(pid, os.WNOHANG)
+            if pid != 0:
+                _trackedPids.discard(pid)
+        except OSError:
+            _trackedPids.discard(pid)
+
+
+def _zombieReaper(signum, frame):
+    for pid in _trackedPids.copy():
+        _tryReap(pid)
+
+
+def registerSignalHandler():
+    """
+    Set up the signal handler so that PIDs are reaped. Should be called once
+    at the start of the program.
+    """
+    signal.signal(signal.SIGCHLD, _zombieReaper)
+
+
+def unregisterSignalHandler():
+    """
+    Stop cleaning PIDs. Should only be used for testing or other specialized
+    use cases.
+    """
+    signal.signal(signal.SIGCHLD, signal.SIG_DFL)
diff --git a/lib/zombiereaper/tests.py b/lib/vdsm/infra/zombiereaper/tests.py
similarity index 97%
rename from lib/zombiereaper/tests.py
rename to lib/vdsm/infra/zombiereaper/tests.py
index 425ac74..a4d9e88 100644
--- a/lib/zombiereaper/tests.py
+++ b/lib/vdsm/infra/zombiereaper/tests.py
@@ -20,6 +20,8 @@
 from time import sleep
 import os
 
+import sys
+sys.path.append("..")
 import zombiereaper
 from cpopen import CPopen
 
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index 84b7883..1eae6c5 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -48,7 +48,7 @@
 import stat
 import threading
 import time
-import zombiereaper
+import vdsm.infra.zombiereaper as zombiereaper
 
 from cpopen import CPopen
 from . import constants
diff --git a/lib/zombiereaper/__init__.py b/lib/zombiereaper/__init__.py
deleted file mode 100644
index 74b311d..0000000
--- a/lib/zombiereaper/__init__.py
+++ /dev/null
@@ -1,32 +0,0 @@
-import os
-import signal
-
-_trackedPids = set()
-
-
-def autoReapPID(pid):
-    _trackedPids.add(pid)
-    # SIGCHLD happend before we added the pid to the set
-    _tryReap(pid)
-
-
-def _tryReap(pid):
-        try:
-            pid, rv = os.waitpid(pid, os.WNOHANG)
-            if pid != 0:
-                _trackedPids.discard(pid)
-        except OSError:
-            _trackedPids.discard(pid)
-
-
-def _zombieReaper(signum, frame):
-    for pid in _trackedPids.copy():
-        _tryReap(pid)
-
-
-def registerSignalHandler():
-    signal.signal(signal.SIGCHLD, _zombieReaper)
-
-
-def unregisterSignalHandler():
-    signal.signal(signal.SIGCHLD, signal.SIG_DFL)
diff --git a/tests/testrunner.py b/tests/testrunner.py
index e606434..ec63f0e 100644
--- a/tests/testrunner.py
+++ b/tests/testrunner.py
@@ -26,7 +26,7 @@
     import pthreading
     pthreading.monkey_patch()
 
-import zombiereaper
+import vdsm.infra.zombiereaper as zombiereaper
 zombiereaper.registerSignalHandler()
 
 from vdsm import utils
diff --git a/vdsm.spec.in b/vdsm.spec.in
index d3c8a9f..9d924f2 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -125,7 +125,7 @@
 Requires: python-ioprocess >= 0.5-1
 Requires: python-pthreading >= 0.1.3-3
 Requires: python-ethtool >= 0.6-3
-Requires: %{name}-python-zombiereaper = %{version}-%{release}
+Requires: %{name}-infra = %{version}-%{release}
 Requires: rpm-python
 Requires: nfs-utils
 Requires: m2crypto
@@ -299,13 +299,14 @@
 %description yajsonrpc
 A JSON RPC server and client implementation.
 
-%package python-zombiereaper
+%package infra
 Summary:        Collects zombie processes automatically
 BuildArch:      noarch
 
 Requires:       python >= 2.6
+Obsoletes:	python-zombiereaper
 
-%description python-zombiereaper
+%description infra
 Collects zombie processes automatically so you don't have to
 
 %package bootstrap
@@ -331,7 +332,7 @@
 
 %package python
 Summary:        VDSM python libraries
-Requires:       %{name}-python-zombiereaper = %{version}-%{release}
+Requires:       %{name}-infra = %{version}-%{release}
 Requires:       python-cpopen >= 1.2.3-5
 Requires:       m2crypto
 Requires:       python-ioprocess >= 0.5-1
@@ -1439,8 +1440,9 @@
 %{python_sitelib}/yajsonrpc/stomp.py*
 %{python_sitelib}/yajsonrpc/stompReactor.py*
 
-%files python-zombiereaper
-%{python_sitelib}/zombiereaper/__init__.py*
+%files infra
+%{python_sitearch}/%{vdsm_name}/infra/zombiereaper/__init__.py*
+%{python_sitearch}/%{vdsm_name}/infra/__init__.py*
 
 %files bootstrap
 %defattr(-, root, root, -)
diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py
index ae3e54a..07789b1 100644
--- a/vdsm/storage/remoteFileHandler.py
+++ b/vdsm/storage/remoteFileHandler.py
@@ -45,7 +45,7 @@
 
 import misc
 import fileUtils
-import zombiereaper
+import vdsm.infra.zombiereaper as zombiereaper
 from vdsm.compat import pickle
 from vdsm import utils
 
diff --git a/vdsm/vdsm b/vdsm/vdsm
index cbc16ff..9587fbb 100755
--- a/vdsm/vdsm
+++ b/vdsm/vdsm
@@ -38,7 +38,7 @@
 from storage.dispatcher import Dispatcher
 from storage.hsm import HSM
 
-import zombiereaper
+import vdsm.infra.zombiereaper as zombiereaper
 import dsaversion
 
 loggerConfFile = constants.P_VDSM_CONF + 'logger.conf'


-- 
To view, visit http://gerrit.ovirt.org/31301
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I86a7d8d2f512395c2f50e21430b9e939f553e91b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Saggi Mizrahi <smizrahi at redhat.com>


More information about the vdsm-patches mailing list