Change in vdsm[master]: vdsm-infra: zombie-reaper refactor
smizrahi at redhat.com
smizrahi at redhat.com
Sun Aug 10 16:03:06 UTC 2014
Saggi Mizrahi has uploaded a new change for review.
Change subject: vdsm-infra: zombie-reaper refactor
......................................................................
vdsm-infra: zombie-reaper refactor
In a general intent to better organize how infra libraries and utilities
are organized.
It uses zombie-reaper as a sample use case.
Also removes the deprecated documentation in /doc/ and adds
documentation for zombie-reaper.
Change-Id: I86a7d8d2f512395c2f50e21430b9e939f553e91b
Signed-off-by: Saggi Mizrahi <smizrahi at redhat.com>
---
M configure.ac
M doc/conf.py
M doc/index.rst
A doc/infra/index.rst
A doc/infra/zombiereaper.rst
D doc/misc/utils.rst
D doc/storage/HSM.rst
D doc/storage/overview.rst
D doc/vdsm/clientIF.rst
D doc/vdsm/vm.rst
M lib/Makefile.am
M lib/vdsm/Makefile.am
C lib/vdsm/infra/Makefile.am
A lib/vdsm/infra/__init__.py
R lib/vdsm/infra/zombiereaper/Makefile.am
A lib/vdsm/infra/zombiereaper/__init__.py
R lib/vdsm/infra/zombiereaper/tests.py
M lib/vdsm/utils.py
D lib/zombiereaper/__init__.py
M tests/testrunner.py
M vdsm.spec.in
M vdsm/storage/remoteFileHandler.py
M vdsm/vdsm
23 files changed, 92 insertions(+), 165 deletions(-)
git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/01/31301/1
diff --git a/configure.ac b/configure.ac
index b7f5108..e67a070 100644
--- a/configure.ac
+++ b/configure.ac
@@ -132,6 +132,8 @@
AC_SUBST([vdsmtsdir], ['${sysconfdir}/pki/vdsm'])
AC_SUBST([vdsmrepo], ['/rhev/data-center'])
AC_SUBST([vdsmpylibdir], ['${pyexecdir}/vdsm'])
+AC_SUBST([vdsminfradir], ['${vdsmpylibdir}/infra'])
+AC_SUBST([zombiereaperdir], ['${vdsminfradir}/zombiereaper'])
AC_SUBST([vdsmtooldir], ['${vdsmpylibdir}/tool'])
AC_SUBST([vdsmtestsdir], ['${datarootdir}/vdsm/tests'])
@@ -320,7 +322,8 @@
lib/vdsm/netlink/Makefile
lib/vdsm/tool/Makefile
lib/yajsonrpc/Makefile
- lib/zombiereaper/Makefile
+ lib/vdsm/infra/Makefile
+ lib/vdsm/infra/zombiereaper/Makefile
tests/Makefile
tests/functional/Makefile
vds_bootstrap/Makefile
diff --git a/doc/conf.py b/doc/conf.py
index 266f7e5..eef8611 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -17,6 +17,7 @@
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
+sys.path.append(os.path.abspath('../lib'))
sys.path.append(os.path.abspath('..'))
# -- General configuration ----------------------------------------------------
diff --git a/doc/index.rst b/doc/index.rst
index c1bbca8..c4156ec 100644
--- a/doc/index.rst
+++ b/doc/index.rst
@@ -9,13 +9,10 @@
Contents:
.. toctree::
- :glob:
:maxdepth: 2
- vdsm/*
- storage/*
- misc/*
-
+ infra/index.rst
+
Indices and tables
==================
diff --git a/doc/infra/index.rst b/doc/infra/index.rst
new file mode 100644
index 0000000..fa010c9
--- /dev/null
+++ b/doc/infra/index.rst
@@ -0,0 +1,7 @@
+VDSM Infrastrature
+==================
+
+Contents:
+
+.. toctree::
+ zombiereaper
diff --git a/doc/infra/zombiereaper.rst b/doc/infra/zombiereaper.rst
new file mode 100644
index 0000000..e39f372
--- /dev/null
+++ b/doc/infra/zombiereaper.rst
@@ -0,0 +1,5 @@
+Zombie Reaper
+=============
+
+.. automodule:: vdsm.infra.zombiereaper
+ :members:
diff --git a/doc/misc/utils.rst b/doc/misc/utils.rst
deleted file mode 100644
index f1c856d..0000000
--- a/doc/misc/utils.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-Documentation for the infamous Utils module
-*******************************************
-
-.. automodule:: vdsm.utils
- :members:
diff --git a/doc/storage/HSM.rst b/doc/storage/HSM.rst
deleted file mode 100644
index de402cd..0000000
--- a/doc/storage/HSM.rst
+++ /dev/null
@@ -1,6 +0,0 @@
-Documentation for the Host Storage Manager
-******************************************
-
-.. automodule:: vdsm.storage.hsm
- :members:
- :noindex:
diff --git a/doc/storage/overview.rst b/doc/storage/overview.rst
deleted file mode 100644
index 8077b4d..0000000
--- a/doc/storage/overview.rst
+++ /dev/null
@@ -1,82 +0,0 @@
-Documentation for the Storage API Overview
-******************************************
-.. autoclass:: vdsm.storage.hsm.HSM
-
-File Based
-==========
-Device
-------
-.. automethod:: vdsm.storage.hsm.HSM.getDeviceList
-
-Volume
-------
-.. automethod:: vdsm.storage.hsm.HSM.getVolumePath
-.. automethod:: vdsm.storage.hsm.HSM.getVolumeSize
-.. automethod:: vdsm.storage.hsm.HSM.prepareVolume
-.. automethod:: vdsm.storage.hsm.HSM.teardownVolume
-
-Structure
-=========
-Storage Domain
---------------
-.. automethod:: vdsm.storage.hsm.HSM.createStorageDomain
-.. automethod:: vdsm.storage.hsm.HSM.validateStorageDomain
-.. automethod:: vdsm.storage.hsm.HSM.formatStorageDomain
-.. automethod:: vdsm.storage.hsm.HSM.forcedDetachStorageDomain
-.. automethod:: vdsm.storage.hsm.HSM.setStorageDomainDescription
-.. automethod:: vdsm.storage.hsm.HSM.getStorageDomainInfo
-.. automethod:: vdsm.storage.hsm.HSM.getStorageDomainStats
-.. automethod:: vdsm.storage.hsm.HSM.getStorageDomainsList
-
-Storage Pool
-------------
-.. automethod:: vdsm.storage.hsm.HSM.createStoragePool
-.. automethod:: vdsm.storage.hsm.HSM.destroyStoragePool
-.. automethod:: vdsm.storage.hsm.HSM.disconnectStoragePool
-.. automethod:: vdsm.storage.hsm.HSM.getStoragePoolInfo
-.. automethod:: vdsm.storage.hsm.HSM.getConnectedStoragePoolsList
-.. automethod:: vdsm.storage.hsm.HSM.refreshStoragePool
-
-Volume Group
-------------
-.. automethod:: vdsm.storage.hsm.HSM.createVG
-.. automethod:: vdsm.storage.hsm.HSM.removeVG
-.. automethod:: vdsm.storage.hsm.HSM.getVGInfo
-.. automethod:: vdsm.storage.hsm.HSM.getVGList
-
-
-Volumes / Images
-----------------
-.. automethod:: vdsm.storage.hsm.HSM.getFloppyList
-.. automethod:: vdsm.storage.hsm.HSM.getImageDomainsList
-.. automethod:: vdsm.storage.hsm.HSM.getImagesList
-.. automethod:: vdsm.storage.hsm.HSM.getIsoList
-.. automethod:: vdsm.storage.hsm.HSM.getVolumeInfo
-.. automethod:: vdsm.storage.hsm.HSM.getVolumesList
-.. automethod:: vdsm.storage.hsm.HSM.refreshVolume
-
-Cluster
-=======
-.. automethod:: vdsm.storage.hsm.HSM.prepareForShutdown
-
-Task Managment
---------------
-.. automethod:: vdsm.storage.hsm.HSM.getAllTasksInfo
-.. automethod:: vdsm.storage.hsm.HSM.getAllTasksStatuses
-.. automethod:: vdsm.storage.hsm.HSM.getTaskInfo
-.. automethod:: vdsm.storage.hsm.HSM.getTaskStatus
-.. automethod:: vdsm.storage.hsm.HSM.stopTask
-
-Storage Pool Managment
-----------------------
-.. automethod:: vdsm.storage.hsm.HSM.connectStoragePool
-.. automethod:: vdsm.storage.hsm.HSM.spmStart
-.. automethod:: vdsm.storage.hsm.HSM.reconstructMaster
-
-Storage Server Managment
-------------------------
-.. automethod:: vdsm.storage.hsm.HSM.connectStorageServer
-.. automethod:: vdsm.storage.hsm.HSM.disconnectStorageServer
-.. automethod:: vdsm.storage.hsm.HSM.discoverSendTargets
-.. automethod:: vdsm.storage.hsm.HSM.validateStorageServerConnection
-.. automethod:: vdsm.storage.hsm.HSM.repoStats
diff --git a/doc/vdsm/clientIF.rst b/doc/vdsm/clientIF.rst
deleted file mode 100644
index d8ebf22..0000000
--- a/doc/vdsm/clientIF.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-Documentation for the clientIF module
-******************************************
-
-.. automodule:: vdsm.clientIF
- :members:
diff --git a/doc/vdsm/vm.rst b/doc/vdsm/vm.rst
deleted file mode 100644
index 39c3984..0000000
--- a/doc/vdsm/vm.rst
+++ /dev/null
@@ -1,5 +0,0 @@
-Documentation for the vm module
-******************************************
-
-.. automodule:: vdsm.vm
- :members:
diff --git a/lib/Makefile.am b/lib/Makefile.am
index 493edc8..ff46328 100644
--- a/lib/Makefile.am
+++ b/lib/Makefile.am
@@ -19,4 +19,4 @@
#
include $(top_srcdir)/build-aux/Makefile.subs
-SUBDIRS = vdsm yajsonrpc zombiereaper
+SUBDIRS = vdsm yajsonrpc
diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am
index 8e90e6e..241b0d6 100644
--- a/lib/vdsm/Makefile.am
+++ b/lib/vdsm/Makefile.am
@@ -19,7 +19,7 @@
#
include $(top_srcdir)/build-aux/Makefile.subs
-SUBDIRS=netlink tool
+SUBDIRS=netlink tool infra
dist_vdsmpylib_PYTHON = \
__init__.py \
diff --git a/lib/zombiereaper/Makefile.am b/lib/vdsm/infra/Makefile.am
similarity index 80%
copy from lib/zombiereaper/Makefile.am
copy to lib/vdsm/infra/Makefile.am
index ec01a0d..475bffb 100644
--- a/lib/zombiereaper/Makefile.am
+++ b/lib/vdsm/infra/Makefile.am
@@ -1,5 +1,5 @@
#
-# Copyright 2008-2011 Red Hat, Inc.
+# Copyright 2013 Hat, Inc.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
@@ -17,16 +17,10 @@
#
# Refer to the README and COPYING files for full details of the license
#
+include $(top_srcdir)/build-aux/Makefile.subs
-zombiereaperdir = $(pythondir)/zombiereaper
+SUBDIRS = zombiereaper
-dist_zombiereaper_PYTHON = \
+dist_vdsminfra_PYTHON = \
__init__.py \
$(NULL)
-
-dist_noinst_PYTHON = \
- tests.py \
- $(NULL)
-
-check-local:
- nosetests tests.py
diff --git a/lib/vdsm/infra/__init__.py b/lib/vdsm/infra/__init__.py
new file mode 100644
index 0000000..c499485
--- /dev/null
+++ b/lib/vdsm/infra/__init__.py
@@ -0,0 +1 @@
+__all__ = ["zombiereaper"]
diff --git a/lib/zombiereaper/Makefile.am b/lib/vdsm/infra/zombiereaper/Makefile.am
similarity index 95%
rename from lib/zombiereaper/Makefile.am
rename to lib/vdsm/infra/zombiereaper/Makefile.am
index ec01a0d..6707514 100644
--- a/lib/zombiereaper/Makefile.am
+++ b/lib/vdsm/infra/zombiereaper/Makefile.am
@@ -18,8 +18,6 @@
# Refer to the README and COPYING files for full details of the license
#
-zombiereaperdir = $(pythondir)/zombiereaper
-
dist_zombiereaper_PYTHON = \
__init__.py \
$(NULL)
diff --git a/lib/vdsm/infra/zombiereaper/__init__.py b/lib/vdsm/infra/zombiereaper/__init__.py
new file mode 100644
index 0000000..5c113ec
--- /dev/null
+++ b/lib/vdsm/infra/zombiereaper/__init__.py
@@ -0,0 +1,52 @@
+"""
+ZombieReaper is a module to handle the annoying problem of cleaning up child
+processes. Specifically, for cases where the result of the child process is not
+needed.
+
+This replaces the common idiom of running a thread which only does
+:py:meth:`subprocess.Popen.wait()`, saving precious threads.
+"""
+
+import os
+import signal
+
+_trackedPids = set()
+
+
+def autoReapPID(pid):
+ """
+ Register a PID to be auto-cleaned.
+ """
+ _trackedPids.add(pid)
+ # SIGCHLD happend before we added the pid to the set
+ _tryReap(pid)
+
+
+def _tryReap(pid):
+ try:
+ pid, rv = os.waitpid(pid, os.WNOHANG)
+ if pid != 0:
+ _trackedPids.discard(pid)
+ except OSError:
+ _trackedPids.discard(pid)
+
+
+def _zombieReaper(signum, frame):
+ for pid in _trackedPids.copy():
+ _tryReap(pid)
+
+
+def registerSignalHandler():
+ """
+ Set up the signal handler so that PIDs are reaped. Should be called once
+ at the start of the program.
+ """
+ signal.signal(signal.SIGCHLD, _zombieReaper)
+
+
+def unregisterSignalHandler():
+ """
+ Stop cleaning PIDs. Should only be used for testing or other specialized
+ use cases.
+ """
+ signal.signal(signal.SIGCHLD, signal.SIG_DFL)
diff --git a/lib/zombiereaper/tests.py b/lib/vdsm/infra/zombiereaper/tests.py
similarity index 97%
rename from lib/zombiereaper/tests.py
rename to lib/vdsm/infra/zombiereaper/tests.py
index 425ac74..a4d9e88 100644
--- a/lib/zombiereaper/tests.py
+++ b/lib/vdsm/infra/zombiereaper/tests.py
@@ -20,6 +20,8 @@
from time import sleep
import os
+import sys
+sys.path.append("..")
import zombiereaper
from cpopen import CPopen
diff --git a/lib/vdsm/utils.py b/lib/vdsm/utils.py
index 84b7883..1eae6c5 100644
--- a/lib/vdsm/utils.py
+++ b/lib/vdsm/utils.py
@@ -48,7 +48,7 @@
import stat
import threading
import time
-import zombiereaper
+import vdsm.infra.zombiereaper as zombiereaper
from cpopen import CPopen
from . import constants
diff --git a/lib/zombiereaper/__init__.py b/lib/zombiereaper/__init__.py
deleted file mode 100644
index 74b311d..0000000
--- a/lib/zombiereaper/__init__.py
+++ /dev/null
@@ -1,32 +0,0 @@
-import os
-import signal
-
-_trackedPids = set()
-
-
-def autoReapPID(pid):
- _trackedPids.add(pid)
- # SIGCHLD happend before we added the pid to the set
- _tryReap(pid)
-
-
-def _tryReap(pid):
- try:
- pid, rv = os.waitpid(pid, os.WNOHANG)
- if pid != 0:
- _trackedPids.discard(pid)
- except OSError:
- _trackedPids.discard(pid)
-
-
-def _zombieReaper(signum, frame):
- for pid in _trackedPids.copy():
- _tryReap(pid)
-
-
-def registerSignalHandler():
- signal.signal(signal.SIGCHLD, _zombieReaper)
-
-
-def unregisterSignalHandler():
- signal.signal(signal.SIGCHLD, signal.SIG_DFL)
diff --git a/tests/testrunner.py b/tests/testrunner.py
index e606434..ec63f0e 100644
--- a/tests/testrunner.py
+++ b/tests/testrunner.py
@@ -26,7 +26,7 @@
import pthreading
pthreading.monkey_patch()
-import zombiereaper
+import vdsm.infra.zombiereaper as zombiereaper
zombiereaper.registerSignalHandler()
from vdsm import utils
diff --git a/vdsm.spec.in b/vdsm.spec.in
index d3c8a9f..9d924f2 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -125,7 +125,7 @@
Requires: python-ioprocess >= 0.5-1
Requires: python-pthreading >= 0.1.3-3
Requires: python-ethtool >= 0.6-3
-Requires: %{name}-python-zombiereaper = %{version}-%{release}
+Requires: %{name}-infra = %{version}-%{release}
Requires: rpm-python
Requires: nfs-utils
Requires: m2crypto
@@ -299,13 +299,14 @@
%description yajsonrpc
A JSON RPC server and client implementation.
-%package python-zombiereaper
+%package infra
Summary: Collects zombie processes automatically
BuildArch: noarch
Requires: python >= 2.6
+Obsoletes: python-zombiereaper
-%description python-zombiereaper
+%description infra
Collects zombie processes automatically so you don't have to
%package bootstrap
@@ -331,7 +332,7 @@
%package python
Summary: VDSM python libraries
-Requires: %{name}-python-zombiereaper = %{version}-%{release}
+Requires: %{name}-infra = %{version}-%{release}
Requires: python-cpopen >= 1.2.3-5
Requires: m2crypto
Requires: python-ioprocess >= 0.5-1
@@ -1439,8 +1440,9 @@
%{python_sitelib}/yajsonrpc/stomp.py*
%{python_sitelib}/yajsonrpc/stompReactor.py*
-%files python-zombiereaper
-%{python_sitelib}/zombiereaper/__init__.py*
+%files infra
+%{python_sitearch}/%{vdsm_name}/infra/zombiereaper/__init__.py*
+%{python_sitearch}/%{vdsm_name}/infra/__init__.py*
%files bootstrap
%defattr(-, root, root, -)
diff --git a/vdsm/storage/remoteFileHandler.py b/vdsm/storage/remoteFileHandler.py
index ae3e54a..07789b1 100644
--- a/vdsm/storage/remoteFileHandler.py
+++ b/vdsm/storage/remoteFileHandler.py
@@ -45,7 +45,7 @@
import misc
import fileUtils
-import zombiereaper
+import vdsm.infra.zombiereaper as zombiereaper
from vdsm.compat import pickle
from vdsm import utils
diff --git a/vdsm/vdsm b/vdsm/vdsm
index cbc16ff..9587fbb 100755
--- a/vdsm/vdsm
+++ b/vdsm/vdsm
@@ -38,7 +38,7 @@
from storage.dispatcher import Dispatcher
from storage.hsm import HSM
-import zombiereaper
+import vdsm.infra.zombiereaper as zombiereaper
import dsaversion
loggerConfFile = constants.P_VDSM_CONF + 'logger.conf'
--
To view, visit http://gerrit.ovirt.org/31301
To unsubscribe, visit http://gerrit.ovirt.org/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: I86a7d8d2f512395c2f50e21430b9e939f553e91b
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Saggi Mizrahi <smizrahi at redhat.com>
More information about the vdsm-patches
mailing list