Change in vdsm[master]: profiling: Add support for memory profiling

fromani at redhat.com fromani at redhat.com
Tue Aug 26 20:30:25 UTC 2014


Francesco Romani has uploaded a new change for review.

Change subject: profiling: Add support for memory profiling
......................................................................

profiling: Add support for memory profiling

There are many python memory profilers, but most of them are
tailored for interactive usage, so they are poorly suited
for a daemon, like VDSM.

This patch adds support for dowser:
<http://www.aminus.net/wiki/Dowser>
<https://pypi.python.org/pypi/dowser/0.2>

which has the following benefits which makes it ina better position
to be used with VDSM:
+ self contained, includes WEB UI
+ allows live monitoring
+ tailored for cherrypy usage, so friendly towards server applications.
+ easily portable (pure package)

It must be noted that dowser has some drawbacks as well
- significat dependencies, both in number and in size (with respect
  to VDSM standards): cherrypy, PIL
- based on gc module

As per the cpu profile already added, this code is meant to be
used only as debug aid or in development environments.
The feature is controlled by a config tunable and disabled by default;
the feature disable itself if any dependency is missed.

Change-Id: Ib56b65513e0118b68cd43791bf655c928d6a26e2
Signed-off-by: Francesco Romani <fromani at redhat.com>
---
M debian/vdsm-python.install
M lib/vdsm/Makefile.am
A lib/vdsm/profile_mem.py
M vdsm.spec.in
M vdsm/vdsm
5 files changed, 89 insertions(+), 0 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/19/32019/1

diff --git a/debian/vdsm-python.install b/debian/vdsm-python.install
index 8d11bfc..f429a91 100644
--- a/debian/vdsm-python.install
+++ b/debian/vdsm-python.install
@@ -15,6 +15,7 @@
 ./usr/lib/python2.7/dist-packages/vdsm/netlink/link.py
 ./usr/lib/python2.7/dist-packages/vdsm/netlink/route.py
 ./usr/lib/python2.7/dist-packages/vdsm/profile.py
+./usr/lib/python2.7/dist-packages/vdsm/profile_mem.py
 ./usr/lib/python2.7/dist-packages/vdsm/qemuimg.py
 ./usr/lib/python2.7/dist-packages/vdsm/sslutils.py
 ./usr/lib/python2.7/dist-packages/vdsm/tool/__init__.py
diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am
index 4bebf28..04c8018 100644
--- a/lib/vdsm/Makefile.am
+++ b/lib/vdsm/Makefile.am
@@ -31,6 +31,7 @@
 	netconfpersistence.py \
 	netinfo.py \
 	profile.py \
+	profile_mem.py \
 	qemuimg.py \
 	sslutils.py \
 	utils.py \
diff --git a/lib/vdsm/profile_mem.py b/lib/vdsm/profile_mem.py
new file mode 100755
index 0000000..a60a833
--- /dev/null
+++ b/lib/vdsm/profile_mem.py
@@ -0,0 +1,83 @@
+#
+# Copyright 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+#
+# Refer to the README and COPYING files for full details of the license
+#
+
+"""
+This module provides memory profiling.
+"""
+
+import threading
+
+from vdsm.config import config
+
+try:
+    # dowser dependencies
+    import cherrypy
+    from PIL import Image, ImageDraw
+    # dowser profiler itself
+    import dowser
+except IOError:
+    _SUPPORTED = False
+else:
+    _SUPPORTED = True
+
+Image  # makes pyflakes happy
+ImageDraw  # same
+
+
+_thread = None
+
+
+def memory_usage_server(port=None):
+    cherrypy.tree.mount(dowser.Root())
+
+    port = (config.getint('vars', 'profile_memory_port')
+            if port is None else port)
+
+    cherrypy.config.update({
+        'environment': 'embedded',
+        'server.socket_port': port})
+
+    cherrypy.engine.start()
+
+
+def start():
+    """ Starts application memory profiling """
+    if _SUPPORTED and is_enabled():
+        _start_profiling()
+
+
+def stop():
+    """ Stops application memory profiling """
+    pass  # not yet needed
+
+
+def is_enabled():
+    return config.getboolean('vars', 'profile_memory_enable')
+
+
+def is_running():
+    return _thread is not None and _thread.is_alive()
+
+
+def _start_profiling():
+    global _thread
+    _thread = threading.Thread(name='memprofile', target=memory_usage_server)
+    _thread.daemon = True
+    _thread.start()
diff --git a/vdsm.spec.in b/vdsm.spec.in
index 5f9488c..233eee6 100644
--- a/vdsm.spec.in
+++ b/vdsm.spec.in
@@ -1184,6 +1184,7 @@
 %{python_sitearch}/%{vdsm_name}/netlink/link.py*
 %{python_sitearch}/%{vdsm_name}/netlink/route.py*
 %{python_sitearch}/%{vdsm_name}/profile.py*
+%{python_sitearch}/%{vdsm_name}/profile_mem.py*
 %{python_sitearch}/%{vdsm_name}/qemuimg.py*
 %{python_sitearch}/%{vdsm_name}/netconfpersistence.py*
 %{python_sitearch}/%{vdsm_name}/sslutils.py*
diff --git a/vdsm/vdsm b/vdsm/vdsm
index e575295..619e57e 100755
--- a/vdsm/vdsm
+++ b/vdsm/vdsm
@@ -34,6 +34,7 @@
 from vdsm.config import config
 from vdsm import libvirtconnection
 from vdsm import profile
+from vdsm import profile_mem
 
 from storage.dispatcher import Dispatcher
 from storage.hsm import HSM
@@ -69,6 +70,7 @@
     zombiereaper.registerSignalHandler()
 
     profile.start()
+    profile_mem.start()
 
     libvirtconnection.start_event_loop()
 
@@ -85,6 +87,7 @@
         while running[0]:
             signal.pause()
 
+        profile_mem.stop()
         profile.stop()
     finally:
         cif.prepareForShutdown()


-- 
To view, visit http://gerrit.ovirt.org/32019
To unsubscribe, visit http://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Ib56b65513e0118b68cd43791bf655c928d6a26e2
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Francesco Romani <fromani at redhat.com>


More information about the vdsm-patches mailing list