Change in vdsm[master]: repoplot: Add storage domains monitors commands

nsoffer at redhat.com nsoffer at redhat.com
Fri Feb 26 23:09:58 UTC 2016


Nir Soffer has uploaded a new change for review.

Change subject: repoplot: Add storage domains monitors commands
......................................................................

repoplot: Add storage domains monitors commands

The time to read from storage is already reported in the repostat
"delay" graph. This graphs show the time to run the dd command. We have
seen cases where the delay was very fast, but running dd command took 15
seconds. I suspect that this issue was caused by overloaded vdsm on
machine with many cores, and should be fixed by using cpu_affinity.

Change-Id: I48b84bf4b9ef7518bbf08675f3dd1a0557c524f9
Relates-To: https://bugzilla.redhat.com/1081962
Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
M contrib/repoplot
1 file changed, 28 insertions(+), 3 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/07/54107/1

diff --git a/contrib/repoplot b/contrib/repoplot
index 3595282..18a8c77 100755
--- a/contrib/repoplot
+++ b/contrib/repoplot
@@ -61,10 +61,10 @@
     parser.add_argument("--width", "-x", dest="width", type=int,
                         help="output file width in inches (default 20)")
     parser.add_argument("--height", "-y", dest="height", type=int,
-                        help="output file height in inches (default 20)")
+                        help="output file height in inches (default 25)")
     parser.add_argument("files", nargs="+",
                         help="vdsm log files to process")
-    parser.set_defaults(name=None, format="pdf", width=20, height=20)
+    parser.set_defaults(name=None, format="pdf", width=20, height=25)
     return parser.parse_args(args)
 
 
@@ -76,6 +76,7 @@
         self.repostats = defaultdict(DomainStats)
         self.lvm_commands = defaultdict(CommandStats)
         self.mailbox_commands = defaultdict(CommandStats)
+        self.monitor_commands = defaultdict(CommandStats)
 
 
 class DomainStats(object):
@@ -103,6 +104,20 @@
         # repoStats, Return response: ...
         (add_repostats, "Run and protect: repoStats, Return response:",
             "repostats"),
+
+        # Match storage domains getReadDelay commands.
+        #
+        # Begin:
+        # Thread-16::DEBUG::2016-02-17
+        # 20:14:37,049::blockSD::436::Storage.Misc.excCmd::(getReadDelay)
+        # /usr/bin/taskset --cpu-list 0-7 /usr/bin/dd ...
+        #
+        # End:
+        # Thread-16::DEBUG::2016-02-17
+        # 20:14:37,061::blockSD::436::Storage.Misc.excCmd::(getReadDelay)
+        # SUCCESS: ...
+        (add_command, "::Storage.Misc.excCmd::(getReadDelay)",
+            "monitor_commands"),
 
         # Match LVM commands begin or end logs.
         #
@@ -202,7 +217,7 @@
 
 
 def plot(stats, filename, size):
-    rows = 4
+    rows = 5
     columns = 1
 
     pyplot.figure(figsize=size, dpi=300)
@@ -257,6 +272,16 @@
     for thread, cs in stats.mailbox_commands.iteritems():
         pyplot.plot(cs.timestamp, cs.runtime)
 
+    pyplot.subplot(rows, columns, 5)
+    pyplot.title("Storage domain monitor commands")
+    pyplot.ylabel("runtime (seconds)")
+    pyplot.xlabel("time")
+    pyplot.grid(True)
+    pyplot.axis([lastcheck.index[0], lastcheck.index[-1], 0, 10])
+
+    for thread, cs in stats.monitor_commands.iteritems():
+        pyplot.plot(cs.timestamp, cs.runtime)
+
     pyplot.savefig(filename, bbox_inches="tight")
 
 


-- 
To view, visit https://gerrit.ovirt.org/54107
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: I48b84bf4b9ef7518bbf08675f3dd1a0557c524f9
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer at redhat.com>


More information about the vdsm-patches mailing list