Change in vdsm[master]: repoplot: Prepare for matching multiple patterns

nsoffer at redhat.com nsoffer at redhat.com
Thu Feb 25 10:58:47 UTC 2016


Nir Soffer has uploaded a new change for review.

Change subject: repoplot: Prepare for matching multiple patterns
......................................................................

repoplot: Prepare for matching multiple patterns

To understand delays in storage domains monitors, we need to plot also
the lvm commands, and possibly other commands (e.g, dd, qemu-img). This
patch changes parsing so we can handle multiple patterns.

I tested multiple patterns using regular expressions, and they are
significantly slower than multiple string.find() on the same line.

Timestamps are needed in all pattern handler functions, so parsing
timestamps moved to separate function.

Change-Id: Id0fbf7bfdf73d0f8088afe459124272782396ede
Signed-off-by: Nir Soffer <nsoffer at redhat.com>
---
M contrib/repoplot
1 file changed, 33 insertions(+), 15 deletions(-)


  git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/13/54013/1

diff --git a/contrib/repoplot b/contrib/repoplot
index 170642a..b6c1d35 100755
--- a/contrib/repoplot
+++ b/contrib/repoplot
@@ -41,8 +41,6 @@
 import pandas
 from matplotlib import pyplot
 
-REPOSTAT = "Run and protect: repoStats, Return response:"
-
 
 def main(args):
     args = parse_args(args)
@@ -79,27 +77,47 @@
 
 def parse(files):
     """
-    Parse repoStat from vdsm log. Return dict of DomainStats objects.
+    Parse patterns from vdsm log. Return dict of DomainStats objects.
     """
+    patterns = [
+        # Match repoStats response log:
+        # jsonrpc.Executor/5::INFO::2016-02-17
+        # 19:26:33,837::logUtils::51::dispatcher::(wrapper) Run and protect:
+        # repoStats, Return response: ...
+        (add_repostats, "Run and protect: repoStats, Return response:"),
+    ]
+
     stats = defaultdict(DomainStats)
 
     for line in fileinput.input(files):
-        pos = line.find(REPOSTAT)
-        if pos == -1:
-            continue
-        timestamp = line.split("::", 3)[2]
-        timestamp, millis = timestamp.split(",", 1)
-        timestamp = pandas.Timestamp(timestamp)
-        response = eval(line[pos + len(REPOSTAT):])
-        for uuid, info in response.items():
-            ds = stats[uuid]
-            ds.timestamp.append(timestamp)
-            ds.lastcheck.append(float(info["lastCheck"]))
-            ds.delay.append(float(info["delay"]))
+        for func, pattern in patterns:
+            start = line.find(pattern)
+            if start != -1:
+                end = start + len(pattern)
+                func(stats, line, start, end)
 
     return stats
 
 
+def add_repostats(stats, line, start, end):
+    """
+    Add repostats samples from repoStats response line
+    """
+    timestamp = parse_timestamp(line)
+    response = eval(line[end:])
+    for uuid, info in response.items():
+        ds = stats[uuid]
+        ds.timestamp.append(timestamp)
+        ds.lastcheck.append(float(info["lastCheck"]))
+        ds.delay.append(float(info["delay"]))
+
+
+def parse_timestamp(line):
+    timestamp = line.split("::", 3)[2]
+    timestamp, millis = timestamp.split(",", 1)
+    return pandas.Timestamp(timestamp)
+
+
 def dataframe(stats, key):
     """
     Create pandas.DataFrame with one column per domain for given key.


-- 
To view, visit https://gerrit.ovirt.org/54013
To unsubscribe, visit https://gerrit.ovirt.org/settings

Gerrit-MessageType: newchange
Gerrit-Change-Id: Id0fbf7bfdf73d0f8088afe459124272782396ede
Gerrit-PatchSet: 1
Gerrit-Project: vdsm
Gerrit-Branch: master
Gerrit-Owner: Nir Soffer <nsoffer at redhat.com>


More information about the vdsm-patches mailing list