[PATCH 1/2] Remove unnecessary interface from 3_vlans_over_active_backup_bond recipe
by Jiri Prochazka
On host2 machine are VLANs configured on one ethernet devices,
therefore second ethernet interface is not used
Signed-off-by: Jiri Prochazka <jprochaz(a)redhat.com>
---
recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml | 5 -----
1 file changed, 5 deletions(-)
diff --git a/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml b/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml
index 1b8d0c9..a4ce2a6 100644
--- a/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml
+++ b/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml
@@ -61,11 +61,6 @@
<host id="testmachine2">
<interfaces>
<eth id="eth1" label="tnet" />
- <eth id="eth2" label="tnet">
- <addresses>
- <address value="2002::2/64" />
- </addresses>
- </eth>
<vlan id="vlan10">
<options>
<option name="vlan_tci" value="10" />
--
2.1.0
9 years
[lnst] recipes: add virtio driver into phase1 and 2 virtual setups
by Jiří Pírko
commit cac8476260d5d77045041331ae7a14fa0d546401
Author: Jan Tluka <jtluka(a)redhat.com>
Date: Mon Apr 20 14:51:38 2015 +0200
recipes: add virtio driver into phase1 and 2 virtual setups
To properly match virtual guest in virtual setups this patch adds
device driver parameter for guest's ethernet device.
Signed-off-by: Jan Tluka <jtluka(a)redhat.com>
Signed-off-by: Jiri Pirko <jiri(a)resnulli.us>
.../phase1/virtual_bridge_vlan_in_guest.xml | 6 +++++-
.../phase1/virtual_bridge_vlan_in_host.xml | 3 +++
.../phase2/virtual_ovs_bridge_vlan_in_guest.xml | 6 +++++-
3 files changed, 13 insertions(+), 2 deletions(-)
---
diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
index fb5bf61..1899b38 100644
--- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
+++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
@@ -20,7 +20,11 @@
</host>
<host id="guest1">
<interfaces>
- <eth id="guestnic" label="to_guest" />
+ <eth id="guestnic" label="to_guest">
+ <params>
+ <param name="driver" value="virtio" />
+ </params>
+ </eth>
<vlan id="vlan10">
<options>
<option name="vlan_tci" value="10" />
diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
index 8636ce0..7fb917e 100644
--- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
+++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
@@ -29,6 +29,9 @@
<host id="guest1">
<interfaces>
<eth id="guestnic" label="to_guest">
+ <params>
+ <param name="driver" value="virtio" />
+ </params>
<addresses>
<address>192.168.10.10/24</address>
<address>2002::10:10/64</address>
diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
index c262cab..eb031b8 100644
--- a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
+++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
@@ -17,7 +17,11 @@
</host>
<host id="guest1">
<interfaces>
- <eth id="guestnic" label="to_guest" />
+ <eth id="guestnic" label="to_guest">
+ <params>
+ <param name="driver" value="virtio" />
+ </params>
+ </eth>
<vlan id="vlan10">
<options>
<option name="vlan_tci" value="10" />
9 years
[lnst] Revert "recipes: use proper virtio_net driver name in virtual setups"
by Jiří Pírko
commit fb4a3b57886c5413f9f742218a71d2c6840790f4
Author: Jan Tluka <jtluka(a)redhat.com>
Date: Mon Apr 20 14:36:31 2015 +0200
Revert "recipes: use proper virtio_net driver name in virtual setups"
This reverts commit 6062bbad7862eccce94a66afbc4b3bb0d487dbfa.
Let's stay with original 'virtio' value so that we won't have to add
workaround in LNST.
Signed-off-by: Jiri Pirko <jiri(a)resnulli.us>
.../phase1/virtual_bridge_vlan_in_guest.xml | 6 +-----
.../phase1/virtual_bridge_vlan_in_host.xml | 3 ---
.../phase2/virtual_ovs_bridge_vlan_in_guest.xml | 6 +-----
.../phase2/virtual_ovs_bridge_vlan_in_host.xml | 2 +-
4 files changed, 3 insertions(+), 14 deletions(-)
---
diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
index 0d448a8..fb5bf61 100644
--- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
+++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
@@ -20,11 +20,7 @@
</host>
<host id="guest1">
<interfaces>
- <eth id="guestnic" label="to_guest">
- <params>
- <param name="driver" value="virtio_net"/>
- </params>
- </eth>
+ <eth id="guestnic" label="to_guest" />
<vlan id="vlan10">
<options>
<option name="vlan_tci" value="10" />
diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
index 26a16c3..8636ce0 100644
--- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
+++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
@@ -29,9 +29,6 @@
<host id="guest1">
<interfaces>
<eth id="guestnic" label="to_guest">
- <params>
- <param name="driver" value="virtio_net"/>
- </params>
<addresses>
<address>192.168.10.10/24</address>
<address>2002::10:10/64</address>
diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
index 0cfd079..c262cab 100644
--- a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
+++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
@@ -17,11 +17,7 @@
</host>
<host id="guest1">
<interfaces>
- <eth id="guestnic" label="to_guest">
- <params>
- <param name="driver" value="virtio_net"/>
- </params>
- </eth>
+ <eth id="guestnic" label="to_guest" />
<vlan id="vlan10">
<options>
<option name="vlan_tci" value="10" />
diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.xml b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.xml
index 6b2c978..d425b82 100644
--- a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.xml
+++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.xml
@@ -24,7 +24,7 @@
<interfaces>
<eth id="guestnic" label="to_guest">
<params>
- <param name="driver" value="virtio_net"/>
+ <param name="driver" value="virtio"/>
</params>
<addresses>
<address>192.168.10.10/24</address>
9 years
[lnst] Remove unnecessary interface from 3_vlans_over_active_backup_bond recipe
by Jiří Pírko
commit 9a714874f6e7dca33b92a9e9467ab431e1e39021
Author: Jiri Prochazka <jprochaz(a)redhat.com>
Date: Mon Apr 20 13:47:42 2015 +0200
Remove unnecessary interface from 3_vlans_over_active_backup_bond recipe
On host2 machine are VLANs configured on one ethernet devices,
therefore second ethernet interface is not used
Signed-off-by: Jiri Prochazka <jprochaz(a)redhat.com>
Signed-off-by: Jiri Pirko <jiri(a)resnulli.us>
.../phase1/3_vlans_over_active_backup_bond.xml | 5 -----
1 files changed, 0 insertions(+), 5 deletions(-)
---
diff --git a/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml b/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml
index 1b8d0c9..a4ce2a6 100644
--- a/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml
+++ b/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml
@@ -61,11 +61,6 @@
<host id="testmachine2">
<interfaces>
<eth id="eth1" label="tnet" />
- <eth id="eth2" label="tnet">
- <addresses>
- <address value="2002::2/64" />
- </addresses>
- </eth>
<vlan id="vlan10">
<options>
<option name="vlan_tci" value="10" />
9 years
[PATCH] recipes: add virtio driver into phase1 and 2 virtual setups
by Jan Tluka
To properly match virtual guest in virtual setups this patch adds
device driver parameter for guest's ethernet device.
Signed-off-by: Jan Tluka <jtluka(a)redhat.com>
---
recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml | 6 +++++-
recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml | 3 +++
.../regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml | 6 +++++-
3 files changed, 13 insertions(+), 2 deletions(-)
diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
index fb5bf61..1899b38 100644
--- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
+++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml
@@ -20,7 +20,11 @@
</host>
<host id="guest1">
<interfaces>
- <eth id="guestnic" label="to_guest" />
+ <eth id="guestnic" label="to_guest">
+ <params>
+ <param name="driver" value="virtio" />
+ </params>
+ </eth>
<vlan id="vlan10">
<options>
<option name="vlan_tci" value="10" />
diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
index 8636ce0..7fb917e 100644
--- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
+++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml
@@ -29,6 +29,9 @@
<host id="guest1">
<interfaces>
<eth id="guestnic" label="to_guest">
+ <params>
+ <param name="driver" value="virtio" />
+ </params>
<addresses>
<address>192.168.10.10/24</address>
<address>2002::10:10/64</address>
diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
index c262cab..eb031b8 100644
--- a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
+++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml
@@ -17,7 +17,11 @@
</host>
<host id="guest1">
<interfaces>
- <eth id="guestnic" label="to_guest" />
+ <eth id="guestnic" label="to_guest">
+ <params>
+ <param name="driver" value="virtio" />
+ </params>
+ </eth>
<vlan id="vlan10">
<options>
<option name="vlan_tci" value="10" />
--
1.9.3
9 years
[lnst: 5/6] Task: LNST integration with PerfRepo
by Jiří Pírko
commit 3b11e390c64ea94a702c12b0b51ae6618aa6259e
Author: Ondrej Lichtner <olichtne(a)redhat.com>
Date: Fri Apr 17 14:56:17 2015 +0200
Task: LNST integration with PerfRepo
This commit extends the TaskAPI to support PerfRepo.
First of all we add 2 new classes: PerfRepoAPI and PerfRepoResult.
These implement a simple API that will be used from python tasks to
interact with PerfRepo.
With that the ControllerAPI class was extended to contain a reference
to a PerfRepoAPI object a method to create a connection to a PerfRepo
instance and two helper functions for retrieving configuration and
mapping information.
An example python task work flow would look something like this:
perf_api = ctl.connect_PerfRepo()
result = perf_api.new_result(tet_name, test_run_name)
result.add_value(metric_name, performance_result)
result.set_tags(list_of_string_tags)
perf_api.save_result(result)
perf_api.compare_to_baseline(result, report_id_with_a_baseline, metric_name)
It is important to know that the compare_to_baseline method expects that
you're always using 3 values to represent a performance result:
metric_name - the median or average result
metric_name_min - metric_name - deviation
metric_name_max - metric_name + deviation
The method returns True if the new result is better than the current
baseline and False if it's worse. The criterium for being worse is that
the two intervals don't have an intersection and "higher is better" or
"lower is better" criterium is stored in Perfrepo.
This might be changed later but it's what we're going with for now.
Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com>
Signed-off-by: Jiri Pirko <jiri(a)resnulli.us>
lnst/Controller/Task.py | 127 +++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 127 insertions(+), 0 deletions(-)
---
diff --git a/lnst/Controller/Task.py b/lnst/Controller/Task.py
index 0aa3d1a..be6b5cc 100644
--- a/lnst/Controller/Task.py
+++ b/lnst/Controller/Task.py
@@ -10,6 +10,11 @@ __author__ = """
rpazdera(a)redhat.com (Radek Pazdera)
"""
+from lnst.Controller.PerfRepo import PerfRepoRESTAPI
+from lnst.Controller.PerfRepo import PerfRepoTestExecution
+from lnst.Controller.PerfRepo import PerfRepoValue
+from lnst.Common.Utils import dot_to_dict, dict_to_dot, list_to_dot
+
# The handle to be imported from each task
ctl = None
@@ -22,6 +27,8 @@ class ControllerAPI(object):
self._ctl = ctl
self._result = True
+ self._perf_repo_api = PerfRepoAPI()
+
self._hosts = {}
for host_id, host in hosts.iteritems():
self._hosts[host_id] = HostAPI(self, host_id, host)
@@ -95,6 +102,32 @@ class ControllerAPI(object):
"""
return self._ctl._get_alias(alias)
+ def connect_PerfRepo(self, hostname, username, password):
+ if not self._perf_repo_api.connected():
+ self._perf_repo_api.connect(hostname, username, password)
+ return self._perf_repo_api
+
+ def get_configuration(self):
+ return self._ctl._recipe["machines"]
+
+ def get_mapping(self):
+ match = self._ctl.get_pool_match()
+ mapping = []
+ for m_id, m in match["machines"].iteritems():
+ machine = {}
+ machine["id"] = m_id
+ machine["pool_id"] = m["target"]
+ machine["hostname"] = m["hostname"]
+ machine["interface"] = []
+ for i_id, i in m["interfaces"].iteritems():
+ interface = {}
+ interface["id"] = i_id
+ interface["pool_id"] = i["target"]
+ interface["hwaddr"] = i["hwaddr"]
+ machine["interface"].append(interface)
+ mapping.append(machine)
+ return mapping
+
class HostAPI(object):
""" An API class representing a host machine. """
@@ -413,3 +446,97 @@ class VolatileValue(object):
def __str__(self):
return str(self.get_val())
+
+class PerfRepoAPI(object):
+ def __init__(self):
+ self._rest_api = None
+
+ def connected(self):
+ return self._rest_api is not None
+
+ def connect(self, hostname, username, password):
+ self._rest_api = PerfRepoRESTAPI(hostname, username, password)
+
+ def new_result(self, testUid, name):
+ result = PerfRepoResult(testUid, name)
+ return result
+
+ def save_result(self, result):
+ if not isinstance(result, PerfRepoResult):
+ raise TaskError("Parameter result must be an instance "\
+ "of PerfRepoResult")
+ elif self._rest_api is None:
+ raise TaskError("Not connected to PerfRepo.")
+ else:
+ self._rest_api.testExecution_create(result.get_testExecution())
+
+ def get_baseline(self, report_id):
+ report = self._rest_api.report_get_by_id(report_id)
+ return report.get_baseline()
+
+ def compare_to_baseline(self, result, report_id, metric_name):
+ baseline = self.get_baseline(report_id)
+ baseline_exec_id = baseline["execId"]
+ baseline_testExec = self._rest_api.testExecution_get(baseline_exec_id)
+ result_testExec = result.get_testExecution()
+
+ return self.compare_testExecutions(result_testExec,
+ baseline_testExec,
+ metric_name)
+
+ def compare_testExecutions(self, first, second, metric_name):
+ first_value = first.get_value(metric_name)
+ first_min = first.get_value(metric_name + "_min")
+ first_max = first.get_value(metric_name + "_max")
+
+ second_value = second.get_value(metric_name)
+ second_min = second.get_value(metric_name + "_min")
+ second_max = second.get_value(metric_name + "_max")
+
+ comp = second_value.get_comparator()
+ print second_min.get_result(), first_max.get_result()
+ if comp == "HB":
+ if second_min.get_result() > first_max.get_result():
+ return False
+ return True
+ elif comp == "LB":
+ if first_min.get_result() > second_max.get_result():
+ return False
+ return True
+ else:
+ return False
+ return False
+
+class PerfRepoResult(object):
+ def __init__(self, testUid, name):
+ self._testExecution = PerfRepoTestExecution()
+ self._testExecution.set_testUid(testUid)
+ self._testExecution.set_name(name)
+ self.set_configuration(ctl.get_configuration())
+ self.set_mapping(ctl.get_mapping())
+
+ def add_value(self, val_name, value):
+ perf_value = PerfRepoValue()
+ perf_value.set_metricName(val_name)
+ perf_value.set_result(value)
+
+ self._testExecution.add_value(perf_value)
+
+ def set_configuration(self, configuration=None):
+ if configuration is None:
+ configuration = ctl.get_configuration()
+ for pair in list_to_dot(configuration, "configuration.", "machine"):
+ self._testExecution.add_parameter(pair[0], pair[1])
+
+ def set_mapping(self, mapping=None):
+ if mapping is None:
+ mapping = ctl.get_mapping()
+ for pair in list_to_dot(mapping, "mapping.", "machine"):
+ self._testExecution.add_parameter(pair[0], pair[1])
+
+ def set_tags(self, tags):
+ for tag in tags:
+ self._testExecution.add_tag(tag)
+
+ def get_testExecution(self):
+ return self._testExecution
9 years
[PATCH 0/4] PerfRepo support
by Ondrej Lichtner
From: Ondrej Lichtner <olichtne(a)redhat.com>
The following set of patches implements basic support for talking to PerfRepo.
There still are some placeholders for functions that are not important for now
and bugs are to be expected.
There is a short example work flow example in the last patch of the set but you
still need to have a PerfRepo instance available to try everything out, I'll be
sending a follow-up email to interested parties that will explain in more detail
how to try this out.
Any comments and ideas are always welcome.
Ondrej Lichtner (4):
Utils: add utility dictionary functions
SlavePool: add hostname and hwaddr to match info
add lnst/Controller/PerfRepo.py
Task: LNST integration with PerfRepo
lnst/Common/Utils.py | 52 +++
lnst/Controller/NetTestController.py | 3 +-
lnst/Controller/NetTestResultSerializer.py | 3 +-
lnst/Controller/PerfRepo.py | 532 +++++++++++++++++++++++++++++
lnst/Controller/SlavePool.py | 17 +-
lnst/Controller/Task.py | 127 +++++++
6 files changed, 728 insertions(+), 6 deletions(-)
create mode 100644 lnst/Controller/PerfRepo.py
--
2.1.0
9 years
[lnst: 4/6] add lnst/Controller/PerfRepo.py
by Jiří Pírko
commit 59b3fd8c52000854b2ea5c2b858c85d03ee58cc0
Author: Ondrej Lichtner <olichtne(a)redhat.com>
Date: Fri Apr 17 14:56:16 2015 +0200
add lnst/Controller/PerfRepo.py
PerfRepo.py is a module that wraps the REST API provided by a PerfRepo
instance. Classes PerfRepoObject, PerfRepoTest, PerfRepoMetric,
PerfRepoTestExecution, PerfRepoValue and PerfRepoReport define data
objects that we can store or retrieve from the PerfRepo database. They
implement methods for setting and getting their attributes and a helper
functions for xml serialization, they can also be initialized from an
xml-serialized object.
The class PerfRepoRESTAPI actually wraps the REST API provided by
PerfRepo.
This is still an experimental implementation so there are some TODOs,
methods that are just placeholders for now and bugs are expected. What
is implemented is what is needed for basic integration with LNST.
Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com>
Signed-off-by: Jiri Pirko <jiri(a)resnulli.us>
lnst/Controller/PerfRepo.py | 532 +++++++++++++++++++++++++++++++++++++++++++
1 files changed, 532 insertions(+), 0 deletions(-)
---
diff --git a/lnst/Controller/PerfRepo.py b/lnst/Controller/PerfRepo.py
new file mode 100644
index 0000000..399f9d5
--- /dev/null
+++ b/lnst/Controller/PerfRepo.py
@@ -0,0 +1,532 @@
+"""
+This module contains the API for PerfRepo.
+
+Copyright 2015 Red Hat, Inc.
+Licensed under the GNU General Public License, version 2 as
+published by the Free Software Foundation; see COPYING for details.
+"""
+
+__author__ = """
+olichtne(a)redhat.com (Ondrej Lichtner)
+"""
+
+import requests
+import collections
+import datetime
+import re
+from types import StringType, NoneType
+from xml.etree import ElementTree
+from xml.etree.ElementTree import Element
+from lnst.Common.Utils import recursive_dict_update
+from lnst.Common.Utils import dot_to_dict
+
+class PerfRepoException(Exception):
+ pass
+
+class PerfRepoObject(object):
+ def __init__(self):
+ pass
+
+ def _set_element_atrib(self, element, name, value):
+ if value != None:
+ element.set(name, value)
+
+ def to_xml(self):
+ pass
+
+ def to_xml_string(self):
+ root = self.to_xml()
+ return ElementTree.tostring(root)
+
+class PerfRepoTest(PerfRepoObject):
+ def __init__(self, xml=None):
+ if type(xml) is NoneType:
+ self._id = None
+ self._name = None
+ self._uid = None
+ self._description = None
+ self._groupid = None
+ self._metrics = []
+ elif type(xml) is StringType or isinstance(xml, Element):
+ if type(xml) is StringType:
+ root = ElementTree.fromstring(xml)
+ else:
+ root = xml
+ if root.tag != "test":
+ raise PerfRepoException("Invalid xml.")
+
+ self._id = root.get("id")
+ self._name = root.get("name")
+ self._uid = root.get("uid")
+ self._groupid = root.get("groupId")
+ self._description = root.find("description").text
+ self._metrics = []
+ for metric in root.find("metrics"):
+ if metric.tag != "metric":
+ continue
+ self._metrics.append(PerfRepoMetric(metric))
+ else:
+ raise PerfRepoException("Parameter xml must be"\
+ " a string, an Element or None")
+
+ def get_id(self):
+ return self._id
+
+ def get_name(self):
+ return self._name
+
+ def get_uid(self):
+ return self._uid
+
+ def get_description(self):
+ return self._description
+
+ def get_groupid(self):
+ return self._groupid
+
+ def get_metrics(self):
+ return self._metrics
+
+ def set_id(self, id):
+ self._id = id
+
+ def set_name(self, name):
+ self._name = name
+
+ def set_uid(self, uid):
+ self._uid = uid
+
+ def set_description(self, description):
+ self._description = description
+
+ def set_groupid(self, groupid):
+ self._groupid = groupid
+
+ def to_xml(self):
+ root = Element('test')
+ self._set_element_atrib(root, 'id', self._id)
+ self._set_element_atrib(root, 'name', self._name)
+ self._set_element_atrib(root, 'uid', self._uid)
+ self._set_element_atrib(root, 'groupId', self._groupid)
+ description = ElementTree.SubElement(root, 'description')
+ description.text = self._description
+ metrics = ElementTree.SubElement(root, 'metrics')
+ for metric in self._metrics:
+ metrics.append(metric.to_xml())
+
+ return root
+
+class PerfRepoTestExecution(PerfRepoObject):
+ def __init__(self, xml=None):
+ if type(xml) is NoneType:
+ self._id = None
+ self._name = None
+ self._started = datetime.datetime.now().isoformat()
+ self._testId = None
+ self._testUid = None
+ self._comment = ""
+
+ self._values = []
+ self._tags = []
+ self._parameters = []
+ elif type(xml) is StringType or isinstance(xml, Element):
+ if type(xml) is StringType:
+ root = ElementTree.fromstring(xml)
+ else:
+ root = xml
+ if root.tag != "testExecution":
+ raise PerfRepoException("Invalid xml.")
+
+ self._id = root.get("id")
+ self._name = root.get("name")
+ self._started = root.get("started")
+ self._testId = root.get("testId")
+ self._testUid = root.get("testUid")
+ self._comment = root.find("comment").text
+
+ self._values = []
+ for value in root.find("values"):
+ if value.tag != "value":
+ continue
+ self._values.append(PerfRepoValue(value))
+
+ self._tags = []
+ for tag in root.find("tags"):
+ if tag.tag != "tag":
+ continue
+ self._tags.append(tag.get("name"))
+
+ self._parameters = []
+ for param in root.find("parameters"):
+ if param.tag != "parameter":
+ continue
+ self._parameters.append((param.get("name"), param.get("value")))
+ else:
+ raise PerfRepoException("Parameter xml must be"\
+ " a string, an Element or None")
+
+ def set_id(self, id):
+ self._id = id
+
+ def set_name(self, name):
+ self._name = name
+
+ def set_started(self, date=None):
+ if isinstance(date, NoneType):
+ date = datetime.datetime.now()
+ self._started = date.isoformat()
+ else:
+ self._started = date
+
+ def set_testId(self, testId):
+ if isinstance(testId, PerfRepoTest):
+ self._testId = testId.get_id()
+ else:
+ self._testId = testId
+
+ def set_testUid(self, testUid):
+ if isinstance(testUid, PerfRepoTest):
+ self._testUid = testUid.get_uid()
+ else:
+ self._testUid = testUid
+
+ def set_comment(self, comment):
+ self._comment = comment
+
+ def add_value(self, value):
+ self._values.append(value)
+
+ def get_values(self):
+ return self._values
+
+ def get_value(self, metric_name):
+ for val in self._values:
+ if val.get_metricName() == metric_name:
+ return val
+
+ def add_tag(self, tag):
+ self._tags.append(tag)
+
+ def add_parameter(self, name, value):
+ self._parameters.append((name, value))
+
+ def to_xml(self):
+ root = Element('testExecution')
+ self._set_element_atrib(root, 'id', self._id)
+ self._set_element_atrib(root, 'name', self._name)
+ self._set_element_atrib(root, 'started', self._started)
+ self._set_element_atrib(root, 'testId', self._testId)
+ self._set_element_atrib(root, 'testUid', self._testUid)
+ comment = ElementTree.SubElement(root, 'comment')
+ comment.text = self._comment
+
+ parameters = ElementTree.SubElement(root, 'parameters')
+ for param in self._parameters:
+ param_elem = ElementTree.SubElement(parameters, 'parameter')
+ self._set_element_atrib(param_elem, "name", param[0])
+ self._set_element_atrib(param_elem, "value", param[1])
+
+ tags = ElementTree.SubElement(root, 'tags')
+ for tag in self._tags:
+ tag_elem = ElementTree.SubElement(tags, 'tag')
+ self._set_element_atrib(tag_elem, "name", tag)
+
+ values = ElementTree.SubElement(root, 'values')
+ for value in self._values:
+ values.append(value.to_xml())
+
+ return root
+
+class PerfRepoValue(PerfRepoObject):
+ def __init__(self, xml=None):
+ if type(xml) is NoneType:
+ self._metricComparator = None
+ self._metricName = None
+ self._result = None
+ self._parameters = []
+ elif type(xml) is StringType or isinstance(xml, Element):
+ if type(xml) is StringType:
+ root = ElementTree.fromstring(xml)
+ else:
+ root = xml
+ if root.tag != "value":
+ raise PerfRepoException("Invalid xml.")
+
+ self._metricComparator = root.get("metricComparator")
+ self._metricName = root.get("metricName")
+ self._result = float(root.get("result"))
+
+ self._parameters = []
+ for param in root.find("parameters"):
+ if param.tag != "parameter":
+ continue
+ self._parameters.append((param.get("name"), param.get("value")))
+ else:
+ raise PerfRepoException("Parameter xml must be"\
+ " a string, an Element or None")
+
+ def set_result(self, result):
+ self._result = result
+
+ def set_comparator(self, comparator):
+ if comparator not in ["HB", "LB"]:
+ raise PerfRepoException("Comparator must be HB/LB.")
+ self._metricComparator = comparator
+
+ def set_metricName(self, name):
+ self._metricName = name
+
+ def add_parameter(self, name, value):
+ self._parameters.append((name, value))
+
+ def get_metricName(self):
+ return self._metricName
+
+ def get_comparator(self):
+ return self._metricComparator
+
+ def get_result(self):
+ return self._result
+
+ def to_xml(self):
+ root = Element('value')
+ self._set_element_atrib(root, 'metricComparator',
+ self._metricComparator)
+ self._set_element_atrib(root, 'metricName', self._metricName)
+ self._set_element_atrib(root, 'result', str(self._result))
+
+ parameters = ElementTree.SubElement(root, 'parameters')
+ for param in self._parameters:
+ param_elem = ElementTree.SubElement(parameters, 'parameter')
+ self._set_element_atrib(param_elem, "name", param[0])
+ self._set_element_atrib(param_elem, "value", param[1])
+ return root
+
+class PerfRepoMetric(PerfRepoObject):
+ def __init__(self, xml=None):
+ if type(xml) is NoneType:
+ self._id = None
+ self._name = None
+ self._description = None
+ self._comparator = None
+ elif type(xml) is StringType or isinstance(xml, Element):
+ if type(xml) is StringType:
+ root = ElementTree.fromstring(xml)
+ else:
+ root = xml
+ if root.tag != "metric":
+ raise PerfRepoException("Invalid xml.")
+
+ self._id = root.get("id")
+ self._name = root.get("name")
+ self._comparator = root.get("comparator")
+ self._description = root.find("description").text
+ else:
+ raise PerfRepoException("Parameter xml must be"\
+ " a string, an Element or None")
+
+ def get_id(self):
+ return self._id
+
+ def get_name(self):
+ return self._id
+
+ def get_description(self):
+ return self._description
+
+ def get_comparator(self):
+ return self._comparator
+
+ def set_id(self, id):
+ self._id = id
+
+ def set_name(self, name):
+ self._name = name
+
+ def set_description(self, description):
+ self._description = description
+
+ def set_comparator(self, comparator):
+ if comapator not in ["HB", "LB"]:
+ raise PerfRepoException("Invalid comparator value.")
+ self._comparator = comparator
+
+ def to_xml(self):
+ root = Element('metric')
+ self._set_element_atrib(root, 'id', self._id)
+ self._set_element_atrib(root, 'name', self._name)
+ description = ElementTree.SubElement(root, 'description')
+ description.text = self._description
+ self._set_element_atrib(root, 'comparator', self._comparator)
+
+ return root
+
+class PerfRepoReport(PerfRepoObject):
+ def __init__(self, xml=None):
+ self._baselines = []
+ if type(xml) is NoneType:
+ self._id = None
+ self._name = None
+ self._type = None
+ self._properties = {}
+ elif type(xml) is StringType or isinstance(xml, Element):
+ if type(xml) is StringType:
+ root = ElementTree.fromstring(xml)
+ else:
+ root = xml
+ if root.tag != "report":
+ raise PerfRepoException("Invalid xml.")
+
+ self._id = root.get("id")
+ self._name = root.get("name")
+ self._type = root.get("type")
+ self._properties = {}
+ for entry in root.find("properties"):
+ if entry.tag != "entry":
+ continue
+ key_tag = entry.find("key")
+ value_tag = entry.find("value")
+ tmp_dict = dot_to_dict(value_tag.get("name"),
+ value_tag.get("value"))
+ recursive_dict_update(self._properties, tmp_dict)
+
+ self._parse_baseline_properties()
+ else:
+ raise PerfRepoException("Parameter xml must be"\
+ " a string, an Element or None")
+
+ def _parse_baseline_properties(self):
+ chart = self._properties["chart0"] #TODO more charts per report?
+ baseline_re = re.compile(r'baseline(\d+)')
+ for key, value in chart.iteritems():
+ match = baseline_re.match(key)
+ if match is None:
+ continue
+ self._baselines.append(value)
+
+ def get_baseline(self, index=-1):
+ return self._baselines[index]
+
+class PerfRepoRESTAPI(object):
+ '''Wrapper class for the REST API provided by PerfRepo'''
+ def __init__(self, url, user, password):
+ self._url = url
+ self._user = user
+ self._password = password
+
+ self._session = requests.Session()
+ self._session.auth = (self._user, self._password)
+ self._session.stream = True
+ self._session.headers['Content-Type'] = 'text/xml'
+
+ def test_get_by_id(self, test_id):
+ get_url = self._url + '/rest/test/id/%s' % test_id
+ response = self._session.get(get_url)
+ if response.status_code != 200:
+ return None
+ else:
+ return PerfRepoTest(response.content)
+
+ def test_get_by_uid(self, test_uid):
+ get_url = self._url + '/rest/test/uid/%s' % test_uid
+ response = self._session.get(get_url)
+ if response.status_code != 200:
+ return None
+ else:
+ return PerfRepoTest(response.content)
+
+ def test_create(self, test):
+ post_url = self._url + '/rest/test/create'
+ response = self._session.post(post_url, data=test.to_xml_string())
+ if response.status_code != 201:
+ return None
+ else:
+ test.set_id(response.headers['Location'])
+ return test
+
+ def test_add_metric(self, test_id, metric):
+ post_url = self._url + '/rest/test/id/%s/addMetric' % test_id
+ response = self._session.post(post_url, data=metric.to_xml_string)
+ if response.status_code != 201:
+ return None
+ else:
+ metric.set_id(response.headers['Location'])
+ return metric
+
+ def test_remove(self, test_id):
+ delete_url = self._url + '/rest/test/id/%s' % test_id
+ response = self._session.delete(delete_url)
+ if response.status_code != 204:
+ return False
+ else:
+ return True
+
+ def metric_get(self, metric_id):
+ get_url = self._url + '/rest/metric/%s' % metric_id
+ response = self._session.get(get_url)
+ if response.status_code != 200:
+ return None
+ else:
+ return PerfRepoMetric(response.content)
+
+ def testExecution_get(self, testExec_id):
+ get_url = self._url + '/rest/testExecution/%s' % testExec_id
+ response = self._session.get(get_url)
+ if response.status_code != 200:
+ return None
+ else:
+ return PerfRepoTestExecution(response.content)
+
+ def testExecution_create(self, testExec):
+ post_url = self._url + '/rest/testExecution/create'
+ response = self._session.post(post_url, data=testExec.to_xml_string())
+ if response.status_code != 201:
+ return None
+ else:
+ testExec.set_id(response.headers['Location'])
+ return testExec
+
+ def testExecution_delete(self, testExec_id):
+ delete_url = self._url + '/rest/testExecution/%s' % testExec_id
+ response = self._session.delete(delete_url)
+ if response.status_code != 204:
+ return False
+ else:
+ return True
+
+ def testExecution_add_value(self, value):
+ post_url = self._url + '/rest/testExecution/addValue'
+ #TODO
+ return self._session.post(post_url, data=value)
+
+ def testExecution_get_attachment(self, attachment_id):
+ get_url = self._url + '/rest/testExecution/attachment/%s' % \
+ attachment_id
+ #TODO
+ return self._session.get(get_url)
+
+ def testExecution_add_attachment(self, testExec_id, attachment):
+ post_url = self._url + '/rest/testExecution/%s/addAttachment' % \
+ testExec_id
+ #TODO
+ return self._session.post(post_url, data=attachment)
+
+ def report_get_by_id(self, report_id):
+ get_url = self._url + '/rest/report/id/%s' % report_id
+ response = self._session.get(get_url)
+ if response.status_code != 200:
+ return None
+ else:
+ return PerfRepoReport(response.content)
+
+ def report_create(self, report):
+ #TODO not needed yet and therefore not tested
+ post_url = self._url + '/rest/report/create'
+ self._session.post(post_url, data=report)
+ return None
+
+ def report_delete(self, report_id):
+ #TODO not needed yet and therefore not tested
+ delete_url = self._url + '/rest/report/delete'
+ self._session.delete(post_url)
+ return None
9 years
[lnst: 3/6] SlavePool: add hostname and hwaddr to match info
by Jiří Pírko
commit 1b8d665ccdf67e9d3be510ce3c818e4c4acc38e8
Author: Ondrej Lichtner <olichtne(a)redhat.com>
Date: Fri Apr 17 14:56:15 2015 +0200
SlavePool: add hostname and hwaddr to match info
This adds hostname and interface hardware addres information to the
dictionary describing the used mapping. This required a minor change to
the interface mapping which is why there are changes in
NetTestController and NetTestResultSerializer as well.
This information is going to be useful when looking at results later
for example in PerfRepo.
Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com>
Signed-off-by: Jiri Pirko <jiri(a)resnulli.us>
lnst/Controller/NetTestController.py | 3 ++-
lnst/Controller/NetTestResultSerializer.py | 3 ++-
lnst/Controller/SlavePool.py | 17 +++++++++++++----
3 files changed, 17 insertions(+), 6 deletions(-)
---
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index c5a9fc9..efeeb92 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -228,7 +228,8 @@ class NetTestController:
logging.info(" Setup is using virtual machines.")
for m_id, m in match["machines"].iteritems():
logging.info(" host \"%s\" uses \"%s\"" % (m_id, m["target"]))
- for if_id, pool_id in m["interfaces"].iteritems():
+ for if_id, match in m["interfaces"].iteritems():
+ pool_id = match["target"]
logging.info(" interface \"%s\" matched to \"%s\"" %\
(if_id, pool_id))
diff --git a/lnst/Controller/NetTestResultSerializer.py b/lnst/Controller/NetTestResultSerializer.py
index 515ead7..9f8955a 100644
--- a/lnst/Controller/NetTestResultSerializer.py
+++ b/lnst/Controller/NetTestResultSerializer.py
@@ -98,7 +98,8 @@ class NetTestResultSerializer:
for m_id, m in match["machines"].iteritems():
output_pairs.append((4*" " + "host \"%s\" uses \"%s\"" %\
(m_id, m["target"]), ""))
- for if_id, pool_id in m["interfaces"].iteritems():
+ for if_id, pool_if in m["interfaces"].iteritems():
+ pool_id = pool_if["target"]
output_pairs.append((6*" " + "interface \"%s\" "\
"matched to \"%s\"" %\
(if_id, pool_id), ""))
diff --git a/lnst/Controller/SlavePool.py b/lnst/Controller/SlavePool.py
index 2602f23..2fed8ad 100644
--- a/lnst/Controller/SlavePool.py
+++ b/lnst/Controller/SlavePool.py
@@ -313,14 +313,15 @@ class SlavePool:
used = []
if_map = self._map["machines"][tm_id]["interfaces"]
for t_if, p_if in if_map.iteritems():
- used.append(p_if)
- if_data = pm["interfaces"][p_if]
+ pool_id = p_if["target"]
+ used.append(pool_id)
+ if_data = pm["interfaces"][pool_id]
iface = machine.new_static_interface(t_if, "eth")
iface.set_hwaddr(if_data["params"]["hwaddr"])
for t_net, p_net in self._map["networks"].iteritems():
- if pm["interfaces"][p_if]["network"] == p_net:
+ if pm["interfaces"][pool_id]["network"] == p_net:
iface.set_network(t_net)
break
@@ -609,11 +610,19 @@ class SetupMapper(object):
for machine in self._machine_stack:
m_map = mapping["machines"][machine["m_id"]] = {}
+
m_map["target"] = machine["current_match"]
+
+ hostname = self._pool[m_map["target"]]["params"]["hostname"]
+ m_map["hostname"] = hostname
+
interfaces = m_map["interfaces"] = {}
if_stack = machine["if_stack"]
for interface in if_stack:
- interfaces[interface["if_id"]] = interface["current_match"]
+ i = interfaces[interface["if_id"]] = {}
+ i["target"] = interface["current_match"]
+ pool_if = self._pool[m_map["target"]]["interfaces"][i["target"]]
+ i["hwaddr"] = pool_if["params"]["hwaddr"]
if self._virtual_matching:
9 years
[lnst: 2/6] Utils: add utility dictionary functions
by Jiří Pírko
commit 637c022832d4234a967b9b76492c0c7646d98035
Author: Ondrej Lichtner <olichtne(a)redhat.com>
Date: Fri Apr 17 14:56:14 2015 +0200
Utils: add utility dictionary functions
This adds a couple of utility functions useful when working with
PerfRepo.
recursive_dict_update
recursively updates nested dictionaries
dot_to_dict
parses a pair of dot separated keys into a dictionary hierarhy and
sets the value to the last key
dict_to_dot
reverse function to dot_to_dict, returns a list of (name, value)
pairs
list_to_dot
transforms a list of values to a list of (name, value) pairs where
the name contains dot separated keys, the last key is indexed with
the values index in the list
Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com>
Signed-off-by: Jiri Pirko <jiri(a)resnulli.us>
lnst/Common/Utils.py | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 52 insertions(+), 0 deletions(-)
---
diff --git a/lnst/Common/Utils.py b/lnst/Common/Utils.py
index 19ac898..fb9d9a2 100644
--- a/lnst/Common/Utils.py
+++ b/lnst/Common/Utils.py
@@ -18,6 +18,7 @@ import tempfile
import subprocess
import errno
import ast
+import collections
from _ast import Call, Attribute
from lnst.Common.ExecCmd import exec_cmd
@@ -192,3 +193,54 @@ def get_module_tools(module_path):
f.close()
return tools
+
+def recursive_dict_update(original, update):
+ for key, value in update.iteritems():
+ if isinstance(value, collections.Mapping):
+ r = recursive_dict_update(original.get(key, {}), value)
+ original[key] = r
+ else:
+ original[key] = update[key]
+ return original
+
+def dot_to_dict(name, value):
+ result = {}
+ last = result
+ last_key = None
+ previous = None
+ for key in name.split('.'):
+ last[key] = {}
+ previous = last
+ last = last[key]
+ last_key = key
+ if last_key != None:
+ previous[last_key] = value
+ return result
+
+def list_to_dot(original_list, prefix="", key=""):
+ return_list = []
+ index = 0
+ for value in original_list:
+ iter_key = prefix + key + str(index) + '.'
+ index += 1
+ if isinstance(value, collections.Mapping):
+ sub_list = dict_to_dot(value, iter_key)
+ return_list.extend(sub_list)
+ elif isinstance(value, list):
+ raise Exception("Nested lists not allowed")
+ else:
+ return_list.append((iter_key, value))
+ return return_list
+
+def dict_to_dot(original_dict, prefix=""):
+ return_list = []
+ for key, value in original_dict.iteritems():
+ if isinstance(value, collections.Mapping):
+ sub_list = dict_to_dot(value, prefix + key + '.')
+ return_list.extend(sub_list)
+ elif isinstance(value, list):
+ sub_list = list_to_dot(value, prefix, key)
+ return_list.extend(sub_list)
+ else:
+ return_list.append((prefix+key, str(value)))
+ return return_list
9 years