From: Ondrej Lichtner olichtne@redhat.com
The following set of patches implements basic support for talking to PerfRepo. There still are some placeholders for functions that are not important for now and bugs are to be expected.
There is a short example work flow example in the last patch of the set but you still need to have a PerfRepo instance available to try everything out, I'll be sending a follow-up email to interested parties that will explain in more detail how to try this out.
Any comments and ideas are always welcome.
Ondrej Lichtner (4): Utils: add utility dictionary functions SlavePool: add hostname and hwaddr to match info add lnst/Controller/PerfRepo.py Task: LNST integration with PerfRepo
lnst/Common/Utils.py | 52 +++ lnst/Controller/NetTestController.py | 3 +- lnst/Controller/NetTestResultSerializer.py | 3 +- lnst/Controller/PerfRepo.py | 532 +++++++++++++++++++++++++++++ lnst/Controller/SlavePool.py | 17 +- lnst/Controller/Task.py | 127 +++++++ 6 files changed, 728 insertions(+), 6 deletions(-) create mode 100644 lnst/Controller/PerfRepo.py
From: Ondrej Lichtner olichtne@redhat.com
This adds a couple of utility functions useful when working with PerfRepo.
recursive_dict_update recursively updates nested dictionaries dot_to_dict parses a pair of dot separated keys into a dictionary hierarhy and sets the value to the last key dict_to_dot reverse function to dot_to_dict, returns a list of (name, value) pairs list_to_dot transforms a list of values to a list of (name, value) pairs where the name contains dot separated keys, the last key is indexed with the values index in the list
Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- lnst/Common/Utils.py | 52 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)
diff --git a/lnst/Common/Utils.py b/lnst/Common/Utils.py index 19ac898..fb9d9a2 100644 --- a/lnst/Common/Utils.py +++ b/lnst/Common/Utils.py @@ -18,6 +18,7 @@ import tempfile import subprocess import errno import ast +import collections from _ast import Call, Attribute from lnst.Common.ExecCmd import exec_cmd
@@ -192,3 +193,54 @@ def get_module_tools(module_path): f.close()
return tools + +def recursive_dict_update(original, update): + for key, value in update.iteritems(): + if isinstance(value, collections.Mapping): + r = recursive_dict_update(original.get(key, {}), value) + original[key] = r + else: + original[key] = update[key] + return original + +def dot_to_dict(name, value): + result = {} + last = result + last_key = None + previous = None + for key in name.split('.'): + last[key] = {} + previous = last + last = last[key] + last_key = key + if last_key != None: + previous[last_key] = value + return result + +def list_to_dot(original_list, prefix="", key=""): + return_list = [] + index = 0 + for value in original_list: + iter_key = prefix + key + str(index) + '.' + index += 1 + if isinstance(value, collections.Mapping): + sub_list = dict_to_dot(value, iter_key) + return_list.extend(sub_list) + elif isinstance(value, list): + raise Exception("Nested lists not allowed") + else: + return_list.append((iter_key, value)) + return return_list + +def dict_to_dot(original_dict, prefix=""): + return_list = [] + for key, value in original_dict.iteritems(): + if isinstance(value, collections.Mapping): + sub_list = dict_to_dot(value, prefix + key + '.') + return_list.extend(sub_list) + elif isinstance(value, list): + sub_list = list_to_dot(value, prefix, key) + return_list.extend(sub_list) + else: + return_list.append((prefix+key, str(value))) + return return_list
From: Ondrej Lichtner olichtne@redhat.com
This adds hostname and interface hardware addres information to the dictionary describing the used mapping. This required a minor change to the interface mapping which is why there are changes in NetTestController and NetTestResultSerializer as well.
This information is going to be useful when looking at results later for example in PerfRepo.
Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- lnst/Controller/NetTestController.py | 3 ++- lnst/Controller/NetTestResultSerializer.py | 3 ++- lnst/Controller/SlavePool.py | 17 +++++++++++++---- 3 files changed, 17 insertions(+), 6 deletions(-)
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py index c5a9fc9..efeeb92 100644 --- a/lnst/Controller/NetTestController.py +++ b/lnst/Controller/NetTestController.py @@ -228,7 +228,8 @@ class NetTestController: logging.info(" Setup is using virtual machines.") for m_id, m in match["machines"].iteritems(): logging.info(" host "%s" uses "%s"" % (m_id, m["target"])) - for if_id, pool_id in m["interfaces"].iteritems(): + for if_id, match in m["interfaces"].iteritems(): + pool_id = match["target"] logging.info(" interface "%s" matched to "%s"" %\ (if_id, pool_id))
diff --git a/lnst/Controller/NetTestResultSerializer.py b/lnst/Controller/NetTestResultSerializer.py index 515ead7..9f8955a 100644 --- a/lnst/Controller/NetTestResultSerializer.py +++ b/lnst/Controller/NetTestResultSerializer.py @@ -98,7 +98,8 @@ class NetTestResultSerializer: for m_id, m in match["machines"].iteritems(): output_pairs.append((4*" " + "host "%s" uses "%s"" %\ (m_id, m["target"]), "")) - for if_id, pool_id in m["interfaces"].iteritems(): + for if_id, pool_if in m["interfaces"].iteritems(): + pool_id = pool_if["target"] output_pairs.append((6*" " + "interface "%s" "\ "matched to "%s"" %\ (if_id, pool_id), "")) diff --git a/lnst/Controller/SlavePool.py b/lnst/Controller/SlavePool.py index 2602f23..2fed8ad 100644 --- a/lnst/Controller/SlavePool.py +++ b/lnst/Controller/SlavePool.py @@ -313,14 +313,15 @@ class SlavePool: used = [] if_map = self._map["machines"][tm_id]["interfaces"] for t_if, p_if in if_map.iteritems(): - used.append(p_if) - if_data = pm["interfaces"][p_if] + pool_id = p_if["target"] + used.append(pool_id) + if_data = pm["interfaces"][pool_id]
iface = machine.new_static_interface(t_if, "eth") iface.set_hwaddr(if_data["params"]["hwaddr"])
for t_net, p_net in self._map["networks"].iteritems(): - if pm["interfaces"][p_if]["network"] == p_net: + if pm["interfaces"][pool_id]["network"] == p_net: iface.set_network(t_net) break
@@ -609,11 +610,19 @@ class SetupMapper(object):
for machine in self._machine_stack: m_map = mapping["machines"][machine["m_id"]] = {} + m_map["target"] = machine["current_match"] + + hostname = self._pool[m_map["target"]]["params"]["hostname"] + m_map["hostname"] = hostname + interfaces = m_map["interfaces"] = {} if_stack = machine["if_stack"] for interface in if_stack: - interfaces[interface["if_id"]] = interface["current_match"] + i = interfaces[interface["if_id"]] = {} + i["target"] = interface["current_match"] + pool_if = self._pool[m_map["target"]]["interfaces"][i["target"]] + i["hwaddr"] = pool_if["params"]["hwaddr"]
if self._virtual_matching:
From: Ondrej Lichtner olichtne@redhat.com
PerfRepo.py is a module that wraps the REST API provided by a PerfRepo instance. Classes PerfRepoObject, PerfRepoTest, PerfRepoMetric, PerfRepoTestExecution, PerfRepoValue and PerfRepoReport define data objects that we can store or retrieve from the PerfRepo database. They implement methods for setting and getting their attributes and a helper functions for xml serialization, they can also be initialized from an xml-serialized object.
The class PerfRepoRESTAPI actually wraps the REST API provided by PerfRepo.
This is still an experimental implementation so there are some TODOs, methods that are just placeholders for now and bugs are expected. What is implemented is what is needed for basic integration with LNST.
Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- lnst/Controller/PerfRepo.py | 532 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 532 insertions(+) create mode 100644 lnst/Controller/PerfRepo.py
diff --git a/lnst/Controller/PerfRepo.py b/lnst/Controller/PerfRepo.py new file mode 100644 index 0000000..399f9d5 --- /dev/null +++ b/lnst/Controller/PerfRepo.py @@ -0,0 +1,532 @@ +""" +This module contains the API for PerfRepo. + +Copyright 2015 Red Hat, Inc. +Licensed under the GNU General Public License, version 2 as +published by the Free Software Foundation; see COPYING for details. +""" + +__author__ = """ +olichtne@redhat.com (Ondrej Lichtner) +""" + +import requests +import collections +import datetime +import re +from types import StringType, NoneType +from xml.etree import ElementTree +from xml.etree.ElementTree import Element +from lnst.Common.Utils import recursive_dict_update +from lnst.Common.Utils import dot_to_dict + +class PerfRepoException(Exception): + pass + +class PerfRepoObject(object): + def __init__(self): + pass + + def _set_element_atrib(self, element, name, value): + if value != None: + element.set(name, value) + + def to_xml(self): + pass + + def to_xml_string(self): + root = self.to_xml() + return ElementTree.tostring(root) + +class PerfRepoTest(PerfRepoObject): + def __init__(self, xml=None): + if type(xml) is NoneType: + self._id = None + self._name = None + self._uid = None + self._description = None + self._groupid = None + self._metrics = [] + elif type(xml) is StringType or isinstance(xml, Element): + if type(xml) is StringType: + root = ElementTree.fromstring(xml) + else: + root = xml + if root.tag != "test": + raise PerfRepoException("Invalid xml.") + + self._id = root.get("id") + self._name = root.get("name") + self._uid = root.get("uid") + self._groupid = root.get("groupId") + self._description = root.find("description").text + self._metrics = [] + for metric in root.find("metrics"): + if metric.tag != "metric": + continue + self._metrics.append(PerfRepoMetric(metric)) + else: + raise PerfRepoException("Parameter xml must be"\ + " a string, an Element or None") + + def get_id(self): + return self._id + + def get_name(self): + return self._name + + def get_uid(self): + return self._uid + + def get_description(self): + return self._description + + def get_groupid(self): + return self._groupid + + def get_metrics(self): + return self._metrics + + def set_id(self, id): + self._id = id + + def set_name(self, name): + self._name = name + + def set_uid(self, uid): + self._uid = uid + + def set_description(self, description): + self._description = description + + def set_groupid(self, groupid): + self._groupid = groupid + + def to_xml(self): + root = Element('test') + self._set_element_atrib(root, 'id', self._id) + self._set_element_atrib(root, 'name', self._name) + self._set_element_atrib(root, 'uid', self._uid) + self._set_element_atrib(root, 'groupId', self._groupid) + description = ElementTree.SubElement(root, 'description') + description.text = self._description + metrics = ElementTree.SubElement(root, 'metrics') + for metric in self._metrics: + metrics.append(metric.to_xml()) + + return root + +class PerfRepoTestExecution(PerfRepoObject): + def __init__(self, xml=None): + if type(xml) is NoneType: + self._id = None + self._name = None + self._started = datetime.datetime.now().isoformat() + self._testId = None + self._testUid = None + self._comment = "" + + self._values = [] + self._tags = [] + self._parameters = [] + elif type(xml) is StringType or isinstance(xml, Element): + if type(xml) is StringType: + root = ElementTree.fromstring(xml) + else: + root = xml + if root.tag != "testExecution": + raise PerfRepoException("Invalid xml.") + + self._id = root.get("id") + self._name = root.get("name") + self._started = root.get("started") + self._testId = root.get("testId") + self._testUid = root.get("testUid") + self._comment = root.find("comment").text + + self._values = [] + for value in root.find("values"): + if value.tag != "value": + continue + self._values.append(PerfRepoValue(value)) + + self._tags = [] + for tag in root.find("tags"): + if tag.tag != "tag": + continue + self._tags.append(tag.get("name")) + + self._parameters = [] + for param in root.find("parameters"): + if param.tag != "parameter": + continue + self._parameters.append((param.get("name"), param.get("value"))) + else: + raise PerfRepoException("Parameter xml must be"\ + " a string, an Element or None") + + def set_id(self, id): + self._id = id + + def set_name(self, name): + self._name = name + + def set_started(self, date=None): + if isinstance(date, NoneType): + date = datetime.datetime.now() + self._started = date.isoformat() + else: + self._started = date + + def set_testId(self, testId): + if isinstance(testId, PerfRepoTest): + self._testId = testId.get_id() + else: + self._testId = testId + + def set_testUid(self, testUid): + if isinstance(testUid, PerfRepoTest): + self._testUid = testUid.get_uid() + else: + self._testUid = testUid + + def set_comment(self, comment): + self._comment = comment + + def add_value(self, value): + self._values.append(value) + + def get_values(self): + return self._values + + def get_value(self, metric_name): + for val in self._values: + if val.get_metricName() == metric_name: + return val + + def add_tag(self, tag): + self._tags.append(tag) + + def add_parameter(self, name, value): + self._parameters.append((name, value)) + + def to_xml(self): + root = Element('testExecution') + self._set_element_atrib(root, 'id', self._id) + self._set_element_atrib(root, 'name', self._name) + self._set_element_atrib(root, 'started', self._started) + self._set_element_atrib(root, 'testId', self._testId) + self._set_element_atrib(root, 'testUid', self._testUid) + comment = ElementTree.SubElement(root, 'comment') + comment.text = self._comment + + parameters = ElementTree.SubElement(root, 'parameters') + for param in self._parameters: + param_elem = ElementTree.SubElement(parameters, 'parameter') + self._set_element_atrib(param_elem, "name", param[0]) + self._set_element_atrib(param_elem, "value", param[1]) + + tags = ElementTree.SubElement(root, 'tags') + for tag in self._tags: + tag_elem = ElementTree.SubElement(tags, 'tag') + self._set_element_atrib(tag_elem, "name", tag) + + values = ElementTree.SubElement(root, 'values') + for value in self._values: + values.append(value.to_xml()) + + return root + +class PerfRepoValue(PerfRepoObject): + def __init__(self, xml=None): + if type(xml) is NoneType: + self._metricComparator = None + self._metricName = None + self._result = None + self._parameters = [] + elif type(xml) is StringType or isinstance(xml, Element): + if type(xml) is StringType: + root = ElementTree.fromstring(xml) + else: + root = xml + if root.tag != "value": + raise PerfRepoException("Invalid xml.") + + self._metricComparator = root.get("metricComparator") + self._metricName = root.get("metricName") + self._result = float(root.get("result")) + + self._parameters = [] + for param in root.find("parameters"): + if param.tag != "parameter": + continue + self._parameters.append((param.get("name"), param.get("value"))) + else: + raise PerfRepoException("Parameter xml must be"\ + " a string, an Element or None") + + def set_result(self, result): + self._result = result + + def set_comparator(self, comparator): + if comparator not in ["HB", "LB"]: + raise PerfRepoException("Comparator must be HB/LB.") + self._metricComparator = comparator + + def set_metricName(self, name): + self._metricName = name + + def add_parameter(self, name, value): + self._parameters.append((name, value)) + + def get_metricName(self): + return self._metricName + + def get_comparator(self): + return self._metricComparator + + def get_result(self): + return self._result + + def to_xml(self): + root = Element('value') + self._set_element_atrib(root, 'metricComparator', + self._metricComparator) + self._set_element_atrib(root, 'metricName', self._metricName) + self._set_element_atrib(root, 'result', str(self._result)) + + parameters = ElementTree.SubElement(root, 'parameters') + for param in self._parameters: + param_elem = ElementTree.SubElement(parameters, 'parameter') + self._set_element_atrib(param_elem, "name", param[0]) + self._set_element_atrib(param_elem, "value", param[1]) + return root + +class PerfRepoMetric(PerfRepoObject): + def __init__(self, xml=None): + if type(xml) is NoneType: + self._id = None + self._name = None + self._description = None + self._comparator = None + elif type(xml) is StringType or isinstance(xml, Element): + if type(xml) is StringType: + root = ElementTree.fromstring(xml) + else: + root = xml + if root.tag != "metric": + raise PerfRepoException("Invalid xml.") + + self._id = root.get("id") + self._name = root.get("name") + self._comparator = root.get("comparator") + self._description = root.find("description").text + else: + raise PerfRepoException("Parameter xml must be"\ + " a string, an Element or None") + + def get_id(self): + return self._id + + def get_name(self): + return self._id + + def get_description(self): + return self._description + + def get_comparator(self): + return self._comparator + + def set_id(self, id): + self._id = id + + def set_name(self, name): + self._name = name + + def set_description(self, description): + self._description = description + + def set_comparator(self, comparator): + if comapator not in ["HB", "LB"]: + raise PerfRepoException("Invalid comparator value.") + self._comparator = comparator + + def to_xml(self): + root = Element('metric') + self._set_element_atrib(root, 'id', self._id) + self._set_element_atrib(root, 'name', self._name) + description = ElementTree.SubElement(root, 'description') + description.text = self._description + self._set_element_atrib(root, 'comparator', self._comparator) + + return root + +class PerfRepoReport(PerfRepoObject): + def __init__(self, xml=None): + self._baselines = [] + if type(xml) is NoneType: + self._id = None + self._name = None + self._type = None + self._properties = {} + elif type(xml) is StringType or isinstance(xml, Element): + if type(xml) is StringType: + root = ElementTree.fromstring(xml) + else: + root = xml + if root.tag != "report": + raise PerfRepoException("Invalid xml.") + + self._id = root.get("id") + self._name = root.get("name") + self._type = root.get("type") + self._properties = {} + for entry in root.find("properties"): + if entry.tag != "entry": + continue + key_tag = entry.find("key") + value_tag = entry.find("value") + tmp_dict = dot_to_dict(value_tag.get("name"), + value_tag.get("value")) + recursive_dict_update(self._properties, tmp_dict) + + self._parse_baseline_properties() + else: + raise PerfRepoException("Parameter xml must be"\ + " a string, an Element or None") + + def _parse_baseline_properties(self): + chart = self._properties["chart0"] #TODO more charts per report? + baseline_re = re.compile(r'baseline(\d+)') + for key, value in chart.iteritems(): + match = baseline_re.match(key) + if match is None: + continue + self._baselines.append(value) + + def get_baseline(self, index=-1): + return self._baselines[index] + +class PerfRepoRESTAPI(object): + '''Wrapper class for the REST API provided by PerfRepo''' + def __init__(self, url, user, password): + self._url = url + self._user = user + self._password = password + + self._session = requests.Session() + self._session.auth = (self._user, self._password) + self._session.stream = True + self._session.headers['Content-Type'] = 'text/xml' + + def test_get_by_id(self, test_id): + get_url = self._url + '/rest/test/id/%s' % test_id + response = self._session.get(get_url) + if response.status_code != 200: + return None + else: + return PerfRepoTest(response.content) + + def test_get_by_uid(self, test_uid): + get_url = self._url + '/rest/test/uid/%s' % test_uid + response = self._session.get(get_url) + if response.status_code != 200: + return None + else: + return PerfRepoTest(response.content) + + def test_create(self, test): + post_url = self._url + '/rest/test/create' + response = self._session.post(post_url, data=test.to_xml_string()) + if response.status_code != 201: + return None + else: + test.set_id(response.headers['Location']) + return test + + def test_add_metric(self, test_id, metric): + post_url = self._url + '/rest/test/id/%s/addMetric' % test_id + response = self._session.post(post_url, data=metric.to_xml_string) + if response.status_code != 201: + return None + else: + metric.set_id(response.headers['Location']) + return metric + + def test_remove(self, test_id): + delete_url = self._url + '/rest/test/id/%s' % test_id + response = self._session.delete(delete_url) + if response.status_code != 204: + return False + else: + return True + + def metric_get(self, metric_id): + get_url = self._url + '/rest/metric/%s' % metric_id + response = self._session.get(get_url) + if response.status_code != 200: + return None + else: + return PerfRepoMetric(response.content) + + def testExecution_get(self, testExec_id): + get_url = self._url + '/rest/testExecution/%s' % testExec_id + response = self._session.get(get_url) + if response.status_code != 200: + return None + else: + return PerfRepoTestExecution(response.content) + + def testExecution_create(self, testExec): + post_url = self._url + '/rest/testExecution/create' + response = self._session.post(post_url, data=testExec.to_xml_string()) + if response.status_code != 201: + return None + else: + testExec.set_id(response.headers['Location']) + return testExec + + def testExecution_delete(self, testExec_id): + delete_url = self._url + '/rest/testExecution/%s' % testExec_id + response = self._session.delete(delete_url) + if response.status_code != 204: + return False + else: + return True + + def testExecution_add_value(self, value): + post_url = self._url + '/rest/testExecution/addValue' + #TODO + return self._session.post(post_url, data=value) + + def testExecution_get_attachment(self, attachment_id): + get_url = self._url + '/rest/testExecution/attachment/%s' % \ + attachment_id + #TODO + return self._session.get(get_url) + + def testExecution_add_attachment(self, testExec_id, attachment): + post_url = self._url + '/rest/testExecution/%s/addAttachment' % \ + testExec_id + #TODO + return self._session.post(post_url, data=attachment) + + def report_get_by_id(self, report_id): + get_url = self._url + '/rest/report/id/%s' % report_id + response = self._session.get(get_url) + if response.status_code != 200: + return None + else: + return PerfRepoReport(response.content) + + def report_create(self, report): + #TODO not needed yet and therefore not tested + post_url = self._url + '/rest/report/create' + self._session.post(post_url, data=report) + return None + + def report_delete(self, report_id): + #TODO not needed yet and therefore not tested + delete_url = self._url + '/rest/report/delete' + self._session.delete(post_url) + return None
From: Ondrej Lichtner olichtne@redhat.com
This commit extends the TaskAPI to support PerfRepo.
First of all we add 2 new classes: PerfRepoAPI and PerfRepoResult. These implement a simple API that will be used from python tasks to interact with PerfRepo.
With that the ControllerAPI class was extended to contain a reference to a PerfRepoAPI object a method to create a connection to a PerfRepo instance and two helper functions for retrieving configuration and mapping information.
An example python task work flow would look something like this:
perf_api = ctl.connect_PerfRepo() result = perf_api.new_result(tet_name, test_run_name) result.add_value(metric_name, performance_result) result.set_tags(list_of_string_tags) perf_api.save_result(result) perf_api.compare_to_baseline(result, report_id_with_a_baseline, metric_name)
It is important to know that the compare_to_baseline method expects that you're always using 3 values to represent a performance result: metric_name - the median or average result metric_name_min - metric_name - deviation metric_name_max - metric_name + deviation
The method returns True if the new result is better than the current baseline and False if it's worse. The criterium for being worse is that the two intervals don't have an intersection and "higher is better" or "lower is better" criterium is stored in Perfrepo.
This might be changed later but it's what we're going with for now.
Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- lnst/Controller/Task.py | 127 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 127 insertions(+)
diff --git a/lnst/Controller/Task.py b/lnst/Controller/Task.py index 0aa3d1a..be6b5cc 100644 --- a/lnst/Controller/Task.py +++ b/lnst/Controller/Task.py @@ -10,6 +10,11 @@ __author__ = """ rpazdera@redhat.com (Radek Pazdera) """
+from lnst.Controller.PerfRepo import PerfRepoRESTAPI +from lnst.Controller.PerfRepo import PerfRepoTestExecution +from lnst.Controller.PerfRepo import PerfRepoValue +from lnst.Common.Utils import dot_to_dict, dict_to_dot, list_to_dot + # The handle to be imported from each task ctl = None
@@ -22,6 +27,8 @@ class ControllerAPI(object): self._ctl = ctl self._result = True
+ self._perf_repo_api = PerfRepoAPI() + self._hosts = {} for host_id, host in hosts.iteritems(): self._hosts[host_id] = HostAPI(self, host_id, host) @@ -95,6 +102,32 @@ class ControllerAPI(object): """ return self._ctl._get_alias(alias)
+ def connect_PerfRepo(self, hostname, username, password): + if not self._perf_repo_api.connected(): + self._perf_repo_api.connect(hostname, username, password) + return self._perf_repo_api + + def get_configuration(self): + return self._ctl._recipe["machines"] + + def get_mapping(self): + match = self._ctl.get_pool_match() + mapping = [] + for m_id, m in match["machines"].iteritems(): + machine = {} + machine["id"] = m_id + machine["pool_id"] = m["target"] + machine["hostname"] = m["hostname"] + machine["interface"] = [] + for i_id, i in m["interfaces"].iteritems(): + interface = {} + interface["id"] = i_id + interface["pool_id"] = i["target"] + interface["hwaddr"] = i["hwaddr"] + machine["interface"].append(interface) + mapping.append(machine) + return mapping + class HostAPI(object): """ An API class representing a host machine. """
@@ -413,3 +446,97 @@ class VolatileValue(object):
def __str__(self): return str(self.get_val()) + +class PerfRepoAPI(object): + def __init__(self): + self._rest_api = None + + def connected(self): + return self._rest_api is not None + + def connect(self, hostname, username, password): + self._rest_api = PerfRepoRESTAPI(hostname, username, password) + + def new_result(self, testUid, name): + result = PerfRepoResult(testUid, name) + return result + + def save_result(self, result): + if not isinstance(result, PerfRepoResult): + raise TaskError("Parameter result must be an instance "\ + "of PerfRepoResult") + elif self._rest_api is None: + raise TaskError("Not connected to PerfRepo.") + else: + self._rest_api.testExecution_create(result.get_testExecution()) + + def get_baseline(self, report_id): + report = self._rest_api.report_get_by_id(report_id) + return report.get_baseline() + + def compare_to_baseline(self, result, report_id, metric_name): + baseline = self.get_baseline(report_id) + baseline_exec_id = baseline["execId"] + baseline_testExec = self._rest_api.testExecution_get(baseline_exec_id) + result_testExec = result.get_testExecution() + + return self.compare_testExecutions(result_testExec, + baseline_testExec, + metric_name) + + def compare_testExecutions(self, first, second, metric_name): + first_value = first.get_value(metric_name) + first_min = first.get_value(metric_name + "_min") + first_max = first.get_value(metric_name + "_max") + + second_value = second.get_value(metric_name) + second_min = second.get_value(metric_name + "_min") + second_max = second.get_value(metric_name + "_max") + + comp = second_value.get_comparator() + print second_min.get_result(), first_max.get_result() + if comp == "HB": + if second_min.get_result() > first_max.get_result(): + return False + return True + elif comp == "LB": + if first_min.get_result() > second_max.get_result(): + return False + return True + else: + return False + return False + +class PerfRepoResult(object): + def __init__(self, testUid, name): + self._testExecution = PerfRepoTestExecution() + self._testExecution.set_testUid(testUid) + self._testExecution.set_name(name) + self.set_configuration(ctl.get_configuration()) + self.set_mapping(ctl.get_mapping()) + + def add_value(self, val_name, value): + perf_value = PerfRepoValue() + perf_value.set_metricName(val_name) + perf_value.set_result(value) + + self._testExecution.add_value(perf_value) + + def set_configuration(self, configuration=None): + if configuration is None: + configuration = ctl.get_configuration() + for pair in list_to_dot(configuration, "configuration.", "machine"): + self._testExecution.add_parameter(pair[0], pair[1]) + + def set_mapping(self, mapping=None): + if mapping is None: + mapping = ctl.get_mapping() + for pair in list_to_dot(mapping, "mapping.", "machine"): + self._testExecution.add_parameter(pair[0], pair[1]) + + def set_tags(self, tags): + for tag in tags: + self._testExecution.add_tag(tag) + + def get_testExecution(self): + return self._testExecution
Fri, Apr 17, 2015 at 02:56:13PM CEST, olichtne@redhat.com wrote:
From: Ondrej Lichtner olichtne@redhat.com
The following set of patches implements basic support for talking to PerfRepo. There still are some placeholders for functions that are not important for now and bugs are to be expected.
There is a short example work flow example in the last patch of the set but you still need to have a PerfRepo instance available to try everything out, I'll be sending a follow-up email to interested parties that will explain in more detail how to try this out.
Any comments and ideas are always welcome.
Ondrej Lichtner (4): Utils: add utility dictionary functions SlavePool: add hostname and hwaddr to match info add lnst/Controller/PerfRepo.py Task: LNST integration with PerfRepo
lnst/Common/Utils.py | 52 +++ lnst/Controller/NetTestController.py | 3 +- lnst/Controller/NetTestResultSerializer.py | 3 +- lnst/Controller/PerfRepo.py | 532 +++++++++++++++++++++++++++++ lnst/Controller/SlavePool.py | 17 +- lnst/Controller/Task.py | 127 +++++++ 6 files changed, 728 insertions(+), 6 deletions(-) create mode 100644 lnst/Controller/PerfRepo.py
patchset applied. Thanks!
lnst-developers@lists.fedorahosted.org
-
Jiri Pirko -
Ondrej Lichtner