LNST-developers August 2020

lnst-developers@lists.fedorahosted.org

4 participants
9 discussions

[DISCUSSION] python version requirement
by Ondrej Lichtner 06 Apr '21

06 Apr '21

Hi all, since we've moved to python3 that is actively developed and versions move between various long term/short term support cycles, we should also adapt LNST to this cycle of updating which minimal version of python LNST requires. TL;DR: The main questions I'm asking are: * how do we _implement_ a python version requirement? * how do we _upgrade_/_migrate_ in the future? * how do we _document_ a python version requirement? * which version do we want to use _now_? More context: I think at the moment we have a "soft" requirement for python3.6. Soft because we: * probably haven't tested on anything older * it's not explicitly configured/documented anywhere At the same time, there are now at least two reasons to start thinking about moving to python3.8: * I remember Perry asking about the f-string feature introduced in 3.8 * while working with Adrian on the TRex refactoring I started thinking about a feature for the lnst.Tests package that I've had in mind for a while, which requires a 3.7 feature * python3.8 is the current version on Fedora32, is available in RHEL8 (via dnf install python38), and python3.7 was skipped The lnst.Tests feature I'm thinking of is "lazy" and "dynamic" loading of BaseTestModule derived modules - for example at the moment, if a Recipe imports any module from lnst.Tests (e.g. lnst.Tests.Ping), the entire package is parsed and "loaded", which means that the python environment will also parse and load lnst.Tests.TRex. This means that a basic hello world recipe that simply calls Ping, will in some way require load time dependencies of TRex. The "lazy" and "dynamic" loading of test modules would ensure that when a recipe calls: from lnst.Tests import Ping Only the Ping module will be parsed, loaded and imported, and nothing else. And the dynamicity here could mean that we could be able to extend test modules exported by the lnst.Tests package via the lnst-ctl config file, for example for user/tester implemented test modules that are not tracked in the main lnst repository. I wrote a rough patch to experiment with this: --- diff --git a/lnst/Tests/__init__.py b/lnst/Tests/__init__.py index f7c6c90..a39b6f4 100644 --- a/lnst/Tests/__init__.py +++ b/lnst/Tests/__init__.py @@ -12,8 +12,26 @@ olichtne(a)redhat.com (Ondrej Lichtner) """ -from lnst.Tests.Ping import Ping -from lnst.Tests.PacketAssert import PacketAssert -from lnst.Tests.Iperf import IperfClient, IperfServer +# from lnst.Tests.Ping import Ping +# from lnst.Tests.PacketAssert import PacketAssert +# from lnst.Tests.Iperf import IperfClient, IperfServer +import importlib + +lazy_load_modules = { + "Ping": "lnst.Tests.Ping", + "PacketAssert": "lnst.Tests.PacketAssert", + "IperfClient": "lnst.Tests.Iperf", + "IperfServer": "lnst.Tests.Iperf", +} + + +def __getattr__(name): + if name not in lazy_load_modules: + raise ImportError("Cannot import {}".format(name)) + mod = importlib.import_module(lazy_load_modules[name]) + globals()[name] = getattr(mod, name) + return globals()[name] + + +# #TODO add support for test classes from lnst-ctl.conf -#TODO add support for test classes from lnst-ctl.conf --- However this requires the ability to define __getattr__ for a module, which is introduced as a python3.7 feature via PEP562 [0]. -Ondrej [0] https://www.python.org/dev/peps/pep-0562/

2 3

[PATCH 1/2] Add support for exporting recipe run data
by pgagne＠redhat.com 31 Aug '20

31 Aug '20

From: Perry Gagne <pgagne(a)redhat.com> Controller.py: Add log_dir to RecipeRun call. Job.py: Added __get_state__ to control what gets pickled during Recipe Run export. Remove netns since it contains a lot of stuff that cant be easily exported. Recipe.py: Added log_dir paramter to RecipeRun in order to more centrally track it so it can be used for exporting. RecipeResults.py: Added __getstate__ to DeviceConfigResult to remove _device during pickling. RecipeRunExport.py: Add RecipeRunExporter and RecipeRunData and related stuff to be used for exporting. Signed-off-by: Perry Gagne <pgagne(a)redhat.com> --- lnst/Controller/Controller.py | 2 +- lnst/Controller/Job.py | 7 ++ lnst/Controller/Recipe.py | 7 +- lnst/Controller/RecipeResults.py | 5 ++ lnst/Controller/RecipeRunExport.py | 110 +++++++++++++++++++++++++++++ 5 files changed, 129 insertions(+), 2 deletions(-) create mode 100644 lnst/Controller/RecipeRunExport.py diff --git a/lnst/Controller/Controller.py b/lnst/Controller/Controller.py index 160e8b7..1641ed3 100644 --- a/lnst/Controller/Controller.py +++ b/lnst/Controller/Controller.py @@ -156,7 +156,7 @@ class Controller(object): logging.info(line) try: self._map_match(match, req, recipe) - recipe._init_run(RecipeRun(match)) + recipe._init_run(RecipeRun(match, log_dir=self._log_ctl.get_recipe_log_path())) recipe.test() except Exception as exc: logging.error("Recipe execution terminated by unexpected exception") diff --git a/lnst/Controller/Job.py b/lnst/Controller/Job.py index 1ec85e3..d0757f6 100644 --- a/lnst/Controller/Job.py +++ b/lnst/Controller/Job.py @@ -230,3 +230,10 @@ class Job(object): attrs.append(repr(self._what)) return ", ".join(attrs) + + def __getstate__(self): + #Remove things that can't be pickled + #TODO figure out better place holder values + state = self.__dict__.copy() + state['_netns'] = None + return state diff --git a/lnst/Controller/Recipe.py b/lnst/Controller/Recipe.py index 26e0737..a9f7514 100644 --- a/lnst/Controller/Recipe.py +++ b/lnst/Controller/Recipe.py @@ -152,10 +152,11 @@ class BaseRecipe(object): level, data_level)) class RecipeRun(object): - def __init__(self, match, desc=None): + def __init__(self, match, desc=None, log_dir=None): self._match = match self._desc = desc self._results = [] + self._log_dir = log_dir def add_result(self, result): if not isinstance(result, BaseResult): @@ -176,6 +177,10 @@ class RecipeRun(object): logging.info("Result: {}, What:".format(result_str)) logging.info("{}".format(result.description)) + @property + def log_dir(self): + return self._log_dir + @property def match(self): return self._match diff --git a/lnst/Controller/RecipeResults.py b/lnst/Controller/RecipeResults.py index 38a4170..941ea8d 100644 --- a/lnst/Controller/RecipeResults.py +++ b/lnst/Controller/RecipeResults.py @@ -118,6 +118,11 @@ class DeviceConfigResult(BaseResult): def device(self): return self._device + def __getstate__(self): + state = self.__dict__.copy() + # Remove things that can't be pickled + state['_device'] = None + return state class DeviceCreateResult(DeviceConfigResult): @property diff --git a/lnst/Controller/RecipeRunExport.py b/lnst/Controller/RecipeRunExport.py new file mode 100644 index 0000000..b0ecccb --- /dev/null +++ b/lnst/Controller/RecipeRunExport.py @@ -0,0 +1,110 @@ +import datetime +import pickle +import os +import logging +from typing import List, Tuple +from lnst.Controller.Recipe import BaseRecipe, RecipeRun + + +class RecipeRunData: + """ + Class used to encapsulate a RecipeRun, this is the object + that will be pickled and output to a file. + + :param recipe_cls: + class of the Recipe. We do not currently pickle the instance + of the recipe itself for ease of exporting. + :type recipe_cls: :py:class: `lnst.Controller.Recipe.BaseRecipe` + + :param param: Copy of Recipe parameters. + :type param: dict + + :param req: Copy of Recipe requirements + :type req: dict + + :param environ: A copy of `os.environ` created when the object is instantiated. + :type environ: dict + + :param run: :py:class:`lnst.Controller.Recipe.RecipeRun` instance of the run + :type run: :py:class:`lnst.Controller.Recipe.RecipeRun` + + :param datetime: A time stamp that is the result of running `datetime.datetime.now()` during instantiation + :type datetime: :py:class:`datetime.datetime` + """ + + def __init__(self, recipe: BaseRecipe, run: RecipeRun): + self.recipe_cls = recipe.__class__ + self.params = recipe.params._to_dict() + self.req = recipe.req._to_dict() + self.environ = os.environ.copy() + self.run = run + self.datetime = datetime.datetime.now() + + +class RecipeRunExporter: + """ + Class used to export recipe runs. + + """ + + def __init__(self, recipe: BaseRecipe): + """ + + :param recipe: Recipe + :type recipe: :py:class: `lnst.Controller.Recipe.BaseRecipe` + """ + self.recipe = recipe + self.recipe_name = self.recipe.__class__.__name__ + + def export_run(self, run: RecipeRun, dir=None, name=None) -> str: + """ + + :param run: The RecipeRun to export + :type run: :py:class:`lnst.Controller.Recipe.RecipeRun` + :param dir: The path to directory to export to. Does not include file name, defaults to `run.log_dir` + :type dir: str + :param name: Name of file to export. Default `<recipename>-run-<timestamp>.dat' + :type name: str + :return: Path (dir+filename) of exported run. + :rtype:str + """ + data = RecipeRunData(self.recipe, run) + if not name: + name = f"{self.recipe_name}-run-{run.datetime:%Y-%m-%d_%H:%M:%S}.dat" + if not dir: + dir = os.path.join(run.log_dir, name) + + with open(dir, 'wb') as f: + pickle.dump(data, f) + + logging.info(f"Exported {self.recipe_name} data to {dir}") + return dir + + +def export_recipe_runs(recipe: BaseRecipe) -> List[Tuple[str, RecipeRun]]: + """ + Helper method that exports all runs in a recipe. + :param recipe: Recipe to export + :type recipe: :py:class: `lnst.Controller.Recipe.BaseRecipe` + :return: list of files that contain exported recipe runs + :rtype: list + """ + exporter = RecipeRunExporter(recipe) + files = [] + for run in recipe.runs: + path = exporter.export_run(run) + files.append((run, path)) + return files + + +def import_recipe_run(path: str) -> RecipeRunData: + """ + Import recipe runs that have been exported using :py:class:`lnst.Controller.RecipeRunExport.RecipeRunExporter` + :param path: path to file to import + :type path: str + :return: `RecipeRun` object containing the run and other metadata. + :rtype: :py:class:`lnst.Controller.RecipeRunExport.RecipeRunData` + """ + with open(path, 'rb') as f: + data = pickle.load(f) + return data -- 2.26.2

3 3

[PATCH] Add support for exporting recipe run data
by pgagne＠redhat.com 27 Aug '20

27 Aug '20

From: Perry Gagne <pgagne(a)redhat.com> Controller.py: Add log_dir to RecipeRun call. Job.py: Added __get_state__ to control what gets pickled during Recipe Run export. Remove netns since it contains a lot of stuff that cant be easily exported. Recipe.py: Added log_dir paramter to RecipeRun in order to more centrally track it so it can be used for exporting. RecipeResults.py: Added __getstate__ to DeviceConfigResult to remove _device during pickling. RecipeRunExport.py: Add RecipeRunExporter and RecipeRunData and related stuff to be used for exporting. Signed-off-by: Perry Gagne <pgagne(a)redhat.com> --- lnst/Controller/Controller.py | 2 +- lnst/Controller/Job.py | 7 ++ lnst/Controller/Recipe.py | 7 +- lnst/Controller/RecipeResults.py | 5 ++ lnst/Controller/RecipeRunExport.py | 110 +++++++++++++++++++++++++++++ 5 files changed, 129 insertions(+), 2 deletions(-) create mode 100644 lnst/Controller/RecipeRunExport.py diff --git a/lnst/Controller/Controller.py b/lnst/Controller/Controller.py index 160e8b7..1641ed3 100644 --- a/lnst/Controller/Controller.py +++ b/lnst/Controller/Controller.py @@ -156,7 +156,7 @@ class Controller(object): logging.info(line) try: self._map_match(match, req, recipe) - recipe._init_run(RecipeRun(match)) + recipe._init_run(RecipeRun(match, log_dir=self._log_ctl.get_recipe_log_path())) recipe.test() except Exception as exc: logging.error("Recipe execution terminated by unexpected exception") diff --git a/lnst/Controller/Job.py b/lnst/Controller/Job.py index 1ec85e3..d0757f6 100644 --- a/lnst/Controller/Job.py +++ b/lnst/Controller/Job.py @@ -230,3 +230,10 @@ class Job(object): attrs.append(repr(self._what)) return ", ".join(attrs) + + def __getstate__(self): + #Remove things that can't be pickled + #TODO figure out better place holder values + state = self.__dict__.copy() + state['_netns'] = None + return state diff --git a/lnst/Controller/Recipe.py b/lnst/Controller/Recipe.py index 26e0737..a9f7514 100644 --- a/lnst/Controller/Recipe.py +++ b/lnst/Controller/Recipe.py @@ -152,10 +152,11 @@ class BaseRecipe(object): level, data_level)) class RecipeRun(object): - def __init__(self, match, desc=None): + def __init__(self, match, desc=None, log_dir=None): self._match = match self._desc = desc self._results = [] + self._log_dir = log_dir def add_result(self, result): if not isinstance(result, BaseResult): @@ -176,6 +177,10 @@ class RecipeRun(object): logging.info("Result: {}, What:".format(result_str)) logging.info("{}".format(result.description)) + @property + def log_dir(self): + return self._log_dir + @property def match(self): return self._match diff --git a/lnst/Controller/RecipeResults.py b/lnst/Controller/RecipeResults.py index 38a4170..941ea8d 100644 --- a/lnst/Controller/RecipeResults.py +++ b/lnst/Controller/RecipeResults.py @@ -118,6 +118,11 @@ class DeviceConfigResult(BaseResult): def device(self): return self._device + def __getstate__(self): + state = self.__dict__.copy() + # Remove things that can't be pickled + state['_device'] = None + return state class DeviceCreateResult(DeviceConfigResult): @property diff --git a/lnst/Controller/RecipeRunExport.py b/lnst/Controller/RecipeRunExport.py new file mode 100644 index 0000000..b0ecccb --- /dev/null +++ b/lnst/Controller/RecipeRunExport.py @@ -0,0 +1,110 @@ +import datetime +import pickle +import os +import logging +from typing import List, Tuple +from lnst.Controller.Recipe import BaseRecipe, RecipeRun + + +class RecipeRunData: + """ + Class used to encapsulate a RecipeRun, this is the object + that will be pickled and output to a file. + + :param recipe_cls: + class of the Recipe. We do not currently pickle the instance + of the recipe itself for ease of exporting. + :type recipe_cls: :py:class: `lnst.Controller.Recipe.BaseRecipe` + + :param param: Copy of Recipe parameters. + :type param: dict + + :param req: Copy of Recipe requirements + :type req: dict + + :param environ: A copy of `os.environ` created when the object is instantiated. + :type environ: dict + + :param run: :py:class:`lnst.Controller.Recipe.RecipeRun` instance of the run + :type run: :py:class:`lnst.Controller.Recipe.RecipeRun` + + :param datetime: A time stamp that is the result of running `datetime.datetime.now()` during instantiation + :type datetime: :py:class:`datetime.datetime` + """ + + def __init__(self, recipe: BaseRecipe, run: RecipeRun): + self.recipe_cls = recipe.__class__ + self.params = recipe.params._to_dict() + self.req = recipe.req._to_dict() + self.environ = os.environ.copy() + self.run = run + self.datetime = datetime.datetime.now() + + +class RecipeRunExporter: + """ + Class used to export recipe runs. + + """ + + def __init__(self, recipe: BaseRecipe): + """ + + :param recipe: Recipe + :type recipe: :py:class: `lnst.Controller.Recipe.BaseRecipe` + """ + self.recipe = recipe + self.recipe_name = self.recipe.__class__.__name__ + + def export_run(self, run: RecipeRun, dir=None, name=None) -> str: + """ + + :param run: The RecipeRun to export + :type run: :py:class:`lnst.Controller.Recipe.RecipeRun` + :param dir: The path to directory to export to. Does not include file name, defaults to `run.log_dir` + :type dir: str + :param name: Name of file to export. Default `<recipename>-run-<timestamp>.dat' + :type name: str + :return: Path (dir+filename) of exported run. + :rtype:str + """ + data = RecipeRunData(self.recipe, run) + if not name: + name = f"{self.recipe_name}-run-{data.datetime:%Y-%m-%d_%H:%M:%S}.dat" + if not dir: + dir = os.path.join(run.log_dir, name) + + with open(dir, 'wb') as f: + pickle.dump(data, f) + + logging.info(f"Exported {self.recipe_name} data to {dir}") + return dir + + +def export_recipe_runs(recipe: BaseRecipe) -> List[Tuple[str, RecipeRun]]: + """ + Helper method that exports all runs in a recipe. + :param recipe: Recipe to export + :type recipe: :py:class: `lnst.Controller.Recipe.BaseRecipe` + :return: list of files that contain exported recipe runs + :rtype: list + """ + exporter = RecipeRunExporter(recipe) + files = [] + for run in recipe.runs: + path = exporter.export_run(run) + files.append((run, path)) + return files + + +def import_recipe_run(path: str) -> RecipeRunData: + """ + Import recipe runs that have been exported using :py:class:`lnst.Controller.RecipeRunExport.RecipeRunExporter` + :param path: path to file to import + :type path: str + :return: `RecipeRun` object containing the run and other metadata. + :rtype: :py:class:`lnst.Controller.RecipeRunExport.RecipeRunData` + """ + with open(path, 'rb') as f: + data = pickle.load(f) + return data -- 2.26.2

2 1

[PATCH] Add support for exporting recipe run data
by pgagne＠redhat.com 27 Aug '20

27 Aug '20

2 1

[PATCH] Recipes.ENRT.DoubleTeamRecipe: remove reversed perf test endpoints
by Jan Tluka 27 Aug '20

27 Aug '20

This scenario is a mirrored setup. There's no real value in running performance test in both directions. If the test requires testing of reversed scenario it should inherit from PerfReversibleFlowMixin class instead. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Recipes/ENRT/DoubleTeamRecipe.py | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lnst/Recipes/ENRT/DoubleTeamRecipe.py b/lnst/Recipes/ENRT/DoubleTeamRecipe.py index 39e069e5..305ffdeb 100644 --- a/lnst/Recipes/ENRT/DoubleTeamRecipe.py +++ b/lnst/Recipes/ENRT/DoubleTeamRecipe.py @@ -92,8 +92,7 @@ class DoubleTeamRecipe(CommonHWSubConfigMixin, OffloadSubConfigMixin, ] def generate_perf_endpoints(self, config): - return [(self.matched.host1.team0, self.matched.host2.team0), - (self.matched.host2.team0, self.matched.host1.team0)] + return [(self.matched.host1.team0, self.matched.host2.team0)] @property def offload_nics(self): -- 2.21.3

1 3

[PATCH] Recipes.ENRT.VlansOverTeamRecipe: extend the class with PerfReversibleFlow mixin
by Jan Tluka 27 Aug '20

27 Aug '20

Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Recipes/ENRT/VlansOverTeamRecipe.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lnst/Recipes/ENRT/VlansOverTeamRecipe.py b/lnst/Recipes/ENRT/VlansOverTeamRecipe.py index ed1e6d53..c9fd8dae 100644 --- a/lnst/Recipes/ENRT/VlansOverTeamRecipe.py +++ b/lnst/Recipes/ENRT/VlansOverTeamRecipe.py @@ -6,6 +6,8 @@ from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( OffloadSubConfigMixin) from lnst.Recipes.ENRT.ConfigMixins.CommonHWSubConfigMixin import ( CommonHWSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.PerfReversibleFlowMixin import ( + PerfReversibleFlowMixin) from lnst.Recipes.ENRT.PingMixins import VlanPingEvaluatorMixin from lnst.RecipeCommon.Ping.PingEndpoints import PingEndpoints from lnst.Devices import VlanDevice @@ -13,7 +15,7 @@ from lnst.Devices.VlanDevice import VlanDevice as Vlan from lnst.Devices import TeamDevice from lnst.Recipes.ENRT.PingMixins import VlanPingEvaluatorMixin -class VlansOverTeamRecipe(VlanPingEvaluatorMixin, +class VlansOverTeamRecipe(PerfReversibleFlowMixin, VlanPingEvaluatorMixin, CommonHWSubConfigMixin, OffloadSubConfigMixin, BaseEnrtRecipe): host1 = HostReq() -- 2.21.3

1 1

[PATCH 1/6] Recipes.ENRT.BasePvPRecipe: fix streams parameter typo
by olichtne＠redhat.com 24 Aug '20

24 Aug '20

From: Ondrej Lichtner <olichtne(a)redhat.com> Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Recipes/ENRT/BasePvPRecipe.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lnst/Recipes/ENRT/BasePvPRecipe.py b/lnst/Recipes/ENRT/BasePvPRecipe.py index 89e4293..690c60a 100644 --- a/lnst/Recipes/ENRT/BasePvPRecipe.py +++ b/lnst/Recipes/ENRT/BasePvPRecipe.py @@ -75,7 +75,7 @@ class BasePvPRecipe(PingTestAndEvaluate, PerfRecipe): perf_duration = IntParam(default=60) perf_iterations = IntParam(default=5) perf_msg_size = IntParam(default=64) - perf_streams = IntParam(default=1) + perf_parallel_streams = IntParam(default=1) nr_hugepages = IntParam(default=13000) # TODO: Allow 1G hugepages as well -- 2.28.0

4 8

[PATCH v2 1/2] Recipes.ENRT.TeamVsBondRecipe: extend the recipe with PerfReversibleFlowMixin
by Jan Tluka 13 Aug '20

13 Aug '20

Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Recipes/ENRT/TeamVsBondRecipe.py | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/lnst/Recipes/ENRT/TeamVsBondRecipe.py b/lnst/Recipes/ENRT/TeamVsBondRecipe.py index 515d2ea9..a617473c 100644 --- a/lnst/Recipes/ENRT/TeamVsBondRecipe.py +++ b/lnst/Recipes/ENRT/TeamVsBondRecipe.py @@ -6,12 +6,14 @@ from lnst.Recipes.ENRT.ConfigMixins.OffloadSubConfigMixin import ( OffloadSubConfigMixin) from lnst.Recipes.ENRT.ConfigMixins.CommonHWSubConfigMixin import ( CommonHWSubConfigMixin) +from lnst.Recipes.ENRT.ConfigMixins.PerfReversibleFlowMixin import ( + PerfReversibleFlowMixin) from lnst.RecipeCommon.Ping.PingEndpoints import PingEndpoints from lnst.Devices import TeamDevice from lnst.Devices import BondDevice -class TeamVsBondRecipe(CommonHWSubConfigMixin, OffloadSubConfigMixin, - BaseEnrtRecipe): +class TeamVsBondRecipe(PerfReversibleFlowMixin, CommonHWSubConfigMixin, + OffloadSubConfigMixin, BaseEnrtRecipe): host1 = HostReq() host1.eth0 = DeviceReq(label="tnet", driver=RecipeParam("driver")) host1.eth1 = DeviceReq(label="tnet", driver=RecipeParam("driver")) @@ -108,8 +110,7 @@ class TeamVsBondRecipe(CommonHWSubConfigMixin, OffloadSubConfigMixin, ] def generate_perf_endpoints(self, config): - return [(self.matched.host1.team0, self.matched.host2.bond0), - (self.matched.host2.bond0, self.matched.host1.team0)] + return [(self.matched.host1.team0, self.matched.host2.bond0)] @property def offload_nics(self): -- 2.21.3

1 2

[PATCH] Recipes.ENRT.TeamVsBondRecipe: extend the recipe with PerfReversibleFlowMixin
by Jan Tluka 12 Aug '20

12 Aug '20

1 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers August 2020