Fri, Dec 18, 2020 at 02:21:23PM CET, olichtne(a)redhat.com wrote:
>From: Ondrej Lichtner <olichtne(a)redhat.com>
>
>Splitting off the perf measurement generation methods into a separate
>smaller package.
>
>This should make it easier to skip and define more custom behaviours for
>the various groups of recipes that we have. To provide more context -
>many of our recipes are baremetal and share the same characteristics
>wrt. flow generation, however for ovs+dpdk recipes or for virtual
>recipes, or for ipsec based recipes we need different groups of
>behaviours. These still share the same basic interface so some form of
>class inheritance is valid.
>
>These will replace the BaseEnrt generation mechanism.
>
>For now this commit adds 3 main generator types:
>* IperfMeasurementGenerator that generates iperf measurements using a
> flow generation algorithm so that multiple combinations of
> ip_protocol*transport_protocol flows are created for each provided
> flow endpoint pair
>* FlowEndpointsStatCPUMeasurementGenerator that generates a cpu
> measurement based on the endpoints of any flow measurements generated
> before this
>* HypervisorsStatCPUMeasurementGenerator that generates a cpu
> measurement based on a "hypervisors" property that should return the
> list of hypervisor hosts in the recipe, this is intended to be used
> with recipes that deal with virtual machines
>
>Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com>
>---
> .../BaseMeasurementGenerator.py | 3 +
> ...lowEndpointsStatCPUMeasurementGenerator.py | 26 ++++
> .../HypervisorsStatCPUMeasurementGenerator.py | 19 +++
> .../IperfMeasurementGenerator.py | 146 ++++++++++++++++++
> .../ENRT/MeasurementGenerators/__init__.py | 0
> 5 files changed, 194 insertions(+)
> create mode 100644
lnst/Recipes/ENRT/MeasurementGenerators/BaseMeasurementGenerator.py
> create mode 100644
lnst/Recipes/ENRT/MeasurementGenerators/FlowEndpointsStatCPUMeasurementGenerator.py
> create mode 100644
lnst/Recipes/ENRT/MeasurementGenerators/HypervisorsStatCPUMeasurementGenerator.py
> create mode 100644
lnst/Recipes/ENRT/MeasurementGenerators/IperfMeasurementGenerator.py
> create mode 100644 lnst/Recipes/ENRT/MeasurementGenerators/__init__.py
>
>diff --git a/lnst/Recipes/ENRT/MeasurementGenerators/BaseMeasurementGenerator.py
b/lnst/Recipes/ENRT/MeasurementGenerators/BaseMeasurementGenerator.py
>new file mode 100644
>index 0000000..c13bfb7
>--- /dev/null
>+++ b/lnst/Recipes/ENRT/MeasurementGenerators/BaseMeasurementGenerator.py
>@@ -0,0 +1,3 @@
>+class BaseMeasurementGenerator(object):
>+ def generate_perf_measurements_combinations(self, config):
>+ return []
I'd simply name this generate_measurements_combinations (remove the
"perf" from the name). Without removing it it makes me think this is
about the perf_tool measurements but when I look at the changes it's
shared by both the cpu and perf tool measurement classes.
I chose the name "generate_perf_measurements_combinations" because:
* the BaseMeasurement class is in the lnst.RecipeCommon.Perf.Measurements package
* the enrt recipes use "net_perf_tool" and "cpu_perf_tool"
parameters,
not just "perf_tool" parameter
But I get what you mean, I'll try and see how it looks or if it can be
improved in a different way.
-Ondrej
Otherwise looks ok.
>diff --git
a/lnst/Recipes/ENRT/MeasurementGenerators/FlowEndpointsStatCPUMeasurementGenerator.py
b/lnst/Recipes/ENRT/MeasurementGenerators/FlowEndpointsStatCPUMeasurementGenerator.py
>new file mode 100644
>index 0000000..7cc41bf
>--- /dev/null
>+++
b/lnst/Recipes/ENRT/MeasurementGenerators/FlowEndpointsStatCPUMeasurementGenerator.py
>@@ -0,0 +1,26 @@
>+from lnst.Common.Parameters import Param
>+
>+from lnst.RecipeCommon.Perf.Measurements.BaseFlowMeasurement import
BaseFlowMeasurement
>+from lnst.RecipeCommon.Perf.Measurements.StatCPUMeasurement import
StatCPUMeasurement
>+
>+from lnst.Recipes.ENRT.MeasurementGenerators.BaseMeasurementGenerator import
BaseMeasurementGenerator
>+
>+class FlowEndpointsStatCPUMeasurementGenerator(BaseMeasurementGenerator):
>+ cpu_perf_tool = Param(default=StatCPUMeasurement)
>+
>+ def generate_perf_measurements_combinations(self, config):
>+ combinations = super().generate_perf_measurements_combinations(config)
>+ for combination in combinations:
>+ cpu_measurement_hosts = self.extract_endpoints(config, combination)
>+ yield [
>+ self.params.cpu_perf_tool(cpu_measurement_hosts)
>+ ] + combination
>+
>+ def extract_endpoints(self, config, measurements):
>+ endpoints = set()
>+ for measurement in measurements:
>+ if isinstance(measurement, BaseFlowMeasurement):
>+ for flow in measurement.flows:
>+ endpoints.add(flow.generator)
>+ endpoints.add(flow.receiver)
>+ return endpoints
>diff --git
a/lnst/Recipes/ENRT/MeasurementGenerators/HypervisorsStatCPUMeasurementGenerator.py
b/lnst/Recipes/ENRT/MeasurementGenerators/HypervisorsStatCPUMeasurementGenerator.py
>new file mode 100644
>index 0000000..9029637
>--- /dev/null
>+++
b/lnst/Recipes/ENRT/MeasurementGenerators/HypervisorsStatCPUMeasurementGenerator.py
>@@ -0,0 +1,19 @@
>+from lnst.Common.Parameters import Param
>+
>+from lnst.RecipeCommon.Perf.Measurements.StatCPUMeasurement import
StatCPUMeasurement
>+
>+from lnst.Recipes.ENRT.MeasurementGenerators.BaseMeasurementGenerator import
BaseMeasurementGenerator
>+
>+class HypervisorsStatCPUMeasurementGenerator(BaseMeasurementGenerator):
>+ cpu_perf_tool = Param(default=StatCPUMeasurement)
>+
>+ def generate_perf_measurements_combinations(self, config):
>+ combinations = super().generate_perf_measurements_combinations(config)
>+ for combination in combinations:
>+ yield [
>+ self.params.cpu_perf_tool(self.hypervisor_hosts)
>+ ] + combination
>+
>+ @property
>+ def hypervisor_hosts(self):
>+ return set()
>diff --git a/lnst/Recipes/ENRT/MeasurementGenerators/IperfMeasurementGenerator.py
b/lnst/Recipes/ENRT/MeasurementGenerators/IperfMeasurementGenerator.py
>new file mode 100644
>index 0000000..4ccf922
>--- /dev/null
>+++ b/lnst/Recipes/ENRT/MeasurementGenerators/IperfMeasurementGenerator.py
>@@ -0,0 +1,146 @@
>+from lnst.Common.Parameters import (
>+ Param,
>+ IntParam,
>+ ListParam,
>+)
>+from lnst.Common.IpAddress import AF_INET, AF_INET6
>+
>+from lnst.RecipeCommon.Perf.Measurements import Flow as PerfFlow
>+from lnst.RecipeCommon.Perf.Measurements import IperfFlowMeasurement
>+
>+from lnst.Recipes.ENRT.MeasurementGenerators.BaseMeasurementGenerator import
BaseMeasurementGenerator
>+
>+class IperfMeasurementGenerator(BaseMeasurementGenerator):
>+ """
>+ :param perf_tests:
>+ Parameter used by the :any:`generate_flow_combinations` generator.
>+ Tells the generator what types of network flow measurements to generate
>+ perf test configurations for.
>+ :type perf_tests: Tuple[str] (default ("tcp_stream",
"udp_stream",
>+ "sctp_stream"))
>+
>+ :param perf_tool_cpu:
>+ Parameter used by the :any:`generate_flow_combinations` generator. To
>+ indicate that the flow measurement should be pinned to a specific CPU
>+ core.
>+ :type perf_tool_cpu: :any:`IntParam` (optional parameter)
>+
>+ :param perf_duration:
>+ Parameter used by the :any:`generate_perf_configurations` generator. To
>+ specify the duration of the performance measurements, in seconds.
>+ :type perf_duration: :any:`IntParam` (default 60)
>+
>+ :param perf_iterations:
>+ Parameter used by the :any:`generate_perf_configurations` generator. To
>+ specify how many times should each performance measurement be repeated
>+ to generate cumulative results which can be statistically analyzed.
>+ :type perf_iterations: :any:`IntParam` (default 5)
>+
>+ :param perf_parallel_streams:
>+ Parameter used by the :any:`generate_flow_combinations` generator. To
>+ specify how many parallel streams of the same network flow should be
>+ measured at the same time.
>+ :type perf_parallel_streams: :any:`IntParam` (default 1)
>+
>+ :param perf_msg_sizes:
>+ Parameter used by the :any:`generate_flow_combinations` generator. To
>+ specify what different message sizes (in bytes) used generated for the
>+ network flow should be tested - each message size resulting in a
>+ separate performance measurement.
>+ :type perf_msg_sizes: List[Int] (default [123])
>+ """
>+
>+ # common perf test params
>+ perf_tests = Param(default=("tcp_stream", "udp_stream",
"sctp_stream"))
>+ perf_tool_cpu = IntParam(mandatory=False)
>+ perf_duration = IntParam(default=60)
>+ perf_iterations = IntParam(default=5)
>+ perf_parallel_streams = IntParam(default=1)
>+ perf_msg_sizes = ListParam(default=[123])
>+
>+ net_perf_tool = Param(default=IperfFlowMeasurement)
>+
>+ def generate_perf_measurements_combinations(self, config):
>+ combinations = super().generate_perf_measurements_combinations(config)
>+ for flow_combination in self.generate_flow_combinations(config):
>+ combinations.append([self.params.net_perf_tool(flow_combination)])
>+ return combinations
>+
>+ def generate_flow_combinations(self, config):
>+ """Base flow combination generator
>+
>+ The generator loops over all endpoint pairs to test performance between
>+ (generated by the :any:`generate_perf_endpoints` method) then over all
>+ the selected :any:`ip_versions` and uses the first IP address fitting
>+ these criteria. Then the generator loops over the selected performance
>+ tests as selected via :any:`perf_tests`, then message sizes from
>+ :any:`msg_sizes`.
>+
>+ :return: list of Flow combinations to measure in parallel
>+ :rtype: List[:any:`PerfFlow`]
>+ """
>+ for client_nic, server_nic in self.generate_perf_endpoints(config):
>+ for ipv in self.params.ip_versions:
>+ ip_filter = {}
>+ if ipv == "ipv4":
>+ ip_filter.update(family=AF_INET)
>+ elif ipv == "ipv6":
>+ ip_filter.update(family=AF_INET6)
>+ ip_filter.update(is_link_local=False)
>+
>+ client_bind = client_nic.ips_filter(**ip_filter)[0]
>+ server_bind = server_nic.ips_filter(**ip_filter)[0]
>+
>+ for perf_test in self.params.perf_tests:
>+ for size in self.params.perf_msg_sizes:
>+ yield [
>+ self._create_perf_flow(
>+ perf_test,
>+ client_nic,
>+ client_bind,
>+ server_nic,
>+ server_bind,
>+ size,
>+ )
>+ ]
>+
>+ def generate_perf_endpoints(self, config):
>+ """Generator for perf endpoints
>+
>+ To be overriden by a derived class.
>+
>+ :return: list of device pairs
>+ :rtype: List[Tuple[:any:`Device`, :any:`Device`]]
>+ """
>+ return []
>+
>+ def _create_perf_flow(
>+ self,
>+ perf_test,
>+ client_nic,
>+ client_bind,
>+ server_nic,
>+ server_bind,
>+ msg_size,
>+ ) -> PerfFlow:
>+ """
>+ Wrapper to create a PerfFlow. Mixins that want to change this behavior (for
example, to reverse the direction)
>+ can override this method as an alternative to overriding
:any:`generate_flow_combinations`
>+ """
>+ return PerfFlow(
>+ type=perf_test,
>+ generator=client_nic.netns,
>+ generator_bind=client_bind,
>+ generator_nic=client_nic,
>+ receiver=server_nic.netns,
>+ receiver_bind=server_bind,
>+ receiver_nic=server_nic,
>+ msg_size=msg_size,
>+ duration=self.params.perf_duration,
>+ parallel_streams=self.params.perf_parallel_streams,
>+ cpupin=(
>+ self.params.perf_tool_cpu
>+ if "perf_tool_cpu" in self.params
>+ else None
>+ ),
>+ )
>diff --git a/lnst/Recipes/ENRT/MeasurementGenerators/__init__.py
b/lnst/Recipes/ENRT/MeasurementGenerators/__init__.py
>new file mode 100644
>index 0000000..e69de29
>--
>2.29.2
>_______________________________________________
>LNST-developers mailing list -- lnst-developers(a)lists.fedorahosted.org
>To unsubscribe send an email to lnst-developers-leave(a)lists.fedorahosted.org
>Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
>List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
>List Archives:
https://lists.fedorahosted.org/archives/list/lnst-developers@lists.fedora...