[PATCH v2 0/7] fixing phase3, extending Netperf and some Version bugfixes

List overview All Threads
Download

newer

older

[PATCH] recipes: update...

[PATCH v2 7/7] Config: redirect...

Ondrej Lichtner

11 Jul 2016 11 Jul '16

6:12 a.m.

From: Ondrej Lichtner olichtne@redhat.com

I still forgot some stuff in the "fix hash related errors" commit so I'm sending a v2.

The extension to Netperf (and the modifications to regression_tests recipes) was confirmed to be working in beaker jobs, so this can definitely be pushed.

And during testing I've also found a two issues with the new feature of checking versions. I've added the fixes to this patchset.

Ondrej Lichtner (7): regression_tests: vxlan_multicast: fix name of mapping file regression_tests: phase3: fix hash related errors Netperf: add option max_deviation Netperf: check errors before the measured results regression_tests: add max_deviation to netperf tests Common: create Version.py Config: redirect git stderr to /dev/null

lnst/Common/Config.py | 7 ++- lnst/Common/Version.py | 14 +++++ recipes/regression_tests/phase1/3_vlans.py | 13 +++-- recipes/regression_tests/phase1/3_vlans.xml | 1 + .../phase1/3_vlans_over_active_backup_bond.xml | 1 + .../regression_tests/phase1/3_vlans_over_bond.py | 13 +++-- .../phase1/3_vlans_over_round_robin_bond.xml | 1 + .../regression_tests/phase1/active_backup_bond.xml | 1 + .../phase1/active_backup_double_bond.xml | 1 + recipes/regression_tests/phase1/bonding_test.py | 13 +++-- .../regression_tests/phase1/round_robin_bond.xml | 1 + .../phase1/round_robin_double_bond.xml | 1 + recipes/regression_tests/phase1/simple_netperf.py | 13 +++-- recipes/regression_tests/phase1/simple_netperf.xml | 1 + ...tual_bridge_2_vlans_over_active_backup_bond.xml | 1 + .../phase1/virtual_bridge_2_vlans_over_bond.py | 13 +++-- .../phase1/virtual_bridge_vlan_in_guest.py | 13 +++-- .../phase1/virtual_bridge_vlan_in_guest.xml | 1 + .../phase1/virtual_bridge_vlan_in_host.py | 13 +++-- .../phase1/virtual_bridge_vlan_in_host.xml | 1 + .../phase2/3_vlans_over_active_backup_team.xml | 1 + .../phase2/3_vlans_over_round_robin_team.xml | 1 + .../regression_tests/phase2/3_vlans_over_team.py | 13 +++-- .../phase2/active_backup_double_team.xml | 1 + .../regression_tests/phase2/active_backup_team.xml | 1 + .../active_backup_team_vs_active_backup_bond.xml | 1 + .../active_backup_team_vs_round_robin_bond.xml | 1 + .../phase2/round_robin_double_team.xml | 1 + .../regression_tests/phase2/round_robin_team.xml | 1 + .../round_robin_team_vs_active_backup_bond.xml | 1 + .../round_robin_team_vs_round_robin_bond.xml | 1 + recipes/regression_tests/phase2/team_test.py | 13 +++-- ...l_ovs_bridge_2_vlans_over_active_backup_bond.py | 13 +++-- ..._ovs_bridge_2_vlans_over_active_backup_bond.xml | 1 + .../phase2/virtual_ovs_bridge_vlan_in_guest.py | 13 +++-- .../phase2/virtual_ovs_bridge_vlan_in_guest.xml | 1 + .../phase2/virtual_ovs_bridge_vlan_in_host.py | 13 +++-- .../phase2/virtual_ovs_bridge_vlan_in_host.xml | 1 + .../regression_tests/phase3/2_virt_ovs_vxlan.py | 49 ++++++++++++---- .../regression_tests/phase3/2_virt_ovs_vxlan.xml | 17 +++--- .../regression_tests/phase3/novirt_ovs_vxlan.py | 25 +++++--- .../regression_tests/phase3/novirt_ovs_vxlan.xml | 1 + recipes/regression_tests/phase3/vxlan_multicast.py | 39 ++++++++++--- .../regression_tests/phase3/vxlan_multicast.xml | 7 ++- recipes/regression_tests/phase3/vxlan_remote.py | 25 +++++--- recipes/regression_tests/phase3/vxlan_remote.xml | 1 + setup.py | 2 +- test_modules/Netperf.py | 66 ++++++++++++++++++---- 48 files changed, 322 insertions(+), 111 deletions(-) create mode 100644 lnst/Common/Version.py

-- 2.9.0

Show replies by date

Ondrej Lichtner

11 Jul 11 Jul

6:12 a.m.

New subject: [PATCH v2 1/7] regression_tests: vxlan_multicast: fix name of mapping file

From: Ondrej Lichtner olichtne@redhat.com

Fixing a copy paste error.

Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- recipes/regression_tests/phase3/vxlan_multicast.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/recipes/regression_tests/phase3/vxlan_multicast.xml b/recipes/regression_tests/phase3/vxlan_multicast.xml index f22610f..cfdb4ac 100644 --- a/recipes/regression_tests/phase3/vxlan_multicast.xml +++ b/recipes/regression_tests/phase3/vxlan_multicast.xml @@ -8,7 +8,7 @@ <alias name="nperf_max_runs" value="5"/> <alias name="nperf_num_parallel" value="1"/> <alias name="nperf_debug" value="0"/> - <alias name="mapping_file" value="vxlan.mapping" /> + <alias name="mapping_file" value="vxlan_multicast.mapping" /> <alias name="net" value="192.168.0"/> <alias name="vxlan_net" value="192.168.100"/> <alias name="vxlan_net6" value="fc00:0:0:0"/>

-- 2.9.0

Ondrej Lichtner

6:12 a.m.

New subject: [PATCH v2 2/7] regression_tests: phase3: fix hash related errors

From: Ondrej Lichtner olichtne@redhat.com

This commit adds regex patterns that should be ignored during hash generation so that the hashes are stable between beaker runs. I also changed the ids of interfaces connecting guests in the 2_virt_ovs_vxlan.xml recipe to be consistent with the other virt recipes.

Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- .../regression_tests/phase3/2_virt_ovs_vxlan.py | 36 +++++++++++++++++----- .../regression_tests/phase3/2_virt_ovs_vxlan.xml | 16 +++++----- .../regression_tests/phase3/novirt_ovs_vxlan.py | 20 +++++++----- recipes/regression_tests/phase3/vxlan_multicast.py | 34 ++++++++++++++------ .../regression_tests/phase3/vxlan_multicast.xml | 4 +-- recipes/regression_tests/phase3/vxlan_remote.py | 20 +++++++----- 6 files changed, 87 insertions(+), 43 deletions(-)

diff --git a/recipes/regression_tests/phase3/2_virt_ovs_vxlan.py b/recipes/regression_tests/phase3/2_virt_ovs_vxlan.py index 45b5873..57f1373 100644 --- a/recipes/regression_tests/phase3/2_virt_ovs_vxlan.py +++ b/recipes/regression_tests/phase3/2_virt_ovs_vxlan.py @@ -129,8 +129,13 @@ if ipv in [ 'ipv4', 'both' ]: result_tcp = perf_api.new_result("tcp_ipv4_id", "tcp_ipv4_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'test_host\d+.hostname', + r'test_host\d+..*hwaddr', + r'machine_h\d+..*ovs\d*.hwaddr', + r'machine_h\d+..*tap\d*.hwaddr', + r'machine_h\d+..*tap\d*.devname']) result_tcp.add_tag(product_name) if nperf_num_parallel > 1: result_tcp.add_tag("multithreaded") @@ -159,8 +164,13 @@ if ipv in [ 'ipv4', 'both' ]: result_udp = perf_api.new_result("udp_ipv4_id", "udp_ipv4_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'test_host\d+.hostname', + r'test_host\d+..*hwaddr', + r'machine_h\d+..*ovs\d*.hwaddr', + r'machine_h\d+..*tap\d*.hwaddr', + r'machine_h\d+..*tap\d*.devname']) result_udp.add_tag(product_name) if nperf_num_parallel > 1: result_udp.add_tag("multithreaded") @@ -189,8 +199,13 @@ if ipv in [ 'ipv6', 'both' ]: result_tcp = perf_api.new_result("tcp_ipv6_id", "tcp_ipv6_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'test_host\d+.hostname', + r'test_host\d+..*hwaddr', + r'machine_h\d+..*ovs\d*.hwaddr', + r'machine_h\d+..*tap\d*.hwaddr', + r'machine_h\d+..*tap\d*.devname']) result_tcp.add_tag(product_name) if nperf_num_parallel > 1: result_tcp.add_tag("multithreaded") @@ -219,8 +234,13 @@ if ipv in [ 'ipv6', 'both' ]: result_udp = perf_api.new_result("udp_ipv6_id", "udp_ipv6_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'test_host\d+.hostname', + r'test_host\d+..*hwaddr', + r'machine_h\d+..*ovs\d*.hwaddr', + r'machine_h\d+..*tap\d*.hwaddr', + r'machine_h\d+..*tap\d*.devname']) result_udp.add_tag(product_name) if nperf_num_parallel > 1: result_udp.add_tag("multithreaded") diff --git a/recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml b/recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml index f16f9bc..a9c97d1 100644 --- a/recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml +++ b/recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml @@ -21,16 +21,16 @@ <address value="{$net}.1/24"/> </addresses> </eth> - <eth id="if2" label="to_guest1"/> - <eth id="if3" label="to_guest2"/> + <eth id="tap1" label="to_guest1"/> + <eth id="tap2" label="to_guest2"/> <ovs_bridge id="ovs1"> <slaves> - <slave id="if2"> + <slave id="tap1"> <options> <option name="ofport_request" value="5"/> </options> </slave> - <slave id="if3"> + <slave id="tap2"> <options> <option name="ofport_request" value="6"/> </options> @@ -80,16 +80,16 @@ <address value="{$net}.2/24"/> </addresses> </eth> - <eth id="if2" label="to_guest3"/> - <eth id="if3" label="to_guest4"/> + <eth id="tap1" label="to_guest3"/> + <eth id="tap2" label="to_guest4"/> <ovs_bridge id="ovs2"> <slaves> - <slave id="if2"> + <slave id="tap1"> <options> <option name="ofport_request" value="5"/> </options> </slave> - <slave id="if3"> + <slave id="tap2"> <options> <option name="ofport_request" value="6"/> </options> diff --git a/recipes/regression_tests/phase3/novirt_ovs_vxlan.py b/recipes/regression_tests/phase3/novirt_ovs_vxlan.py index f69e77f..d3b0ac9 100644 --- a/recipes/regression_tests/phase3/novirt_ovs_vxlan.py +++ b/recipes/regression_tests/phase3/novirt_ovs_vxlan.py @@ -80,8 +80,9 @@ if ipv in [ 'ipv4', 'both' ]: result_tcp = perf_api.new_result("tcp_ipv4_id", "tcp_ipv4_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'interface_ovs\d+.hwaddr']) result_tcp.add_tag(product_name) if nperf_num_parallel > 1: result_tcp.add_tag("multithreaded") @@ -111,8 +112,9 @@ if ipv in [ 'ipv4', 'both' ]: result_udp = perf_api.new_result("udp_ipv4_id", "udp_ipv4_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'interface_ovs\d+.hwaddr']) result_udp.add_tag(product_name) if nperf_num_parallel > 1: result_udp.add_tag("multithreaded") @@ -144,8 +146,9 @@ if ipv in [ 'ipv6', 'both' ]: result_tcp = perf_api.new_result("tcp_ipv6_id", "tcp_ipv6_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'interface_ovs\d+.hwaddr']) result_tcp.add_tag(product_name) if nperf_num_parallel > 1: result_tcp.add_tag("multithreaded") @@ -175,8 +178,9 @@ if ipv in [ 'ipv6', 'both' ]: result_udp = perf_api.new_result("udp_ipv6_id", "udp_ipv6_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'interface_ovs\d+.hwaddr']) result_udp.add_tag(product_name) if nperf_num_parallel > 1: result_udp.add_tag("multithreaded") diff --git a/recipes/regression_tests/phase3/vxlan_multicast.py b/recipes/regression_tests/phase3/vxlan_multicast.py index 9cfa900..293e8ec 100644 --- a/recipes/regression_tests/phase3/vxlan_multicast.py +++ b/recipes/regression_tests/phase3/vxlan_multicast.py @@ -54,7 +54,7 @@ if nperf_cpupin: g1.run("service irqbalance stop")

m1_phy1 = m1.get_interface("eth1") - m1_phy2 = m1.get_interface("eth2") + m1_phy2 = m1.get_interface("tap1") m2_phy1 = m2.get_interface("eth1") g1_phy1 = g1.get_interface("eth1") dev_list = [(m1, m1_phy1), (m1, m1_phy2), (m2, m2_phy1), (g1, g1_phy1)] @@ -102,8 +102,12 @@ if ipv in [ 'ipv4', 'both' ]: result_tcp = perf_api.new_result("tcp_ipv4_id", "tcp_ipv4_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'testmachine\d+.interface_tap\d+.hwaddr', + r'guest\d+.hostname', + r'guest\d+.interface_eth\d+.hwaddr', + r'test_if.hwaddr']) result_tcp.add_tag(product_name) if nperf_num_parallel > 1: result_tcp.add_tag("multithreaded") @@ -133,8 +137,12 @@ if ipv in [ 'ipv4', 'both' ]: result_udp = perf_api.new_result("udp_ipv4_id", "udp_ipv4_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'testmachine\d+.interface_tap\d+.hwaddr', + r'guest\d+.hostname', + r'guest\d+.interface_eth\d+.hwaddr', + r'test_if.hwaddr']) result_udp.add_tag(product_name) if nperf_num_parallel > 1: result_udp.add_tag("multithreaded") @@ -165,8 +173,12 @@ if ipv in [ 'ipv6', 'both' ]: result_tcp = perf_api.new_result("tcp_ipv6_id", "tcp_ipv6_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'testmachine\d+.interface_tap\d+.hwaddr', + r'guest\d+.hostname', + r'guest\d+.interface_eth\d+.hwaddr', + r'test_if.hwaddr']) result_tcp.add_tag(product_name) if nperf_num_parallel > 1: result_tcp.add_tag("multithreaded") @@ -196,8 +208,12 @@ if ipv in [ 'ipv6', 'both' ]: result_udp = perf_api.new_result("udp_ipv6_id", "udp_ipv6_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'testmachine\d+.interface_tap\d+.hwaddr', + r'guest\d+.hostname', + r'guest\d+.interface_eth\d+.hwaddr', + r'test_if.hwaddr']) result_udp.add_tag(product_name) if nperf_num_parallel > 1: result_udp.add_tag("multithreaded") diff --git a/recipes/regression_tests/phase3/vxlan_multicast.xml b/recipes/regression_tests/phase3/vxlan_multicast.xml index cfdb4ac..87b2cb3 100644 --- a/recipes/regression_tests/phase3/vxlan_multicast.xml +++ b/recipes/regression_tests/phase3/vxlan_multicast.xml @@ -17,11 +17,11 @@ <host id="testmachine1"> <interfaces> <eth id="eth1" label="tnet"/> - <eth id="eth2" label="to_guest1"/> + <eth id="tap1" label="to_guest1"/> <bridge id="br0"> <slaves> <slave id="eth1"/> - <slave id="eth2"/> + <slave id="tap1"/> </slaves> <addresses> <address value="{$net}.1/24" /> diff --git a/recipes/regression_tests/phase3/vxlan_remote.py b/recipes/regression_tests/phase3/vxlan_remote.py index 9086a1e..712c34b 100644 --- a/recipes/regression_tests/phase3/vxlan_remote.py +++ b/recipes/regression_tests/phase3/vxlan_remote.py @@ -76,8 +76,9 @@ if ipv in [ 'ipv4', 'both' ]: result_tcp = perf_api.new_result("tcp_ipv4_id", "tcp_ipv4_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'test_if.hwaddr']) result_tcp.add_tag(product_name) if nperf_num_parallel > 1: result_tcp.add_tag("multithreaded") @@ -107,8 +108,9 @@ if ipv in [ 'ipv4', 'both' ]: result_udp = perf_api.new_result("udp_ipv4_id", "udp_ipv4_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'test_if.hwaddr']) result_udp.add_tag(product_name) if nperf_num_parallel > 1: result_udp.add_tag("multithreaded") @@ -143,8 +145,9 @@ if ipv in [ 'ipv6', 'both' ]: result_tcp = perf_api.new_result("tcp_ipv6_id", "tcp_ipv6_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'test_if.hwaddr']) result_tcp.add_tag(product_name) if nperf_num_parallel > 1: result_tcp.add_tag("multithreaded") @@ -174,8 +177,9 @@ if ipv in [ 'ipv6', 'both' ]: result_udp = perf_api.new_result("udp_ipv6_id", "udp_ipv6_result", hash_ignore=[ - 'kernel_release', - 'redhat_release']) + r'kernel_release', + r'redhat_release', + r'test_if.hwaddr']) result_udp.add_tag(product_name) if nperf_num_parallel > 1: result_udp.add_tag("multithreaded")

-- 2.9.0

Ondrej Lichtner

6:12 a.m.

New subject: [PATCH v2 3/7] Netperf: add option max_deviation

From: Ondrej Lichtner olichtne@redhat.com

This option limits the maximum allowed deviation. If the measured deviation exceeds this limit the result will automatically be FAIL. This is because sometimes the deviation gets so big that it overlaps with the specified threshold interval resulting in a PASS even though the result is so unstable that it shouldn't be compared to anything.

The option accepts either a percentual value describing the maximum deviation or an absolute value in the same format that we use for specifying the threshold.

Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- test_modules/Netperf.py | 42 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+)

diff --git a/test_modules/Netperf.py b/test_modules/Netperf.py index 6367808..5a9ffe7 100644 --- a/test_modules/Netperf.py +++ b/test_modules/Netperf.py @@ -56,6 +56,9 @@ class Netperf(TestGeneric): else: self._threshold_interval = None

+ self._max_deviation = self._parse_max_deviation( + self.get_opt("max_deviation", default=None)) + def _is_omni(self): return self._testname in self.omni_tests

@@ -313,6 +316,21 @@ class Netperf(TestGeneric): return {"rate": threshold_rate, "unit": threshold_unit_type}

+ def _parse_max_deviation(self, deviation): + if deviation is None: + return None + percentual_deviation = r"(\d+(.\d+)?)\s*%" + match = re.match(percentual_deviation, deviation) + if match: + return {"type": "percent", + "value": float(match.group(1))} + else: + val = self._parse_threshold(deviation) + if val is not None: + return {"type": "absolute", + "value": val} + return None + def _sum_results(self, first, second): result = {}

@@ -457,6 +475,30 @@ class Netperf(TestGeneric): rate_dev_pretty = self._pretty_rate(rate_deviation, unit=rate_pretty["unit"])

res_val = False + if self._max_deviation is not None: + if self._max_deviation["type"] == "percent": + percentual_deviation = (rate_deviation / rate) * 100 + if percentual_deviation > self._max_deviation["value"]: + res_val = False + res_data["msg"] = "Measured rate %.2f +-%.2f %s has bigger "\ + "deviation than allowed (+-%.2f %%)" %\ + (rate_pretty["rate"], + rate_dev_pretty["rate"], + rate_pretty["unit"], + self._max_deviation["value"]) + return (res_val, res_data) + elif self._max_deviation["type"] == "absolute": + if rate_deviation > self._max_deviation["value"]["rate"]: + pretty_deviation = self._pretty_rate(self._max_deviation["value"]["rate"]) + res_val = False + res_data["msg"] = "Measured rate %.2f +-%.2f %s has bigger "\ + "deviation than allowed (+-%.2f %s)" %\ + (rate_pretty["rate"], + rate_dev_pretty["rate"], + rate_pretty["unit"], + pretty_deviation["rate"], + pretty_deviation["unit"]) + return (res_val, res_data) if self._threshold_interval is not None: result_interval = (rate - rate_deviation, rate + rate_deviation)

-- 2.9.0

Ondrej Lichtner

6:12 a.m.

New subject: [PATCH v2 4/7] Netperf: check errors before the measured results

From: Ondrej Lichtner olichtne@redhat.com

There's no point in doing a comparison against threshold if the netperf command returned errors. Moving this check before the threshold comparison. I also added returns to the individual threshold comparison branches where it's not necessary to continue in the function execution.

Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- test_modules/Netperf.py | 24 +++++++++++++----------- 1 file changed, 13 insertions(+), 11 deletions(-)

diff --git a/test_modules/Netperf.py b/test_modules/Netperf.py index 5a9ffe7..d09bdfc 100644 --- a/test_modules/Netperf.py +++ b/test_modules/Netperf.py @@ -474,6 +474,17 @@ class Netperf(TestGeneric): rate_pretty = self._pretty_rate(rate) rate_dev_pretty = self._pretty_rate(rate_deviation, unit=rate_pretty["unit"])

+ if rv != 0 and self._runs == 1: + res_data["msg"] = "Could not get performance throughput!" + logging.info(res_data["msg"]) + return (False, res_data) + elif rv != 0 and self._runs > 1: + res_data["msg"] = "At least one of the Netperf runs failed, "\ + "check the logs and result data for more "\ + "information." + logging.info(res_data["msg"]) + return (False, res_data) + res_val = False if self._max_deviation is not None: if self._max_deviation["type"] == "percent": @@ -517,6 +528,7 @@ class Netperf(TestGeneric): threshold_pretty["rate"], threshold_dev_pretty["rate"], threshold_pretty["unit"]) + return (res_val, res_data) else: res_val = True res_data["msg"] = "Measured rate %.2f +-%.2f %s is higher "\ @@ -527,6 +539,7 @@ class Netperf(TestGeneric): threshold_pretty["rate"], threshold_dev_pretty["rate"], threshold_pretty["unit"]) + return (res_val, res_data) else: if rate > 0.0: res_val = True @@ -536,17 +549,6 @@ class Netperf(TestGeneric): (rate_pretty["rate"], rate_dev_pretty["rate"], rate_pretty["unit"]) - - if rv != 0 and self._runs == 1: - res_data["msg"] = "Could not get performance throughput!" - logging.info(res_data["msg"]) - return (False, res_data) - elif rv != 0 and self._runs > 1: - res_data["msg"] = "At least one of the Netperf runs failed, "\ - "check the logs and result data for more "\ - "information." - logging.info(res_data["msg"]) - return (False, res_data) return (res_val, res_data)

def run(self):

-- 2.9.0

Ondrej Lichtner

6:12 a.m.

New subject: [PATCH v2 5/7] regression_tests: add max_deviation to netperf tests

From: Ondrej Lichtner olichtne@redhat.com

This commit adds a max_deviation parameter to all Netperf test_module clients in our regression_tests directory. The value of the parameter can be controlled through alias "nperf_max_dev" that I've added to all the recipes and set the default value to 20%. I chose the default value by going through the PASS values we usually see on these tests. None of these normally exceed 20% so it should be safe.

Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- recipes/regression_tests/phase1/3_vlans.py | 13 +++++++++---- recipes/regression_tests/phase1/3_vlans.xml | 1 + .../phase1/3_vlans_over_active_backup_bond.xml | 1 + recipes/regression_tests/phase1/3_vlans_over_bond.py | 13 +++++++++---- .../phase1/3_vlans_over_round_robin_bond.xml | 1 + recipes/regression_tests/phase1/active_backup_bond.xml | 1 + .../regression_tests/phase1/active_backup_double_bond.xml | 1 + recipes/regression_tests/phase1/bonding_test.py | 13 +++++++++---- recipes/regression_tests/phase1/round_robin_bond.xml | 1 + recipes/regression_tests/phase1/round_robin_double_bond.xml | 1 + recipes/regression_tests/phase1/simple_netperf.py | 13 +++++++++---- recipes/regression_tests/phase1/simple_netperf.xml | 1 + .../virtual_bridge_2_vlans_over_active_backup_bond.xml | 1 + .../phase1/virtual_bridge_2_vlans_over_bond.py | 13 +++++++++---- .../regression_tests/phase1/virtual_bridge_vlan_in_guest.py | 13 +++++++++---- .../phase1/virtual_bridge_vlan_in_guest.xml | 1 + .../regression_tests/phase1/virtual_bridge_vlan_in_host.py | 13 +++++++++---- .../regression_tests/phase1/virtual_bridge_vlan_in_host.xml | 1 + .../phase2/3_vlans_over_active_backup_team.xml | 1 + .../phase2/3_vlans_over_round_robin_team.xml | 1 + recipes/regression_tests/phase2/3_vlans_over_team.py | 13 +++++++++---- .../regression_tests/phase2/active_backup_double_team.xml | 1 + recipes/regression_tests/phase2/active_backup_team.xml | 1 + .../phase2/active_backup_team_vs_active_backup_bond.xml | 1 + .../phase2/active_backup_team_vs_round_robin_bond.xml | 1 + recipes/regression_tests/phase2/round_robin_double_team.xml | 1 + recipes/regression_tests/phase2/round_robin_team.xml | 1 + .../phase2/round_robin_team_vs_active_backup_bond.xml | 1 + .../phase2/round_robin_team_vs_round_robin_bond.xml | 1 + recipes/regression_tests/phase2/team_test.py | 13 +++++++++---- .../virtual_ovs_bridge_2_vlans_over_active_backup_bond.py | 13 +++++++++---- .../virtual_ovs_bridge_2_vlans_over_active_backup_bond.xml | 1 + .../phase2/virtual_ovs_bridge_vlan_in_guest.py | 13 +++++++++---- .../phase2/virtual_ovs_bridge_vlan_in_guest.xml | 1 + .../phase2/virtual_ovs_bridge_vlan_in_host.py | 13 +++++++++---- .../phase2/virtual_ovs_bridge_vlan_in_host.xml | 1 + recipes/regression_tests/phase3/2_virt_ovs_vxlan.py | 13 +++++++++---- recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml | 1 + recipes/regression_tests/phase3/novirt_ovs_vxlan.py | 5 +++++ recipes/regression_tests/phase3/novirt_ovs_vxlan.xml | 1 + recipes/regression_tests/phase3/vxlan_multicast.py | 5 +++++ recipes/regression_tests/phase3/vxlan_multicast.xml | 1 + recipes/regression_tests/phase3/vxlan_remote.py | 5 +++++ recipes/regression_tests/phase3/vxlan_remote.xml | 1 + 44 files changed, 160 insertions(+), 52 deletions(-)

diff --git a/recipes/regression_tests/phase1/3_vlans.py b/recipes/regression_tests/phase1/3_vlans.py index 49020db..8144815 100644 --- a/recipes/regression_tests/phase1/3_vlans.py +++ b/recipes/regression_tests/phase1/3_vlans.py @@ -43,6 +43,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([m1, m2], pr_user_comment) @@ -111,7 +112,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev }) netperf_cli_udp = ctl.get_module("Netperf", options={ @@ -123,7 +125,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev }) netperf_cli_tcp6 = ctl.get_module("Netperf", options={ @@ -135,7 +138,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev }) netperf_cli_udp6 = ctl.get_module("Netperf", options={ @@ -147,7 +151,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase1/3_vlans.xml b/recipes/regression_tests/phase1/3_vlans.xml index 03cf350..5c0f1fe 100644 --- a/recipes/regression_tests/phase1/3_vlans.xml +++ b/recipes/regression_tests/phase1/3_vlans.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="3_vlans.mapping" /> <alias name="vlan10_net" value="192.168.10"/> <alias name="vlan10_tag" value="10"/> diff --git a/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml b/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml index 9f4f932..f9b4922 100644 --- a/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml +++ b/recipes/regression_tests/phase1/3_vlans_over_active_backup_bond.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="3_vlans_over_active_backup_bond.mapping" /> <alias name="vlan10_net" value="192.168.10"/> <alias name="vlan10_tag" value="10"/> diff --git a/recipes/regression_tests/phase1/3_vlans_over_bond.py b/recipes/regression_tests/phase1/3_vlans_over_bond.py index a96e604..41f2b95 100644 --- a/recipes/regression_tests/phase1/3_vlans_over_bond.py +++ b/recipes/regression_tests/phase1/3_vlans_over_bond.py @@ -42,6 +42,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([m1, m2], pr_user_comment) @@ -112,7 +113,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev }) netperf_cli_udp = ctl.get_module("Netperf", options={ @@ -124,7 +126,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev }) netperf_cli_tcp6 = ctl.get_module("Netperf", options={ @@ -136,7 +139,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev }) netperf_cli_udp6 = ctl.get_module("Netperf", options={ @@ -148,7 +152,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase1/3_vlans_over_round_robin_bond.xml b/recipes/regression_tests/phase1/3_vlans_over_round_robin_bond.xml index 1f77a46..d22fa7b 100644 --- a/recipes/regression_tests/phase1/3_vlans_over_round_robin_bond.xml +++ b/recipes/regression_tests/phase1/3_vlans_over_round_robin_bond.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5"/> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="3_vlans_over_round_robin_bond.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase1/active_backup_bond.xml b/recipes/regression_tests/phase1/active_backup_bond.xml index 11c5f28..0b79f4c 100644 --- a/recipes/regression_tests/phase1/active_backup_bond.xml +++ b/recipes/regression_tests/phase1/active_backup_bond.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="active_backup_bond.mapping" /> <alias name="net" value="192.168.0"/> </define> diff --git a/recipes/regression_tests/phase1/active_backup_double_bond.xml b/recipes/regression_tests/phase1/active_backup_double_bond.xml index ccf1ba1..bc39db8 100644 --- a/recipes/regression_tests/phase1/active_backup_double_bond.xml +++ b/recipes/regression_tests/phase1/active_backup_double_bond.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="active_backup_double_bond.mapping" /> <alias name="net" value="192.168.0"/> </define> diff --git a/recipes/regression_tests/phase1/bonding_test.py b/recipes/regression_tests/phase1/bonding_test.py index 7d8a48e..39e7df8 100644 --- a/recipes/regression_tests/phase1/bonding_test.py +++ b/recipes/regression_tests/phase1/bonding_test.py @@ -42,6 +42,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([m1, m2], pr_user_comment) @@ -117,7 +118,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -130,7 +132,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -144,7 +147,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev }) netperf_cli_udp6 = ctl.get_module("Netperf", options={ @@ -157,7 +161,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase1/round_robin_bond.xml b/recipes/regression_tests/phase1/round_robin_bond.xml index 861e47c..9b28cbc 100644 --- a/recipes/regression_tests/phase1/round_robin_bond.xml +++ b/recipes/regression_tests/phase1/round_robin_bond.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5"/> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="round_robin_bond.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase1/round_robin_double_bond.xml b/recipes/regression_tests/phase1/round_robin_double_bond.xml index 0083b4b..15798f1 100644 --- a/recipes/regression_tests/phase1/round_robin_double_bond.xml +++ b/recipes/regression_tests/phase1/round_robin_double_bond.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5"/> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="round_robin_double_bond.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase1/simple_netperf.py b/recipes/regression_tests/phase1/simple_netperf.py index 811dd58..fe9d96b 100644 --- a/recipes/regression_tests/phase1/simple_netperf.py +++ b/recipes/regression_tests/phase1/simple_netperf.py @@ -42,6 +42,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([m1, m2], pr_user_comment) @@ -77,7 +78,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs" : nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -90,7 +92,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs" : nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -103,7 +106,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs" : nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp6 = ctl.get_module("Netperf", @@ -116,7 +120,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs" : nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_srv = ctl.get_module("Netperf", diff --git a/recipes/regression_tests/phase1/simple_netperf.xml b/recipes/regression_tests/phase1/simple_netperf.xml index cfc2d85..0f106aa 100644 --- a/recipes/regression_tests/phase1/simple_netperf.xml +++ b/recipes/regression_tests/phase1/simple_netperf.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default" /> <alias name="nperf_num_parallel" value="2" /> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="simple_netperf.mapping" /> <alias name="net" value="192.168.101" /> </define> diff --git a/recipes/regression_tests/phase1/virtual_bridge_2_vlans_over_active_backup_bond.xml b/recipes/regression_tests/phase1/virtual_bridge_2_vlans_over_active_backup_bond.xml index ff5d8d6..5b5223e 100644 --- a/recipes/regression_tests/phase1/virtual_bridge_2_vlans_over_active_backup_bond.xml +++ b/recipes/regression_tests/phase1/virtual_bridge_2_vlans_over_active_backup_bond.xml @@ -8,6 +8,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mtu" value="1500" /> <alias name="mapping_file" value="virtual_bridge_2_vlans_over_active_backup_bond.mapping" /> <alias name="vlan10_net" value="192.168.10"/> diff --git a/recipes/regression_tests/phase1/virtual_bridge_2_vlans_over_bond.py b/recipes/regression_tests/phase1/virtual_bridge_2_vlans_over_bond.py index 780e85b..37f703f 100644 --- a/recipes/regression_tests/phase1/virtual_bridge_2_vlans_over_bond.py +++ b/recipes/regression_tests/phase1/virtual_bridge_2_vlans_over_bond.py @@ -47,6 +47,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([h1, g1, g2, h2, g3, g4], pr_user_comment) @@ -125,7 +126,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "runs": nperf_max_runs, "netperf_opts" : "-L %s" % (g3_guestnic.get_ip(0)), - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -139,7 +141,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "runs": nperf_max_runs, "netperf_opts" : "-L %s" % (g3_guestnic.get_ip(0)), - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -154,7 +157,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "runs": nperf_max_runs, "netperf_opts" : "-L %s -6" % (g3_guestnic.get_ip(1)), - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp6 = ctl.get_module("Netperf", @@ -169,7 +173,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "runs": nperf_max_runs, "netperf_opts" : "-L %s -6" % (g3_guestnic.get_ip(1)), - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.py b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.py index 416f469..b4b5c6c 100644 --- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.py +++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.py @@ -43,6 +43,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([h1, g1, h2], pr_user_comment) @@ -111,7 +112,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -124,7 +126,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -138,7 +141,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp6 = ctl.get_module("Netperf", @@ -152,7 +156,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml index 6aba6f8..7768542 100644 --- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml +++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_guest.xml @@ -8,6 +8,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mtu" value="1500" /> <alias name="mapping_file" value="virtual_bridge_vlan_in_guest.mapping" /> <alias name="vlan10_net" value="192.168.10"/> diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.py b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.py index 03dd3bf..1ebdd91 100644 --- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.py +++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.py @@ -43,6 +43,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([h1, g1, h2], pr_user_comment) @@ -110,7 +111,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -123,7 +125,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -137,7 +140,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp6 = ctl.get_module("Netperf", @@ -151,7 +155,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml index 27583ac..a38848b 100644 --- a/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml +++ b/recipes/regression_tests/phase1/virtual_bridge_vlan_in_host.xml @@ -8,6 +8,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mtu" value="1500" /> <alias name="mapping_file" value="virtual_bridge_vlan_in_host.mapping" /> <alias name="vlan10_net" value="192.168.10"/> diff --git a/recipes/regression_tests/phase2/3_vlans_over_active_backup_team.xml b/recipes/regression_tests/phase2/3_vlans_over_active_backup_team.xml index 2bcfc88..c2a5117 100644 --- a/recipes/regression_tests/phase2/3_vlans_over_active_backup_team.xml +++ b/recipes/regression_tests/phase2/3_vlans_over_active_backup_team.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="3_vlans_over_active_backup_team.mapping" /> <alias name="vlan10_net" value="192.168.10"/> <alias name="vlan10_tag" value="10"/> diff --git a/recipes/regression_tests/phase2/3_vlans_over_round_robin_team.xml b/recipes/regression_tests/phase2/3_vlans_over_round_robin_team.xml index 1a374bd..7576ec0 100644 --- a/recipes/regression_tests/phase2/3_vlans_over_round_robin_team.xml +++ b/recipes/regression_tests/phase2/3_vlans_over_round_robin_team.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5" /> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="3_vlans_over_round_robin_team.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase2/3_vlans_over_team.py b/recipes/regression_tests/phase2/3_vlans_over_team.py index 2a33268..e9cae83 100644 --- a/recipes/regression_tests/phase2/3_vlans_over_team.py +++ b/recipes/regression_tests/phase2/3_vlans_over_team.py @@ -42,6 +42,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase2/active_backup_double_team.xml b/recipes/regression_tests/phase2/active_backup_double_team.xml index 235872b..3825fb0 100644 --- a/recipes/regression_tests/phase2/active_backup_double_team.xml +++ b/recipes/regression_tests/phase2/active_backup_double_team.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="active_backup_double_team.mapping" /> <alias name="net" value="192.168.0"/> </define> diff --git a/recipes/regression_tests/phase2/active_backup_team.xml b/recipes/regression_tests/phase2/active_backup_team.xml index 721edeb..abf4451 100644 --- a/recipes/regression_tests/phase2/active_backup_team.xml +++ b/recipes/regression_tests/phase2/active_backup_team.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="active_backup_team.mapping" /> <alias name="net" value="192.168.0"/> </define> diff --git a/recipes/regression_tests/phase2/active_backup_team_vs_active_backup_bond.xml b/recipes/regression_tests/phase2/active_backup_team_vs_active_backup_bond.xml index b2fc512..e0cb9c1 100644 --- a/recipes/regression_tests/phase2/active_backup_team_vs_active_backup_bond.xml +++ b/recipes/regression_tests/phase2/active_backup_team_vs_active_backup_bond.xml @@ -9,6 +9,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="active_backup_team_vs_active_backup_bond.mapping" /> <alias name="net" value="192.168.0"/> </define> diff --git a/recipes/regression_tests/phase2/active_backup_team_vs_round_robin_bond.xml b/recipes/regression_tests/phase2/active_backup_team_vs_round_robin_bond.xml index 3d40c72..983a512 100644 --- a/recipes/regression_tests/phase2/active_backup_team_vs_round_robin_bond.xml +++ b/recipes/regression_tests/phase2/active_backup_team_vs_round_robin_bond.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5" /> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="active_backup_team_vs_round_robin_bond.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase2/round_robin_double_team.xml b/recipes/regression_tests/phase2/round_robin_double_team.xml index 4acc6c6..a4cb9cb 100644 --- a/recipes/regression_tests/phase2/round_robin_double_team.xml +++ b/recipes/regression_tests/phase2/round_robin_double_team.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5" /> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="round_robin_double_team.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase2/round_robin_team.xml b/recipes/regression_tests/phase2/round_robin_team.xml index 95ecac2..9ff89eb 100644 --- a/recipes/regression_tests/phase2/round_robin_team.xml +++ b/recipes/regression_tests/phase2/round_robin_team.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5" /> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="round_robin_team.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase2/round_robin_team_vs_active_backup_bond.xml b/recipes/regression_tests/phase2/round_robin_team_vs_active_backup_bond.xml index 99f1767..0d15dda 100644 --- a/recipes/regression_tests/phase2/round_robin_team_vs_active_backup_bond.xml +++ b/recipes/regression_tests/phase2/round_robin_team_vs_active_backup_bond.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5" /> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="round_robin_team_vs_active_backup_bond.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase2/round_robin_team_vs_round_robin_bond.xml b/recipes/regression_tests/phase2/round_robin_team_vs_round_robin_bond.xml index e03c64d..4b31194 100644 --- a/recipes/regression_tests/phase2/round_robin_team_vs_round_robin_bond.xml +++ b/recipes/regression_tests/phase2/round_robin_team_vs_round_robin_bond.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5" /> <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="round_robin_team_vs_round_robin_bond.mapping" /> </define> <network> diff --git a/recipes/regression_tests/phase2/team_test.py b/recipes/regression_tests/phase2/team_test.py index 8ad9758..1aa0d0f 100644 --- a/recipes/regression_tests/phase2/team_test.py +++ b/recipes/regression_tests/phase2/team_test.py @@ -41,6 +41,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([m1, m2], pr_user_comment) @@ -116,7 +117,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -129,7 +131,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -143,7 +146,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev }) netperf_cli_udp6 = ctl.get_module("Netperf", options={ @@ -156,7 +160,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_2_vlans_over_active_backup_bond.py b/recipes/regression_tests/phase2/virtual_ovs_bridge_2_vlans_over_active_backup_bond.py index 9ce9048..d795714 100644 --- a/recipes/regression_tests/phase2/virtual_ovs_bridge_2_vlans_over_active_backup_bond.py +++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_2_vlans_over_active_backup_bond.py @@ -47,6 +47,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([h1, g1, g2, h2, g3, g4], pr_user_comment) @@ -124,7 +125,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "runs": nperf_max_runs, "netperf_opts" : "-L %s" % (g3_guestnic.get_ip(0)), - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -138,7 +140,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "runs": nperf_max_runs, "netperf_opts" : "-L %s" % (g3_guestnic.get_ip(0)), - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -153,7 +156,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "runs": nperf_max_runs, "netperf_opts" : "-L %s -6" % (g3_guestnic.get_ip(1)), - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp6 = ctl.get_module("Netperf", @@ -168,7 +172,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "runs": nperf_max_runs, "netperf_opts" : "-L %s -6" % (g3_guestnic.get_ip(1)), - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_2_vlans_over_active_backup_bond.xml b/recipes/regression_tests/phase2/virtual_ovs_bridge_2_vlans_over_active_backup_bond.xml index 172f02f..4f34ac5 100644 --- a/recipes/regression_tests/phase2/virtual_ovs_bridge_2_vlans_over_active_backup_bond.xml +++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_2_vlans_over_active_backup_bond.xml @@ -8,6 +8,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="virtual_ovs_bridge_2_vlans_over_active_backup_bond.mapping" /> <alias name="vlan10_net" value="192.168.10"/> <alias name="vlan10_tag" value="10"/> diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.py b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.py index b72b660..c96e2a6 100644 --- a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.py +++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.py @@ -43,6 +43,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([h1, g1, h2], pr_user_comment) @@ -108,7 +109,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -121,7 +123,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -135,7 +138,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp6 = ctl.get_module("Netperf", @@ -149,7 +153,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml index 020b9a3..bfea7a0 100644 --- a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml +++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_guest.xml @@ -8,6 +8,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="virtual_ovs_bridge_vlan_in_guest.mapping" /> <alias name="vlan10_net" value="192.168.10"/> <alias name="vlan10_tag" value="10"/> diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.py b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.py index a2382cd..eaa1cab 100644 --- a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.py +++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.py @@ -43,6 +43,7 @@ nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_mode = ctl.get_alias("nperf_mode") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([h1, g1, h2], pr_user_comment) @@ -107,7 +108,8 @@ netperf_cli_tcp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp = ctl.get_module("Netperf", @@ -120,7 +122,8 @@ netperf_cli_udp = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_tcp6 = ctl.get_module("Netperf", @@ -134,7 +137,8 @@ netperf_cli_tcp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

netperf_cli_udp6 = ctl.get_module("Netperf", @@ -148,7 +152,8 @@ netperf_cli_udp6 = ctl.get_module("Netperf", "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : p_opts6, - "debug" : nperf_debug + "debug" : nperf_debug, + "max_deviation" : nperf_max_dev })

if nperf_mode == "multi": diff --git a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.xml b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.xml index 2715c0d..ea8f41a 100644 --- a/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.xml +++ b/recipes/regression_tests/phase2/virtual_ovs_bridge_vlan_in_host.xml @@ -8,6 +8,7 @@ <alias name="nperf_mode" value="default"/> <alias name="nperf_num_parallel" value="2"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="virtual_ovs_bridge_vlan_in_host.mapping" /> <alias name="vlan10_net" value="192.168.10"/> <alias name="vlan10_tag" value="10"/> diff --git a/recipes/regression_tests/phase3/2_virt_ovs_vxlan.py b/recipes/regression_tests/phase3/2_virt_ovs_vxlan.py index 57f1373..5ebaacc 100644 --- a/recipes/regression_tests/phase3/2_virt_ovs_vxlan.py +++ b/recipes/regression_tests/phase3/2_virt_ovs_vxlan.py @@ -42,6 +42,7 @@ nperf_cpupin = ctl.get_alias("nperf_cpupin") nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([guest1, guest2, guest3, guest4], @@ -152,7 +153,8 @@ if ipv in [ 'ipv4', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": nperf_opts, - "debug": nperf_debug}, + "debug": nperf_debug, + "max_deviation": nperf_max_dev}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs)

@@ -187,7 +189,8 @@ if ipv in [ 'ipv4', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts": nperf_opts, - "debug": nperf_debug}, + "debug": nperf_debug, + "max_deviation": nperf_max_dev}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs)

@@ -222,7 +225,8 @@ if ipv in [ 'ipv6', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : nperf_opts + "-6", - "debug": nperf_debug}, + "debug": nperf_debug, + "max_deviation": nperf_max_dev}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs)

@@ -257,7 +261,8 @@ if ipv in [ 'ipv6', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "netperf_opts" : nperf_opts + "-6", - "debug": nperf_debug}, + "debug": nperf_debug, + "max_deviation": nperf_max_dev}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs)

diff --git a/recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml b/recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml index a9c97d1..e2e7ee7 100644 --- a/recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml +++ b/recipes/regression_tests/phase3/2_virt_ovs_vxlan.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5"/> <alias name="nperf_num_parallel" value="1"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="2_virt_ovs_vxlan.mapping" /> <alias name="net" value="192.168.2"/> <alias name="vxlan_net" value="192.168.100"/> diff --git a/recipes/regression_tests/phase3/novirt_ovs_vxlan.py b/recipes/regression_tests/phase3/novirt_ovs_vxlan.py index d3b0ac9..5419ad9 100644 --- a/recipes/regression_tests/phase3/novirt_ovs_vxlan.py +++ b/recipes/regression_tests/phase3/novirt_ovs_vxlan.py @@ -36,6 +36,7 @@ nperf_cpupin = ctl.get_alias("nperf_cpupin") nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([h1, h2], pr_user_comment) @@ -100,6 +101,7 @@ if ipv in [ 'ipv4', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts": nperf_opts}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -132,6 +134,7 @@ if ipv in [ 'ipv4', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts": nperf_opts}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -166,6 +169,7 @@ if ipv in [ 'ipv6', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts" : nperf_opts + "-6"}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -198,6 +202,7 @@ if ipv in [ 'ipv6', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts" : nperf_opts + "-6"}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) diff --git a/recipes/regression_tests/phase3/novirt_ovs_vxlan.xml b/recipes/regression_tests/phase3/novirt_ovs_vxlan.xml index 25404b7..98f5f1b 100644 --- a/recipes/regression_tests/phase3/novirt_ovs_vxlan.xml +++ b/recipes/regression_tests/phase3/novirt_ovs_vxlan.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5"/> <alias name="nperf_num_parallel" value="1"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="novirt_ovs_vxlan.mapping" /> <alias name="net" value="192.168.2"/> <alias name="vxlan_net" value="192.168.100"/> diff --git a/recipes/regression_tests/phase3/vxlan_multicast.py b/recipes/regression_tests/phase3/vxlan_multicast.py index 293e8ec..0c29977 100644 --- a/recipes/regression_tests/phase3/vxlan_multicast.py +++ b/recipes/regression_tests/phase3/vxlan_multicast.py @@ -37,6 +37,7 @@ nperf_cpupin = ctl.get_alias("nperf_cpupin") nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([m1, m2], pr_user_comment) @@ -125,6 +126,7 @@ if ipv in [ 'ipv4', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts": nperf_opts}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -160,6 +162,7 @@ if ipv in [ 'ipv4', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts": nperf_opts}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -196,6 +199,7 @@ if ipv in [ 'ipv6', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts" : nperf_opts + " -6"}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -231,6 +235,7 @@ if ipv in [ 'ipv6', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts" : nperf_opts + "-6"}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) diff --git a/recipes/regression_tests/phase3/vxlan_multicast.xml b/recipes/regression_tests/phase3/vxlan_multicast.xml index 87b2cb3..38b150f 100644 --- a/recipes/regression_tests/phase3/vxlan_multicast.xml +++ b/recipes/regression_tests/phase3/vxlan_multicast.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5"/> <alias name="nperf_num_parallel" value="1"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="vxlan_multicast.mapping" /> <alias name="net" value="192.168.0"/> <alias name="vxlan_net" value="192.168.100"/> diff --git a/recipes/regression_tests/phase3/vxlan_remote.py b/recipes/regression_tests/phase3/vxlan_remote.py index 712c34b..12050d9 100644 --- a/recipes/regression_tests/phase3/vxlan_remote.py +++ b/recipes/regression_tests/phase3/vxlan_remote.py @@ -36,6 +36,7 @@ nperf_cpupin = ctl.get_alias("nperf_cpupin") nperf_cpu_util = ctl.get_alias("nperf_cpu_util") nperf_num_parallel = int(ctl.get_alias("nperf_num_parallel")) nperf_debug = ctl.get_alias("nperf_debug") +nperf_max_dev = ctl.get_alias("nperf_max_dev") pr_user_comment = ctl.get_alias("perfrepo_comment")

pr_comment = generate_perfrepo_comment([m1, m2], pr_user_comment) @@ -96,6 +97,7 @@ if ipv in [ 'ipv4', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts": nperf_opts}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -128,6 +130,7 @@ if ipv in [ 'ipv4', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts": nperf_opts}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -165,6 +168,7 @@ if ipv in [ 'ipv6', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts" : nperf_opts + " -6"}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) @@ -197,6 +201,7 @@ if ipv in [ 'ipv6', 'both' ]: "cpu_util" : nperf_cpu_util, "runs": nperf_max_runs, "debug": nperf_debug, + "max_deviation": nperf_max_dev, "netperf_opts" : nperf_opts + "-6"}, baseline = baseline, timeout = (netperf_duration + nperf_reserve)*nperf_max_runs) diff --git a/recipes/regression_tests/phase3/vxlan_remote.xml b/recipes/regression_tests/phase3/vxlan_remote.xml index 2290458..040479e 100644 --- a/recipes/regression_tests/phase3/vxlan_remote.xml +++ b/recipes/regression_tests/phase3/vxlan_remote.xml @@ -8,6 +8,7 @@ <alias name="nperf_max_runs" value="5"/> <alias name="nperf_num_parallel" value="1"/> <alias name="nperf_debug" value="0"/> + <alias name="nperf_max_dev" value="20%"/> <alias name="mapping_file" value="vxlan_remote.mapping" /> <alias name="net" value="192.168.0"/> <alias name="vxlan_net" value="192.168.100"/>

-- 2.9.0

Ondrej Lichtner

6:12 a.m.

New subject: [PATCH v2 6/7] Common: create Version.py

From: Ondrej Lichtner olichtne@redhat.com

Separating the version number into it's own file to avoid dependency resolution issues.

Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- lnst/Common/Config.py | 3 +-- lnst/Common/Version.py | 14 ++++++++++++++ setup.py | 2 +- 3 files changed, 16 insertions(+), 3 deletions(-) create mode 100644 lnst/Common/Version.py

diff --git a/lnst/Common/Config.py b/lnst/Common/Config.py index 7b4d17e..0b8b03e 100644 --- a/lnst/Common/Config.py +++ b/lnst/Common/Config.py @@ -17,11 +17,10 @@ import subprocess from lnst.Common.Utils import bool_it from lnst.Common.NetUtils import verify_mac_address from lnst.Common.Colours import get_preset_conf +from lnst.Common.Version import LNSTMajorVersion

DefaultRPCPort = 9999

-LNSTMajorVersion = '11' - class ConfigError(Exception): pass

diff --git a/lnst/Common/Version.py b/lnst/Common/Version.py new file mode 100644 index 0000000..bafbb83 --- /dev/null +++ b/lnst/Common/Version.py @@ -0,0 +1,14 @@ +""" +Module containing just the version number of the package, needs to be separated +to avoid dependency resolution conflicst during setup.py execution. + +Copyright 2016 Red Hat, Inc. +Licensed under the GNU General Public License, version 2 as +published by the Free Software Foundation; see COPYING for details. +""" + +__autor__ = """ +olichtne@redhat.com (Ondrej Lichtner) +""" + +LNSTMajorVersion = '11' diff --git a/setup.py b/setup.py index 144e25c..41b86b3 100755 --- a/setup.py +++ b/setup.py @@ -23,7 +23,7 @@ import gzip import os from time import gmtime, strftime from distutils.core import setup -from lnst.Common.Config import LNSTMajorVersion +from lnst.Common.Version import LNSTMajorVersion

def process_template(template_path, values): template_name_re = ".in$"

-- 2.9.0

Ondrej Lichtner

6:12 a.m.

New subject: [PATCH v2 7/7] Config: redirect git stderr to /dev/null

From: Ondrej Lichtner olichtne@redhat.com

When trying to get the git revision fails it prints out a 'fatal: ...' error message to stderr. Since we don't care about how the git command fails we can redirect the error to /dev/null so that it's not mixed in with normal lnst output.

Signed-off-by: Ondrej Lichtner olichtne@redhat.com --- lnst/Common/Config.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lnst/Common/Config.py b/lnst/Common/Config.py index 0b8b03e..5362da1 100644 --- a/lnst/Common/Config.py +++ b/lnst/Common/Config.py @@ -39,7 +39,9 @@ class Config(): abspath = os.path.abspath(__file__) dname = os.path.dirname(abspath) os.chdir(dname) - head = subprocess.check_output(['git', 'rev-parse', 'HEAD']).strip() + with open(os.devnull, 'w') as null: + head = subprocess.check_output(['git', 'rev-parse', 'HEAD'], + stderr=null).strip() return head except subprocess.CalledProcessError: return LNSTMajorVersion

-- 2.9.0

Jan Tluka

9:17 a.m.

New subject: [PATCH v2 0/7] fixing phase3, extending Netperf and some Version bugfixes

Mon, Jul 11, 2016 at 01:12:00PM CEST, olichtne@redhat.com wrote:

...

From: Ondrej Lichtner olichtne@redhat.com

I still forgot some stuff in the "fix hash related errors" commit so I'm sending a v2.

The extension to Netperf (and the modifications to regression_tests recipes) was confirmed to be working in beaker jobs, so this can definitely be pushed.

And during testing I've also found a two issues with the new feature of checking versions. I've added the fixes to this patchset.

Acked-by: Jan Tluka jtluka@redhat.com

Jiri Prochazka

12 Jul 12 Jul

2:27 a.m.

2016-07-11 16:17 GMT+02:00 Jan Tluka jtluka@redhat.com:

...

Mon, Jul 11, 2016 at 01:12:00PM CEST, olichtne@redhat.com wrote:

...
From: Ondrej Lichtner olichtne@redhat.com

I still forgot some stuff in the "fix hash related errors" commit so I'm sending a v2.

The extension to Netperf (and the modifications to regression_tests recipes) was confirmed to be working in beaker jobs, so this can definitely be pushed.

And during testing I've also found a two issues with the new feature of checking versions. I've added the fixes to this patchset.

Acked-by: Jan Tluka jtluka@redhat.com

Acked-by: Jiri Prochazka jprochaz@redhat.com

...

LNST-developers mailing list lnst-developers@lists.fedorahosted.org https://lists.fedorahosted.org/admin/lists/lnst-developers@lists.fedorahoste...

3071

Age (days ago)

3072

Last active (days ago)

lnst-developers@lists.fedorahosted.org

9 comments

3 participants

tags (0)

participants (3)

Jan Tluka
Jiri Prochazka
Ondrej Lichtner