LNST-developers August 2012

lnst-developers@lists.fedorahosted.org

6 participants
99 discussions

[lnst] recipes: team: rename forgotten netmachineconfigs
by Jiří Pírko 31 Aug '12

31 Aug '12

commit 1a33b6e04aa4a1a9bff61f6aef420aa5c35e2be1 Author: Jiri Pirko <jiri(a)resnulli.us> Date: Thu Aug 23 15:37:15 2012 +0200 recipes: team: rename forgotten netmachineconfigs Signed-off-by: Jiri Pirko <jiri(a)resnulli.us> recipes/team/netconfig-team_ab_002.xml | 38 ++++++++++++++++++++++++++++++++ recipes/team/recipe_001.xml | 4 +- recipes/team/recipe_002.xml | 4 +- 3 files changed, 42 insertions(+), 4 deletions(-) --- diff --git a/recipes/team/netconfig-team_ab_002.xml b/recipes/team/netconfig-team_ab_002.xml new file mode 100644 index 0000000..d3ed392 --- /dev/null +++ b/recipes/team/netconfig-team_ab_002.xml @@ -0,0 +1,38 @@ +<netconfig> + <interface id="1" phys_id="1" type="eth"> + <options> + <option name="teamd_port_config"> + { + "prio": -10, + "sticky": true + } + </option> + </options> + </interface> + <interface id="2" phys_id="2" type="eth"> + <options> + <option name="teamd_port_config"> + { + "prio": 100 + } + </option> + </options> + </interface> + <interface id="3" type="team"> + <options> + <option name="teamd_config"> + { + "runner": {"name": "activebackup"}, + "link_watch": {"name": "ethtool"} + } + </option> + </options> + <slaves> + <slave id="1"/> + <slave id="2"/> + </slaves> + <addresses> + <address value="{$testip}"/> + </addresses> + </interface> +</netconfig> diff --git a/recipes/team/recipe_001.xml b/recipes/team/recipe_001.xml index b18940e..ad5a536 100644 --- a/recipes/team/recipe_001.xml +++ b/recipes/team/recipe_001.xml @@ -4,14 +4,14 @@ <alias name="testip" value="192.168.111.1/24"/> </define> <machine id="1"> - <machineconfig source="netmachineconfig-test1.xml"/> + <machineconfig source="machineconfig-test1.xml"/> <netconfig source="netconfig-team_rr.xml"/> </machine> <define> <alias name="testip" value="192.168.111.2/24"/> </define> <machine id="2"> - <machineconfig source="netmachineconfig-test2.xml"/> + <machineconfig source="machineconfig-test2.xml"/> <netconfig source="netconfig-team_rr.xml"/> </machine> </machines> diff --git a/recipes/team/recipe_002.xml b/recipes/team/recipe_002.xml index 0abe1ef..d985507 100644 --- a/recipes/team/recipe_002.xml +++ b/recipes/team/recipe_002.xml @@ -4,14 +4,14 @@ <alias name="testip" value="192.168.111.1/24"/> </define> <machine id="1"> - <machineconfig source="netmachineconfig-test1.xml"/> + <machineconfig source="machineconfig-test1.xml"/> <netconfig source="netconfig-team_ab_001.xml"/> </machine> <define> <alias name="testip" value="192.168.111.2/24"/> </define> <machine id="2"> - <machineconfig source="netmachineconfig-test2.xml"/> + <machineconfig source="machineconfig-test2.xml"/> <netconfig source="netconfig-team_ab_001.xml"/> </machine> </machines>

1 0

hwaddr for layered stacks
by Jan Tluka 31 Aug '12

31 Aug '12

Hi guys, I came over specific case that results in exception. I have eth interface put into vlan. LNST has problems with resolving hwaddr function template for the vlan interface. The traceback File "/home/igyn/tmp/lnst/Common/XmlTemplates.py", line 202, in _process_func_template result = self._call_preprocessor_func(func_name, param_values) File "/home/igyn/tmp/lnst/Common/XmlTemplates.py", line 210, in _call_preprocessor_func result = self._hwaddr_func(params) File "/home/igyn/tmp/lnst/Common/XmlTemplates.py", line 262, in _hwaddr_func mac_addr = machine['netconfig'][if_id]['hwaddr'] KeyError: 'hwaddr' The netconfig snippet: <netconfig> <interface id="1" phys_id="1" type="eth" /> <interface id="201" type="vlan"> <options> <option name="vlan_tci" value="22" /> </options> <slaves> <slave id="1" /> </slaves> <addresses> <address value="192.168.122.201/24"/> </addresses> </interface> </netconfig> The command snippet: <command machine_id="1" type="test" value="PktgenTx"> <options> <option name="netdev_name" value="{devname(1,100)}" /> <option name="addr" value="{ip(2,201)}" /> <option name="hwaddr" value="{hwaddr(2,201)}" /> <option name="skb_clone" value="0" /> <option name="count" value="1000000" /> </options> </command> Since the 'hwaddr' is added for physical interfaces only (having phys_id attribute), we're screwed in case of vlan, bond, etc. hwaddr resolutions. This is needed for example for PktGenTx test case that needs mac address where to sent traffic. I'd like to get your opinions whether we should set hwaddr in some cases. For example in case of vlan over eth the hw address of vlan interface is the same as underlying physical interface. On the other hand, in case of bonding the hwaddr does not have to be stable, so the hwaddr should not be set. Another approach would be to use the underlying hwaddr as a workaround. It would work for vlan case. Will it work in case of bonding as well? I'd be happy for any suggestions, comments. Thanks, Jan

2 1

[PATCH] NetTest: Adding MachinePool module
by Radek Pazdera 30 Aug '12

30 Aug '12

This commit introduces MachinePool into NetTestController. MachinePool will be responsible for managing local machine configs at controller. NetTestController will search for available configs on startup and then be able to use the machines for testing. Template matching of test setups against the pool should be done in the future. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> --- NetTest/MachinePool.py | 61 ++++++++++++++++++++++++++++++++++++++++++ NetTest/NetTestController.py | 3 ++ 2 files changed, 64 insertions(+), 0 deletions(-) create mode 100644 NetTest/MachinePool.py diff --git a/NetTest/MachinePool.py b/NetTest/MachinePool.py new file mode 100644 index 0000000..2712f09 --- /dev/null +++ b/NetTest/MachinePool.py @@ -0,0 +1,61 @@ +""" +This module contains implementaion of MachinePool class that +can be used to maintain a cluster of test machines. + +Copyright 2012 Red Hat, Inc. +Licensed under the GNU General Public License, version 2 as +published by the Free Software Foundation; see COPYING for details. +""" + +__author__ = """ +rpazdera(a)redhat.com (Radek Pazdera) +""" + +import logging +import os +import re +from Common.XmlProcessing import XmlDomTreeInit +from NetTestParse import MachineConfigParse + +class MachinePool: + """ This class is responsible for managing test machines + that are available at controler and can be used for + testing via template matching + """ + + _machines = {} + + def __init__(self, pool_dirs): + for pool_dir in pool_dirs: + self.add_dir(pool_dir) + + def add_dir(self, pool_dir): + dentries = os.listdir(pool_dir) + + for dirent in dentries: + self.add_file("%s/%s" % (pool_dir, dirent)) + + def add_file(self, filepath): + if os.path.isfile(filepath) and re.search("\.xml$", filepath, re.I): + dom_init = XmlDomTreeInit() + dom = dom_init.parse_file(filepath) + + dirname, basename = os.path.split(filepath) + + parser = MachineConfigParse() + parser.set_include_root(dirname) + parser.disable_events() + + machine = {"info": {}, "netdevices": {}} + machine_id = re.sub("\.xml$", "", basename, flags=re.I) + parser.set_machine(machine_id, machine) + + machineconfig = dom.getElementsByTagName("machineconfig")[0] + parser.parse(machineconfig) + self._machines[machine_id] = machine + + def get_machines(self): + return self._machines + + def match_setup(self, templates): + pass diff --git a/NetTest/NetTestController.py b/NetTest/NetTestController.py index 41707c5..b028e35 100644 --- a/NetTest/NetTestController.py +++ b/NetTest/NetTestController.py @@ -26,6 +26,7 @@ from NetTest.NetTestCommand import NetTestCommand, str_command from Common.LoggingServer import LoggingServer from Common.VirtUtils import VirtNetCtl, VirtDomainCtl, BridgeCtl from Common.Utils import wait_for +from NetTest.MachinePool import MachinePool MAC_POOL_RANGE = {"start": "52:54:01:00:00:01", "end": "52:54:01:FF:FF:FF"} @@ -43,6 +44,8 @@ class NetTestController: self._res_serializer = res_serializer self._remote_capture_files = {} + self._machine_pool = MachinePool([]) + self._recipe = {} definitions = {"recipe": self._recipe} -- 1.7.7.6

1 0

[lnst trac] #12: Rename netmachineconfig to machineconfig
by fedora-badges 29 Aug '12

29 Aug '12

#12: Rename netmachineconfig to machineconfig -------------------------+----------------------- Reporter: rpazdera | Owner: somebody Type: enhancement | Status: new Priority: major | Milestone: Component: component1 | Version: Keywords: | Blocked By: Blocking: #10 | -------------------------+----------------------- We agreed on the meeting, that in spite the upcoming development, we'd like to change the name netmachineconfig to machineconfig, to make it shorter and more comprehensible. -- Ticket URL: <https://fedorahosted.org/lnst/ticket/12> lnst <http://example.org/> My example project

1 2

[lnst] recipes: Adapt recipes to the netmachineconfig change
by Jiří Pírko 23 Aug '12

23 Aug '12

commit f948351f6a5e900506d4513c4ba3d99f60fdc42d Author: Radek Pazdera <rpazdera(a)redhat.com> Date: Thu Aug 23 13:55:35 2012 +0200 recipes: Adapt recipes to the netmachineconfig change I changed the old netmachineconfig tags in all recipes within the recipe directory. I couldn't test all the changes though, because I have a different test setup. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> recipes/multicast/multicast-all.xml | 8 ++++---- ...ineconfig-test1.xml => machineconfig-test1.xml} | 4 ++-- ...ineconfig-test2.xml => machineconfig-test2.xml} | 4 ++-- recipes/team/recipe_001.xml | 4 ++-- recipes/team/recipe_002.xml | 4 ++-- 5 files changed, 12 insertions(+), 12 deletions(-) --- diff --git a/recipes/multicast/multicast-all.xml b/recipes/multicast/multicast-all.xml index f4dc82c..0796059 100644 --- a/recipes/multicast/multicast-all.xml +++ b/recipes/multicast/multicast-all.xml @@ -10,7 +10,7 @@ </define> <machines> <machine id="1"> - <netmachineconfig> + <machineconfig> <info hostname="{$hostname1}" libvirt_domain="{$virtdomain1}" rootpass="redhat"/> <netdevices> @@ -19,7 +19,7 @@ <netdevice network="tnet" phys_id="2" type="eth"/> </libvirt_create> </netdevices> - </netmachineconfig> + </machineconfig> <netconfig> <interface id="1" phys_id="1" type="eth"> <addresses> @@ -34,7 +34,7 @@ </netconfig> </machine> <machine id="2"> - <netmachineconfig> + <machineconfig> <info hostname="{$hostname2}" libvirt_domain="{$virtdomain2}" rootpass="redhat"/> <netdevices> @@ -42,7 +42,7 @@ <netdevice network="tnet" phys_id="1" type="eth"/> </libvirt_create> </netdevices> - </netmachineconfig> + </machineconfig> <netconfig> <interface id="1" phys_id="1" type="eth"> <addresses> diff --git a/recipes/team/netmachineconfig-test1.xml b/recipes/team/machineconfig-test1.xml similarity index 86% rename from recipes/team/netmachineconfig-test1.xml rename to recipes/team/machineconfig-test1.xml index dc01321..1591a01 100644 --- a/recipes/team/netmachineconfig-test1.xml +++ b/recipes/team/machineconfig-test1.xml @@ -1,7 +1,7 @@ -<netmachineconfig> +<machineconfig> <info hostname="192.168.122.182" rootpass="aaa"/> <netdevices> <netdevice type="eth" phys_id="1" hwaddr="52:54:00:3d:c7:6d" network="testing"/> <netdevice type="eth" phys_id="2" hwaddr="52:54:00:73:15:c2" network="testing"/> </netdevices> -</netmachineconfig> +</machineconfig> diff --git a/recipes/team/netmachineconfig-test2.xml b/recipes/team/machineconfig-test2.xml similarity index 86% rename from recipes/team/netmachineconfig-test2.xml rename to recipes/team/machineconfig-test2.xml index 5272314..10a2cf7 100644 --- a/recipes/team/netmachineconfig-test2.xml +++ b/recipes/team/machineconfig-test2.xml @@ -1,7 +1,7 @@ -<netmachineconfig> +<machineconfig> <info hostname="192.168.122.223" rootpass="aaa"/> <netdevices> <netdevice type="eth" phys_id="1" hwaddr="52:54:00:99:bb:27" network="testing"/> <netdevice type="eth" phys_id="2" hwaddr="52:54:00:b7:cc:fb" network="testing"/> </netdevices> -</netmachineconfig> +</machineconfig> diff --git a/recipes/team/recipe_001.xml b/recipes/team/recipe_001.xml index ef2f9e7..b18940e 100644 --- a/recipes/team/recipe_001.xml +++ b/recipes/team/recipe_001.xml @@ -4,14 +4,14 @@ <alias name="testip" value="192.168.111.1/24"/> </define> <machine id="1"> - <netmachineconfig source="netmachineconfig-test1.xml"/> + <machineconfig source="netmachineconfig-test1.xml"/> <netconfig source="netconfig-team_rr.xml"/> </machine> <define> <alias name="testip" value="192.168.111.2/24"/> </define> <machine id="2"> - <netmachineconfig source="netmachineconfig-test2.xml"/> + <machineconfig source="netmachineconfig-test2.xml"/> <netconfig source="netconfig-team_rr.xml"/> </machine> </machines> diff --git a/recipes/team/recipe_002.xml b/recipes/team/recipe_002.xml index 232df3b..0abe1ef 100644 --- a/recipes/team/recipe_002.xml +++ b/recipes/team/recipe_002.xml @@ -4,14 +4,14 @@ <alias name="testip" value="192.168.111.1/24"/> </define> <machine id="1"> - <netmachineconfig source="netmachineconfig-test1.xml"/> + <machineconfig source="netmachineconfig-test1.xml"/> <netconfig source="netconfig-team_ab_001.xml"/> </machine> <define> <alias name="testip" value="192.168.111.2/24"/> </define> <machine id="2"> - <netmachineconfig source="netmachineconfig-test2.xml"/> + <machineconfig source="netmachineconfig-test2.xml"/> <netconfig source="netconfig-team_ab_001.xml"/> </machine> </machines>

1 0

[lnst] NetTestParse: Renaming netmachineconfig to machineconfig
by Jiří Pírko 23 Aug '12

23 Aug '12

commit f0f43afb277bd7e118420004f0a91291ecdd6898 Author: Radek Pazdera <rpazdera(a)redhat.com> Date: Thu Aug 23 13:55:34 2012 +0200 NetTestParse: Renaming netmachineconfig to machineconfig This patch changes <netmachineconfig> tag to <machineconfig>. This change will be necessary in some future development so I went ahead and made it already. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> NetTest/NetTestParse.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) --- diff --git a/NetTest/NetTestParse.py b/NetTest/NetTestParse.py index 21bfdd5..4f54624 100644 --- a/NetTest/NetTestParse.py +++ b/NetTest/NetTestParse.py @@ -94,12 +94,12 @@ class MachineParse(RecipeParser): self._machine["netdevices"] = {} self._machine["netconfig"] = {} - scheme = {"netmachineconfig": self._netmachineconfig, + scheme = {"machineconfig": self._machineconfig, "netconfig": self._netconfig } self._process_child_nodes(node, scheme) - def _netmachineconfig(self, node, params): - subparser = NetMachineConfigParse(self) + def _machineconfig(self, node, params): + subparser = MachineConfigParse(self) subparser.set_machine(self._id, self._machine) subparser.parse(node) @@ -108,7 +108,8 @@ class MachineParse(RecipeParser): subparser.set_machine(self._id, self._machine) subparser.parse(node) -class NetMachineConfigParse(RecipeParser): + +class MachineConfigParse(RecipeParser): _machine_id = None _machine = None

1 0

[lnst] NetTestController: Write traceback to debug on error
by Jiří Pírko 23 Aug '12

23 Aug '12

commit 7289159bf7f7eb8718cccb87affc784197232d03 Author: Radek Pazdera <rpazdera(a)redhat.com> Date: Thu Aug 23 11:30:45 2012 +0200 NetTestController: Write traceback to debug on error When the recipe execution fails with an exception in NetTestController, the exception is always caught and rethrown after propper cleanup has been done. This unfortunatelly leads to loosing the original exception traceback which is very helpful for debugging. The original traceback is now logged into debug when exception is caught and then rethrown. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> Common/Logs.py | 6 ++++++ NetTest/NetTestController.py | 4 +++- NetTest/NetTestSlave.py | 7 ++----- 3 files changed, 11 insertions(+), 6 deletions(-) --- diff --git a/Common/Logs.py b/Common/Logs.py index ac0d5d3..0772f1f 100644 --- a/Common/Logs.py +++ b/Common/Logs.py @@ -12,9 +12,15 @@ jzupka(a)redhat.com (Jiri Zupka) import os, sys, shutil, datetime from logging import Formatter import logging.handlers +import traceback LOCAL_IP = "(127.0.0.1)" +def log_exc_traceback(): + cmd_type, value, tb = sys.exc_info() + exception = traceback.format_exception(cmd_type, value, tb) + logging.debug(''.join(exception)) + class FindRootPathError(Exception): def __init__(self, path, rootFolder): Exception.__init__(self, path, rootFolder) diff --git a/NetTest/NetTestController.py b/NetTest/NetTestController.py index c5a218c..41707c5 100644 --- a/NetTest/NetTestController.py +++ b/NetTest/NetTestController.py @@ -14,7 +14,7 @@ jpirko(a)redhat.com (Jiri Pirko) import logging import socket import os -from Common.Logs import Logs +from Common.Logs import Logs, log_exc_traceback from Common.SshUtils import scp_from_remote from pprint import pprint, pformat from Common.XmlRpc import ServerProxy @@ -295,6 +295,7 @@ class NetTestController: try: self._ntparse.parse_recipe() except Exception, exc: + log_exc_traceback() logging.debug("Exception raised during recipe parsing. "\ "Deconfiguring machines.") self._deconfigure_slaves() @@ -377,6 +378,7 @@ class NetTestController: try: res = self._run_recipe() except Exception, exc: + log_exc_traceback() err = exc if packet_capture: diff --git a/NetTest/NetTestSlave.py b/NetTest/NetTestSlave.py index a7722e4..8065591 100644 --- a/NetTest/NetTestSlave.py +++ b/NetTest/NetTestSlave.py @@ -11,7 +11,7 @@ __author__ = """ jpirko(a)redhat.com (Jiri Pirko) """ -from Common.Logs import Logs +from Common.Logs import Logs, log_exc_traceback import signal import select, logging import os @@ -114,10 +114,7 @@ class NetTestSlaveXMLRPC: try: return NetTestCommand(command).run() except: - import sys, traceback - cmd_type, value, tb = sys.exc_info() - exception = traceback.format_exception(cmd_type, value, tb) - logging.error(''.join(exception)) + log_exc_traceback() raise CommandException(command) def machine_cleanup(self):

1 0

[lnst] NetConfigDevice: set team slaves up
by Jiří Pírko 23 Aug '12

23 Aug '12

commit 55585bda07e29e844b4a56475b6b77cd4d7ef212 Author: Ondrej Lichtner <olichtne(a)redhat.com> Date: Thu Aug 23 11:21:35 2012 +0200 NetConfigDevice: set team slaves up In case team slaves were specified both in teamd_config and using <slave> tags, their state was not correctly set to UP. This commit fixes that. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> NetConfig/NetConfigDevice.py | 6 ++++++ 1 files changed, 6 insertions(+), 0 deletions(-) --- diff --git a/NetConfig/NetConfigDevice.py b/NetConfig/NetConfigDevice.py index 24799b5..2dd8d1a 100644 --- a/NetConfig/NetConfigDevice.py +++ b/NetConfig/NetConfigDevice.py @@ -234,6 +234,11 @@ class NetConfigDeviceTeam(NetConfigDeviceGeneric): port_netdev = self._config[slave_id] NetConfigDevice(port_netdev, self._config).down() + def _ports_up(self): + for slave_id in get_slaves(self._netdev): + port_netdev = self._config[slave_id] + NetConfigDevice(port_netdev, self._config).up() + def configure(self): self._ports_down() @@ -250,6 +255,7 @@ class NetConfigDeviceTeam(NetConfigDeviceGeneric): for slave_id in get_slaves(self._netdev): self.slave_add(slave_id) + self._ports_up() def deconfigure(self): for slave_id in get_slaves(self._netdev):

1 0

Machine Pool Roadmap
by Radek Pazdera 23 Aug '12

23 Aug '12

Hi! I was thinking about how to grasp the machine pool at controller idea and what would be the best thing to start the development with. The first thing we should do is change <netmachineconfig> tag to only <machineconfig>, because some internal code should be renamed along with this. I will look at this (hopefuly) today and send a patch. It will break all the recipes (again), but the sooner we do it, the better. After that is sorted out, we can proceed to implementing the config support so LNST controller is able to search for and load configs on startup [1]. This will consist from a number of smaller tasks: * Create new module (Common.Config?) with tools that will check for configuration in various locations and make it available to the controller. NetTestController should be able to call this and access the values. * Some small changes will be necessary in NetTestParse so the subparser NetMachineConfigParse can be used to parse the machines in the pool directory. When this is ready, we can go ahead and start working on the hard part - the templates themselves. Some further (not that big, hopefully) changes to the parser will be necessary. Basically, we'll need to incorporate only some sort of raw XML-DOM preprocessing before the interpreting starts. And than we can start workin on the matching algorithm. Does this sound ok? [1] https://fedorahosted.org/lnst/ticket/13 Cheers, -Radek

2 1

[PATCH 1/2] NetTestParse: Renaming netmachineconfig to machineconfig
by Radek Pazdera 23 Aug '12

23 Aug '12

This patch changes <netmachineconfig> tag to <machineconfig>. This change will be necessary in some future development so I went ahead and made it already. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> --- NetTest/NetTestParse.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/NetTest/NetTestParse.py b/NetTest/NetTestParse.py index 21bfdd5..4f54624 100644 --- a/NetTest/NetTestParse.py +++ b/NetTest/NetTestParse.py @@ -94,12 +94,12 @@ class MachineParse(RecipeParser): self._machine["netdevices"] = {} self._machine["netconfig"] = {} - scheme = {"netmachineconfig": self._netmachineconfig, + scheme = {"machineconfig": self._machineconfig, "netconfig": self._netconfig } self._process_child_nodes(node, scheme) - def _netmachineconfig(self, node, params): - subparser = NetMachineConfigParse(self) + def _machineconfig(self, node, params): + subparser = MachineConfigParse(self) subparser.set_machine(self._id, self._machine) subparser.parse(node) @@ -108,7 +108,8 @@ class MachineParse(RecipeParser): subparser.set_machine(self._id, self._machine) subparser.parse(node) -class NetMachineConfigParse(RecipeParser): + +class MachineConfigParse(RecipeParser): _machine_id = None _machine = None -- 1.7.7.6

1 1

← Newer
1
2
3
4
5
6
7
8
9
10
Older →

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers August 2012