LNST-developers April 2012

lnst-developers@lists.fedorahosted.org

4 participants
18 discussions

Packet Assert Proposal
by Radek Pazdera 01 May '12

01 May '12

Hi, I did a series of tests for LNST lately where I needed to check whether packets with some properties left or arrived to some interface. The usual way of doing this is to run tcpdump with some filter, grep the results for the required properties and see if you get something. I thought it might be useful to be able to do this natively in LNST. Maybe crate new command type like this: <command type="packet_assert" timeout="10" bg_id="5"> <option name="interface" value="192.168.1.100" /> <option name="filter" value="igmp" /> <option name="grep_for" value="igmp query v{$igmp_version}" /> <option name="grep_for" value="> {$multiaddr}" /> <option name="min_count" value="5" /> </command> Do you think it's a good idea or should I do it some other way/not do things like this at all? We could discuss it together with the previous sysctl proposal if you're interested. Thank you, Radek Pazdera

2 1

Sysctl Proposal
by Radek Pazdera 30 Apr '12

30 Apr '12

Hi, I was working on some tests for LNST where I needed to change kernel settings in /proc/sys a lot. Now this can be achieved by using the exec command, but I thought, that it could be a good idea to integrate this natively into LNST since this could be frequently used in kernel networking tests. There are two places (that I know of) where you'd want to call sysctl: * in the recipe with effect through the end of the recipe * inside of TestGeneric child classes with effect only inside the class I think that writing a class with the following methods might work: * set values with sysctl (original value of the option will be kept) * restore original values Each test could have an instance of this class available and the restore method would be called in destructor. And also NetTestController would have one that would be passed to the slaves in _prepare_slaves() and then restored in _cleanup_slaves(). Big plus of natively controlling sysctl calls is (in my opinion), that we could easily save the initial values and then restore them when test is over. Honza Tluka told me that you've already considered such feature and proposed, that we could meet up sometime and discuss the idea further. What do you think? Thank you, Radek Pazdera

1 0

[PATCH] Add skb_clone option to PktgenTx test
by Jan Tluka 29 Apr '12

29 Apr '12

The pktgen's skb_clone > 0 is not valid for bonds/vlans in RHEL. This is due to fix of bug https://bugzilla.redhat.com/show_bug.cgi?id=678794. Following patch adds skb_clone option that can be set to 0 for such cases. The default value remains 100000. Tested by myself on RHEL6. --- Tests/TestPktgenTx.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) diff --git a/Tests/TestPktgenTx.py b/Tests/TestPktgenTx.py index 5e48f2c..5ec500f 100644 --- a/Tests/TestPktgenTx.py +++ b/Tests/TestPktgenTx.py @@ -57,6 +57,7 @@ class TestPktgenTx(TestGeneric): addr = self.get_mopt("addr", opt_type="addr") hwaddr = self.get_mopt("hwaddr") vlan_tci = self.get_opt("vlan_tci", default=0) + skb_clone = self.get_opt("skb_clone", default=100000) exec_cmd("modprobe pktgen") @@ -67,7 +68,7 @@ class TestPktgenTx(TestGeneric): for dev_name in dev_names: pgwrkr.add_device(dev_name) pg = Pktgen("/proc/net/pktgen/%s" % dev_name) - pg.set("clone_skb 100000") + pg.set("clone_skb %s" % skb_clone) pg.set("pkt_size 60") pg.set("dst %s" % addr) pg.set("dst_mac %s" % hwaddr) -- 1.7.6.5

2 1

[lnst] Add skb_clone option to PktgenTx test
by Jiří Pírko 29 Apr '12

29 Apr '12

commit ae61c8a03824d1ba32a29106207296176c94e54f Author: Jan Tluka <jtluka(a)redhat.com> Date: Thu Apr 26 17:45:15 2012 +0200 Add skb_clone option to PktgenTx test The pktgen's skb_clone > 0 is not valid for bonds/vlans in RHEL. This is due to fix of bug https://bugzilla.redhat.com/show_bug.cgi?id=678794. Following patch adds skb_clone option that can be set to 0 for such cases. The default value remains 100000. Tested by myself on RHEL6. Tests/TestPktgenTx.py | 3 ++- 1 files changed, 2 insertions(+), 1 deletions(-) --- diff --git a/Tests/TestPktgenTx.py b/Tests/TestPktgenTx.py index 5e48f2c..5ec500f 100644 --- a/Tests/TestPktgenTx.py +++ b/Tests/TestPktgenTx.py @@ -57,6 +57,7 @@ class TestPktgenTx(TestGeneric): addr = self.get_mopt("addr", opt_type="addr") hwaddr = self.get_mopt("hwaddr") vlan_tci = self.get_opt("vlan_tci", default=0) + skb_clone = self.get_opt("skb_clone", default=100000) exec_cmd("modprobe pktgen") @@ -67,7 +68,7 @@ class TestPktgenTx(TestGeneric): for dev_name in dev_names: pgwrkr.add_device(dev_name) pg = Pktgen("/proc/net/pktgen/%s" % dev_name) - pg.set("clone_skb 100000") + pg.set("clone_skb %s" % skb_clone) pg.set("pkt_size 60") pg.set("dst %s" % addr) pg.set("dst_mac %s" % hwaddr)

1 0

[PATCH 1/2] NetTestParse: Source attr support for all tags
by Radek Pazdera 19 Apr '12

19 Apr '12

From: Radek Pazdera <rpazdera(a)redhat.com> This commit introduces a possibility of including various parts of recipe through 'source' attribute (as it were with netmachineconfig, netconfig and command_sequence tags) to all tags in the XML. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> --- NetTest/NetTestParse.py | 59 +++++++++++++++++++++++++++-------------------- 1 files changed, 34 insertions(+), 25 deletions(-) diff --git a/NetTest/NetTestParse.py b/NetTest/NetTestParse.py index c4b553b..a6991ff 100644 --- a/NetTest/NetTestParse.py +++ b/NetTest/NetTestParse.py @@ -25,6 +25,9 @@ def load_file(filename): class WrongCommandSequenceException(Exception): pass +class WrongIncludeSource(Exception): + pass + class NetTestParse: def __init__(self, recipe_path): recipe_path = os.path.expanduser(recipe_path) @@ -37,22 +40,12 @@ class NetTestParse: def _parse_machine(self, dom_machine): machine = {} - dom_netmachineconfig = dom_machine.getElementsByTagName("netmachineconfig")[0] - dom_netconfig = dom_machine.getElementsByTagName("netconfig")[0] - source = str(dom_netmachineconfig.getAttribute("source")) - if source: - file_path = self._get_referenced_xml_path(source) - netmachineconfig_xml = load_file(file_path) - else: - netmachineconfig_xml = dom_netmachineconfig.toxml() + dom_netmachineconfig = dom_machine.getElementsByTagName("netmachineconfig")[0] + netmachineconfig_xml = dom_netmachineconfig.toxml() - source = str(dom_netconfig.getAttribute("source")) - if source: - file_path = self._get_referenced_xml_path(source) - netconfig_xml = load_file(file_path) - else: - netconfig_xml = dom_netconfig.toxml() + dom_netconfig = dom_machine.getElementsByTagName("netconfig")[0] + netconfig_xml = dom_netconfig.toxml() ncparse = NetConfigParse(netmachineconfig_xml) machine["info"] = ncparse.get_machine_info() @@ -72,6 +65,8 @@ class NetTestParse: def parse_recipe(self): recipe = {} dom = parseString(self._recipe_xml_string) + + self._load_included_parts(dom) dom_nettestrecipe = dom.getElementsByTagName("nettestrecipe")[0] dom_machines_grp = dom_nettestrecipe.getElementsByTagName("machines") @@ -86,6 +81,30 @@ class NetTestParse: def get_recipe(self): return self._recipe + def _load_included_parts(self, dom_node): + if dom_node.nodeType == dom_node.ELEMENT_NODE: + source = str(dom_node.getAttribute("source")) + if source: + file_path = self._get_referenced_xml_path(source) + xml_data = load_file(file_path) + + dom = parseString(xml_data) + loaded_node = None + try: + loaded_node = dom.getElementsByTagName(dom_node.nodeName)[0] + except Exception: + err = ("No '%s' node present in included file '%s'." + % (dom_node.nodeName, file_path)) + raise WrongIncludeSource(err) + + parent = dom_node.parentNode + parent.replaceChild(loaded_node, dom_node) + self._load_included_parts(loaded_node) + return + + for child in dom_node.childNodes: + self._load_included_parts(child) + def _parse_command_option(self, dom_option, options): logging.debug("Parsing command option") option_type = str(dom_option.getAttribute("type")) @@ -193,18 +212,8 @@ class NetTestParse: def parse_recipe_command_sequence(self): sequence = [] dom_sequences = self._dom_nettestrecipe.getElementsByTagName("command_sequence") - for dom_sequence in dom_sequences: - source = str(dom_sequence.getAttribute("source")) - if source: - """ - If source attribute is present, load sequence command - from referenced xml file. - """ - file_path = self._get_referenced_xml_path(source) - xml_data = load_file(file_path) - dom = parseString(xml_data) - dom_sequence = dom.getElementsByTagName("command_sequence")[0] + for dom_sequence in dom_sequences: dom_commands = dom_sequence.getElementsByTagName("command") for dom_command in dom_commands: sequence.append(self._parse_command(dom_command)) -- 1.7.7.6

2 2

[lnst] NetTestParse: Adding aliases support
by Jiří Pírko 19 Apr '12

19 Apr '12

commit 38f1c8755865b92d2999b674fcf6cf30520c11c5 Author: Radek Pazdera <rpazdera(a)redhat.com> Date: Thu Apr 19 14:29:30 2012 +0200 NetTestParse: Adding aliases support This commit enhances NetTestParser with the ability to define and resolve aliases through the recipe. Aliases can be defined anywhere in the recipe using <define> tag. For instance: <define> <alias name="source_ip" value="192.168.0.1" /> <alias name="mask" value="24" /> </define> Referencing aliases can be done from text elements and element attributes in the XML as follows: <address value="{$source_ip}/{$mask}" /> This patch also enables accessing recipe_eval functionality in command_sequence tag through this convention. You can reference $recipe variable, for example: <command type="test" value="Multicast" machine_id="1" timeout="30"> <options> <option name="setup" value="max_groups" /> <option name="interface" value="{$recipe['machines'][1]['netconfig'][1]['addresses'][0]}" /> <option name="condition" value="max_groups > 0" /> </options> </command> Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> NetTest/NetTestParse.py | 87 +++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 81 insertions(+), 6 deletions(-) --- diff --git a/NetTest/NetTestParse.py b/NetTest/NetTestParse.py index a6991ff..10ff4fc 100644 --- a/NetTest/NetTestParse.py +++ b/NetTest/NetTestParse.py @@ -13,6 +13,7 @@ jpirko(a)redhat.com (Jiri Pirko) from xml.dom.minidom import parseString import logging import os +import re from NetConfig.NetConfigParse import NetConfigParse from NetTest.NetTestCommand import str_command @@ -62,6 +63,16 @@ class NetTestParse: machines[machine_id] = self._parse_machine(dom_machine) return machines + def _parse_definitions(self, dom_definitions_grp): + definitions = {} + for dom_definitions_item in dom_definitions_grp: + dom_aliases = dom_definitions_item.getElementsByTagName("alias") + for dom_alias in dom_aliases: + alias_name = str(dom_alias.getAttribute("name")) + alias_value = str(dom_alias.getAttribute("value")) + definitions[alias_name] = alias_value + return definitions + def parse_recipe(self): recipe = {} dom = parseString(self._recipe_xml_string) @@ -69,10 +80,18 @@ class NetTestParse: self._load_included_parts(dom) dom_nettestrecipe = dom.getElementsByTagName("nettestrecipe")[0] + dom_definitions_grp = dom_nettestrecipe.getElementsByTagName("define") + self._definitions = self._parse_definitions(dom_definitions_grp) + for define_tag in dom_definitions_grp: + parent = define_tag.parentNode + parent.removeChild(define_tag) + dom_machines_grp = dom_nettestrecipe.getElementsByTagName("machines") + self._expand_group(dom_machines_grp) recipe["machines"] = self._parse_machines(dom_machines_grp) dom_switches_grp = dom_nettestrecipe.getElementsByTagName("switches") + self._expand_group(dom_switches_grp) recipe["switches"] = self._parse_machines(dom_switches_grp) self._recipe = recipe @@ -105,6 +124,15 @@ class NetTestParse: for child in dom_node.childNodes: self._load_included_parts(child) + def _recipe_eval(self, eval_data): + try: + return str(eval("self._recipe%s" % eval_data)) + except (KeyError, IndexError): + print self._recipe + logging.error("Wrong recipe_eval value \"%s\" passed" + % eval_data) + raise Exception + def _parse_command_option(self, dom_option, options): logging.debug("Parsing command option") option_type = str(dom_option.getAttribute("type")) @@ -115,12 +143,7 @@ class NetTestParse: elif option_type == "recipe_eval": name = str(dom_option.getAttribute("name")) orig_value = str(dom_option.getAttribute("value")) - try: - value = str(eval("self._recipe%s" % orig_value)) - except (KeyError, IndexError): - logging.error("Wrong recipe_eval value \"%s\" passed" - % orig_value) - raise Exception + value = str(self._recipe_eval(orig_value)) else: logging.error("Unknown option type \"%s\"" % option_type) raise Exception("Unknown option type") @@ -212,6 +235,7 @@ class NetTestParse: def parse_recipe_command_sequence(self): sequence = [] dom_sequences = self._dom_nettestrecipe.getElementsByTagName("command_sequence") + self._expand_group(dom_sequences, recipe_eval=True) for dom_sequence in dom_sequences: dom_commands = dom_sequence.getElementsByTagName("command") @@ -220,3 +244,54 @@ class NetTestParse: self._check_sequence(sequence) self._recipe["sequence"] = sequence + + def _expand(self, node, recipe_eval=False): + if node.nodeType == node.ELEMENT_NODE: + i = 0 + num_attributes = node.attributes.length + while(i < num_attributes): + attr = node.attributes.item(i) + attr.value = self._expand_string(attr.value, recipe_eval) + i += 1 + elif node.nodeType == node.TEXT_NODE: + node.data = self._expand_string(node.data, recipe_eval) + + for child in node.childNodes: + self._expand(child, recipe_eval) + + def _expand_group(self, group, recipe_eval=False): + for node in group: + self._expand(node, recipe_eval) + + def _expand_string(self, string, recipe_eval): + eval_re = "\{\$recipe([^\{\}]+)\}" + alias_re = "\{\$([^\{\}]*)\}" + while True: + eval_match = re.search(eval_re, string) + if eval_match: + eval_string = eval_match.group(0) + eval_data = eval_match.group(1) + if recipe_eval: + string = string.replace(eval_string, + self._recipe_eval(eval_data)) + continue + else: + err = ("Accessing $recipe allowed only from command sequence: %s" + % string) + raise KeyError(err) + alias_match = re.search(alias_re, string) + if alias_match: + alias = alias_match.group(0) + alias_name = alias_match.group(1) + try: + string = string.replace(alias, + self._definitions[alias_name]) + continue + except KeyError, err: + raise Exception("Alias '%s' doesn't exist!" % str(err)) + + + if not (eval_match and alias_match): + break + + return string

1 0

[lnst] NetTestParse: Source attr support for all tags
by Jiří Pírko 19 Apr '12

19 Apr '12

commit c38ca17e9c5dfecdc9d120d14a683cc64f7e6514 Author: Radek Pazdera <rpazdera(a)redhat.com> Date: Thu Apr 19 14:29:29 2012 +0200 NetTestParse: Source attr support for all tags This commit introduces a possibility of including various parts of recipe through 'source' attribute (as it were with netmachineconfig, netconfig and command_sequence tags) to all tags in the XML. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> NetTest/NetTestParse.py | 59 +++++++++++++++++++++++++++-------------------- 1 files changed, 34 insertions(+), 25 deletions(-) --- diff --git a/NetTest/NetTestParse.py b/NetTest/NetTestParse.py index c4b553b..a6991ff 100644 --- a/NetTest/NetTestParse.py +++ b/NetTest/NetTestParse.py @@ -25,6 +25,9 @@ def load_file(filename): class WrongCommandSequenceException(Exception): pass +class WrongIncludeSource(Exception): + pass + class NetTestParse: def __init__(self, recipe_path): recipe_path = os.path.expanduser(recipe_path) @@ -37,22 +40,12 @@ class NetTestParse: def _parse_machine(self, dom_machine): machine = {} - dom_netmachineconfig = dom_machine.getElementsByTagName("netmachineconfig")[0] - dom_netconfig = dom_machine.getElementsByTagName("netconfig")[0] - source = str(dom_netmachineconfig.getAttribute("source")) - if source: - file_path = self._get_referenced_xml_path(source) - netmachineconfig_xml = load_file(file_path) - else: - netmachineconfig_xml = dom_netmachineconfig.toxml() + dom_netmachineconfig = dom_machine.getElementsByTagName("netmachineconfig")[0] + netmachineconfig_xml = dom_netmachineconfig.toxml() - source = str(dom_netconfig.getAttribute("source")) - if source: - file_path = self._get_referenced_xml_path(source) - netconfig_xml = load_file(file_path) - else: - netconfig_xml = dom_netconfig.toxml() + dom_netconfig = dom_machine.getElementsByTagName("netconfig")[0] + netconfig_xml = dom_netconfig.toxml() ncparse = NetConfigParse(netmachineconfig_xml) machine["info"] = ncparse.get_machine_info() @@ -72,6 +65,8 @@ class NetTestParse: def parse_recipe(self): recipe = {} dom = parseString(self._recipe_xml_string) + + self._load_included_parts(dom) dom_nettestrecipe = dom.getElementsByTagName("nettestrecipe")[0] dom_machines_grp = dom_nettestrecipe.getElementsByTagName("machines") @@ -86,6 +81,30 @@ class NetTestParse: def get_recipe(self): return self._recipe + def _load_included_parts(self, dom_node): + if dom_node.nodeType == dom_node.ELEMENT_NODE: + source = str(dom_node.getAttribute("source")) + if source: + file_path = self._get_referenced_xml_path(source) + xml_data = load_file(file_path) + + dom = parseString(xml_data) + loaded_node = None + try: + loaded_node = dom.getElementsByTagName(dom_node.nodeName)[0] + except Exception: + err = ("No '%s' node present in included file '%s'." + % (dom_node.nodeName, file_path)) + raise WrongIncludeSource(err) + + parent = dom_node.parentNode + parent.replaceChild(loaded_node, dom_node) + self._load_included_parts(loaded_node) + return + + for child in dom_node.childNodes: + self._load_included_parts(child) + def _parse_command_option(self, dom_option, options): logging.debug("Parsing command option") option_type = str(dom_option.getAttribute("type")) @@ -193,18 +212,8 @@ class NetTestParse: def parse_recipe_command_sequence(self): sequence = [] dom_sequences = self._dom_nettestrecipe.getElementsByTagName("command_sequence") - for dom_sequence in dom_sequences: - source = str(dom_sequence.getAttribute("source")) - if source: - """ - If source attribute is present, load sequence command - from referenced xml file. - """ - file_path = self._get_referenced_xml_path(source) - xml_data = load_file(file_path) - dom = parseString(xml_data) - dom_sequence = dom.getElementsByTagName("command_sequence")[0] + for dom_sequence in dom_sequences: dom_commands = dom_sequence.getElementsByTagName("command") for dom_command in dom_commands: sequence.append(self._parse_command(dom_command))

1 0

[PATCH 1/2] NetTestParse: Source attr support for all tags
by Radek Pazdera 19 Apr '12

19 Apr '12

From: Radek Pazdera <rpazdera(a)redhat.com> This commit introduces a possibility of including various parts of recipe through 'source' attribute (as it were with netmachineconfig, netconfig and command_sequence tags) to all tags in the XML. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> --- NetTest/NetTestParse.py | 68 +++++++++++++++++++++++++++++----------------- 1 files changed, 43 insertions(+), 25 deletions(-) diff --git a/NetTest/NetTestParse.py b/NetTest/NetTestParse.py index c4b553b..7cf6ff0 100644 --- a/NetTest/NetTestParse.py +++ b/NetTest/NetTestParse.py @@ -25,6 +25,9 @@ def load_file(filename): class WrongCommandSequenceException(Exception): pass +class WrongIncludeSource(Exception): + pass + class NetTestParse: def __init__(self, recipe_path): recipe_path = os.path.expanduser(recipe_path) @@ -37,22 +40,12 @@ class NetTestParse: def _parse_machine(self, dom_machine): machine = {} - dom_netmachineconfig = dom_machine.getElementsByTagName("netmachineconfig")[0] - dom_netconfig = dom_machine.getElementsByTagName("netconfig")[0] - source = str(dom_netmachineconfig.getAttribute("source")) - if source: - file_path = self._get_referenced_xml_path(source) - netmachineconfig_xml = load_file(file_path) - else: - netmachineconfig_xml = dom_netmachineconfig.toxml() + dom_netmachineconfig = dom_machine.getElementsByTagName("netmachineconfig")[0] + netmachineconfig_xml = dom_netmachineconfig.toxml() - source = str(dom_netconfig.getAttribute("source")) - if source: - file_path = self._get_referenced_xml_path(source) - netconfig_xml = load_file(file_path) - else: - netconfig_xml = dom_netconfig.toxml() + dom_netconfig = dom_machine.getElementsByTagName("netconfig")[0] + netconfig_xml = dom_netconfig.toxml() ncparse = NetConfigParse(netmachineconfig_xml) machine["info"] = ncparse.get_machine_info() @@ -72,6 +65,8 @@ class NetTestParse: def parse_recipe(self): recipe = {} dom = parseString(self._recipe_xml_string) + + self._load_included_parts(dom) dom_nettestrecipe = dom.getElementsByTagName("nettestrecipe")[0] dom_machines_grp = dom_nettestrecipe.getElementsByTagName("machines") @@ -86,6 +81,39 @@ class NetTestParse: def get_recipe(self): return self._recipe + def _load_included_parts(self, dom_node): + if dom_node.nodeType == dom_node.ELEMENT_NODE: + source = str(dom_node.getAttribute("source")) + if source: + file_path = self._get_referenced_xml_path(source) + xml_data = load_file(file_path) + + dom = parseString(xml_data) + loaded_node = None + try: + loaded_node = dom.getElementsByTagName(dom_node.nodeName)[0] + except Exception: + err = ("No '%s' node present in included file '%s'." + % (dom_node.nodeName, file_path)) + raise WrongIncludeSource(err) + + parent = dom_node.parentNode + parent.replaceChild(loaded_node, dom_node) + self._load_included_parts(loaded_node) + return + + for child in dom_node.childNodes: + self._load_included_parts(child) + + def _recipe_eval(self, eval_data): + try: + return str(eval("self._recipe%s" % eval_data)) + except (KeyError, IndexError): + print self._recipe + logging.error("Wrong recipe_eval value \"%s\" passed" + % eval_data) + raise Exception + def _parse_command_option(self, dom_option, options): logging.debug("Parsing command option") option_type = str(dom_option.getAttribute("type")) @@ -193,18 +221,8 @@ class NetTestParse: def parse_recipe_command_sequence(self): sequence = [] dom_sequences = self._dom_nettestrecipe.getElementsByTagName("command_sequence") - for dom_sequence in dom_sequences: - source = str(dom_sequence.getAttribute("source")) - if source: - """ - If source attribute is present, load sequence command - from referenced xml file. - """ - file_path = self._get_referenced_xml_path(source) - xml_data = load_file(file_path) - dom = parseString(xml_data) - dom_sequence = dom.getElementsByTagName("command_sequence")[0] + for dom_sequence in dom_sequences: dom_commands = dom_sequence.getElementsByTagName("command") for dom_command in dom_commands: sequence.append(self._parse_command(dom_command)) -- 1.7.7.6

2 2

[PATCH 2/2] NetTestParse: Adding aliases support
by Radek Pazdera 19 Apr '12

19 Apr '12

From: Radek Pazdera <rpazdera(a)redhat.com> This commit enhances NetTestParser with the ability to define and resolve aliases through the recipe. Aliases can be defined anywhere in the recipe using <define> tag. For instance: <define> <alias name="source_ip" value="192.168.0.1" /> <alias name="mask" value="24" /> </define> Referencing aliases can be done from text elements and element attributes in the XML as follows: <address value="{$source_ip}/{$mask}" /> This patch enables accessing recipe_eval functionality in command_sequence tag through this convention. You can reference $recipe variable, for example: <command type="test" value="Multicast" machine_id="1" timeout="30"> <options> <option name="setup" value="max_groups" /> <option name="interface" value="{$recipe['machines'][1]['netconfig'][1]['addresses'][0]}" /> <option name="condition" value="max_groups > 0" /> </options> </command> Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> --- NetTest/NetTestParse.py | 78 +++++++++++++++++++++++++++++++++++++++++++--- 1 files changed, 72 insertions(+), 6 deletions(-) diff --git a/NetTest/NetTestParse.py b/NetTest/NetTestParse.py index 7cf6ff0..10ff4fc 100644 --- a/NetTest/NetTestParse.py +++ b/NetTest/NetTestParse.py @@ -13,6 +13,7 @@ jpirko(a)redhat.com (Jiri Pirko) from xml.dom.minidom import parseString import logging import os +import re from NetConfig.NetConfigParse import NetConfigParse from NetTest.NetTestCommand import str_command @@ -62,6 +63,16 @@ class NetTestParse: machines[machine_id] = self._parse_machine(dom_machine) return machines + def _parse_definitions(self, dom_definitions_grp): + definitions = {} + for dom_definitions_item in dom_definitions_grp: + dom_aliases = dom_definitions_item.getElementsByTagName("alias") + for dom_alias in dom_aliases: + alias_name = str(dom_alias.getAttribute("name")) + alias_value = str(dom_alias.getAttribute("value")) + definitions[alias_name] = alias_value + return definitions + def parse_recipe(self): recipe = {} dom = parseString(self._recipe_xml_string) @@ -69,10 +80,18 @@ class NetTestParse: self._load_included_parts(dom) dom_nettestrecipe = dom.getElementsByTagName("nettestrecipe")[0] + dom_definitions_grp = dom_nettestrecipe.getElementsByTagName("define") + self._definitions = self._parse_definitions(dom_definitions_grp) + for define_tag in dom_definitions_grp: + parent = define_tag.parentNode + parent.removeChild(define_tag) + dom_machines_grp = dom_nettestrecipe.getElementsByTagName("machines") + self._expand_group(dom_machines_grp) recipe["machines"] = self._parse_machines(dom_machines_grp) dom_switches_grp = dom_nettestrecipe.getElementsByTagName("switches") + self._expand_group(dom_switches_grp) recipe["switches"] = self._parse_machines(dom_switches_grp) self._recipe = recipe @@ -124,12 +143,7 @@ class NetTestParse: elif option_type == "recipe_eval": name = str(dom_option.getAttribute("name")) orig_value = str(dom_option.getAttribute("value")) - try: - value = str(eval("self._recipe%s" % orig_value)) - except (KeyError, IndexError): - logging.error("Wrong recipe_eval value \"%s\" passed" - % orig_value) - raise Exception + value = str(self._recipe_eval(orig_value)) else: logging.error("Unknown option type \"%s\"" % option_type) raise Exception("Unknown option type") @@ -221,6 +235,7 @@ class NetTestParse: def parse_recipe_command_sequence(self): sequence = [] dom_sequences = self._dom_nettestrecipe.getElementsByTagName("command_sequence") + self._expand_group(dom_sequences, recipe_eval=True) for dom_sequence in dom_sequences: dom_commands = dom_sequence.getElementsByTagName("command") @@ -229,3 +244,54 @@ class NetTestParse: self._check_sequence(sequence) self._recipe["sequence"] = sequence + + def _expand(self, node, recipe_eval=False): + if node.nodeType == node.ELEMENT_NODE: + i = 0 + num_attributes = node.attributes.length + while(i < num_attributes): + attr = node.attributes.item(i) + attr.value = self._expand_string(attr.value, recipe_eval) + i += 1 + elif node.nodeType == node.TEXT_NODE: + node.data = self._expand_string(node.data, recipe_eval) + + for child in node.childNodes: + self._expand(child, recipe_eval) + + def _expand_group(self, group, recipe_eval=False): + for node in group: + self._expand(node, recipe_eval) + + def _expand_string(self, string, recipe_eval): + eval_re = "\{\$recipe([^\{\}]+)\}" + alias_re = "\{\$([^\{\}]*)\}" + while True: + eval_match = re.search(eval_re, string) + if eval_match: + eval_string = eval_match.group(0) + eval_data = eval_match.group(1) + if recipe_eval: + string = string.replace(eval_string, + self._recipe_eval(eval_data)) + continue + else: + err = ("Accessing $recipe allowed only from command sequence: %s" + % string) + raise KeyError(err) + alias_match = re.search(alias_re, string) + if alias_match: + alias = alias_match.group(0) + alias_name = alias_match.group(1) + try: + string = string.replace(alias, + self._definitions[alias_name]) + continue + except KeyError, err: + raise Exception("Alias '%s' doesn't exist!" % str(err)) + + + if not (eval_match and alias_match): + break + + return string -- 1.7.7.6

1 0

[PATCH] SshUtils: Fix hanging slaves when using -e option
by Radek Pazdera 18 Apr '12

18 Apr '12

From: Radek Pazdera <rpazdera(a)redhat.com> This patch fixes problem with haning processes on slave machines, after remote test execution using the -e option of nettestctl.py. The problem is, when `ssh' is killed on the client side, the corresponding remote `sshd' is terminated, but it doesn't signal bash with SIGHUP if there's no tty allocated. Which (in this case leaves two processes hanging on the slave machine (/bin/sh and ./nettestslave.py). The solution here is passing -t agrument to `ssh', which results in forced allocation of pseudo-tty. There's one issue with this solution: when the remotely executed command finishes, `ssh' terminates the connection which reults in printing "Connection closed by remote host" even if everything was ok (return code 0). So I also had to remove the error check associated with this message. But I tested this and it still works fine, because if the connection is closed by error, `ssh' exits with 255, LNST can recognize that and still report it as an error. Signed-off-by: Radek Pazdera <rpazdera(a)redhat.com> --- Common/SshUtils.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) diff --git a/Common/SshUtils.py b/Common/SshUtils.py index 109af71..e306976 100644 --- a/Common/SshUtils.py +++ b/Common/SshUtils.py @@ -142,7 +142,8 @@ def _remote_login(session, username, password, prompt, timeout=10): msg = "Got username prompt after password prompt" raise LoginAuthenticationError(msg, text) elif match == 3: # "Connection closed" - raise LoginError("Client said 'connection closed'", text) + logging.log(logging.INFO, "Remote command execution successful") + break elif match == 4: # "Connection refused" raise LoginError("Client said 'connection refused'", text) elif match == 5: # "Please wait" @@ -183,7 +184,7 @@ def remote_login(host, port, username, password, prompt, linesep="\n", command = "" else: command = "-C '%s'" % (command) - cmd = ("ssh -o ServerAliveInterval=5 -o ServerAliveCountMax=2 -o UserKnownHostsFile=/dev/null " + cmd = ("ssh -t -o ServerAliveInterval=5 -o ServerAliveCountMax=2 -o UserKnownHostsFile=/dev/null " "-p %s %s@%s %s" % #-o PreferredAuthentications=password (port, username, host, command)) -- 1.7.7.6

2 1

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers April 2012