LNST-developers February 2016

lnst-developers@lists.fedorahosted.org

5 participants
11 discussions

PLEASE READ - PyRecipes final discussion with drafts - Opinions requested
by Jiri Prochazka 02 Mar '16

02 Mar '16

Hello, this mail is status update on PyRecipes. I'm sorry there is no visible progress yet, but it's because I was considering several different formats and tried to satisfy both parties, which won't be probably possible. Current use cases of LNST (RedHat vs Mellanox) ============================================== Currently, we have both sides, each with their own use case scenarios. Red Hat is using LNST for performance regression testing, along with PerfRepo. Each test uses same tools (ethtool, netperf, ping). The setups are very similar, as for hosts and eth ifaces, what is changing is soft interface setup (bond, team, VLAN, ovs, bridge). Setup of soft interfaces is currently done via XML and in task is only execution of test tools and additional setup (ethtool, MTU). Currently, the code is duplicated in every task (perfrepo methods, ethtool setups, mtu setups, netperf inits and calls) so a new layer of abstraction would be welcomed in order to simplify the code and maintainability. As for Mellanox side, I can only speak about what I can see in switchdev recipes. Their approach is to define only hardware interfaces in XML and do all soft interface setting in the task. In order to create an abstraction layer a TestLib was created, which looks really appealing to me. What do RH/Mellanox expect from PyRecipes ========================================= We had a video call in January about PyRecipes with olichtne (RH) and jpirko (Mellanox). Both sides had different view of PyRecipes. Red Hat POW ----------- 1. Wants to be able to define soft and hard interfaces together in setup 2. Wants to be able to combine network setup with different tasks 3. Task should be understood as a function, with network as argument 4. Task should be generic Mellanox POW ------------ 1. Wants to get rid of ID's (hosts and interfaces) 2. Wants soft iface definition only in task, not in setup 3. Task should be specific 4. Wrappers for generic stuff 5. 1 task == 1 test == 1 file, do not combine it Proposed approach #1 (Mlx like) =============================== Description: Setup and task is in one file, no IDs are used, soft interface definition is part of task Example: import lnst m1 = lnst.add_host() m2 = lnst.add_host() m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet") m2_eth1 = m2.add_interface(label="tnet") while match(match=lnst.SingleMatch): m1_team = m1.create_team([m1_eth1, m1_eth2], ip="1.2.3.4/24") m2_eth1.reset(["1.2.3.5/24"]) ping_mod = ... m1_team.run(ping_mod) Proposed approach #2 (RH like) ============================== Description: Soft interfaces can be defined in both setup() and task() methods. IDs muset used due to different scopes of variables. Setup can be in separate file and can be imported in multiple tasks. In one file, multiple tasks can be called. Example: import lnst def setup(): m1 = lnst.add_host("m1") m2 = lnst.add_host("m2") m1_eth1 = m1.add_interface(id="eth1", label="tnet") m1_eth2 = m1.add_interface(id="eth2", label="tnet") m2_eth1 = m2.add_interface(id="eth1", label="tnet", ip="1.1.1.1/24") m1.create_team(id="team1", slaves=[m1_eth1, m1_eth2], ip="1.1.1.2/24") def task(): m1 = lnst.get_host("m1") m2 = lnst.get_host("m2") m1_team = m1.get_interface("team1") m2_eth1 = m2.get_interface("eth1") ping_mod = ... m1_team.run(ping_mod) lnst.run(match=lnst.SingleMatch, setup, task) Proposed approach #3 (RH like) ============================== Description: Task method is portable, it uses machine and interface objects as args so no IDs are required. Soft interfaces can be created in both do_task and in setup phase. In one file, multiple tasks can be called. Example: import lnst def do_task(m1, if1, if2): ping_mod = ...src=if1, dst=if2... m1.run(ping_mod) m1 = lnst.add_machine() m2 = lnst.add_machine() m1_eth1 = m1.add_interface(label="tnet") m1_eth2 = m1.add_interface(label="tnet") m1_team = m1.create_team(slaves=[m1_eth1, m1_eth2], ip="1.1.1.1/24") m2_eth1 = m2.add_interface(label="tnet", ip="1.1.1.2/24") while lnst.match(match=lnst.SingleMatch): do_task(m1, m1_team, m2_eth1) Summary ======= Drafts above are not meant to be final, it sure can be improved and modified to satisfy our needs. But we need to come to conclusion for both Mlx and RH side, so I can start working on it. Some important questions regarding PyRecipes: --------------------------------------------- I. Soft interfaces - in setup phase and task phase or only in task phase? II. Portability - one task == one recipe, or allow combinations of networks with different tasks? III. Should task be generic or specific? IV. Do we have to get rid of IDs? My opinion on the matter ======================== Favourite approach - #3 Answers to questions: --------------------- I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe mentality, will bring easier maintaining II. Portability - one task == one recipe - even our use case shows, that tests don't allow so much combination (1 task is being used in avg. 1-2 recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so important to use to preserve it III. Specific task - generic stuff can be defined by a new layer of abstraction, like TestLib in switchdev tests IV. I do not think IDs are such evil that we should get rid of it, altough I agree object oriented approach (which we want to follow, thus PyRecipes became a thing in the first place) should be only using instances of objects in both task and setup. Summary ======= Please, devs from RH and Mlx, take a look on these drafts and send an email with your opinion on the matter. Ideally, next week starting Wed I would like to have it decided so I can start implementing it. In the end, probably we will have to compromise, but hopefully, both parties will end up satisfied and all this work will lead to improving of the quality of the whole LNST. Thanks for reading, Jiri Prochazka

5 7

[PATCH 01/12] Config: add get_section_values method
by Ondrej Lichtner 01 Mar '16

01 Mar '16

From: Ondrej Lichtner <olichtne(a)redhat.com> This method returns all the option values in a specific section. Previously we just had the get_section method which returns the internal dict structure of the Config class which is not very usable in the application. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Common/Config.py | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lnst/Common/Config.py b/lnst/Common/Config.py index 97a82e4..de7ba4a 100644 --- a/lnst/Common/Config.py +++ b/lnst/Common/Config.py @@ -155,6 +155,16 @@ class Config(): raise ConfigError(msg) return self._options[section] + def get_section_values(self, section): + if section not in self._options: + msg = 'Unknow section: %s' % section + raise ConfigError(msg) + + res = {} + for opt_name, opt in self._options[section].items(): + res[opt_name] = opt["value"] + return res + def get_option(self, section, option): sect = self.get_section(section) if option not in sect: -- 2.7.2

2 12

[PATCH 10/12] SecureSocket: make cryptography import optional
by Ondrej Lichtner 01 Mar '16

01 Mar '16

From: Ondrej Lichtner <olichtne(a)redhat.com> The cryptography library is imported only when authentication methods that need it are requested. If the library is missing, the user will get an error message explaining the problem. It's a pretty hackish way of doing the imports but I'm not aware of a better solution... I'm open to suggestions if there are any. Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com> --- lnst/Common/SecureSocket.py | 72 ++++++++++++++++++++++++++++++++++------- lnst/Controller/CtlSecSocket.py | 40 +++++++++++++++++++---- lnst/Slave/SlaveSecSocket.py | 42 ++++++++++++++++++++---- 3 files changed, 131 insertions(+), 23 deletions(-) diff --git a/lnst/Common/SecureSocket.py b/lnst/Common/SecureSocket.py index d319c54..cd149c2 100644 --- a/lnst/Common/SecureSocket.py +++ b/lnst/Common/SecureSocket.py @@ -19,17 +19,6 @@ import os import cPickle import hashlib import hmac -import cryptography.exceptions -from cryptography.hazmat.primitives import hashes -from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes -from cryptography.hazmat.primitives.asymmetric import padding, ec -from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey -from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePublicKey -from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey -from cryptography.hazmat.primitives.asymmetric.rsa import RSAPublicKey -from cryptography.hazmat.primitives.asymmetric.dsa import DSAPrivateKey -from cryptography.hazmat.primitives.asymmetric.dsa import DSAPublicKey -from cryptography.hazmat.backends import default_backend DH_GROUP = {"p": int("0xFFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD1"\ "29024E088A67CC74020BBEA63B139B22514A08798E3404DD"\ @@ -75,6 +64,58 @@ if SRP_GROUP["p"].bit_length()%8: class SecSocketException(Exception): pass +cryptography = None +hashes = None +Cipher = None +algorithms = None +modes = None +padding = None +ec = None +EllipticCurvePrivateKey = None +EllipticCurvePublicKey = None +RSAPrivateKey = None +RSAPublicKey = None +DSAPrivateKey = None +DSAPublicKey = None +default_backend = None +cryptography_imported = False +def cryptography_imports(): + global cryptography_imported + if cryptography_imported: + return + + global cryptography + global hashes + global Cipher + global algorithms + global modes + global padding + global ec + global EllipticCurvePrivateKey + global EllipticCurvePublicKey + global RSAPrivateKey + global RSAPublicKey + global DSAPrivateKey + global DSAPublicKey + global default_backend + + try: + import cryptography.exceptions + from cryptography.hazmat.primitives import hashes + from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes + from cryptography.hazmat.primitives.asymmetric import padding, ec + from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey + from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePublicKey + from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey + from cryptography.hazmat.primitives.asymmetric.rsa import RSAPublicKey + from cryptography.hazmat.primitives.asymmetric.dsa import DSAPrivateKey + from cryptography.hazmat.primitives.asymmetric.dsa import DSAPublicKey + from cryptography.hazmat.backends import default_backend + cryptography_imported = True + except ImportError: + raise SecSocketException("Library 'cryptography' missing "\ + "can't establish secure channel.") + class SecureSocket(object): def __init__(self, soc): self._role = None @@ -112,6 +153,7 @@ class SecureSocket(object): def _add_mac_sign(self, data): if not self._current_write_spec["mac_key"]: return data + cryptography_imports() msg = str(self._current_write_spec["seq_num"]) + str(len(data)) + data signature = hmac.new(self._current_write_spec["mac_key"], @@ -124,6 +166,7 @@ class SecureSocket(object): def _del_mac_sign(self, signed_data): if not self._current_read_spec["mac_key"]: return signed_data + cryptography_imports() signed_msg = cPickle.loads(signed_data) data = signed_msg["data"] @@ -140,6 +183,7 @@ class SecureSocket(object): def _add_padding(self, data): if not self._current_write_spec["enc_key"]: return data + cryptography_imports() block_size = algorithms.AES.block_size/8 pad_length = block_size - (len(data) % block_size) @@ -152,6 +196,7 @@ class SecureSocket(object): def _del_padding(self, data): if not self._current_read_spec["enc_key"]: return data + cryptography_imports() pad_length = int(data[-1].encode("hex"), 16) for char in data[-pad_length]: @@ -163,6 +208,7 @@ class SecureSocket(object): def _add_encrypt(self, data): if not self._current_write_spec["enc_key"]: return data + cryptography_imports() iv = os.urandom(algorithms.AES.block_size/8) mode = modes.CBC(iv) @@ -180,6 +226,7 @@ class SecureSocket(object): def _del_encrypt(self, data): if not self._current_read_spec["enc_key"]: return data + cryptography_imports() encrypted_msg = cPickle.loads(data) encrypted_data = encrypted_msg["enc_data"] @@ -313,6 +360,7 @@ class SecureSocket(object): server_spec = self._next_read_spec else: raise SecSocketException("Socket without a role!") + cryptography_imports() aes_keysize = max(algorithms.AES.key_sizes)/8 mac_keysize = hashlib.sha256().block_size @@ -334,6 +382,7 @@ class SecureSocket(object): return def _sign_data(self, data, privkey): + cryptography_imports() if isinstance(privkey, DSAPrivateKey): signer = privkey.signer(hashes.SHA256()) elif isinstance(privkey, RSAPrivateKey): @@ -349,6 +398,7 @@ class SecureSocket(object): return signer.finalize() def _verify_signature(self, pubkey, data, signature): + cryptography_imports() if isinstance(pubkey, DSAPublicKey): verifier = pubkey.verifier(signature, hashes.SHA256()) elif isinstance(pubkey, RSAPublicKey): diff --git a/lnst/Controller/CtlSecSocket.py b/lnst/Controller/CtlSecSocket.py index b4c857a..1f16055 100644 --- a/lnst/Controller/CtlSecSocket.py +++ b/lnst/Controller/CtlSecSocket.py @@ -19,13 +19,37 @@ from lnst.Common.SecureSocket import SecureSocket from lnst.Common.SecureSocket import DH_GROUP, SRP_GROUP from lnst.Common.SecureSocket import SecSocketException from lnst.Common.Config import lnst_config -from cryptography.hazmat.primitives import serialization as ser -from cryptography.hazmat.primitives.serialization import load_pem_private_key -from cryptography.hazmat.primitives.serialization import load_pem_public_key -from cryptography.hazmat.primitives.serialization import load_ssh_public_key -from cryptography.hazmat.backends import default_backend -backend = default_backend() +ser = None +load_pem_private_key = None +load_pem_public_key = None +load_ssh_public_key = None +backend = None +cryptography_imported = False +def cryptography_imports(): + global cryptography_imported + if cryptography_imported: + return + + global ser + global load_pem_private_key + global load_pem_public_key + global load_ssh_public_key + global backend + + try: + import cryptography + import cryptography.hazmat.primitives.serialization as ser + from cryptography.hazmat.primitives.serialization import load_pem_private_key + from cryptography.hazmat.primitives.serialization import load_pem_public_key + from cryptography.hazmat.primitives.serialization import load_ssh_public_key + from cryptography.hazmat.backends import default_backend + except ImportError: + raise SecSocketException("Library 'cryptography' missing "\ + "can't establish secure channel.") + + backend = default_backend() + cryptography_imported = True class CtlSecSocket(SecureSocket): def __init__(self, soc): @@ -57,10 +81,13 @@ class CtlSecSocket(SecureSocket): logging.warning(" NO AUTHENTICATION IN PLACE") logging.warning("SECURE CHANNEL IS VULNERABLE TO MIM ATTACKS") logging.warning("===========================================") + cryptography_imports() self._dh_handshake() elif sec_params["auth_type"] == "ssh": + cryptography_imports() self._ssh_handshake() elif sec_params["auth_type"] == "pubkey": + cryptography_imports() ctl_identity = sec_params["identity"] ctl_key_path = sec_params["privkey"] try: @@ -81,6 +108,7 @@ class CtlSecSocket(SecureSocket): self._pubkey_handshake(ctl_identity, ctl_key, srv_key) elif sec_params["auth_type"] == "password": + cryptography_imports() self._passwd_handshake(sec_params["auth_passwd"]) else: raise SecSocketException("Unknown authentication method.") diff --git a/lnst/Slave/SlaveSecSocket.py b/lnst/Slave/SlaveSecSocket.py index 9e3cd92..8810bac 100644 --- a/lnst/Slave/SlaveSecSocket.py +++ b/lnst/Slave/SlaveSecSocket.py @@ -19,13 +19,39 @@ import logging from lnst.Common.SecureSocket import SecureSocket from lnst.Common.SecureSocket import DH_GROUP, SRP_GROUP from lnst.Common.SecureSocket import SecSocketException -from cryptography.hazmat.primitives import serialization as ser -from cryptography.hazmat.primitives.serialization import load_pem_private_key -from cryptography.hazmat.primitives.serialization import load_pem_public_key -from cryptography.hazmat.primitives.serialization import load_ssh_public_key -from cryptography.hazmat.backends import default_backend -backend = default_backend() +ser = None +load_pem_private_key = None +load_pem_public_key = None +load_ssh_public_key = None +backend = None +cryptography_imported = False +def cryptography_imports(): + global cryptography_imported + if cryptography_imported: + return + + global ser + global load_pem_private_key + global load_pem_public_key + global load_ssh_public_key + global backend + + try: + import cryptography + import cryptography.hazmat.primitives.serialization as ser + from cryptography.hazmat.primitives.serialization import load_pem_private_key + from cryptography.hazmat.primitives.serialization import load_pem_public_key + from cryptography.hazmat.primitives.serialization import load_ssh_public_key + from cryptography.hazmat.backends import default_backend + except ImportError: + logging.error("Library 'cryptography' missing "\ + "can't establish secure channel.") + raise SecSocketException("Library 'cryptography' missing "\ + "can't establish secure channel.") + + backend = default_backend() + cryptography_imported = True class SlaveSecSocket(SecureSocket): def __init__(self, soc): @@ -55,10 +81,13 @@ class SlaveSecSocket(SecureSocket): logging.warning(" NO AUTHENTICATION IN PLACE") logging.warning("SECURE CHANNEL IS VULNERABLE TO MIM ATTACKS") logging.warning("===========================================") + cryptography_imports() self._dh_handshake() elif sec_params["auth_types"] == "ssh": + cryptography_imports() self._ssh_handshake() elif sec_params["auth_types"] == "pubkey": + cryptography_imports() srv_key = None with open(sec_params["privkey"], 'r') as f: srv_key = load_pem_private_key(f.read(), None, backend) @@ -73,6 +102,7 @@ class SlaveSecSocket(SecureSocket): self._pubkey_handshake(srv_key, ctl_pubkeys) elif sec_params["auth_types"] == "password": + cryptography_imports() self._passwd_handshake(sec_params["auth_password"]) else: raise SecSocketException("Unknown authentication method.") -- 2.7.2

2 6

[PATCH 0/7] RFC implementing secure Controller<->Slave communication
by Ondrej Lichtner 29 Feb '16

29 Feb '16

From: Ondrej Lichtner <olichtne(a)redhat.com> First of all, this isn't a patchset to be pulled into the repository yet, just a request for comments to see if anything can be improved, while I work on finishing some minor problem areas: * this breaks the lnst-pool-wizard because it doesn't use SecureSockets yet * this breaks the lnst-ctl deconfigure command for the same reason The following patch set implements secure communication between the Controller and the Slave. It implements 4 different authentication mechanisms: none - no authentication, just DH secret negotiation and encryption password - password authenticated secret negotiation using SRP-6a protocol and encryption pubkey - DH secret negotiation, authentication by using a private-public key pairs ssh - the same as pubkey, but uses ssh keys already present on the system By default, both the Controller and Slave assume the "none" mechanism so everything should work just as before without any additional work needed. To try the other mechanisms you'll need to configure both the Controller and the Slave as follows: password: on Slave edit the lnst-slave.conf like this: [security] auth_types = password auth_password = your_chosen_password on Controller edit the SlaveMachineXML file of the slave and add this: <security> <auth_type>password</auth_type> <auth_password>your_chosen_password</auth_password> </security> under the main <slavemachine> element pubkey: generate a private-public key pair on both the Slave and the Controller and exchange the public keys between the machines. on Slave edit the lnst-slave.conf file like this: [security] auth_types = pubkey privkey = path/to/slave_private_key ctl_pubkeys = path/to/pubkeys_dir/ on Controller edit the SlaveMachineXML and add this: <security> <auth_type>pubkey</auth_type> <pubkey_path>path/to/slave_pubkey</pubkey_path> </security> under the main <slavemachine> element and edit the lnst-ctl.conf like this: [security] identity = ctl_name privkey = path/to/ctl_privatekey On the Slave the Controller public key needs to be placed into the ctl_pubkeys directory and the filename must match the value of the identity option in the lnst-ctl.conf ssh: have ssh presetup between the Controller and Slave machines - the slave needs its sshd private keys to be located in /etc/ssh (ssh_host.*key files) and the controllers public key must be in the ~/.ssh/authorized_keys file On the Controller you need to have the ~/.ssh/id_rsa file to contain a private RSA key (the public part is in the slaves authorized_keys file) and you also need to have at least one of the Slaves public keys located in the ~/.ssh/known_hosts file you can easily achieve this by just starting the sshd daemon on the Slave machine and on the Controller generate the key with ssh-keygen -t rsa -C "your_email(a)example.com" and run ssh-copy-id username@slavehostname then configure the LNST Slave - lnst-slave.conf: [security] auth_types = ssh and on the Controller edit the Slave Machine XML file like this: <security> <auth_type>pubkey</auth_type> <pubkey_path>path/to/slave_pubkey</pubkey_path> </security> Ondrej Lichtner (7): Config: add get_section_values method Config: add security section and options to config Add SecureSocket classes SlaveMachineXML: add security information Machine: add security parameters of the slave machine Use SecureSockets for Ctl<->Slave communication Machine: rename method configure to init_connection lnst/Common/Config.py | 44 ++++ lnst/Common/ConnectionHandler.py | 39 +--- lnst/Common/SecureSocket.py | 377 ++++++++++++++++++++++++++++++++++ lnst/Controller/CtlSecSocket.py | 318 ++++++++++++++++++++++++++++ lnst/Controller/Machine.py | 17 +- lnst/Controller/NetTestController.py | 2 +- lnst/Controller/SlaveMachineParser.py | 31 +++ lnst/Controller/SlavePool.py | 9 +- lnst/Slave/NetTestSlave.py | 13 +- lnst/Slave/SlaveSecSocket.py | 323 +++++++++++++++++++++++++++++ schema-sm.rng | 29 +++ 11 files changed, 1163 insertions(+), 39 deletions(-) create mode 100644 lnst/Common/SecureSocket.py create mode 100644 lnst/Controller/CtlSecSocket.py create mode 100644 lnst/Slave/SlaveSecSocket.py -- 2.7.1

2 10

[PATCH] recipes: switchdev: Test LAG leave flow
by Ido Schimmel 28 Feb '16

28 Feb '16

When port netdevs are leaving a bridged LAG device, then the bridge's cleanup sequence isn't invoked and the switch driver needs to do the necessary cleanup. Adjust current LAG recipes to test the flow. Note that I'm creating a new LAG device on the switch for the second fastpath test, as currently switchdev drivers suffer from an ordering problem, which first requires the LAG to be populated and only then bridged. Hopefully, this will be resolved soon. Signed-off-by: Ido Schimmel <idosch(a)mellanox.com> --- recipes/switchdev/l2-006-bridge_team.py | 28 ++++++++++++++- recipes/switchdev/l2-015-bridge_team_vlan1d.py | 48 ++++++++++++++++++++++++-- 2 files changed, 72 insertions(+), 4 deletions(-) diff --git a/recipes/switchdev/l2-006-bridge_team.py b/recipes/switchdev/l2-006-bridge_team.py index fd648c9..48b9752 100644 --- a/recipes/switchdev/l2-006-bridge_team.py +++ b/recipes/switchdev/l2-006-bridge_team.py @@ -31,7 +31,8 @@ def do_task(ctl, hosts, ifaces, aliases): sw_lag2 = sw.create_team(slaves=[sw_if3, sw_if4], config=team_config) - sw.create_bridge(slaves=[sw_lag1, sw_lag2], options={"vlan_filtering": 1}) + sw_br = sw.create_bridge(slaves=[sw_lag1, sw_lag2], + options={"vlan_filtering": 1}) sleep(15) @@ -40,6 +41,31 @@ def do_task(ctl, hosts, ifaces, aliases): tl.netperf_tcp(m1_lag1, m2_lag1) tl.netperf_udp(m1_lag1, m2_lag1) + sw_lag1.slave_del(sw_if1.get_id()) + sw_lag1.slave_del(sw_if2.get_id()) + + m1_lag1.slave_del(m1_if1.get_id()) + + # Make sure slowpath is working. + sw_if1.reset(ip=["192.168.102.10/24", "2003::1/64"]) + m1_if1.reset(ip=["192.168.102.11/24", "2003::2/64"]) + + sleep(15) + + tl.ping_simple(sw_if1, m1_if1) + + # Repopulate the LAGs and make sure fastpath is OK. + sw_lag3 = sw.create_team(slaves=[sw_if1, sw_if2], + config=team_config) + sw_br.slave_add(sw_lag3.get_id()) + m1_lag1.slave_add(m1_if1.get_id()) + + sleep(15) + + tl.ping_simple(m1_lag1, m2_lag1) + tl.netperf_tcp(m1_lag1, m2_lag1) + tl.netperf_udp(m1_lag1, m2_lag1) + do_task(ctl, [ctl.get_host("machine1"), ctl.get_host("machine2"), ctl.get_host("switch")], diff --git a/recipes/switchdev/l2-015-bridge_team_vlan1d.py b/recipes/switchdev/l2-015-bridge_team_vlan1d.py index 3bea6e2..723a04a 100644 --- a/recipes/switchdev/l2-015-bridge_team_vlan1d.py +++ b/recipes/switchdev/l2-015-bridge_team_vlan1d.py @@ -34,15 +34,17 @@ def do_task(ctl, hosts, ifaces, aliases): sw_lag1 = sw.create_team(slaves=[sw_if1, sw_if2], config=team_config) sw_lag2 = sw.create_team(slaves=[sw_if3, sw_if4], config=team_config) br_options = {"vlan_filtering": 1} - sw.create_bridge(slaves=[sw_lag1, sw_lag2], options=br_options) + sw_br1 = sw.create_bridge(slaves=[sw_lag1, sw_lag2], options=br_options) sw_lag1_10 = sw.create_vlan(sw_lag1, 10) sw_lag2_10 = sw.create_vlan(sw_lag2, 10) - sw.create_bridge(slaves=[sw_lag1_10, sw_lag2_10], options=br_options) + sw_br2 = sw.create_bridge(slaves=[sw_lag1_10, sw_lag2_10], + options=br_options) sw_lag1_20 = sw.create_vlan(sw_lag1, 20) sw_lag2_21 = sw.create_vlan(sw_lag2, 21) - sw.create_bridge(slaves=[sw_lag1_20, sw_lag2_21], options=br_options) + sw_br3 = sw.create_bridge(slaves=[sw_lag1_20, sw_lag2_21], + options=br_options) sleep(15) @@ -60,6 +62,46 @@ def do_task(ctl, hosts, ifaces, aliases): tl.netperf_tcp(m1_lag1_20, m2_lag1_21) tl.netperf_udp(m1_lag1_20, m2_lag1_21) + sw_lag1.slave_del(sw_if1.get_id()) + sw_lag1.slave_del(sw_if2.get_id()) + + m1_lag1.slave_del(m1_if1.get_id()) + + # Make sure slowpath is working. + sw_if1.reset(ip=test_ip(4, 1)) + m1_if1.reset(ip=test_ip(4, 2)) + + sleep(15) + + tl.ping_simple(sw_if1, m1_if1) + + # Repopulate the LAGs and make sure fastpath is OK. + sw_lag3 = sw.create_team(slaves=[sw_if1, sw_if2], + config=team_config) + sw_br1.slave_add(sw_lag3.get_id()) + + sw_lag3_10 = sw.create_vlan(sw_lag3, 10) + sw_br2.slave_add(sw_lag3_10.get_id()) + + sw_lag3_20 = sw.create_vlan(sw_lag3, 20) + sw_br3.slave_add(sw_lag3_20.get_id()) + + m1_lag1.slave_add(m1_if1.get_id()) + + sleep(15) + + tl.ping_simple(m1_lag1, m2_lag1) + tl.netperf_tcp(m1_lag1, m2_lag1) + tl.netperf_udp(m1_lag1, m2_lag1) + + tl.ping_simple(m1_lag1_10, m2_lag1_10) + tl.netperf_tcp(m1_lag1_10, m2_lag1_10) + tl.netperf_udp(m1_lag1_10, m2_lag1_10) + + tl.ping_simple(m1_lag1_20, m2_lag1_21) + tl.netperf_tcp(m1_lag1_20, m2_lag1_21) + tl.netperf_udp(m1_lag1_20, m2_lag1_21) + do_task(ctl, [ctl.get_host("machine1"), ctl.get_host("machine2"), ctl.get_host("switch")], -- 2.4.10

2 1

[PATCH] Controller: remove libvirt root user restriction
by Jan Tluka 18 Feb '16

18 Feb '16

It is possible to manage virtual machines without root privileges. The check for root is therefore no more required. Follow further steps to allow non-root user to use LNST for virtual setups. 1. First you need to use policy kit to tweak libvirtd. # echo 'polkit.addRule(function(action, subject) { if (action.id == "org.libvirt.unix.manage" && subject.isInGroup("wheel")) { return polkit.Result.YES; } }); ' > /etc/polkit-1/rules.d/80-libvirt.rules then (re)start libvirtd service, # systemctl enable libvirtd # systemctl start libvirtd 2. Add the non-root user to group 'wheel'. 3. The last bit is to run lnst-ctl with following environment variable exported: LIBVIRT_DEFAULT_URI=qemu:///system lnst-ctl ... Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- lnst/Controller/NetTestController.py | 6 ------ 1 file changed, 6 deletions(-) diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py index cce1fec..8fdd43e 100644 --- a/lnst/Controller/NetTestController.py +++ b/lnst/Controller/NetTestController.py @@ -232,12 +232,6 @@ class NetTestController: msg = "This setup cannot be provisioned with the current pool." raise NoMatchError(msg) - if sp.is_setup_virtual() and os.geteuid() != 0: - msg = "Provisioning this setup requires additional configuration "\ - "of the virtual hosts in the pool. LNST needs root "\ - "priviledges so it can connect to qemu." - raise NetTestError(msg) - def print_match_description(self): sp = self._slave_pool match = sp.get_match() -- 2.4.3

1 0

[PATCH] recipes: remove duplicate wait from simple_netperf task
by Jan Tluka 18 Feb '16

18 Feb '16

There were two ctl.wait() calls in the task. One is redundant so I removed it. Signed-off-by: Jan Tluka <jtluka(a)redhat.com> --- recipes/regression_tests/phase1/simple_netperf.py | 2 -- 1 file changed, 2 deletions(-) diff --git a/recipes/regression_tests/phase1/simple_netperf.py b/recipes/regression_tests/phase1/simple_netperf.py index 6762a60..407ee5d 100644 --- a/recipes/regression_tests/phase1/simple_netperf.py +++ b/recipes/regression_tests/phase1/simple_netperf.py @@ -53,8 +53,6 @@ if nperf_cpupin: for m, d in [ (m1, m1_testiface), (m2, m2_testiface) ]: pin_dev_irqs(m, d, 0) -ctl.wait(15) - p_opts = "-L %s" % (m2_testiface.get_ip(0)) if nperf_cpupin and nperf_mode != "multi": p_opts += " -T%s,%s" % (nperf_cpupin, nperf_cpupin) -- 2.4.3

1 0

[PATCH] recipes: switchdev: Test toggling of VLAN flags on 802.1q bridge
by Ido Schimmel 18 Feb '16

18 Feb '16

Signed-off-by: Ido Schimmel <idosch(a)mellanox.com> --- recipes/switchdev/l2-008-bridge_vlan1q_sanity.py | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/recipes/switchdev/l2-008-bridge_vlan1q_sanity.py b/recipes/switchdev/l2-008-bridge_vlan1q_sanity.py index 5572017..555891b 100644 --- a/recipes/switchdev/l2-008-bridge_vlan1q_sanity.py +++ b/recipes/switchdev/l2-008-bridge_vlan1q_sanity.py @@ -78,6 +78,22 @@ def do_task(ctl, hosts, ifaces, aliases): sleep(1) tl.ping_simple(m1_if1, m2_if1) + sw_if2.add_br_vlan(500, pvid=True, untagged=False) + sleep(1) + tl.ping_simple(m1_if1, m2_if1, fail_expected=True) + + sw_if2.add_br_vlan(500, pvid=True, untagged=True) + sleep(1) + tl.ping_simple(m1_if1, m2_if1) + + sw_if2.add_br_vlan(500, pvid=False, untagged=True) + sleep(1) + tl.ping_simple(m1_if1, m2_if1, fail_expected=True) + + sw_if2.add_br_vlan(500, pvid=True, untagged=True) + sleep(1) + tl.ping_simple(m1_if1, m2_if1) + sw_br.slave_del(sw_if1.get_id()) sleep(5) -- 2.4.10

2 1

[PATCH v2 1/4] lnst: Fix dynamic addition / deletion of slaves
by Ido Schimmel 17 Feb '16

17 Feb '16

The slaves of an interface are determined during creation time and stored in two objects: * Interface on the controller * NetConfigDeviceGeneric on the slave machine When slaves are dynamically added / deleted during a Python task the 'slaves' data structures in these objects are not updated, which can cause several problems: If a slave is removed from its master, then during cleanup the slave machine will try to remove it again and thereby generate an error. This requires us to update the data struct on the slave machine. If a slave is added to a master and that master is later reset(), then the newly added slave won't be configured correctly after the reset(). This requires us to update the data struct on the controller. Cc: Andrew Lunn <andrew(a)lunn.ch> Fixes: af025186a329 ("InterfaceAPI: Introduce slave_{add, del}") Signed-off-by: Ido Schimmel <idosch(a)mellanox.com> --- v1->v2: * Align team devices with bridge and only add / delete slaves from the "slaves" list when dynamically configured. --- lnst/Controller/Machine.py | 12 ++++++++++++ lnst/Slave/NetConfigDevice.py | 18 ++++++++++++++---- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/lnst/Controller/Machine.py b/lnst/Controller/Machine.py index 884b315..9897580 100644 --- a/lnst/Controller/Machine.py +++ b/lnst/Controller/Machine.py @@ -591,6 +591,12 @@ class Interface(object): else: self._master["other"].append(master) + def del_master(self, master): + if self._master["primary"] is master: + self._master["primary"] = None + else: + self._master["other"].remove(master) + def get_primary_master(self): return self._master["primary"] @@ -601,6 +607,10 @@ class Interface(object): else: iface.add_master(self) + def del_slave(self, iface): + iface.del_master(self) + del self._slaves[iface.get_id()] + def set_slave_option(self, slave_id, name, value): if slave_id not in self._slave_options: self._slave_options[slave_id] = [] @@ -812,8 +822,10 @@ class Interface(object): def slave_add(self, if_id): self._machine._rpc_call_x(self._netns, "slave_add", self._id, if_id) + self.add_slave(self._machine.get_interface(if_id)) def slave_del(self, if_id): + self.del_slave(self._machine.get_interface(if_id)) self._machine._rpc_call_x(self._netns, "slave_del", self._id, if_id) class StaticInterface(Interface): diff --git a/lnst/Slave/NetConfigDevice.py b/lnst/Slave/NetConfigDevice.py index 60be2ec..dd053a6 100644 --- a/lnst/Slave/NetConfigDevice.py +++ b/lnst/Slave/NetConfigDevice.py @@ -212,8 +212,10 @@ class NetConfigDeviceBridge(NetConfigDeviceGeneric): def slave_add(self, slave_id): self._add_rm_port("add", slave_id) + self._dev_config["slaves"].append(slave_id) def slave_del(self, slave_id): + self._dev_config["slaves"].remove(slave_id) self._add_rm_port("del", slave_id) class NetConfigDeviceMacvlan(NetConfigDeviceGeneric): @@ -379,14 +381,14 @@ class NetConfigDeviceTeam(NetConfigDeviceGeneric): self._ports_down() for slave_id in get_slaves(self._dev_config): - self.slave_add(slave_id) + self._slave_add(slave_id) self._ports_up() def deconfigure(self): for slave_id in get_slaves(self._dev_config): - self.slave_del(slave_id) + self._slave_del(slave_id) - def slave_add(self, slave_id): + def _slave_add(self, slave_id): dev_name = self._dev_config["name"] port_dev = self._if_manager.get_mapped_device(slave_id) port_name = port_dev.get_name() @@ -400,13 +402,21 @@ class NetConfigDeviceTeam(NetConfigDeviceGeneric): port_dev.down() exec_cmd("teamdctl %s %s port add %s" % (dbus_option, dev_name, port_name)) - def slave_del(self, slave_id): + def slave_add(self, slave_id): + self._slave_add(slave_id) + self._dev_config["slaves"].append(slave_id) + + def _slave_del(self, slave_id): dev_name = self._dev_config["name"] port_dev = self._if_manager.get_mapped_device(slave_id) port_name = port_dev.get_name() dbus_option = "-D" if self._should_enable_dbus() else "" exec_cmd("teamdctl %s %s port remove %s" % (dbus_option, dev_name, port_name)) + def slave_del(self, slave_id): + self._dev_config["slaves"].remove(slave_id) + self._slave_del(slave_id) + class NetConfigDeviceOvsBridge(NetConfigDeviceGeneric): _modulename = "openvswitch" _moduleload = True -- 2.4.10

2 4

[PATCH 1/4] lnst: Fix dynamic addition / deletion of slaves
by Ido Schimmel 17 Feb '16

17 Feb '16

The slaves of an interface are determined during creation time and stored in two objects: * Interface on the controller * NetConfigDeviceGeneric on the slave machine When slaves are dynamically added / deleted during a Python task the 'slaves' data structures in these objects are not updated, which can cause several problems: If a slave is removed from its master, then during cleanup the slave machine will try to remove it again and thereby generate an error. This requires us to update the data struct on the slave machine. If a slave is added to a master and that master is later reset(), then the newly added slave won't be configured correctly after the reset(). This requires us to update the data struct on the controller. Cc: Andrew Lunn <andrew(a)lunn.ch> Fixes: af025186a329 ("InterfaceAPI: Introduce slave_{add, del}") Signed-off-by: Ido Schimmel <idosch(a)mellanox.com> --- lnst/Controller/Machine.py | 12 ++++++++++++ lnst/Slave/NetConfigDevice.py | 4 ++++ 2 files changed, 16 insertions(+) diff --git a/lnst/Controller/Machine.py b/lnst/Controller/Machine.py index 884b315..9897580 100644 --- a/lnst/Controller/Machine.py +++ b/lnst/Controller/Machine.py @@ -591,6 +591,12 @@ class Interface(object): else: self._master["other"].append(master) + def del_master(self, master): + if self._master["primary"] is master: + self._master["primary"] = None + else: + self._master["other"].remove(master) + def get_primary_master(self): return self._master["primary"] @@ -601,6 +607,10 @@ class Interface(object): else: iface.add_master(self) + def del_slave(self, iface): + iface.del_master(self) + del self._slaves[iface.get_id()] + def set_slave_option(self, slave_id, name, value): if slave_id not in self._slave_options: self._slave_options[slave_id] = [] @@ -812,8 +822,10 @@ class Interface(object): def slave_add(self, if_id): self._machine._rpc_call_x(self._netns, "slave_add", self._id, if_id) + self.add_slave(self._machine.get_interface(if_id)) def slave_del(self, if_id): + self.del_slave(self._machine.get_interface(if_id)) self._machine._rpc_call_x(self._netns, "slave_del", self._id, if_id) class StaticInterface(Interface): diff --git a/lnst/Slave/NetConfigDevice.py b/lnst/Slave/NetConfigDevice.py index 60be2ec..8adcf2b 100644 --- a/lnst/Slave/NetConfigDevice.py +++ b/lnst/Slave/NetConfigDevice.py @@ -212,8 +212,10 @@ class NetConfigDeviceBridge(NetConfigDeviceGeneric): def slave_add(self, slave_id): self._add_rm_port("add", slave_id) + self._dev_config["slaves"].append(slave_id) def slave_del(self, slave_id): + self._dev_config["slaves"].remove(slave_id) self._add_rm_port("del", slave_id) class NetConfigDeviceMacvlan(NetConfigDeviceGeneric): @@ -399,8 +401,10 @@ class NetConfigDeviceTeam(NetConfigDeviceGeneric): exec_cmd("teamdctl %s %s port config update %s \"%s\"" % (dbus_option, dev_name, port_name, teamd_port_config)) port_dev.down() exec_cmd("teamdctl %s %s port add %s" % (dbus_option, dev_name, port_name)) + self._dev_config["slaves"].append(slave_id) def slave_del(self, slave_id): + self._dev_config["slaves"].remove(slave_id) dev_name = self._dev_config["name"] port_dev = self._if_manager.get_mapped_device(slave_id) port_name = port_dev.get_name() -- 2.4.10

1 4

[PATCH 1/4] NetTestResultSerializer: sort machines in result setup description
by Jiri Prochazka 08 Feb '16

08 Feb '16

For prettier output machines in result setup description are sorted by id's of machines Signed-off-by: Jiri Prochazka <jprochaz(a)redhat.com> --- lnst/Controller/NetTestResultSerializer.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lnst/Controller/NetTestResultSerializer.py b/lnst/Controller/NetTestResultSerializer.py index fbe4392..e4fe086 100644 --- a/lnst/Controller/NetTestResultSerializer.py +++ b/lnst/Controller/NetTestResultSerializer.py @@ -94,7 +94,7 @@ class NetTestResultSerializer: "Setup is using virtual machines.", "")) - for m_id, m in match["machines"].iteritems(): + for m_id, m in sorted(match["machines"].iteritems()): output_pairs.append((4*" " + "host \"%s\" uses \"%s\"" %\ (m_id, m["target"]), "")) for if_id, pool_if in m["interfaces"].iteritems(): -- 2.4.3

3 8

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

LNST-developers February 2016