PLEASE READ - PyRecipes final discussion with drafts - Opinions requested
by Jiri Prochazka
Hello,
this mail is status update on PyRecipes. I'm sorry there is no visible
progress yet, but it's because I was considering several different formats
and tried to satisfy both parties, which won't be probably possible.
Current use cases of LNST (RedHat vs Mellanox)
==============================================
Currently, we have both sides, each with their own use case scenarios. Red
Hat is using LNST for performance regression testing, along with PerfRepo.
Each test uses same tools (ethtool, netperf, ping). The setups are very
similar, as for hosts and eth ifaces, what is changing is soft interface
setup (bond, team, VLAN, ovs, bridge). Setup of soft interfaces is
currently done via XML and in task is only execution of test tools and
additional setup (ethtool, MTU). Currently, the code is duplicated in every
task (perfrepo methods, ethtool setups, mtu setups, netperf inits and
calls) so a new layer of abstraction would be welcomed in order to simplify
the code and maintainability.
As for Mellanox side, I can only speak about what I can see in switchdev
recipes. Their approach is to define only hardware interfaces in XML and do
all soft interface setting in the task. In order to create an abstraction
layer a TestLib was created, which looks really appealing to me.
What do RH/Mellanox expect from PyRecipes
=========================================
We had a video call in January about PyRecipes with olichtne (RH) and
jpirko (Mellanox). Both sides had different view of PyRecipes.
Red Hat POW
-----------
1. Wants to be able to define soft and hard interfaces together in setup
2. Wants to be able to combine network setup with different tasks
3. Task should be understood as a function, with network as argument
4. Task should be generic
Mellanox POW
------------
1. Wants to get rid of ID's (hosts and interfaces)
2. Wants soft iface definition only in task, not in setup
3. Task should be specific
4. Wrappers for generic stuff
5. 1 task == 1 test == 1 file, do not combine it
Proposed approach #1 (Mlx like)
===============================
Description: Setup and task is in one file, no IDs are used, soft interface
definition is part of task
Example:
import lnst
m1 = lnst.add_host()
m2 = lnst.add_host()
m1_eth1 = m1.add_interface(label="tnet")
m1_eth2 = m1.add_interface(label="tnet")
m2_eth1 = m2.add_interface(label="tnet")
while match(match=lnst.SingleMatch):
m1_team = m1.create_team([m1_eth1, m1_eth2], ip="1.2.3.4/24")
m2_eth1.reset(["1.2.3.5/24"])
ping_mod = ...
m1_team.run(ping_mod)
Proposed approach #2 (RH like)
==============================
Description: Soft interfaces can be defined in both setup() and task()
methods. IDs muset used due to different scopes of variables. Setup can be
in separate file and can be imported in multiple tasks. In one file,
multiple tasks can be called.
Example:
import lnst
def setup():
m1 = lnst.add_host("m1")
m2 = lnst.add_host("m2")
m1_eth1 = m1.add_interface(id="eth1", label="tnet")
m1_eth2 = m1.add_interface(id="eth2", label="tnet")
m2_eth1 = m2.add_interface(id="eth1", label="tnet", ip="1.1.1.1/24")
m1.create_team(id="team1", slaves=[m1_eth1, m1_eth2], ip="1.1.1.2/24")
def task():
m1 = lnst.get_host("m1")
m2 = lnst.get_host("m2")
m1_team = m1.get_interface("team1")
m2_eth1 = m2.get_interface("eth1")
ping_mod = ...
m1_team.run(ping_mod)
lnst.run(match=lnst.SingleMatch,
setup,
task)
Proposed approach #3 (RH like)
==============================
Description: Task method is portable, it uses machine and interface objects
as args so no IDs are required. Soft interfaces can be created in both
do_task and in setup phase. In one file, multiple tasks can be called.
Example:
import lnst
def do_task(m1, if1, if2):
ping_mod = ...src=if1, dst=if2...
m1.run(ping_mod)
m1 = lnst.add_machine()
m2 = lnst.add_machine()
m1_eth1 = m1.add_interface(label="tnet")
m1_eth2 = m1.add_interface(label="tnet")
m1_team = m1.create_team(slaves=[m1_eth1, m1_eth2], ip="1.1.1.1/24")
m2_eth1 = m2.add_interface(label="tnet", ip="1.1.1.2/24")
while lnst.match(match=lnst.SingleMatch):
do_task(m1, m1_team, m2_eth1)
Summary
=======
Drafts above are not meant to be final, it sure can be improved and
modified to satisfy our needs. But we need to come to conclusion for both
Mlx and RH side, so I can start working on it.
Some important questions regarding PyRecipes:
---------------------------------------------
I. Soft interfaces - in setup phase and task phase or only in task phase?
II. Portability - one task == one recipe, or allow combinations of networks
with different tasks?
III. Should task be generic or specific?
IV. Do we have to get rid of IDs?
My opinion on the matter
========================
Favourite approach - #3
Answers to questions:
---------------------
I. Soft interfaces - only in task phase - to follow 1 task == 1 recipe
mentality, will bring easier maintaining
II. Portability - one task == one recipe - even our use case shows, that
tests don't allow so much combination (1 task is being used in avg. 1-2
recipes in phase1, 2-3 recipes in phase2), so I don't thinks its so
important to use to preserve it
III. Specific task - generic stuff can be defined by a new layer of
abstraction, like TestLib in switchdev tests
IV. I do not think IDs are such evil that we should get rid of it, altough
I agree object oriented approach (which we want to follow, thus PyRecipes
became a thing in the first place) should be only using instances of
objects in both task and setup.
Summary
=======
Please, devs from RH and Mlx, take a look on these drafts and send an email
with your opinion on the matter. Ideally, next week starting Wed I would
like to have it decided so I can start implementing it.
In the end, probably we will have to compromise, but hopefully, both
parties will end up satisfied and all this work will lead to improving of
the quality of the whole LNST.
Thanks for reading,
Jiri Prochazka
7 years, 7 months
[PATCH 01/12] Config: add get_section_values method
by Ondrej Lichtner
From: Ondrej Lichtner <olichtne(a)redhat.com>
This method returns all the option values in a specific section.
Previously we just had the get_section method which returns the internal
dict structure of the Config class which is not very usable in the
application.
Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com>
---
lnst/Common/Config.py | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/lnst/Common/Config.py b/lnst/Common/Config.py
index 97a82e4..de7ba4a 100644
--- a/lnst/Common/Config.py
+++ b/lnst/Common/Config.py
@@ -155,6 +155,16 @@ class Config():
raise ConfigError(msg)
return self._options[section]
+ def get_section_values(self, section):
+ if section not in self._options:
+ msg = 'Unknow section: %s' % section
+ raise ConfigError(msg)
+
+ res = {}
+ for opt_name, opt in self._options[section].items():
+ res[opt_name] = opt["value"]
+ return res
+
def get_option(self, section, option):
sect = self.get_section(section)
if option not in sect:
--
2.7.2
7 years, 7 months
[PATCH 10/12] SecureSocket: make cryptography import optional
by Ondrej Lichtner
From: Ondrej Lichtner <olichtne(a)redhat.com>
The cryptography library is imported only when authentication methods
that need it are requested. If the library is missing, the user will get
an error message explaining the problem. It's a pretty hackish way of
doing the imports but I'm not aware of a better solution... I'm open to
suggestions if there are any.
Signed-off-by: Ondrej Lichtner <olichtne(a)redhat.com>
---
lnst/Common/SecureSocket.py | 72 ++++++++++++++++++++++++++++++++++-------
lnst/Controller/CtlSecSocket.py | 40 +++++++++++++++++++----
lnst/Slave/SlaveSecSocket.py | 42 ++++++++++++++++++++----
3 files changed, 131 insertions(+), 23 deletions(-)
diff --git a/lnst/Common/SecureSocket.py b/lnst/Common/SecureSocket.py
index d319c54..cd149c2 100644
--- a/lnst/Common/SecureSocket.py
+++ b/lnst/Common/SecureSocket.py
@@ -19,17 +19,6 @@ import os
import cPickle
import hashlib
import hmac
-import cryptography.exceptions
-from cryptography.hazmat.primitives import hashes
-from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
-from cryptography.hazmat.primitives.asymmetric import padding, ec
-from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey
-from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePublicKey
-from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey
-from cryptography.hazmat.primitives.asymmetric.rsa import RSAPublicKey
-from cryptography.hazmat.primitives.asymmetric.dsa import DSAPrivateKey
-from cryptography.hazmat.primitives.asymmetric.dsa import DSAPublicKey
-from cryptography.hazmat.backends import default_backend
DH_GROUP = {"p": int("0xFFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD1"\
"29024E088A67CC74020BBEA63B139B22514A08798E3404DD"\
@@ -75,6 +64,58 @@ if SRP_GROUP["p"].bit_length()%8:
class SecSocketException(Exception):
pass
+cryptography = None
+hashes = None
+Cipher = None
+algorithms = None
+modes = None
+padding = None
+ec = None
+EllipticCurvePrivateKey = None
+EllipticCurvePublicKey = None
+RSAPrivateKey = None
+RSAPublicKey = None
+DSAPrivateKey = None
+DSAPublicKey = None
+default_backend = None
+cryptography_imported = False
+def cryptography_imports():
+ global cryptography_imported
+ if cryptography_imported:
+ return
+
+ global cryptography
+ global hashes
+ global Cipher
+ global algorithms
+ global modes
+ global padding
+ global ec
+ global EllipticCurvePrivateKey
+ global EllipticCurvePublicKey
+ global RSAPrivateKey
+ global RSAPublicKey
+ global DSAPrivateKey
+ global DSAPublicKey
+ global default_backend
+
+ try:
+ import cryptography.exceptions
+ from cryptography.hazmat.primitives import hashes
+ from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
+ from cryptography.hazmat.primitives.asymmetric import padding, ec
+ from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePrivateKey
+ from cryptography.hazmat.primitives.asymmetric.ec import EllipticCurvePublicKey
+ from cryptography.hazmat.primitives.asymmetric.rsa import RSAPrivateKey
+ from cryptography.hazmat.primitives.asymmetric.rsa import RSAPublicKey
+ from cryptography.hazmat.primitives.asymmetric.dsa import DSAPrivateKey
+ from cryptography.hazmat.primitives.asymmetric.dsa import DSAPublicKey
+ from cryptography.hazmat.backends import default_backend
+ cryptography_imported = True
+ except ImportError:
+ raise SecSocketException("Library 'cryptography' missing "\
+ "can't establish secure channel.")
+
class SecureSocket(object):
def __init__(self, soc):
self._role = None
@@ -112,6 +153,7 @@ class SecureSocket(object):
def _add_mac_sign(self, data):
if not self._current_write_spec["mac_key"]:
return data
+ cryptography_imports()
msg = str(self._current_write_spec["seq_num"]) + str(len(data)) + data
signature = hmac.new(self._current_write_spec["mac_key"],
@@ -124,6 +166,7 @@ class SecureSocket(object):
def _del_mac_sign(self, signed_data):
if not self._current_read_spec["mac_key"]:
return signed_data
+ cryptography_imports()
signed_msg = cPickle.loads(signed_data)
data = signed_msg["data"]
@@ -140,6 +183,7 @@ class SecureSocket(object):
def _add_padding(self, data):
if not self._current_write_spec["enc_key"]:
return data
+ cryptography_imports()
block_size = algorithms.AES.block_size/8
pad_length = block_size - (len(data) % block_size)
@@ -152,6 +196,7 @@ class SecureSocket(object):
def _del_padding(self, data):
if not self._current_read_spec["enc_key"]:
return data
+ cryptography_imports()
pad_length = int(data[-1].encode("hex"), 16)
for char in data[-pad_length]:
@@ -163,6 +208,7 @@ class SecureSocket(object):
def _add_encrypt(self, data):
if not self._current_write_spec["enc_key"]:
return data
+ cryptography_imports()
iv = os.urandom(algorithms.AES.block_size/8)
mode = modes.CBC(iv)
@@ -180,6 +226,7 @@ class SecureSocket(object):
def _del_encrypt(self, data):
if not self._current_read_spec["enc_key"]:
return data
+ cryptography_imports()
encrypted_msg = cPickle.loads(data)
encrypted_data = encrypted_msg["enc_data"]
@@ -313,6 +360,7 @@ class SecureSocket(object):
server_spec = self._next_read_spec
else:
raise SecSocketException("Socket without a role!")
+ cryptography_imports()
aes_keysize = max(algorithms.AES.key_sizes)/8
mac_keysize = hashlib.sha256().block_size
@@ -334,6 +382,7 @@ class SecureSocket(object):
return
def _sign_data(self, data, privkey):
+ cryptography_imports()
if isinstance(privkey, DSAPrivateKey):
signer = privkey.signer(hashes.SHA256())
elif isinstance(privkey, RSAPrivateKey):
@@ -349,6 +398,7 @@ class SecureSocket(object):
return signer.finalize()
def _verify_signature(self, pubkey, data, signature):
+ cryptography_imports()
if isinstance(pubkey, DSAPublicKey):
verifier = pubkey.verifier(signature, hashes.SHA256())
elif isinstance(pubkey, RSAPublicKey):
diff --git a/lnst/Controller/CtlSecSocket.py b/lnst/Controller/CtlSecSocket.py
index b4c857a..1f16055 100644
--- a/lnst/Controller/CtlSecSocket.py
+++ b/lnst/Controller/CtlSecSocket.py
@@ -19,13 +19,37 @@ from lnst.Common.SecureSocket import SecureSocket
from lnst.Common.SecureSocket import DH_GROUP, SRP_GROUP
from lnst.Common.SecureSocket import SecSocketException
from lnst.Common.Config import lnst_config
-from cryptography.hazmat.primitives import serialization as ser
-from cryptography.hazmat.primitives.serialization import load_pem_private_key
-from cryptography.hazmat.primitives.serialization import load_pem_public_key
-from cryptography.hazmat.primitives.serialization import load_ssh_public_key
-from cryptography.hazmat.backends import default_backend
-backend = default_backend()
+ser = None
+load_pem_private_key = None
+load_pem_public_key = None
+load_ssh_public_key = None
+backend = None
+cryptography_imported = False
+def cryptography_imports():
+ global cryptography_imported
+ if cryptography_imported:
+ return
+
+ global ser
+ global load_pem_private_key
+ global load_pem_public_key
+ global load_ssh_public_key
+ global backend
+
+ try:
+ import cryptography
+ import cryptography.hazmat.primitives.serialization as ser
+ from cryptography.hazmat.primitives.serialization import load_pem_private_key
+ from cryptography.hazmat.primitives.serialization import load_pem_public_key
+ from cryptography.hazmat.primitives.serialization import load_ssh_public_key
+ from cryptography.hazmat.backends import default_backend
+ except ImportError:
+ raise SecSocketException("Library 'cryptography' missing "\
+ "can't establish secure channel.")
+
+ backend = default_backend()
+ cryptography_imported = True
class CtlSecSocket(SecureSocket):
def __init__(self, soc):
@@ -57,10 +81,13 @@ class CtlSecSocket(SecureSocket):
logging.warning(" NO AUTHENTICATION IN PLACE")
logging.warning("SECURE CHANNEL IS VULNERABLE TO MIM ATTACKS")
logging.warning("===========================================")
+ cryptography_imports()
self._dh_handshake()
elif sec_params["auth_type"] == "ssh":
+ cryptography_imports()
self._ssh_handshake()
elif sec_params["auth_type"] == "pubkey":
+ cryptography_imports()
ctl_identity = sec_params["identity"]
ctl_key_path = sec_params["privkey"]
try:
@@ -81,6 +108,7 @@ class CtlSecSocket(SecureSocket):
self._pubkey_handshake(ctl_identity, ctl_key, srv_key)
elif sec_params["auth_type"] == "password":
+ cryptography_imports()
self._passwd_handshake(sec_params["auth_passwd"])
else:
raise SecSocketException("Unknown authentication method.")
diff --git a/lnst/Slave/SlaveSecSocket.py b/lnst/Slave/SlaveSecSocket.py
index 9e3cd92..8810bac 100644
--- a/lnst/Slave/SlaveSecSocket.py
+++ b/lnst/Slave/SlaveSecSocket.py
@@ -19,13 +19,39 @@ import logging
from lnst.Common.SecureSocket import SecureSocket
from lnst.Common.SecureSocket import DH_GROUP, SRP_GROUP
from lnst.Common.SecureSocket import SecSocketException
-from cryptography.hazmat.primitives import serialization as ser
-from cryptography.hazmat.primitives.serialization import load_pem_private_key
-from cryptography.hazmat.primitives.serialization import load_pem_public_key
-from cryptography.hazmat.primitives.serialization import load_ssh_public_key
-from cryptography.hazmat.backends import default_backend
-backend = default_backend()
+ser = None
+load_pem_private_key = None
+load_pem_public_key = None
+load_ssh_public_key = None
+backend = None
+cryptography_imported = False
+def cryptography_imports():
+ global cryptography_imported
+ if cryptography_imported:
+ return
+
+ global ser
+ global load_pem_private_key
+ global load_pem_public_key
+ global load_ssh_public_key
+ global backend
+
+ try:
+ import cryptography
+ import cryptography.hazmat.primitives.serialization as ser
+ from cryptography.hazmat.primitives.serialization import load_pem_private_key
+ from cryptography.hazmat.primitives.serialization import load_pem_public_key
+ from cryptography.hazmat.primitives.serialization import load_ssh_public_key
+ from cryptography.hazmat.backends import default_backend
+ except ImportError:
+ logging.error("Library 'cryptography' missing "\
+ "can't establish secure channel.")
+ raise SecSocketException("Library 'cryptography' missing "\
+ "can't establish secure channel.")
+
+ backend = default_backend()
+ cryptography_imported = True
class SlaveSecSocket(SecureSocket):
def __init__(self, soc):
@@ -55,10 +81,13 @@ class SlaveSecSocket(SecureSocket):
logging.warning(" NO AUTHENTICATION IN PLACE")
logging.warning("SECURE CHANNEL IS VULNERABLE TO MIM ATTACKS")
logging.warning("===========================================")
+ cryptography_imports()
self._dh_handshake()
elif sec_params["auth_types"] == "ssh":
+ cryptography_imports()
self._ssh_handshake()
elif sec_params["auth_types"] == "pubkey":
+ cryptography_imports()
srv_key = None
with open(sec_params["privkey"], 'r') as f:
srv_key = load_pem_private_key(f.read(), None, backend)
@@ -73,6 +102,7 @@ class SlaveSecSocket(SecureSocket):
self._pubkey_handshake(srv_key, ctl_pubkeys)
elif sec_params["auth_types"] == "password":
+ cryptography_imports()
self._passwd_handshake(sec_params["auth_password"])
else:
raise SecSocketException("Unknown authentication method.")
--
2.7.2
7 years, 7 months
[PATCH 0/7] RFC implementing secure Controller<->Slave communication
by Ondrej Lichtner
From: Ondrej Lichtner <olichtne(a)redhat.com>
First of all, this isn't a patchset to be pulled into the repository yet, just a
request for comments to see if anything can be improved, while I work on
finishing some minor problem areas:
* this breaks the lnst-pool-wizard because it doesn't use SecureSockets yet
* this breaks the lnst-ctl deconfigure command for the same reason
The following patch set implements secure communication between the Controller
and the Slave. It implements 4 different authentication mechanisms:
none - no authentication, just DH secret negotiation and encryption
password - password authenticated secret negotiation using SRP-6a protocol and
encryption
pubkey - DH secret negotiation, authentication by using a private-public key
pairs
ssh - the same as pubkey, but uses ssh keys already present on the system
By default, both the Controller and Slave assume the "none" mechanism so
everything should work just as before without any additional work needed. To try
the other mechanisms you'll need to configure both the Controller and the Slave
as follows:
password:
on Slave edit the lnst-slave.conf like this:
[security]
auth_types = password
auth_password = your_chosen_password
on Controller edit the SlaveMachineXML file of the slave and add this:
<security>
<auth_type>password</auth_type>
<auth_password>your_chosen_password</auth_password>
</security>
under the main <slavemachine> element
pubkey:
generate a private-public key pair on both the Slave and the Controller and
exchange the public keys between the machines.
on Slave edit the lnst-slave.conf file like this:
[security]
auth_types = pubkey
privkey = path/to/slave_private_key
ctl_pubkeys = path/to/pubkeys_dir/
on Controller edit the SlaveMachineXML and add this:
<security>
<auth_type>pubkey</auth_type>
<pubkey_path>path/to/slave_pubkey</pubkey_path>
</security>
under the main <slavemachine> element
and edit the lnst-ctl.conf like this:
[security]
identity = ctl_name
privkey = path/to/ctl_privatekey
On the Slave the Controller public key needs to be placed into the
ctl_pubkeys directory and the filename must match the value of the identity
option in the lnst-ctl.conf
ssh:
have ssh presetup between the Controller and Slave machines - the slave
needs its sshd private keys to be located in /etc/ssh (ssh_host.*key files)
and the controllers public key must be in the ~/.ssh/authorized_keys file
On the Controller you need to have the ~/.ssh/id_rsa file to contain a
private RSA key (the public part is in the slaves authorized_keys file) and
you also need to have at least one of the Slaves public keys located in the
~/.ssh/known_hosts file
you can easily achieve this by just starting the sshd daemon on the Slave
machine and on the Controller generate the key with
ssh-keygen -t rsa -C "your_email(a)example.com"
and run ssh-copy-id username@slavehostname
then configure the LNST Slave - lnst-slave.conf:
[security]
auth_types = ssh
and on the Controller edit the Slave Machine XML file like this:
<security>
<auth_type>pubkey</auth_type>
<pubkey_path>path/to/slave_pubkey</pubkey_path>
</security>
Ondrej Lichtner (7):
Config: add get_section_values method
Config: add security section and options to config
Add SecureSocket classes
SlaveMachineXML: add security information
Machine: add security parameters of the slave machine
Use SecureSockets for Ctl<->Slave communication
Machine: rename method configure to init_connection
lnst/Common/Config.py | 44 ++++
lnst/Common/ConnectionHandler.py | 39 +---
lnst/Common/SecureSocket.py | 377 ++++++++++++++++++++++++++++++++++
lnst/Controller/CtlSecSocket.py | 318 ++++++++++++++++++++++++++++
lnst/Controller/Machine.py | 17 +-
lnst/Controller/NetTestController.py | 2 +-
lnst/Controller/SlaveMachineParser.py | 31 +++
lnst/Controller/SlavePool.py | 9 +-
lnst/Slave/NetTestSlave.py | 13 +-
lnst/Slave/SlaveSecSocket.py | 323 +++++++++++++++++++++++++++++
schema-sm.rng | 29 +++
11 files changed, 1163 insertions(+), 39 deletions(-)
create mode 100644 lnst/Common/SecureSocket.py
create mode 100644 lnst/Controller/CtlSecSocket.py
create mode 100644 lnst/Slave/SlaveSecSocket.py
--
2.7.1
7 years, 7 months
[PATCH] recipes: switchdev: Test LAG leave flow
by Ido Schimmel
When port netdevs are leaving a bridged LAG device, then the bridge's
cleanup sequence isn't invoked and the switch driver needs to do the
necessary cleanup.
Adjust current LAG recipes to test the flow.
Note that I'm creating a new LAG device on the switch for the second
fastpath test, as currently switchdev drivers suffer from an ordering
problem, which first requires the LAG to be populated and only then
bridged. Hopefully, this will be resolved soon.
Signed-off-by: Ido Schimmel <idosch(a)mellanox.com>
---
recipes/switchdev/l2-006-bridge_team.py | 28 ++++++++++++++-
recipes/switchdev/l2-015-bridge_team_vlan1d.py | 48 ++++++++++++++++++++++++--
2 files changed, 72 insertions(+), 4 deletions(-)
diff --git a/recipes/switchdev/l2-006-bridge_team.py b/recipes/switchdev/l2-006-bridge_team.py
index fd648c9..48b9752 100644
--- a/recipes/switchdev/l2-006-bridge_team.py
+++ b/recipes/switchdev/l2-006-bridge_team.py
@@ -31,7 +31,8 @@ def do_task(ctl, hosts, ifaces, aliases):
sw_lag2 = sw.create_team(slaves=[sw_if3, sw_if4],
config=team_config)
- sw.create_bridge(slaves=[sw_lag1, sw_lag2], options={"vlan_filtering": 1})
+ sw_br = sw.create_bridge(slaves=[sw_lag1, sw_lag2],
+ options={"vlan_filtering": 1})
sleep(15)
@@ -40,6 +41,31 @@ def do_task(ctl, hosts, ifaces, aliases):
tl.netperf_tcp(m1_lag1, m2_lag1)
tl.netperf_udp(m1_lag1, m2_lag1)
+ sw_lag1.slave_del(sw_if1.get_id())
+ sw_lag1.slave_del(sw_if2.get_id())
+
+ m1_lag1.slave_del(m1_if1.get_id())
+
+ # Make sure slowpath is working.
+ sw_if1.reset(ip=["192.168.102.10/24", "2003::1/64"])
+ m1_if1.reset(ip=["192.168.102.11/24", "2003::2/64"])
+
+ sleep(15)
+
+ tl.ping_simple(sw_if1, m1_if1)
+
+ # Repopulate the LAGs and make sure fastpath is OK.
+ sw_lag3 = sw.create_team(slaves=[sw_if1, sw_if2],
+ config=team_config)
+ sw_br.slave_add(sw_lag3.get_id())
+ m1_lag1.slave_add(m1_if1.get_id())
+
+ sleep(15)
+
+ tl.ping_simple(m1_lag1, m2_lag1)
+ tl.netperf_tcp(m1_lag1, m2_lag1)
+ tl.netperf_udp(m1_lag1, m2_lag1)
+
do_task(ctl, [ctl.get_host("machine1"),
ctl.get_host("machine2"),
ctl.get_host("switch")],
diff --git a/recipes/switchdev/l2-015-bridge_team_vlan1d.py b/recipes/switchdev/l2-015-bridge_team_vlan1d.py
index 3bea6e2..723a04a 100644
--- a/recipes/switchdev/l2-015-bridge_team_vlan1d.py
+++ b/recipes/switchdev/l2-015-bridge_team_vlan1d.py
@@ -34,15 +34,17 @@ def do_task(ctl, hosts, ifaces, aliases):
sw_lag1 = sw.create_team(slaves=[sw_if1, sw_if2], config=team_config)
sw_lag2 = sw.create_team(slaves=[sw_if3, sw_if4], config=team_config)
br_options = {"vlan_filtering": 1}
- sw.create_bridge(slaves=[sw_lag1, sw_lag2], options=br_options)
+ sw_br1 = sw.create_bridge(slaves=[sw_lag1, sw_lag2], options=br_options)
sw_lag1_10 = sw.create_vlan(sw_lag1, 10)
sw_lag2_10 = sw.create_vlan(sw_lag2, 10)
- sw.create_bridge(slaves=[sw_lag1_10, sw_lag2_10], options=br_options)
+ sw_br2 = sw.create_bridge(slaves=[sw_lag1_10, sw_lag2_10],
+ options=br_options)
sw_lag1_20 = sw.create_vlan(sw_lag1, 20)
sw_lag2_21 = sw.create_vlan(sw_lag2, 21)
- sw.create_bridge(slaves=[sw_lag1_20, sw_lag2_21], options=br_options)
+ sw_br3 = sw.create_bridge(slaves=[sw_lag1_20, sw_lag2_21],
+ options=br_options)
sleep(15)
@@ -60,6 +62,46 @@ def do_task(ctl, hosts, ifaces, aliases):
tl.netperf_tcp(m1_lag1_20, m2_lag1_21)
tl.netperf_udp(m1_lag1_20, m2_lag1_21)
+ sw_lag1.slave_del(sw_if1.get_id())
+ sw_lag1.slave_del(sw_if2.get_id())
+
+ m1_lag1.slave_del(m1_if1.get_id())
+
+ # Make sure slowpath is working.
+ sw_if1.reset(ip=test_ip(4, 1))
+ m1_if1.reset(ip=test_ip(4, 2))
+
+ sleep(15)
+
+ tl.ping_simple(sw_if1, m1_if1)
+
+ # Repopulate the LAGs and make sure fastpath is OK.
+ sw_lag3 = sw.create_team(slaves=[sw_if1, sw_if2],
+ config=team_config)
+ sw_br1.slave_add(sw_lag3.get_id())
+
+ sw_lag3_10 = sw.create_vlan(sw_lag3, 10)
+ sw_br2.slave_add(sw_lag3_10.get_id())
+
+ sw_lag3_20 = sw.create_vlan(sw_lag3, 20)
+ sw_br3.slave_add(sw_lag3_20.get_id())
+
+ m1_lag1.slave_add(m1_if1.get_id())
+
+ sleep(15)
+
+ tl.ping_simple(m1_lag1, m2_lag1)
+ tl.netperf_tcp(m1_lag1, m2_lag1)
+ tl.netperf_udp(m1_lag1, m2_lag1)
+
+ tl.ping_simple(m1_lag1_10, m2_lag1_10)
+ tl.netperf_tcp(m1_lag1_10, m2_lag1_10)
+ tl.netperf_udp(m1_lag1_10, m2_lag1_10)
+
+ tl.ping_simple(m1_lag1_20, m2_lag1_21)
+ tl.netperf_tcp(m1_lag1_20, m2_lag1_21)
+ tl.netperf_udp(m1_lag1_20, m2_lag1_21)
+
do_task(ctl, [ctl.get_host("machine1"),
ctl.get_host("machine2"),
ctl.get_host("switch")],
--
2.4.10
7 years, 7 months
[PATCH] Controller: remove libvirt root user restriction
by Jan Tluka
It is possible to manage virtual machines without root privileges. The
check for root is therefore no more required.
Follow further steps to allow non-root user to use LNST for virtual
setups.
1. First you need to use policy kit to tweak libvirtd.
# echo 'polkit.addRule(function(action, subject) {
if (action.id == "org.libvirt.unix.manage" && subject.isInGroup("wheel")) {
return polkit.Result.YES;
}
});
' > /etc/polkit-1/rules.d/80-libvirt.rules
then (re)start libvirtd service,
# systemctl enable libvirtd
# systemctl start libvirtd
2. Add the non-root user to group 'wheel'.
3. The last bit is to run lnst-ctl with following environment variable
exported:
LIBVIRT_DEFAULT_URI=qemu:///system lnst-ctl ...
Signed-off-by: Jan Tluka <jtluka(a)redhat.com>
---
lnst/Controller/NetTestController.py | 6 ------
1 file changed, 6 deletions(-)
diff --git a/lnst/Controller/NetTestController.py b/lnst/Controller/NetTestController.py
index cce1fec..8fdd43e 100644
--- a/lnst/Controller/NetTestController.py
+++ b/lnst/Controller/NetTestController.py
@@ -232,12 +232,6 @@ class NetTestController:
msg = "This setup cannot be provisioned with the current pool."
raise NoMatchError(msg)
- if sp.is_setup_virtual() and os.geteuid() != 0:
- msg = "Provisioning this setup requires additional configuration "\
- "of the virtual hosts in the pool. LNST needs root "\
- "priviledges so it can connect to qemu."
- raise NetTestError(msg)
-
def print_match_description(self):
sp = self._slave_pool
match = sp.get_match()
--
2.4.3
7 years, 7 months
[PATCH] recipes: remove duplicate wait from simple_netperf task
by Jan Tluka
There were two ctl.wait() calls in the task. One is redundant so I
removed it.
Signed-off-by: Jan Tluka <jtluka(a)redhat.com>
---
recipes/regression_tests/phase1/simple_netperf.py | 2 --
1 file changed, 2 deletions(-)
diff --git a/recipes/regression_tests/phase1/simple_netperf.py b/recipes/regression_tests/phase1/simple_netperf.py
index 6762a60..407ee5d 100644
--- a/recipes/regression_tests/phase1/simple_netperf.py
+++ b/recipes/regression_tests/phase1/simple_netperf.py
@@ -53,8 +53,6 @@ if nperf_cpupin:
for m, d in [ (m1, m1_testiface), (m2, m2_testiface) ]:
pin_dev_irqs(m, d, 0)
-ctl.wait(15)
-
p_opts = "-L %s" % (m2_testiface.get_ip(0))
if nperf_cpupin and nperf_mode != "multi":
p_opts += " -T%s,%s" % (nperf_cpupin, nperf_cpupin)
--
2.4.3
7 years, 7 months
[PATCH] recipes: switchdev: Test toggling of VLAN flags on 802.1q bridge
by Ido Schimmel
Signed-off-by: Ido Schimmel <idosch(a)mellanox.com>
---
recipes/switchdev/l2-008-bridge_vlan1q_sanity.py | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/recipes/switchdev/l2-008-bridge_vlan1q_sanity.py b/recipes/switchdev/l2-008-bridge_vlan1q_sanity.py
index 5572017..555891b 100644
--- a/recipes/switchdev/l2-008-bridge_vlan1q_sanity.py
+++ b/recipes/switchdev/l2-008-bridge_vlan1q_sanity.py
@@ -78,6 +78,22 @@ def do_task(ctl, hosts, ifaces, aliases):
sleep(1)
tl.ping_simple(m1_if1, m2_if1)
+ sw_if2.add_br_vlan(500, pvid=True, untagged=False)
+ sleep(1)
+ tl.ping_simple(m1_if1, m2_if1, fail_expected=True)
+
+ sw_if2.add_br_vlan(500, pvid=True, untagged=True)
+ sleep(1)
+ tl.ping_simple(m1_if1, m2_if1)
+
+ sw_if2.add_br_vlan(500, pvid=False, untagged=True)
+ sleep(1)
+ tl.ping_simple(m1_if1, m2_if1, fail_expected=True)
+
+ sw_if2.add_br_vlan(500, pvid=True, untagged=True)
+ sleep(1)
+ tl.ping_simple(m1_if1, m2_if1)
+
sw_br.slave_del(sw_if1.get_id())
sleep(5)
--
2.4.10
7 years, 7 months
[PATCH v2 1/4] lnst: Fix dynamic addition / deletion of slaves
by Ido Schimmel
The slaves of an interface are determined during creation time and
stored in two objects:
* Interface on the controller
* NetConfigDeviceGeneric on the slave machine
When slaves are dynamically added / deleted during a Python task the
'slaves' data structures in these objects are not updated, which can
cause several problems:
If a slave is removed from its master, then during cleanup the slave
machine will try to remove it again and thereby generate an error. This
requires us to update the data struct on the slave machine.
If a slave is added to a master and that master is later reset(), then
the newly added slave won't be configured correctly after the reset().
This requires us to update the data struct on the controller.
Cc: Andrew Lunn <andrew(a)lunn.ch>
Fixes: af025186a329 ("InterfaceAPI: Introduce slave_{add, del}")
Signed-off-by: Ido Schimmel <idosch(a)mellanox.com>
---
v1->v2:
* Align team devices with bridge and only add / delete slaves
from the "slaves" list when dynamically configured.
---
lnst/Controller/Machine.py | 12 ++++++++++++
lnst/Slave/NetConfigDevice.py | 18 ++++++++++++++----
2 files changed, 26 insertions(+), 4 deletions(-)
diff --git a/lnst/Controller/Machine.py b/lnst/Controller/Machine.py
index 884b315..9897580 100644
--- a/lnst/Controller/Machine.py
+++ b/lnst/Controller/Machine.py
@@ -591,6 +591,12 @@ class Interface(object):
else:
self._master["other"].append(master)
+ def del_master(self, master):
+ if self._master["primary"] is master:
+ self._master["primary"] = None
+ else:
+ self._master["other"].remove(master)
+
def get_primary_master(self):
return self._master["primary"]
@@ -601,6 +607,10 @@ class Interface(object):
else:
iface.add_master(self)
+ def del_slave(self, iface):
+ iface.del_master(self)
+ del self._slaves[iface.get_id()]
+
def set_slave_option(self, slave_id, name, value):
if slave_id not in self._slave_options:
self._slave_options[slave_id] = []
@@ -812,8 +822,10 @@ class Interface(object):
def slave_add(self, if_id):
self._machine._rpc_call_x(self._netns, "slave_add", self._id, if_id)
+ self.add_slave(self._machine.get_interface(if_id))
def slave_del(self, if_id):
+ self.del_slave(self._machine.get_interface(if_id))
self._machine._rpc_call_x(self._netns, "slave_del", self._id, if_id)
class StaticInterface(Interface):
diff --git a/lnst/Slave/NetConfigDevice.py b/lnst/Slave/NetConfigDevice.py
index 60be2ec..dd053a6 100644
--- a/lnst/Slave/NetConfigDevice.py
+++ b/lnst/Slave/NetConfigDevice.py
@@ -212,8 +212,10 @@ class NetConfigDeviceBridge(NetConfigDeviceGeneric):
def slave_add(self, slave_id):
self._add_rm_port("add", slave_id)
+ self._dev_config["slaves"].append(slave_id)
def slave_del(self, slave_id):
+ self._dev_config["slaves"].remove(slave_id)
self._add_rm_port("del", slave_id)
class NetConfigDeviceMacvlan(NetConfigDeviceGeneric):
@@ -379,14 +381,14 @@ class NetConfigDeviceTeam(NetConfigDeviceGeneric):
self._ports_down()
for slave_id in get_slaves(self._dev_config):
- self.slave_add(slave_id)
+ self._slave_add(slave_id)
self._ports_up()
def deconfigure(self):
for slave_id in get_slaves(self._dev_config):
- self.slave_del(slave_id)
+ self._slave_del(slave_id)
- def slave_add(self, slave_id):
+ def _slave_add(self, slave_id):
dev_name = self._dev_config["name"]
port_dev = self._if_manager.get_mapped_device(slave_id)
port_name = port_dev.get_name()
@@ -400,13 +402,21 @@ class NetConfigDeviceTeam(NetConfigDeviceGeneric):
port_dev.down()
exec_cmd("teamdctl %s %s port add %s" % (dbus_option, dev_name, port_name))
- def slave_del(self, slave_id):
+ def slave_add(self, slave_id):
+ self._slave_add(slave_id)
+ self._dev_config["slaves"].append(slave_id)
+
+ def _slave_del(self, slave_id):
dev_name = self._dev_config["name"]
port_dev = self._if_manager.get_mapped_device(slave_id)
port_name = port_dev.get_name()
dbus_option = "-D" if self._should_enable_dbus() else ""
exec_cmd("teamdctl %s %s port remove %s" % (dbus_option, dev_name, port_name))
+ def slave_del(self, slave_id):
+ self._dev_config["slaves"].remove(slave_id)
+ self._slave_del(slave_id)
+
class NetConfigDeviceOvsBridge(NetConfigDeviceGeneric):
_modulename = "openvswitch"
_moduleload = True
--
2.4.10
7 years, 7 months
[PATCH 1/4] lnst: Fix dynamic addition / deletion of slaves
by Ido Schimmel
The slaves of an interface are determined during creation time and
stored in two objects:
* Interface on the controller
* NetConfigDeviceGeneric on the slave machine
When slaves are dynamically added / deleted during a Python task the
'slaves' data structures in these objects are not updated, which can
cause several problems:
If a slave is removed from its master, then during cleanup the slave
machine will try to remove it again and thereby generate an error. This
requires us to update the data struct on the slave machine.
If a slave is added to a master and that master is later reset(), then
the newly added slave won't be configured correctly after the reset().
This requires us to update the data struct on the controller.
Cc: Andrew Lunn <andrew(a)lunn.ch>
Fixes: af025186a329 ("InterfaceAPI: Introduce slave_{add, del}")
Signed-off-by: Ido Schimmel <idosch(a)mellanox.com>
---
lnst/Controller/Machine.py | 12 ++++++++++++
lnst/Slave/NetConfigDevice.py | 4 ++++
2 files changed, 16 insertions(+)
diff --git a/lnst/Controller/Machine.py b/lnst/Controller/Machine.py
index 884b315..9897580 100644
--- a/lnst/Controller/Machine.py
+++ b/lnst/Controller/Machine.py
@@ -591,6 +591,12 @@ class Interface(object):
else:
self._master["other"].append(master)
+ def del_master(self, master):
+ if self._master["primary"] is master:
+ self._master["primary"] = None
+ else:
+ self._master["other"].remove(master)
+
def get_primary_master(self):
return self._master["primary"]
@@ -601,6 +607,10 @@ class Interface(object):
else:
iface.add_master(self)
+ def del_slave(self, iface):
+ iface.del_master(self)
+ del self._slaves[iface.get_id()]
+
def set_slave_option(self, slave_id, name, value):
if slave_id not in self._slave_options:
self._slave_options[slave_id] = []
@@ -812,8 +822,10 @@ class Interface(object):
def slave_add(self, if_id):
self._machine._rpc_call_x(self._netns, "slave_add", self._id, if_id)
+ self.add_slave(self._machine.get_interface(if_id))
def slave_del(self, if_id):
+ self.del_slave(self._machine.get_interface(if_id))
self._machine._rpc_call_x(self._netns, "slave_del", self._id, if_id)
class StaticInterface(Interface):
diff --git a/lnst/Slave/NetConfigDevice.py b/lnst/Slave/NetConfigDevice.py
index 60be2ec..8adcf2b 100644
--- a/lnst/Slave/NetConfigDevice.py
+++ b/lnst/Slave/NetConfigDevice.py
@@ -212,8 +212,10 @@ class NetConfigDeviceBridge(NetConfigDeviceGeneric):
def slave_add(self, slave_id):
self._add_rm_port("add", slave_id)
+ self._dev_config["slaves"].append(slave_id)
def slave_del(self, slave_id):
+ self._dev_config["slaves"].remove(slave_id)
self._add_rm_port("del", slave_id)
class NetConfigDeviceMacvlan(NetConfigDeviceGeneric):
@@ -399,8 +401,10 @@ class NetConfigDeviceTeam(NetConfigDeviceGeneric):
exec_cmd("teamdctl %s %s port config update %s \"%s\"" % (dbus_option, dev_name, port_name, teamd_port_config))
port_dev.down()
exec_cmd("teamdctl %s %s port add %s" % (dbus_option, dev_name, port_name))
+ self._dev_config["slaves"].append(slave_id)
def slave_del(self, slave_id):
+ self._dev_config["slaves"].remove(slave_id)
dev_name = self._dev_config["name"]
port_dev = self._if_manager.get_mapped_device(slave_id)
port_name = port_dev.get_name()
--
2.4.10
7 years, 7 months