Hello!
This is (another) one of my endless proposal emails, sorry about that :).
We discussed a way of enabling users to define their own tests and tools without the necessity to add modules to site-packages/ on the controller and all the slaves by hand.
This should be done by creating a directory at the controller machine that will contain the tests. The controller machine will send them to the slave through a xml-rpc method and the slave will be able to use them.
I dug into this a little more today and I had some ideas that I'd like to present here.
There are two things we deal with nowadays. Python test modules that represent Test* commands directly and test tools that can be in an arbitrary format. The tools are later used by the modules, to perform test tasks.
I personally think, that this model is great. It allows you to set up everything you need at the controller, e.g, prepare the test tools, write a new command, that will allow using them with LNST, write the recipe and then one-click execute them in your lab. You don't have to ssh to 5 (or even more) different machines to install the tools. That's the most tedious thing about networking tests. Manually preparing all the machines for the tests. I think the key feature of LNST facility is making the test setup and execution as easy and as much automated as possible.
Currently, the test modules (Test/Test*.py) are an integral part of the LNST package. They are available to controller machines as well as to slave machines. The inconsistency here is, that controller doesn't need to execute them. It only wants to take them and send them over to the slaves. I think that we could take them out from LNST core and treat them a little differently.
There are two categories of tests:
* upstream - currently lnst/Tests/Test*.py * user-defined - configurable; defaults to ~/.lnst/Tests
We could put all the upstream tests to /usr/share/lnst instead of having them within the tree directly. The test execution would then work like this:
1. Controller starts the test 2. It will contact the slave and send him hashes of all the Test*.py commands that will be necessary to execute the recipe 3. Slave will look within his tests cache located in /var/cache/lnst to see if it does have the tests already 4. If not, controller will transfer all the files to the slave 5. The test can begin :-)
The key feature is, that the slave would not search any directories other than his test cache, which would after a few executed recipes get almost 100% hit-rate. The great thing about the hashes is, that in case the test is changed in any way at the controller, it will be transfered again to the slave.
Consequentially to this, we could pack LNST into two (maybe three due to some common code) different packages:
* lnst-controller * lnst-slave * (lnst-common)
What do you think?
-Radek
Thu, Oct 18, 2012 at 12:43:00PM CEST, rpazdera@redhat.com wrote:
Hello!
This is (another) one of my endless proposal emails, sorry about that :).
We discussed a way of enabling users to define their own tests and tools without the necessity to add modules to site-packages/ on the controller and all the slaves by hand.
This should be done by creating a directory at the controller machine that will contain the tests. The controller machine will send them to the slave through a xml-rpc method and the slave will be able to use them.
I dug into this a little more today and I had some ideas that I'd like to present here.
There are two things we deal with nowadays. Python test modules that represent Test* commands directly and test tools that can be in an arbitrary format. The tools are later used by the modules, to perform test tasks.
I personally think, that this model is great. It allows you to set up everything you need at the controller, e.g, prepare the test tools, write a new command, that will allow using them with LNST, write the recipe and then one-click execute them in your lab. You don't have to ssh to 5 (or even more) different machines to install the tools. That's the most tedious thing about networking tests. Manually preparing all the machines for the tests. I think the key feature of LNST facility is making the test setup and execution as easy and as much automated as possible.
Currently, the test modules (Test/Test*.py) are an integral part of the LNST package. They are available to controller machines as well as to slave machines. The inconsistency here is, that controller doesn't need to execute them. It only wants to take them and send them over to the slaves. I think that we could take them out from LNST core and treat them a little differently.
There are two categories of tests:
- upstream - currently lnst/Tests/Test*.py
- user-defined - configurable; defaults to ~/.lnst/Tests
We could put all the upstream tests to /usr/share/lnst instead of having them within the tree directly. The test execution would then work like this:
- Controller starts the test
- It will contact the slave and send him hashes of all the Test*.py commands that will be necessary to execute the recipe
- Slave will look within his tests cache located in /var/cache/lnst to see if it does have the tests already
- If not, controller will transfer all the files to the slave
- The test can begin :-)
The key feature is, that the slave would not search any directories other than his test cache, which would after a few executed recipes get almost 100% hit-rate. The great thing about the hashes is, that in case the test is changed in any way at the controller, it will be transfered again to the slave.
Consequentially to this, we could pack LNST into two (maybe three due to some common code) different packages:
- lnst-controller
- lnst-slave
- (lnst-common)
What do you think?
I like this very much. In fact I got a similar idea in the middle of reading your email :)
Let's do this.
-Radek _______________________________________________ LNST-developers mailing list LNST-developers@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/lnst-developers
Thu, Oct 18, 2012 at 01:14:39PM CEST, jpirko@redhat.com wrote:
Thu, Oct 18, 2012 at 12:43:00PM CEST, rpazdera@redhat.com wrote:
Hello!
This is (another) one of my endless proposal emails, sorry about that :).
We discussed a way of enabling users to define their own tests and tools without the necessity to add modules to site-packages/ on the controller and all the slaves by hand.
This should be done by creating a directory at the controller machine that will contain the tests. The controller machine will send them to the slave through a xml-rpc method and the slave will be able to use them.
I dug into this a little more today and I had some ideas that I'd like to present here.
There are two things we deal with nowadays. Python test modules that represent Test* commands directly and test tools that can be in an arbitrary format. The tools are later used by the modules, to perform test tasks.
I personally think, that this model is great. It allows you to set up everything you need at the controller, e.g, prepare the test tools, write a new command, that will allow using them with LNST, write the recipe and then one-click execute them in your lab. You don't have to ssh to 5 (or even more) different machines to install the tools. That's the most tedious thing about networking tests. Manually preparing all the machines for the tests. I think the key feature of LNST facility is making the test setup and execution as easy and as much automated as possible.
Currently, the test modules (Test/Test*.py) are an integral part of the LNST package. They are available to controller machines as well as to slave machines. The inconsistency here is, that controller doesn't need to execute them. It only wants to take them and send them over to the slaves. I think that we could take them out from LNST core and treat them a little differently.
There are two categories of tests:
- upstream - currently lnst/Tests/Test*.py
- user-defined - configurable; defaults to ~/.lnst/Tests
We could put all the upstream tests to /usr/share/lnst instead of having them within the tree directly. The test execution would then work like this:
- Controller starts the test
- It will contact the slave and send him hashes of all the Test*.py commands that will be necessary to execute the recipe
- Slave will look within his tests cache located in /var/cache/lnst to see if it does have the tests already
- If not, controller will transfer all the files to the slave
- The test can begin :-)
The key feature is, that the slave would not search any directories other than his test cache, which would after a few executed recipes get almost 100% hit-rate. The great thing about the hashes is, that in case the test is changed in any way at the controller, it will be transfered again to the slave.
Consequentially to this, we could pack LNST into two (maybe three due to some common code) different packages:
- lnst-controller
- lnst-slave
- (lnst-common)
What do you think?
I like this very much. In fact I got a similar idea in the middle of reading your email :)
Let's do this.
Yeah, I can't think of any reason why we should not do it this way and the concept sounds very reasonable. Great idea!
Ack
-Radek _______________________________________________ LNST-developers mailing list LNST-developers@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/lnst-developers
LNST-developers mailing list LNST-developers@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/lnst-developers
On Thu, Oct 18, 2012 at 01:31:18PM +0200, Jan Tluka wrote:
Thu, Oct 18, 2012 at 01:14:39PM CEST, jpirko@redhat.com wrote:
Thu, Oct 18, 2012 at 12:43:00PM CEST, rpazdera@redhat.com wrote:
Hello!
This is (another) one of my endless proposal emails, sorry about that :).
We discussed a way of enabling users to define their own tests and tools without the necessity to add modules to site-packages/ on the controller and all the slaves by hand.
This should be done by creating a directory at the controller machine that will contain the tests. The controller machine will send them to the slave through a xml-rpc method and the slave will be able to use them.
I dug into this a little more today and I had some ideas that I'd like to present here.
There are two things we deal with nowadays. Python test modules that represent Test* commands directly and test tools that can be in an arbitrary format. The tools are later used by the modules, to perform test tasks.
I personally think, that this model is great. It allows you to set up everything you need at the controller, e.g, prepare the test tools, write a new command, that will allow using them with LNST, write the recipe and then one-click execute them in your lab. You don't have to ssh to 5 (or even more) different machines to install the tools. That's the most tedious thing about networking tests. Manually preparing all the machines for the tests. I think the key feature of LNST facility is making the test setup and execution as easy and as much automated as possible.
Currently, the test modules (Test/Test*.py) are an integral part of the LNST package. They are available to controller machines as well as to slave machines. The inconsistency here is, that controller doesn't need to execute them. It only wants to take them and send them over to the slaves. I think that we could take them out from LNST core and treat them a little differently.
There are two categories of tests:
- upstream - currently lnst/Tests/Test*.py
- user-defined - configurable; defaults to ~/.lnst/Tests
We could put all the upstream tests to /usr/share/lnst instead of having them within the tree directly. The test execution would then work like this:
- Controller starts the test
- It will contact the slave and send him hashes of all the Test*.py commands that will be necessary to execute the recipe
- Slave will look within his tests cache located in /var/cache/lnst to see if it does have the tests already
- If not, controller will transfer all the files to the slave
- The test can begin :-)
The key feature is, that the slave would not search any directories other than his test cache, which would after a few executed recipes get almost 100% hit-rate. The great thing about the hashes is, that in case the test is changed in any way at the controller, it will be transfered again to the slave.
Consequentially to this, we could pack LNST into two (maybe three due to some common code) different packages:
- lnst-controller
- lnst-slave
- (lnst-common)
What do you think?
I like this very much. In fact I got a similar idea in the middle of reading your email :)
Let's do this.
Yeah, I can't think of any reason why we should not do it this way and the concept sounds very reasonable. Great idea!
Ack
Thank you for the feedback guys!
I worked more on the file transitions from the controller to the slaves and there are some more problems to be addressed.
Besides the tests, we have the the test tools, which can be virtually anything from a set of tools in C, to shell, ruby or perl scripts. We already have the multicast tools there for instance.
They need to be transfered and compiled on the slave machines in order to be used. We could do this by a reeeeally light-weight packages.
The controller would have the tools available in archives like this:
lnst-multicast-tools.tar.gz: multicast.h ... and all the other sources ... Makefile AND: lnst-setup.sh
LNST would then send the tools to a slave machine, unpack the archive and execute lnst-setup.sh. This script would prepare the tools for use for testing. In this case, the script would look like this:
#!/bin/bash
make
The slave would also keep an index of the archive hashes with file corresponding names. The package would be retransfered and rebuild only in case of a hash change.
I like this approach because it is easy for us to implement and it doesn't limit the user in any way for chosing test tools. You could wrap up tests from beaker for instance and run them in LNST.
Radek
-Radek _______________________________________________ LNST-developers mailing list LNST-developers@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/lnst-developers
LNST-developers mailing list LNST-developers@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/lnst-developers
lnst-developers@lists.fedorahosted.org