On 15.9.2015 12:08, Bohuslav Kabrda wrote:
----- Original Message -----
> Hi all,
>
> since we are preparing Fedora Developer Portal[0, testing instance]
> which will feature various assistants (language assistants, Vagrant
> assistant) I would like to make sure that they actually work and are
> up-to-date. E.g. we mention Ruby assistant for installing Rails that
> does not install right set of packages on Fedora 22[0]. Therefore it
> would be quite beneficial having tests/CI for those assistants (at least
> the important ones).
>
> Testing it manually is time-consuming and currently not done by anyone.
> And I think that's not acceptable for the future especially if
> DevAssistant gets such a big promotion in almost any content on the
> Fedora Developer Portal.
>
> Ideas how to solve it?
Hey,
this has been on my mind for quite some time. I'm not sure how precisely the solution
should look like, but I have some thoughts:
- I think we should have a testing framework for assistants (doesn't have to be
complex, just several testing libraries glued together in the right way(TM))
- I think that generally it would make sense to use BDD for testing assistants, Gherkin
[1] would be nice. We could have test like:
Given I install Django assistant
When I run Django assistant
And I provide "foo" as project name
Then a directory called "foo" should be created in "~"
And the directory "~/foo" should contain "setup.py" file
- So putting together behave (which implements BDD testing with Gherkin for Python) with
e.g. py.test should do the trick (in fact, the BDD based testing framework wouldn't
necessarily have to be written in Python...)
- The testing framework should probably test assistants in containers to prevent messing
with user's system and achieve reproducible results
I'd love to get something like this going but, as usual, time is the issue. If you
know someone who would like to work on this, I'll be happy to provide him with more
detailed notes and guide him through this.
I've already discussed using Tunir [1] for testing all this in Docker
containers with different Fedoras on e.g. Travis CI with Kushal Das (the
author of Tunir) and it seems quite possible.
I will look at it when our current server-client goal is reached.
[1]
http://tunir.readthedocs.org/
--
Miro Hrončok
--
Phone: +420777974800
IRC: mhroncok