----- Original Message -----
Hi all,
since we are preparing Fedora Developer Portal[0, testing instance]
which will feature various assistants (language assistants, Vagrant
assistant) I would like to make sure that they actually work and are
up-to-date. E.g. we mention Ruby assistant for installing Rails that
does not install right set of packages on Fedora 22[0]. Therefore it
would be quite beneficial having tests/CI for those assistants (at least
the important ones).
Testing it manually is time-consuming and currently not done by anyone.
And I think that's not acceptable for the future especially if
DevAssistant gets such a big promotion in almost any content on the
Fedora Developer Portal.
Ideas how to solve it?
Hey,
this has been on my mind for quite some time. I'm not sure how precisely the solution
should look like, but I have some thoughts:
- I think we should have a testing framework for assistants (doesn't have to be
complex, just several testing libraries glued together in the right way(TM))
- I think that generally it would make sense to use BDD for testing assistants, Gherkin
[1] would be nice. We could have test like:
Given I install Django assistant
When I run Django assistant
And I provide "foo" as project name
Then a directory called "foo" should be created in "~"
And the directory "~/foo" should contain "setup.py" file
- So putting together behave (which implements BDD testing with Gherkin for Python) with
e.g. py.test should do the trick (in fact, the BDD based testing framework wouldn't
necessarily have to be written in Python...)
- The testing framework should probably test assistants in containers to prevent messing
with user's system and achieve reproducible results
I'd love to get something like this going but, as usual, time is the issue. If you
know someone who would like to work on this, I'll be happy to provide him with more
detailed notes and guide him through this.
Thanks,
Slavek
[1]
https://en.wikipedia.org/wiki/Behavior-driven_development#Behavioural_spe...
[2]
https://pypi.python.org/pypi/behave