On Mon, Sep 01, 2014 at 09:38:09PM +0200, Michal Židek wrote:
But the question here is: Do we need a framework for
this? I think it would be nice, and we may think
about adopting/implementing one in the future if the
script that we have now will not be sufficient, BUT I think
if we fix the few issues I mentioned in this thread
we will be happy (or at least mostly happy) about what
we have and will not need to expand functionality
in the future too much ( == we do not need any
framwork).
If I understood the meeting we had last week, this
script is part of tier 1 and 2 of the 3 tiered
test suite we were talking about and the framework/
tools used in the 3rd tier will be different (probably
adopted from IPA). That is why this script already
does almost everything we want from it.
I completely agree with what Michal said above.
For the CI tests we are planning in the mid-term, we really
should be using something very similar to what IPA has. IIRC the only
feature the IPA CI library was missing was VM provisioning. Currently
their tests require a hardcoded list of VMs in the Brno VM lab.
I suggest the best course of action would be to meet with the IPA
developers (IIRC Martin Kosek should be setting those meetings) and look
at how their test library can be improved and collaborated on.
Their tests are written in Python which is a language understood my many
developers, yet easy to use and expressive.