Before we had tests for the Cobbler API directly, that missed the XMLRPC
API.
Later we had tests for the remote.py API, but not the webapp.
We never really had solid command line tests.
Now we have a command line that uses new invocations on top of things,
and a web app that is slightly different than before.
I think this is a good occasion to revisit (i.e. gut) our existing test
infrastructure.
Ideally I think I would like to see tests that:
(A) Exercise the command line directly, and check that it makes the
changes it is supposed to make via the presense/absense/correctness of
JSON files, filesystem files, and correct output in some cases -- rather
not testing just the internals.
(B) Doing the same, eventually, for Cobbler Web, whether through
Selenium or something else.
What we currently have for api.py unit tests are largely /not/ correct
anymore, and they are difficult to maintain. So, once we conclude the
CLI modifications I
think that would be a good thing to do.
Does anyone with experience in test automation want to help with
something like this?
I'd like to see more real world test cases hitting the full scope of the
application, versus just doing what are just a little bit beyond "unit"
tests now (which from experience are not that useful).
I'd basically (for starters) just like to make sure that all actions can
fire, all pages don't break, and that object manipulations all take
effect ... so that if we are later working around on
optimizations/cleanup to any of the underlying code/infrastructure we
have something to detect unexpected breakage.
--Michael
Show replies by date