Greetings,
I've been doing some work on our autobuild setup. I'm about done with what I was going to do right now so here are some thoughts about the state of things before I move on to something else.
The Matahari Project has been running an autobuild instance for a good while. You can find the latest autobuild results here:
http://et3.redhat.com/hosted/matahari-rawhide/
If you look in the matahari source, the top level directory has a script, autobuild.sh. This is what is run by autobuild. It uses mock to do builds of both matahari and mingw32-matahari RPMs. There has recently been some expansion to what autobuild covers:
1) The C API unit tests are now run.
2) Dave Johnson submitted a suite of tests that exercise the QMF APIs. (Thanks, Dave!) I have put some work into making them compatible with running inside of mock.
There are fairly extensive tests written against the Host and Sysconfig QMF APIs that run in autobuild. It runs some scaled down tests (ensure the agent starts, connects to a broker, and we can retrieve a property) for the Resources, Services, and Network QMF APIs.
Potential future improvements for the backlog:
1) Resources, Services, Network QMF APIs
We have code that exercises these APIs more than what is being done in autobuild at the moment. These APIs perform operations that aren't really compatible with running inside of mock, so we'll have to explore a different approach on enabling automated extensive tests of these APIs (such as spinning up a VM).
2) Windows runtime tests
We do automated Windows builds, but we don't run them. It would be good to eventually spin up a Windows VM, install everything and run it there, too.
3) Code coverage
It would be nice to have an idea of the code coverage from these tests. I've used lcov [1] before and it worked quite well. I think we should integrate it at some point.
4) D-bus tests
We've got a number of tests running against the QMF APIs, but it would be good to add tests that exercise the APIs over D-bus, as well.
Thanks,
[1] http://ltp.sourceforge.net/coverage/lcov.php