Continuous Integration results

Radek Novacek rnovacek at redhat.com
Tue Aug 27 14:32:34 UTC 2013


Hi all,

BuildBot instance is up and running for some time now [1]. 

There is a couple of issues I would like to discuss here:

1) The tests are fragile
  
  When some test fails to clean after itself it might affect other test or the 
next run of the test. We want to make sure that we make the tests as solid as 
possible but there will always be troubles (like issues in provider backends - 
blivet, NetworkManager). I'm working on improving this - I'm going to remove 
all openlmi-* packages, CIMOM and any other leftovers (like CIMOM repository 
in /var) before (or rather after?) the test.

2) Results are not visible enough

  There is IRC bot on #openlmi channel of freenode notifying when test status 
changes (it used to fail and now it succeeds and vice versa). But it not 
visible enough. Also we don't want to send email to this list because it will 
be quite a lot of traffic. So, should I create another mailing list? openlmi-
build? openlmi-ci?

3) Building machines updates

  Now the test machines are updated manually but this should be probably 
automated too. But we need to know whether the build/test fails because of 
change in providers or system update. So the workflow should probably be like:

* Build our packages and run the test suite
* Update the system
* Build and test our packages again to see if the result is the same
* Report it

Does it make sense? Any better ideas?



Tips and suggestions how to deal with these issues are welcomed.

Radek Novacek

[1] http://openlmi-rnovacek.rhcloud.com/waterfall


More information about the openlmi-devel mailing list