Automated API compatibility checking

Lukas Krejci lkrejci at redhat.com
Fri Aug 10 16:45:20 UTC 2012


We've had a number of problems in the past with accidentally breaking API 
compatibility of our remote (or otherwise accessible) APIs which lead to 
unpleasant surprises for our users.

Internally we have experience with using animal-sniffer maven plugin which we 
use to ensure that our codebase doesn't use any API's not present in the 
minimal supported Java version (currently it's Java 6).

Our hope was to employ animal-sniffer in a broader effort of ensuring that our 
codebase doesn't introduce any unplanned incompatibilities of publicly 
accessible APIs of RHQ itself.

While working on this for some time I came to an important conclusion: Animal-
sniffer is NOT a tool for checking API incompatibilities. It is a tool for 
ensuring that your code doesn't use APIs of third party libraries that the 
target enviroment might not have available.

So we needed something different. Our testers use the java api compliance 
checker  (http://ispras.linuxbase.org/index.php/Java_API_Compliance_Checker) - 
a perl program that analyses jar files for API changes. While it generates 
quite nice reports, it is not nicely integrateable (if that's a word) into our 
build process.

Another tool that is feature-wise comparable to the above compliance checker 
is Clirr (http://mojo.codehaus.org/clirr-maven-plugin/index.html). It comes 
with a nice maven integration and is able to produce its reports in either 
text, xml or participate in Maven's site generation producing detailed html 
report.

Even though we can get our APIs checked for compatiblity and we have that 
phase integrated with our build, this is still not a fully automated process.

The problem is that the tool still finds some false positives.

For example, it flags a method added to an interface as an API breaking change. 
While this is true in general case because any possible implementation of that 
interface becomes invalid, in case of our remote API interfaces it is not 
considered a breaking change - the users are not supposed to implement these 
interfaces.

On the other hand this tool is great to detect the API breakage in the areas 
we've had most problems with - the agent plugins. How many times has it 
happened that a change in a "parent" plugin broke a plugin that inherited from 
it?

So apart from tooling, this email is also to inform you about the initial set 
of modules, that I've set up for API checking:

rhq-enterprise-server (only *Remote classes and ServerVersion)
rhq-core-domain
rhq-core-util
rhq-core-plugin-api
rhq-scripting-api
rhq-augeas-plugin
rhq-database-plugin
jopr-jboss-as-plugin
jopr-jboss-as-5-plugin
rhq-jboss-as-7-plugin
rhq-jmx-plugin

The API checking is switched on by specifying the RHQ version to check 
against:

mvn install -Dsignature-check-base-version=4.4.0

This will produce XML reports in the target of the above mentioned projects.

Also if you manage to run the site generation on our build (i've not been able 
to do so on the whole project, only module by module), you'll get a nice HTML 
report on the compatibility in given module.

Note that this code is not yet committed as I first want to disscuss the choice 
of the plugin to use and the set of the modules subject to API checking.

Cheers,

Lukas



More information about the rhq-devel mailing list