Automated API compatibility checking
Lukas Krejci
lkrejci at redhat.com
Fri Aug 10 16:45:20 UTC 2012
We've had a number of problems in the past with accidentally breaking API
compatibility of our remote (or otherwise accessible) APIs which lead to
unpleasant surprises for our users.
Internally we have experience with using animal-sniffer maven plugin which we
use to ensure that our codebase doesn't use any API's not present in the
minimal supported Java version (currently it's Java 6).
Our hope was to employ animal-sniffer in a broader effort of ensuring that our
codebase doesn't introduce any unplanned incompatibilities of publicly
accessible APIs of RHQ itself.
While working on this for some time I came to an important conclusion: Animal-
sniffer is NOT a tool for checking API incompatibilities. It is a tool for
ensuring that your code doesn't use APIs of third party libraries that the
target enviroment might not have available.
So we needed something different. Our testers use the java api compliance
checker (http://ispras.linuxbase.org/index.php/Java_API_Compliance_Checker) -
a perl program that analyses jar files for API changes. While it generates
quite nice reports, it is not nicely integrateable (if that's a word) into our
build process.
Another tool that is feature-wise comparable to the above compliance checker
is Clirr (http://mojo.codehaus.org/clirr-maven-plugin/index.html). It comes
with a nice maven integration and is able to produce its reports in either
text, xml or participate in Maven's site generation producing detailed html
report.
Even though we can get our APIs checked for compatiblity and we have that
phase integrated with our build, this is still not a fully automated process.
The problem is that the tool still finds some false positives.
For example, it flags a method added to an interface as an API breaking change.
While this is true in general case because any possible implementation of that
interface becomes invalid, in case of our remote API interfaces it is not
considered a breaking change - the users are not supposed to implement these
interfaces.
On the other hand this tool is great to detect the API breakage in the areas
we've had most problems with - the agent plugins. How many times has it
happened that a change in a "parent" plugin broke a plugin that inherited from
it?
So apart from tooling, this email is also to inform you about the initial set
of modules, that I've set up for API checking:
rhq-enterprise-server (only *Remote classes and ServerVersion)
rhq-core-domain
rhq-core-util
rhq-core-plugin-api
rhq-scripting-api
rhq-augeas-plugin
rhq-database-plugin
jopr-jboss-as-plugin
jopr-jboss-as-5-plugin
rhq-jboss-as-7-plugin
rhq-jmx-plugin
The API checking is switched on by specifying the RHQ version to check
against:
mvn install -Dsignature-check-base-version=4.4.0
This will produce XML reports in the target of the above mentioned projects.
Also if you manage to run the site generation on our build (i've not been able
to do so on the whole project, only module by module), you'll get a nice HTML
report on the compatibility in given module.
Note that this code is not yet committed as I first want to disscuss the choice
of the plugin to use and the set of the modules subject to API checking.
Cheers,
Lukas
More information about the rhq-devel
mailing list