On Friday I went to execute MeasurementDataManagerBeanTest, which is now in the
server/itests-2 module. I was unpleasantly surprised by the execution time. Running
this,
$ mvn -o clean test -Dtest=MeasurementDataManagerBeanTest
was taking upwards of 50 seconds. I am running on a fast machine as well so I imagine the
situation only gets more bleak on slower hardware. There is nothing crazy going on in
MeasurementDataManagerBeanTest to cause such a long build time. I need to write more
tests, but I consider this is a blocker. The problem has to do with Arquillian, or more
specifically, how we are using it. In a local branch I did some prototyping using
AuthorizationManagerBeanTest, for reasons I will explain later, and was able to get the
execution time down to about 10 seconds. There are a few things I did to bring down the
execution time.
1) Do not redeploy AS 7 every test run.
Even when not doing a clean, I was still seeing 40+ second build times. Simply commenting
out the antrun plugin in pom.xml sped things up a little bit. We could put that plugin in
a profile so we are not always executing the Ant script(s) to deploy our AS 7 server. I do
something similar with Cassandra tests.
2) Use remote container instead of managed container
Arquillian handles starting and stopping a managed container whereas the lifecycle of a
remote container is not handled by Arquillian. I kept my AS 7 instance running. Maybe Jay
and/or Mazz have some insights into whether or not there might be issues with using a
remote container.
3) Introduce more modularity
I studied the timestamps in the log statements of tests runs to get a better feel for
where the most time is being spent. While there is not one huge culprit, the lack of
modularity is a big problem, and it will only get worse. We have lots of EJBs, and they
are split across only 2 EJB JAR files. One contains entity beans, and the other contains
all of our SLSBs and MDBs. The latter corresponds to the very large server/jar module.
When you go to run an Arquillian test, we deploy all of those EJBs along with some other
components. Given the sheer number of them, it is going to take some time for the
deployment to run.
In order to speed up deployments and execution times, we need to limit what we deploy to
test code, the production code under test, and required runtime dependencies. This however
is not possible with the source code organized and packaged as it is. In my branch I chose
AuthorizationManagerBean because of its minimal dependencies. It minimized the amount of
refactoring I had to do to get things running. I created a new module named
server-authorization that produces an EJB JAR artifact. My test produces/deploys a
JavaArchive much like the example at
http://arquillian.org/guides/getting_started_rinse_and_repeat/.
Modularizing our EJBs makes the build more complex, but it brings a lot of benefits. Prior
to the AS7 merge to master, SLSBs along with tests lived in server/jar. I rarely would
execute all of the tests in server/jar because of how long they take to run. Even running
a single test class took a non-trivial amount time due to the embedded container start
up/initialization time. Now consider the process with our Arquillian set up. Suppose I
make a change to MeasurementDataManagerBean that I want to test. Here are the steps,
Make code changes to MeasurementDataManagerBean and to MeasurementDataManagerBeanTest
mvn install server/jar
mvn install server/ear
mvn test -Dtest=MeasurementDataManagerBeanTest (from tests-2 module)
In addition to the already very long running time for the test, I have to rebuild both
server/jar and the EAR. With what I have done, both MeasurementDataManagerBean and
MeasurementDataManagerBeanTest might live in a module named server-metrics. The steps to
test some changes would be as follows,
Make code changes to MeasurementDataManagerBean and to MeasurementDataManagerBeanTest
mvn test -Dtest=MeasurementDataManagerBeanTest
There is no need to rebuild any other modules and the test presumably will execute a lot
faster as we only deploy the required artifacts instead of the whole EAR. There are more
benefits to increases modularity, but I will postpone those to a later email since this
has one has already gotten rather length.
- John