testing without the embedded container

Joseph Marques jmarques at redhat.com
Sun Dec 5 22:48:23 UTC 2010


Some comments / questions in line...

On 11/29/2010 03:33 PM, John Sanda wrote:
> A while back when I was testing JPQL queries that I had updated to
> account for resource types being deleted, I got tired off using the
> embedded container. The start up time in particular is what wore my
> patience thin.

Just an FYI, but I've found that the best way to write the 
implementation for some (non-trivial) NamedQuery definition is *not* to 
start by writing a unit test.  Since you might need to write it, tweak 
it, write it, tweak it...I've found the best way is to try and mimic for 
HQL/JPQL what you would get by using a native SQL client.  To that end, 
I will either:

1) Use some IDE that supports HQL/JPQL execution against the local 
database.  I use the hibernate tools plugin for eclipse which allows me 
to write the query and get results all without leaving the IDE.  It 
provides things like syntax assistance, code highlighting, and more.
2) Fire up an instance of some RHQ server and use the 
"/admin/test/hibernate.jsp" page.  This allows me to easily build 
queries piecemeal, adding and testing named parameters of any type, 
doing immediate testing and rapid verification of all types of queries 
(select/insert/update/delete), etc.

Once you're satisfied with the initial development of the query, then it 
can be converted into a NamedQuery and have unit tests written for it.

> ...
> 3) Integration with DbUnit. Unitils executes a dbunit data set
> immediately before each test method is executed
> ...

Before each method, or before each class?

> There are a number of upsides to this change. The overall build time for
> core/domain is a good bit faster.

If all of the tests in core/domain are being executed, I wouldn't have 
expected the end-to-end running time to be reduced all that much.  The 
embedded container bootstrap only runs once, so it's cost would be 
amortized over the test phase for the entire module.  As the number of 
tests grow, their contribution to overall execution time should outpace 
any bootstrap time.  Are you saying that you saw a reduction in time 
elsewhere besides just the embedded container bootstrap?

> More importantly though, executing a
> single test class is substantially faster since there is no longer the
> overhead of bootstrapping the embedded container.

Yeah, I can *definitely* see how if you're writing individual tests this 
will come in handy.  What would be even better is if a developer didn't 
have to drop to the command line at all.  It would be nice if they could 
- from within their IDE - quickly and easily execute all of the test 
methods in a single test class, or even just execute a single test 
method.  If we can lower the learning curve to that point, I think more 
people would be more willing to write more comprehensive test suites 
(because they could be produced in a fraction of the time that it takes 
to do so today).

> ... This *does* mean that tests are
> committing transactions, and that's ok since each test resets the
> database (or more precisely, relevant parts of the database) before it
> runs. Committing transactions has the added benefit of being able to
> inspect the state of the database after a test failure.

Being able to inspect the database locally is nice, but when we run the 
tests in a Hudson environment we need to make sure our assertion 
messages provide adequate information about failures to aide developers 
in more quickly analysing the root cause without needing access to the 
remote DB instance.  It's even more important to have good assert 
messages for the cases where something fails in Hudson but not 
locally...or fails locally but not in Hudson -- in these cases having 
access to the database may not be as useful as a message indicating what 
the expectations where at the point of failure.

> ...
> Using dbunit can and does simplify the set up logic for tests. It
> provides a way to do test set up independent of the code under test. It
> also makes for good documentation and makes tests more readable.
> ...
> Defining the dbunit data sets in xml might be considered another
> downside. It limits the use of some refactoring tools. If I change part
> of an entity mapping in the Java source file, I will have to
> independently make any changes necessary to the data set file.

Sometimes the data construction itself is the thing that needs testing, 
which happens in the case where the unit test's job is to explicitly 
test that we have set up the entity cascading rules properly (inserts, 
updates, delete-orphans, etc).  In this case, we can't make the test set 
up independent of the code under test, because the set up of data is 
literally the thing that needs testing.

> ...
> I want to emphasize that my goal was not to simply replace the embedded
> container with unitils. Rather, the intention was to make tests faster,
> more concise, more reliabe, and easier to maintain. Unitils is just one
> way to achieve this, but I do think it is a very good way to achieve it.
> I did not integrate unitils into any of the server/jar tests since we
> rely heavily on the embedded container there for accessing our SLSBs.

Keep in mind that we're starting to move away from NamedQueries these 
days in favor of criteria-based generated queries.  These have proven to 
be useful in a wide variety of circumstances thus far, and should over 
time supplant the need for writing functionality using NamedQueries in 
all but the most extreme cases.  So, one thing we should probably look 
in to doing is leveraging the JPATest class for unit testing criteria 
objects.


More information about the rhq-devel mailing list