detailed code review for delete-agent-plugin branch
John Sanda
jsanda at redhat.com
Thu Dec 2 17:15:24 UTC 2010
On 12/2/10 11:41 AM, Joseph Marques wrote:
>
>
> commit 988bafd8d08916e30c93b4fa79d6e24a588a0401
> Author: John Sanda <jsanda at redhat.com>
> Date: Tue Nov 9 16:33:44 2010 -0500
>
> Fixing tests that were failing due to database clean up running too soon
>
> MetadataManager previously defined @BeforeClass and @AfterClass
> methods for
> setting up and tearing down each plugin meta data test class. The
> database was
> reset in the @BeforeClass method. Turns out that while the first few test
> methods of say OperationMetadataManagerBeanTest run, then before the
> rest of its
> test methods have executed the @BeforeClass method is executed for say
> ContentMetadataManagerBeanTest which wind up clearing out the same
> test data
> that OperationMetadataManagerBeanTest was using. To remedy the
> situation, I
> have made every configuration/test method in each
> XXXMetadataManagerBeanTest
> part of the plugin.metadata group. Then I changed the @BeforeClass and
> @AfterClass methods into @BeforeGroup and @AfterGroup methods
> respectively,
> esnuring that they only run once for the entire group.
>
> * so by virtue of using the strategy of completely wiping out the db,
> does this prevent
> us from leverage testng to run our tests in parallel now? how much
> slower does the
> test suite execute in this branch versus master?
>
Initially I was not using dbunit or anything else to reset tables that
are effected during a test run. I was using the pattern used in other
tests which is to rollback the commit and the end of the test run;
however, I needed to commit transactions in order to verify the state of
things. We have this in other tests as well in other parts of the code
base and the catch is, you might get unexpected results if you run those
multiple times without restoring the database. So after a test run and
prior to the next one I was restoring the database by running dbsetup.
And by test run, I don't mean hudson. I am talking about running test as
I was writing the code which would be rather frequent. Having to run
dbsetup as I was doing was flat out painful and really slowing me down.
Not only was it slow, there were plenty of times that I simply forgot to
run dbsetup, and tests would subsequently fail as a result.
While I do clear out a substantial number of tables, I do not completely
wipe out the database. I was not aware that we are running tests in
parallel; however, tests in the delete-agent-plugin branch are running
in the same way as they are in master. TestNG is pretty flexible in how
it allows you to specify what you want run in parallel. You can specify
entire suites be run in parallel, as well as methods and classes.
Granted if we start running tests in parallel, some of the set up logic
I put in place for these tests might have to be revisited, but I think
the same can be said for tests in other parts of the code base as well.
As for how overall execution time, I cannot say. I have not paid too
close attention. I do know though that due to the reasons I described
above running tests on my box while I am developing is faster and more
efficient. I expect the overall execution time to be at least somewhat
slower in the delete-agent-plugin branch than in master if for no other
reason than the fact that I added a substantial number of tests in the
branch. You can look at the coverage report numbers to get a better
sense of this.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: https://fedorahosted.org/pipermail/rhq-devel/attachments/20101202/173e029f/attachment.html
More information about the rhq-devel
mailing list