Reduce Database Configuration Complexity - Proposal

Ian Springer ian.springer at redhat.com
Tue Jul 26 15:10:18 UTC 2011


>>  A lot of the scripts/tests only use rhq.test.ds (and completely ignore rhq.dev.ds or the flag).

This is how it's supposed to work - tests should always use the test db (the one defined by the rhq.test.ds props). The 'db' prop is only used to toggle which db dbsetup/dbupgrade/dbreset should use.

>>  The original dbreset was not making full use of the settings either. At any single time, it is possible to use only one set.

That is also how it's suppose to work. dbsetup/dbupgrade/dbreset should 
only use a single db per mvn run - the one specified by the 'db' prop. 
If db=dev, then the db specified by the rhq.dev.ds props should be used, 
and if db=test, the db specified by the rhq.test.ds props should be 
used. This is meant to allow a dev to specify which db they would like 
to setup, upgrade, or reset.

Stefan, in case you haven't already found this, here's the wiki page 
that documents the db build props:

http://rhq-project.org/display/RHQ/Advanced+Build+Notes#AdvancedBuildNotes-PurgingandUpdatingtheDatabaseSchema

Basically, the test db is supposed to always be the one used by tests, 
and the dev db is used as the dev-container's db (configured via 
rhq-server.properties). I think the reason we split it into two db's 
configured via two sets of properties is to support running tests with 
the dev profile enabled and knowing that those tests will not mess with 
your dev-container's db and potentially corrupt it, and that the 
dev-container will not mess with the test db and potentially cause false 
failures. I think most devs have the dev profile activated by default in 
their settings.xml. Assuming that's true, if there was only one set of 
db props and the dev profile set those props to the dev db, whenever a 
dev wanted to run the domain or server-jar tests, they would have to 
remember to first edit their settings.xml and deactivate the dev 
profile; otherwise, their dev db would get stomped on by the tests. You 
could always specify the test db by overriding the db props via -D on 
the command line or by specifying an alternate settings file 
(test-settings.xml), but then if you happened to also build with 
-Penterprise, your dev-containers db would suddenly get switched to the 
test db. However, that may be acceptable, since it's not often devs will 
need to run the tests and rebuild the dev-container in the same mvn run. 
It's also worth considering the use case where a dev uses postgres as 
their dev db, but occasionally wants to run the tests against an oracle 
db - in that case, overriding the props via the command line would not 
be practical since they'd have to override all of them, not just 
db.name. So for that case they'd have to use an alternate settings.xml 
that specifies the Oracle db and remember not to enable the enterprise 
profile.

I have been happy with the way it is, because I just configure both db's 
in my settings.xml and never have to worry about any of the above. At 
the time the change was made, I got the impression other members of the 
team also were happier with it. I'm not sure what you mean about falling 
back to the other db for dbreset, etc. If the props for the db specified 
by the 'db' prop aren't defined or have invalid values, I would still 
use them and just let it fail. There are defaults for both sets of 
properties defined in the root pom, and these can be overridden in 
settings.xml as desired.

-- Ian

On 07/25/2011 05:56 PM, John Mazzitelli wrote:
> there was a reason we have two. we USED to have one, like you propose.
> We added the second.
>
> Ian knows more (we did this a few years ago) and it was mainly to
> support (IIRC) the ability to build releases in addition to being able
> to build dev builds.
>
> Wait for ips to come back and let him chime in on this.
>
> On 07/25/2011 05:27 PM, Stefan Negrea wrote:
>> Hello Everybody,
>>
>> I have been working on BZ 7222298 (https://bugzilla.redhat.com/show_bug.cgi?id=722298) to port dbreset to Oracle. During this work I got confused over data source configuration.
>>
>> Currently there are two sets of settings (rhq.test.ds and rhq.dev.ds) for data source configuration (primarily used for unit tests). The two sets are almost identical (username, passsword, type-mapping), the main difference is db-name. Additionally, there is a flag used to switch between the two sets. And here is where the confusion starts... Not all the scripts use (or use correctly) the flag to switch between the settings. A lot of the scripts/tests only use rhq.test.ds (and completely ignore rhq.dev.ds or the flag). The original dbreset was not making full use of the settings either. At any single time, it is possible to use only one set.
>>
>> My initial dbreset port was very confusing and complicated because there are so many settings. I had to take into account fall-back scenarios (when one setting was not defined for one profile but it was a for the other). So, I took a step back and looked at ways to simplify/improve this.
>>
>> And here is my proposal:
>> 1) Remove both rhq.dev.ds and rhq.test.ds; and remove the flag to switch between the two sets.
>> 2) Consolidate everything into one set of settings - rhq.ds.
>> 3) To use a different database for building/testing/resetting the db then just override rhq.ds (eg. -Drhq.ds.db-name=rhq_dev) at command line. Or better, create separate profiles in .m2/settings.xml and activate each profile as needed for maven builds.
>>
>> Having only one set will:
>> 1) make it easy to understand data source settings (only rhq.ds to configure)
>> 2) reduce the code complexity (no fall-back, no flags to switch between settings).
>> 3) make it easy to override the correct settings for your particular database configuration.
>> 4) make new unit tests consistent and reduce the possibility to make mistakes.
>>
>> Here is the branch with the changes I made for dbreset:
>> http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=shortlog;h=refs/heads/stefan/dbreset
>>
>> And here is the commit with most of the changes proposed for rhq.ds:
>> http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=commit;h=187643
>>
>>
>> Any suggestions or input for my proposal? Is there a reason to keep two sets of tests? Are there any use cases that cannot be accomplished by profiles/settings override?
>>
>>
>> Thank you,
>> Stefan Negrea
>>
>> Software Engineer
>> _______________________________________________
>> rhq-devel mailing list
>> rhq-devel at lists.fedorahosted.org
>> https://fedorahosted.org/mailman/listinfo/rhq-devel
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel


-- 
Ian Springer
Principal Software Developer
JBoss Operations Network
Red Hat
ian.springer at redhat.com



More information about the rhq-devel mailing list