Reduce Database Configuration Complexity - Proposal

Stefan Negrea snegrea at redhat.com
Thu Jul 28 19:09:47 UTC 2011


I talked yesterday with Ian (thanks Ian!) to clarify how rhq.test.ds and rhq.dev.ds should work. My proposal cannot implement some essential use cases supported by two sets of settings. So, I will revert my proposed change. I will implement dbreset to comply with the way the two sets should be used. I will also review and update the rest of the code to make correct use of the flags and settings. 

Here are the rules on how the two sets should be used:
1) Always use rhq.test.ds for unit tests.
2) Use rhq.dev.ds for dbreset/dbsetup/dbupgrade if the dev profile (-Pdev) is activated. Fallback to rhq.test.ds if the dev profile is not activated.
3) Use rhq.dev.ds for building dev containers if the dev profile is activate. Fallback to settings stored in the container configuration is the dev profile is not activated.

I will update the wiki page from Ian's email to further clarify these options.

Any feedback or suggestions?


Thank you,
Stefan Negrea

Software Engineer

----- Original Message -----
> From: "Ian Springer" <ian.springer at redhat.com>
> To: rhq-devel at lists.fedorahosted.org
> Sent: Tuesday, July 26, 2011 10:10:18 AM
> Subject: Re: Reduce Database Configuration Complexity - Proposal
> >>  A lot of the scripts/tests only use rhq.test.ds (and completely
> >>  ignore rhq.dev.ds or the flag).
> 
> This is how it's supposed to work - tests should always use the test
> db (the one defined by the rhq.test.ds props). The 'db' prop is only
> used to toggle which db dbsetup/dbupgrade/dbreset should use.
> 
> >>  The original dbreset was not making full use of the settings
> >>  either. At any single time, it is possible to use only one set.
> 
> That is also how it's suppose to work. dbsetup/dbupgrade/dbreset
> should
> only use a single db per mvn run - the one specified by the 'db' prop.
> If db=dev, then the db specified by the rhq.dev.ds props should be
> used,
> and if db=test, the db specified by the rhq.test.ds props should be
> used. This is meant to allow a dev to specify which db they would like
> to setup, upgrade, or reset.
> 
> Stefan, in case you haven't already found this, here's the wiki page
> that documents the db build props:
> 
> http://rhq-project.org/display/RHQ/Advanced+Build+Notes#AdvancedBuildNotes-PurgingandUpdatingtheDatabaseSchema
> 
> Basically, the test db is supposed to always be the one used by tests,
> and the dev db is used as the dev-container's db (configured via
> rhq-server.properties). I think the reason we split it into two db's
> configured via two sets of properties is to support running tests with
> the dev profile enabled and knowing that those tests will not mess
> with
> your dev-container's db and potentially corrupt it, and that the
> dev-container will not mess with the test db and potentially cause
> false
> failures. I think most devs have the dev profile activated by default
> in
> their settings.xml. Assuming that's true, if there was only one set of
> db props and the dev profile set those props to the dev db, whenever a
> dev wanted to run the domain or server-jar tests, they would have to
> remember to first edit their settings.xml and deactivate the dev
> profile; otherwise, their dev db would get stomped on by the tests.
> You
> could always specify the test db by overriding the db props via -D on
> the command line or by specifying an alternate settings file
> (test-settings.xml), but then if you happened to also build with
> -Penterprise, your dev-containers db would suddenly get switched to
> the
> test db. However, that may be acceptable, since it's not often devs
> will
> need to run the tests and rebuild the dev-container in the same mvn
> run.
> It's also worth considering the use case where a dev uses postgres as
> their dev db, but occasionally wants to run the tests against an
> oracle
> db - in that case, overriding the props via the command line would not
> be practical since they'd have to override all of them, not just
> db.name. So for that case they'd have to use an alternate settings.xml
> that specifies the Oracle db and remember not to enable the enterprise
> profile.
> 
> I have been happy with the way it is, because I just configure both
> db's
> in my settings.xml and never have to worry about any of the above. At
> the time the change was made, I got the impression other members of
> the
> team also were happier with it. I'm not sure what you mean about
> falling
> back to the other db for dbreset, etc. If the props for the db
> specified
> by the 'db' prop aren't defined or have invalid values, I would still
> use them and just let it fail. There are defaults for both sets of
> properties defined in the root pom, and these can be overridden in
> settings.xml as desired.
> 
> -- Ian
> 
> On 07/25/2011 05:56 PM, John Mazzitelli wrote:
> > there was a reason we have two. we USED to have one, like you
> > propose.
> > We added the second.
> >
> > Ian knows more (we did this a few years ago) and it was mainly to
> > support (IIRC) the ability to build releases in addition to being
> > able
> > to build dev builds.
> >
> > Wait for ips to come back and let him chime in on this.
> >
> > On 07/25/2011 05:27 PM, Stefan Negrea wrote:
> >> Hello Everybody,
> >>
> >> I have been working on BZ 7222298
> >> (https://bugzilla.redhat.com/show_bug.cgi?id=722298) to port
> >> dbreset to Oracle. During this work I got confused over data source
> >> configuration.
> >>
> >> Currently there are two sets of settings (rhq.test.ds and
> >> rhq.dev.ds) for data source configuration (primarily used for unit
> >> tests). The two sets are almost identical (username, passsword,
> >> type-mapping), the main difference is db-name. Additionally, there
> >> is a flag used to switch between the two sets. And here is where
> >> the confusion starts... Not all the scripts use (or use correctly)
> >> the flag to switch between the settings. A lot of the scripts/tests
> >> only use rhq.test.ds (and completely ignore rhq.dev.ds or the
> >> flag). The original dbreset was not making full use of the settings
> >> either. At any single time, it is possible to use only one set.
> >>
> >> My initial dbreset port was very confusing and complicated because
> >> there are so many settings. I had to take into account fall-back
> >> scenarios (when one setting was not defined for one profile but it
> >> was a for the other). So, I took a step back and looked at ways to
> >> simplify/improve this.
> >>
> >> And here is my proposal:
> >> 1) Remove both rhq.dev.ds and rhq.test.ds; and remove the flag to
> >> switch between the two sets.
> >> 2) Consolidate everything into one set of settings - rhq.ds.
> >> 3) To use a different database for building/testing/resetting the
> >> db then just override rhq.ds (eg. -Drhq.ds.db-name=rhq_dev) at
> >> command line. Or better, create separate profiles in
> >> .m2/settings.xml and activate each profile as needed for maven
> >> builds.
> >>
> >> Having only one set will:
> >> 1) make it easy to understand data source settings (only rhq.ds to
> >> configure)
> >> 2) reduce the code complexity (no fall-back, no flags to switch
> >> between settings).
> >> 3) make it easy to override the correct settings for your
> >> particular database configuration.
> >> 4) make new unit tests consistent and reduce the possibility to
> >> make mistakes.
> >>
> >> Here is the branch with the changes I made for dbreset:
> >> http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=shortlog;h=refs/heads/stefan/dbreset
> >>
> >> And here is the commit with most of the changes proposed for
> >> rhq.ds:
> >> http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=commit;h=187643
> >>
> >>
> >> Any suggestions or input for my proposal? Is there a reason to keep
> >> two sets of tests? Are there any use cases that cannot be
> >> accomplished by profiles/settings override?
> >>
> >>
> >> Thank you,
> >> Stefan Negrea
> >>
> >> Software Engineer
> >> _______________________________________________
> >> rhq-devel mailing list
> >> rhq-devel at lists.fedorahosted.org
> >> https://fedorahosted.org/mailman/listinfo/rhq-devel
> > _______________________________________________
> > rhq-devel mailing list
> > rhq-devel at lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/rhq-devel
> 
> 
> --
> Ian Springer
> Principal Software Developer
> JBoss Operations Network
> Red Hat
> ian.springer at redhat.com
> 
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel


More information about the rhq-devel mailing list