Reduce Database Configuration Complexity - Proposal

Charles Crouch ccrouch at redhat.com
Thu Jul 28 22:06:55 UTC 2011



----- Original Message -----
> I talked yesterday with Ian (thanks Ian!) to clarify how rhq.test.ds
> and rhq.dev.ds should work. My proposal cannot implement some
> essential use cases supported by two sets of settings. So, I will
> revert my proposed change. I will implement dbreset to comply with the
> way the two sets should be used. I will also review and update the
> rest of the code to make correct use of the flags and settings.
> 
> Here are the rules on how the two sets should be used:
> 1) Always use rhq.test.ds for unit tests.
> 2) Use rhq.dev.ds for dbreset/dbsetup/dbupgrade if the dev profile
> (-Pdev) is activated. Fallback to rhq.test.ds if the dev profile is
> not activated.
> 3) Use rhq.dev.ds for building dev containers if the dev profile is
> activate. Fallback to settings stored in the container configuration
> is the dev profile is not activated.
> 
> I will update the wiki page from Ian's email to further clarify these
> options.
> 
> Any feedback or suggestions?

Lets doc the heck out of this, with examples. So that new folks coming to the project can more easily get their head around the differences.

> 
> 
> Thank you,
> Stefan Negrea
> 
> Software Engineer
> 
> ----- Original Message -----
> > From: "Ian Springer" <ian.springer at redhat.com>
> > To: rhq-devel at lists.fedorahosted.org
> > Sent: Tuesday, July 26, 2011 10:10:18 AM
> > Subject: Re: Reduce Database Configuration Complexity - Proposal
> > >>  A lot of the scripts/tests only use rhq.test.ds (and completely
> > >>  ignore rhq.dev.ds or the flag).
> >
> > This is how it's supposed to work - tests should always use the test
> > db (the one defined by the rhq.test.ds props). The 'db' prop is only
> > used to toggle which db dbsetup/dbupgrade/dbreset should use.
> >
> > >>  The original dbreset was not making full use of the settings
> > >>  either. At any single time, it is possible to use only one set.
> >
> > That is also how it's suppose to work. dbsetup/dbupgrade/dbreset
> > should
> > only use a single db per mvn run - the one specified by the 'db'
> > prop.
> > If db=dev, then the db specified by the rhq.dev.ds props should be
> > used,
> > and if db=test, the db specified by the rhq.test.ds props should be
> > used. This is meant to allow a dev to specify which db they would
> > like
> > to setup, upgrade, or reset.
> >
> > Stefan, in case you haven't already found this, here's the wiki page
> > that documents the db build props:
> >
> > http://rhq-project.org/display/RHQ/Advanced+Build+Notes#AdvancedBuildNotes-PurgingandUpdatingtheDatabaseSchema
> >
> > Basically, the test db is supposed to always be the one used by
> > tests,
> > and the dev db is used as the dev-container's db (configured via
> > rhq-server.properties). I think the reason we split it into two db's
> > configured via two sets of properties is to support running tests
> > with
> > the dev profile enabled and knowing that those tests will not mess
> > with
> > your dev-container's db and potentially corrupt it, and that the
> > dev-container will not mess with the test db and potentially cause
> > false
> > failures. I think most devs have the dev profile activated by
> > default
> > in
> > their settings.xml. Assuming that's true, if there was only one set
> > of
> > db props and the dev profile set those props to the dev db, whenever
> > a
> > dev wanted to run the domain or server-jar tests, they would have to
> > remember to first edit their settings.xml and deactivate the dev
> > profile; otherwise, their dev db would get stomped on by the tests.
> > You
> > could always specify the test db by overriding the db props via -D
> > on
> > the command line or by specifying an alternate settings file
> > (test-settings.xml), but then if you happened to also build with
> > -Penterprise, your dev-containers db would suddenly get switched to
> > the
> > test db. However, that may be acceptable, since it's not often devs
> > will
> > need to run the tests and rebuild the dev-container in the same mvn
> > run.
> > It's also worth considering the use case where a dev uses postgres
> > as
> > their dev db, but occasionally wants to run the tests against an
> > oracle
> > db - in that case, overriding the props via the command line would
> > not
> > be practical since they'd have to override all of them, not just
> > db.name. So for that case they'd have to use an alternate
> > settings.xml
> > that specifies the Oracle db and remember not to enable the
> > enterprise
> > profile.
> >
> > I have been happy with the way it is, because I just configure both
> > db's
> > in my settings.xml and never have to worry about any of the above.
> > At
> > the time the change was made, I got the impression other members of
> > the
> > team also were happier with it. I'm not sure what you mean about
> > falling
> > back to the other db for dbreset, etc. If the props for the db
> > specified
> > by the 'db' prop aren't defined or have invalid values, I would
> > still
> > use them and just let it fail. There are defaults for both sets of
> > properties defined in the root pom, and these can be overridden in
> > settings.xml as desired.
> >
> > -- Ian
> >
> > On 07/25/2011 05:56 PM, John Mazzitelli wrote:
> > > there was a reason we have two. we USED to have one, like you
> > > propose.
> > > We added the second.
> > >
> > > Ian knows more (we did this a few years ago) and it was mainly to
> > > support (IIRC) the ability to build releases in addition to being
> > > able
> > > to build dev builds.
> > >
> > > Wait for ips to come back and let him chime in on this.
> > >
> > > On 07/25/2011 05:27 PM, Stefan Negrea wrote:
> > >> Hello Everybody,
> > >>
> > >> I have been working on BZ 7222298
> > >> (https://bugzilla.redhat.com/show_bug.cgi?id=722298) to port
> > >> dbreset to Oracle. During this work I got confused over data
> > >> source
> > >> configuration.
> > >>
> > >> Currently there are two sets of settings (rhq.test.ds and
> > >> rhq.dev.ds) for data source configuration (primarily used for
> > >> unit
> > >> tests). The two sets are almost identical (username, passsword,
> > >> type-mapping), the main difference is db-name. Additionally,
> > >> there
> > >> is a flag used to switch between the two sets. And here is where
> > >> the confusion starts... Not all the scripts use (or use
> > >> correctly)
> > >> the flag to switch between the settings. A lot of the
> > >> scripts/tests
> > >> only use rhq.test.ds (and completely ignore rhq.dev.ds or the
> > >> flag). The original dbreset was not making full use of the
> > >> settings
> > >> either. At any single time, it is possible to use only one set.
> > >>
> > >> My initial dbreset port was very confusing and complicated
> > >> because
> > >> there are so many settings. I had to take into account fall-back
> > >> scenarios (when one setting was not defined for one profile but
> > >> it
> > >> was a for the other). So, I took a step back and looked at ways
> > >> to
> > >> simplify/improve this.
> > >>
> > >> And here is my proposal:
> > >> 1) Remove both rhq.dev.ds and rhq.test.ds; and remove the flag to
> > >> switch between the two sets.
> > >> 2) Consolidate everything into one set of settings - rhq.ds.
> > >> 3) To use a different database for building/testing/resetting the
> > >> db then just override rhq.ds (eg. -Drhq.ds.db-name=rhq_dev) at
> > >> command line. Or better, create separate profiles in
> > >> .m2/settings.xml and activate each profile as needed for maven
> > >> builds.
> > >>
> > >> Having only one set will:
> > >> 1) make it easy to understand data source settings (only rhq.ds
> > >> to
> > >> configure)
> > >> 2) reduce the code complexity (no fall-back, no flags to switch
> > >> between settings).
> > >> 3) make it easy to override the correct settings for your
> > >> particular database configuration.
> > >> 4) make new unit tests consistent and reduce the possibility to
> > >> make mistakes.
> > >>
> > >> Here is the branch with the changes I made for dbreset:
> > >> http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=shortlog;h=refs/heads/stefan/dbreset
> > >>
> > >> And here is the commit with most of the changes proposed for
> > >> rhq.ds:
> > >> http://git.fedorahosted.org/git/?p=rhq/rhq.git;a=commit;h=187643
> > >>
> > >>
> > >> Any suggestions or input for my proposal? Is there a reason to
> > >> keep
> > >> two sets of tests? Are there any use cases that cannot be
> > >> accomplished by profiles/settings override?
> > >>
> > >>
> > >> Thank you,
> > >> Stefan Negrea
> > >>
> > >> Software Engineer
> > >> _______________________________________________
> > >> rhq-devel mailing list
> > >> rhq-devel at lists.fedorahosted.org
> > >> https://fedorahosted.org/mailman/listinfo/rhq-devel
> > > _______________________________________________
> > > rhq-devel mailing list
> > > rhq-devel at lists.fedorahosted.org
> > > https://fedorahosted.org/mailman/listinfo/rhq-devel
> >
> >
> > --
> > Ian Springer
> > Principal Software Developer
> > JBoss Operations Network
> > Red Hat
> > ian.springer at redhat.com
> >
> > _______________________________________________
> > rhq-devel mailing list
> > rhq-devel at lists.fedorahosted.org
> > https://fedorahosted.org/mailman/listinfo/rhq-devel
> _______________________________________________
> rhq-devel mailing list
> rhq-devel at lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/rhq-devel


More information about the rhq-devel mailing list