Data safety for our Fedora releases... :)

Steve Linabery slinaber at redhat.com
Wed Jun 20 15:54:09 UTC 2012


On Wed, Jun 20, 2012 at 11:49:34AM -0400, Jason Guiditta wrote:
> On 20/06/12 11:31 -0400, Scott Seago wrote:
> >On 06/20/2012 10:51 AM, Justin Clift wrote:
> <snip>
> >>As a first thought, this kind of presents two potential approaches:
> >>
> >> a) If a new Aeolus version needs a db change, let's not push that
> >>    new Aeolus version to the Fedora official repos.
> >>
> >>    Example, Fedora 16 wouldn't get 0.11.0 (if 0.11.0 requires
> >>    db changes - no idea, it's just an example). :)
> >>
> >>    This doesn't stop us from pushing 0.11.0 to a new Fedora release
> >>    which has never had 0.10.2. (or rawhide, who's purpose is testing
> >>    new stuff)
> >>
> 
> Not disagreeing with any of this, just attempting to clarify some
> thing we discussed in irc.
> 
> >Yeah, that's the part about 'all new sprint releases go straight
> >into fedora' that's always worried me. Well destabilization in
> >general, not just db upgrades. That said, any actual development
> >sprint is almost certainly to include db migrations. I don't think
> >I've seen one yet that doesn't. Even bugfixes may include them,
> >and may include other required "upgrades" that aren't schema
> >changes (i.e. 'rake dc:upgrade' vs 'rake db:migrate'
> >
> 
> So, I think a makes perfect sense as a solution for _right now_, we
> push a stable release to a given fedora, then only bugfix releases
> there from that point forward, full stop.  Newer rpms can be had at
> the user's own risk, from our own hosted yum repos.  Latest release
> always goes into rawhide for fedora.  Later, we move on to....
> 
> >> b) We figure out some way of upgrading the database automatically.
> >>
> >>    So, Aeolus 0.10.2 in F16 example again.  SysAdmin does "yum update",
> >>    maybe gets a new kernel at the same time... system reboots... and
> >>    Aeolus 0.11.0 is installed, running fine, and has automatically
> >>    updated the db as needed in the background.  (no SysAdmin
> >>    intervention required).
> >>
> >This is where we get in trouble with the current implementation.
> >The simplest way of attempting this would be to have the service
> >'start' script upgrade if necessary. However, since we currently
> >have two processes (conductor itself and dbomatic), they'd both
> >need to be stopped to run db:migrate, since migrating a database
> >with a live server pointing to it will have unpredictable results,
> >since it implies that either before or after the migrate the code
> >and db were out of sync, potentially corrupting data.
> >
> >As we discussed on irc, we may need to handle this with puppet
> >somehow -- since the service stop; upgrade; service restart will
> >need to be coordinated across services (potentially running on
> >different boxes -- but that's a future extension).
> >
> 
> While we may still not want to push such upgrades to fedora proper due
> to policy there (as mentioned by eck), we can certainly make it easier
> for users and potentially downstream customers by setting up puppet to
> manage these updates.  The nutshell of what we discussed it doing:
> * notice rpms have been updated (and possibly determine if following
>   steps are required)
> * stop services for suite (mostly to make sure nobody is using the db
>   while upgrades happen)
> * run any upgrade/migration scripts needed
> * start services back up.
> 
> -j
> 
> >>There might be other approaches, but the two above stood out as most
> >>obvious.  a) sounds like the easiest, but it's probably not the
> >>right move.  Several months down the track from now, being stuck with
> >>"ancient" versions of Aeolus releases in Fedora is unlikely to work for
> >>us.
> >>
> >>Piwik (for example) can do in place database upgrades, performed by the
> >>running app itself.  It's not going to be the only one whose approach
> >>we could copy from... ;)
> >>
> >How does it handle multiple db connections? or does it not have
> >multiple services that talks to the same database?
> >>Does anyone have experience with this kind of thing with other db backed
> >>web apps?
> >>
> >>Regards and best wishes,
> >>
> >>Justin Clift
> >>
> >>--
> >>Aeolus Community Manager
> >>http://www.aeolusproject.org
> >>
> >Scott
> >

ACK



More information about the aeolus-devel mailing list