Data safety for our Fedora releases... :)

Jason Guiditta jguiditt at redhat.com
Wed Jun 20 15:49:34 UTC 2012


On 20/06/12 11:31 -0400, Scott Seago wrote:
>On 06/20/2012 10:51 AM, Justin Clift wrote:
<snip>
>>As a first thought, this kind of presents two potential approaches:
>>
>>  a) If a new Aeolus version needs a db change, let's not push that
>>     new Aeolus version to the Fedora official repos.
>>
>>     Example, Fedora 16 wouldn't get 0.11.0 (if 0.11.0 requires
>>     db changes - no idea, it's just an example). :)
>>
>>     This doesn't stop us from pushing 0.11.0 to a new Fedora release
>>     which has never had 0.10.2. (or rawhide, who's purpose is testing
>>     new stuff)
>>

Not disagreeing with any of this, just attempting to clarify some
thing we discussed in irc.

>Yeah, that's the part about 'all new sprint releases go straight into 
>fedora' that's always worried me. Well destabilization in general, not 
>just db upgrades. That said, any actual development sprint is almost 
>certainly to include db migrations. I don't think I've seen one yet 
>that doesn't. Even bugfixes may include them, and may include other 
>required "upgrades" that aren't schema changes (i.e. 'rake dc:upgrade' 
>vs 'rake db:migrate'
>

So, I think a makes perfect sense as a solution for _right now_, we
push a stable release to a given fedora, then only bugfix releases
there from that point forward, full stop.  Newer rpms can be had at
the user's own risk, from our own hosted yum repos.  Latest release
always goes into rawhide for fedora.  Later, we move on to....

>>  b) We figure out some way of upgrading the database automatically.
>>
>>     So, Aeolus 0.10.2 in F16 example again.  SysAdmin does "yum update",
>>     maybe gets a new kernel at the same time... system reboots... and
>>     Aeolus 0.11.0 is installed, running fine, and has automatically
>>     updated the db as needed in the background.  (no SysAdmin
>>     intervention required).
>>
>This is where we get in trouble with the current implementation. The 
>simplest way of attempting this would be to have the service 'start' 
>script upgrade if necessary. However, since we currently have two 
>processes (conductor itself and dbomatic), they'd both need to be 
>stopped to run db:migrate, since migrating a database with a live 
>server pointing to it will have unpredictable results, since it 
>implies that either before or after the migrate the code and db were 
>out of sync, potentially corrupting data.
>
>As we discussed on irc, we may need to handle this with puppet somehow 
>-- since the service stop; upgrade; service restart will need to be 
>coordinated across services (potentially running on different boxes -- 
>but that's a future extension).
>

While we may still not want to push such upgrades to fedora proper due
to policy there (as mentioned by eck), we can certainly make it easier
for users and potentially downstream customers by setting up puppet to
manage these updates.  The nutshell of what we discussed it doing:
* notice rpms have been updated (and possibly determine if following
   steps are required)
* stop services for suite (mostly to make sure nobody is using the db
   while upgrades happen)
* run any upgrade/migration scripts needed
* start services back up.

-j

>>There might be other approaches, but the two above stood out as most
>>obvious.  a) sounds like the easiest, but it's probably not the
>>right move.  Several months down the track from now, being stuck with
>>"ancient" versions of Aeolus releases in Fedora is unlikely to work for
>>us.
>>
>>Piwik (for example) can do in place database upgrades, performed by the
>>running app itself.  It's not going to be the only one whose approach
>>we could copy from... ;)
>>
>How does it handle multiple db connections? or does it not have 
>multiple services that talks to the same database?
>>Does anyone have experience with this kind of thing with other db backed
>>web apps?
>>
>>Regards and best wishes,
>>
>>Justin Clift
>>
>>--
>>Aeolus Community Manager
>>http://www.aeolusproject.org
>>
>Scott
>



More information about the aeolus-devel mailing list