RFC: Integrating Aeolus with Heat

Ian Main imain at redhat.com
Mon Sep 10 18:15:19 UTC 2012


On Fri, Sep 07, 2012 at 02:35:39PM +0200, Jan Provaznik wrote:
> On 09/04/2012 11:25 PM, Ian Main wrote:
> >On Mon, Sep 03, 2012 at 12:44:05PM +0200, Jan Provazník wrote:
> >>On 08/30/2012 10:23 PM, Ian Main wrote:
> >>>On Thu, Aug 30, 2012 at 09:21:37AM +0200, Jan Provaznik wrote:
> >>>>On 08/29/2012 09:55 PM, Ian Main wrote:
> >>>>>On Mon, Aug 27, 2012 at 02:47:42PM +0200, Jan Provaznik wrote:
> >>>>>>On 08/21/2012 06:15 PM, Tomas Sedovic wrote:

[snip]

> >I think we would have to try it and see.  Ultimately it all comes from a
> >database over a network connection.  Querying heat only adds one more
> >http request/socket layer.  Perhaps some time munging data but all in
> >all it's fairly light weight stuff.
> >
> 
> Hi, sorry for late response, there was a deadline in last days...
> 
> Honestly I don't think that it would be light weight stuff, Heat
> will have sooner or later some authentication mechanism, also it
> will do another query (multiple queries) into its own DB backend.

As would Aeolus.. what I'm saying is the data and querying is similar in
both cases, there's just 1 extra http connection between heat and
aeolus.  The trick would be getting the queries fine grained enough in
heat to allow aeolus to make single requests to get the info it needs.

> >I think it's important to really consider what is involved in duplicating
> 
> As I mentioned in the first mail in this thread Conductor needs
> primarily information about instance changes, from other resources
> it checks also realms availability (maybe we might need check hw
> profile changes in future).
> 
> >that data.  It has been difficult in the past eg with dbomatic
> >because it is a very hard problem.  Even with events this doesn't
> >change.  What happens when a service goes down for a time?  You need to
> >replay events, read logs etc.. people spend a lot of time and a lot of
> 
> Well, for callback-related issues were discussed in a thread about
> DC tracker and it wouldn't be so hard (it would be combination of
> retry+replay).

This is what dbomatic had when it was using condor FYI.

> What about this:
> instead of callbacks we could use a messaging queue (an AMQP
> implementation, e.g. RabbitMQ). Integrate this into Heat and
> Conductor would be matter of few seconds (there are clients for most
> of programming languages). This would be quite robust solution.

That would probably be a better idea.  You will still need replay and
I'm not sure you can ever say you are getting 100% of the events, but I
think that is true with any solution.

> The same messaging might be used also for sending notifications
> between Conductor and Imagefactory.
> 
> >money trying to solve data replication.  IMO if you can access the
> >primary data source for your implementation that will be a far more
> 
> Don't forget that Heat is *not* primary data source (only cloud
> provider is), And BTW, Heat will have to do the same "data
> mirroring" (if you mean instance state checking by this term) itself
> and if it's going to use dc-api you will probably end up with
> polling providers.

Yes very true, but I what I am saying is why do it more than needed?

>From what I've seen the real issue is actually ActiveRecord.  The data
model is so tightly tied into the logic and application that using some
other data source becomes very difficult.  I did some searching and I
think what I would propose is using something like the Repository
Pattern:

http://msdn.microsoft.com/en-us/library/ff649690.aspx

(Sorry it's an msdn page but it's the best description I could find.)

The idea would be to create a layer to allow for storage of data in
multiple places, including another web service.  Perhaps this would be
useful for the "pluggable backend" idea?

Just a thought anyway, I'm probably in over my head now.. I'm no rails
pro :).


    Ian




More information about the aeolus-devel mailing list