RFC: Integrating Aeolus with Heat

Mo Morsi mmorsi at redhat.com
Mon Aug 27 16:16:12 UTC 2012


On 08/21/2012 12:15 PM, Tomas Sedovic wrote:
> Hey Folks,
>
> I've been contributing to the Heat project lately and I think it's
> gotten to a point where we (the Aeolus community) could look at it and
> figure out if it's something we want to use.
>
> Please forgive me the length of this email. I tried to make it as
> short but there's a lot to cover. Grab your favourite beverage and sit
> comfortably.
>
>
> A little background
> -------------------
>
> Heat is a project that provides orchestration and high availability to
> OpenStack. Its API is modelled after Amazon CloudFormation.
>
> What it does in Aeolus' terms is: you create a deployable and pass it
> to Heat API along with any launch-time parameters. Heat will decide on
> the correct order of the resources (instances, databases, load
> balancers, remote storage, floating IP addresses, etc.) and launches
> and configures the whole thing to produce a running deployment.
>
> Heat started about five months ago and hopes to become one of the core
> OpenStack components. It's written in Python following the OpenStack
> development model and tools.

Is Heat implemented as a centralized python service? How does the user
currently invoke that service and how does the service in return managed
the openstack components?

Is there a hard dependency of openstack in heat? Will that be removed /
made optional when the deltacloud backend is in place?


>
>
> What's this to do with Aeolus?
> ------------------------------
>
> We're trying to solve some of the same problems:
>
> * launching multiple instances in correct order
> * passing parameters that are only known when an instance is launched
> * configuring and advertising the services running inside the VMs

Does Heat require anything to be installed on the instances themselves?



>
> If we could outsource this to a project whose sole focus is solving
> these issues, we could focus on things that make Aeolus unique:
> building great UI, tools and APIs for managing cross-cloud deployments.
>
> As a side benefit, we would get the current and future features of
> Heat for free:
>
> * high availability
> * auto-scaling
> * load-balancing
>

<snip>

>
>
> ### Template format ###
>
> Aeolus uses a different format from Heat to describe deployables.
>
> We need to settle on what's the best thing to use. Options:
>
> 1) Translate Aeolus Deployable XML to Amazon CloudFormation JSON
>
> 2) Implement Aeolus Deployable XML format in Heat
>
> 3) Adopt CloudFormation JSON in Conductor
>
> 4) Have both Aeouls and Heat adopt another (open standard) format
>
> With all likelihood Heat will reject 2), we won't switch to 3) and
> since *both* communities would have to accept and implement 4), we'll
> probably stick to 1) for some time.
>
> Still, listing it here as it needs to be decided.

Curious as to why you went w/ the CloudFormation JSON as the deployable
descriptor format? Is this what openstack uses?

Supporting multiple deployable formats would be a nice feature in either
application but obviously puts a burden on the developers / testers /
etc. I'm not 100% familiar w/ the intricacies of the formats, but if
they are simple enough and map well enough to each other, a barebones
utility to do the conversion may be optimal for the time being.



>
>
> ### Heat-Conductor communication ###
>
> Heat is working on implementing a proper RESTful API that will
>
> 1) be more consistent with other OpenStack components
> 2) suck less
>
> Once that's done, Conductor should probably adopt this API.
>
> We then need to figure out how to pass data from Heat back to
> Conductor. If Conductor's launching instances via Heat, how does that
> information get updated?
>
> 1) Conductor asks Heat every time it needs the data
>
> 2) Conductor asks Heat periodically and caches the results
>
> 3) Heat allows Conductor to register a callback -- either via HTTP
> (webhooks) or via RPC -- and tells Conductor when stuff happens (e.g.
> an instance was launched, crashed, etc.)
>
> 4) Conductor continues to use DBomatic to query the instances directly
> via Deltacloud, ignoring any additional information Heat may provide
>
> For the proof of concept, everything other than 1) is a premature
> optimization and then we'll see.
>
> I think 4) would be rather brittle especially since adopting Heat
> might possibly let us drop DBomatic entirely.

Some of this reminds me of condor, we were using that for a while in
sort of an orchestration manner. It was responsible for tracking the
lifetimes of instances on different cloud providers and dbomatic was
initially written to track events through condor logs. It seems the
level of abstraction is a bit different as Heat represents resources
using the openstack model.



>
>
> ### Is Heat going to be the only way to launch deployments in
> Conductor or will it be an optional part? ###
>
> We will need to see how stable and usable Heat is. How easy it is to
> deploy and keep up (it would be another dependency, after all) and how
> much benefit and code cleanup it brings to Aeolus.


I'd be in favour of keeping this optional for the time being. It sounds
like Heat would be a useful add on, but I'd rather keep the required
dependency set as small as possible. Even external components that are
currently in place should be made modular / pluggable: image management,
config servers, system engines, etc.

   -Mo




More information about the aeolus-devel mailing list