Notifications between Aeolus components

Hugh Brock hbrock at redhat.com
Tue Jan 8 16:45:03 UTC 2013


On Tue, Jan 08, 2013 at 02:41:51PM +0100, Jan Provaznik wrote:
> Hi folks,
> I'd like to discuss a topic which was briefly touched on Aeolus
> developer Conference in Brno. Many components of Aeolus project need
> to send notifications to other components:
> Heat -> Conductor (deployment/instance state changes)
> Conductor -> Winged Monkey (deployment/instance state changes)
> Imagefactory -> Conductor (notifications about image build&upload state)
> DC tracker* -> Heat (notifications about instance state changes)
> DC tracker -> Conductor (notifications about other provider resource
> changes, realms availability, hw profile changes)
> 
> * DC Tracker - this component is actually not agreed yet, it was
> just a proposal some time ago, but I believe this will be needed.
> 
> As far as I know there is no a unified plan how to deal with
> notifications between components. At this point notifications are
> implemented only in Imagefactory. This implementation is for now
> quite simple - a notification callback is sent back as http PUT
> request. It doesn't cover any failure situations (network error,
> auth error, Conductor is not running...), so if a request is not
> successful, no retry is done.
> 
> I think we need more robust notification system between all
> components above, this system should support at least:
> 1) retry on failure
> 2) keep correct order of notifications
> 3) support authentication
> 
> And here it comes...
> 
> Why not use a message bus (an AMQP implementation) for communication
> between all involved components?
> - it supports all required features out of the box
> - clients exists for all languages involved in Aeolus project
> - notifications will be solved in the same way between all components
> 
> Or is there some other solution how to solve notifications as
> painless as possible while keeping required robustness? What is your
> preferred solution of this problem?

I have hated on message bus solutions publicly in the past, so it seems
appropriate for me to weigh in now :).

What you're suggesting appears to make good sense. However, based on
past (bitter, painful, very expensive) experience, I would like us to
approach the message bus question very carefully, according to a few
principles: 

* Any message bus, regardless how robust or stable, introduces
  complexity and dependencies to our code. Any proposal to add message
  bus use to one of our components should include some consideration of
  the costs and benefits. In other words, I want to see a solid
  justification of why REST callbacks are inadequate for a particular
  API connection before we dive into AMQP.

* The message bus we choose should be one that other upstream cloud
  projects commonly use. (What is OpenStack using, for example? Is oVirt
  using anything?). It should also be available across our target
  developer and end user platforms.

* The message bus we choose must support all the encryption and
  authentication mechanisms that the app supports. This means LDAP,
  oAuth, and (eventually) kerberos.

* Unless it is incredibly expensive to build it this way, I'd like the
  message bus to be optional wherever possible -- meaning, fall back to
  a simple listener/callback over REST architecture whenever possible.

Darts welcome :)...

--Hugh


-- 
== Hugh Brock, hbrock at redhat.com                                   ==
== Engineering Manager, Cloud BU                                   ==
== Aeolus Project: Manage virtual infrastructure across clouds.    ==
== http://aeolusproject.org                                        ==

"I know that you believe you understand what you think I said, but I’m
not sure you realize that what you heard is not what I meant."
--Robert McCloskey


More information about the aeolus-devel mailing list