On 01/08/2013 11:45 AM, Hugh Brock wrote:
On Tue, Jan 08, 2013 at 02:41:51PM +0100, Jan Provaznik wrote:
> Hi folks,
> I'd like to discuss a topic which was briefly touched on Aeolus
> developer Conference in Brno. Many components of Aeolus project need
> to send notifications to other components:
> Heat -> Conductor (deployment/instance state changes)
> Conductor -> Winged Monkey (deployment/instance state changes)
> Imagefactory -> Conductor (notifications about image build&upload state)
> DC tracker* -> Heat (notifications about instance state changes)
> DC tracker -> Conductor (notifications about other provider resource
> changes, realms availability, hw profile changes)
>
> * DC Tracker - this component is actually not agreed yet, it was
> just a proposal some time ago, but I believe this will be needed.
>
> As far as I know there is no a unified plan how to deal with
> notifications between components. At this point notifications are
> implemented only in Imagefactory. This implementation is for now
> quite simple - a notification callback is sent back as http PUT
> request. It doesn't cover any failure situations (network error,
> auth error, Conductor is not running...), so if a request is not
> successful, no retry is done.
>
> I think we need more robust notification system between all
> components above, this system should support at least:
> 1) retry on failure
> 2) keep correct order of notifications
> 3) support authentication
>
> And here it comes...
>
> Why not use a message bus (an AMQP implementation) for communication
> between all involved components?
> - it supports all required features out of the box
> - clients exists for all languages involved in Aeolus project
> - notifications will be solved in the same way between all components
>
> Or is there some other solution how to solve notifications as
> painless as possible while keeping required robustness? What is your
> preferred solution of this problem?
I have hated on message bus solutions publicly in the past, so it seems
appropriate for me to weigh in now :).
What you're suggesting appears to make good sense. However, based on
past (bitter, painful, very expensive) experience, I would like us to
approach the message bus question very carefully, according to a few
principles:
* Any message bus, regardless how robust or stable, introduces
complexity and dependencies to our code. Any proposal to add message
bus use to one of our components should include some consideration of
the costs and benefits. In other words, I want to see a solid
justification of why REST callbacks are inadequate for a particular
API connection before we dive into AMQP.
+1. We tried AMQP / QMF before and it ended up being a pain. Introduced
another service / point of failure.
In general I'd prefer to see more decoupling between components so they
can be used independently, rather than trying to come up w/ efficient
ways to tie things together.
We need to ask ourselves, "what will grow the community / userbase" as
opposed to "what is needed for the technical aspects of the project".
* The message bus we choose should be one that other upstream cloud
projects commonly use. (What is OpenStack using, for example? Is oVirt
using anything?). It should also be available across our target
developer and end user platforms.
* The message bus we choose must support all the encryption and
authentication mechanisms that the app supports. This means LDAP,
oAuth, and (eventually) kerberos.
As a side note, I've been becoming a big fan of JSON-RPC [1] and wrote
the Ruby implementation as a side project here [2].
It's simple, supports any transport that you could desire (HTTP, AMQP,
TCP, SSL, Websockets, and more), can use any authentication scheme that
you desire on the backend and more.
Even gave a presentation / demo of it at the Brno-Ruby group when I was
in Brno last spring [3].
-Mo
[1]
http://en.wikipedia.org/wiki/JSON-RPC
[2]
https://github.com/movitto/rjr
[3]
http://mo.morsi.org/blog/node/365