On 01/09/2013 07:57 AM, Mo Morsi wrote:
On 01/08/2013 11:45 AM, Hugh Brock wrote:
> On Tue, Jan 08, 2013 at 02:41:51PM +0100, Jan Provaznik wrote:
>> Hi folks,
>> I'd like to discuss a topic which was briefly touched on Aeolus
>> developer Conference in Brno. Many components of Aeolus project need
>> to send notifications to other components:
>> Heat -> Conductor (deployment/instance state changes)
>> Conductor -> Winged Monkey (deployment/instance state changes)
>> Imagefactory -> Conductor (notifications about image build&upload state)
>> DC tracker* -> Heat (notifications about instance state changes)
>> DC tracker -> Conductor (notifications about other provider resource
>> changes, realms availability, hw profile changes)
>>
>> * DC Tracker - this component is actually not agreed yet, it was
>> just a proposal some time ago, but I believe this will be needed.
>>
>> As far as I know there is no a unified plan how to deal with
>> notifications between components. At this point notifications are
>> implemented only in Imagefactory. This implementation is for now
>> quite simple - a notification callback is sent back as http PUT
>> request. It doesn't cover any failure situations (network error,
>> auth error, Conductor is not running...), so if a request is not
>> successful, no retry is done.
>>
>> I think we need more robust notification system between all
>> components above, this system should support at least:
>> 1) retry on failure
>> 2) keep correct order of notifications
>> 3) support authentication
>>
>> And here it comes...
>>
>> Why not use a message bus (an AMQP implementation) for communication
>> between all involved components?
>> - it supports all required features out of the box
>> - clients exists for all languages involved in Aeolus project
>> - notifications will be solved in the same way between all components
>>
>> Or is there some other solution how to solve notifications as
>> painless as possible while keeping required robustness? What is your
>> preferred solution of this problem?
> I have hated on message bus solutions publicly in the past, so it seems
> appropriate for me to weigh in now :).
>
> What you're suggesting appears to make good sense. However, based on
> past (bitter, painful, very expensive) experience, I would like us to
> approach the message bus question very carefully, according to a few
> principles:
>
> * Any message bus, regardless how robust or stable, introduces
> complexity and dependencies to our code. Any proposal to add message
> bus use to one of our components should include some consideration of
> the costs and benefits. In other words, I want to see a solid
> justification of why REST callbacks are inadequate for a particular
> API connection before we dive into AMQP.
+1. We tried AMQP / QMF before and it ended up being a pain. Introduced
another service / point of failure.
Just thinking out loud... QMF is a pain but is
dead now I think. if you
do callbacks you have to buy off on alot of networking. Can you put
image factory in EC2 and run Aeolous in house?
-- bk