On 09/04/2012 05:25 PM, Ian Main wrote:
On Mon, Sep 03, 2012 at 12:44:05PM +0200, Jan Provazník wrote:
> On 08/30/2012 10:23 PM, Ian Main wrote:
>> On Thu, Aug 30, 2012 at 09:21:37AM +0200, Jan Provaznik wrote:
>>> On 08/29/2012 09:55 PM, Ian Main wrote:
>>>> On Mon, Aug 27, 2012 at 02:47:42PM +0200, Jan Provaznik wrote:
>>>>> On 08/21/2012 06:15 PM, Tomas Sedovic wrote:
>>>>>> Hey Folks,
>>>>>>
>>>> [snip]
>>>>
>>>>>> ### Querying Heat data from Conductor ###
>>>>>>
>>>>>> Heat doesn't support any callbacks. When Conductor wants to
know details
>>>>>> about the stack it launched, it will use the CloudFormation API
to query
>>>>>> the data.
>>>>>>
>>>>>> For the proof of concept stage, we will just issue the query to
Heat
>>>>>> upon every relevant UI action: e.g. `ListStacks` when showing
>>>>>> deployables in the UI, `DescribeStackResource` when shoving a
details of
>>>>>> a single deployable, `DescribeStackEvents` to get deployable
events, etc.
>>>>>>
>>>>> This is OK for POC, but it would be really nice to have callback
>>>>> support for real integration.
>>>>>
>>>>> nit: you probably meant 'deployment' instead of
'deployable' in the
>>>>> paragraph above.
>>>> I am curious as to why you think it is necessary to use callbacks and
>>>> mirror the data held in heat within aeolus?
>>>>
>>>> Ian
>>>>
>>> Conductor needs to know if/when a deployment or single instance
>>> changed its state (is this what you mean by mirroring data?). W/o
>>> notification support on Heat side, Conductor would have to poll Heat
>>> which is painful (dbomatic-like service presence on conductor side)
>>> and not very effective.
>> I agree dbomatic type service is error prone. However mirroring data
> >from one service to another is a very difficult problem to solve well
>> and have it be reliable.
>>
>> Is this required for some sort of reporting? If it is just for the
> Yes, reporting and keeping history logs about instances is part of
> Conductor. Conductor also uses this information when choosing a
> provider when launching an instance and also for quota checking.
This could be done either way, but really you are just needing a tally
of instances per user and per cloud. I'm not saying it is ideal but I
wouldn't say it's impossible or even unwise to consider direct querying
even here.
One difficulty is that if we're talking about making heat optional,
heat
and any other launching infrastructure (including, perhaps, the
current/legacy one if that remains) will need to handle quota/instance
state and queries/etc in the same way. Currently instance
metadata/state/quota checking is tracked in conductor itself. As long as
heat is optional, I'm not sure how we would change that. If heat became
a complete replacement, we _could_ query all of this stuff live (rather
than caching), but there are a lot of moving parts in the existing
infrastructure that would need to be rewritten. After the pain of
handling image metadata as a separate server (with data only available
with live calls outside), we're in the process of moving that back into
conductor. We need to be careful about deciding to do the opposite with
instances and deployments. I'm not saying we can't/shouldn't do that,
but we'd better make sure we've got answers to the various pain points
-- performance, searching, object associations, permissions, etc. For
permissions, in particular, even if all instance/deployment metadata
were in heat (and only queried live from Conductor), we'd at least need
placeholder objects on our side, so that we don't lose the ability to
manage permissions on a per-object basis.
>> user to view the states then that can be done on an as-needed
basis by
>> contacting heat. Even reporting is part of the AWS cloudformations API
>> and events for stacks are supposed to be kept around for something like
>> 90 days (iirc).
>>
>> Personally I very much question the need to mirror the data heat retains
>> into Aeolus as these kinds of things are very error prone and difficult.
>> Unless there is some kind of special need for reporting etc. the data
>> could just as easily be queried directly.
>>
>> Ian
>>
> Besides the needs listed above, I'm afraid there might be
> performance issues with querying Heat directly per each user
> request.
I think we would have to try it and see. Ultimately it all comes from a
database over a network connection. Querying heat only adds one more
http request/socket layer. Perhaps some time munging data but all in
all it's fairly light weight stuff.
I think it's important to really consider what is involved in duplicating
that data. It has been difficult in the past eg with dbomatic
because it is a very hard problem. Even with events this doesn't
change. What happens when a service goes down for a time? You need to
replay events, read logs etc.. people spend a lot of time and a lot of
money trying to solve data replication. IMO if you can access the
primary data source for your implementation that will be a far more
robust solution. Yes, this may mean having to sort out performance
issues, but I think that is ultimately an easier problem.
To summarize the more
rambling comments above, the issues we'd need to
resolve if we decide not to store instance/deployment data in conductor
anymore are:
1) performance concerns (what level of caching is appropriate? what's
the performance of the Heat API?, etc.)
2) data association concerns. This relates, in part, to performance --
things that are now retrievable with a single query may now require a
separate HEAT api call for each row returned, what about search, etc.
For things like the permissions system, which is based on the ability to
apply the generic permissions checking and filtering queries to various
object types, it becomes an issue not so much related to performance but
related to maintaining a list of "heat-owned" objects in Conductor. This
wouldn't be a full data cache but just a list of objects by ID (and
perhaps name) that would need to exist in both places -- i.e. we
wouldn't need to sync on update -- just on creation/deletion.
3) handling the "is heat optional?" question and the implications on the
model.
Ian
Scott