[PATCH conductor 0/2] #786844 - Reporting on deleted instances (v2)

Scott Seago sseago at redhat.com
Wed Feb 15 15:19:35 UTC 2012


On 02/15/2012 09:49 AM, Matt Wagner wrote:
> On Wed, Feb 15, 2012 at 07:02:19AM -0500, Javier Pena wrote:
>> If I understood this correctly, this means that in a scenario where we have a local RHEV-M deployment managed by Aeolus, and the RHEV-M server goes nuts for some time, what would happen is:
>>
>> 1. Aeolus would try to identify the state of the instances running on RHEV-M twice, failing both times.
>> 2. After this failure, remove all instances from the DB
>>
>> Is this correct, or am I missing something obvious?
> Yes, unfortunately, that's exactly what would happen. We wait a couple
> of times to try to guard against one-time problems, but if it persists,
> that code will indeed remove all the instances from Conductor.
>
> Is this sort of failure something that happens in the wild? If so, this
> solution is probably not adequate.
>
> -- Matt
It depends on what kind of 'goes nuts' we're talking about. If RHEV 
isn't responding, then we won't remove any instances. However, if RHEV 
appears to be working normally except it's not reporting any instances, 
I don't think we have any choice but to remove them (although maybe a 
"provider-side deleted" state makes more  sense, that way if they 
suddenly re-appear dbomatic can recover. If we simply delete them then 
they're gone forever from conductor.

The difficulty here is that from the conductor/deltacloud point of view, 
this "vanished provider instances" case is exactly what we'd expect to 
see if a user manually went to the RHEV or ec2 console and stopped and 
deleted the instances, and if that's what happened, then deleting is 
exactly what we want to happen.

If we take the "provider-side deleted" state approach (or whatever state 
makes the most sense), then we can provide an easy way for an admin (or 
the instance owner) to clear out these instances once they've confirmed 
that the things are really gone (perhaps via the not-yet-implemented 
alerts mechanism).

Scott




More information about the aeolus-devel mailing list