On 12/01/2011 01:41 PM, Tomas Hrcka wrote:
> Hi all,
>
> the time has come, and we need to somehow implement background jobs. I
> am curently working on task which involve long run operations on
> provider side(ie. stop and delete). So I can make the UI not responding
> for 2-4 mins, but that is not what we want.
>
Hi,
it's only "destroy" operation which takes long time. This operation can
be triggered on changing instance state to "stop". "stop" state is
set
by dbomatic which is running on background, so "destroy" operation
should always run on background too -> no slow UI response in this case.
As I said before, we (will) need background job on conductor side anyway
because of mass operations on instances, example: if a user selects
multiple instances running on various providers, one of providers
doesn't respond because of firewall/connection error, the conductor user
will wait for a response forever or will get connection timeout error
w/o knowing on which instance a command was executed. Another use case
is "build+push" feature which we had to postpone - having background job
solves this too.
> Question is what will we use for performing background jobs. On
>
ruby-toolbox.com [1] is overview of ruby projects for background jobs.
> Or we can implement our own solution.
>
> [1] -
https://www.ruby-toolbox.com/categories/Background_Jobs
>
>
Yes, there are many nice tools we can use. As it was already mentioned
in this thread we have been using DelayedJob in conductor before and
from what I remember it worked fine, but not saying we must use this again.
Another option is to extend dbomatic to do this job, but I don't think
it's best solution.
Since this background job integration probably won't be part of 1.0
version we have enough time to think about best solution.
Jan
Jan,
Since we are speaking post 1.0, an alternative to this problem of
deployable stopping without blocking the conductor UI is to use the API
that Tomas and Padraig implemented in [1].
In essence, the general idea would be to drive start/stop from a
separate process (pacemaker cloud's cpe process) via the API described
by Tomas. Pacemaker cloud would take care of calling the deltacloud
APIs to start and stop instances and provide feedback of what happened
to conductor. Conductor would tell CPE to start/stop instances and
provide feedback given by DPE to the administrator removing policy
execution from conductor and more tightly focusing conductor as an OAM UI.
This offers several advantages:
* Multiple assembly deployable support with full HA
+ Rigorous active fault detection of resources (applications)
+ Rigorous active fault detection of assemblies
+ multiple assemblies can have start ordering per user policy
+ When a failure is detected, children assemblies can be properly
recovered by restart or reconfiguratoin
+ A recovery escalation policy can be enforced on fault detection
* Clean separation of OAM policy generation (occurs in conductor) and
OAM policy execution (occurs in CPE). This separation removes any
blocking that occurs in conductor with the current implementation.
The disadvantage is the rework involved however most of the building
blocks are in place in the code base limiting the rework.
[1]