On Mon, 2009-09-21 at 13:14 -0700, Ian Main wrote:
So I've been looking over some things and doing a lot of
pondering on
how to move forward with taskomatic for deltacloud.
The plan I'm working with now is to:
- Finish the infrastructure for dependency information between tasks
and implement threading. I think it would be wise to just do this
up front so that as we implement tasks we can figure out the
dependencies and implement them. The task dependencies will work as
David and I worked out before for ovirt, using a class for each task
type, an instance for each action will be added to a list which can
be scanned by each instance to determine if it can run. I actually
don't think this will add too much difficulty given the limited number
of tasks in dcloud.
Sounds good - the devil is in the details with this, but that's a good
start. And deltacloud desperately needs queue parallelization, since we
can't have the queue stall just because we lost connection to EC2 (but
still have connections to RHEV-M)
- Implement a seperate thread that deals with updates from the
service
providers. This will basically be the "dbomatic" for dcloud. The
poll interval will be based on an XML config value in each driver and
will likely be fairly longish (I'm thinking 60sec or so depending).
- Plumb in a system that allows quicker updates to objects on which
recent actions have taken place. For example when an instance is
started we want to have more frequent polling of that instance until
its state changes so that we can more quickly inform the user. I am
unsure whether I will just use a seperate thread (or even be part of
the task) to do the updates or somehow notify the main update
thread. I suspect another thread will be easier..
How about just sticking a 'frequency' and a 'next_poll' property into
each task, so that your dbomatic thread polls for status updates for all
tasks whose 'next_poll' is in the past and sets next_poll to next_poll +
frequency after each successful poll ? State changes (or any change) to
an action can adjust frequency. That way, you don't need to distinguish
between slow and fast polling tasks.
- I'm not going to worry about torqeubox for now. If we do end
up
using it it may be useful to use the queuing system as a method to
notify taskomatic of new tasks.
Absolutely agreed. Getting something working with whatever is familiar
is the top priority.
David