On 06/07/2012 02:29 PM, Scott Seago wrote:
On 06/07/2012 12:49 PM, Mark McLoughlin wrote:
> On Wed, 2012-06-06 at 11:47 -0400, Jason Guiditta wrote:
>> We also have in the incubator[1] the new rails engine that acts as a
>> reusable client for factory
>> [1]
https://github.com/aeolus-incubator/image-management-engine
> Am I roughly correct in saying that this is?
>
> - An image registry API which exposes the mapping of an image
> identifier to cloud provider specific images, perhaps also tracking
> multiple versions/builds of an image
>
> Your "client for factory" description makes it sound like you can submit
> a build request through image-management-service, but I don't see that
> in the code?
>
One thing -- the 'engine' name here may be confusing people again.
this is not a separately-running service like 'system engine' or
'cloud engine' but a "rails engine" -- essentially a plugin. When we
use the image management engine as part of conductor, it's pulled into
the conductor application - we integrate with its model so we get
fully-integrated permissions, internal database assocaitions among
resources, etc.
But, since it's a separate rails engine, some other application or
client can also incorporate the codebase and interact with factory for
non-conductor use cases. However, I don't imagine you'd be able to
spawn off a separate "standalone" image management service to interact
with the _same_ dataset as that managed by a conductor instance, since
when it's pulled into conductor the image-management-engine model will
be extended with conductor-specific things, such as permissions,
connections with conductor environments, etc.
> Whatever ... this kind of effort is exactly the way to be moving
> forward. Kudos to you and Martyn.
>
> One suggestion I'd make for this would be to concentrate 100% on a nice
> API and CLI for it. Make it easy to integrate with it. Make it so you
> can go and promote this and image factory together completely
> independent of conductor and all the rest.
>
But, yes, this last part is the goal, as I mentioned above.
Going back to some of the original points about splitting up conductor
into something less monolithic and more consumable by other users, I
think the model here with the image management engine may be a better
one to follow than the idea of trying to shoehorn it all into
deltacloud modules/plugins. Otherwise we run the risk of deltacloud
itself being overcomplicated and obscuring its stated purpose.
I guess my issue is if we're going to have a place where we're
plugging in separate smaller components that work together for
something larger, it seems that the conductor side would be the
logical starting point, since 1) that's where we already have this
sort of functionality and 2) most of the things we're discussing doing
here seem to have a lot more to do with the overall goals of the
aeolus project than they are to the conductor project.
s/conductor/deltacloud/ here ^^^^^^^^^
Also, if we take conductor as the starting point, it's actually
possible to get to the end goal by little steps, so we don't have a
long period of breakage/destabilization where we come out at the end
with fewer things than we started with (even if they are better
designed).
So conductor includes several features that we'd like to separate into
smaller chunks that can be used independently. In all cases, once we
identify something to separate, we need to decide whether it's being
separated into a completely separate service/process that will (in the
fully-integrated use case) communicate with conductor via API, or
whether it will be more of a plugin which (again in the
fully-integrated use case) will be more of a callable library
(possibly even extended) within the conductor rails application.
Among these features:
1) image authoring and building service (already separated as
imagefactory)
2) image management component (originally separated as iwhd, being
rewritten as a plugin (image-management-engine)
3) multi-instance deployment with service configuration, dependency
handling, etc
4) cross-cloud launching (i.e. take a user request to launch
something; figure out where to place it based on resource availability
and other business logic)
(and possibly more)
From the other replies to the thread I've seen various proposals that
essentially amounted to trying to take either 3) or 4) out and make it
a deltacloud plugin. I would suggest we start on the other side --
again largely to allow incremental progress and to avoid a big
de-stabilizing massive integration which would inevitably result from
a ground-up rewrite. Basically we'd pick 3) or 4) and rewrite the
existing functionality using a rails engine. Create something
self-contained that would work on its own, but would still lend itself
to a fully-integrated conductor application much like we have now --
but with two separate (smaller) codebases, one of which can be used
independently of the other. If there's something that conductor uses
internally now that we _also_ need as a deltacloud plugin, that could
be done too -- but again I'd say we should _start_ by making the
conductor component pluggable, and then refactor the plugin such that
it could be used in both deltacloud _and_ conductor.
Most important in the above is that we avoid a ground-up rewrite all
at once. We take individual bits that already work in conductor and
refactor them to create a pluggable component and (possibly in some
cases) refactor to the point it can be used by deltacloud as well --
but if we go the other way (starting with a new deltacloud plugin for
something that wasn't done in deltacloud before), then the integration
effort on the conductor side will be much more difficult.
Scott