On 06/07/2012 12:49 PM, Mark McLoughlin wrote:
On Wed, 2012-06-06 at 11:47 -0400, Jason Guiditta wrote:
> We also have in the incubator[1] the new rails engine that acts as a
> reusable client for factory
> [1]
https://github.com/aeolus-incubator/image-management-engine
Am I roughly correct in saying that this is?
- An image registry API which exposes the mapping of an image
identifier to cloud provider specific images, perhaps also tracking
multiple versions/builds of an image
Your "client for factory" description makes it sound like you can submit
a build request through image-management-service, but I don't see that
in the code?
One thing -- the 'engine' name here may be confusing people again. this
is not a separately-running service like 'system engine' or 'cloud
engine' but a "rails engine" -- essentially a plugin. When we use the
image management engine as part of conductor, it's pulled into the
conductor application - we integrate with its model so we get
fully-integrated permissions, internal database assocaitions among
resources, etc.
But, since it's a separate rails engine, some other application or
client can also incorporate the codebase and interact with factory for
non-conductor use cases. However, I don't imagine you'd be able to spawn
off a separate "standalone" image management service to interact with
the _same_ dataset as that managed by a conductor instance, since when
it's pulled into conductor the image-management-engine model will be
extended with conductor-specific things, such as permissions,
connections with conductor environments, etc.
Whatever ... this kind of effort is exactly the way to be moving
forward. Kudos to you and Martyn.
One suggestion I'd make for this would be to concentrate 100% on a nice
API and CLI for it. Make it easy to integrate with it. Make it so you
can go and promote this and image factory together completely
independent of conductor and all the rest.
But, yes, this last part is the goal, as I mentioned above.
Going back to some of the original points about splitting up conductor
into something less monolithic and more consumable by other users, I
think the model here with the image management engine may be a better
one to follow than the idea of trying to shoehorn it all into deltacloud
modules/plugins. Otherwise we run the risk of deltacloud itself being
overcomplicated and obscuring its stated purpose.
I guess my issue is if we're going to have a place where we're plugging
in separate smaller components that work together for something larger,
it seems that the conductor side would be the logical starting point,
since 1) that's where we already have this sort of functionality and 2)
most of the things we're discussing doing here seem to have a lot more
to do with the overall goals of the aeolus project than they are to the
conductor project.
Also, if we take conductor as the starting point, it's actually possible
to get to the end goal by little steps, so we don't have a long period
of breakage/destabilization where we come out at the end with fewer
things than we started with (even if they are better designed).
So conductor includes several features that we'd like to separate into
smaller chunks that can be used independently. In all cases, once we
identify something to separate, we need to decide whether it's being
separated into a completely separate service/process that will (in the
fully-integrated use case) communicate with conductor via API, or
whether it will be more of a plugin which (again in the fully-integrated
use case) will be more of a callable library (possibly even extended)
within the conductor rails application.
Among these features:
1) image authoring and building service (already separated as imagefactory)
2) image management component (originally separated as iwhd, being
rewritten as a plugin (image-management-engine)
3) multi-instance deployment with service configuration, dependency
handling, etc
4) cross-cloud launching (i.e. take a user request to launch something;
figure out where to place it based on resource availability and other
business logic)
(and possibly more)
From the other replies to the thread I've seen various proposals that
essentially amounted to trying to take either 3) or 4) out and make it a
deltacloud plugin. I would suggest we start on the other side -- again
largely to allow incremental progress and to avoid a big de-stabilizing
massive integration which would inevitably result from a ground-up
rewrite. Basically we'd pick 3) or 4) and rewrite the existing
functionality using a rails engine. Create something self-contained that
would work on its own, but would still lend itself to a fully-integrated
conductor application much like we have now -- but with two separate
(smaller) codebases, one of which can be used independently of the
other. If there's something that conductor uses internally now that we
_also_ need as a deltacloud plugin, that could be done too -- but again
I'd say we should _start_ by making the conductor component pluggable,
and then refactor the plugin such that it could be used in both
deltacloud _and_ conductor.
Most important in the above is that we avoid a ground-up rewrite all at
once. We take individual bits that already work in conductor and
refactor them to create a pluggable component and (possibly in some
cases) refactor to the point it can be used by deltacloud as well -- but
if we go the other way (starting with a new deltacloud plugin for
something that wasn't done in deltacloud before), then the integration
effort on the conductor side will be much more difficult.
Scott