Winged Monkey

Hugh Brock hbrock at redhat.com
Thu Oct 25 20:32:35 UTC 2012


On Thu, Oct 25, 2012 at 07:30:55PM +0200, Jaromír Coufal wrote:
> Hello folks,
> 
> I was also feeling and saying (as Matt wrote few mails ago), that
> with Winked Monkey, we are going to duplicate part of Conductor and
> trying to avoid its difficult parts.
> =
> My idea about how we can deal with this situation in Conductor is
> based on user privileges and hiding sections, which user cannot
> access. Let me explain on simple example:
> 
> * Situation (based on Winked Monkey purpose):
> - We have a user who doesn't (or shouldn't) care about creating
> images, pushing to providers, setting environments or pools, (...).
> - User wants just to start and stop images, which he has on
> different provider accounts.

[many worthy ideas snipped]

> 
> In Winked Monkey, as I found out, there is not even counted with
> these necessary steps (talking about steps 1 and 2). Adding these
> features, we are slowly going to lighter version of Conductor.
> =
> What I really like about Winked Monkey is, that we started to think
> about UI from different point of view and started to care just about
> users who are running images. Which definitely can inspire and help
> us to improve UX.
> 
> Another value I see in Winked Monkey project is, that we will
> approach broader audience and community of users - those having just
> few images across different providers.
> But my question is, if this segment of users is big enough? Because
> from my point of view, being individual (or small company) running
> just few instances, I am usually choosing one provider and running
> all instances there - in this case I would not search for
> application unifying UI for running instances.
> (maybe I am wrong on this one)
> =
> Having talks about this particular type of users and creating
> muck-ups for their use-cases is definitely valuable for us. I feel,
> and I am sorry to express my worries, that having stand-alone
> application as Winked Monkey is (especially implementation and
> maintenance) would consume lot of effort and resources. I think,
> that we can achieve similar result with improving Conductor for less
> effort.

Thanks so much for spending so much time thinking about this
Jarda. I think your analysis of the details is accurate, but I'm going
to dispute the conclusion you come to by taking a big step backward to
look at the whole picture of Aeolus.

(This is going to be a major topic of conversation in our meetings in
Brno week after next I hope.)

How will we know Aeolus is successful as a project? What are its goals?

I believe the goal of Aeolus has always been to make *all* cross-cloud
operations as easy as using a single cloud, to the extent that it is
possible to do so. I believe this is a worthy goal, because proprietary
cloud APIs and tools are as dangerous as the proprietary compilers of
the 1980s that inspired the free software movement. I believe we have
made good progress toward this goal although we have a long way to go
yet. 

I believe we will know Aeolus is successful when we Red Hat folks
comprise less than 50% of the developer community, and when we (Red Hat
Inc.)  have to have significant arguments with that community in order
to get features in that we want in our productized version of it. By
this measure Aeolus has made 0 progress toward being successful in its
lifetime :(.

Finally, I believe that unless Aeolus is successful upstream, it will
eventually fail as a product or be absorbed into something else. I
believe the clock for this is ticking. We have no time to lose, in other
words. 

So, how can we build a user and developer community from the ground up,
around a problem space as esoteric as "save the world from cloud
lock-in"?

This is hard work but there are certain lessons we can learn from other
community projects that are successful. I'll take OpenStack as a model
here; although it, too, is far from perfect, it is by any measure a
successful upstream project.

Lesson 1: A single upstream project should do one thing,
well. Deltacloud is actually a good example of this; it has profited by
limiting itself to a stateless cloud API. Most of the OpenStack
services -- Glance, Swift, Cinder, Nova, Quantum -- also conform to this
principle. Heat has also explicitly tried to limit itself to one key
feature -- manage multi-tier applications on OpenStack -- with
reasonable success.

Lesson 2: Build things in small pieces that a single developer can
understand without too much effort. Our community will, in the end, be
built by interested developers who find a piece of what we're doing
valuable to something they're doing. We may at times finding ourselves
doing *more work* to keep things small. This is a perfectly reasonable
cost for the benefit we will derive from upstream adoption.

Lesson 2b: Keep project teams small. If a project is complex enough that
it needs more than three developers to make forward progress in it, it
will soon be incomprehensible to outsiders, thereby failing Lesson 2.

Lesson 3: The umbrella project for a set of individual projects should
have good, open governance, in which not just coding standards and API
styles but also the overall project goals and metrics are agreed by a
committee in an open process that has at least some degree of
formality. In other words, it should be possible to debate the structure
of an API, put it to a vote, and have it be a settled question from
thereon. Otherwise the individual projects under the umbrella will fail
Lessons 1 and 2.

Lesson 4: The umbrella project must itself have a compelling goal that
gets people excited and is easily understood. OpenStack is a
free-software reimplementation of EC2. Everybody wants private cloud and
everybody wants to not be locked in, so OpenStack is really
compelling. (Note Aeolus has quite similar goals, just on a meta-cloud
level.)

Lesson 5: The umbrella project has to get talked about at a lot of
conferences by a lot of different people.

Now, let's measure the current state of Aeolus against these rules,
where Aeolus (for the sake of argument) is comprised of Deltacloud,
Image Factory/Oz, Audrey, Conductor, and recent arrivals Alberich and
Tim.

Deltacloud largely succeeds across all of the lessons above, which may
explain why it is one of two Aeolus projects (with Oz being the other
one) that has gotten significant traction upstream. 

Image Factory does well on goals 1 and 2, although it has suffered by
being tied to an Image Warehouse tool that no one really understands the
purpose of. 

Oz has in fact gotten a lot of interest upstream although as far as I
know it hasn't drawn too many actual contributions. But it certainly
does well on goals 1 and 2 -- it does one thing well, and it's certainly
small enough that a single developer can easily grasp it.

Tim is a bit too early to rate, but it appears to pass lessons 1 and 2,
as does Alberich.

Audrey is also small, but it suffers from being directly tied to
Conductor, so in a sense it fails lesson 1 by doing nothing at all on
its own.

Now... let's take a look at Conductor. Conductor:

* Provides a UI for building and pushing images and wrapping them in
  deployables
* Provides a UI for launching deployables and setting launchtime
  parameters for them
* Provides a UI for managing deployables and instances while they are
  running and tracking them over time
* Provides the back end facilities for doing the launching itself,
  handling failures and exceptions, and actually setting the config
  values post launch
* Provides an extensive and complex "matching" mechanism that, given a
  user's privileges, the requirements of the application being launched,
  the relative weight of the particular cloud providers that are
  available to launch on, and the user's quota on each provider, will
  choose a suitable cloud provider to launch on
* Provides all the UI required to manage the inputs to the matching
  mechanism
* Provides full RBAC services for all of the entities in the system
* Provides a UI for managing the RBAC system
* Provides an event logging facility so that users can retrieve and
  track instance lifecycle events
* Provides graphs of the event logs

Clearly Conductor fails lesson 1. With more than 50 model classes and at
least one pair of controllers that actually do the same thing, it also
fails lesson 2.

Finally, let's look at the umbrella project, Aeolus. Aeolus clearly
fails lesson 3 because historically *I* have provided the governance
with my mighty ><}}}*>. While I find this method of governance quite
satisfying, I am forced to admit that it neither scales nor works for an
upstream project where there might be people who do not work for
me. Sigh. (Scales, get it? The ><}}}*> doesn't *scale*. I kill me!)

Does Aeolus pass Lesson 4? If not, we should all hang it up and go do
something else. Fortunately, I think Aeolus does pass lesson 4,
handily. I think there is real need for a set of cross-cloud tools, from
small shops that have nothing but an EC2 account and a few in-house
machines running VMWare, to individual developers who want to launch a
multi-tier app on whatever the cheapest public cloud happens to be at
the time, to larger organizations that want to provide a fully managed
self-service environment to internal users. I think Aeolus is well
positioned to fill this need, and I've seen other closed source tools
that do so I'm confident the need is real.

What about lesson 5? We have tried to make some headway here but we've
not done well at coming up with a good community marketing plan and
sticking with it. My hope is that improving our score on lesson 3 will
help with this.

So, where does this leave us?

1. We need to fix the governance. This means that you folks, the Aeolus
developers, need to figure out what model we are going to follow for
making architectural decisions, API decisions, coding style decisions,
and so on, put it in place, and stick to it. I would suggest that that
model be more sophisticated than "argue about it on the mailing list and
never come to a conclusion," which is what we've done to date.

2. Once we've fixed the governance, I believe we need to fix Conductor,
and I believe that fixing Conductor requires breaking it into individual
pieces that do one thing, well. One set of suggestions for those pieces,
which you may or may not agree with, is the following:

* A standalone self-service portal (i.e. Winged Monkey). Provides
  management of single instances or n-tier apps. Will work all by itself
  in front of rhev-m, openstack, vsphere, ec2, rackspace, or will work
  connecting to Heat connected to one of those things, or will work
  connected to Conductor. Appeals just as much to the individual
  developer or guy-with-a-webapp as to the corporation with
  self-service users connecting to Conductor pools.

* A standalone catalog manager service. Provides API to retrieve deployables,
  resolve the templates they require into actual build images (via
  querying Tim presumably), answer the question "what can I launch on
  this cloud."

* A standalone app design/catalog manager UI that would provide the app
  designer a GUI for Tim and the catalog manager

* A launching/state tracking mechanism -- i.e. Heat. Takes a
  CloudFormations template and launches it on a given cloud. Tracks the
  state of the instances in the template and manages failures
  appropriately (by restarting, auto-scaling, etc.)

* An event tracking service. Provides up-to-date views into instance and
  app events collected over time via an API. Clients would include
  Winged Monkey and other sophisticated reporting services.

* A cloud abstraction layer or "synthetic cloud" provider (the real meat
  of Conductor). Given configuration input by an admin, aggregates a set
  of cloud accounts and providers into a single cloud that it presents
  to the user via an API. Allows admins to shape provided clouds
  according to cost and resource needs, provider track record
  ("quality"), quota, other factors.

* Other thin UI layers as appropriate. I would prefer to see us move to
  the pattern OpenStack and OpenShift and Katello (to some extent) are
  using here, which is a set of services that provide APIs only
  (probably REST but I don't really care to be honest) and a stateless
  UI layer in front -- model code, sure, but no access to storage other
  than via the constituent projects that make up the back end. (This is
  another reason we need better governance.)

(Note I've left out Alberich here, not because I don't think it's
important but because it's more of a building block that all of these
services might use to implement their own authorization schema. There
will be other things in Conductor that fall into the same category.)

This is, more or less, the list of things that Conductor does. No wonder
it has 50+ model classes...

Now, the above is just my list. I think it's a pretty good list, and
I've thought about it a lot, but it is not a mandate (see Lesson 3). You
can see though that the reasons for doing something like Winged Monkey
go way beyond the merits of Winged Monkey itself, or any kind of
WM-vs-Conductor contest. WM is much more tied to the idea of let's make
Aeolus a great upstream project by taking the bits that might be useful
on their own and making them available for use. I think it's tough to
argue with this approach, frankly.

Well... that's a lot of writing for a Thursday afternoon. If you've made
it this far, please, let me know if you think I'm nuts or if you have a
better list of discrete Conductor components. Also, proposals for
governance models are encouraged...

Take care,
--Hugh

-- 
== Hugh Brock, hbrock at redhat.com                                   ==
== Engineering Manager, Cloud BU                                   ==
== Aeolus Project: Manage virtual infrastructure across clouds.    ==
== http://aeolusproject.org                                        ==

"I know that you believe you understand what you think I said, but I’m
not sure you realize that what you heard is not what I meant."
--Robert McCloskey



More information about the aeolus-devel mailing list