Aeolus Community Assessment

Jason Guiditta jguiditt at redhat.com
Wed Jun 6 15:47:21 UTC 2012


On 04/06/12 15:39 -0400, Chris Alfonso wrote:
>Problem Statement:
>Aeolus projects upstream community is Red Hat, full stop.
>
>It's probably too easy to get caught up in hand waving and just say "it's important to have a healthy upstream aeolus community."  I've been thinking a lot about what the benefits really are to having a "healthy" community, and what it means to be such.  I want to keep this real and not go all academic, but I will add references to really smart people's research and perspective to add some sort of credibility (because you guys know I keep booze at my desk and as such might think I've already started the weekend).
>
>/me pours a shot.
>

Hope you still have that shot, I am about to disagree with, or at
least question, many things listed below. Note, I think this is really
2 discussions munged into one, so we are likly to go far off track
here:
* Discussion 1 - 'upstream community'
* Discussion 2 - 'proposed arch changes'  

While I realize the arguement is that 2 will affect 1, I disagree
heartily.  Having a broader community involvement will get more
opinions on 2, thus either justifying it or coming up with some other
alternative we may not have thought of.

>Quickly, the value proposition of having a community (Why it matters that we create a community)[1]:
>* Security
>* Affordability
>* Transparency
>* Perpetuity
>* Interoperablity
>* Localization
>
>A thriving, healthy open source community is one that yields the value proposition listed above.  Just throwing a license header in project and making it available for the masses to download is not the blueprint to building a community.  All this does is make the project source code 'open'.  Without a community, there simply is no project [2]. Furthermore, we can formulate how the Aeolus projects reach the mountain-top that is a vibrant open source community - if we agree on the value proposition listed above as the output of a thriving, healthy community.
>
>Typical paths to a successfull community
>- Seed the project development[3]
>- Get feedback from early and often releases[3]

This one we have utterly failed on, and is why we find ourselves with
something that is a bit, _ahem_, complex to fully use.

If could wander off on a minor rant, this is a part of the project
that has disturbed me greatly for some time, and I have complained
about before.  While we have arguably built something useful that can
later be expanded/changed to fit user needs, I think the primary
reason we have something as complex as we do today is our utter
failure to get feedback early and do frequent (even if small) releases
that add core functionality requested by users.  I understand there is
some amount of seeding that needed doing, but even taking that into
account, I think we built far too much and made far too many
assumptions about what our potential users would want.  This caused a
certain amount of overengineering and a lot of reworking of the same
parts of the application.  This has manifested in numerous ways, as a
sample:
* Time consuming complete rewrites of the UI (with no functionality
   added, and sometimes functionality removed, or made harder to use)
* Splitting out a scheduler system before we even had anything to
   schedule, and choosing one that is incredibly difficult for anyone
   but an expert to understand, debug, or enhance
* Trying to encapsulate some very generic concept of a 'blob store
   that is s3 compatible', and trying to make it able to do anything,
   but having not a single clear way to use it to begin with.

Sadly, there are many more (and we have, or are, overcoming them), but
the point is that we attempted to do too much right off the bat
instead of addressing a small set of useful scenarios to start with,
while keeping in mind some of these ideas that we think people would
want eventually.  You can build small while thinking big.  In other
words, we should have provided something with less functionality, and
provided it earlier, made more of an effort to get people on board,
and then gradually built out some of these things based on what users
were asking for.  I have more thoughts and examples, but I have
probably lost most readers already, so I will save further words on
this topic.

>- Attract users of the software[2]
>- Attract developers[2]
>- Letting go[2]

Agree with all these points (rant aside, which was actually violent
agreement).

>
>We, as a Red Hat sponsored group of hooligans, and I mean that in the most affectionate way possible, have done a tremendous job seeding the aeolus projects.  We've created a metric ton of feature and function.  So if we've built some pretty cool features...why aren't they coming[5]?  It's necessary to address each of the factors at some point, but I'm just going to focus on how we attract users and developers in this mail.  I think we need to ask ourselves some tough questions and be honest with our answers.
>
>1. Are we building function and feature that people want (individuals, entreprenuers, enterprises) to use?  If so, is the barrier to setup and use aelous low or high?
I am going to answer this one here, then respond to your answer below.

I think the answer for part 1 is 'who knows?' due to the limited scope of our
community.  We have not had broad enough engagement to really get a
fair feel for this yet, imo. We have a number of different opinions on
directions and usage among developers and product management, let
alone factoring in what outside community members would find useful.

For part 2, I think there is a mixed answer.  _If_ you are installing
on RHEL or Fedora, and are approaching the simpler use cases (say, ec2
as a provider), I think our setup curve is not bad.  Bugs aside, a
simple yum install and run of aeolus-configure will get you pretty
well ready to go.  However, finding where to do things in the web app
itself needs some more work, and the entire flow is a bit on the
complex side.  Now, if you want to use one of the more complicated
providers like rhev or vmware, things fall down even more.  They work
just fine, but the setup/configuration is far more difficult and prone
to inexplicable breakage for a newcomer than it should be.

>2. How easy is it for developers to get involved and understand the components involved and then debug, path, contribute changes?

At the component level, I think the answer is 'not as easy as it
could/should be', but some of this as already being addressed in v2 of
factory, and the consumption of it via the image management engine,
which is what conductor will use (as well as other clients of factory,
should they so choose).  Note this comment is strictly about
understanding the components.

Debugging and contributing are much more involved than they should be
as well, though again, some of that becomes a bit simplified soon (even
moreso if we use deltacloud as a library rather than another service,
as has been mentioned).  Part of the problem though, is that the
concepts we are modelling are fairly complex, and there is a learning
curve to understand what the different objects are meant to
encapsulate.
>3. Are we doing a good job letting our target users know aeolus exists?
>

Clearly we are not, though efforts seem to be underway to remedy
this.

>There are plenty more questions we can analyze to come up with a self 
>evaluation, but I think the above 3 are sufficient for this already 
>too long email.
>
>- First, did we build something that is useful, I say yes.  However, we built something that is less useful than using most cloud provider management tools directly (because we went for the common denominator feature/api base).  There are some scenarios in which aeolus is easy to setup, and there are others that are mindblowingly difficult and confusing.  The learning curve for an admin and a user is fairly steep as well.  If you don't believe me, you probably didn't hang out in the SA training classes, or you started working on the projects at their inception and have become blind to the complexities, or you've never had the floppyinject-hook have its way with you!
>

I think whether we built something useful was not really the question.
Absolutely it is useful (warts and all), but the unanswered question
remains 'did we build something people want to use?'.  Agree that it
provides less than cloud management tools, and this dovetails nicely
with what I was trying to say abouve about building smaller chunks to
start with and getting people interested and asking for specific
features much earlier in the process.

>- Second, all aeolus developers are Red Hat folks.  There are probably a number of reasons for this: perhaps the components that make up aeolus don't lend themselves well enough to reuse in other projects, they aren't puzzle pieces that can be plugged in to solve any other problem than what we've decided to solve. The design priniciples of each of the projects differ from one another.  The languages themselves used between the projects are different.  The frameworks used between the projects are different.  The protocols used between the projects are different. What we've done is create an enormous learning curve for any developer that dares to take on the challenge of contributing (and they haven't even looked at the development process guidelines at this point).
>
I have to disagree with some of this.  I think at least some
components do lend themselves, at least conceptually to reuse in
other projects.  Examples include image factory and deltacloud (yes,
not purely aeolus I guess). We also have in the incubator[1] the new
rails engine that acts as a reusable client for factory, an event
library (currently in conductor, but could easily be split out into
its own gem if there were a demand), and a growing api and cli tooling
that can be built around for other systems.

I also disagree with the 'use different protocols'.  While this was
true at one point, I do not think it is anymore.  Pretty much all
components communicate via a REST api.  Iwhd does has some weaknesses
here, but those are already being addressed as well.  What different
inter-component protocols are you thinking of? Am I forgetting
something here?

Lastly, somewhat agree on the different languages thing in the respect
that if I want to add, say, full stack support for CoolNewProvider, I
would both have to make some small additions to the ruby codebase as
well as add building functionality in factory using python. However, I
dont see this as a huge drawback, especially as we build out api for
adding new provider types on the conductor side.  This means you could
just make some cli/api calls there to add your provider, and then
build the plugin for factory as needed.  I suppose there is a chance
that deltacloud would need a new driver, but that has rarely been the
blocker to this point.  Further I think all this can be alleviated by
being responsive to potential contributors' needs.  If someone shows
up wanting support for CoolNewProvider, we should find out what parts
they are comfortable helping with, and point them to the right place
to submit feature requests for any other missing pieces.


>- Third, we have folks go to a user group here and there, *I think*. Developers aren't good salesmen or ad-men, in general.  Since we are not creating a market with aeolus, we are way behind a lot of other projects in this space, we had probably better pony up and do a much better job letting folks know aeolus exists and is here to take down the competition, because we're better than them!  We need to get a fleet of folks traveling to trade-shows, meetups, usergroups, conferences.  Put together consistent, professional marketing materials and get it in people's faces.
>

Agree

>markmc made a few suggestions to address making aeolus components more useful to the masses, and I tend to agree they are great places to start aeolus components without having to setup all the infrastructure.
>
Ok, this is what I call Discussion 2, and is the part I tend to more
agree with matty_dubs' comments, and (I think) lutter, though they may
not have made it into this thread.  Basically, much, if not all of the
below is addressing core functionality of conductor, which has been
referred to both as IaaS and a 'cloud broker'.  I tend to think of it
more as the latter, but that is not really that relevant to the
current discussion I don't think.  I'll briefly respond to some
specific points below, but I am all for better compartmentalizing
things where possible.  However, I also think that doing that too soon
or without a clear reuse need is not beneficial.

>(1) Credentials delegation plugin for deltacloud
>
>Add a plugin interface to deltacloud which enables credential re-mapping.
>Write a plugin which authenticates the incoming user against its own
>user DB and grants any valid user access to a configured set of
>credentials
>The idea here is to allow deltacloud to be used as a proxy by which
>someone can share access to a set of credentials with others, without
>actually sharing the credentials themselves

This seems to me to be exactly a large purpose of conductor, why would
we move it into deltacloud?  I would argue that conductor is the
proxy, and if we were to use deltacloud as a library instead of
another service, the user would need to only install and configure
conductor to get the same benefit I think you are trying to describe.
Now, does it make sense to pull the credential-mapping functionality
we _already have_ into a plugin or rails engine?  Perhaps, but I am
not completely convinced yet.
>
>(2) Notifications of interesting deltacloud events
>Add a plugin interface which is called with details of any interesting
>events happening in deltacloud.
>Provide plugins for publishing those notifications over Qpid, RabbitMQ,
>ZeroMQ, syslog, etc.
>The idea here is that the person sharing their credentials can use this
>notifications system to do chargeback
>
Again, we already have this in conductor/dbomatic.  Dbomatic checks
with deltacloud for (certain) events, and we have a small library in
condcutor (which is actually already structured for reuse) that writes
events to syslog.  Further, it is designed for a 'swappable backend'
meaning if you want to write a converter to output to something other
than syslog, you are welcome and able to do so [2].  If Deltacloud added
the return of events via http calls back to conductor, the current
setup would log any such events we were interested in. Maybe I should
move that library out into its own gem if people other than conductor
are wanting to use it.

>(3) Track resource ownership
>If user A launches an instance via the credential re-mapping plugin,
>then user B should not be able to see the instance or kill it.
>Requires the plugin to have a DB which tracks the ownership of instances
>
Again, Conductor does this already, no?

>(4) Add instance state change event notifications
>There should be a way to have a service monitor clouds (via deltacloud)
>for the state of an instance and notify you of changes via a message
>bus.
This seems largely the same as 2 above to me.

>Is this external to deltacloud, just polling via deltacloud? If so, what
>credentials is it using to do the polling?
Yes, dbomatic does the polling, and that is because of the current
strict RESTful design of deltacloud combined with the fact that
providers do not send events that deltacloud can then bubble up -
someone has to poll them for those changes, and deltacloud currently
acts as a pass-through for requesting these events.  If deltacloud
adds a callback mechanism, that may remove one layer of polling, but I
am not convinced it entirely would.  Only the client of deltacloud
knows what events they are interested in, and I dont think deltacloud
itself should have to know this.  I guess it could work if there were a
way for deltacloud to ask a provider for 'all current events' for a
given user, but seems like that could be an ugly mess.  Lastly, it
uses the credentials passed in by conductor/dbomatic on each request,
they are never stored by deltacloud - this keeps them in one place,
conductor.

>Is it part of the credential re-mapping plugin, since that has access to
>credentials? But surely this is useful even if you're not using
>credential re-mapping? Does it publish these state changes via the same mechanism as
>deltacloud's notification system?
>
>(4) Add per-user quotas
>The admin sharing a set of credentials should be able to set quotas on
>each of the user accounts. This is for safety sake, not as a replacement
>for chargeback.
>The plugin needs to listen for instance launch notifications and
>instance state change notifications in order to update the quota usage
>of each user.
>
Again, this is in conductor.  There area few layers of quota supported
there, in fact.  Again, could this become better encapulated?  Very
likely, but it does exist, and I dont see why deltacloud needs to be
involved.

>(5) Extend (1) to support credentials for multiple drivers/providers
>
Done

>(6) Experiment with ways for deltacloud (via a plugin or driver) to map
>incoming requests to one of multiple clouds
>
Done, see conductor.

>(7) Explore using Heat on top of deltacloud
>
>Suggested Actions:
>1 - take one some of the suggested architectural changes that make the aeolus components more reusable in projects outside aeolus as a whole
>2 - set up a schedule to start marketing the components and aeolus as a whole, go out and get community members instead of waiting for them to show up
>
Agree with 2.

>If you think this is bonkers or if you have any other opinions on how we can better foster the community, let's hear it.
>
I think it is bonkers (1 only) :)  Seriously, I think that you raise a number
of good points, but do not see the value in most of this
re-architecture.  It seems to be again (as we have done so many times
in the past) moving existing functionality around into new places, with
no obvious gain.  I think we would be much better served looking at
each of these _existing_ pieces of functionality and seeing if we can
better encapsulate them, and where, if anywhere, someone might want to
use them outside of their current scope.  Even in the latter case, I
think the _code_ should be better encapulated first, and separate
libraries created only when there is a real need.  Remember that every
library we split out will add to our release overhead - new gems, new
rpms, new insert-distribution-method-here.  I am not saying never to
do it, but that having 20 little libraries that are only used together
is not a big gain over well-defined and encapsulated code that happens
to live in the same codebase (and yes, this is a part that in many
cases needs some love).

Thanks for starting this discussion, I think we have the potential to
correct some long-standing issues here if people participate and are
honest in their replies (and hopefully slightly less long winded than
myself).

-j

[1] https://github.com/aeolus-incubator/image-management-engine
[2]
https://www.aeolusproject.org/redmine/projects/aeolus/wiki/Feature_1_0_Syslog_Event_Documentation




More information about the aeolus-devel mailing list