Aeolus Community Assessment

Chris Alfonso calfonso at redhat.com
Wed Jun 6 16:25:21 UTC 2012


On 06/06/12 11:47 -0400, Jason Guiditta wrote:
>On 04/06/12 15:39 -0400, Chris Alfonso wrote:
>>Problem Statement:
>>Aeolus projects upstream community is Red Hat, full stop.
>>
>>It's probably too easy to get caught up in hand waving and just say "it's important to have a healthy upstream aeolus community."  I've been thinking a lot about what the benefits really are to having a "healthy" community, and what it means to be such.  I want to keep this real and not go all academic, but I will add references to really smart people's research and perspective to add some sort of credibility (because you guys know I keep booze at my desk and as such might think I've already started the weekend).
>>
>>/me pours a shot.
>>
>
>Hope you still have that shot, I am about to disagree with, or at
>least question, many things listed below. Note, I think this is really
>2 discussions munged into one, so we are likly to go far off track
>here:
>* Discussion 1 - 'upstream community'
>* Discussion 2 - 'proposed arch changes'
>
>While I realize the arguement is that 2 will affect 1, I disagree
>heartily.  Having a broader community involvement will get more
>opinions on 2, thus either justifying it or coming up with some other
>alternative we may not have thought of.
>
For discussion 2, the suggested changes were based on the question of whether or not we have built components that are usable by a community outside Red Hat. The architectural changes should only be taken seriously if the honest answer to the preceeding question is "no, I suppose we haven't built something reusable by other projects or applications."  I do realize Conductor has features to support quotas, credential mapping and the like. However, my point is we have an opportunity to showcase these features as an extension of deltacloud, an apache project with a substantial brand associated with it.  At the end of the day I think the most important aspect of this conversation is that the developers and engineers should be deciding/morphing the aeolus components into what they should be (based upon community feedback).  A real concious effort needs to be made on how to grow the community and evolve the aeolus projects. 
>>Quickly, the value proposition of having a community (Why it matters that we create a community)[1]:
>>* Security
>>* Affordability
>>* Transparency
>>* Perpetuity
>>* Interoperablity
>>* Localization
>>
>>A thriving, healthy open source community is one that yields the value proposition listed above.  Just throwing a license header in project and making it available for the masses to download is not the blueprint to building a community.  All this does is make the project source code 'open'.  Without a community, there simply is no project [2]. Furthermore, we can formulate how the Aeolus projects reach the mountain-top that is a vibrant open source community - if we agree on the value proposition listed above as the output of a thriving, healthy community.
>>
>>Typical paths to a successfull community
>>- Seed the project development[3]
>>- Get feedback from early and often releases[3]
>
>This one we have utterly failed on, and is why we find ourselves with
>something that is a bit, _ahem_, complex to fully use.
>
>If could wander off on a minor rant, this is a part of the project
>that has disturbed me greatly for some time, and I have complained
>about before.  While we have arguably built something useful that can
>later be expanded/changed to fit user needs, I think the primary
>reason we have something as complex as we do today is our utter
>failure to get feedback early and do frequent (even if small) releases
>that add core functionality requested by users.  I understand there is
>some amount of seeding that needed doing, but even taking that into
>account, I think we built far too much and made far too many
>assumptions about what our potential users would want.  This caused a
>certain amount of overengineering and a lot of reworking of the same
>parts of the application.  This has manifested in numerous ways, as a
>sample:
>* Time consuming complete rewrites of the UI (with no functionality
>  added, and sometimes functionality removed, or made harder to use)
>* Splitting out a scheduler system before we even had anything to
>  schedule, and choosing one that is incredibly difficult for anyone
>  but an expert to understand, debug, or enhance
>* Trying to encapsulate some very generic concept of a 'blob store
>  that is s3 compatible', and trying to make it able to do anything,
>  but having not a single clear way to use it to begin with.
>
>Sadly, there are many more (and we have, or are, overcoming them), but
>the point is that we attempted to do too much right off the bat
>instead of addressing a small set of useful scenarios to start with,
>while keeping in mind some of these ideas that we think people would
>want eventually.  You can build small while thinking big.  In other
>words, we should have provided something with less functionality, and
>provided it earlier, made more of an effort to get people on board,
>and then gradually built out some of these things based on what users
>were asking for.  I have more thoughts and examples, but I have
>probably lost most readers already, so I will save further words on
>this topic.
>
+1000
>>- Attract users of the software[2]
>>- Attract developers[2]
>>- Letting go[2]
>
>Agree with all these points (rant aside, which was actually violent
>agreement).
>
>>
>>We, as a Red Hat sponsored group of hooligans, and I mean that in the most affectionate way possible, have done a tremendous job seeding the aeolus projects.  We've created a metric ton of feature and function.  So if we've built some pretty cool features...why aren't they coming[5]?  It's necessary to address each of the factors at some point, but I'm just going to focus on how we attract users and developers in this mail.  I think we need to ask ourselves some tough questions and be honest with our answers.
>>
>>1. Are we building function and feature that people want (individuals, entreprenuers, enterprises) to use?  If so, is the barrier to setup and use aelous low or high?
>I am going to answer this one here, then respond to your answer below.
>
>I think the answer for part 1 is 'who knows?' due to the limited scope of our
>community.  We have not had broad enough engagement to really get a
>fair feel for this yet, imo. We have a number of different opinions on
>directions and usage among developers and product management, let
>alone factoring in what outside community members would find useful.
>
>For part 2, I think there is a mixed answer.  _If_ you are installing
>on RHEL or Fedora, and are approaching the simpler use cases (say, ec2
>as a provider), I think our setup curve is not bad.  Bugs aside, a
>simple yum install and run of aeolus-configure will get you pretty
>well ready to go.  However, finding where to do things in the web app
>itself needs some more work, and the entire flow is a bit on the
>complex side.  Now, if you want to use one of the more complicated
>providers like rhev or vmware, things fall down even more.  They work
>just fine, but the setup/configuration is far more difficult and prone
>to inexplicable breakage for a newcomer than it should be.
>
I tend to agree with you in that, we don't know the answer.  If we have built something useful, then folks just don't know about it.  If we haven't, then we should.  Either way we have work to accomplish.  First this is to figure out if developers outside our small group find value in the aeolus components.  It's a bit of a chicken and egg thing, we are guessing until we have feedback.
>>2. How easy is it for developers to get involved and understand the components involved and then debug, path, contribute changes?
>
>At the component level, I think the answer is 'not as easy as it
>could/should be', but some of this as already being addressed in v2 of
>factory, and the consumption of it via the image management engine,
>which is what conductor will use (as well as other clients of factory,
>should they so choose).  Note this comment is strictly about
>understanding the components.
>
>Debugging and contributing are much more involved than they should be
>as well, though again, some of that becomes a bit simplified soon (even
>moreso if we use deltacloud as a library rather than another service,
>as has been mentioned).  Part of the problem though, is that the
>concepts we are modelling are fairly complex, and there is a learning
>curve to understand what the different objects are meant to
>encapsulate.
>>3. Are we doing a good job letting our target users know aeolus exists?
>>
>
>Clearly we are not, though efforts seem to be underway to remedy
>this.
>
>>There are plenty more questions we can analyze to come up with a 
>>self evaluation, but I think the above 3 are sufficient for this 
>>already too long email.
>>
>>- First, did we build something that is useful, I say yes.  However, we built something that is less useful than using most cloud provider management tools directly (because we went for the common denominator feature/api base).  There are some scenarios in which aeolus is easy to setup, and there are others that are mindblowingly difficult and confusing.  The learning curve for an admin and a user is fairly steep as well.  If you don't believe me, you probably didn't hang out in the SA training classes, or you started working on the projects at their inception and have become blind to the complexities, or you've never had the floppyinject-hook have its way with you!
>>
>
>I think whether we built something useful was not really the question.
>Absolutely it is useful (warts and all), but the unanswered question
>remains 'did we build something people want to use?'.  Agree that it
>provides less than cloud management tools, and this dovetails nicely
>with what I was trying to say abouve about building smaller chunks to
>start with and getting people interested and asking for specific
>features much earlier in the process.
>
>>- Second, all aeolus developers are Red Hat folks.  There are probably a number of reasons for this: perhaps the components that make up aeolus don't lend themselves well enough to reuse in other projects, they aren't puzzle pieces that can be plugged in to solve any other problem than what we've decided to solve. The design priniciples of each of the projects differ from one another.  The languages themselves used between the projects are different.  The frameworks used between the projects are different.  The protocols used between the projects are different. What we've done is create an enormous learning curve for any developer that dares to take on the challenge of contributing (and they haven't even looked at the development process guidelines at this point).
>>
>I have to disagree with some of this.  I think at least some
>components do lend themselves, at least conceptually to reuse in
>other projects.  Examples include image factory and deltacloud (yes,
>not purely aeolus I guess). We also have in the incubator[1] the new
>rails engine that acts as a reusable client for factory, an event
>library (currently in conductor, but could easily be split out into
>its own gem if there were a demand), and a growing api and cli tooling
>that can be built around for other systems.
>
>I also disagree with the 'use different protocols'.  While this was
>true at one point, I do not think it is anymore.  Pretty much all
>components communicate via a REST api.  Iwhd does has some weaknesses
>here, but those are already being addressed as well.  What different
>inter-component protocols are you thinking of? Am I forgetting
>something here?
We've made improvements here for sure, but we still use SSL connections between conductor and config server, which is interesting if you're doing oauth debugging.  We don't between the conductor and IF and iwhd, which has it's good and bad, but it's inconsistent and is a bit of a surprise when you're a developer debugging stuff.
>
>Lastly, somewhat agree on the different languages thing in the respect
>that if I want to add, say, full stack support for CoolNewProvider, I
>would both have to make some small additions to the ruby codebase as
>well as add building functionality in factory using python. However, I
>dont see this as a huge drawback, especially as we build out api for
>adding new provider types on the conductor side.  This means you could
>just make some cli/api calls there to add your provider, and then
>build the plugin for factory as needed.  I suppose there is a chance
>that deltacloud would need a new driver, but that has rarely been the
>blocker to this point.  Further I think all this can be alleviated by
>being responsive to potential contributors' needs.  If someone shows
>up wanting support for CoolNewProvider, we should find out what parts
>they are comfortable helping with, and point them to the right place
>to submit feature requests for any other missing pieces.
>
>
You don't see it as a drawback, that's fine.  But the reality is it requires additional investement in a skillset that a developer may or may not have.  It makes the pool of developers from which to pull from smaller.  It's a barrier to adoption.
>>- Third, we have folks go to a user group here and there, *I think*. Developers aren't good salesmen or ad-men, in general.  Since we are not creating a market with aeolus, we are way behind a lot of other projects in this space, we had probably better pony up and do a much better job letting folks know aeolus exists and is here to take down the competition, because we're better than them!  We need to get a fleet of folks traveling to trade-shows, meetups, usergroups, conferences.  Put together consistent, professional marketing materials and get it in people's faces.
>>
>
>Agree
>
>>markmc made a few suggestions to address making aeolus components more useful to the masses, and I tend to agree they are great places to start aeolus components without having to setup all the infrastructure.
>>
>Ok, this is what I call Discussion 2, and is the part I tend to more
>agree with matty_dubs' comments, and (I think) lutter, though they may
>not have made it into this thread.  Basically, much, if not all of the
>below is addressing core functionality of conductor, which has been
>referred to both as IaaS and a 'cloud broker'.  I tend to think of it
>more as the latter, but that is not really that relevant to the
>current discussion I don't think.  I'll briefly respond to some
>specific points below, but I am all for better compartmentalizing
>things where possible.  However, I also think that doing that too soon
>or without a clear reuse need is not beneficial.
>
>>(1) Credentials delegation plugin for deltacloud
>>
>>Add a plugin interface to deltacloud which enables credential re-mapping.
>>Write a plugin which authenticates the incoming user against its own
>>user DB and grants any valid user access to a configured set of
>>credentials
>>The idea here is to allow deltacloud to be used as a proxy by which
>>someone can share access to a set of credentials with others, without
>>actually sharing the credentials themselves
>
>This seems to me to be exactly a large purpose of conductor, why would
>we move it into deltacloud?  I would argue that conductor is the
>proxy, and if we were to use deltacloud as a library instead of
>another service, the user would need to only install and configure
>conductor to get the same benefit I think you are trying to describe.
>Now, does it make sense to pull the credential-mapping functionality
>we _already have_ into a plugin or rails engine?  Perhaps, but I am
>not completely convinced yet.
>>
>>(2) Notifications of interesting deltacloud events
>>Add a plugin interface which is called with details of any interesting
>>events happening in deltacloud.
>>Provide plugins for publishing those notifications over Qpid, RabbitMQ,
>>ZeroMQ, syslog, etc.
>>The idea here is that the person sharing their credentials can use this
>>notifications system to do chargeback
>>
>Again, we already have this in conductor/dbomatic.  Dbomatic checks
>with deltacloud for (certain) events, and we have a small library in
>condcutor (which is actually already structured for reuse) that writes
>events to syslog.  Further, it is designed for a 'swappable backend'
>meaning if you want to write a converter to output to something other
>than syslog, you are welcome and able to do so [2].  If Deltacloud added
>the return of events via http calls back to conductor, the current
>setup would log any such events we were interested in. Maybe I should
>move that library out into its own gem if people other than conductor
>are wanting to use it.
>
>>(3) Track resource ownership
>>If user A launches an instance via the credential re-mapping plugin,
>>then user B should not be able to see the instance or kill it.
>>Requires the plugin to have a DB which tracks the ownership of instances
>>
>Again, Conductor does this already, no?
>
>>(4) Add instance state change event notifications
>>There should be a way to have a service monitor clouds (via deltacloud)
>>for the state of an instance and notify you of changes via a message
>>bus.
>This seems largely the same as 2 above to me.
>
>>Is this external to deltacloud, just polling via deltacloud? If so, what
>>credentials is it using to do the polling?
>Yes, dbomatic does the polling, and that is because of the current
>strict RESTful design of deltacloud combined with the fact that
>providers do not send events that deltacloud can then bubble up -
>someone has to poll them for those changes, and deltacloud currently
>acts as a pass-through for requesting these events.  If deltacloud
>adds a callback mechanism, that may remove one layer of polling, but I
>am not convinced it entirely would.  Only the client of deltacloud
>knows what events they are interested in, and I dont think deltacloud
>itself should have to know this.  I guess it could work if there were a
>way for deltacloud to ask a provider for 'all current events' for a
>given user, but seems like that could be an ugly mess.  Lastly, it
>uses the credentials passed in by conductor/dbomatic on each request,
>they are never stored by deltacloud - this keeps them in one place,
>conductor.
>
>>Is it part of the credential re-mapping plugin, since that has access to
>>credentials? But surely this is useful even if you're not using
>>credential re-mapping? Does it publish these state changes via the same mechanism as
>>deltacloud's notification system?
>>
>>(4) Add per-user quotas
>>The admin sharing a set of credentials should be able to set quotas on
>>each of the user accounts. This is for safety sake, not as a replacement
>>for chargeback.
>>The plugin needs to listen for instance launch notifications and
>>instance state change notifications in order to update the quota usage
>>of each user.
>>
>Again, this is in conductor.  There area few layers of quota supported
>there, in fact.  Again, could this become better encapulated?  Very
>likely, but it does exist, and I dont see why deltacloud needs to be
>involved.
>
>>(5) Extend (1) to support credentials for multiple drivers/providers
>>
>Done
>
>>(6) Experiment with ways for deltacloud (via a plugin or driver) to map
>>incoming requests to one of multiple clouds
>>
>Done, see conductor.
>
>>(7) Explore using Heat on top of deltacloud
>>
>>Suggested Actions:
>>1 - take one some of the suggested architectural changes that make the aeolus components more reusable in projects outside aeolus as a whole
>>2 - set up a schedule to start marketing the components and aeolus as a whole, go out and get community members instead of waiting for them to show up
>>
>Agree with 2.
>
>>If you think this is bonkers or if you have any other opinions on how we can better foster the community, let's hear it.
>>
>I think it is bonkers (1 only) :)  Seriously, I think that you raise a number
>of good points, but do not see the value in most of this
>re-architecture.  It seems to be again (as we have done so many times
>in the past) moving existing functionality around into new places, with
>no obvious gain.  I think we would be much better served looking at
>each of these _existing_ pieces of functionality and seeing if we can
>better encapsulate them, and where, if anywhere, someone might want to
>use them outside of their current scope.  Even in the latter case, I
>think the _code_ should be better encapulated first, and separate
>libraries created only when there is a real need.  Remember that every
>library we split out will add to our release overhead - new gems, new
>rpms, new insert-distribution-method-here.  I am not saying never to
>do it, but that having 20 little libraries that are only used together
>is not a big gain over well-defined and encapsulated code that happens
>to live in the same codebase (and yes, this is a part that in many
>cases needs some love).
>
>Thanks for starting this discussion, I think we have the potential to
>correct some long-standing issues here if people participate and are
>honest in their replies (and hopefully slightly less long winded than
>myself).
>
I really appreciate that you've taken the time to respond to this thread in such detail and with such honesty.  I think the best thing we can do is be honest with outselves about where we want the community to end up, analyze what is keeping us from getting there, and rectifying anything that needs it.  The developers on the upstream should be able to evaluate and organize this, rather than just code, code, code, code, code...  In summary,  it takes more than just code to build a community.

- Chris
>-j
>
>[1] https://github.com/aeolus-incubator/image-management-engine
>[2]
>https://www.aeolusproject.org/redmine/projects/aeolus/wiki/Feature_1_0_Syslog_Event_Documentation
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <https://lists.fedorahosted.org/pipermail/aeolus-devel/attachments/20120606/5db840fc/attachment.sig>


More information about the aeolus-devel mailing list