On Wed, Jun 06, 2012 at 11:47:21AM -0400, Jason Guiditta wrote:
On 04/06/12 15:39 -0400, Chris Alfonso wrote:
>Problem Statement:
>Aeolus projects upstream community is Red Hat, full stop.
>
>It's probably too easy to get caught up in hand waving and just say
"it's important to have a healthy upstream aeolus community." I've been
thinking a lot about what the benefits really are to having a "healthy"
community, and what it means to be such. I want to keep this real and not go all
academic, but I will add references to really smart people's research and perspective
to add some sort of credibility (because you guys know I keep booze at my desk and as such
might think I've already started the weekend).
>
>/me pours a shot.
>
Hope you still have that shot, I am about to disagree with, or at
least question, many things listed below. Note, I think this is really
2 discussions munged into one, so we are likly to go far off track
here:
* Discussion 1 - 'upstream community'
* Discussion 2 - 'proposed arch changes'
I had initially replied when this thread started elsewhere, before it
came to aeolus-devel. I'm going to attempt to reiterate some of what I
said in the earlier thread, and also reply to some of Jay's points here.
Hopefully it doesn't turn into a huge mess.
I agree with Jay. I think these are two separate discussions, though
perhaps not entirely unrelated.
While I realize the arguement is that 2 will affect 1, I disagree
heartily. Having a broader community involvement will get more
opinions on 2, thus either justifying it or coming up with some other
alternative we may not have thought of.
>Quickly, the value proposition of having a community (Why it matters that we create a
community)[1]:
>* Security
>* Affordability
>* Transparency
>* Perpetuity
>* Interoperablity
>* Localization
>
>A thriving, healthy open source community is one that yields the value proposition
listed above. Just throwing a license header in project and making it available for the
masses to download is not the blueprint to building a community. All this does is make
the project source code 'open'. Without a community, there simply is no project
[2]. Furthermore, we can formulate how the Aeolus projects reach the mountain-top that is
a vibrant open source community - if we agree on the value proposition listed above as the
output of a thriving, healthy community.
>
>Typical paths to a successfull community
>- Seed the project development[3]
>- Get feedback from early and often releases[3]
This one we have utterly failed on, and is why we find ourselves with
something that is a bit, _ahem_, complex to fully use.
If could wander off on a minor rant, this is a part of the project
that has disturbed me greatly for some time, and I have complained
about before. While we have arguably built something useful that can
later be expanded/changed to fit user needs, I think the primary
reason we have something as complex as we do today is our utter
failure to get feedback early and do frequent (even if small) releases
that add core functionality requested by users.
Honestly, I think there are two separate areas for improvement here.
One is that we aren't really properly releasing things with any
regularity. We do sprints, and sometimes release things on our
fedorapeople.org account. The last mention of an Aeolus release on our
website was 0.8.0, on January 21. The stuff we have in Fedora is
currently broken. So there's... room for improvement... with our release
process and regularity.
But a second bit of what I think you're getting at is that we're not
doing a very good job of collecting or soliciting user feedback. I'd
love to build not what we think sounds good, but what users actually
want. I'd love to see us building out features in response to real-world
user stories, versus to pave the way to features that we think might be
nice to have. (This isn't to suggest that we don't care about users,
just that I'd rather hear from the horse's mouse what they want. (Also,
this is not to suggest that I think that our users are horses.))
I understand there is
some amount of seeding that needed doing, but even taking that into
account, I think we built far too much and made far too many
assumptions about what our potential users would want. This caused a
certain amount of overengineering and a lot of reworking of the same
parts of the application. This has manifested in numerous ways, as a
sample:
* Time consuming complete rewrites of the UI (with no functionality
added, and sometimes functionality removed, or made harder to use)
I'm a little biased in favor of our current UI efforts which are trying
to achieve a consistent interface with Katello, but this _is_ the fourth
UI I've seen in ~18 months on the project.
* Splitting out a scheduler system before we even had anything to
schedule, and choosing one that is incredibly difficult for anyone
but an expert to understand, debug, or enhance
* Trying to encapsulate some very generic concept of a 'blob store
that is s3 compatible', and trying to make it able to do anything,
but having not a single clear way to use it to begin with.
You know, this is an interesting point I may revisit in the later
discussion. iwhd is a great "blob store that is S3 compatible" IMHO, but
it doesn't really fit the needs of Conductor, so there's talk of us
finding something new to use.
Sadly, there are many more (and we have, or are, overcoming them),
but
the point is that we attempted to do too much right off the bat
instead of addressing a small set of useful scenarios to start with,
while keeping in mind some of these ideas that we think people would
want eventually. You can build small while thinking big. In other
words, we should have provided something with less functionality, and
provided it earlier, made more of an effort to get people on board,
and then gradually built out some of these things based on what users
were asking for.
+1000 to "we should have provided something with less functionality...
earlier... [and] made more of an effort to get people on board." User
feedback early and often will work out so much better than trying to
build the perfect system in a vacuum.
I have more thoughts and examples, but I have
probably lost most readers already, so I will save further words on
this topic.
>- Attract users of the software[2]
>- Attract developers[2]
>- Letting go[2]
Agree with all these points (rant aside, which was actually violent
agreement).
>
>We, as a Red Hat sponsored group of hooligans, and I mean that in the most
affectionate way possible, have done a tremendous job seeding the aeolus projects.
We've created a metric ton of feature and function. So if we've built some pretty
cool features...why aren't they coming[5]? It's necessary to address each of the
factors at some point, but I'm just going to focus on how we attract users and
developers in this mail. I think we need to ask ourselves some tough questions and be
honest with our answers.
>
>1. Are we building function and feature that people want (individuals, entreprenuers,
enterprises) to use? If so, is the barrier to setup and use aelous low or high?
I am going to answer this one here, then respond to your answer below.
I think the answer for part 1 is 'who knows?' due to the limited scope of our
community. We have not had broad enough engagement to really get a
fair feel for this yet, imo. We have a number of different opinions on
directions and usage among developers and product management, let
alone factoring in what outside community members would find useful.
Not only "Who knows?", but that I think it's a question for potential
users, not us developers. I hope we did build something that they want,
and personally think we did, but I'm rather biased on the matter.
For part 2, I think there is a mixed answer. _If_ you are
installing
on RHEL or Fedora, and are approaching the simpler use cases (say, ec2
as a provider), I think our setup curve is not bad. Bugs aside, a
simple yum install and run of aeolus-configure will get you pretty
well ready to go. However, finding where to do things in the web app
itself needs some more work, and the entire flow is a bit on the
complex side. Now, if you want to use one of the more complicated
providers like rhev or vmware, things fall down even more. They work
just fine, but the setup/configuration is far more difficult and prone
to inexplicable breakage for a newcomer than it should be.
You're right, though the "bugs aside" about not working on Fedora is a
big problem IMHO.
I think aeolus-configure is really slick and makes things 100 times
easier than it used to be, but "edit this yaml file based on what this
wiki page suggests" is _still_ not very intuitive for RHEV and vSphere
configs. I'm not sure just yet on how we can solve that, but it's
definitely something deserving some attention.
>2. How easy is it for developers to get involved and understand
the components involved and then debug, path, contribute changes?
At the component level, I think the answer is 'not as easy as it
could/should be', but some of this as already being addressed in v2 of
factory, and the consumption of it via the image management engine,
which is what conductor will use (as well as other clients of factory,
should they so choose). Note this comment is strictly about
understanding the components.
This isn't the answer, but FWIW,
https://www.aeolusproject.org/redmine/projects/aeolus/wiki/Aeolus_Components
is a big help in figuring it out IMHO. Maybe it shouldn't take a
complicated diagram like that, though.
Debugging and contributing are much more involved than they should
be
as well, though again, some of that becomes a bit simplified soon (even
moreso if we use deltacloud as a library rather than another service,
as has been mentioned). Part of the problem though, is that the
concepts we are modelling are fairly complex, and there is a learning
curve to understand what the different objects are meant to
encapsulate.
I agree 100% here, though I think that if we're _really_ good at making
the "complicated stuff" involved in cross-cloud deployments easy for our
users, we'd also end up making it easier for developers. That's
definitely easier said than done, though.
>3. Are we doing a good job letting our target users know aeolus
exists?
>
Clearly we are not, though efforts seem to be underway to remedy
this.
Are they? I know we've talked about outreach for a long time, but I'm
yet to see substantial returns on whatever investments we've made. We've
had some interest, but there are definitely not huge flocks of people.
>There are plenty more questions we can analyze to come up with a
>self evaluation, but I think the above 3 are sufficient for this
>already too long email.
>
>- First, did we build something that is useful, I say yes. However, we built
something that is less useful than using most cloud provider management tools directly
(because we went for the common denominator feature/api base). There are some scenarios
in which aeolus is easy to setup, and there are others that are mindblowingly difficult
and confusing. The learning curve for an admin and a user is fairly steep as well. If
you don't believe me, you probably didn't hang out in the SA training classes, or
you started working on the projects at their inception and have become blind to the
complexities, or you've never had the floppyinject-hook have its way with you!
>
I think whether we built something useful was not really the question.
Absolutely it is useful (warts and all), but the unanswered question
remains 'did we build something people want to use?'. Agree that it
provides less than cloud management tools, and this dovetails nicely
with what I was trying to say abouve about building smaller chunks to
start with and getting people interested and asking for specific
features much earlier in the process.
I think it would be interesting to look at what our "competitors" offer
and see how we compare. (Please note that I'm not proposing that we
should rush to copy them. I just have an embarrassingly minimal
understanding of who else is in this space and how we compare, and I
think it could be interesting to see how they present concepts.)
>- Second, all aeolus developers are Red Hat folks. There are
probably a number of reasons for this: perhaps the components that make up aeolus
don't lend themselves well enough to reuse in other projects, they aren't puzzle
pieces that can be plugged in to solve any other problem than what we've decided to
solve. The design priniciples of each of the projects differ from one another. The
languages themselves used between the projects are different. The frameworks used between
the projects are different. The protocols used between the projects are different. What
we've done is create an enormous learning curve for any developer that dares to take
on the challenge of contributing (and they haven't even looked at the development
process guidelines at this point).
>
I have to disagree with some of this.
I want to disagree with another bit of this, though it doesn't change
your point -- not 100% of development has come from Red Hat. Surely an
overwhelming majority, but we've had a few greatly-appreciated patches
from the community, and also received feedback and suggestions from
others in the community that have helped us. It's certainly still the
case that 99% of development comes from Red Hat employees, and that we
have a huge way to go. I don't disagree with that at all. I just want to
make sure we acknowledge that the little bit of community support we
have had is very welcomed and appreciated.
I think at least some
components do lend themselves, at least conceptually to reuse in
other projects. Examples include image factory and deltacloud (yes,
not purely aeolus I guess). We also have in the incubator[1] the new
rails engine that acts as a reusable client for factory, an event
library (currently in conductor, but could easily be split out into
its own gem if there were a demand), and a growing api and cli tooling
that can be built around for other systems.
Yes, agreed. I do think there's a lot of room for improvement in terms
of reusable components, especially around the now-rather-large Conductor
app.
I also disagree with the 'use different protocols'. While
this was
true at one point, I do not think it is anymore. Pretty much all
components communicate via a REST api. Iwhd does has some weaknesses
here, but those are already being addressed as well. What different
inter-component protocols are you thinking of? Am I forgetting
something here?
Yeah, agreed. As Chris points out in his reply (I'm replying at a weird
time in this conversation, perhaps), we're inconsistent with our HTTPS
use, but fundamentally it's REST either way and whether it's a secure
connection is just an implementation detail IMHO. More consistent (or
configurable) HTTPS and OAuth use would be a plus, but I don't think the
inconsistencies constitute "different protocols." We're much, much
better here than we used to be.
Lastly, somewhat agree on the different languages thing in the
respect
that if I want to add, say, full stack support for CoolNewProvider, I
would both have to make some small additions to the ruby codebase as
well as add building functionality in factory using python. However, I
dont see this as a huge drawback, especially as we build out api for
adding new provider types on the conductor side.
Yeah, +1 here. While I think it's important for people to be able to add
support for new providers, I'm not sure too many people will be adding
new providers, versus fixing bugs / adding new functionality. I don't
love that we've used different languages, and I'd fight hard if someone
tried to add a new component in _another_ language. But I don't think
it would make sense to rewrite anything just so it used the same
language.
This means you could
just make some cli/api calls there to add your provider, and then
build the plugin for factory as needed. I suppose there is a chance
that deltacloud would need a new driver, but that has rarely been the
blocker to this point. Further I think all this can be alleviated by
being responsive to potential contributors' needs. If someone shows
up wanting support for CoolNewProvider, we should find out what parts
they are comfortable helping with, and point them to the right place
to submit feature requests for any other missing pieces.
>- Third, we have folks go to a user group here and there, *I think*. Developers
aren't good salesmen or ad-men, in general. Since we are not creating a market with
aeolus, we are way behind a lot of other projects in this space, we had probably better
pony up and do a much better job letting folks know aeolus exists and is here to take down
the competition, because we're better than them! We need to get a fleet of folks
traveling to trade-shows, meetups, usergroups, conferences. Put together consistent,
professional marketing materials and get it in people's faces.
>
Agree
To me, this is the biggest thing we need to concentrate on. While
there's a lot that we can improve on, I think our biggest problem is
that there really aren't many people using Aeolus, or, from what I can
tell, even kicking the tires. Sure, we should make it easier for them
when they find us, but what we've done so far to get the word out
doesn't seem to be working very effectively.
>markmc made a few suggestions to address making aeolus components
more useful to the masses, and I tend to agree they are great places to start aeolus
components without having to setup all the infrastructure.
>
Ok, this is what I call Discussion 2, and is the part I tend to more
agree with matty_dubs' comments, and (I think) lutter, though they may
not have made it into this thread.
The more time goes by, the more set in the belief that this is not the
solution to our problem. There is a good deal of rearchitecting that
makes sense, and some of it _will_ make components more attractive. But
it's not like, if we break Conductor up into components and shove them
into other projects, a thriving community will suddenly materialize.
If we set out with the intention of making new components smaller, more
modular components, I think we'll reap the technical and architectural
benefits, and get the bonus of them being more attractive independently.
It might even make sense to break up some existing complex functionality
over time into more discrete components. But I think it only makes sense
if we do it where there's a clear technical reason to do so. Shuffling
around the architecture and pushing chunks of Conductor into other
projects doesn't feel like an effective use of our time on its own.
Basically, much, if not all of the
below is addressing core functionality of conductor, which has been
referred to both as IaaS and a 'cloud broker'. I tend to think of it
more as the latter, but that is not really that relevant to the
current discussion I don't think. I'll briefly respond to some
specific points below, but I am all for better compartmentalizing
things where possible. However, I also think that doing that too soon
or without a clear reuse need is not beneficial.
>(1) Credentials delegation plugin for deltacloud
>
>Add a plugin interface to deltacloud which enables credential re-mapping.
>Write a plugin which authenticates the incoming user against its own
>user DB and grants any valid user access to a configured set of
>credentials
>The idea here is to allow deltacloud to be used as a proxy by which
>someone can share access to a set of credentials with others, without
>actually sharing the credentials themselves
You know, the more I think about this, the more I feel like there's a
sort of mis-match in the discussion. I think you have a well-thought-out
plan on how this should be implemented, while many of us are still at
the beginning, "Should we think about starting to move to another
architecture long-term?" phase. I sort of feel like you're describing a
series of carefully-thought-out football plays, when the rest of us are
discussing whether we should play football. The plan may well be
brilliant and exactly what we need, but at least I'm not yet in the
right mindset to appreciate it.
At a high level, it feels like the intention is to break up Conductor
into a series of pieces. Some will bring currently Conductor
functionality into Deltacloud, others will be standalone clients, and
others I'm not sure on. I think some of that makes sense. Conductor is a
big app, and some bits of functionality should probably be extracted
long-term. And if Deltacloud is wanting to become a stateful server, I
think it might make sense to push some logic into Deltacloud.
But I'm not sold on the concept that moving nearly all of what Conductor
currently does back into Deltacloud (recall that it started as a
component of Deltacloud and was moved off into an independent component)
is going to do a ton for us. And like Jay's example of iwhd above, if we
spend _too_ much time focusing on building out something modular and of
interest to everyone, we end up with a wonky component that doesn't even
meet our own needs.
I think it makes a lot of sense to look at building out new
functionality into smaller components, perhaps following the Rails
Engine approach. And I think that we should give a bit of a look at
whether dbomatic, which needs some TLC anyway, can fill some of the
notification / polling needs you described.
While I don't necessarily disagree with the specifics steps you
mentioned, I don't think we're yet at a point where my comments on
specifics ones would be valuable.
-- Matt