On 06/22/2011 03:45 PM, Steven Dake wrote:
On 06/22/2011 11:54 AM, Scott Seago wrote:
> On 06/22/2011 12:52 PM, Steven Dake wrote:
>> It would be helpful to have a debate about the limits we intend to
>> support with aeolus atleast from an engineering perspective. This
>> allows us to communicate with our user community and helps fedora test
>> days get some idea of what they should be trying to exercise or might
>> find out of scope.
>>
>> The first step is identifying the limiting factors. The second step is
>> associating a number with them and what contributes to that limit. This
>> info provides developers with a ballpark number to shoot for
>> implementation wise. Once we have this info, we can put it on the
>> Aeolus website for all to see :)
>>
>> Let us focus on creating a list of limiting factors first.
>>
>> I'll start off with some limiting factors I find interesting for my work:
>> 1. Limit of deployables in the system
>> 2. Limit of assemblies in the system
> These two no longer apply since they're no longer centrally managed. At
> this point deployables are just xml files that are http-accessible --
Since they are http accessible, they are stored somewhere. This leads
to the following applications of a limit (although perhaps very large):
1. The time to find the deployable xml data-blob given a large number
of deployables
If we're just storing them as files served by http, this limit
would be
large indeed.
2. The storage required for storing each deployable xml blob
Same as above
3. The cli may only be able to query a certain limit of
deployables
when iterating
etc.
This one wouldn't apply to the conductor side of things. If (when) we
build some sort of deployable management application that maintains,
allows queries over, etc deployables, then all of the above might apply.
At this point the limit of conductor's interaction with the deployable
store is to pull the contents at
http://my-server.com/my-deployable.xml.
> so you can have as many as you want. Assemblies are just defined
inline
> in the deployable definition file, so the same comment applies.
>> 3. Limit of images in the system
> And to split this out -- we're storing these in a hierarchy in iwhd:
> - Image (version-independent) -- i.e. "jboss AS"
> - Build (target-independent) -- i.e. "jboss AS version 1.1"
> - Target image -- i.e. jboss AS, version 1.1 for ec2
> - Provider image -- i.e. jboss AS, version 1.1 for ec2-us-east-1,
> ami-12345
>
> Are we more concerned with total number of objects in iwhd (i.e. sum of
> the above) or the number of each type, or the number with a given parent?
We would need your expertise there to define the specific limits. See
example above (re storage and display).
So this is probably mostly a question of IWHD performance as well as
performance of the IWHD client models in conductor (i.e. at what point
does Images.all fall apart due to the size of the dataset).
Coming from the other side, do we have any numbers as to what sort of
minimum limits we'd like to be able to set? i.e. if the software can
handle 10,000 but from a use case/requirements point of view we are only
asking to be able to handle 1000 we're all set here -- if we need to
support 10,000 but the current implementation can only handle 1000 it's
more difficult to work through.
There's also the question of total storage available -- more images mean
we might fill up the filesystem, but this isn't something we can set an
across-the-board limit on, since it's a function of total space available.
>> 4. Limit of assemblies in a deployable
> Right now we're not enforcing a hard limit, but it might make sense to
> at least target a certain range (i.e. deployables with> 5 assemblies
> are not recommended, etc)
We don't have to enforce the limits, but should have some idea of what
we think they would be given our overall system.
Sure. So here the biggest factor is one of scheduling -- since all
instances in a deployment must be launched in the same provider, as the
set of instances you need to launch together becomes larger, the
probability of success drops since you may not find a single provider
account with the excess capacity to launch it all in one place.
Scott