On 06/01/2011 11:18 AM, Mark McLoughlin wrote:
On Fri, 2011-05-27 at 12:50 -0400, Hugh Brock wrote:
> On Fri, May 27, 2011 at 05:20:51PM +0100, Mark McLoughlin wrote:
>> On Thu, 2011-05-26 at 23:49 -0400, Scott Seago wrote:
>>> On 05/26/2011 10:57 AM, Mark McLoughlin wrote:
>>> We should clarify at what levels we store quota. We've already
>>> implemented instance quota in conductor -- i.e. how many instances can
>>> be run at once by various measures:
>>> 1) user quota: how many instances an end user can run at once,
>>> regardless of pool or provider account
>>> 2) pool quota: how many instances can run in a pool, regardless of
>>> running user or provider account
>>> 3) environment/pool family quota: how many instances can run in a
>>> particular environment, regardless of pool, running user, or provider
>>> account
>>> 4) provider account quota: how many instances can run in a provider
>>> account, regardless of user, pool, or environment.
>> Well, I don't know what the rationale for those quotas was, so I can't
>> really comment.
>>
>> To me, quota is only really important as a replacement for billing.
>> Anything an admin might want to set up billing for, they will need
>> quotas for in the absence of billing.
> I think that's correct, with the exception of the pool quota and the
> pool family/environment quota. The only reason that quota exists is to
> keep one environment from DOS-ing the provider accounts that back it,
> in the event another environment is also allowed to access those
> accounts.
Apologies for getting off-topic a little, but ...
Say you have two provider accounts - one for RHEV-M and one for EC2.
Your RHEV-M install is fairly small and you can't handle more than 20
instances. Your EC2 account can handle an unlimited number of instances.
Ahh -- that's what the provider account quota is for -- at least at the
aggregate level. So in addition to whatever other quotas you want to
enforce below, in this case you'd also want to set a RHEV account quota
to 20 so condor won't try to schedule more than 20 there.
You want to allow some dev workload on RHEV-M, but not at the expense
of
forcing all production workload onto EC2.
That's the kind of case you mean, right?
With your description of a pool family quota, you'd need to limit the
dev pool family to e.g. 10 instances in order to guarantee that there is
always at least 10 instances available for the production pool family.
That seems like a unnatural constraint, since you have unlimited
resources on the EC2 account.
I don't think you'd need to limit pool family in this way -- unless
there's some requirement that each pool family schedule equally on both
providers (which would be yet another policy question), you could simply
set the pool family quotas as high as you'd like. Now if it violates
some requirement for 'dev' to use up rhev and have all of prod in ec2,
then some of the "combined" quota types you mention below could be
created -- or even a non-quota policy -- i.e. "prefer the provider with
the most available quota before the one that's almost full"
I would have imagined something like "the dev pool family can
only have
5 instances on RHEV-M, but can have an unlimited number of EC2
instances".
Sure, that would make sense too if we want to define various "combined"
or "composite" quota types. In other words, right now we have 4 types of
quota:
1) User
2) Pool
3) Pool Family/Environment
4) Provider account
The above would be a fifth one:
5) PoolFamily/Environment-ProviderAccount
But we could add 10 more by combining the basic types in more ways, so
we really need a clear set of use cases to help define where we'll need
to restrict instance usage.
> Same thing goes for pool quota -- If multiple teams share
> the "dev" environment, we want a way to limit how many instances each
> team can use so that one team doesn't monopolize the whole
> environment.
So, each team has a separate pool?
That's one way of doing it. Really an admin can set up pools however it
makes sense for the org. The original design called for each
self-service user to have his/her own "personal" Pool -- then we decided
to make self-service users share a single pool (by default). But if you
want to create a shared quota for a team, a team pool is the most
straightforward way of doing it.
Since a pool belongs to only one env, a team may have multiple pools --
i.e. a member of the conductor dev team may have access to the
"conductor dev" pool and the "conductor QE" pool.
Continuing the example above, if you had two teams, would you have
to
put a quota of e.g. 5 on each pool? Again, why have this limit when you
have don't have a limit on the EC2 account?
You may want this limit if you want to limit the number of resources a
team uses (since even if ec2 may be "unlimited" it's not "unlimited and
free")
Does it not more like "each team's pool can only use 50% of
the
resources of the dev pool family" - i.e. limiting the pool to using 5
RHEV-M instances, but only limiting the EC2 usage to 50% of unlimited?
So yeah, we'd have a couple options here:
1) create a 6th quota type "Pool/ProviderAccount"
or
2) add a notion of fractional pool definition which might allow a pool
to have a quota of N% of the PoolFamily quota, and M% of a
PoolFamily/ProviderAccount quota
This starts to get complex fast -- so we should identify the minimum set
of quotas we need -- any more and we risk a UI nightmare.
Interesting, though, that quotas aren't replaceable by billing in
these
use cases, unlike some of the other use cases I'd thought about.
Cheers,
Mark.
Scott