On 05/12/2011 10:16 AM, Greg Blomquist wrote:
On your first point, there's no simple way to guarantee that the necessary packages are in a template to support late-bound services if users don't refer to the packagegroup that defines those dependencies when they're creating the template, though we can point out when users are adding services to an instance of a template which wasn't explicity deisgned to accommodate that specific service.
Oh, I should quickly point out here that instead of "templates", we're probably talking about "icicle" (as long as it's still called that). IOW, the thing that is produced by Image Factory when it bakes an image based on a template.
Not necessarily. A deployable combines a set of templates which, at launch time, will get mapped to a set of images (icicle) on a given cloud provider by condor. At the point when we create the deployable,though, the notion is that it needs to work for all providers for which the templates can be built.
This gives us a thing that actually lists all the packages installed in the image. Which gets us closer to being able to detect whether a particular deployable (with late-bound services) is going to violate the pre-defined list of packages in an image.
So the problem here is when we assign services to the deployable, the images may not have been built yet -- and if they have, they may not be complete (additional providers added later, etc).
The questions I have regarding the puppet master are:
- Where is the puppet master hosted? This is not hugely important,
but I want to see if we're thinking about the n+1 (or n+m) problem of instance proliferation needed to support post-boot configuration. If the puppet master is part of the config server, then we're back to n+1. If it needs to be separate (for some reason I don't see right now) then we're to n+2. We have to watch that "+m" piece carefully.
I'd imagine it would need to be part of config server (from the Conductor point of view, it's an implementation detail -- we just need to be told where to send the config scripts)
- Is the user responsible for creating this ahead of time manually?
Or, is this something that could be part of the global setup of cloud engine? Probably short term the answer is, "this is manual". Perhaps longer term this could be something automagically configured when a provider is added to Cloud Engine.
If a config server is needed for each provider (i.e. one in ec2-us-east, one in rhev-server-2.redhat.com, etc) then ideally this happens automatically when we add a provider. I'm not sure whether that sort of thing would be in scope for 1.0 or if it would be a documented manual step for each provider.
The need to store a different image for each variation of the services that might run on a single OS, and to do that for multiple providers, will turn into a constraint on scalability once we have users seeking to manage hundreds, or thousands, of instances.
This is where I get hung up.
I see that there's two competing issues here. One one hand, we want to provide the user with a way to spin up an instance an be running quickly. This requires that the image be pre-loaded with all (or almost all) the packages required to be running for instances based on that image.
On the other hand, we want to provide the user with flexibility of being able to customize the instance with post-boot configuration so that they might spin up several instances from one image and have those instances be fairly distinct from each other (e.g., db server and http server from one image).
It sounds like the solution being talked about is to front-load all the packages into an image so that it might be able to satisfy both requirements simultaneously. That might be a good solution for the interim, but I think the image proliferation problem is going to get real really fast despite this solution.
Well I don't think we'd want to front-load all services in general. I'd imagine this sort of "generic template that contains the packages for a bunch of services" would mostly be useful for various "utility" services -- things that tend to rely on a common set of basic packages, and for which the service-specific packages are small. For more complex services, etc -- presumably you've got more special-purpose templates that are built around a specific function (i.e. an "IPA" template, an "aeolus" template, etc) -- in this case, of course, you _do_ know which services you're going to be supporting.
In addition, for services that require packages that are not normally considered default OS packages, you're probably already choosing the service in the template explicitly, since that's the easiest way to get the required packages.
The big piece I don't have a good grasp on is which requirement is more important. Image proliferation looks like it costs money (thinking of EC2), since the user has to store all those images, probably in an S3 bucket, right? Whereas spin up time is operational cost to an IT center (i.e., the site is down, disaster recovery must kick in NOW). I guess the real answer is that this particular balance of requirements is going to change with each customer. And, we need to be able to support both models.
Scott