On 12/20/2011 10:22 AM, Steven Lawrance wrote:
Excerpts from Yaniv Kaul's message of Tue Dec 20 17:00:54 +1000
> On 12/19/2011 09:16 PM, Bill Peck wrote:
>> 3 - Provide images for operating systems that are difficult to
>> automate (Windows)
> Windows is not that difficult to provision automatically, from my
> experience. It's very configurable and scriptable. But you need the
> environment set up (Windows RIS services, local Windows Update server,
> etc.). I used to have that (all in VMs, btw!).
Our approach might depend on how much flexibility we want or require
with each of these other operating systems. Automatic provisioning of
Windows (without images) sounds easy enough, but we would need a
sensible way to model the requirements for a job (for all variants of
all the OSes we are going to support). If maintaining a handful of
standard images isn't going to generate too much overhead, it would make
this aspect simpler. (And there is the performance advantage, mentioned
>> thinking out loud here, here is how we currently process a recipe.
>> - A new recipe comes in that requires an x86_64 system with 4 gigs of
>> ram and 20 gig of disk space on Distro X.
> - We support 32bit as well, of course.
> - Q: do you actually need 20GB of disks? If not really, you can use thin
> provisioning, which will be quicker to provision. RAW would provide
> better performance, though.
If the recipe requires at least 20GB of disk, then yes, it should have a
device with at least that amount of disk available because there's no
telling what the job will do with it. Bill's example is for a job that
might be submitted with something like:
<memory op=">=" value="4000"/>
<arch op="=" value="x86_64"/>
<key_value key="DISKSPACE" op=">="
Currently, we just search for all systems that match those requirements.
But there's nothing to say this recipe couldn't run in a VM but we would
need to first check whether RHEVM can provision such a guest, and if so,
create a specific guest definition.
OK, so if DISKSPACE is mentioned, we should probably go with RAW
pre-allocated storage, otherwise with a standard 100GB or so
thin-provisioned disk, I guess.
>> Because we want to create the system dynamically we can't rely on sql
>> to alert us that a match is available. I'm concerned with
>> scalability, I don't think you remember this, but old rhts attempted
>> to match every queued recipe every time through the loop. It was
>> horrendous! The more queued recipes you had the longer it took to do
>> one loop. The sql method works wonderfully because we join on the
>> system being available, we never see recipes that can't be serviced.
>> So my concern is how we implement this with RHEVM without creating a
>> bottle neck?
> You CAN rely on a match 'partially' available - a template may already
> exist that matches your requirement. The 'partially' refers to the fact
> you may not be able to execute a snapshot of that template due to lack
> of resources.
So basically RHEVM will tell us "yes, this guest could be created, just
not right now"?
Knowing whether we can provision that VM immediately is important
because otherwise we can move onto the search for a "real" system that
matches the requirements, i.e. we wouldn't want a job held up because it
*could* be run via RHEV while there are idle baremetal boxes sitting in
the lab. But at the same time, we don't want to poll RHEVM continually
for all jobs sitting in the queue.
The only real way to know if a guest can run is actually execute it.
Could RHEVM notify Beaker of changes in available resources (such as
when a guest is destroyed or created)? Maybe Beaker itself could track
RHEV's available resources and just poll for changes at the start of
each loop over the queued recipes, rather than repeatedly asking RHEVM
whether it could provision each one?
All events can be registered to and sent via an email notification.
But I think we are optimizing for the worst case scenario, where we ran
out of resources.
I expect us to have enough resources to handle peak loads.
And btw, we are talking about tens to hundreds of concurrent instances,
> Alternatively, (and something we are looking for future versions to be
> built in to the product) - have X number of systems ready for
> For example, our internal continuous integration system has several VMs
> which are always ready for re-install if needed, used as RHEVM servers.
> There's a fixed number of them.
> For phase 1, I'd start with that, see how it goes. Then we can add
> dynamically provisioned systems.
I'm not sure I follow -- do you mean as target/test hosts? That's more
or less the approach we already take; we have quite a few "systems"
available to be scheduled in Beaker that are virt guests with fixed
memory and storage. So the whole goal here is dynamically provisioned
systems in that sense.
The 'dynamic' is in the sense that you just need to 'prep' them, not do
the whole allocation:
1. The VM is already created
2. Its disks are defined.
3. It has a network interface (may add more, but this brings more
4. You may change the number of vCPUs and memory
5. execute and prep it
6. Assign it to a user
So this is 'half dynamic'. A more dynamic would replace step 1 with:
1. Create a VM with a template
And completely loose would be:
1. Create a VM from a blank template
5. Provision an OS on that blank VM (installed via PXE/CDROM)
6. Assign to a user.
> Note that creating VMs should be merely a snapshot from an existing
> template and then 'prepping' the VM (sysprep in Windows, something
> similar in Linux).
> That should be quick - take ~5 minutes, tops. If you are installing from
> scratch it may take a bit more, similar to a physical system.
This speed-up is a good benefit, but I think first of all we really need
to be able to install from scratch just like we currently do, by
tweaking the kickstarts (and allowing users to submit custom kickstarts)
so that we can run more recipes as-is.
I'm not sure I agree. Certainly for Windows, where Sysprep is an
established process and is well integrated into RHEVM.
For Linux, see http://libguestfs.org/virt-sysprep.1.html
But we can start one way and convert to the other later on.
Beaker-devel mailing list