We've been talking about possibly changing the existing dynamic virtualization to work with OpenStack instead of oVirt for a while, and I've now hammered that into a hopefully coherent concrete proposal:
http://beaker-project.org/dev/proposals/dynamic-virtualization.html
The key points in there are too:
- associate lab controllers with OpenStack regions - map host filtering to flavour filtering - generate bootstrap images in Glance as part of distro import - use the bootstrap images for dynamic provisioning (rather than configuring netboot and then rebooting the system) - ensure console logging is hooked up properly
Provisioning based on specific images is explicitly deferred, since we'd have to map a bunch of the stuff we do in the kickstart post to cloud-init, and I'd prefer to put that off to a later iteration. By contrast, I think everything above is essential for dynamic virtualisation to be as good as what people are used to with bare metal provisioning (the lack of console logging with no obvious way to add it was one of the final nails in the coffin of the previous oVirt based approach).
Cheers, Nick.
On 12/18/2013 06:23 PM, Nick Coghlan wrote:
We've been talking about possibly changing the existing dynamic virtualization to work with OpenStack instead of oVirt for a while, and I've now hammered that into a hopefully coherent concrete proposal:
http://beaker-project.org/dev/proposals/dynamic-virtualization.html
The key points in there are too:
- associate lab controllers with OpenStack regions
- map host filtering to flavour filtering
- generate bootstrap images in Glance as part of distro import
- use the bootstrap images for dynamic provisioning (rather than
configuring netboot and then rebooting the system)
- ensure console logging is hooked up properly
Gah, missed one: we also need to decide on what we want to do in terms of network configuration. I did think of this (that's why the note is there about only supporting Havana and later), I just forgot to include the relevant details in the first draft.
Anyway, because of the way OpenStack works, Beaker is going to have to take care of properly associating machines with IP addresses, and making sure they can see the rest of the Beaker lab. We can't just declare it a "lab management DHCP configuration problem" the way we do for bare metal systems :)
One key requirement is that the systems provisioned in OpenStack still need to be able to connect to the lab controller, so all provisioned systems will need to be connected back to the lab network, even if all the recipes in the recipe set are running in OpenStack.
To delay the point where we start dynamically creating and destroying network definitions, I suggest we just run with a really simple model for the initial iteration: as part of the per-lab-controller configuration, we say which network to use. Then we can just configure the network and its routing appropriately in OpenStack, and just deal with handing out IP addresses appropriately when creating the VMs.
At some point in the future we may explore isolated subnets per test, but there are limits to the effectiveness of that since the systems will always need to be able to access at least the lab controller, even if we block them off from other parts of the lab.
Cheers, Nick.
On 12/18/2013 06:23 AM, Nick Coghlan wrote:
We've been talking about possibly changing the existing dynamic virtualization to work with OpenStack instead of oVirt for a while, and I've now hammered that into a hopefully coherent concrete proposal:
http://beaker-project.org/dev/proposals/dynamic-virtualization.html
I did read the proposal, it is good and clear.
The key points in there are too:
- associate lab controllers with OpenStack regions
- map host filtering to flavour filtering
- generate bootstrap images in Glance as part of distro import
- use the bootstrap images for dynamic provisioning (rather than
configuring netboot and then rebooting the system)
- ensure console logging is hooked up properly
Provisioning based on specific images is explicitly deferred, since we'd have to map a bunch of the stuff we do in the kickstart post to cloud-init, and I'd prefer to put that off to a later iteration. By
Is mapping the post install stage to cloud-init significantly harder than providing the boostrap images that launch anaconda?
contrast, I think everything above is essential for dynamic virtualisation to be as good as what people are used to with bare metal provisioning (the lack of console logging with no obvious way to add it was one of the final nails in the coffin of the previous oVirt based approach).
Cheers, Nick.
On 12/19/2013 03:27 AM, Lucas Meneghel Rodrigues wrote:
On 12/18/2013 06:23 AM, Nick Coghlan wrote:
We've been talking about possibly changing the existing dynamic virtualization to work with OpenStack instead of oVirt for a while, and I've now hammered that into a hopefully coherent concrete proposal:
http://beaker-project.org/dev/proposals/dynamic-virtualization.html
I did read the proposal, it is good and clear.
Thanks!
The key points in there are too:
- associate lab controllers with OpenStack regions
- map host filtering to flavour filtering
- generate bootstrap images in Glance as part of distro import
- use the bootstrap images for dynamic provisioning (rather than
configuring netboot and then rebooting the system)
- ensure console logging is hooked up properly
Provisioning based on specific images is explicitly deferred, since we'd have to map a bunch of the stuff we do in the kickstart post to cloud-init, and I'd prefer to put that off to a later iteration. By
Is mapping the post install stage to cloud-init significantly harder than providing the boostrap images that launch anaconda?
Yes, for a few different reasons. Firstly, we use some anaconda features to configure yum repos and install packages on the target system, and would have to come up with new ways of doing those in a cloud-init based solution.
Secondly, by building on https://github.com/redhat-openstack/image-building-poc (which is what I'm suggesting we do in order to create these initial bootstrapping images), we can eventually start creating the appropriate full system images in Glance when a distro is imported, rather than relying on them being created externally and just providing a way to pass the image name into Beaker. (While requesting specific images is something we think we should support, that's separate from transparently using OpenStack whenever feasible).
Finally, we currently support a variety of kickstart specific customisation options through ks_meta, ks_appends and custom kickstart templates. By bootstrapping Anaconda rather than a full system image, those will just work, whereas if we try to jump straight to image based provisioning, there will be quite a few existing recipes that will still need to use bare metal systems because they need the customisation options.
By doing the bootstrapping option first, when we later add support for image based provisioning, we simply won't support those kickstart specific customisation options, but will instead add new ones that are compatible with both kickstart and cloud-init.
I'll add some more of this rationale to the proposal today, since it's a good question.
Cheers, Nick.
On 12/19/2013 12:05 PM, Nick Coghlan wrote:
I'll add some more of this rationale to the proposal today, since it's a good question.
Updated proposal is up at: http://beaker-project.org/dev/proposals/dynamic-virtualization.html
It has the changes discussed in this thread (simple networking config, more discussion of why we're going with bootstrapping Anaconda for the initial iteration), and also a couple of other significant changes:
- the initial iteration won't touch Cinder at all. Instead, we'll run with just the ephemeral disk space provided as part of the chosen VM flavour. Using <disk/> criteria in your host filtering will disqualify the recipe from dynamic virtualisation for the time being.
- Dan pointed me towards iPXE (http://ipxe.org/) which we can configure to retrieve the boot image over HTTP. This means we can just define *one* standard iPXE image, that we can then use to boot any distro tree. This means we won't need to mess about creating images and uploading them to Glance until we do image based provisioning.
Cheers, Nick.
beaker-devel@lists.fedorahosted.org