Nick, would it be possible to support some more general API to these "cloudy"
things? So
we could connect beaker to openstack, or ec2, or... anything else :). Deltacloud for
example, if we don't have anything better (I'm not tracking it that much).
Cheers,
J.
On 06/02/2014 04:24 AM, Nick Coghlan wrote:
Dan has been working on implementing the OpenStack design proposal,
and
the preliminary experimental version of that will be available in Beaker
0.17. This initial version allows individual users to supply appropriate
credentials to log into OpenStack on their behalf, and then tried to run
single-host recipe sets submitted by that user on OpenStack rather than
bare metal whenever possible. The scheduler will choose an appropriate
VM flavour (based on hostRequires), dynamically install a distro via
iPXE and then run the recipe on the fresh instance. While the use of
iPXE means this still incurs the Anaconda install time, it will still be
faster than most bare metal systems (since there is no server POST to
run through) and it also means that all the usual kickstart based tweaks
still work.
The current plan is to get this iPXE based version working cleanly to
thrash out most of the Beaker/OpenStack integration issues, and only
then start looking at image based provisioning. Once we have image based
provisioning working in OpenStack, we can then looking at supporting it
natively in Beaker. This approach is to keep the number of unknowns in
play at any given point in time at least somewhat manageable. While it's
going to take a while to get there, the end result should provide a nice
flexible system where simple requests without stringent hardware
requirements can be offloaded to generic compute resources in OpenStack,
while more exotic hardware requests will stay under the management of
Beaker's relatively fine-grained inventory system.
As we work through the initial integration, I'm expecting three possible
outcomes for various things:
- "still needs work" on the Beaker side
- "configuration constraints" in terms of consequences for particular
design decisions for an associated OpenStack instance
- "OpenStack RFEs" for things that currently seem to be inherent
limitations of OpenStack
At this point, we have the following items in each category:
Still needs work on the Beaker side:
* console log retrieval (although this will be somewhat clunky for the
time being - see the RFE section)
* using the trust delegation model created for Heat rather than storing
raw credentials
* improving the scalability of the OpenStack integration to better
handle simultaneous scheduling of at least dozens (preferably hundreds)
of recipes without stalling the whole scheduling system
* expanding to support multi-host recipes
Configuration constraints:
* Short version: use routable IPs for an OpenStack instance paired with
Beaker rather than relying on the NAT system
* Long version:
* if you don't use routable IPs, you won't get usable SSH access to
dynamic VMs without setting up some kind of SSH proxy infrastructure (we
tried & failed to get this working with floating IPs - that doesn't
work, because the VM doesn't know its own external IP address)
* if you don't use routable IPs, you won't be able to mix-and-match
Beaker systems and OpenStack VMs in multi-host recipes (once we
implement that)
* if you don't use routable IPs, you won't get usable DNS names for
the systems
OpenStack RFEs:
* the current console log access APIs don't support console log
streaming properly - we'll file some specific RFEs once we get a basic
version based on the existing APIs working
* it currently looks like the setup process for trust delegation will be
quite messy from a user experience perspective - we may file some RFEs
about that, depending on how it turns out
* OpenStack Designate ("DNS-as-a-Service") will hopefully address some
of the limitations of non-routable IPs, but in the meantime, we'll just
be saying "if you use non-routable IPs, here's a bunch of things that
won't work for dynamic VMs"
Cheers,
Nick.