Hi Scott,
On Mon, 2011-05-23 at 01:02 -0400, Scott Seago wrote:
On 05/20/2011 04:21 PM, Mark McLoughlin wrote:
> Finally, FWIW, I'm pretty excited that this will get us out of some of
> the ratholes we've been in lately.
>
> Cheers,
> Mark.
>
> Changes From Before
> ===================
>
> - There are two XML document formats - deployables and templates - and
> they are quite independent
>
> - Deployables (or rather, assemblies) contain image references rather
> than template references
>
> - An image reference maps to multiple versions of the image, where
> each version has multiple provider images
>
> - Images can be built from a template, or imported from elsewhere
>
Regarding the above three bullet points, I spoke with Hugh at length
about the model change here on Friday, and I should note that this is
both a model change _and_ a terminology change.
Yes, absolutely
I'm fine with the
change, but I'd like to describe my understanding of both the old and
new models in more detail here to make sure that we're all clear on
what's happening. In addition, I think the above description misses one
level in the hierarchy, but it is one which is largely an implementation
detail (although a fairly important one)
First, the old (current Conductor codebase) model:
- A Template is a target-and-provider-independent way of referencing
a Provider Image (i.e. the 'template' is what an end user selects in the
UI that maps -- once Condor does its thing -- to a particular Image on a
particular provider.) In addition, a Template includes the XML
description which tells Image Factory how to build Images for the
various back end providers. Note that in the current model the Template
does double duty here -- it's the specification as to what to build
_and_ the object which references a series of _equivalent_ provider
images on various back ends. In the new model these two concepts get
split apart
Right
- An Image is the output of the 'build' API call on Image
Factory
which is the result of building a disk image for a single target which
is not yet associated with a particular provider. In other words, an ec2
image is the root FS image in the proper format for ec2 which has not
yet been bundled or registered as an AMI in a particular region with a
particular owner. For RHEV, this would be the KVM image which has not
yet been uploaded/enabled for a particular RHEV server. From an
implementation POV, this is an important intermediate step (which I
believe is missing in Mark's writeup), as we don't need to generate a
new from-scratch build for each ec2 region or for each RHEV server we
want to upload to
- A Provider Image is an image uploaded (bundled/registered/etc) to a
particular provider (ec2-us-east,
my-rhev-instance1.redhat.com, etc)
Right, I think some notion of Image vs Provider Image is there in the my
writeup as the target-and-provider-agnostic disk image associated with
an image version
From what you say, though, it sounds like we don't have the notion
of a
target-and-provider-agnostic disk image, only a provider-agnostic disk
image. Hmm
So there are 2 main changes here in the new schema. First of all, the
Template concept is split into two separate (but related) resources. The
Template describes the specification for the series of images that will
be built from it (i.e. the input to Image Factory), and the Image
describes a particular build in a target-independent way -- i.e. an
Image reference maps to a series of related builds, for various targets
that are to be considered equivalent (i.e. built at roughly the same
time for ec2-us-east, ec2-us-west, various private cloud instances,
etc.). Adding versioning to this results in, for a given Template, a
version-independent Image reference (perhaps just a name) and a series
of Image Versions (i.e. builds), each of which points to one or more
provider images (via the yet-unnamed intermediary that we used to call
Image)
Yes. And note that not all Images are associated with an origin Template
- i.e. images that are imported are represented by an Image with no
associated Template
The second change is one of nomenclature. Now that the term
"Image" is
used to describe a target-independent set of equivalent images -- the
thing referenced by assemblies -- rather than "Template", we need a new
name for what was "Image" -- i.e. a build for a particular
target/provider type but not associated with a specific provider. This
is something that will be used primarily for the Image Factory API (i.e.
the output from the 'build' action and the input to the 'push' action.
Assemblies and Deployables won't need to worry about it -- they specify
an image, and condor will map this to a Provider Image. For the moment
I'm calling this a TargetImage -- it's a valid name, I'm open to
suggestions for a better name.
Okay, makes sense.
I think I'd like to try something like this:
- A Provider Image has a pair of metadata tags - (Target, Provider) -
and the Provider part of the pair is optional
That way, we get the concept of a target specific image but not
associated with a particular provider
Or, to use a concrete example, consider the following collection of
objects:
JBossAS-template (Template XML doc -- belongs_to image)
JBossAS (image) (has_one Template (optional) )
JBossAS 1.0 (Image version)
JBossAS 1.0-ec2 (Target Image)
JBossAS-ec2-us-east (Provider Image)
JBossAS-ec2-us-west (Provider Image)
JBossAS-rhev (Target Image)
JBossAS uploaded to
rhev1.redhat.com (Provider Image)
JBossAS uploaded to
rhev2.redhat.com (Provider Image)
Yep, that sounds right
or to define rails-y associations (this isn't an implementation
suggestion, as for the moment _none_ of these objects are proposed to
belong in the conductor DB, but I just wanted to show the relationships
in another way to make sure it's clear):
class Template
belongs_to :image
end
class Image
has_one :template
has_many :image_versions
end
class ImageVersion
belongs_to :image
has_many :target_images
end
class TargetImage
belongs_to :image_version
has_many :provider_images
end
class ProviderImage
belongs_to :target_image
end
Yep, that's it.
Also, I think a Template can be built multiple times, perhaps by
different users, to give different Images. In the new model, a template
is just referenced by a URL and supplied to image factory, and there's
no reason why the same URL couldn't be used to build different Images,
perhaps with a different set of package updates
Given that and my other suggestion above (i.e. collapse TargetImage into
ProviderImage) gives:
class Template
has_many :image
end
class Image
belongs_to :template
has_many :image_versions
end
class ImageVersion
belongs_to :image
has_many :provider_images
end
class ProviderImage
belongs_to :image
belongs_to :provider_type
belongs_to :provider
end
> - Services are at the deployable level, not the template
level
>
Does this mean we won't be embedding the service scripts (puppet, etc)
into the template -- only the packages required by the service? If the
latter, will they be specified manually?
Right, service scripts won't be included in the template
Services may install packages. If the deployable author wishes to avoid
that, she needs to ensure the template has the required packages
In future, we may try to have an administrator level policy to prevent
services installing updates, but that's hand-waving territory for the
moment
From the discussions during the week, it turns out that the constraint
around preventing services from installing deployables isn't important.
The most we need do is initially is to provide template authors with
tools to make it easy to ensure that the packages required for services
are installed
> - Templates expose parameters, but they're not described
as services
>
How are we exposing parameters without defining the scripts which use
these parameters? Or are we separating the scripts from the notion of
services?
Templates may have scripts (or perhaps only a single script) with
parameters, but they aren't services
> - Use of ruby string interpolation as a more flexible and
expressive way
> to manipulate parameters
>
> Parameters and Returns
> ======================
>
> At each level of the model, the concept of parameters and returns are
> used.
>
> A parameter is an input and a return is an output. Parameters are
> typed, returns are strings. Both have names and human consumable
> descriptions.
>
> When declaring a parameter that is made available by a level of the
> model to the level above it, you do e.g.
>
> <param type="string">
> <name>admin_user</name>
> <description>Administrator's username</description>
> <value>admin</value>
> </param>
>
> A default value may be supplied, essentially making the parameter
> optional.
>
> When supplying a parameter from a level of the model to the level
> below it you do e.g.
>
> <param>
> <name>admin_user</name>
> <value>admin</value>
> </param>
>
> i.e. the type attribute and description element is irrelevant.
>
> When declaring a return that is supplied from a level of the model to
> the level above it, you do e.g.
>
> <return>
> <name>http_port</name>
> <description>HTTP port number</description>
> <value>80</value>
> </return>
>
> A<value> element of a parameter or return may use Ruby's string
> interpolation to reference other parameter or return values e.g.
>
> <return>
> <description>URL of the Redmine project management
system</description>
>
<value>http://#{frontend.ip_address}:#{frontend.http_port}/</value>
> </return>
>
> or:
>
> <params>
> <param>
> <name>url</name>
> <value>http://#{hostname}:#{http_port + 100}/redmine/</value>
> </param>
> </params>
>
> Services
> ========
>
> Services encapsulate a configuration script or recipe for a configuration
> management tool.
>
> So, for example, a script referenced by a URI:
>
> <service>
> <script href="...."/>
> <params/>
> <returns/>
> <script>
> </service>
>
> or a script embedded in the document:
>
> <service>
> <script>
> <contents><![CDATA[
> #!/bin/bash
> ...
> ]]</contents>
> <params/>
> <returns/>
> </script>
> <service>
>
> Parameters are passed to the script on stdin using a<params/> element
> and the script writes a<returns/> element to stdout.
>
> Puppet manifests and chef recipes are handled similarly using<puppet/>
> and<chef/> elements with appropriate ways for params and returns to be
> exchanged.
>
Just to clarify, as I'm a bit confused here. Above you said that we
define services only in assemblies/deployables (not templates), but that
templates will describe parameters. Are these params without scripts, or
something else?
They are scripts (or it is a script) which takes parameters but we're
not describing as a service
Also, is the intent to only load the puppet/chef/etc scripts on
firstboot (not in the image itself)?
Yep
> Images
> ======
>
> The term "image" refers to an abstract concept here, as opposed to a
binary
> disk image.
>
> An image is created by importing as a disk image from elsewhere or by building
> a template.
>
> An image has multiple versions. Each version has a set of provider images and
> (optionally) a provider agnostic disk image.
>
OK, I think the provider-agnostic disk image is what I'm calling a
TargetImage above -- if by this you mean an image that's ec2-specific
but not registered/bundled as an AMI in a particular region.
No, I was thinking this provider-agnostic disk image was
provider-type-agnostic :)
I'm pretty convinced now just by what you say that we don't have such a
concept
I think we may be making this more complex if we make that
intermediate
stage optional -- i.e. it may be simpler to always have this
image,although there may be provider types (presumably mostly non-ec2
public providers) for which the TargetImage is mainly a placeholder. In
any case,this is an impl detail that we can figure out the right answer
for as we get there, as I don't think it will have any significant API
implications (except possibly for Image Factory).
Unless you're referring to the provider-and-target-agnostic image (i.e.
could point to _any_ provider, not just providers of a specific type) --
in which case the above comments about TargetImage being missing in this
description still apply.
Yep, it was the latter and you're right
Either of your Image/ImageVersion/ProviderImage/TargetImage or my
Image/ProviderImage schemes makes sense to me
> The concept exists to allow deployables to refer to disk images
without
> referencing a template, a specific template build or provider image.
>
In fact, deployables will never reference a provider image directly
(unless we're changing this now too) -- the deployable should always
reference Images in a provider-agnostic way.
I think deployables should be able to reference the Image but also
optionally reference the ImageVersion
i.e. a deployable author can choose whether to use the latest version
available or use a specific version
> Assemblies
> ==========
>
> Assemblies encapsulate an image reference and a set of services.
>
> An image is referenced by ID. e.g.
>
> <assembly>
> <name>frontend</name>
> <image id="11abf870-894c-4336-bc9f-37904c394924">
> ...
> </image>
> </assemly>
>
> Optionally, the image reference can include a version id:
>
> <assembly>
> <name>frontend</name>
> <image id="11abf870-894c-4336-bc9f-37904c394924"
version="be2a8e6">
> ...
> </image>
> </assemly>
>
> This ID and optional version can be resolved by conductor, via metadata
> and tags IWHD, to the set of provider image builds associated with the
> image.
>
> The image reference also describes the parameters to pass to instance
> when it is started:
>
> <assembly>
> <name>frontend</name>
> <image id="11abf870-894c-4336-bc9f-37904c394924">
> ...
> <params>
> <param>
> <name>http_port</name>
> <value>#{http_port}</value>
> </param>
> ...
> </params>
> </image>
> <assembly>
>
> When an instance is launched, it can return information and these are
> also described in the image reference:
>
> <assembly>
> <name>frontend</name>
> <image id="11abf870-894c-4336-bc9f-37904c394924">
> ...
> <returns>
> <return>
> <name>http_port</name>
> </return>
> </returns>
> </image>
> <assembly>
>
> Like all other objects, assemblies have returns. However, they also
> implicitly have some returns which do not need to be explicitly
> defined, like ip_address.
>
> Finally, Assemblies include a list of services:
>
> <assembly>
> ...
> <services>
> <service/>
> <service/>
> ...
> </services>
> </assemblies>
>
> == Deployables ==
>
> A deployable encapsulates all the information required to launch a set
> of cooperating instances. It contains:
>
> - Its name and description:
>
> <deployable>
> <name>Redmine</name>
> <description>Redmine is a web-based project management
application</description>
> ...
> </deployable>
>
> - Some parameters that the user launching the deployable may
> specify:
>
> <deployable>
> ...
> <params>
> <param type="string">
> <name>admin_user</name>
> <description>Administrator's username</description>
> <value>admin</value>
> </param>
> <param type="string">
> <name>admin_passwd</name>
> <description>Administrator's password</description>
> <secret>true</secret>
> </param>
> <param type="int">
> <name>http_port</name>
> <description>HTTP port number</description>
> <value>80</value>
> </param>
> <param type="string">
> <name>timezone</name>
> <description>Timezone in which to run the application>
> <values>
> <value>America/New_York</value>
> <value>Europe/Paris</value>
> </value>
> </param>
> ...
> </deployable>
>
> When a user launches a deployable, a form is displayed allowing
> the user to enter values for the parameters. The form is
> pre-populated with default values. Where multiple default values
> are specified, a drop-down box is used to allow the user choose
> one of the values.
>
> - Assemblies, which may be included inline in the deployable:
>
> <deployable>
> ...
> <assemblies>
> <assembly>
> <name>frontend</name>
> <image id="11abf870-894c-4336-bc9f-37904c394924">
> <params>
> <param>
> <name>redmine_admin</name>
> <value>#{admin_user}</value>
> </param>
> <param>
> <name>redmine_passwd</name>
> <value>#{admin_passwd}</value>
> </param>
> </params>
> </image>
> <assembly>
> </assemblies>
> ...
> </deployable>
>
> Notice that because this assembly is inline, it can directly
> reference the deployables parameter values.
>
> Assemblies can also be included by reference:
>
> <deployable>
> ...
> <assemblies>
> <assembly href="http://myserver/redmine/frontend.assy">
> <params>
> <param>
> <name>admin_user</name>
> <value>#{admin_user}</value>
> </param>
> <param>
> <name>admin_passwd</name>
> <value>#{admin_passwd}</value>
> </param>
> </params>
> </assembly>
> </assemblies>
> </deployable>
>
> Here, because the assembly is included by reference, the
> deployable must explicitly pass the parameters required by the
> assembly. In this example, it has two parameters and we pass the
> values of the equivalent deployable parameters straight
> through.
>
> It is common to use the return values of an assembly as the
> paramaters to another assembly e.g.
>
> <assembly href="http://myserver/redmine/caching-proxy.assy">
> <param>
> <name>base_url</name>
>
<value>http://#{frontend.ip_address}:#{frontend.http_port}/</value>
> </param>
> </assembly>
>
> - Return values which may be displayed to the user e.g.
>
> <deployable>
> ...
> <returns>
> <return>
> <description>URL of the Redmine project management
system</description>
>
<value>http://#{frontend.ip_address}:#{frontend.http_port}/</value>
> </return>
> </returns>
> </deployable>
>
> In summary, a deployable has a description, a set of typed parameters
> with defaults, a set of (inline or included by reference) assemblies
> and their parameter values and a set of user consumable return
> values.
>
> In conductor, a user launches a deployable simply by supplying a URI
> to the deployable definition, or by selecting from a list of
> deployable URIs supplied by the administrator.
>
> Templates
> =========
>
> Templates are an XML document which describe how to build an image and
> what the parameters and returns of the image will be. For example:
>
> <template>
> <name>win2kjeos</name>
> <os>
> <name>Windows</name>
> <version>2008</version>
> <arch>x86_64</arch>
> <install type='iso'>
> <iso>http://directory_path/windows2008x64.iso</iso>
> </install>
> <key></key>
> </os>
> <description>Windows 2008</description>
> <repos>
> <repo url="smb://domain\user:UserName@IP\share"
name="Default"/>
> <repo url="smb://domain\user:UserName@IP\another_share"
name="Alternative"/>
> </repos>
> <packages>
> <package>
> <repo name="Default"/>
> <name>Dot Net 4.0</name>
> <file>dotNetFx40_Full_x86_x64.exe</file>
> <arguments>/passive</arguments>
> </package>
> <package>
> <repo name="Alternative"/>
> <name>Winrar</name>
> <file>winrarx64393.exe</file>
> <arguments>/s</arguments>
> </package>
> </packages>
> </template>
>
> (Yes, that's copied and pasted from imgfac(1) man page :-)
>
> Image Warehouse Metadata
> ========================
>
> Image warehouse will contain enough metadata for conductor to resolve
> an image reference to a set of provider images.
>
> One possible way of storing that metadata would be as follows:
>
> - An image descriptor XML document describing the image is stored as
> an object. The UUID of the object is the image ID.
>
> - Each provider agnostic disk image is also an object and represents
> a build of a template. So an image version/build is identified by
> this object's UUID. The other attributes of this object include the
> image UUID and UUID of its parent version, if any.
>
One question though -- "provider agnostic disk image" is what we've been
calling "Image" throught here, right? Or is there something else I've
missed here?
In your scheme, this would be ImageVersion. I realize now that there is
no disk image associated with this concept
And we'd insert target image here once we all agree 1) that we
need it
and 2) what it's called
> - Each provider image has the disk image UUID as an attribute.
>
> A special case we need to think about is where an image is imported
> from e.g. EC2. In that case, the provider agnostic disk image doesn't
> exist, so perhaps we'd create an empty object with the metadata we
> need.
>
Yes -- I think we'd represent everything but the Template at least as
placeholders. So we'll have an Image object (for inclusion in
assemblies), and a single Provider Image (the one we imported) -- but no
Template (since we didn't build it).
These aren't useless placholders either -- for example,what if I have
some completely separate way of getting images built (say pulled
directly from a vendor,etc) and I know that I have an ec2-us-east AMI,
an ec2-us-west AMI, and a RHEV image, all of which are equivalent -- I'd
want to import all 3 of them into IWHD but all pointing to the same
provider-agnostic disk image -- so any assemblies that use this image
can be instantiated in all 3 back ends.
Exactly, yes!
> In order to find all the provider images associated with an
image, you
> find all the disk images referencing the image UUID and find all the
> provider images referencing all the disk image UUIDs.
>
OK, I'm starting to think we're proposing the same thing here with
different words again. Image, Disk Image, Provider Image -- are we
saying Image can reference all providers, all types (ec2, RHEV,
etc),disk image applies to multiple providers but to one target type
(ec2, multiple regions _or_ RHEV, multiple servers), and of course
Provider Image is a particular image in a particular provider (i.e. a
single AMI, an image in one RHEV server, etc.) -- or is there a
different distinction between "image" and "disk image" ?
Hopefully it's clear at this point that I was just confused in thinking
that there was a provider-and-target-agnostic disk image somewhere
If that doesn't exist, it is replaced by the metadata-only object called
ImageVersion in your scheme
Thanks for the feedback Scott. To be crystal clear, here's what I think
needs to be fixed in the writeup:
- Kill off the DiskImage concept and replace with ImageVersion
- Either add TargetImage, or make the provider optional in
ProviderImage's (target, provider) pair
- Clarify that service scripts aren't included in templates, but that
templates can include a script which takes parameters
This morning, I'm going to look in detail at how this all maps to image
warehouse metadata and the queries conductor will need etc. Hopefully
that level of detail will help us be more confident that this all works
Cheers,
Mark.