BZ 716286
by Jan Provazník
Patchset moves 'match' method into instance model. With this user gets more
details why instance launch failed.
This patch depends on "BZ 716294 - Fixed quota checking"
12 years, 10 months
[PATCH conductor] BZ 716294 - Fixed quota checking
by Jan Provazník
From: Jan Provaznik <jprovazn(a)redhat.com>
Most important change is that quota.running_instances now includes 'pending' and
'shutting_down' instances too. Till now only 'running' instances were counted.
Also an instance is set as 'pending' once it's passed to condor in
condormatic_instance_create. Above changes are important because of
(mostly) removing atomic problem with quota check: running instances quota is
checked before passing job to condor, but instances running state is set in
dbomatic (after instance is really running) -> in this time quota check
didn't work properly.
---
src/app/models/instance_observer.rb | 21 ++++++++++-----------
src/app/util/condormatic.rb | 3 ++-
src/spec/models/instance_observer_spec.rb | 20 +++++++++++++++++++-
3 files changed, 31 insertions(+), 13 deletions(-)
diff --git a/src/app/models/instance_observer.rb b/src/app/models/instance_observer.rb
index eedf813..800da10 100644
--- a/src/app/models/instance_observer.rb
+++ b/src/app/models/instance_observer.rb
@@ -1,6 +1,7 @@
class InstanceObserver < ActiveRecord::Observer
ACTIVE_STATES = [ Instance::STATE_PENDING, Instance::STATE_RUNNING, Instance::STATE_SHUTTING_DOWN, Instance::STATE_STOPPED ]
+ RUNNING_STATES = [ Instance::STATE_PENDING, Instance::STATE_RUNNING, Instance::STATE_SHUTTING_DOWN ]
def before_save(an_instance)
if an_instance.changed?
@@ -42,18 +43,16 @@ class InstanceObserver < ActiveRecord::Observer
if parent
quota = parent.quota
if quota
- if state_to == Instance::STATE_RUNNING
- quota.running_instances += 1
- elsif state_from == Instance::STATE_RUNNING
- quota.running_instances -= 1
- end
+ if !RUNNING_STATES.include?(state_from) && RUNNING_STATES.include?(state_to)
+ quota.running_instances += 1
+ elsif RUNNING_STATES.include?(state_from) && !RUNNING_STATES.include?(state_to)
+ quota.running_instances -= 1
+ end
- if state_from != nil
- if !ACTIVE_STATES.include?(state_from) && ACTIVE_STATES.include?(state_to)
- quota.total_instances += 1
- elsif ACTIVE_STATES.include?(state_from) && !ACTIVE_STATES.include?(state_to)
- quota.total_instances -= 1
- end
+ if !ACTIVE_STATES.include?(state_from) && ACTIVE_STATES.include?(state_to)
+ quota.total_instances += 1
+ elsif ACTIVE_STATES.include?(state_from) && !ACTIVE_STATES.include?(state_to)
+ quota.total_instances -= 1
end
quota.save!
end
diff --git a/src/app/util/condormatic.rb b/src/app/util/condormatic.rb
index 107ad6e..4b42dee 100644
--- a/src/app/util/condormatic.rb
+++ b/src/app/util/condormatic.rb
@@ -67,7 +67,7 @@ def match(instance)
# FIXME: we should have something smarter here that prioritizes
# and/or chooses the "cheapest" possibility. For now, just return the
# first that fits under quota
- if Quota.can_create_instance?(instance, match.account)
+ if Quota.can_start_instance?(instance, match.account)
return match
end
end
@@ -167,6 +167,7 @@ def condormatic_instance_create(task)
instance.state = Instance::STATE_CREATE_FAILED
else
task.state = Task::STATE_PENDING
+ instance.state = Instance::STATE_PENDING
end
instance.save!
task.save!
diff --git a/src/spec/models/instance_observer_spec.rb b/src/spec/models/instance_observer_spec.rb
index 767712b..69d702c 100644
--- a/src/spec/models/instance_observer_spec.rb
+++ b/src/spec/models/instance_observer_spec.rb
@@ -139,7 +139,10 @@ describe InstanceObserver do
end
end
- it "should update a pool, cloud account and user quota when an instance state goes from running to another active state" do
+ it "should not update pool, cloud account and user quota when an instance state goes from pending to running to shutting down" do
+ @instance.state = Instance::STATE_PENDING
+ @instance.save!
+
@instance.state = Instance::STATE_RUNNING
@instance.save!
@@ -148,6 +151,21 @@ describe InstanceObserver do
[@provider_account_quota, @pool_quota, @user_quota].each do |quota|
quota = Quota.find(quota.id)
+ quota.running_instances.should == 1
+ quota.total_instances.should == 1
+ end
+ end
+
+
+ it "should update a pool, cloud account and user quota when an instance state goes from running to stopped state" do
+ @instance.state = Instance::STATE_RUNNING
+ @instance.save!
+
+ @instance.state = Instance::STATE_STOPPED
+ @instance.save!
+
+ [@provider_account_quota, @pool_quota, @user_quota].each do |quota|
+ quota = Quota.find(quota.id)
quota.running_instances.should == 0
quota.total_instances.should == 1
--
1.7.4
12 years, 10 months
Image Building UI
by Chris Lalancette
All,
I hate to bring this topic back up, but I thought I would share some
feedback I have received about the new UI.
Yesterday I gave some training to a bunch of people about Aeolus, and
the feedback I received was that having a command-line-only tool to build
images is a pretty jarring experience. That is, the lack of an image building
UI in the web UI is seen as a large omission. This isn't the first time I've
heard this particular feedback, either, but it was a significant enough
surprise to the audience that I thought I would share it.
I totally understand the reasons that we removed the image building UI,
and I also think that the command-line image building tool is useful,
particularly for complex images. However, I think a simple UI for image
building like we had before might go a long way to quelling some of these
concerns.
--
Chris Lalancette
12 years, 10 months
[PATCH 1/6] Remove trailing whitespace.
by Chris Lalancette
Signed-off-by: Chris Lalancette <clalance(a)redhat.com>
---
VMDKstream.py | 42 +++++++++++++++++++++---------------------
1 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/VMDKstream.py b/VMDKstream.py
index 0bf26d1..f588e0a 100755
--- a/VMDKstream.py
+++ b/VMDKstream.py
@@ -2,16 +2,16 @@
# encoding: utf-8
# Copyright (C) 2011 Red Hat, Inc.
-#
+#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
-#
+#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
-#
+#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston,
@@ -19,7 +19,7 @@
# also available at http://www.gnu.org/copyleft/gpl.html.
#
# First cut module to convert raw disk images into stream-optimized VMDK files
-#
+#
# See the "Specification Document" referenced in Wikipedia for more details:
#
# http://en.wikipedia.org/wiki/VMDK
@@ -56,17 +56,17 @@ SECTOR_SIZE = 512
# Descriptor Template
image_descriptor_template='''# Description file created by VMDK stream converter
-version=1
+version=1
# Believe this is random
-CID=7e5b80a7
+CID=7e5b80a7
# Indicates no parent
-parentCID=ffffffff
-createType="streamOptimized"
+parentCID=ffffffff
+createType="streamOptimized"
# Extent description
RDONLY #SECTORS# SPARSE "call-me-stream.vmdk"
-# The Disk Data Base
+# The Disk Data Base
#DDB
ddb.adapterType = "lsilogic"
@@ -80,7 +80,7 @@ ddb.virtualHWVersion = "7"'''
def create_sparse_header(inFileSectors = None, descriptorOffset = 1 ,
- descriptorSize = None, gdOffset = 0xFFFFFFFFFFFFFFFF ):
+ descriptorSize = None, gdOffset = 0xFFFFFFFFFFFFFFFF ):
# While theoretically variable we set these based on current VMWare convention
grainSize = 128
numGTEsPerGT = 512
@@ -90,21 +90,21 @@ def create_sparse_header(inFileSectors = None, descriptorOffset = 1 ,
# The following are always fixed in the "stream-optimized" format we are creating
compressAlgorithm = 1
flags = 0x30001
- rgdOffset = 0
+ rgdOffset = 0
# We are building from scratch so an unclean shutdown is not possible
uncleanShutdown = 0
# Build the struct
header_list = [ MAGIC_NUMBER, formatVersion, flags, inFileSectors, grainSize, descriptorOffset, descriptorSize,
- numGTEsPerGT, rgdOffset, gdOffset, overHead, uncleanShutdown, '\n', ' ', '\r', '\n',
+ numGTEsPerGT, rgdOffset, gdOffset, overHead, uncleanShutdown, '\n', ' ', '\r', '\n',
compressAlgorithm ]
for i in range(433):
header_list.append(0)
header_struct = "=IIIQQQQIQQQBccccH433B"
- return struct.pack(header_struct, *header_list)
+ return struct.pack(header_struct, *header_list)
-def create_marker(numSectors = None, size = None, marker_type = None):
+def create_marker(numSectors = None, size = None, marker_type = None):
marker_list = [ numSectors, size, marker_type ]
for i in range(496):
marker_list.append(0)
@@ -113,8 +113,8 @@ def create_marker(numSectors = None, size = None, marker_type = None):
def create_grain_marker(location = None, size = None):
# The grain marker is special in that the data follows immediately after it
- # without a pad
- grain_marker_struct = "=QI"
+ # without a pad
+ grain_marker_struct = "=QI"
return struct.pack(grain_marker_struct, location, size)
def divro(num, den):
@@ -141,7 +141,7 @@ def sector_pointer(file_object):
if file_location % SECTOR_SIZE:
raise VMDKStreamException("Asked for a sector pointer on a file whose r/w pointer is not sector aligned")
else:
- return file_location / SECTOR_SIZE
+ return file_location / SECTOR_SIZE
def write_grain_table(outfile, grain_table, gtes_per_gt = 512):
@@ -204,7 +204,7 @@ def convert_to_stream(infilename, outfilename):
debug_print("DEBUG: Descriptor takes up (%s) sectors" % desc_sectors )
image_descriptor += image_descriptor_pad
- image_header = create_sparse_header(inFileSectors = infileSectors, descriptorSize = desc_sectors)
+ image_header = create_sparse_header(inFileSectors = infileSectors, descriptorSize = desc_sectors)
outfile = open(outfilename, "wb")
outfile.write(image_header)
@@ -245,7 +245,7 @@ def convert_to_stream(infilename, outfilename):
# Create a compressed grain
currentGrainTable.append(sector_pointer(outfile))
compChunk = zlib.compress(inChunk)
- grain_marker = create_grain_marker(inputSectorPointer, len(compChunk))
+ grain_marker = create_grain_marker(inputSectorPointer, len(compChunk))
grainPad, writeSectors = pad_to_sector(len(compChunk) + len(grain_marker))
outfile.write(grain_marker)
outfile.write(compChunk)
@@ -284,8 +284,8 @@ def convert_to_stream(infilename, outfilename):
debug_print("Grain directory length (%d)" % (len(grainDirectory)) )
debug_print("Grain directory: ")
debug_print(grainDirectory)
- outfile.write(struct.pack(grainDirectoryStruct, *grainDirectory))
-
+ outfile.write(struct.pack(grainDirectoryStruct, *grainDirectory))
+
# footer marker
outfile.write(create_marker(1,0,MARKER_FOOTER))
--
1.7.4.4
12 years, 10 months
#1870 - Support nil HWPs
by Matt Wagner
I've been blocked on getting vSphere instances to launch because it's not possible to enter a nil front-end HWP, and back-end HWPs with nil values were not accepted. This changes the code to permit both, and updates tests to support this.
-- Matt
12 years, 10 months
#1788 - "Download key" link on Deployment instances list
by Matt Wagner
After launching some deployments today in testing, it occurred to me that to actually connect to any of the instances I was launching in EC2 required that I enter the URL for /conductor/instances and then find them in the list to be able to download the key.
This patch adds a "Download key" link to the pretty view of the instance list on Deployment#show. It didn't make sense to me to duplicate it on filter view.
12 years, 10 months
BZ 716286
by Jan Provazník
Patchset moves 'match' method into instance model. With this user gets more
details why instance launch failed.
12 years, 10 months
[PATCH] Add <commands> tag to Red Hat produced VMs (Fedora, RHEL)
by Steven Dake
The <commads> tag allows the running of multiple commands during the customize
phase of oz. An example format would be:
<commands>
<command>
yum -y update
</command>
</commands>
This will trigger a yum update during the customize phase. This patch
does not address other platforms, although they should be easy enough to add
with some cut and paste from RedHat.py.
Signed-off-by: Steven Dake <sdake(a)redhat.com>
---
docs/tdl.rng | 18 ++++++++++++++++++
oz/RedHat.py | 8 ++++++--
oz/TDL.py | 23 +++++++++++++++++++++++
3 files changed, 47 insertions(+), 2 deletions(-)
diff --git a/docs/tdl.rng b/docs/tdl.rng
index bc0c20d..03a4793 100644
--- a/docs/tdl.rng
+++ b/docs/tdl.rng
@@ -90,6 +90,24 @@
</element>
</optional>
<optional>
+ <element name='commands'>
+ <oneOrMore>
+ <element name='command'>
+ <attribute name='name'>
+ <text/>
+ </attribute>
+ <attribute name='type'>
+ <choice>
+ <value>raw</value>
+ <value>base64</value>
+ </choice>
+ </attribute>
+ <text/>
+ </element>
+ </oneOrMore>
+ </element>
+ </optional>
+ <optional>
<element name='repositories'>
<zeroOrMore>
<element name='repository'>
diff --git a/oz/RedHat.py b/oz/RedHat.py
index 577f539..a519ce8 100644
--- a/oz/RedHat.py
+++ b/oz/RedHat.py
@@ -538,14 +538,18 @@ class RedHatCDYumGuest(RedHatCDGuest):
self.customize_files(guestaddr)
+ self.log.debug("Running custom commands")
+ for name, content in self.tdl.commands.items():
+ self.guest_execute_command(guestaddr, '%s' % (content))
+
self.log.debug("Syncing")
self.guest_execute_command(guestaddr, 'sync')
def customize(self, libvirt_xml):
self.log.info("Customizing image")
- if not self.tdl.packages and not self.tdl.files:
- self.log.info("No additional packages or files to install, skipping customization")
+ if not self.tdl.packages and not self.tdl.files and not self.tdl.commands:
+ self.log.info("No additional packages, files or commands to install, skipping customization")
return
self.collect_setup(libvirt_xml)
diff --git a/oz/TDL.py b/oz/TDL.py
index 5cc081f..e2e9a20 100644
--- a/oz/TDL.py
+++ b/oz/TDL.py
@@ -124,6 +124,9 @@ class TDL(object):
dictionary may be empty.
files - A dictionary of file contents to be added to the
operating system. The dictionary is indexed by filename.
+ commands - A dictionary of commands to run inside the guest VM. The
+ dictionary is indexed by commands. This dictionary may be
+ empty.
"""
def __init__(self, xmlstring):
self.doc = None
@@ -247,6 +250,26 @@ class TDL(object):
self.repositories[name] = Repository(name, url, signed)
+ self.commands = {}
+ for command in self.doc.xpathEval('/template/commands/command'):
+ name = command.prop('name')
+ if name is None:
+ raise oz.OzException.OzException("Command without a name was given")
+ contenttype = command.prop('type')
+ if contenttype is None:
+ contenttype = 'raw'
+
+ content = command.getContent().strip()
+ if contenttype == 'raw':
+ self.commands[name] = content
+ elif contenttype == 'base64':
+ if len(content) == 0:
+ self.commands[name] = ""
+ else:
+ self.commands[name] = base64.b64decode(content)
+ else:
+ raise oz.OzException.OzException("File type for %s must be 'raw' or 'base64'" % (name))
+
def __del__(self):
if self.doc is not None:
self.doc.freeDoc()
--
1.7.4.4
12 years, 10 months
How Audrey, Conductor, and Audrey's config server interact and their relationship to a unique vm instance id
by Steven Dake
Hi,
Tracking down how to get a unique machine ID into the VMs for
/var/lib/dbus/machne_id, had a conversation with Chris re how all these
parts interact.
Here is a summary of an IRC conversation with Chris:
A software component called Audrey runs inside the VM and replaces
rc.local before image upload. This component reads a configuration
script from the configuration server. The script could co whatever it
likes, such as start services (service httpd start) or write files, or
do any other interesting work to get the vm into an operational state.
It runs as S99 in the runlevel list (last).
When the vm has booted, audrey (the configuration tool on the vm)
contacts a configuration server. Currently this config server (a
component of audrey) is located in the cloud provider, but could also be
in a central location (TBD). Audrey can gather a unique id from the
configuration server.
Conductor tells the config server what the config should be for a
particular VM instance UUID. Conductor then starts the VM. The VM then
contacts the config server to insert its configuration giblets into the
system. The VM is then booted.
Regards
-steve
12 years, 10 months