Hi, this patchset adds deployment state attribute. This attribute will be used for tracking deployment's state which will be needed especially for doing rollback when launching a deployment. It also replaces 'status' method which computed deployment's state from states of all instances.
This patchset introduces state transitions (IOW now it's not possible to change deployment's state from 'running' to 'pending' or from 'stopped' to 'shutting down'). deployment's state is now mostly changed on a user's action (e.g. a user clicks stop button -> state is changed to 'shutting down'), though some transitions are done automatically ('pending' -> 'running', 'shutting down' -> 'stopped', all rollback state transitions will be automatic too).
Drawback of setting state when an action is performed is that deployment's state may not describe real state of all instances properly, for example if a deployment is running and all instances are suddenly changed to 'stopped' (from outside of conductor), then deployment's state will be 'incomplete' until a user stops/deletes the deployment from conductor.
Which reminds me I've added another deployment state: 'incomplete', this state is used when a deployment is running and then some (or all) instances unexpectedly change state (wiki page Robust_instance_launching is updated too).
From: Jan Provaznik jprovazn@redhat.com
https://www.aeolusproject.org/redmine/issues/3159 --- src/app/models/deployment.rb | 10 ++++++++++ .../migrate/20120416080744_add_deployment_state.rb | 9 +++++++++ 2 files changed, 19 insertions(+), 0 deletions(-) create mode 100644 src/db/migrate/20120416080744_add_deployment_state.rb
diff --git a/src/app/models/deployment.rb b/src/app/models/deployment.rb index 4f80aff..b2db8c8 100644 --- a/src/app/models/deployment.rb +++ b/src/app/models/deployment.rb @@ -77,6 +77,16 @@ class Deployment < ActiveRecord::Base USER_MUTABLE_ATTRS = ['name'] STATE_MIXED = "mixed"
+ STATE_NEW = "new" + STATE_PENDING = "pending" + STATE_RUNNING = "running" + STATE_SHUTTING_DOWN = "shutting_down" + STATE_STOPPED = "stopped" + STATE_FAILED = "failed" + STATE_ROLLBACK_IN_PROGRESS = "rollback_in_progress" + STATE_ROLLBACK_COMPLETE = "rollback_complete" + STATE_ROLLBACK_FAILED = "rollback_failed" + validate :validate_xml validate :validate_launch_parameters
diff --git a/src/db/migrate/20120416080744_add_deployment_state.rb b/src/db/migrate/20120416080744_add_deployment_state.rb new file mode 100644 index 0000000..3e96544 --- /dev/null +++ b/src/db/migrate/20120416080744_add_deployment_state.rb @@ -0,0 +1,9 @@ +class AddDeploymentState < ActiveRecord::Migration + def self.up + add_column :deployments, :state, :string + end + + def self.down + remove_column :deployments, :state + end +end
From: Jan Provaznik jprovazn@redhat.com
https://www.aeolusproject.org/redmine/issues/3160 --- src/app/controllers/deployments_controller.rb | 3 +++ src/app/models/deployment.rb | 9 +++++++++ 2 files changed, 12 insertions(+), 0 deletions(-)
diff --git a/src/app/controllers/deployments_controller.rb b/src/app/controllers/deployments_controller.rb index d259a8f..3cdc9c6 100644 --- a/src/app/controllers/deployments_controller.rb +++ b/src/app/controllers/deployments_controller.rb @@ -304,6 +304,9 @@ class DeploymentsController < ApplicationController errors = []
@deployments_to_stop.each do |deployment| + # TODO: move all this logic to model + deployment.state = Deployment::STATE_SHUTTING_DOWN + deployment.save! deployment.instances.each do |instance| log_prefix = "#{t('deployments.deployment')}: #{instance.deployment.name}, #{t('instances.instance')}: #{instance.name}" begin diff --git a/src/app/models/deployment.rb b/src/app/models/deployment.rb index b2db8c8..3347367 100644 --- a/src/app/models/deployment.rb +++ b/src/app/models/deployment.rb @@ -73,6 +73,7 @@ class Deployment < ActiveRecord::Base before_create :inject_launch_parameters before_create :generate_uuid before_create :set_pool_family + before_create :set_new_state
USER_MUTABLE_ATTRS = ['name'] STATE_MIXED = "mixed" @@ -161,6 +162,8 @@ class Deployment < ActiveRecord::Base return end
+ self.state = Deployment::STATE_SHUTTING_DOWN + save! if instances.all? {|i| i.destroyable? or i.state == Instance::STATE_RUNNING} destroy_deployment_config # The deployment will be destroyed from an InstanceObserver callback once @@ -212,6 +215,8 @@ class Deployment < ActiveRecord::Base # # TODO: need to be able to handle deployable-level errors # + self.state = STATE_PENDING + save! status = { :errors => {}, :successes => {} } status[:errors] = create_instances_with_params(user) all_inst_match, account, errors = find_match_with_common_account @@ -571,4 +576,8 @@ class Deployment < ActiveRecord::Base end errors end + + def set_new_state + self.state ||= STATE_NEW + end end
From: Jan Provaznik jprovazn@redhat.com
https://www.aeolusproject.org/redmine/issues/3160 --- src/app/models/deployment.rb | 40 +++++++++++++++++- src/app/models/instance.rb | 4 ++ src/app/models/instance_observer.rb | 1 + src/spec/models/deployment_spec.rb | 81 ++++++++++++++++++++++++++++++----- 4 files changed, 114 insertions(+), 12 deletions(-)
diff --git a/src/app/models/deployment.rb b/src/app/models/deployment.rb index 3347367..fe8008a 100644 --- a/src/app/models/deployment.rb +++ b/src/app/models/deployment.rb @@ -76,11 +76,11 @@ class Deployment < ActiveRecord::Base before_create :set_new_state
USER_MUTABLE_ATTRS = ['name'] - STATE_MIXED = "mixed"
STATE_NEW = "new" STATE_PENDING = "pending" STATE_RUNNING = "running" + STATE_INCOMPLETE = "incomplete" STATE_SHUTTING_DOWN = "shutting_down" STATE_STOPPED = "stopped" STATE_FAILED = "failed" @@ -450,6 +450,17 @@ class Deployment < ActiveRecord::Base instances.select {|instance| instance.failed?} end
+ def update_state(changed_instance) + transition_method = "state_transition_from_#{state}".to_sym + send(transition_method, changed_instance) if self.respond_to?(transition_method, true) + if state_changed? + save! + true + else + false + end + end + PRESET_FILTERS_OPTIONS = []
private @@ -580,4 +591,31 @@ class Deployment < ActiveRecord::Base def set_new_state self.state ||= STATE_NEW end + + def state_transition_from_pending(instance) + if instances.all? {|i| i.state == Instance::STATE_RUNNING} + self.state = STATE_RUNNING + elsif Instance::FAILED_STATES.include?(instance.state) + # TODO: initiate rollback. For now if an error occurs, deployment will + # stay in pending state + end + end + + def state_transition_from_running(instance) + if instance.state != STATE_RUNNING + self.state = STATE_INCOMPLETE + end + end + + def state_transition_from_incomplete(instance) + if instances.all? {|i| i.state == Instance::STATE_RUNNING} + self.state = STATE_RUNNING + end + end + + def state_transition_from_shutting_down(instance) + if instance.state == Instance::STATE_STOPPED and instances.all? {|i| i.inactive?} + self.state = STATE_STOPPED + end + end end diff --git a/src/app/models/instance.rb b/src/app/models/instance.rb index ee5b8df..d6eacfc 100644 --- a/src/app/models/instance.rb +++ b/src/app/models/instance.rb @@ -343,6 +343,10 @@ class Instance < ActiveRecord::Base FAILED_STATES.include?(state) end
+ def inactive? + (FAILED_STATES + [STATE_STOPPED]).include?(state) + end + def requires_config_server? not instance_config_xml.nil? or assembly_xml.requires_config_server? end diff --git a/src/app/models/instance_observer.rb b/src/app/models/instance_observer.rb index 244a8a4..77b2e12 100644 --- a/src/app/models/instance_observer.rb +++ b/src/app/models/instance_observer.rb @@ -95,6 +95,7 @@ class InstanceObserver < ActiveRecord::Observer # This can get stale, so reload it -- if it exists instance.deployment.reload if instance.deployment if instance.state_changed? + instance.deployment.update_state(instance) if instance.deployment event = Event.new(:source => instance, :event_time => DateTime.now, :summary => "state changed to #{instance.state}", :change_hash => instance.changes, :status_code => instance.state) diff --git a/src/spec/models/deployment_spec.rb b/src/spec/models/deployment_spec.rb index 2878520..69fc583 100644 --- a/src/spec/models/deployment_spec.rb +++ b/src/spec/models/deployment_spec.rb @@ -325,20 +325,79 @@ describe Deployment do end end
- describe "deployment_state" do - it "should be pending if only some instances are running" do - deployment = Factory.build :deployment - instance = Factory.build(:mock_running_instance, :deployment => deployment) - instance2 = Factory.build(:mock_pending_instance, :deployment => deployment) - deployment.stub(:instances) { [instance, instance2] } - deployment.status.should == :pending + describe "deployment state" do + context "in running state" do + before :each do + @deployment.state = Deployment::STATE_RUNNING + @deployment.save! + @inst1 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_RUNNING) + @inst2 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_RUNNING) + @deployment.instances << @inst1 + @deployment.instances << @inst2 + end + + it "should be incomplete if only some instances are running" do + @inst1.state = Instance::STATE_PENDING + @inst1.save! + @deployment.reload + @deployment.state.should == Deployment::STATE_INCOMPLETE + end end
- it "should be stopped if no instances exist" do - deployment = Factory.build :deployment - deployment.stub(:instances) {[]} - deployment.status.should == :stopped + context "in pending state" do + before :each do + @deployment.state = Deployment::STATE_PENDING + @deployment.save! + @inst1 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_PENDING) + @inst2 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_PENDING) + @deployment.instances << @inst1 + @deployment.instances << @inst2 + end + + it "should be pending if only some instances are running" do + @inst1.state = Instance::STATE_RUNNING + @inst1.save! + @deployment.reload + @deployment.state.should == Deployment::STATE_PENDING + end + + it "should be running if all instances are running" do + @inst1.state = Instance::STATE_RUNNING + @inst1.save! + @inst2.state = Instance::STATE_RUNNING + @inst2.save! + @deployment.reload + @deployment.state.should == Deployment::STATE_RUNNING + end + end + + context "in incomplete state" do + before :each do + @deployment.state = Deployment::STATE_INCOMPLETE + @deployment.save! + @inst1 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_RUNNING) + @inst2 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_PENDING) + @deployment.instances << @inst1 + @deployment.instances << @inst2 + end + + it "should be incomplete if no instances are running" do + @inst1.state = Instance::STATE_ERROR + @inst1.save! + @inst2.state = Instance::STATE_ERROR + @inst2.save! + @deployment.reload + @deployment.state.should == Deployment::STATE_INCOMPLETE + end + + it "should be running if all instances are running" do + @inst2.state = Instance::STATE_RUNNING + @inst2.save! + @deployment.reload + @deployment.state.should == Deployment::STATE_RUNNING + end end + end
describe ".instances.instance_parameters" do
On Wednesday 25 April 2012 14:55:01 jprovazn@redhat.com wrote:
From: Jan Provaznik jprovazn@redhat.com
https://www.aeolusproject.org/redmine/issues/3160
src/app/models/deployment.rb | 40 +++++++++++++++++- src/app/models/instance.rb | 4 ++ src/app/models/instance_observer.rb | 1 + src/spec/models/deployment_spec.rb | 81 ++++++++++++++++++++++++++++++----- 4 files changed, 114 insertions(+), 12 deletions(-)
diff --git a/src/app/models/deployment.rb b/src/app/models/deployment.rb index 3347367..fe8008a 100644 --- a/src/app/models/deployment.rb +++ b/src/app/models/deployment.rb @@ -76,11 +76,11 @@ class Deployment < ActiveRecord::Base before_create :set_new_state
USER_MUTABLE_ATTRS = ['name']
STATE_MIXED = "mixed"
STATE_NEW = "new" STATE_PENDING = "pending" STATE_RUNNING = "running"
- STATE_INCOMPLETE = "incomplete" STATE_SHUTTING_DOWN = "shutting_down" STATE_STOPPED = "stopped" STATE_FAILED = "failed"
@@ -450,6 +450,17 @@ class Deployment < ActiveRecord::Base instances.select {|instance| instance.failed?} end
- def update_state(changed_instance)
- transition_method = "state_transition_from_#{state}".to_sym
- send(transition_method, changed_instance) if
self.respond_to?(transition_method, true) + if state_changed?
save!
true
else
false
end
end
PRESET_FILTERS_OPTIONS = []
private
@@ -580,4 +591,31 @@ class Deployment < ActiveRecord::Base def set_new_state self.state ||= STATE_NEW end
- def state_transition_from_pending(instance)
- if instances.all? {|i| i.state == Instance::STATE_RUNNING}
self.state = STATE_RUNNING
- elsif Instance::FAILED_STATES.include?(instance.state)
# TODO: initiate rollback. For now if an error occurs, deployment
will + # stay in pending state
- end
- end
- def state_transition_from_running(instance)
- if instance.state != STATE_RUNNING
self.state = STATE_INCOMPLETE
- end
- end
- def state_transition_from_incomplete(instance)
- if instances.all? {|i| i.state == Instance::STATE_RUNNING}
self.state = STATE_RUNNING
- end
- end
- def state_transition_from_shutting_down(instance)
- if instance.state == Instance::STATE_STOPPED and instances.all? {|i|
i.inactive?} + self.state = STATE_STOPPED
- end
- end
end diff --git a/src/app/models/instance.rb b/src/app/models/instance.rb index ee5b8df..d6eacfc 100644 --- a/src/app/models/instance.rb +++ b/src/app/models/instance.rb @@ -343,6 +343,10 @@ class Instance < ActiveRecord::Base FAILED_STATES.include?(state) end
- def inactive?
- (FAILED_STATES + [STATE_STOPPED]).include?(state)
- end
- def requires_config_server? not instance_config_xml.nil? or assembly_xml.requires_config_server? end
diff --git a/src/app/models/instance_observer.rb b/src/app/models/instance_observer.rb index 244a8a4..77b2e12 100644 --- a/src/app/models/instance_observer.rb +++ b/src/app/models/instance_observer.rb @@ -95,6 +95,7 @@ class InstanceObserver < ActiveRecord::Observer # This can get stale, so reload it -- if it exists instance.deployment.reload if instance.deployment if instance.state_changed?
instance.deployment.update_state(instance) if instance.deployment event = Event.new(:source => instance, :event_time => DateTime.now, :summary => "state changed to #{instance.state}", :change_hash => instance.changes, :status_code => :instance.state)
diff --git a/src/spec/models/deployment_spec.rb b/src/spec/models/deployment_spec.rb index 2878520..69fc583 100644 --- a/src/spec/models/deployment_spec.rb +++ b/src/spec/models/deployment_spec.rb @@ -325,20 +325,79 @@ describe Deployment do end end
- describe "deployment_state" do
- it "should be pending if only some instances are running" do
deployment = Factory.build :deployment
instance = Factory.build(:mock_running_instance, :deployment =>
deployment) - instance2 = Factory.build(:mock_pending_instance, :deployment => deployment) - deployment.stub(:instances) { [instance, instance2] }
deployment.status.should == :pending
- describe "deployment state" do
- context "in running state" do
before :each do
@deployment.state = Deployment::STATE_RUNNING
@deployment.save!
@inst1 = Factory.create(:instance, :deployment => @deployment,
:state => Instance::STATE_RUNNING) + @inst2 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_RUNNING) + @deployment.instances << @inst1
@deployment.instances << @inst2
end
it "should be incomplete if only some instances are running" do
@inst1.state = Instance::STATE_PENDING
^^ in fact there should be other state than Instance::STATE_PENDING(transition from RUNNING to PENDING is not allowed)
@inst1.save!
@deployment.reload
@deployment.state.should == Deployment::STATE_INCOMPLETE
endend
- it "should be stopped if no instances exist" do
deployment = Factory.build :deployment
deployment.stub(:instances) {[]}
deployment.status.should == :stopped
- context "in pending state" do
before :each do
@deployment.state = Deployment::STATE_PENDING
@deployment.save!
@inst1 = Factory.create(:instance, :deployment => @deployment,
:state => Instance::STATE_PENDING) + @inst2 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_PENDING) + @deployment.instances << @inst1
@deployment.instances << @inst2
end
it "should be pending if only some instances are running" do
@inst1.state = Instance::STATE_RUNNING
@inst1.save!
@deployment.reload
@deployment.state.should == Deployment::STATE_PENDING
end
it "should be running if all instances are running" do
@inst1.state = Instance::STATE_RUNNING
@inst1.save!
@inst2.state = Instance::STATE_RUNNING
@inst2.save!
@deployment.reload
@deployment.state.should == Deployment::STATE_RUNNING
end
- end
- context "in incomplete state" do
before :each do
@deployment.state = Deployment::STATE_INCOMPLETE
@deployment.save!
@inst1 = Factory.create(:instance, :deployment => @deployment,
:state => Instance::STATE_RUNNING) + @inst2 = Factory.create(:instance, :deployment => @deployment, :state => Instance::STATE_PENDING) + @deployment.instances << @inst1
@deployment.instances << @inst2
end
it "should be incomplete if no instances are running" do
@inst1.state = Instance::STATE_ERROR
@inst1.save!
@inst2.state = Instance::STATE_ERROR
@inst2.save!
@deployment.reload
@deployment.state.should == Deployment::STATE_INCOMPLETE
end
it "should be running if all instances are running" do
@inst2.state = Instance::STATE_RUNNING
@inst2.save!
@deployment.reload
@deployment.state.should == Deployment::STATE_RUNNING
end
end
end
describe ".instances.instance_parameters" do
From: Jan Provaznik jprovazn@redhat.com
https://www.aeolusproject.org/redmine/issues/3199 --- src/app/helpers/mustache_helper.rb | 4 ++-- src/app/models/deployment.rb | 16 +++------------- src/app/views/pools/_deployments.html.haml | 4 ++-- src/config/locales/en.yml | 24 +++++++++++++++++++++--- 4 files changed, 28 insertions(+), 20 deletions(-)
diff --git a/src/app/helpers/mustache_helper.rb b/src/app/helpers/mustache_helper.rb index 16464f1..eed0223 100644 --- a/src/app/helpers/mustache_helper.rb +++ b/src/app/helpers/mustache_helper.rb @@ -47,8 +47,8 @@ module MustacheHelper :name => deployment.name, :path => deployment_path(deployment), :filter_view_path => deployment_path(deployment, :view => :filter), - :status => deployment.status, - :translated_state => I18n.t("deployments.status.#{deployment.status}"), + :status => deployment.state, + :translated_state => I18n.t("deployments.status_description.#{deployment.state}"), :uptime => count_uptime(deployment.uptime_1st_instance), :deployable_xml_name => deployment.deployable_xml.name, :created_at => deployment.created_at.to_s, diff --git a/src/app/models/deployment.rb b/src/app/models/deployment.rb index fe8008a..4ad2082 100644 --- a/src/app/models/deployment.rb +++ b/src/app/models/deployment.rb @@ -387,16 +387,6 @@ class Deployment < ActiveRecord::Base end end
- def status - if instances.any? and instances.all? {|i| i.state == Instance::STATE_RUNNING} - :running - elsif instances.empty? or instances.all? {|i| (Instance::FAILED_STATES + [Instance::STATE_STOPPED]).include?(i.state)} - :stopped - else - :pending - end - end - # A deployment "starts" when _all_ instances begin to run def start_time if instances.deployed.count == instances.count && ev = events.find_by_status_code(:all_running) @@ -419,9 +409,9 @@ class Deployment < ActiveRecord::Base def as_json(options={}) json = super(options).merge({ :deployable_xml_name => deployable_xml.name, - :status => status, - :translated_status => I18n.t("instances.states.#{status}"), - :status_description => I18n.t("deployments.status.#{status}"), + :status => state, + :translated_status => I18n.t("deployments.status.#{state}"), + :status_description => I18n.t("deployments.status_description.#{state}"), :instances_count => instances.count, :failed_instances_count => failed_instances.count, :instances_count_text => I18n.t('instances.instances', :count => instances.count.to_i), diff --git a/src/app/views/pools/_deployments.html.haml b/src/app/views/pools/_deployments.html.haml index 4052d91..c535f93 100644 --- a/src/app/views/pools/_deployments.html.haml +++ b/src/app/views/pools/_deployments.html.haml @@ -5,8 +5,8 @@ %li.deployment %h3.name = link_to deployment.name, deployment_path(deployment) - %span{ :class => "status #{deployment.status}", :title => t("deployments.status.#{deployment.status}")} - = deployment.status + %span{ :class => "status #{deployment.state}", :title => t("deployments.status_description.#{deployment.state}")} + = deployment.state %dl.statistics %ul %li.right diff --git a/src/config/locales/en.yml b/src/config/locales/en.yml index f6d3623..c918ece 100644 --- a/src/config/locales/en.yml +++ b/src/config/locales/en.yml @@ -182,9 +182,27 @@ en: list_of_events: List of deployment events state: State status: - running: All instances are running - pending: Some (but not all) instances are running - stopped: No instances are currently running + new: New + pending: "Pending" + running: "Running" + incomplete: "Incomplete" + shutting_down: "Shutting down" + stopped: "Stopped" + failed: "Failed" + rollback_in_progress: "Rollback in progress" + rollback_complete: "Rollback complete" + rollback_failed: "Rollback failed" + status_description: + new: "Deployment wasn't started" + pending: "Deployment is starting up" + running: "All instances are running" + incomplete: "Some instances are not running" + shutting_down: "Deployment is shutting down" + stopped: "All instances are stopped" + failed: "All instances are in failed state" + rollback_in_progress: "Launch failed, rollback is in progress" + rollback_complete: "Rollback successfully completed" + rollback_failed: "Rollback failed, re-launch terminated" launch_time_params: title: "Launch-time parameters" configure_launch_time_params: "Configure launch-time parameters for your deployment:"
On Wednesday 25 April 2012 14:54:58 jprovazn@redhat.com wrote:
Hi, this patchset adds deployment state attribute. This attribute will be used for tracking deployment's state which will be needed especially for doing rollback when launching a deployment. It also replaces 'status' method which computed deployment's state from states of all instances.
This patchset introduces state transitions (IOW now it's not possible to change deployment's state from 'running' to 'pending' or from 'stopped' to 'shutting down'). deployment's state is now mostly changed on a user's action (e.g. a user clicks stop button -> state is changed to 'shutting down'), though some transitions are done automatically ('pending' -> 'running', 'shutting down' -> 'stopped', all rollback state transitions will be automatic too).
Drawback of setting state when an action is performed is that deployment's state may not describe real state of all instances properly, for example if a deployment is running and all instances are suddenly changed to 'stopped' (from outside of conductor), then deployment's state will be 'incomplete' until a user stops/deletes the deployment from conductor.
Which reminds me I've added another deployment state: 'incomplete', this state is used when a deployment is running and then some (or all) instances unexpectedly change state (wiki page Robust_instance_launching is updated too).
ACK. Everything work fine, tests are green(one inline note in new spec examples ).
On 05/03/2012 04:15 PM, Jozef Zigmund wrote:
On Wednesday 25 April 2012 14:54:58 jprovazn@redhat.com wrote:
Hi, this patchset adds deployment state attribute. This attribute will be used for tracking deployment's state which will be needed especially for doing rollback when launching a deployment. It also replaces 'status' method which computed deployment's state from states of all instances.
This patchset introduces state transitions (IOW now it's not possible to change deployment's state from 'running' to 'pending' or from 'stopped' to 'shutting down'). deployment's state is now mostly changed on a user's action (e.g. a user clicks stop button -> state is changed to 'shutting down'), though some transitions are done automatically ('pending' -> 'running', 'shutting down' -> 'stopped', all rollback state transitions will be automatic too).
Drawback of setting state when an action is performed is that deployment's state may not describe real state of all instances properly, for example if a deployment is running and all instances are suddenly changed to 'stopped' (from outside of conductor), then deployment's state will be 'incomplete' until a user stops/deletes the deployment from conductor.
Which reminds me I've added another deployment state: 'incomplete', this state is used when a deployment is running and then some (or all) instances unexpectedly change state (wiki page Robust_instance_launching is updated too).
ACK. Everything work fine, tests are green(one inline note in new spec examples ).
Pushed with the updated spc test. Thanks. Jan
aeolus-devel@lists.fedorahosted.org