RFC: Factory support for nested build requests

Hugh Brock hbrock at redhat.com
Thu Jan 10 17:27:20 UTC 2013


On Thu, Jan 10, 2013 at 08:59:08AM -0600, Ian McLeod wrote:
> On Thu, 2013-01-10 at 09:08 -0500, Hugh Brock wrote:
> > On Thu, Jan 10, 2013 at 10:18:23AM +0100, Jan Provaznik wrote:
> > > On 01/10/2013 12:46 AM, Matt Wagner wrote:
> > > >Hi folks,
> > > >
> > > >This email is to solicit input on a discussion that came up on IRC
> > > >tonight, with a proposal to have Image Factory allow us to request the
> > > >builds of child objects at the same time we are requesting the builds
> > > >for their parents.
> > > >
> > > >Jan and I hit an issue tonight with submitting builds to Image
> > > >Factory. The problem is essentially caused by the fact that, in our
> > > >new (reimplemented) build_all method[1], we submit a request to build
> > > >a new image version, and the immediately commence building target
> > > >images.
> > > >
> > > >This appears to typically work fine when building for a single
> > > >provider, but when there are multiple providers, this often causes the
> > > >build to fail, as we end up submitting the target_image builds before
> > > >the image_version build has completed.
> > > >
> > > >In our discussion, we talked about the implementation of this. The
> > > >"right way" in Conductor would be to set up a background task to
> > > >handle this. We would submit a request to build an image version, and
> > > >then wait for a callback from Factory saying it was done. When it was
> > > >done, we would then submit tasks for each target image we wished to
> > > >build.
> > > >
> > > 
> > > Only a nit: a background task is not necessarily needed, we could
> > > hook sending of image build requests to image version's active
> > > record after_update callback (which would be called once 'COMPLETE'
> > > status is set on image version).
> > > 
> > > >(We would have to do this eventually anyway, to allow us to
> > > >automatically push the image when we got a callback that the build had
> > > >completed. One-call build-and-push would help here, but this isn't
> > > >what we're talking about in this email.)
> > > >
> > > >Ian proposed that this might be done in Factory itself:
> > > >
> > > ><imcleod> matty_dubs: Would it be super-helpful to you to be able to
> > > >request child image types while the parent is still in progress?
> > > >Because we could do that.  We'd have to add some polling internal to
> > > >factory, but it's not particularly onerous.
> > > >
> > > >My gut is that this would make things significantly simpler, at least
> > > >on the Conductor side. We wouldn't need to worry about queuing tasks
> > > >to be sent upon receipt of callbacks. It also just feels
> > > >architecturally cleaner, to not have two applications that need to
> > > >know a lot about the inner workings of the other.
> > > >
> > > >Ian is going to investigate a bit to get an estimate of how involved
> > > >this would be. But in the event that it's not too involved, I propose
> > > >that this would be the best course of action, versus us trying to
> > > >implement the same sort of thing in Conductor.
> > > >
> > > >Do any of you folks have opinions on the matter?
> > > >
> > > 
> > > Agree with you, it would be best to have support for this directly
> > > in Imagefactory if possible.
> > 
> > Seems to me this is an important part of Factory's value proposition,
> > actually. The intent of Factory is to hide the implementation details
> > around build/push/register for all clouds. If it's really doing that
> > then callers shouldn't need to be aware of the intermediate stages. 
> 
> We aspire to be valuable Hugh.
> 
> Factory v2 has, for some time, supported a mode of operation where we
> hide the existence of the parent object if the user wants us to.  That
> is, you can just request a target_image or a provider_image and Factory
> will transparently create the parent objects and update you when the
> object you've requested is finished.
> 
> The first iteration of Tim actually took advantage of this and ignored
> base images entirely.
> 
> The revised Tim is now doing something different.  It's using the base
> image concept but it's also making child object creation requests before
> the parent has finished.  In some cases, it's making multiple child
> object requests.
> 
> As I understand it, Matt was some way down the road of using our status
> callbacks to address this, by waiting for success or failure of the
> parent before requesting the child.
> 
> It was in this context that I made my IRC offer early yesterday evening
> to attempt to hide this completely in the factory.
> 
> I have a candidate patch and will be testing it this morning.
> 

Look at Mcleod, adding value... sheesh... Nice one Ian. I'll look
forward to seeing how it plays out.

--H

-- 
== Hugh Brock, hbrock at redhat.com                                   ==
== Engineering Manager, Cloud BU                                   ==
== Aeolus Project: Manage virtual infrastructure across clouds.    ==
== http://aeolusproject.org                                        ==

"I know that you believe you understand what you think I said, but I’m
not sure you realize that what you heard is not what I meant."
--Robert McCloskey


More information about the aeolus-devel mailing list