RFC: Factory support for nested build requests

Ian McLeod imcleod at redhat.com
Thu Jan 10 14:59:08 UTC 2013


On Thu, 2013-01-10 at 09:08 -0500, Hugh Brock wrote:
> On Thu, Jan 10, 2013 at 10:18:23AM +0100, Jan Provaznik wrote:
> > On 01/10/2013 12:46 AM, Matt Wagner wrote:
> > >Hi folks,
> > >
> > >This email is to solicit input on a discussion that came up on IRC
> > >tonight, with a proposal to have Image Factory allow us to request the
> > >builds of child objects at the same time we are requesting the builds
> > >for their parents.
> > >
> > >Jan and I hit an issue tonight with submitting builds to Image
> > >Factory. The problem is essentially caused by the fact that, in our
> > >new (reimplemented) build_all method[1], we submit a request to build
> > >a new image version, and the immediately commence building target
> > >images.
> > >
> > >This appears to typically work fine when building for a single
> > >provider, but when there are multiple providers, this often causes the
> > >build to fail, as we end up submitting the target_image builds before
> > >the image_version build has completed.
> > >
> > >In our discussion, we talked about the implementation of this. The
> > >"right way" in Conductor would be to set up a background task to
> > >handle this. We would submit a request to build an image version, and
> > >then wait for a callback from Factory saying it was done. When it was
> > >done, we would then submit tasks for each target image we wished to
> > >build.
> > >
> > 
> > Only a nit: a background task is not necessarily needed, we could
> > hook sending of image build requests to image version's active
> > record after_update callback (which would be called once 'COMPLETE'
> > status is set on image version).
> > 
> > >(We would have to do this eventually anyway, to allow us to
> > >automatically push the image when we got a callback that the build had
> > >completed. One-call build-and-push would help here, but this isn't
> > >what we're talking about in this email.)
> > >
> > >Ian proposed that this might be done in Factory itself:
> > >
> > ><imcleod> matty_dubs: Would it be super-helpful to you to be able to
> > >request child image types while the parent is still in progress?
> > >Because we could do that.  We'd have to add some polling internal to
> > >factory, but it's not particularly onerous.
> > >
> > >My gut is that this would make things significantly simpler, at least
> > >on the Conductor side. We wouldn't need to worry about queuing tasks
> > >to be sent upon receipt of callbacks. It also just feels
> > >architecturally cleaner, to not have two applications that need to
> > >know a lot about the inner workings of the other.
> > >
> > >Ian is going to investigate a bit to get an estimate of how involved
> > >this would be. But in the event that it's not too involved, I propose
> > >that this would be the best course of action, versus us trying to
> > >implement the same sort of thing in Conductor.
> > >
> > >Do any of you folks have opinions on the matter?
> > >
> > 
> > Agree with you, it would be best to have support for this directly
> > in Imagefactory if possible.
> 
> Seems to me this is an important part of Factory's value proposition,
> actually. The intent of Factory is to hide the implementation details
> around build/push/register for all clouds. If it's really doing that
> then callers shouldn't need to be aware of the intermediate stages. 

We aspire to be valuable Hugh.

Factory v2 has, for some time, supported a mode of operation where we
hide the existence of the parent object if the user wants us to.  That
is, you can just request a target_image or a provider_image and Factory
will transparently create the parent objects and update you when the
object you've requested is finished.

The first iteration of Tim actually took advantage of this and ignored
base images entirely.

The revised Tim is now doing something different.  It's using the base
image concept but it's also making child object creation requests before
the parent has finished.  In some cases, it's making multiple child
object requests.

As I understand it, Matt was some way down the road of using our status
callbacks to address this, by waiting for success or failure of the
parent before requesting the child.

It was in this context that I made my IRC offer early yesterday evening
to attempt to hide this completely in the factory.

I have a candidate patch and will be testing it this morning.

> I'd support adding one or two folks whose eyes don't bleed when looking
> at Python to help Ian and Steve with this effort, if necessary...
> 
> --Hugh
> 

-- 
Ian McLeod - Red Hat
office: 312.660.3539
mobile: 312.899.6736
rh internal: (81) 33539



More information about the aeolus-devel mailing list