On 06/11/2013 11:23 AM, Raymond Mancy wrote:
----- Original Message -----
> From: "Nick Coghlan" <ncoghlan(a)redhat.com>
> To: beaker-devel(a)lists.fedorahosted.org
> Sent: Tuesday, June 11, 2013 11:16:47 AM
> Subject: Re: [Beaker-devel] Better handling of pending status updates when scheduling
recipes
>
> On 06/11/2013 10:06 AM, Raymond Mancy wrote:
>>
>>
>> ----- Original Message -----
>>> From: "Nick Coghlan" <ncoghlan(a)redhat.com>
>>> what we really want is a way to
>>> flag those systems as off limits for the remainder of this pass over the
>>> queue, *without* attempting to schedule the dirty recipe immediately.
>>>
>>
>> That was attempted with this patch[1].
>>
>> In testing I could not get it to run at a reasonable speed. I could take a
>> look
>> at it again with refreshed eyes I guess.
>
> You could also use a client side system ID cache in the scheduling loop
> to filter out the off limits systems rather than trying to do it through
> the database.
Please Explain :-)
Rather than hitting up the database to figure out which systems are off
limits and try to get *exactly one* suitable system back for each
recipe, maintain a Python set in the scheduling loop that holds the
system IDs for "off limits" systems.
When we get the list of candidate systems back from the DB for a recipe:
1. If the recipe is dirty, add the first candidate system that isn't
already in the "off limits" set to the "off limits" set.
2. If the recipe is clean, schedule it on the first candidate system
that isn't already in the "off limits" set
3. If there are no candidate systems that aren't in the "off limits"
set, simply continue on to the next recipe
This way, on each pass through the loop, dirty recipes get a tentative
reservation (by placing the system ID in the scheduling loop's "off
limits" set), while clean recipes get assigned to a system.
The ABBA deadlock is prevented by the fact that when the second recipe
is considered, the contested machine will already be in the "off limits"
set and thus skipped over client side.
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Test Automation Team Lead
Beaker Development Lead (
http://beaker-project.org/)
PulpDist Development Lead (
http://pulpdist.readthedocs.org)