[Beaker-devel] First draft of reservations design proposal

Nick Coghlan ncoghlan at redhat.com
Thu Sep 26 08:47:05 UTC 2013


The first draft of the improved reservations and loans design proposal
is up at:


http://gerrit.beaker-project.org/#/c/2302/1/dev/proposals/improved-reservations-and-loans.rst

Very rough sketch at the moment, so I'm mostly looking for feedback on
whether or not the general concepts seem sound.

I've quoted the whole thing below, since I'm not really after
line-by-line review comments at this stage, so email seems like a more
appropriate venue.

Cheers,
Nick.

Improved System Reservations and Loans
======================================

:Author: Nick Coghlan
:Status: Proposed
:Target Release: 0.16


Abstract
--------

This proposal supplements the current timed system reservation mechanism
with a new mechanism that is independent of the lab controller's external
watchdog timer. This new reservation timer is then used to offer time
limited manual reservations and automatic reservation of systems when
using an alternative harness or after a recipe is aborted.

A similar time limiting mechanism is proposed for system loans, as well as
the ability to indicate when reserving a system in Manual mode that the
loan should be returned automatically when the reservation is returned.

Finally, a command line interface will be provided for system loan
management.

.. note::

   Writing this out, I suspect it needs to be done in two phases, with phase
   one covering the reservation expiry times and automatically returning
   loans when the reservation is returned, while phase two covers loan
   expiry.


Harness independent time limited reservations
---------------------------------------------

A new "expiry date" attribute will be added to reservation records.

For systems in Manual mode, the web UI will be updated to allow
specification
of a duration when taking the system. The default will continue to be no
time limit, but if a time limit is requested, then the default will be
24 hours (configurable as a global Beaker server setting). If a time limit
is requested, then the reservation expiry date will be set appropriately.

For systems in Automated mode, the reservation expiry date will not be
used by default. Instead, job submitters will be able to opt in to the new
mechanism by specifying a new ``reservesys`` element as part of the recipe
definition.

Ignoring things like making the attributes optional and giving them a
specific data types, the new ``reservesys`` element is defined as::

    <element name="reservesys">
        <attribute name="onabort"/>
        <attribute name="onfail"/>
        <attribute name="onwarn"/>
        <attribute name="onpass"/>
        <attribute name="duration"/>
    </element>

By default, the system would be reserved regardless of how the recipe
finishes. The duration setting would control how long the automatic
reservation would be for (defaulting to 24 hours)

onabort/fail/warn/pass control whether or not the system is
actually reserved in the relevant situations:

* onabort = reserve system if a task or the overall recipe aborts
* onfail = reserve system if at least one task reports a failed result
* onwarn = reserve system if at least one task reports a warning
* onpass = reserve system if none of the above occur

All four settings would default to "true", so the reservation is
unconditional by default.

If any of the chosen conditions is triggered, the appropriate expiry time
will be recorded on the reservation once the recipe has completed rather
than returning the system immediately.

If the recipe is explicitly cancelled, then the system will not be reserved.

While a reservation is active, it may be extended. The existing watchdog
extension commands will be updated to also extended the reservation when
needed.

The Beaker scheduling daemon will periodically check for expired
reservations
and automatically return them. For systems in Automated mode, the expiry of
the post-completion reservation will not alter the result reported for the
recipe.

.. note::

   Perhaps Beaker should send a warning email 25 hours prior to expiry of
   the reservation?

(Also see :issue:`639938`)

Automatically returning loans
-----------------------------

When reserving a system, a user will be able to choose to automatically
return a loan. If this option is specified, then when the reservation is
returned, the active system loan will also be returned.


Time limited loans
------------------

System loans will be separated out to be tracked explicitly (similar to
the handling of reservations, only without the "type" field). When loaning
the system, the user granting the loan will be able to specify a duration
of the loan, which will be recorded as a loan expiry time. The default will
continue to be no time limit, but if a time limit is requested, then the
default will be 7 days (configurable as a global Beaker server setting).

While a loan is active, it may be extended. However, the user extending the
loan must have ``loan-self`` or ``loan-any`` permissions on the system.

If a system has a loan expiry time configured, then all reservations made
by that user must be time limited, defaulting to the loan expiry time.
Attempting to extend reservations beyond the end of the loan period will
fail.

The Beaker scheduling daemon will periodically check for expired loans
and automatically return them. For reserved systems where the reservation
is from a user that is no longer permitted to access the system once the
loan has been returned, it will also return the reservation or cancel the
recipe as appropriate.

.. note::

   Perhaps Beaker should send a warning email 25 hours prior to expiry of
   the loan?

User interface proposals
------------------------

Web UI
~~~~~~

TBD

Command line
~~~~~~~~~~~~

TBD

Deferred features
-----------------

* Updating the scheduled provisioning mechanism for systems in Automated
  mode to use the new harness independent mechanism rather than the
  reservesys task

* Allowing the ``reservesys`` element to be specified at the recipe set
level
  to reserve all systems in the recipe set whenever one or more of them
  encounters a problem.


Rejected features
-----------------

* Allowing the ``reservesys`` element to be specified at the job level,
  since it isn't clear how that would work when recipe sets are run at
  different times.

* Having "onpass" default to false in the reservesys element. While this is
  desirable in some respects, having different defaults for one of the
  items is difficult to document clearly.

-- 
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane

Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)


More information about the Beaker-devel mailing list