[Beaker-devel] Docs/overview for using autotest in Beaker?

Don Zickus dzickus at redhat.com
Fri Nov 8 19:38:20 UTC 2013


On Tue, Nov 05, 2013 at 12:00:59PM +1000, Nick Coghlan wrote:
> > 
> > will request BEAKER_RECIPE_ID xml from BEAKER_LAB_CONTROLLER_URL, convert
> > it into a control file for autotest to consume and start running tests
> > (and reporting them to beaker).
> 
> OK, that sounds reasonable (especially once the RPM packaging is brought
> up to date)
> 
> >> 2. What do you need to set as the alternative harness in
> >> http://beaker-project.org/docs/alternative-harnesses/#selecting-an-alternative-harness-in-your-recipe
> > 
> > I need 'AUTOTEST_HARNESS=beaker' set.
> 
> OK, in order to get the
> (ks_meta="harness=<package-or-URL-to-yum-install>") recipe attribute to
> work properly, it looks like we may also need a
> "autotest-beaker-harness" RPM that:

Yeah, Bill explained (and showed) how that works.  So now I have the right
variables.  Though I was caught by surprise when my tasks timed out
because nothing communicated to beaker that distribution-install was
successfull. ;-)

> 
> 1. Depends on autotest itself
> 2. Sets "AUTOTEST_HARNESS=beaker" in the system environment variables
> 3. Arranges for "autotest-local bootstrap" to run on the next reboot

Yeah, I will sort this out a little later.  Using Bill's idea of the %post
section is probably a good start for now.

> 
> Otherwise you'll need to write a Beaker task that installs beah just to
> bootstrap autotest, and installing beah is one of the things using an
> alternative harness is supposed to avoid.
> 
> >> ?
> >> 3. What's known to work, what's still quirky? (e.g. reboot)
> > 
> > reboot doesn't work for some reason.  I don't think running actual
> > autotest-tests work.  I have only run with beaker tests through a job.
> > And also running locally doesn't work (even though the tests are locally
> > installed [unless you hack up some recipe id workarounds..]).
> 
> We're going to have to get the reboot glitch sorted in order to
> interoperate with the alternative harness selection mechanism.

Yup. :-)

> 
> >> 4. What's different (if anything) compared to normal usage of autotest?
> > 
> > Normal usage of autotest is 'autotest-local <some control file>' or
> > 'autotest-local run <testname>' (where testname is from 'autotest-local
> > list')
> > 
> > But that is running autotest locally/offline.  I can get it to work with
> > beaker tests, but I have to tweak some env variables to fake it (not
> > ideal).  So run now it has to run automated, which isn't ideal either.
> 
> The current recommendation for local beah testing is to assume offline
> execution if JOBID isn't set:
> http://beaker-project.org/docs/user-guide/multihost.html#standalone-execution-of-multihost-tests
> 
> Something similar may work here: if AUTOTEST_HARNESS=beaker is set, but
> the BEAKER_* environment variables are missing, perhaps there could be
> some way to provide the recipe definition XML file directly rather than
> needing to retrieve it from the server?

Well, we still have /root/RECIPE.txt and LABC_URL.  Combine those pieces
and one can figure out how to get the xml file in an automated way.

> 
> >> At the moment, it's impossible for even the core Beaker team to help
> >> with this (let alone anyone else that may be interested), as we don't
> >> even know how to get set up to run it.
> > 
> > Yeah, I was trying to figure out a workflow that I wanted to run autotest
> > with.
> > 
> > Initially I wanted to run tests locally for developers and through
> > automation for QE folks.  I can run autotest tests for developers and
> > beaker tests for QE folks.  Ideally I need to mix and match in both
> > directions. :-/
> > 
> > It shouldn't be hard to do, just need to hack up some glue code to detect
> > and run things properly.
> > 
> >>
> >> I'd also like to submit a talk on Beaker to the Continuous Integration
> >> miniconf at LCA, and that becomes a lot more compelling when I can point
> >> to a post about *how* to run autotest inside Beaker, rather than just to
> >> the pull request that made it possible.
> > 
> > Yeah, my stuff needs more work.  And more thought.  Perhaps some feedback
> > about workflows you had in mind, might help target this a little more.
> > 
> > I'll keep poking at it this week.
> 
> Hopefully the above helps - it sounds to me like a way to provide an XML
> definition directly for local testing + the "autotest-beaker-harness"
> RPM to integrate with the alternative harness selection mechanism are
> the key next steps.
> 
> Then on to the fun part of getting non-Beaker autotest tests running
> inside Beaker :)

Yeah, getting there.  For now how about a beaker patch?

Cheers,
Don

--------------------8<----------------
From: Don Zickus <dzickus at redhat.com>
Date: Fri, 8 Nov 2013 14:03:34 -0500
Subject: [PATCH] Tasks: make reservesys always exit 0

Different harnesses have a different environment than what reservesys
was coded for.  As a result, the task may fail for something silly

/usr/bin/killall rhts-test-runner.sh
 rhts-test-runner.sh: no process found

because the test harness I am using, doesn't run rhts-test-runner.sh.

I do not believe there is any reason to have reservesys fail.  The
test was built to always succeed.  Force that success with an
'exit 0' at the end.

Signed-off-by: Don Zickus <dzickus at redhat.com>
---
 Tasks/distribution/reservesys/runtest.sh |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/Tasks/distribution/reservesys/runtest.sh b/Tasks/distribution/reservesys/runtest.sh
index 3a79b90..b4353e3 100755
--- a/Tasks/distribution/reservesys/runtest.sh
+++ b/Tasks/distribution/reservesys/runtest.sh
@@ -242,3 +242,6 @@ RprtRslt $TEST PASS 0
 
 # stop rhts service, So that reserve workflow works with test reboot support.
 STOPRHTS
+
+# different harness may cause this script to fail, always return zero
+exit 0
-- 
1.7.1



More information about the Beaker-devel mailing list