[Beaker-devel] Docs/overview for using autotest in Beaker?
Don Zickus
dzickus at redhat.com
Fri Nov 8 19:38:20 UTC 2013
On Tue, Nov 05, 2013 at 12:00:59PM +1000, Nick Coghlan wrote:
> >
> > will request BEAKER_RECIPE_ID xml from BEAKER_LAB_CONTROLLER_URL, convert
> > it into a control file for autotest to consume and start running tests
> > (and reporting them to beaker).
>
> OK, that sounds reasonable (especially once the RPM packaging is brought
> up to date)
>
> >> 2. What do you need to set as the alternative harness in
> >> http://beaker-project.org/docs/alternative-harnesses/#selecting-an-alternative-harness-in-your-recipe
> >
> > I need 'AUTOTEST_HARNESS=beaker' set.
>
> OK, in order to get the
> (ks_meta="harness=<package-or-URL-to-yum-install>") recipe attribute to
> work properly, it looks like we may also need a
> "autotest-beaker-harness" RPM that:
Yeah, Bill explained (and showed) how that works. So now I have the right
variables. Though I was caught by surprise when my tasks timed out
because nothing communicated to beaker that distribution-install was
successfull. ;-)
>
> 1. Depends on autotest itself
> 2. Sets "AUTOTEST_HARNESS=beaker" in the system environment variables
> 3. Arranges for "autotest-local bootstrap" to run on the next reboot
Yeah, I will sort this out a little later. Using Bill's idea of the %post
section is probably a good start for now.
>
> Otherwise you'll need to write a Beaker task that installs beah just to
> bootstrap autotest, and installing beah is one of the things using an
> alternative harness is supposed to avoid.
>
> >> ?
> >> 3. What's known to work, what's still quirky? (e.g. reboot)
> >
> > reboot doesn't work for some reason. I don't think running actual
> > autotest-tests work. I have only run with beaker tests through a job.
> > And also running locally doesn't work (even though the tests are locally
> > installed [unless you hack up some recipe id workarounds..]).
>
> We're going to have to get the reboot glitch sorted in order to
> interoperate with the alternative harness selection mechanism.
Yup. :-)
>
> >> 4. What's different (if anything) compared to normal usage of autotest?
> >
> > Normal usage of autotest is 'autotest-local <some control file>' or
> > 'autotest-local run <testname>' (where testname is from 'autotest-local
> > list')
> >
> > But that is running autotest locally/offline. I can get it to work with
> > beaker tests, but I have to tweak some env variables to fake it (not
> > ideal). So run now it has to run automated, which isn't ideal either.
>
> The current recommendation for local beah testing is to assume offline
> execution if JOBID isn't set:
> http://beaker-project.org/docs/user-guide/multihost.html#standalone-execution-of-multihost-tests
>
> Something similar may work here: if AUTOTEST_HARNESS=beaker is set, but
> the BEAKER_* environment variables are missing, perhaps there could be
> some way to provide the recipe definition XML file directly rather than
> needing to retrieve it from the server?
Well, we still have /root/RECIPE.txt and LABC_URL. Combine those pieces
and one can figure out how to get the xml file in an automated way.
>
> >> At the moment, it's impossible for even the core Beaker team to help
> >> with this (let alone anyone else that may be interested), as we don't
> >> even know how to get set up to run it.
> >
> > Yeah, I was trying to figure out a workflow that I wanted to run autotest
> > with.
> >
> > Initially I wanted to run tests locally for developers and through
> > automation for QE folks. I can run autotest tests for developers and
> > beaker tests for QE folks. Ideally I need to mix and match in both
> > directions. :-/
> >
> > It shouldn't be hard to do, just need to hack up some glue code to detect
> > and run things properly.
> >
> >>
> >> I'd also like to submit a talk on Beaker to the Continuous Integration
> >> miniconf at LCA, and that becomes a lot more compelling when I can point
> >> to a post about *how* to run autotest inside Beaker, rather than just to
> >> the pull request that made it possible.
> >
> > Yeah, my stuff needs more work. And more thought. Perhaps some feedback
> > about workflows you had in mind, might help target this a little more.
> >
> > I'll keep poking at it this week.
>
> Hopefully the above helps - it sounds to me like a way to provide an XML
> definition directly for local testing + the "autotest-beaker-harness"
> RPM to integrate with the alternative harness selection mechanism are
> the key next steps.
>
> Then on to the fun part of getting non-Beaker autotest tests running
> inside Beaker :)
Yeah, getting there. For now how about a beaker patch?
Cheers,
Don
--------------------8<----------------
From: Don Zickus <dzickus at redhat.com>
Date: Fri, 8 Nov 2013 14:03:34 -0500
Subject: [PATCH] Tasks: make reservesys always exit 0
Different harnesses have a different environment than what reservesys
was coded for. As a result, the task may fail for something silly
/usr/bin/killall rhts-test-runner.sh
rhts-test-runner.sh: no process found
because the test harness I am using, doesn't run rhts-test-runner.sh.
I do not believe there is any reason to have reservesys fail. The
test was built to always succeed. Force that success with an
'exit 0' at the end.
Signed-off-by: Don Zickus <dzickus at redhat.com>
---
Tasks/distribution/reservesys/runtest.sh | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/Tasks/distribution/reservesys/runtest.sh b/Tasks/distribution/reservesys/runtest.sh
index 3a79b90..b4353e3 100755
--- a/Tasks/distribution/reservesys/runtest.sh
+++ b/Tasks/distribution/reservesys/runtest.sh
@@ -242,3 +242,6 @@ RprtRslt $TEST PASS 0
# stop rhts service, So that reserve workflow works with test reboot support.
STOPRHTS
+
+# different harness may cause this script to fail, always return zero
+exit 0
--
1.7.1
More information about the Beaker-devel
mailing list