On Tue, Nov 05, 2013 at 12:00:59PM +1000, Nick Coghlan wrote:
> will request BEAKER_RECIPE_ID xml from BEAKER_LAB_CONTROLLER_URL, convert
> it into a control file for autotest to consume and start running tests
> (and reporting them to beaker).
OK, that sounds reasonable (especially once the RPM packaging is brought
up to date)
>> 2. What do you need to set as the alternative harness in
> I need 'AUTOTEST_HARNESS=beaker' set.
OK, in order to get the
(ks_meta="harness=<package-or-URL-to-yum-install>") recipe attribute to
work properly, it looks like we may also need a
"autotest-beaker-harness" RPM that:
Yeah, Bill explained (and showed) how that works. So now I have the right
variables. Though I was caught by surprise when my tasks timed out
because nothing communicated to beaker that distribution-install was
1. Depends on autotest itself
2. Sets "AUTOTEST_HARNESS=beaker" in the system environment variables
3. Arranges for "autotest-local bootstrap" to run on the next reboot
Yeah, I will sort this out a little later. Using Bill's idea of the %post
section is probably a good start for now.
Otherwise you'll need to write a Beaker task that installs beah just to
bootstrap autotest, and installing beah is one of the things using an
alternative harness is supposed to avoid.
>> 3. What's known to work, what's still quirky? (e.g. reboot)
> reboot doesn't work for some reason. I don't think running actual
> autotest-tests work. I have only run with beaker tests through a job.
> And also running locally doesn't work (even though the tests are locally
> installed [unless you hack up some recipe id workarounds..]).
We're going to have to get the reboot glitch sorted in order to
interoperate with the alternative harness selection mechanism.
>> 4. What's different (if anything) compared to normal usage of autotest?
> Normal usage of autotest is 'autotest-local <some control file>' or
> 'autotest-local run <testname>' (where testname is from
> But that is running autotest locally/offline. I can get it to work with
> beaker tests, but I have to tweak some env variables to fake it (not
> ideal). So run now it has to run automated, which isn't ideal either.
The current recommendation for local beah testing is to assume offline
execution if JOBID isn't set:
Something similar may work here: if AUTOTEST_HARNESS=beaker is set, but
the BEAKER_* environment variables are missing, perhaps there could be
some way to provide the recipe definition XML file directly rather than
needing to retrieve it from the server?
Well, we still have /root/RECIPE.txt and LABC_URL. Combine those pieces
and one can figure out how to get the xml file in an automated way.
>> At the moment, it's impossible for even the core Beaker team to help
>> with this (let alone anyone else that may be interested), as we don't
>> even know how to get set up to run it.
> Yeah, I was trying to figure out a workflow that I wanted to run autotest
> Initially I wanted to run tests locally for developers and through
> automation for QE folks. I can run autotest tests for developers and
> beaker tests for QE folks. Ideally I need to mix and match in both
> directions. :-/
> It shouldn't be hard to do, just need to hack up some glue code to detect
> and run things properly.
>> I'd also like to submit a talk on Beaker to the Continuous Integration
>> miniconf at LCA, and that becomes a lot more compelling when I can point
>> to a post about *how* to run autotest inside Beaker, rather than just to
>> the pull request that made it possible.
> Yeah, my stuff needs more work. And more thought. Perhaps some feedback
> about workflows you had in mind, might help target this a little more.
> I'll keep poking at it this week.
Hopefully the above helps - it sounds to me like a way to provide an XML
definition directly for local testing + the "autotest-beaker-harness"
RPM to integrate with the alternative harness selection mechanism are
the key next steps.
Then on to the fun part of getting non-Beaker autotest tests running
inside Beaker :)
Yeah, getting there. For now how about a beaker patch?
From: Don Zickus <dzickus(a)redhat.com>
Date: Fri, 8 Nov 2013 14:03:34 -0500
Subject: [PATCH] Tasks: make reservesys always exit 0
Different harnesses have a different environment than what reservesys
was coded for. As a result, the task may fail for something silly
rhts-test-runner.sh: no process found
because the test harness I am using, doesn't run rhts-test-runner.sh.
I do not believe there is any reason to have reservesys fail. The
test was built to always succeed. Force that success with an
'exit 0' at the end.
Signed-off-by: Don Zickus <dzickus(a)redhat.com>
Tasks/distribution/reservesys/runtest.sh | 3 +++
1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/Tasks/distribution/reservesys/runtest.sh
index 3a79b90..b4353e3 100755
@@ -242,3 +242,6 @@ RprtRslt $TEST PASS 0
# stop rhts service, So that reserve workflow works with test reboot support.
+# different harness may cause this script to fail, always return zero