Future directions for the Beaker test harness
by Nick Coghlan
A few weeks back, Amit put together a proof of concept for running the
test harness in a container, rather than directly on the host
(http://gerrit.beaker-project.org/#/c/3199).
That proof of concept relies on restraint, the new reference harness,
that is intended to eventually replace beah
(https://beaker-project.org/dev/proposals/reference-harness.html)
At the same time, I don't think restraint is currently getting the level
of review and testing that it needs to mature into a plausible
replacement for beah as the default harness.
I think Amit's proposed patch provides a possible way forward:
1. Accept the initial approach where restraint is the *only* supported
harness when running inside a container. Specifying both
"contained_harness" and "harness" as ks_meta variables should be an
error at this point (side note: 'harness' should also be documented
along with all the other ks_meta variables, with a link to
https://beaker-project.org/docs/alternative-harnesses/).
2. Recommend publishing both beah *and* restraint in the harness repos
for Beaker installations. This will not only make restraint available
for container based testing, but also make it readily available via
"harness=restraint" for normal testing, without needing to add a custom
repo definition.
3. Once we have container based testing working reliably with restraint,
drop the restriction against using alternative harnesses in containers.
The priority at the moment though is to get something working that can
run on an Atomic Host, and still provide a relatively normal execution
environment for the executed tasks. Supporting alternative harnesses
*inside* containers is a nice-to-have that can wait until later - by
flatly disallowing it, we ensure we don't have to spend any time working
on container related issues that don't impact restraint. For the initial
iteration, we can also ignore the question of choosing the base image
used to run the harness, as well as being able to start and stop other
containers on the host.
I've filed an RFE for 0.18 on that basis:
https://bugzilla.redhat.com/show_bug.cgi?id=1131388
As part of this, we may also want to move restraint from Bill's personal
account on GitHub to the main Beaker project account, but I don't think
that's particularly urgent at this point.
Regards,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
HSS Provisioning Architect
6 years
Beaker inventory: smolt & lshw comparison
by Amit Saha
Hi all,
As part of the migration to "lshw" for beaker's inventory task, I ran a comparison of the "smolt"
based tasks with the "lshw" based task.
The "raw" data is available in the "comparison.html" file under each arch sub-directory at:
https://amitksaha.fedorapeople.org/beaker-inventory-comparison/comparison...
Here are the summarized differences:
** i686/AMD/athlon (32-bit system)
CPU flags
=========
* lshw categorises "fpu_exception" as a CPU flag, adds "wp" as cpu flag too.
(This is because we assume all the stuff lshw lists as CPU capabilities,
we consider them flags)
Others
======
* USBID retrieved by lshw with repetition, smolt didn't retrieve any
* Same PCIID retrieved by lshw, however there is repetition of a device (for PCIID as well)
* lshw is not able to detect the system model (same as smolt)
* lshw sets arch to i686 instead of i386 by smolt
* lshw fails to get the system vendor (same as smolt)
* lshw fails to get the FORMFACTOR
* CPUVENDOR obtained by smolt is AuthenticAMD, lshw sets it to "Advanced Micro Devices [AMD]"
** ia64
CPU flags
=========
* No CPU flags retrieved by lshw
Others
======
* Bulk of the USB IDs retrieved are 0000:0000s
* lshw is not able to detect the system model
* CPUFAMILY is reported by smolt as 1, by lshw as 2
* lshw fails to get the system vendor
* lshw fails to get the FORMFACTOR, same as smolt
* CPUVENDOR obtained by smolt is genuineintel, lshw sets it to "Intel corp."
* CPU stepping obtained by smolt is 0, whereas it is None by lshw
* PCIIDS retrieved by lshw contain one missing PCIID and a number of repetitions
* Memory reported by smolt is 7863 MB, 8192 by smolt. (**Needs more investigation**)
** ppc64/Power 6
CPU flags
=========
* No CPU flags retrieved by either
Others
======
* USBID retrieved by lshw, none by smolt, contains one repetition
* lshw is not able to detect the system model, same as smolt
* lshw fails to get the system vendor
* lshw reports a repeating PCIID (same as smolt)
** ppc64/Power 7
CPU flags
=========
* CPU flags retrieved by lshw as "performance-monitor"
Others
======
* No USB devices reported by either
* lshw is not able to detect the system model, same as smolt
* lshw fails to get the system vendor
* lshw and smolt both doesn't report any PCIIDs
** s390
CPU flags
=========
* CPU flags reported by lshw are each duplicated
Others
======
* No USBIDs retrieved by lshw, none by smolt
* lshw is not able to detect the system model, same as smolt
* lshw fails to get the system vendor, same as smolt
* lshw reports a repeating PCIID (same as smolt)
* Memory is reported as 2001 MB by both, system has 2048 MB
** x86_64/amd
CPU flags
=========
* lshw uses x86-64 in place of lm
* lshw categorises "fpu_exception" and "cpufreq" as a CPU flags
(This is because we assume all the stuff lshw lists as CPU capabilities,
we consider them flags)
Others
======
* More USBID retrieved by lshw, however there is repetition of a device (for PCIID as well)
* lshw is not able to detect the system model
* lshw sets VIRT_IOMMU to False, whereas smolt has it to True
* lshw sets arch to x86-64 instead of x86_64
* smolt gets CPUMODEL as " AMD Eng Sample, 1S256146U8K54_35/25/20_2/8 ", lshw gets it as "Opteron"
* lshw fails to get the system vendor
* lshw fails to get the FORMFACTOR
* CPUVENDOR obtained by smolt is AuthenticAMD, lshw sets it to "Advanced Micro Devices [AMD]"
* smolt gets 3811 MB of memory, lshw gets None on an 3919 MB system (**Needs more investigation**)
** x86_64/intel
CPU flags
=========
* lshw uses x86-64 in place of lm
* lshw categorises "fpu_exception" and "cpufreq" as a CPU flags
(This is because we assume all the stuff lshw lists as CPU capabilities,
we consider them flags)
Others
======
* More USBID retrieved by lshw, however there is repetition of a device
* lshw is not able to detect the system model
* lshw sets arch to x86-64 instead of x86_64
* smolt gets CPUMODEL as "Intel(R) Xeon(TM) CPU 3.60GHz", lshw gets it as "Xeon"
* lshw fails to get the system vendor
* CPUVENDOR/vendor obtained by smolt is GenuineIntel, lshw sets it to Intel Corp.
* lshw gets 4096 MB of memory correctly
** x86_64/kvm guest
CPU flags
=========
* lshw uses x86-64 in place of lm
* lshw categorises "fpu_exception" as a CPU flag
(This is because we assume all the stuff lshw lists as CPU capabilities,
we consider them flags)
Others
======
* USBID retrieved by lshw, none by smolt
* lshw is not able to detect the system model
* lshw sets arch to x86-64 instead of x86_64
* lshw fails to get the system vendor
* CPUVENDOR/vendor obtained by smolt is GenuineIntel, lshw sets it to Intel Corp.
* lshw fails finds one device less (PCIID)
* lshw gets 2048 MB of memory correctly
** Common to all archs
For all the architectures, the devices list returned by the lshw task is fewer than those
returned by smolt. The devices need to be individually compared to come up with a more accurate
answer.
PS: arm 32-bit and 64-bit comparison: Not possible, since we cannot run any smolt supported distro
on these.
The next plan of action is to fix the issues that we have found above - either in "beaker-system-scan"
or "lshw" itself.
Best,
Amit.
--
Amit Saha
SED, Hosted & Shared Services
Red Hat, Inc.
8 years, 11 months
Task /distribution/reservesys is broken on Fedora21
by Jirka Hladky
Hello,
extendtesttime.sh does not work on Fedora21 Server:
[root@karkulka-02 tests]# extendtesttime.sh 48
Extending reservation time 48
ERROR: 2 args provided
Usage:
rhts-test-checkin <lab_server> <hostname> <jobid> <test> <killtime>
<testid>
root: rhts-test-checkin karkulka-02.lab.eng.brq.redhat.com 48h
How to reproduce?
Privison in Beaker server with "Fedora-Server-21_Beta Server x86_64".
After installation, login on the server and try to run extendtesttime.sh
Bellow is an example XML generated by Beaker "Provision" command.
Thanks
Jirka
<job retention_tag="scratch">
<whiteboard>
Provision Fedora-Server-21_Beta Server x86_64
</whiteboard>
<recipeSet priority="High">
<recipe kernel_options="console=tty0 console=ttyS1,115200n81
ksdevice=C8:1F:66:F4:15:C4 rdblacklist=mtip32xx" kernel_options_post=""
ks_meta="docker_package=docker-io end=%end grubport=0x02f8
has_autopart_type=True has_chrony=True has_leavebootorder=True
has_repo_cost=True has_systemd=True has_unsupported_hardware=True"
role="RECIPE_MEMBERS" whiteboard="">
<autopick random="false"/>
<watchdog panic="ignore"/>
<packages/>
<ks_appends/>
<repos/>
<distroRequires>
<and>
<distro_family op="=" value="Fedora21"/>
<distro_variant op="=" value="Server"/>
<distro_name op="=" value="Fedora-Server-21_Beta"/>
<distro_arch op="=" value="x86_64"/>
</and>
</distroRequires>
<hostRequires>
<and>
<hostname op="="
value="karkulka-02.lab.eng.brq.redhat.com"/>
<system_type op="=" value="Machine"/>
</and>
</hostRequires>
<partitions/>
<task name="/distribution/install" role="STANDALONE"/>
<task name="/distribution/reservesys" role="STANDALONE">
<params>
<param name="RESERVETIME" value="86400"/>
</params>
</task>
</recipe>
</recipeSet>
</job>
9 years, 3 months
Group SSH keys on manually provisioned systems
by Nick Coghlan
Is there a way to get a whole group's SSH keys onto a system without
going through the scheduler? Or is manual provisioning currently
restricted to only added the SSH key of the user provisioning the system?
Cheers,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
HSS Provisioning Architect
9 years, 4 months
Requesting reservation of a previously submitted recipe?
by Nick Coghlan
It occurred to me today that with the server side reservation system
added in 0.17, that actually becomes something that could be made
configurable for a previously submitted recipe.
The kinds of cases where that seemed potentially interesting to me were:
* After manually kicking a recipe that stalled for some reason (the case
that prompted the idea)
* After noticing something odd in the execution of a long running recipe
* Switching from "always" to "only on failure" if you forget to make it
conditional (or vice-versa)
* Adding or removing the reservation request if simply made a mistake at
submission time
There'd be a bit fiddling involved - new CLI commands to manage it, new
web UI elements on the job details page.
If we did it at all, perhaps it would be best left until after the
results page redesign?
Cheers,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
HSS Provisioning Architect
9 years, 4 months
Subtest support in unittest2!
by Nick Coghlan
Robert Collins has backported one of my favourite Python 3 features to
unittest2: subtests!
See the Python 3 docs [1] for details, but the basic idea is to let you
easily split up a data driven test such that:
1. All iterations execute, even if some of the checks fail
2. Each failure is reported separately, with relevant details you provide
For example:
===================
class NumbersTest(unittest.TestCase):
def test_even(self):
"""
Test that numbers between 0 and 5 are all even.
"""
for i in range(0, 6):
with self.subTest(i=i):
self.assertEqual(i % 2, 0)
===================
Will check all values from 0 to 5, and report separate failures for 1, 3
and 5.
The only other particular notable new feature is the addition of the
"assertLogs" context manager to test cases, which makes it easier to
check logging within the current process is performed correctly as part
of unit tests.
Cheers,
Nick.
[1]
https://docs.python.org/3/library/unittest.html#distinguishing-test-itera...
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
HSS Provisioning Architect
9 years, 4 months
Using shell "if" to conditionalize snippet inclusion is error-prone?
by Jun'ichi Nomura
Hi,
In beaker 0.18, "rhts_post" snippet has a following construct:
if [ -f /etc/sysconfig/readahead ] ; then
{% snippet 'readahead_sysconfig' %}
fi
This is error-prone because if we have site-local "readahead_sysconfig"
snippet, which is empty, generated shell script will be broken.
(Actually, I've been using empty "readahead_sysconfig" snippet)
So it might be better to move if/then/fi into the snippet like below.
Same might apply to "virt_console_post".
---
Jun'ichi Nomura, NEC Corporation
diff --git a/Server/bkr/server/snippets/readahead_sysconfig b/Server/bkr/server/snippets/readahead_sysconfig
index 830f20b..393c215 100644
--- a/Server/bkr/server/snippets/readahead_sysconfig
+++ b/Server/bkr/server/snippets/readahead_sysconfig
@@ -1,3 +1,4 @@
+if [ -f /etc/sysconfig/readahead ] ; then
cat >>/etc/sysconfig/readahead <<EOF
# readahead conflicts with auditd, see bug 561486 for detailed explanation.
@@ -8,3 +9,4 @@
READAHEAD_COLLECT="no"
READAHEAD_COLLECT_ON_RPM="no"
EOF
+fi
diff --git a/Server/bkr/server/snippets/rhts_post b/Server/bkr/server/snippets/rhts_post
index bce2347..1635406 100644
--- a/Server/bkr/server/snippets/rhts_post
+++ b/Server/bkr/server/snippets/rhts_post
@@ -15,9 +15,7 @@ fi
# Enable post-install boot notification
{% snippet 'post_anamon' %}
-if [ -f /etc/sysconfig/readahead ] ; then
{% snippet 'readahead_sysconfig' %}
-fi
{% snippet 'linkdelay' %}
{# We normally want to make sure the system time is accurate, in case
9 years, 4 months