Hello Beaker,
One of the jobs we run uses shared storage. For example:
HostA with Driver1
HostB with Driver2
HostC with Driver3
All connected to StorageX.
We can't run the test on more than one host at a time since it would
conflict.
Being able to specify the following in the job xml is the goal:
<recipeSet>
<recipe/> <!-- Require HostA -->
<resource/> <!-- Require StorageX -->
</recipeSet>
<recipeSet>
<recipe/> <!-- Require HostB -->
<resource/> <!-- Require StorageX -->
</recipeSet>
<recipeSet>
<recipe/> <!-- Require HostC -->
<resource/> <!-- Require StorageX -->
</recipeSet>
I know there is a resource type for recipe but this doesn't seem like a
good solution. I did try an experiment where I created StorageX host in
beaker with no power control and scheduled three recipesets like above.
But this hack won't work because a watchdog with a NULL expire time is
set and that means it will sit there forever. Even if it did work,
maybe if we created a dummy power type, we would still end up with an abort.
I don't think it would take much work to support this type of
multi-host/resource requirement.
Right now we have the following model
Object Recipe, which has the following attributes: distro_tree,
resource, rendered_kickstart, watchdog, systems, dyn_systems, tasks,
dyn_tasks, tags, repos, rpms, logs, custom_packages, ks_appends
Object MachineRecipe which inherits from Recipe, and adds a guests
attribute.
Object GuestRecipe which inherits from Recipe.
A ResourceRecipe object wouldn't need most of these attributes, probably
just the following:
resource, systems, and dyn_systems. (no watchdog, just return when
recipeSet completes or aborts)
And resource currently maps to a RecipeResource which can be one of the
following:
SystemResource
VirtResource
GuestResource
I think adding another Object here would make sense, but the term
Resource is overused and its a little confusing right now.
If done correctly we could re-use the filtering code in needproperty,
but I think the only things we could search on for resources would be
the following: Name, lab_controller, and key_values? (storing how these
shared resources are connected would be another option I suppose, but
maybe too complicated for a first implementation?)
I'm sure there are other things to consider, we could support setting up
and tearing down shared resources (almost like a power on/off command).
I want to be mindful of things like that but I don't want to be
paralysed with what if's and never get anything done.
I know you guys have been working hard on the groups features and want
to re-do the scheduler as well. Maybe this would be something for after
the scheduler re-write? I don't see any mention of this on the roadmap
[1] (although the roadmap is quite full already!).
In any event, I'm interested in hearing your ideas and feedback.
Thanks!
1 - http://beaker-project.org/dev/tech-roadmap.html
Hi Don, Bill, LMR,
I saw that the pull request for Beaker harness API support in autotest has been merged. I'd like to link to that feature from beaker-project.org, so I was wondering if there was anything published anywhere (even a blog or mailing list post) that covered the details of:
1. What to add to a job/recipe definition to specify autotest as the harness
2. What, if anything, to add to the task list in the recipe to configure autotest appropriately
3. What, if anything, to include in an autotest job to make it suitable for running inside Beaker
4. How the result reporting works when using autotest as a harness inside Beaker (Do the results just go to Beaker? Or does normal autotest reporting still happen as well?)
If there isn't, we'd be happy to accept a patch on Gerrit to add an autotest section to http://beaker-project.org/docs/alternative-harnesses/ :)
Somewhat related, is there a tentative date yet for the next autotest release?
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)
I noticed that there is no sub-command to list the existing policy for a system, so
tried writing one [1]. My primary objective is to of course having it as a sub-command,
unless there was another reason other than time that it wasn't added earlier.
The second objective is to use it as the second example in our developer guide
to show how you expose controller methods and write a client to access it.
Example usage:
# bkr policy-list --system beaker-test-vm1 --hub http://localhost/bkr --username admin --password foobar
{
"rules": [
{
"everybody": true,
"permission": "control_system",
"group": null,
"id": 1,
"user": null
},
{
"everybody": true,
"permission": "reserve",
"group": null,
"id": 4,
"user": null
}
],
"possible_permissions": [
{
"value": "edit_policy",
. ..
. ..
The authentication bit is not required I suspect.
[1] http://paste.fedoraproject.org/41757/13800067/
Thoughts and suggestions welcome.
Best,
Amit.
--
Amit Saha <http://echorand.me>
Infrastructure Engineering and Development
Red Hat, Inc.
I am beginning to think why don't we maintain the Beaker tasks in a separate repository?
Each task has and will have their own version/release info, so there won't be a "release"
of the tasks per se.
We have made task upgrades in the past at various points during the release cycle, so there
doesn't seem to be any pressing reason not to do this.
Thoughts?
Best,
-Amit.
--
Amit Saha <http://echorand.me>
Infrastructure Engineering and Development
Red Hat, Inc.
Currently, Beaker's default test harness is Beah [1]. This post will attempt to
summarize how Beah is currently developed, released and upgraded with the
rationale explained at the end. To learn briefly about Beah in the context of
Beaker, please refer to the architecture guide [2].
*Current development process*
Bugs against Beah are currently filed using the "test harness"
component under Beaker's bugzilla. All development happens on the
"master" branch.
Here is a brief overview of what is the current approach of contributing a feature,
a bug fix or another change to Beah:
- Clone beah: git clone git://git.beaker-project.org/beah
- Create a new branch for the changes, say mybranch
- Once you are happy with the changes, push your branch to gerrit:
git push git+ssh://gerrit.beaker-project.org:29418/beah mybranch:refs/for/master
- Once the change has been approved, the patch is merged to master
*Release and upgrade process*
Currently, the release process involves tagging a new release using
tito and pushing the tag to remote beah repository. Beah releases
happen at the same time as new Beaker releases. Beah packages are served from
a yum repository at [3]. If there is a new Beah release, the harness packages have
to be updated on the Beaker server using the beaker-repo-update utility.
*Existing Beah documentation*
These are the current "mentions" of Beah in the Beaker documentation.
- http://beaker-project.org/docs/architecture-guide/capabilities.html#test-ha…
- http://beaker-project.org/docs/admin-guide/upgrading.html#new-harness-packa…
- http://beaker-project.org/docs/user-guide/task-metadata.html
*Future plan*
As is fairly apparent from the above summary, the release cycle and documentation of
Beah is tightly coupled with Beaker's. In fact, currently there is no independent
documentation about what Beah is, where to file a bug against it or how code changes
can be contributed.
The rationale behind this post is to consolidate the current state
and use it as a base for decoupling Beah's documentation from Beaker's and gradually
adopt a separate release cycle for Beah. Once we have done that for Beah, we
hope to do the same for some of the other projects listed at
http://git.beaker-project.org/cgit/.
Thoughts, suggestions and comments welcome.
[1] http://git.beaker-project.org/cgit/beah/
[2] http://beaker-project.org/docs/architecture-guide/capabilities.html
[3] http://beaker-project.org/yum/harness/
Best,
Amit.
--
Amit Saha <http://echorand.me>
Infrastructure Engineering and Development
Red Hat, Inc.
I have created a 'develop' branch for Beah and hence new patches should now be
submitted against it.
Quick ref: git push git+ssh://gerrit.beaker-project.org:29418/beah yourbranch:refs/for/develop
Best,
Amit.
--
Amit Saha <http://echorand.me>
Infrastructure Engineering and Development
Red Hat, Inc.
On behalf of the Beaker development team, I'm happy to announce the release of Beaker 0.15.1!
It can be found on beaker-project.org[1].
This release fixes a number of regressions found in the 0.15.0 release, as well as some other
minor bug fixes. The full changelog can be see here[2].
Regards
Raymond.
1 http://beaker-project.org/releases/
2 http://beaker-project.org/releases/beaker-0.15.1-ChangeLog
I read an interesting article on the weekend about the collaborative
nature of corporate software development
(https://medium.com/better-humans/2e2ed4daeb99)
While the whole article was quite informative, what specifically caught
my attention were some of the comments on UI testing, especially in
conjunction with some comments Ray made on IRC about the challenages of
effectively reviewing UI changes on Gerrit.
Specifically:
1. Gerrit 2.7 [1] includes the ability to display image files inline in
the code review (showing the old and new versions adjacent to each other
rather than a diff)
2. Facebook's Huxley [2] is based on automatically taking screenshots
based on a webdriver test and committing them to Git
It seems to me we may be able to kill two birds with one stone here: by
creating Huxley tests for various parts of the UI, we'll get screenshot
changes to review as part of the commits, *and* potentially be able to
automate regression testing for cosmetic problems that are painful to
test with explicit WebDriver API calls.
The same article also referenced this UI acceptance testing tool which
looks potentially useful for acceptance testing purposes:
https://github.com/bslatkin/dpxdt
Unlike Huxley, which is designed to integrate with the normal code
review and automated regression testing processes for a project, dpxdt
(pronounced "depicted") is designed to run as a separate service,
capturing screenshots and generating perceptual diffs which are then
approved (or not) based on whether they represent deliberate or neutral
changes or UI regressions.
Regards,
Nick.
[1]
http://gerrit-documentation.googlecode.com/svn/ReleaseNotes/ReleaseNotes-2.…
[2] https://github.com/facebook/huxley
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)
Overall results:
964 tests, 5 failures, 0 errors, 3 skips in 3032.7s
Key packages and versions:
python-sqlalchemy-0.8.2-1.fc20.x86_64, python-requests-1.1.0-4.fc20.noarch,
python-formencode-1.2.6-2.fc20.noarch
I deliberately used python-requests-1.1.0-4.fc20.noarch, instead of python-requests-1.2.3-5.fc20 (latest on Fedora 20).
Failures:
These all result in assertion failures with the new version for python-formencode:
bkr.inttest.client.test_group_create:GroupCreateTest.test_group_create
bkr.inttest.client.test_group_modify:GroupModifyTest.test_group_modify_display_name
bkr.inttest.client.test_group_modify:GroupModifyTest.test_group_modify_group_name
bkr.inttest.server.selenium.test_group_edit:TestGroupsWD.test_check_group_name_display_name_length
The problem is the error message has been "fixed" in the new version, but it remains the same
on the older versions. Here is an example:
- Enter a value less than 255 characters long?
+ Enter a value not more than 255 characters long?
So shall we instead have an "or" condition here in the assertion so that it tests
against both possible error messages?
"Known" Fedora failure:
bkr.inttest.server.test_jobs:TestJobsController.test_job_xml_can_be_roundtripped
With python-requests-1.2.3-5.fc20, this patch is needed:
diff --git a/Client/src/bkr/client/__init__.py b/Client/src/bkr/client/__init__.py
index eb4a4fd..2414bf7 100644
--- a/Client/src/bkr/client/__init__.py
+++ b/Client/src/bkr/client/__init__.py
@@ -63,7 +63,7 @@ def request(self, method, url, **kwargs):
# turn 'json' parameter into a suitably formatted request
if 'json' in kwargs:
kwargs['data'] = json.dumps(kwargs.pop('json'))
- kwargs.setdefault('headers', []).append(('Content-Type', 'application/json'))
+ kwargs.setdefault('headers', {}).update({'Content-Type': 'application/json'})
return super(BeakerClientRequestsSession, self).request(method, url, **kwargs)
return BeakerClientRequestsSession()
Fortunately, this is compatible with 1.1, so we should get this patch merged.
Thoughts on the above suggested changes?
Best,
Amit.
--
Amit Saha <http://echorand.me>
Infrastructure Engineering and Development
Red Hat, Inc.
For Beaker 0.15, we migrated away from using tito to generate RPMs (we
wanted to start building release candidate RPMs and it was easier to
replace tito with some suitable scripts than it was to figure out how to
get tito to do what we wanted).
The new commands are in the Misc directory: rpmbuild.sh and tag-release.sh
Misc/rpmbuild.sh is just a thin wrapper around rpmbuild (use rmpbuild.sh
-bs and mock to generate RPMs for other versions)
$ ./rpmbuild.sh
Usage: -bs|-bb <rpmbuild-options...>
Hint: -bs builds SRPM, -bb builds RPM, refer to rpmbuild(8)
Misc/tag-release.sh handles the edits needed to bump the version number
and then commits and tags the result.
$ ./tag-release.sh
Usage: ./tag-release.sh <version> [<prerelease>]
Example: ./tag-release.sh 0.15.0 rc1
The latter change means that the "*.py.in" input files that were
previously used to inject the version are now gone, and Beaker should
run from a fresh clone without needing to run "make" first.
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)