Adding resource management to beaker
by Bill Peck
Hello Beaker,
One of the jobs we run uses shared storage. For example:
HostA with Driver1
HostB with Driver2
HostC with Driver3
All connected to StorageX.
We can't run the test on more than one host at a time since it would
conflict.
Being able to specify the following in the job xml is the goal:
<recipeSet>
<recipe/> <!-- Require HostA -->
<resource/> <!-- Require StorageX -->
</recipeSet>
<recipeSet>
<recipe/> <!-- Require HostB -->
<resource/> <!-- Require StorageX -->
</recipeSet>
<recipeSet>
<recipe/> <!-- Require HostC -->
<resource/> <!-- Require StorageX -->
</recipeSet>
I know there is a resource type for recipe but this doesn't seem like a
good solution. I did try an experiment where I created StorageX host in
beaker with no power control and scheduled three recipesets like above.
But this hack won't work because a watchdog with a NULL expire time is
set and that means it will sit there forever. Even if it did work,
maybe if we created a dummy power type, we would still end up with an abort.
I don't think it would take much work to support this type of
multi-host/resource requirement.
Right now we have the following model
Object Recipe, which has the following attributes: distro_tree,
resource, rendered_kickstart, watchdog, systems, dyn_systems, tasks,
dyn_tasks, tags, repos, rpms, logs, custom_packages, ks_appends
Object MachineRecipe which inherits from Recipe, and adds a guests
attribute.
Object GuestRecipe which inherits from Recipe.
A ResourceRecipe object wouldn't need most of these attributes, probably
just the following:
resource, systems, and dyn_systems. (no watchdog, just return when
recipeSet completes or aborts)
And resource currently maps to a RecipeResource which can be one of the
following:
SystemResource
VirtResource
GuestResource
I think adding another Object here would make sense, but the term
Resource is overused and its a little confusing right now.
If done correctly we could re-use the filtering code in needproperty,
but I think the only things we could search on for resources would be
the following: Name, lab_controller, and key_values? (storing how these
shared resources are connected would be another option I suppose, but
maybe too complicated for a first implementation?)
I'm sure there are other things to consider, we could support setting up
and tearing down shared resources (almost like a power on/off command).
I want to be mindful of things like that but I don't want to be
paralysed with what if's and never get anything done.
I know you guys have been working hard on the groups features and want
to re-do the scheduler as well. Maybe this would be something for after
the scheduler re-write? I don't see any mention of this on the roadmap
[1] (although the roadmap is quite full already!).
In any event, I'm interested in hearing your ideas and feedback.
Thanks!
1 - http://beaker-project.org/dev/tech-roadmap.html
9 years, 5 months
External tasks proposal up on beaker-project.org
by Nick Coghlan
I decided it made more sense to publish Bill's external tasks library on
beaker-project.org and start a thread here, rather than continue to
conduct the high level design discussions in the Gerrit review.
Firstly, I like the idea of "fetch" and "rpm" nodes in the task XML. The
proposal should be more explicit about the fact that the RPM node is
only present in the recipe XML when it is retrieved through the stable
harness API, and not in other cases (such as when cloning a job).
I *don't* like the way the proposal refers to *removing* the task
library. The task library isn't going anywhere, we're just providing a
way to bypass it when doing so makes sense. The task library is still an
incredibly convenient way to publish utilities like the standard tasks
(http://beaker-project.org/docs/user-guide/beaker-provided-tasks.html),
and potentially future higher level utilities like properly configured
KDC, LDAP and qpid servers.
Finally, I'd prefer it if the proposal was backed by a core
"/distribution/gitfetch" task, together with the ability to flag a task
in the task library as "external". When a task was flagged as external,
then Beaker would *require* that a "fetch" element exist, and would
display that URL when appropriate. Given the UI issues in the Beaker
0.15 upgrade, I'm concerned about breaking the fundamental assumption of
every task being associated with an entry in the task library. By using
an "external" flag and an implied task name (instead of leaving the task
library reference NULL), it changes the failure mode from failing to
handle an unexpected NULL or None value to display the task name as
usual rather than the desired URL details.
Regards,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)
10 years, 1 month
OpenStack based dynamic virtualization
by Nick Coghlan
We've been talking about possibly changing the existing dynamic
virtualization to work with OpenStack instead of oVirt for a while, and
I've now hammered that into a hopefully coherent concrete proposal:
http://beaker-project.org/dev/proposals/dynamic-virtualization.html
The key points in there are too:
- associate lab controllers with OpenStack regions
- map host filtering to flavour filtering
- generate bootstrap images in Glance as part of distro import
- use the bootstrap images for dynamic provisioning (rather than
configuring netboot and then rebooting the system)
- ensure console logging is hooked up properly
Provisioning based on specific images is explicitly deferred, since we'd
have to map a bunch of the stuff we do in the kickstart post to
cloud-init, and I'd prefer to put that off to a later iteration. By
contrast, I think everything above is essential for dynamic
virtualisation to be as good as what people are used to with bare metal
provisioning (the lack of console logging with no obvious way to add it
was one of the final nails in the coffin of the previous oVirt based
approach).
Cheers,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)
10 years, 3 months
/virt/install problems with guest console logs
by Jan Stancek
Hi,
I was looking into problem where we get duplicated output on RHEL5 Xen
guest console logs after reboot and while testing my patch I found 2 more issues.
I made a patch for problem 1, but I'm not sure what we can do about 2 and 3.
Do we still need rhts-compat? Is there an option to turn it off so people could
try to run their tests without it and check if any tests need it?
Problem 1: Duplicate output in console logs on RHEL5 xen guests
----------------------------------------------------------------
http://gerrit.beaker-project.org/#/c/2630/
Problem 2: second serial port on guest is suffering from plymouth interference
-------------------------------------------------------------------------------
See this Bug for details:
Bug 1043261 - plymouth breaks serial console on kvm guest
And by the looks of it, it's problem with rhts-compat never completing:
https://bugzilla.redhat.com/show_bug.cgi?id=1043261#c2
Problem 3: Duplicate output in console logs on RHEL6
-----------------------------------------------------
After this BZ:
Bug 978419 - RFE: Change in rhel6 guest kickstarts
we have following in kickstarts: --append="console=ttyS0,115200 console=ttyS1,115200",
which is problem, because we also pass "serial". Anaconda implicitly adds one more
"console=ttyS0" if it finds "serial" on command line.
For example:
If you look at console.log, you can see that lines get duplicated:
2013-12-15 08:16:30,165 beah start_server: INFO ################################
2013-12-15 08:16:30,165 beah start_server: INFO ################################
2013-12-15 08:16:30,176 beah start_server: INFO # Starting a Controller... #
2013-12-15 08:16:30,176 beah start_server: INFO ################################
2013-12-15 08:16:30,176 beah start_server: INFO Controller: BackendListener listening on :12432
2013-12-15 08:16:30,176 beah start_server: INFO # Starting a Controller... #
2013-12-15 08:16:30,176 beah start_server: INFO ################################
2013-12-15 08:16:30,176 beah start_server: INFO Controller: BackendListener listening on :12432
Problem is again that plymouth keeps running and duplicates
all output on console for each "console=ttyS0" it finds on kernel command line.
Normally plymouth would be stopped when all runlevel 5 services start,
but because rhts-compat runs forever then also plymouth runs forever:
https://bugzilla.redhat.com/show_bug.cgi?id=1043261#c2
Regards,
Jan
10 years, 3 months
Ending RHEL5 client RPM updates?
by Nick Coghlan
The Beaker 0.15.2 release won't have an associated RHEL5 client RPM
update - it turns out we have a conditional expression (if/else) in the
current client code, and that's a syntax error on Python 2.4.
It isn't actually a functional loss for any users running the client on
RHEL5, since new Beaker CLI commands don't work in the RHEL5 client
anyway - they all rely on the requests module, and that is Python 2.6+ only.
So, the question at this point is: do we just declare 0.15.1 the last
RHEL5 compatible client release, and promise to maintain backwards
compatibility of the XML-RPC interfaces it relies on until we're
prepared to drop RHEL5 client support entirely? This would mean checking
that the client commands in 0.15.1 client all still work with new server
releases, but those tests could be run on any client system rather than
needing to be specifically run on RHEL5.
The alternative is to start testing RHEL5 more consistently earlier in
the development cycle, and that seems like a lot of work for next to no
gain.
Cheers,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)
10 years, 3 months
Disambiguation of beaker.fedoraproject.org
by Tim Flink
There still seems to be a bit of confusion as to the current state of
the beaker deployment that we've been working on for Fedora.
To improve clarity on the exact state of the current install, I've made
some changes to what was formerly beaker.fedoraproject.org:
- The beaker server is now available at beaker-dev.fedoraproject.org
- beaker.fedoraproject.org points to a static html page explaining
what beaker-dev.fedoraproject.org is, how to reach it and that it is
currently access restricted while the security issues are dealt with.
I've also enabled SSL with self-signed certs and disallowed non-https
traffic to the beaker server but the previous IP access restrictions
still apply.
Tim
10 years, 3 months
Docs/overview for using autotest in Beaker?
by Nick Coghlan
Hi Don, Bill, LMR,
I saw that the pull request for Beaker harness API support in autotest has been merged. I'd like to link to that feature from beaker-project.org, so I was wondering if there was anything published anywhere (even a blog or mailing list post) that covered the details of:
1. What to add to a job/recipe definition to specify autotest as the harness
2. What, if anything, to add to the task list in the recipe to configure autotest appropriately
3. What, if anything, to include in an autotest job to make it suitable for running inside Beaker
4. How the result reporting works when using autotest as a harness inside Beaker (Do the results just go to Beaker? Or does normal autotest reporting still happen as well?)
If there isn't, we'd be happy to accept a patch on Gerrit to add an autotest section to http://beaker-project.org/docs/alternative-harnesses/ :)
Somewhat related, is there a tentative date yet for the next autotest release?
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Testing Solutions Team Lead
Beaker Development Lead (http://beaker-project.org/)
10 years, 3 months
Reorganising the system page
by Dan Callaghan
For bug 1014438 I am working on reorganising the system page to use
space more efficiently and to make things easier to find.
Attached are two screenshots of my ideas.
The top section ("system at a glance") gives an overview of what the
system is and its current state. The idea is to display the most
commonly needed information and the most commonly performed actions at
the top, in a compact layout. It occupies much less vertical space than
the system form which currently appears at the top of the system page.
The three columns in the "at a glance" section are: description, health,
and current usage. The health column has either a "Report problem"
button or a "Remove from service" button according to context. Similarly
the current usage column shows "Reserve", "Take", or "Request loan"
according to context.
The tabs in the sidebar are all the familiar tabs that exist right now
on the system page. There are also a few additions, to cover
functionality that is currently crammed awkwardly into the system form.
The rest remain largely unchanged so that this doesn't need to turn into
a massive rewrite.
Let me know what you think.
--
Dan Callaghan <dcallagh(a)redhat.com>
Software Engineer, Hosted & Shared Services
Red Hat, Inc.
10 years, 3 months
New sub-command to list access policy for a system
by Amit Saha
I noticed that there is no sub-command to list the existing policy for a system, so
tried writing one [1]. My primary objective is to of course having it as a sub-command,
unless there was another reason other than time that it wasn't added earlier.
The second objective is to use it as the second example in our developer guide
to show how you expose controller methods and write a client to access it.
Example usage:
# bkr policy-list --system beaker-test-vm1 --hub http://localhost/bkr --username admin --password foobar
{
"rules": [
{
"everybody": true,
"permission": "control_system",
"group": null,
"id": 1,
"user": null
},
{
"everybody": true,
"permission": "reserve",
"group": null,
"id": 4,
"user": null
}
],
"possible_permissions": [
{
"value": "edit_policy",
. ..
. ..
The authentication bit is not required I suspect.
[1] http://paste.fedoraproject.org/41757/13800067/
Thoughts and suggestions welcome.
Best,
Amit.
--
Amit Saha <http://echorand.me>
Infrastructure Engineering and Development
Red Hat, Inc.
10 years, 3 months