Beaker-devel February 2014

beaker-devel@lists.fedorahosted.org

6 participants
10 discussions

Adding resource management to beaker
by Bill Peck 21 Oct '14

21 Oct '14

Hello Beaker, One of the jobs we run uses shared storage. For example: HostA with Driver1 HostB with Driver2 HostC with Driver3 All connected to StorageX. We can't run the test on more than one host at a time since it would conflict. Being able to specify the following in the job xml is the goal: <recipeSet> <recipe/>  <resource/>  </recipeSet> <recipeSet> <recipe/>  <resource/>  </recipeSet> <recipeSet> <recipe/>  <resource/>  </recipeSet> I know there is a resource type for recipe but this doesn't seem like a good solution. I did try an experiment where I created StorageX host in beaker with no power control and scheduled three recipesets like above. But this hack won't work because a watchdog with a NULL expire time is set and that means it will sit there forever. Even if it did work, maybe if we created a dummy power type, we would still end up with an abort. I don't think it would take much work to support this type of multi-host/resource requirement. Right now we have the following model Object Recipe, which has the following attributes: distro_tree, resource, rendered_kickstart, watchdog, systems, dyn_systems, tasks, dyn_tasks, tags, repos, rpms, logs, custom_packages, ks_appends Object MachineRecipe which inherits from Recipe, and adds a guests attribute. Object GuestRecipe which inherits from Recipe. A ResourceRecipe object wouldn't need most of these attributes, probably just the following: resource, systems, and dyn_systems. (no watchdog, just return when recipeSet completes or aborts) And resource currently maps to a RecipeResource which can be one of the following: SystemResource VirtResource GuestResource I think adding another Object here would make sense, but the term Resource is overused and its a little confusing right now. If done correctly we could re-use the filtering code in needproperty, but I think the only things we could search on for resources would be the following: Name, lab_controller, and key_values? (storing how these shared resources are connected would be another option I suppose, but maybe too complicated for a first implementation?) I'm sure there are other things to consider, we could support setting up and tearing down shared resources (almost like a power on/off command). I want to be mindful of things like that but I don't want to be paralysed with what if's and never get anything done. I know you guys have been working hard on the groups features and want to re-do the scheduler as well. Maybe this would be something for after the scheduler re-write? I don't see any mention of this on the roadmap [1] (although the roadmap is quite full already!). In any event, I'm interested in hearing your ideas and feedback. Thanks! 1 - http://beaker-project.org/dev/tech-roadmap.html

2 2

Demo of system page improvements
by Dan Callaghan 04 Mar '14

04 Mar '14

I've set up a temporary Beaker instance to demonstrate my proposed system page improvements [1] which will hopefully be landing in the 0.17 release. The demo instance is here: http://209.132.179.159/bkr/ You can log in as "demo" with password "demo". There are no real systems or lab controllers attached, and no distros imported, so it's not a fully functional Beaker instance. But I created some dummy systems with a variety of permissions granted, so it's enough to try out most of the system page features. If you're interested in the evolution of the system page, please take a look and reply with your feedback. [1] http://beaker-project.org/dev/proposals/system-page-improvements.html -- Dan Callaghan <dcallagh(a)redhat.com> Software Engineer, Hosted & Shared Services Red Hat, Inc.

3 10

Handling unrecognised distro names in the default templates
by Nick Coghlan 02 Mar '14

02 Mar '14

Currently, the default kickstart snippets are written such that any unknown distro names are handled as if they were Red Hat Enterprise Linux 6. We've dealt with Fedora by mentioning it explicitly in the templates when necessary. That's worked OK up until now, but there's likely to be a CentOS 7 beta some time in the next few months, and the handling of that and other Red Hat Enterprise Linux 7 derivatives means that this isn't going to work any more, since we can't hardcode special case handling of every possible derivative. The simplest resolution I have been able to come up with is the following: 1. Add a "base_distro" context variable in the kickstart rendering. By default, this is the same as "distro". 2. Update the kickstart snippets to use base_distro rather than distro 3. Add an optional "base_distro" field to our distro records. If that is set, base_distro in the template rendering context is to that rather than yo the same thing as distro. It's too late to get that into 0.16, but we could do it for 0.17. Does anyone see a simpler way to resolve the problem? If not, I'll create an RFE for this and allocate it to 0.17. Cheers, Nick. -- Nick Coghlan Red Hat Hosted & Shared Services Software Engineering & Development, Brisbane Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)

3 7

Currently recorded inventory data
by Amit Saha 27 Feb '14

27 Feb '14

As a first step towards migration to a lshw based inventory gathering system [1], I have first collected the data that we are currently gathering. Here they are with the occasional note: CPU === - Arch - Vendor - Model - Model name - Speed - Processors (number of CPUs) - Cores - Sockets - CPU Flags - Family - Stepping Notes: same fields for all archs are recorded. Some of them are not relevant to non-x86 architectures and are hard coded to 0. On the other hand, these archs have some fields of potential interest which are hence not being recorded. Devices ======= - Vendor ID - Device ID - subsys Vendor ID - subsys Device ID - bus - driver - type - description Misc others =========== - Running under a hypervisor or not - Memory - Number of NUMA nodes - System vendor - System model - System form factor Disk ==== - Disks: Model, sector size, geometry, etc. Key value data ============== - Uses the data above to create a collection of key value fields - Creates some additional fields (such as PCIID and USBID for devices) Data collected but not stored ============================= - Some of the data collected are not stored currently: kernel modules, network interfaces, using multipath or not, disk controller with the disk If there is anything I may have missed, please add to it. If there is anything which we should gather, please do the same. [1] http://beaker-project.org/dev/proposals/inventory-lshw-migration.html Best, Amit. -- Amit Saha <http://echorand.me> Software Engineering and Development Red Hat, Inc.

1 0

Beaker Gerrit upgrade 2014-02-26 01:00 UTC
by Dan Callaghan 26 Feb '14

26 Feb '14

Tomorrow I will take down Beaker's Gerrit installation for upgrading to version 2.8.1, which has a number of interesting new features: https://gerrit-documentation.storage.googleapis.com/ReleaseNotes/ReleaseNot… I'll start the upgrade at 01:00 UTC, in theory the outage should only last a few minutes but it looks like a tricky upgrade so I'm giving some heads up just in case. -- Dan Callaghan <dcallagh(a)redhat.com> Software Engineer, Hosted & Shared Services Red Hat, Inc.

3 3

External tasks support merged to the 0.16 development branch
by Nick Coghlan 21 Feb '14

21 Feb '14

Dan merged the external task support to the development branch this afternoon: http://git.beaker-project.org/cgit/beaker/commit/?h=develop&id=07869a2c0e33… The design proposal has also been updated based on the merged implementation: http://beaker-project.org/dev/proposals/external-tasks.html Note that the default beah harness is *not* compatible with the use of fetch URLs - for the time being, this feature is designed to be used solely with alternate harnesses (at least until we get restraint to a state where we're happy with it replacing beah as the default harness). Cheers, Nick. -- Nick Coghlan Red Hat Hosted & Shared Services Software Engineering & Development, Brisbane Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)

1 0

Getting Beaker into Fedora
by Raymond Mancy 20 Feb '14

20 Feb '14

Hi all, We've recently decided to take another look at getting Beaker into Fedora. Reading through the long list of packaging guidelines[1] that are required to be met, I've come up with, what I believe to be, the problems that would need to be resolved before Beaker would meet these guidelines. I list them below in no particular order. Rpmlint: Some errors, moslty the fiddling we do with RPM_BUILD_ROOT Changelog: No details about changes included in the release, only an upstream link. The guidelines say, "Links to upstream changelogs can be entered for those who want additional information". So it seems a bit unclear to me as to whether or not 'additional' is the operative word here or not. Documentation: According to the guidelines, 'relevent' documentation in the source must be inclded marked with %doc. AFAICT, the only docs in the documentation folder that are included in the binary rpms are the man pages, are a recreation of the releases page from the website. Although the guidelines do talk about irrelevant docs, and what they are, I'm not sure if the rest of our docs fall into that category. PIE: We need to add instructions so that Beaker can make use of position-independent execution. Requiring Base Package: Subpackages do not have a versioned Requires of their base package Macros: We currently mix and match some macros (well, on only one case). Need to pick one style. Directory ownership: Not all directories that beaker creates are owned by it. Web content: We keep some beaker content in /var/www rather than /usr/share. Unit tests: Need to execute our unittests in the %check section Provides/Obsoletes: Need to add versions to some of our Provides/Obsoletes entries If anyone can point out any misunderstanding of the guidelines, or in general anything else to add, please do. Cheers, Raymond. 1 https://fedoraproject.org/wiki/Packaging:Guidelines

3 8

Beaker 0.15.4 released
by Dan Callaghan 17 Feb '14

17 Feb '14

On behalf of the Beaker development team, I'm pleased to announce that Beaker 0.15.4 is now available from beaker-project.org [1]. To go along with this Beaker release, we have also released new versions of the default beah test harness and the standard reservesys and install tasks. The new beah release fixes some issues with the recently released IPv6 support. In addition, a new ks_meta variable `beah_rpm` allows you to select an older harness version in your recipe, to make it easier to debug and work around issues with the harness. For example, you can use ks_meta="beah_rpm=beah-0.6.48" to select the last IPv4-only version of the harness. For system owners, the edit_policy permission now also grants the ability to change the system owner. This brings it back in line with the "admin group" functionality in Beaker 0.14. A complete description of the bug fixes in this release can be found in the updated Beaker 0.15 release notes [2]. The detailed list of all changes made since Beaker 0.15.3 is also available [3]. [1] http://beaker-project.org/releases/ [2] http://beaker-project.org/docs/whats-new/release-0.15.html#beaker-0-15-4 [3] http://beaker-project.org/releases/beaker-0.15.4-ChangeLog -- Dan Callaghan <dcallagh(a)redhat.com> Software Engineer, Hosted & Shared Services Red Hat, Inc.

1 0

Public Beakerlib newsletter
by Nick Coghlan 17 Feb '14

17 Feb '14

One of the QE folks has started a series of posts about using Beakerlib to write Beaker tests: http://blog.afri.cz/ The first gives a nice overview of what Beakerlib is and why it exists (and why it's a shell library): http://blog.afri.cz/?p=6 Cheers, Nick. -- Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia

1 0

External tasks proposal up on beaker-project.org
by Nick Coghlan 06 Feb '14

06 Feb '14

I decided it made more sense to publish Bill's external tasks library on beaker-project.org and start a thread here, rather than continue to conduct the high level design discussions in the Gerrit review. Firstly, I like the idea of "fetch" and "rpm" nodes in the task XML. The proposal should be more explicit about the fact that the RPM node is only present in the recipe XML when it is retrieved through the stable harness API, and not in other cases (such as when cloning a job). I *don't* like the way the proposal refers to *removing* the task library. The task library isn't going anywhere, we're just providing a way to bypass it when doing so makes sense. The task library is still an incredibly convenient way to publish utilities like the standard tasks (http://beaker-project.org/docs/user-guide/beaker-provided-tasks.html) and potentially future higher level utilities like properly configured KDC, LDAP and qpid servers. Finally, I'd prefer it if the proposal was backed by a core "/distribution/gitfetch" task, together with the ability to flag a task in the task library as "external". When a task was flagged as external, then Beaker would *require* that a "fetch" element exist, and would display that URL when appropriate. Given the UI issues in the Beaker 0.15 upgrade, I'm concerned about breaking the fundamental assumption of every task being associated with an entry in the task library. By using an "external" flag and an implied task name (instead of leaving the task library reference NULL), it changes the failure mode from failing to handle an unexpected NULL or None value to display the task name as usual rather than the desired URL details. Regards, Nick. -- Nick Coghlan Red Hat Hosted & Shared Services Software Engineering & Development, Brisbane Testing Solutions Team Lead Beaker Development Lead (http://beaker-project.org/)

2 1

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Beaker-devel February 2014