Beaker-devel May 2013

beaker-devel@lists.fedorahosted.org

5 participants
12 discussions

The march towards Beaker 1.0
by Nick Coghlan 04 Jun '13

04 Jun '13

Beaker has been doing 0.x releases for a long time. We've been talking about the release later this month being 1.0, but before we declare we've reached that milestone, I'd really like us to be at the point where someone could reasonably expect to grab the Beaker RPMs from beaker-project.org, and using just our documentation, set up and start running their own Beaker instance, including writing new tests using either beah or autotest. We're not there yet, and we're not going to get there this month, so the next release will be 0.13 and we'll continue with regular 0.x releases until we're happy we have something we consider worthy of the "Beaker 1.0" title. I don't think we're all that far away - we may need a 0.14, but I'll be surprised if we see 0.15. For planning purposes, I'd like to keep marking the backlog as "1.0" - the 0.x releases will then just be snapshots that make useful chunks of functionality available to users, rather than having to wait until we're done with everything we want to include for the 1.0 milestone. This will throw out the currently predicted releases in http://beaker-project.org/dev/proposals/handling-large-installations.html and the associated design proposals, but I'm not really worried about that. In some respects, it works out better, as it means we can add the "Effective Job Priorities" proposal to our targets for Beaker 1.0, and that's the *real* objective of several of the changes we're currently working on. Once we're happy we know what we would like to include in 1.0, then I can go back and adjust those design proposals and the tech roadmap accordingly. Based on the off-list discussions, here's what I would like to aim for as our 1.0 milestone: * The Effective Job Priorities design proposal (and its dependencies) * Full Fedora compatibility (including the test suite) * A stable alternate harness API (ideally with the corresponding autotest patches merged on their side) * A harness independent reservesys mechanism * Moving any remaining hardcoded Red Hat specific settings into configuration files * Documentation improvements, including * an architectural guide that at least explains all of Beaker's external interfaces * an explanation of the local watchdog hooks * more general advice on job design (including effectively using features like the job matrix) * Self tests that exercise beah, the legacy harness API and the new harness API * Having a public page giving at least basic info on how Beaker is used at Red Hat Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Test Automation Team Lead Beaker Development Lead (http://beaker-project.org/) PulpDist Development Lead (http://pulpdist.readthedocs.org)

2 3

Standalone Selenium test jar requirement bumped to 2.33
by Nick Coghlan 28 May '13

28 May '13

If you rebase on develop, be aware I just updated the required version of the standalone Selenium jar file to 2.33 (https://code.google.com/p/selenium/downloads/detail?name=selenium-server-st…) This will cause a couple of known failures until we get the associated test suite compatibility fixes merged as well (which shouldn't be far away - they're part of Dan's fixes to get the dogfood task running properly again) I'll update the instructions in the dev guide later today, too. Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Test Automation Team Lead Beaker Development Lead (http://beaker-project.org/) PulpDist Development Lead (http://pulpdist.readthedocs.org)

2 1

The nose-progressive plugin
by Nick Coghlan 28 May '13

28 May '13

I was recently introduced to the "nose-progressive" plugin, which changes the way nose displays results to a single updated line, with any error reports printed immediately and a nice progress bar indicating how far through the test run is (in terms of number of cases). Really easy to setup and use with our tests: sudo yum install python-nose-progressive ./run-tests.sh --with-progressive And thanks to dcallagh for packaging it for Fedora :) Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Test Automation Team Lead Beaker Development Lead (http://beaker-project.org/) PulpDist Development Lead (http://pulpdist.readthedocs.org)

3 2

Nightly RPMs on beaker-project.org
by Dan Callaghan 24 May '13

24 May '13

I've just set up a cron job to build Beaker RHEL6 RPMs from the develop branch in git each night. They are uploaded to beaker-project.org here: http://beaker-project.org/nightlies/ This is also usable as a yum repository, so if you like to live on the wild side you can install directly from there :-) -- Dan Callaghan <dcallagh(a)redhat.com> Software Engineer, Infrastructure Engineering and Development Red Hat, Inc.

2 1

beaker-project.org down for upgrades
by Nick Coghlan 24 May '13

24 May '13

Just a heads up that beaker-project.org is currently down while it is updated to Fedora 18, as well as to take advantage of a free upgrade offer from Linode (512 MiB RAM -> 1 GiB). Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Test Automation Team Lead Beaker Development Lead (http://beaker-project.org/) PulpDist Development Lead (http://pulpdist.readthedocs.org)

2 1

Beaker road map updated
by Nick Coghlan 20 May '13

20 May '13

The Bugzilla downtime today ended up providing me with a good opportunity to bring the technical road map up to date. The major changes are to set our objectives for Beaker 1.0 and to clearly document beah's current status as fully supported, but not actively developed. * http://beaker-project.org/dev/tech-roadmap.html#objectives-for-beaker-1-0 * http://beaker-project.org/dev/tech-roadmap.html#the-status-of-beah I also added several references to other projects (like Rickshaw, Ansible and cloud-init) that may be of interest for various parts of the road map. The site update also covered the documentation and draft release notes for 0.13 changes that have already been merged: * http://beaker-project.org/docs-develop/whats-new/#unreleased-changes The full diff is at http://git.beaker-project.org/cgit/beaker-project.org/commit/?id=346da20496… Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Test Automation Team Lead Beaker Development Lead (http://beaker-project.org/) PulpDist Development Lead (http://pulpdist.readthedocs.org)

1 1

Defining a more useful component list in Bugzilla
by Nick Coghlan 13 May '13

13 May '13

The current component list in Bugzilla has a few limitations, with some bugs being thrown at a component more or less at random, since they don't fit any of the existing components. In other cases, the component is too broad, failing to make appropriate distinctions in the assignments. Here's the current component list: command line Doc inventory qpid lab controller scheduler reports tests test harness web UI Those actually aren't *that* bad, there are just some missing ones and a couple could do with a name change. Suggested name/scope changes: qpid -> notifications (covering both AMQP sending and receiving and email sending) tests -> test suite inventory -> inventory task test harness -> beah test harness command line -> bkr CLI Suggested new components: beaker-wizard CLI other tasks (covers all standard tasks other than the inventory task) harness API (for the new stable harness API) server XML-RPC (for central issues that aren't related to the web UI or scheduler) multiple (problem/feature that spans multiple components) unknown (for bugs that haven't been triaged yet) Fedora compatibility I'm *not* sure those proposals actually make sense, but I wanted to provide a starting point for discussion. Cheers, Nick. -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Test Automation Team Lead Beaker Development Lead (http://beaker-project.org/) PulpDist Development Lead (http://pulpdist.readthedocs.org)

4 4

New Beaker API quirks
by Don Zickus 12 May '13

12 May '13

Hi Dan, I have been enjoying the ease to code against the new Beaker API. The GET, POST, PUSH stuff keeps things really simple. But during my testing I noticed a few quirks that Bill P. suggested I email you about for your opinion. - When I POST/PUT to a url, if the url has an extra '/' in the path, like /recipes/(recipe_id)/tasks/(task_id)//status the command gets rejected. Not sure if that is an http thing or a server thing. Not to much of a problem, just forces me to clean up my code (which is good!). Just thought I would mention it. - when I PUT a zero length file to the server, it fails with a 400 code. Now I should be filtering those (and will now). But it did take me an hour before I realized that of my list of files, the rejected ones were of zero length. Again could be on purpose, not sure. - Because the server does not have a mechanism to 'push' and abort the client (other than power off), the client could still be running and trying to pushing results back to the server. a POST of a task_result gave me an 'Internal Server Error 500'. Will that be the expected response? I can trap for that and abort the client. PUT of logs is still allowed even though the task aborted on the server side. Bill thought that should be fixed. Though I am not sure what the expectations are. That's it so far. Thanks, Don

2 3

Handling exit status of the commands execution from all the nodes
by shwetha 07 May '13

07 May '13

Hi All , In a multi-node testing environment we have to fail a test case when certain commands on any one of the node fails . For example : if mounting fails on client node , that test case execution should not be continued. A message to be sent to all other machines i.e masternode/node/peer in this case to stop the execution. How are we going to handle this situation. *This is the basic requirement for multi-node testing.* We want to check the exit status after every step. For example in most of our testcases the set-up includes : 1) peer probing 2) volume_create 3) volume_info 4) volume_start 5) volume_status 6) mount In every step we should check the exit status from all the nodes and if any one node has non-zero exit status we should stop that test case and move on to next test case in that task. -Shwetha

1 0

Technical roadmap updated
by Nick Coghlan 03 May '13

03 May '13

I've just published an update [1] to the tech roadmap at http://beaker-project.org/tech-roadmap.html Three changes of note: - I moved the "Live Dashboard" down into the exploration section. It would still be nice to have it, but it turns out that Graphite's rendering API plus the ReloadEvery extension for Firefox ([2]) isn't a bad interim solution :) - I moved the OpenStack integration up into the planned section. oVirt Engine is great if you're wanting to run a long lived high availability core service, but not so good if you're after throwaway VMs to run a single recipe. By contrast, OpenStack's reputation is pretty much the exact opposite (great for spinning up instances quickly, less great if you then want to keep them running forever), so it will hopefully be a better fit for our use case. - I added Jenkins integration to the "exploration" section. While you *can* use Beaker for continuous integration (as Dan's patchbot shows), Jenkins is really the king of that space. I see a lot of potential value in an approach where Jenkins buildbots are used for initial smoke tests on core platforms, and then you can configure a Jenkins plugin to kick off a more comprehensive set of integration tests in Beaker. Cheers, Nick. [1] http://git.beaker-project.org/cgit/beaker-project.org/commit/?id=43ad402a8f… [2] https://addons.mozilla.org/en-US/firefox/addon/reloadevery/ -- Nick Coghlan Red Hat Infrastructure Engineering & Development, Brisbane Test Automation Team Lead Beaker Development Lead (http://beaker-project.org/) PulpDist Development Lead (http://pulpdist.readthedocs.org)

1 0

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

Beaker-devel May 2013