Future directions for the Beaker test harness
by Nick Coghlan
A few weeks back, Amit put together a proof of concept for running the
test harness in a container, rather than directly on the host
(http://gerrit.beaker-project.org/#/c/3199).
That proof of concept relies on restraint, the new reference harness,
that is intended to eventually replace beah
(https://beaker-project.org/dev/proposals/reference-harness.html)
At the same time, I don't think restraint is currently getting the level
of review and testing that it needs to mature into a plausible
replacement for beah as the default harness.
I think Amit's proposed patch provides a possible way forward:
1. Accept the initial approach where restraint is the *only* supported
harness when running inside a container. Specifying both
"contained_harness" and "harness" as ks_meta variables should be an
error at this point (side note: 'harness' should also be documented
along with all the other ks_meta variables, with a link to
https://beaker-project.org/docs/alternative-harnesses/).
2. Recommend publishing both beah *and* restraint in the harness repos
for Beaker installations. This will not only make restraint available
for container based testing, but also make it readily available via
"harness=restraint" for normal testing, without needing to add a custom
repo definition.
3. Once we have container based testing working reliably with restraint,
drop the restriction against using alternative harnesses in containers.
The priority at the moment though is to get something working that can
run on an Atomic Host, and still provide a relatively normal execution
environment for the executed tasks. Supporting alternative harnesses
*inside* containers is a nice-to-have that can wait until later - by
flatly disallowing it, we ensure we don't have to spend any time working
on container related issues that don't impact restraint. For the initial
iteration, we can also ignore the question of choosing the base image
used to run the harness, as well as being able to start and stop other
containers on the host.
I've filed an RFE for 0.18 on that basis:
https://bugzilla.redhat.com/show_bug.cgi?id=1131388
As part of this, we may also want to move restraint from Bill's personal
account on GitHub to the main Beaker project account, but I don't think
that's particularly urgent at this point.
Regards,
Nick.
--
Nick Coghlan
Red Hat Hosted & Shared Services
Software Engineering & Development, Brisbane
HSS Provisioning Architect
5 years, 8 months
Adding system firmware to system details
by Don Zickus
Hi Dan,
Shawn is working with Jon and myself to add system firmware/bios info to the
beaker database to be displayed on the system details page.
The idea was to make it easy to find how old a bios is on our lab machines
and see which ones need an update.
Shawn made all the front end changes in the inventory job, but the backend
work is becoming a challenge.
It seems like we are going to have to add a table entry to the database in
server/model/inventory.py? Which would probably cause a database migration
event.
For now, we are using numa_nodes as our template to copy the output of lshw
(from the inventory script) into the database and onto the webpage.
Are we going in the right direction? Is this more work than we thought?
Then as a follow-on, we are assuming the database has to be updated to add
this entry, would it make sense to convert the whole database to a generic
key/value table to dynamically support new fields without having to migrate
the whole database all the time? And then to prevent random junk from being
added by the inventory script (or other script), have a whitelist filter
that only allows certain keys to be added/updated. Maintaining the
whitelist would be easier on the database then adding table entries.
This is just us trying to understand the architecture a little more and
trying to see where we can add some value to make it easier to maintain our
tests.
Cheers,
Don
6 years, 4 months
webdriver cache?
by Shawn Doherty
Hello all.
I have a question regarding WebDriver testing. I was working on creating
an integration test. When I would change code in the widgets.py file, my
integration test was not responsive to my changes, and seemed to be holding
on to information from previous versions.
Is there a cache in the webdriver or something that needs to be updated
when we make a change?
Thanks, Shawn
--
Shawn Doherty
Software Engineer, Kernel-HW
Red Hat
<https://www.redhat.com>
314 Littleton Rd
Westford, MA 01886
sdoherty(a)redhat.com T: 19785891080 INTERNAL:-8131080 IM: sdoherty
<https://red.ht/sig>
6 years, 5 months
Re: Fwd: Change in beaker[develop]: Add ExcludeAll button to Excluded Families
by Don Zickus
Hi Dan,
How do we move Gerrit changes forward?
Cheers,
Don
On Thu, Jun 22, 2017 at 02:16:44PM -0400, Shawn Doherty wrote:
> ---------- Forwarded message ----------
> From: Jonathan Toppins (Code Review) <gerrit(a)beaker-project.org>
> Date: Thu, Jun 15, 2017 at 2:27 PM
> Subject: Change in beaker[develop]: Add ExcludeAll button to Excluded
> Families
> To: Dan Callaghan <dcallagh(a)redhat.com>
> Cc: sdoherty(a)redhat.com
>
>
> Jonathan Toppins would like Dan Callaghan to *review* this change.
>
> View Change <https://gerrit.beaker-project.org/5712>
>
> Add ExcludeAll button to Excluded Families
>
> List of excluded families is large in some cases. Added a button to the
> exclude page that will toggle checkbox of Major releases.
>
> Change-Id: I3e86fbf4421023eb591d838ed5259825f5feb03a
> Signed-off-by: Shawn Doherty <sdoherty(a)redhat.com>
> ---
> M Server/bkr/server/static/javascript/util.js
> M Server/bkr/server/widgets.py
> 2 files changed, 9 insertions(+), 0 deletions(-)
>
> git pull ssh://gerrit.beaker-project.org:29418/beaker refs/changes/12/5712/1
>
> To view, visit change 5712 <https://gerrit.beaker-project.org/5712>. To
> unsubscribe, visit settings <https://gerrit.beaker-project.org/settings>.
> Gerrit-Project: beaker
> Gerrit-Branch: develop
> Gerrit-MessageType: newchange
> Gerrit-Change-Id: I3e86fbf4421023eb591d838ed5259825f5feb03a
> Gerrit-Change-Number: 5712
> Gerrit-PatchSet: 1
> Gerrit-Owner: sdoherty(a)redhat.com
> Gerrit-Reviewer: Dan Callaghan <dcallagh(a)redhat.com>
>
>
>
> --
>
> Shawn Doherty
>
> Software Engineer, Kernel-HW
>
> Red Hat
>
> <https://www.redhat.com>
>
> 314 Littleton Rd
>
> Westford, MA 01886
>
> sdoherty(a)redhat.com T: 19785891080 INTERNAL:-8131080 IM: sdoherty
> <https://red.ht/sig>
6 years, 5 months
converting CSV_import_export to flask and Backbone.js
by Don Zickus
Hi Dan,
I thought instead of describing what I am doing, I would just show you the
patch so you can point at the problem quicker.
As I mentioned before, I decided to just randomly grab a page like
CSV_import_export and try to convert it to flask. I started by mimicing
reserve_workflow thinking it had a similar display and POST frontend.
However, I found out, I can get the page to render the title but that is
about it.
Even on the reserve_workflow page, I don't get the pretty Distro and other
forms (without my changes applied too).
So I can't tell if my patch below is technically wrong (I am sure it is
missing lots of pieces) or my devel env is not quite configured correctly
(because reserve_workflow doesn't quite work either).
Thoughts? Help?
Cheers,
Don
diff --git a/Server/assets/csv-export.js b/Server/assets/csv-export.js
new file mode 100644
index 0000000..aed9db9
--- /dev/null
+++ b/Server/assets/csv-export.js
@@ -0,0 +1,35 @@
+
+// This program is free software; you can redistribute it and/or modify
+// it under the terms of the GNU General Public License as published by
+// the Free Software Foundation; either version 2 of the License, or
+// (at your option) any later version.
+
+;(function () {
+
+var CSVExportSelection = Backbone.Model.extend({
+});
+
+window.CSVExport = Backbone.View.extend({
+ template: JST['csv-export'],
+ events: {
+ 'submit form': 'submit',
+ },
+ initialize: function (options) {
+ this.csv_types = options.options['csv_types']
+ this.render();
+ },
+ render: function () {
+ this.$el.html(this.template(this));
+ },
+ submit: function (evt) {
+ evt.preventDefault();
+ var xhr = $.ajax({
+ url: 'action_export',
+ type: 'POST',
+ data: this.selection.attributes,
+ traditional: true,
+ });
+ },
+});
+
+})();
diff --git a/Server/assets/jst/csv-export.html b/Server/assets/jst/csv-export.html
new file mode 100644
index 0000000..eec875e
--- /dev/null
+++ b/Server/assets/jst/csv-export.html
@@ -0,0 +1,19 @@
+<form class="form-horizontal">
+ <fieldset>
+ <div class="control-group">
+ <label class="control-label">CSV TYPE</label>
+ <div class="controls">
+ <% _.each(csv_types, function (csv) { %>
+ <label class="radio">
+ <input type="radio" name="csv_type" value="<%- csv.toLowerCase() %>"
+ <% if (csv == 'System' ) { %>checked<% } %>
+ />
+ <%- csv %>
+ </label>
+ </div>
+ </div>
+ <div class="form-actions">
+ <button class="btn btn-primary" type="submit">Export CSV</button>
+ </div>
+ </fieldset>
+</form>
diff --git a/Server/bkr/server/CSV_import_export.py b/Server/bkr/server/CSV_import_export.py
index 41fcb26..94bb00f 100644
--- a/Server/bkr/server/CSV_import_export.py
+++ b/Server/bkr/server/CSV_import_export.py
@@ -10,7 +10,11 @@
from bkr.server import identity
from bkr.server.xmlrpccontroller import RPCRoot
from tempfile import NamedTemporaryFile
-from cherrypy.lib.cptools import serve_file
+#from cherrypy.lib.cptools import serve_file
+from flask import send_file, request
+from bkr.server.flask_util import render_tg_template
+
+from bkr.server.app import app
from bkr.server.model import (System, SystemType, Activity, SystemActivity,
User, Group, LabController, LabInfo,
OSMajor, OSVersion,
@@ -70,6 +74,64 @@ def line_num(self):
def fieldnames(self):
return self.reader.fieldnames
+# For XMLRPC methods in this class.
+exposed = False
+
+export_help_text = XML(u'<span>Refer to the <a href="http://beaker-project.org/docs/'
+ 'admin-guide/interface.html#export" target="_blank">'
+ 'documentation</a> to learn more about the exported data.</span>').expand()
+import_help_text = XML(u'<span>Refer to the <a href="http://beaker-project.org/docs/'
+ 'admin-guide/interface.html#import" target="_blank">'
+ 'documentation</a> for details about the supported CSV format.</span>').expand()
+
+upload = widgets.FileField(name='csv_file', label='Import CSV', \
+ help_text = import_help_text)
+download = RadioButtonList(name='csv_type', label='CSV Type',
+ options=[('system', 'Systems'),
+ ('system_id', 'Systems (for modification)'),
+ ('labinfo', 'System LabInfo'),
+ ('power', 'System Power'),
+ ('exclude', 'System Excluded Families'),
+ ('install', 'System Install Options'),
+ ('keyvalue', 'System Key/Values'),
+ ('system_pool', 'System Pools'),
+ ('user_group', 'User Groups')],
+ default='system',
+ help_text = export_help_text)
+exportform = HorizontalForm(
+ 'export',
+ action = 'export data',
+ submit_text = _(u'Export CSV'),
+)
+
+(a)app.route('/csv', methods=['GET'])
+(a)identity.require(identity.not_anonymous())
+def index():
+ options = {}
+ options['csv_types'] = ('system', 'system_id', 'labinfo', 'power',
+ 'exclude', 'install', 'keyvalue', 'system_pool',
+ 'user_group')
+ #return render_template('assets.jst.csv_export.html', options=options)
+ return render_tg_template('bkr.server.templates.csv_export', {
+ 'title' : u'CSV Export Don',
+ 'options' : options
+ })
+
+(a)app.route('/csv/action_export', methods=['POST'])
+(a)identity.require(identity.not_anonymous())
+def action_export():
+ csv_type = request.form.get('csv_type')
+ logger.debug("DON: request: %s" % csv_type())
+
+ file = NamedTemporaryFile()
+ log = self.to_csv(file, csv_type)
+ file.seek(0)
+
+ logger.debug('DON CSV export with send_file type: %s', csv_type)
+ return send_file(file.name, mimetype="text/csv",
+ as_attachment=True,
+ attachment_filename="%s.csv" % csv_type)
+
class CSV(RPCRoot):
# For XMLRPC methods in this class.
exposed = False
@@ -139,6 +201,7 @@ def action_export(self, csv_type, *args, **kw):
log = self.to_csv(file, csv_type)
file.seek(0)
+ logger.debug('DON CSV export with send_file type: %s', csv_type)
return serve_file(file.name, contentType="text/csv",
disposition="attachment",
name="%s.csv" % csv_type)
diff --git a/Server/bkr/server/assets.py b/Server/bkr/server/assets.py
index 0c5a117..71747f4 100644
--- a/Server/bkr/server/assets.py
+++ b/Server/bkr/server/assets.py
@@ -90,6 +90,7 @@ def _create_env(source_dir, output_dir, **kwargs):
'pools.js',
'query-builder.js',
'reserve-workflow.js',
+ 'csv-export.js',
'installation-model.js',
'task-library-model.js',
'scheduler-model.js',
diff --git a/Server/bkr/server/controllers.py b/Server/bkr/server/controllers.py
index cece8db..936cdc2 100644
--- a/Server/bkr/server/controllers.py
+++ b/Server/bkr/server/controllers.py
@@ -146,7 +146,7 @@ class Root(RPCRoot):
users = Users()
arches = Arches()
auth = Auth()
- csv = CSV()
+ #csv = CSV()
jobs = Jobs()
recipesets = RecipeSets()
recipes = Recipes()
diff --git a/Server/bkr/server/templates/csv_export.kid b/Server/bkr/server/templates/csv_export.kid
new file mode 100644
index 0000000..848cfff
--- /dev/null
+++ b/Server/bkr/server/templates/csv_export.kid
@@ -0,0 +1,22 @@
+<!DOCTYPE html>
+<html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://purl.org/kid/ns#"
+ py:extends="'master.kid'">
+<head>
+<title>$title</title>
+</head>
+<body>
+<div class="page-header">
+ <h1>$title</h1>
+</div>
+<div class="csv_export"></div>
+<script type="text/javascript">
+var csvexport = new CSVExport(
+$(function () {
+ new CSVExport({
+ el: '.csv_export',
+ options: ${tg.to_json(options)},
+ });
+});
+</script>
+</body>
+</html>
6 years, 6 months
beaker server migration
by Don Zickus
Hi Dan,
A recurring theme we seem to encounter when working through various features
we want to implement in beaker, is the challenges the front-end poses to add
code. It isn't just a straight-forward add code (once you understand the
technology), but there is an added bonus of understanding the strange quirks
the legacy code is laying upon the current code. :-) But you already know
that. :-)
I wanted to reach out and understand what the initial thoughts were to
migrate away from the legacy stuff. Obviously your team made an initial
effort and stopped. And there is still plenty of work to be done. Was a
process to move forward ever documented?
Granted things are a little bit better now that we understand how to run the
server in developer mode better and have the corresponding lengthy
dependencies installed. But I still struggle just to convert one page to
strictly flask and still involves using kid files to pass back to
cherrypy???
With all the changing technology out there, it would be nice to have beaker
be updated easily to match our needs. So I was trying to wrap my head
around an easy process to migrate things to a better place.
Cheers,
Don
6 years, 6 months