Slight change to release notes process
by Nick Coghlan
All,
Dan figured out a way to make the pending release note patches version
independent, by placing them all into a "documentation/whats-new/next/"
subdirectory. They'll then be edited together into the actual release
notes after we hit beta for the release.
We've made that change in git, so if you have any current patches with
release notes up on Gerrit or in a working copy, they will need to be
rebased and the release note snippet moved from "release-0.12/" to "next/".
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
11 years
Re: [Beaker-devel] Reorganising Power tab
by Raymond Mancy
----- Original Message -----
> From: "Nick Coghlan" <ncoghlan(a)redhat.com>
> To: beaker-dev-list(a)redhat.com
> Sent: Tuesday, February 26, 2013 1:35:24 PM
> Subject: Re: Reorganising Power tab
>
> On 02/26/2013 01:01 PM, Dan Callaghan wrote:
> > Excerpts from Raymond Mancy's message of 2013-02-25 22:24:14 +1000:
> >> A new BZ which enables us to manually clear netboot configuration
> >> files is causing
> >> the Power and Provision tabs to become a bit confusing.
> >>
> >> Currently, the 'Recent Power Commands' table on the 'Power' tab is
> >> used to display
> >> recent entries in the CommandActivity table. This table actually
> >> records both
> >> power and provisioning actions (and perhaps anything else we deem
> >> necessary in the
> >> future). So if you want to show the user their operation has been
> >> successfully queued
> >> , the operation needs to happen on the 'Power' tab.
> >>
> >> I suggest that we remove the CommandActivity table out of the
> >> Power tab
> >> and give it its own tab, similar to the history tab. That way we
> >> can make it a
> >> first class citizen in its own right without compromising the UI.
> >> We also
> >> then avoid the trap of feeling that we have to group things onto a
> >> tab
> >> based on underlying implementation realities.
> >
> > Does that mean the command buttons (Power On, Power Off, Reboot,
> > Interrupt, and the forthcoming Clear Netboot) will all move to the
> > new
> > Command tab also?
>
> I actually thought the conclusion of the IRC discussion was that we
> would simply rename the existing Power tab to "Commands" rather than
> creating a new tab.
>
It was, but I dont' think that is the right conclusion. I'm not a UX designer,
but to me it doesn't make sense to arrange items together just because
they happen to use the same underlying communication mechanism (in this case
the CommandActivity table), when they are otherwise more or less unrelated in their
purpose. It would be akin to arranging everything together just because they yse
XMLRPC. I think clearing the netboot files from the same place where we create
them makes as much sense as anything else.
> Cheers,
> Nick.
>
> P.S. I don't see any reason for this thread to be on the internal dev
> list rather than the public one. Please default to the public list
> for
> any Beaker design discussions that don't touch on the RH internal
> Beaker
> instance or HSS internal processes.
>
> --
> Nick Coghlan
> Red Hat Infrastructure Engineering & Development, Brisbane
>
> Python Applications Team Lead
> Beaker Development Lead (http://beaker-project.org/)
> PulpDist Development Lead (http://pulpdist.readthedocs.org)
>
> --
> This is a Red Hat internal mailing list
>
11 years
Harness API proposal
by Dan Callaghan
I've just published the initial version my design proposal for a stable
harness API [1], following Beaker's new design proposal process. The API
it describes is mostly the same as my original draft from an earlier
thread on this list [2]. I added a "none" result ("N/A" was awkward
because of the slash and "Not Applicable" because of its two words) and
a way for the harness to attach arbitrary URLs as logs, as suggested by
Bill [3].
The other additions are the part about how Beaker configures the
harness, and how users will be able to select an alternative harness.
I'm particularly interested in feedback about the post-install
configuration part. I based the /root/RECIPE.TXT file and the
environment variables on what we have now, but they are not necessarily
the nicest way to do it. As I see it, passing information on the
filesystem could be problematic because it's not cross-platform (the
path would have to be different on Windows) and it might give us SELinux
headaches. On the other hand, passing information by environment
variables will work cross-platform but it is much harder for Beaker to
actually *set* the variables in a cross-platform way. Right now we are
relying on /etc/profile being sourced, which is okay for SysV init since
the scripts are in bash, but systemd services are started with a clean
environment and so we would need to somehow get the env vars into the
right unit file. And it would be different again on the other platforms.
So I'm not really sure what the best way is.
I think Nick also has some suggestions about the alternative harness
selection too... so, let the discussions begin!
Coming soon: my "reference harness" proposal to go along with this
one...
[1] http://beaker-project.org/dev/proposals/harness-api.html
[2] http://thread.gmane.org/gmane.comp.systems.beaker.devel/405
[3] http://article.gmane.org/gmane.comp.systems.beaker.devel/416
--
Dan Callaghan <dcallagh(a)redhat.com>
Software Engineer, Infrastructure Engineering and Development
Red Hat, Inc.
11 years
Re: [Beaker-devel] Harness API proposal
by Jan Stancek
Hi,
would it make sense to add to harness API:
GET /recipes/(recipe_id)/logs/(path)
so harness could provide URL or content to test?
One of the requests in past was to be able to get to your own console.log,
so you can for example check output of kdump kernel.
Regards,
Jan
11 years
Task Syncing Script
by Amit Saha
Hello all,
I have a preliminary version of a script to sync the tasks between two Beaker instances. The changeset is here [1].
The script uses Kobo [2] client library to make XML-RPC calls, and hence you would need to have it installed.
The syncing protocol of the script is as follows:
- Task doesn't exist in destination: copy it.
- Task exists in destination: Overwrite it, if it is a different version on the source.
(The assumption here is that the source is always right)
- Tasks which exist on the destination and not on the source are left untouched
You can specify the following options to the script:
Options:
--version show program's version number and exit
-h, --help show this help message and exit
-s SOURCE, --source=SOURCE
Source Beaker Instance
-d DESTINATION, --destination=DESTINATION
Destination Beaker Instance
-u USERNAME, --username=USERNAME
Username for the destination server
-p PASSWORD, --password=PASSWORD
Password for the destination server
-k, --kerberos Specify to use Kerberos authentication
--krb_realm=KRB_REALM
Specify Kerberos realm
--krb_service=KRB_SERVICE
Specify Kerberos service
Sample use cases:
Using Kerberos authentication:
$ beaker-sync-tasks --source=http://127.0.0.1/bkr --destination=http://my-remote-beaker --kerberos
Using Username/password
$ beaker-sync-tasks --source=http://127.0.0.1/bkr --destination=http://my-remote-beaker --username <username>
Password:
$ beaker-sync-tasks --source=http://127.0.0.1/bkr --destination=http://my-remote-beaker --username <username> --password <mypass>
A first sanity check of the script would be to give the same source and destination:
$ beaker-sync-tasks --source=http://127.0.0.1/bkr --destination=http://127.0.0.1/bkr --username beakeradmin
Password:
Getting the list of tasks from source and destination..
Finding tasks to upload to destination..
No tasks to be uploaded to destination from source
[1] http://gerrit.beaker-project.org/#/c/1727/
[2] https://fedorahosted.org/kobo/
I welcome feedback on the design and functionality of the script.
Best,
Amit.
--
Amit Saha <http://echorand.me>
Infrastructure Engineering and Development
Red Hat, Inc.
11 years, 1 month
Group ownership of systems
by Nick Coghlan
One of the pieces of feedback I have received on the System Pools design
proposal is that it seems very confusing and overly complicated to
people that just want the ability to set an "owning group" on a system,
similar to what we're planning for submission of group jobs in 1.0.
So, while I still really like the idea of system pools as an
*implementation* technique, I think the draft proposal needs some
significant UI redesign in order to mostly hide their existence from the
end user.
My current thought is to change the initial proposal as follows (while
keeping the underlying system pool infrastructure):
- system pools will get an "owning group"
- the owning group automatically has all permissions on that pool
- every user group gets a "System Pool" attribute
- these pools are created automatically whenever a new group is created
- the user group is automatically granted all permissions on its own
system pool
- pools for existing groups will be created as part of the 1.1 data
migration
- the underlying naming scheme will be something simple like
"<groupname>-system-pool" (while user groups and system pools are
technically separate name spaces, so we could just reuse the group
names, I expect that would end up being confusing in the long run, since
it would hide the underlying system pool concept *too* much)
- the UI for systems will allow you to set an "owning group". Behind the
scenes, this will add the system to the group system pool.
- the UI for group management will allow you to edit the policy for the
group's system pool
- the UI for group management will allow to get a list of all systems in
the group's system pool
And then explicitly defer the following features:
- providing "Edit pool policy" as a permission on the pool (instead
limit it to the owning group)
- creating pools manually (rather than as part of group creation)
- selecting an existing pool as the group pool when creating a new group
- adding a system to multiple pools
- bulk operations on system pools (script the CLI or XML-RPC API for now)
Thoughts?
Regards,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
11 years, 1 month
Release notes infrastructure added to develop
by Nick Coghlan
The new release notes infrastructure has been merged into the develop
branch.
The process is fairly simple: if you're making a user/admin facing
change for 1.0, it must be captured in a reStructuredText file under
"documentation/whats-new/release-1.0/". Any affected patch won't make it
through code review until the release note is present.
Providing technical release notes for "under the hood" stuff (like
replacing SimpleXMLRpcServer on the lab controllers with gevent+flask)
is encouraged, but not required.
The exact formatting and wording of the draft notes isn't a big deal,
the main requirement during development is to make sure that the gist of
any changes is captured in that directory, so that we don't forget to
mention it when preparing the final version of the notes. Using separate
files rather than all updating a single file should help to avoid
spurious merge conflicts on Gerrit.
Once we hit feature freeze in early March, I'll take care of
reformatting the notes with an eye to making them more readable
(splitting them out into admin/user/developer notes, grouping notes for
related features together, etc).
For reference, basic instructions are kept as a comment in:
http://git.beaker-project.org/cgit/beaker/tree/documentation/whats-new/in...
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
11 years, 1 month
Testing scheduler race conditions
by Nick Coghlan
Ray asked about possible approaches to testing scheduler race conditions
for http://gerrit.beaker-project.org/#/c/1718/
Here's my suggestion:
1. Set up 2 systems, A and B
2. Set up a pair of jobs, each containing a recipeset with a pair of
recipes targeting the respective systems (so we will have recipes 1A,
1B, 2A, 2B queued in that order)
3. Mark system A unavailable
4. Monkeypatch beakerd.schedule_queued_recipe with a version that, after
calling the regular version, marks system A as available, if recipe 1B
was just scheduled
5. Call beakerd.schedule_queued_recipes
6. Ensure 1B is scheduled, 1A, 2A and 2B are still queued
7. Call beakerd.schedule_queued_recipes again
8. Ensure 1A and 1B are scheduled, 2A and 2B are still queued
9. Make sure to undo the monkeypatch even if an exception is thrown!
FWIW, Michael Foord's "mock" library makes this kind of temporary
replacement of functions in the module under test really easy, and
self.addCleanup in unittest2 is really handy as well. The unittest2
feature set hit the stdlib unittest in 2.7, and mock was added in 3.3 as
unittest.mock. If we're ever looking for more things to do,
adding/upgrading those dependencies could be worthwhile. You can
definitely live without them, but they're nice to have.
(The upgraded unittest in 2.7 is actually one of the main benefits of
running our automated test suite on Fedora during development: you get
much more useful error messages out of self.assertEqual when comparing
containers, so even 2.6 compatible tests benefit from the upgrade!)
Cheers,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
11 years, 1 month
Opening up Beaker's design processes
by Nick Coghlan
Traditionally, Beaker's design processes have been fairly closed. If you
weren't part of the Beaker team, participating directly in conversations
on IRC or following the right bugs in Bugzilla or code reviews on
Gerrit, major changes in Beaker's capabilities could come as a complete
surprise. When release announcements were made (and they generally
weren't announced on this list), they consisted of a series of bug
numbers and titles, without a clear explanation of what the new release
might mean for *users* of an upgraded Beaker installation.
We don't think this is a healthy situation for the project to be in, and
so we're taking a number of steps to help change that in the run up to
Beaker 1.0 (currently expected in mid-to-late March).
1. We have now published a technical roadmap at
http://beaker-project.org/tech-roadmap.html
We aim for this to provide a high level view of what the Beaker team see
as Beaker's major current shortcomings, as well as some insight into
what we're actively working on. We value external contributions to any
part of the system, especially those that align with this road map.
2. Proposals for major new features and architectural changes in Beaker
will now be published at http://beaker-project.org/dev/
I've been a CPython core developer for several years, and one of the
things I think we get fairly right is the Python Enhancement Proposal
process for discussing and reviewing significant changes to the language
and standard library. By introducing a similar process for Beaker, we
hope to stimulate early feedback from our users and other developers,
resulting in more appropriate final designs.
If you have an idea for a design proposal (especially if it's related to
a topic already on the technical road map) please send it to this list
(beaker-devel(a)lists.fedorahosted.org) for review and discussion.
3. Future Beaker releases will be accompanied by user and admin oriented
release notes (not just a list of references to developer oriented
Bugzilla entries)
Again influenced by CPython's development processes (in this case, the
"What's New in Python X.Y?" guide that accompanies each major release),
we hope that these enhanced release notes will help Beaker users become
aware of the interesting new capabilities provided in each release, as
well as which significant bug fixes allow things that were previously
impossible.
Another change that was made some time ago, but perhaps not widely
advertised, is the provision of a public "Developer Guide" at
http://beaker-project.org/dev-guide.html, as well as
the web service API documentation at http://beaker-project.org/server-api/
Note, all of the links above are accessible from the "For Developers"
section on the main Beaker help page at http://beaker-project.org/help.html
Regards,
Nick.
--
Nick Coghlan
Red Hat Infrastructure Engineering & Development, Brisbane
Python Applications Team Lead
Beaker Development Lead (http://beaker-project.org/)
PulpDist Development Lead (http://pulpdist.readthedocs.org)
11 years, 1 month