result logs
by Bill Peck
Hi Marian,
I have updated both the beah harness and the rhts helpers but I'm still
not getting results associated with results, right now they are
associated with the task. Is that piece still to be done?
Thanks!
13 years, 8 months
job_matrix
by Raymond Mancy
Hi Bill, I've just commited the latest of the job matrix.
It's still very much WIP but it will run some queries and give some results.
For the past few hours I've been working on forming the data in a way that is suitable
for the DataGrid to display.It's proving a bit tricky, so for the time being
I've put the code back to how it was (displaying it incorrectly with multiple arches).
Tomorrow with fresh eyes I should be able to nut it out.
Raymond
13 years, 8 months
Harness: plan
by Marian Csontos
Hi,
I identified following critical issues totalling in ~8.5d dividing us
from usable harness. I would like to keep ~2d reserve for newly found
problems. I tried not to be too optimistic, thus these estimates should
represent upper boundaries.
Testing: run more tests (~1d, @ASAP, ongoing)
- test local watchdog, multihost
- retest on RHEL4, RHEL5
- report any new critical issues.
Fix: Leaking memory (1.5d, work-in-progress)
Known problem with multi-host and forwarder.
Add: Improved reporting (2d)
Some events go unnoticed. Capture and format them and upload as text
file to Scheduler. Audience: tester.
Create an event log. Capture all incoming events (excluding
file_write`s) and upload to server. Audience: tools.
Add: Runtime storage (1d)
At the moment not everything what needs to be persistent i.e. to survive
harness restart is. And the little what is persistent is using simple
directory being pickled to/unpickled from disk, which is suboptimal for
larger quantities.
Will use shelve to store persistent data. shelve is working fine with
Python 2.3.
Fix: Synchronization issues (~2d)
Task was for unknown reasons not finished correctly and results were
assigned to following task.
Other, non-critical, issues:
Change: Local watchdog (~1d)
- this is currently implemented in rhts-test-runner.sh
- want to move it to Controller
Fix: backends do not daemonize properly. (~2d)
Backends can not be stopped with 'service ... stop'.
Fix: EL4: beah-srv not restarted. (~1d)
May be timing issue, trying to listen on busy TCP/IP port.
And more for next point release.
Planned work on LVM (1d spread all over next week)
- testing changes against nightly builds.
- improving test
-- Marian
13 years, 8 months
UUID in filenames
by Bill Peck
Hi Marian,
Is it possible to strip the UUID from the log name when submitting it to
Beaker? Its quite noisy when trying to show the filenames on the web page.
Thanks!
13 years, 8 months
Re: [Beaker-devel] deleting branches after merge
by Raymond Mancy
----- "Marian Csontos" <mcsontos(a)redhat.com> wrote:
> From: "Marian Csontos" <mcsontos(a)redhat.com>
> To: "Bill Peck" <bpeck(a)redhat.com>
> Cc: "List for the development of the Beaker project" <beaker-devel(a)lists.fedorahosted.org>
> Sent: Thursday, January 14, 2010 9:10:05 AM GMT +10:00 Brisbane
> Subject: Re: [Beaker-devel] deleting branches after merge
>
> On 01/12/2010 04:19 PM, Bill Peck wrote:
> > On 01/12/2010 10:16 AM, Marian Csontos wrote:
> >> On 01/12/2010 03:11 PM, Bill Peck wrote:
> >>> On 01/12/2010 04:21 AM, Marian Csontos wrote:
> >>>> On 01/11/2010 09:18 PM, Bill Peck wrote:
> >>>>> Hello Everyone,
> >>>>>
> >>>>> FYI - Just to keep things clean I've been deleting RFE and
> bugfix
> >>>>> branches after merging them into master. Obviously I won't
> delete
> >>>>> any
> >>>>> stable branches (we only have master right now).
> >>>> If anyone would like to keep branch's history, it is possible to
>
> >>>> use rebase instead of merge, just do:
> >>>> git pull --merge
> >> ^^^^^
> >> This should be
> >> git pull --rebase
> >> of course.
> >>
> >>>>
> >>> Can you explain in more detail how this would work and what the
> >>> differences are?
> >>>
> >>> I was told you should *NOT* use rebase if you are sharing your
> >>> repo. In fact, I thought rebase removed history.
> >> Partially correct. Before publishing changes, you can do pretty
> much
> >> anything including rewriting history.
> >> It is the history on the server which counts.
> >>
> >> You can run
> >> git pull --rebase
> >> [ which applies changes between origin..HEAD on top of branch ]
> >> and, after solving any conflicts, run
> >> git push
> >> This way, one will get rid of the branch as well as keep all the
> >> history.
> >> See
> >> man git-rebase
> >> for details.
> >>
> >> <Advertising-message>
> >> Sometimes it is undesirable to keep history as is, and
> >> git rebase -i
> >> allows you to split/join/reorder commits before pushing them.
> >> I love it.
> >>
> >
> > I'm pretty sure this only works if you are working on a branch by
> > yourself! If you do this while sharing with others git doesn't have
> > the info needed to resolve conflicts! I think this is dangerous.
We do work on local branches though. When you do a
git fetch
git checkout -b master origin/master
You are on a local branch and you should be able to rebase your local master
from the origin/master until the cows come home. What you DONT want to do
is somehow rebase the branch that origin/master refers to. That is when you are rebasing
a non local branch (i.e one that others pull from)
In fact this makes much more sense to rebase on master.
If you do a git log on master you'll see how there are various
merges with master...i.e merges onto itself (well technically not itself, merges from rmancy onto the master branch, both
happen to have the same name).
We should be able to do a rebase when working on master, thus we preserve a clear
and linear history of master.
Just before I did a bug fix commit directly to master. Since I had last updated master though, a commit had been
made to it. So something like this went down
git add <file>
git commit
git push origin master:master
*opps non fast forward error*
git pull --merge
git push origin master:master
now the last three entries in the git log are this
commit 1f5f2590d13c04efbb77200d2f70079d028e5e3d
Merge: 6315b2b b60a4b9
Author: Raymond Mancy <rmancy(a)redhat.com>
Date: Thu Jan 14 10:01:02 2010 +1000
Merge branch 'master' of ssh://rmancy@git.fedoraproject.org/git/beaker
commit 6315b2bf863e1edc0840d5235e58f3cf41299140
Author: Raymond Mancy <rmancy(a)redhat.com>
Date: Thu Jan 14 09:17:39 2010 +1000
bz555184 - ISE when creating new system
commit b60a4b94eb903fc9380bcc7d8f951031ba79a2ea
Author: Bill Peck <bpeck(a)redhat.com>
Date: Tue Jan 12 20:20:41 2010 -0500
minor fix
So the top merge is a merge of my master into the current master.It merged the previous two commits.
Kinda seems unecessary huh ?
if I had done a rebase instead that merge commit would not need to be there.
> Of course it is! And people do like dangerous things! ;-)
>
> RULE1:
> One should never do a rebase of shared branch anyone might have pulled
> from.
I'm pretty sure none of us are working directly on a shared branch.
Or at least we shouldnt be. Working on local branches of shared branches
is what we do.
> One thus always pulls from upstream to merge changes into such a
> branch.
>
> But, every rule has an exception, and may be following is one:
>
> EXCEPTION1:
> In theory, a "closed" branch (or more precisley the branch to be
> merged
> upstream and deleted), might be a good candidate for rebase.
>
> I have done a little homework, and it might work, if we wanted to keep
>
> more-linear history.
> May be not worth all the effort, but I wanted to learn more about git,
>
> and now I have to. :-)
>
> Let's have a development branch DevBranch and target branch Master,
> then
> git checkout DevBranch
> git checkout -b TempRebaseBranch
> git rebase Master
I've found that rebasing a branch with another branch that is significantly
different code wise is hard. I found there was a lot of manual conflict resolving involved.
It seemed to me kind of error prone with all the manual merges I had to do. In these instances I found a git pull --merge easier.
The history looked 'prettier' with a merge but it was also a very long history. If I have been working on a
branch for 2 months and I have 100 commits to it, those commit logs are going to end up in master.
Perhaps I'd rather see in the master log that my branch was merged into master, and if I want to look
further at the commits of that branch I can do so myself. Different people have different opinions about this
I'm relatively new to git, but this has been my experience and understanding. If you're interested in what
Linus said about git in response to our redhat colleague here in Brisbane, see this http://www.mail-archive.com/dri-devel@lists.sourceforge.net/msg39091.html
I don't think there is anything patently wrong with the way we've been doing things. At the end of the day I guess it's
just a tool to help us do our job, so whatever works for us.
Raymond
> will create new private branch TempRebaseBranch, containing all
> changes
> from DevBranch, but based on Master, while DevBranch will not be
> affected.
>
> -- Marian
>
> _______________________________________________
> Beaker-devel mailing list
> Beaker-devel(a)lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/beaker-devel
13 years, 8 months
deleting branches after merge
by Bill Peck
Hello Everyone,
FYI - Just to keep things clean I've been deleting RFE and bugfix
branches after merging them into master. Obviously I won't delete any
stable branches (we only have master right now).
-Bill
13 years, 8 months
Re: [Beaker-devel] deleting branches after merge
by Raymond Mancy
----- "Marian Csontos" <mcsontos(a)redhat.com> wrote:
> From: "Marian Csontos" <mcsontos(a)redhat.com>
> To: "Bill Peck" <bpeck(a)redhat.com>
> Cc: "List for the development of the Beaker project" <beaker-devel(a)lists.fedorahosted.org>
> Sent: Wednesday, January 13, 2010 1:16:34 AM GMT +10:00 Brisbane
> Subject: Re: [Beaker-devel] deleting branches after merge
>
> On 01/12/2010 03:11 PM, Bill Peck wrote:
> > On 01/12/2010 04:21 AM, Marian Csontos wrote:
> >> On 01/11/2010 09:18 PM, Bill Peck wrote:
> >>> Hello Everyone,
> >>>
> >>> FYI - Just to keep things clean I've been deleting RFE and bugfix
> >>> branches after merging them into master. Obviously I won't delete
> any
> >>> stable branches (we only have master right now).
> >> If anyone would like to keep branch's history, it is possible to
> use
> >> rebase instead of merge, just do:
> >> git pull --merge
> ^^^^^
> This should be
> git pull --rebase
> of course.
>
Another way to have a look at the history of a merged branch is to do a git log on the
branch's hash. So if you do a git log on say master branch and you see something like this
commit f0da09579a0a7f9e89087c4e36e4e29ae6046a8d
Merge: d462dea 0f9363b
Author: Bill Peck <bpeck(a)redhat.com>
Date: Mon Jan 11 13:15:13 2010 -0500
Merge branch 'master' into ticket51
You can do 'git log d62dea' which will show you the log history of that branch that was merged.
Raymond
> >>
> > Can you explain in more detail how this would work and what the
> > differences are?
> >
> > I was told you should *NOT* use rebase if you are sharing your repo.
>
> > In fact, I thought rebase removed history.
> Partially correct. Before publishing changes, you can do pretty much
> anything including rewriting history.
> It is the history on the server which counts.
>
> You can run
> git pull --rebase
> [ which applies changes between origin..HEAD on top of branch ]
> and, after solving any conflicts, run
> git push
> This way, one will get rid of the branch as well as keep all the
> history.
> See
> man git-rebase
> for details.
>
> <Advertising-message>
> Sometimes it is undesirable to keep history as is, and
> git rebase -i
> allows you to split/join/reorder commits before pushing them.
> I love it.
>
> -- Marian
>
> >
> >> -- Marian
> >>> -Bill
> >>> _______________________________________________
> >>> Beaker-devel mailing list
> >>> Beaker-devel(a)lists.fedorahosted.org
> >>> https://fedorahosted.org/mailman/listinfo/beaker-devel
> >>
> >
>
> _______________________________________________
> Beaker-devel mailing list
> Beaker-devel(a)lists.fedorahosted.org
> https://fedorahosted.org/mailman/listinfo/beaker-devel
13 years, 8 months
Re: [Beaker-devel] Alternative to disable custom columns.
by Raymond Mancy
----- "Bill Peck" <bpeck(a)redhat.com> wrote:
> From: "Bill Peck" <bpeck(a)redhat.com>
> To: "Raymond Mancy" <rmancy(a)redhat.com>
> Cc: "beaker-devel" <beaker-devel(a)lists.fedorahosted.org>
> Sent: Tuesday, January 12, 2010 6:16:31 AM GMT +10:00 Brisbane
> Subject: Re: Alternative to disable custom columns.
>
> On 01/08/2010 07:58 AM, Raymond Mancy wrote:
> > Hi Bill,
> >
> > Something wrong with the net connection I'm using. Can't connect to
> the repo.
> >
> > Here is the patch file for my latest changes as per your spec
> below.
> >
> > I've made it so whenever the user tries to return no columns, they
> are returned the default columns.
> > I can't think of a single use case where they would want to return
> no data from the search...
> >
> > Cheers
> >
> > Raymond
> >
>
> Hi Raymond,
>
> I just finished testing your latest code for ticket51. Seems to work
>
> well after a couple minor adjustments..
>
> 1 - class Utility was defined twice, Second one overrode the previous
>
> one so methods were missing. I remove the extra class and moved the
> one
> method in it to the other. Not sure what happened here? Bad merge?
>
Utility class in ticket51 is the one that was full of methods.
I also defined the Utility class in bz544347 (as it was not based on ticketg51, it did not have the Utility class already).
So when the two codes would merge the methods of the Utility class in bz544347 would need to have been moved in with
all the Utility methods of ticket51 (which you say you did).
> 2 - I made the following change..
> @@ -519,9 +518,7 @@ class Root(RPCRoot):
>
> - systems = sys_search.return_results()
> - new_systems = System.all(identity.current.user,system =
> systems)
> - return new_systems
> + return sys_search.return_results()
>
> I don't understand why the call to System.all(), we already start out
>
> with a System query object which started from one of the following:
> - System.all()
> - System.available()
> - System.free()
>
I remember I wrote that line a while ago. It was removed in an earlier commit (actually you removed it in commit e88cb). Ticket51 was branched from quite an old bit of code. It looks I missed it in my merge of master -> ticket51 though.
>
> I did a couple different searches and everything acts as expected.
> Started from Available link or Free, also tried Machine and Laptop.
> All
> works as expected.
>
> I think we will have requests for more columns to be added in the
> future.. We will probably want to move the checkbox list into a table
>
> so it takes up less room. Maybe 3 columns across? But that can be in
>
> an update. :-)
>
No worries.
> I won't merge into master until I hear back. :-)
>
> Thanks!
>
>
Thanks Bill. Hopefully that answers your questions.
If you could let me know once ticket51 is in master, I will need to merge master into a couple of other branches
I have.
Cheers
> > ----- "Bill Peck"<bpeck(a)redhat.com> wrote:
> >
> >
> >> From: "Bill Peck"<bpeck(a)redhat.com>
> >> To: "Raymond Mancy"<rmancy(a)redhat.com>
> >> Cc: "beaker-devel"<beaker-devel(a)lists.fedorahosted.org>
> >> Sent: Friday, January 8, 2010 4:46:57 AM GMT +10:00 Brisbane
> >> Subject: Re: Alternative to disable custom columns.
> >>
> >> On 01/07/2010 12:48 AM, Raymond Mancy wrote:
> >>
> >>> Hi Bill, Just wanting your thoughts on a good alternative to the
> >>>
> >> disable result column checkbox on ticket51.
> >>
> >>> The only reasonable alternative I can come up with is the
> >>>
> >> following.
> >>
> >>> Initially, in the checkbox list of result columns, have our
> default
> >>>
> >> columns selected (System,Status,Vendor...etc).
> >>
> >>> Have another link up the top that allows them to populate that
> >>>
> >> checkbox list with only default values (so they
> >>
> >>> can easily go from a heavily customised result set back to the
> >>>
> >> default).
> >>
> >>> I'm trying to make it easy for the user to go between a
> customised
> >>>
> >> search result and a default search result. If
> >>
> >>> they wanted to go between the two quite often it could get
> fiddley
> >>>
> >> if they had to do it all manually. Maybe I'm
> >>
> >>> overstating this feature though.
> >>>
> >>>
> >>> Raymond
> >>>
> >>>
> >>>
> >>>
> >>>
> >> Hi Raymond,
> >>
> >> Here are my thoughts on testing ticket51 so far.
> >>
> >> - Default to having all current columns selected. (The ones we
> >> normally
> >> show)
> >> - Have three "shortcut" links, Select All, Select None, and Select
> >> Default.
> >> - move these to under the checkbox div. I don't think we
> want to
> >>
> >> see them unless we are actually clicking and unclicking
> checkboxes.
> >> Users will be confused otherwise.
> >> - Remove the "Disable Custom Result Columns" checkbox
> >>
> >> Assuming we can get the above correct then we could just have one
> code
> >>
> >> path for the grid.
> >> It would also allow a user to set their preferred columns in
> prefs.
> >>
> >> btw - while testing I received a 500 ISE when selecting
> >> System/PowerType. I think this blows up because of the nested
> >> Power->Type relation. Might need to move PowerType to a Power
> search
> >>
> >> object which has Type in it. thoughts?
> >>
> >> Thanks!
> >>
13 years, 8 months
Re: [Beaker-devel] Alternative to disable custom columns.
by Raymond Mancy
Hi Bill,
Something wrong with the net connection I'm using. Can't connect to the repo.
Here is the patch file for my latest changes as per your spec below.
I've made it so whenever the user tries to return no columns, they are returned the default columns.
I can't think of a single use case where they would want to return no data from the search...
Cheers
Raymond
----- "Bill Peck" <bpeck(a)redhat.com> wrote:
> From: "Bill Peck" <bpeck(a)redhat.com>
> To: "Raymond Mancy" <rmancy(a)redhat.com>
> Cc: "beaker-devel" <beaker-devel(a)lists.fedorahosted.org>
> Sent: Friday, January 8, 2010 4:46:57 AM GMT +10:00 Brisbane
> Subject: Re: Alternative to disable custom columns.
>
> On 01/07/2010 12:48 AM, Raymond Mancy wrote:
> > Hi Bill, Just wanting your thoughts on a good alternative to the
> disable result column checkbox on ticket51.
> >
> > The only reasonable alternative I can come up with is the
> following.
> >
> > Initially, in the checkbox list of result columns, have our default
> columns selected (System,Status,Vendor...etc).
> > Have another link up the top that allows them to populate that
> checkbox list with only default values (so they
> > can easily go from a heavily customised result set back to the
> default).
> >
> > I'm trying to make it easy for the user to go between a customised
> search result and a default search result. If
> > they wanted to go between the two quite often it could get fiddley
> if they had to do it all manually. Maybe I'm
> > overstating this feature though.
> >
> >
> > Raymond
> >
> >
> >
> >
>
> Hi Raymond,
>
> Here are my thoughts on testing ticket51 so far.
>
> - Default to having all current columns selected. (The ones we
> normally
> show)
> - Have three "shortcut" links, Select All, Select None, and Select
> Default.
> - move these to under the checkbox div. I don't think we want to
>
> see them unless we are actually clicking and unclicking checkboxes.
> Users will be confused otherwise.
> - Remove the "Disable Custom Result Columns" checkbox
>
> Assuming we can get the above correct then we could just have one code
>
> path for the grid.
> It would also allow a user to set their preferred columns in prefs.
>
> btw - while testing I received a 500 ISE when selecting
> System/PowerType. I think this blows up because of the nested
> Power->Type relation. Might need to move PowerType to a Power search
>
> object which has Type in it. thoughts?
>
> Thanks!
13 years, 8 months
Re: [Beaker-devel] result_upload_file, handling attachments and other relationships
by Marian Csontos
Hi Bill,
rhts uses "late binding" - it uploads a file and only then would say it
was resultLog.
I would like to avoid caching files in backends,
and I would like to go on (at least for a while) with original
rhts-devel-test-env and submit all patches upstream. At least while it
will be used by the two systems.
Fortunately rhts is under our control and the change is simple enough -
requires only to change report_log function in rhts-db-submit-result:
+ bkr = beaker()
+ if bkr:
+ resp = session.results.resultLog(log_type, result_id, prettyname)
rhts.uploadWrapper(session,log_name,recipetestid, prettyname)
- resp = session.results.resultLog(log_type, result_id, prettyname)
+ if not bkr:
+ resp = session.results.resultLog(log_type, result_id, prettyname)
...and...
+def beaker():
+ return os.environ.get('FROM_BEAH')
Any advice on this particular solution?
However, we might run into situation, where it will not be that easy.
Do you think it would be possible to change server in such case?
I would like to suggest a change on server: I see no need to
differentiate between {task,recipe,result}_upload_file - it is just a
file. What really matters is "linking" i.e. what we link the uploaded
file to. In these cases we could create link from {recipe,task,result}
as appropriate. IMO it would make sense keeping these in DB.
The same applies to any other kind of objects. At the moment there is
not much of them. But I would like to introduce sections as I mentioned
before. A large part of beakerlib tries to handle this.
Moreover, at the moment we accept only single task at the time and
single "event-stream" for a task. I think it would make sense having
multiple streams and multiple tasks/processes running in parallel and we
would need to identify these. And we would want to link fork and wait
points, may be synchronization points for easier log analysis. Without
linking we might end up with a big mess.
Any of these stream could have child streams, results or log messages
(debug, warning, error) attached.
IMO table display would be the most natural way to display these -
vertical axe for time and horizontal for particular streams.
This does not look like something to be done by end of January, may be
2011, but let's make it possible.
Cheers,
-- Marian
13 years, 8 months