Hi Don, Bill, LMR,
I saw that the pull request for Beaker harness API support in autotest has been merged. I'd like to link to that feature from beaker-project.org, so I was wondering if there was anything published anywhere (even a blog or mailing list post) that covered the details of:
1. What to add to a job/recipe definition to specify autotest as the harness 2. What, if anything, to add to the task list in the recipe to configure autotest appropriately 3. What, if anything, to include in an autotest job to make it suitable for running inside Beaker 4. How the result reporting works when using autotest as a harness inside Beaker (Do the results just go to Beaker? Or does normal autotest reporting still happen as well?)
If there isn't, we'd be happy to accept a patch on Gerrit to add an autotest section to http://beaker-project.org/docs/alternative-harnesses/ :)
Somewhat related, is there a tentative date yet for the next autotest release?
Cheers, Nick.
On Sun, Aug 04, 2013 at 11:52:41PM -0400, Nick Coghlan wrote:
Hi Don, Bill, LMR,
I saw that the pull request for Beaker harness API support in autotest has been merged. I'd like to link to that feature from beaker-project.org, so I was wondering if there was anything published anywhere (even a blog or mailing list post) that covered the details of:
- What to add to a job/recipe definition to specify autotest as the harness
- What, if anything, to add to the task list in the recipe to configure autotest appropriately
- What, if anything, to include in an autotest job to make it suitable for running inside Beaker
- How the result reporting works when using autotest as a harness inside Beaker (Do the results just go to Beaker? Or does normal autotest reporting still happen as well?)
If there isn't, we'd be happy to accept a patch on Gerrit to add an autotest section to http://beaker-project.org/docs/alternative-harnesses/ :)
Somewhat related, is there a tentative date yet for the next autotest release?
Yeah I will have to sit down and write one up. There were still a couple of quirks with the workflow (rebooting for one), that I need to iron out. But I am hoping to re-visit that stuff soon in the next couple of weeks. I need to wrap up another project first.
Cheers,. Don
On 08/09/2013 12:27 AM, Don Zickus wrote:
On Sun, Aug 04, 2013 at 11:52:41PM -0400, Nick Coghlan wrote:
Hi Don, Bill, LMR,
I saw that the pull request for Beaker harness API support in autotest has been merged. I'd like to link to that feature from beaker-project.org, so I was wondering if there was anything published anywhere (even a blog or mailing list post) that covered the details of:
- What to add to a job/recipe definition to specify autotest as the harness
- What, if anything, to add to the task list in the recipe to configure autotest appropriately
- What, if anything, to include in an autotest job to make it suitable for running inside Beaker
- How the result reporting works when using autotest as a harness inside Beaker (Do the results just go to Beaker? Or does normal autotest reporting still happen as well?)
If there isn't, we'd be happy to accept a patch on Gerrit to add an autotest section to http://beaker-project.org/docs/alternative-harnesses/ :)
Somewhat related, is there a tentative date yet for the next autotest release?
Yeah I will have to sit down and write one up. There were still a couple of quirks with the workflow (rebooting for one), that I need to iron out. But I am hoping to re-visit that stuff soon in the next couple of weeks. I need to wrap up another project first.
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
1. How do you get the experimental version of autotest to play with? Is there a pre-built RPM somewhere that people can use, or do they have to build their own? 2. What do you need to set as the alternative harness in http://beaker-project.org/docs/alternative-harnesses/#selecting-an-alternati... ? 3. What's known to work, what's still quirky? (e.g. reboot) 4. What's different (if anything) compared to normal usage of autotest?
At the moment, it's impossible for even the core Beaker team to help with this (let alone anyone else that may be interested), as we don't even know how to get set up to run it.
I'd also like to submit a talk on Beaker to the Continuous Integration miniconf at LCA, and that becomes a lot more compelling when I can point to a post about *how* to run autotest inside Beaker, rather than just to the pull request that made it possible.
Regards, Nick.
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
On 08/09/2013 12:27 AM, Don Zickus wrote:
On Sun, Aug 04, 2013 at 11:52:41PM -0400, Nick Coghlan wrote:
Hi Don, Bill, LMR,
I saw that the pull request for Beaker harness API support in autotest has been merged. I'd like to link to that feature from beaker-project.org, so I was wondering if there was anything published anywhere (even a blog or mailing list post) that covered the details of:
- What to add to a job/recipe definition to specify autotest as the harness
- What, if anything, to add to the task list in the recipe to configure autotest appropriately
- What, if anything, to include in an autotest job to make it suitable for running inside Beaker
- How the result reporting works when using autotest as a harness inside Beaker (Do the results just go to Beaker? Or does normal autotest reporting still happen as well?)
If there isn't, we'd be happy to accept a patch on Gerrit to add an autotest section to http://beaker-project.org/docs/alternative-harnesses/ :)
Somewhat related, is there a tentative date yet for the next autotest release?
Yeah I will have to sit down and write one up. There were still a couple of quirks with the workflow (rebooting for one), that I need to iron out. But I am hoping to re-visit that stuff soon in the next couple of weeks. I need to wrap up another project first.
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own? 2. What do you need to set as the alternative harness in http://beaker-project.org/docs/alternative-harnesses/#selecting-an-alternati... ? 3. What's known to work, what's still quirky? (e.g. reboot) 4. What's different (if anything) compared to normal usage of autotest?
At the moment, it's impossible for even the core Beaker team to help with this (let alone anyone else that may be interested), as we don't even know how to get set up to run it.
I'd also like to submit a talk on Beaker to the Continuous Integration miniconf at LCA, and that becomes a lot more compelling when I can point to a post about *how* to run autotest inside Beaker, rather than just to the pull request that made it possible.
Hmm, I am trying to figure out the workflow I had. Logging back into my old systems is slowly jogging my memory. Let me beat on it some more. Most of my workflow was using beaker tests, so something like
autotest-local bootstrap <http://xml file>
would download the remote xml, convert it to a control file and then run it. Or if it didn't run it, 'autotest-local list' was supposed to show the control to run with 'autotest-local run <control file>'.
I'll work on it tomorrow.
Now that the new api has been released for a while and upstream autotest has most of my code (if not all), then I can simplify my kvm environment to a normal environment and play with it there.
Cheers, Don
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
So playing with things a little more..
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own?
Unfortunately, I see autotest-0.15 has not been packaged with Fedora 20 for some reason.
Lucas,
Why is Fedora using a version of autotest from a year ago instead of something far more recent????
So git clone https://github.com/autotest/autotest.git
will get you the latest autotest.
python setup.py build python setup.py install
(though I had to disable version=version.get_version in setup.py and comment out the BEAKER_RECIPE_ID and BEAKER_LAB_CONTROLLER_URL hacks in client/harness_beaker.py)
will install the autotest pieces
Then as long as you have in your `env` BEAKER_RECIPE_ID and BEAKER_LAB_CONTROLLER_URL defined and AUTOTEST_HARNESS=beaker setup as meta data
autotest-local bootstrap
will request BEAKER_RECIPE_ID xml from BEAKER_LAB_CONTROLLER_URL, convert it into a control file for autotest to consume and start running tests (and reporting them to beaker).
- What do you need to set as the alternative harness in
http://beaker-project.org/docs/alternative-harnesses/#selecting-an-alternati...
I need 'AUTOTEST_HARNESS=beaker' set.
? 3. What's known to work, what's still quirky? (e.g. reboot)
reboot doesn't work for some reason. I don't think running actual autotest-tests work. I have only run with beaker tests through a job. And also running locally doesn't work (even though the tests are locally installed [unless you hack up some recipe id workarounds..]).
- What's different (if anything) compared to normal usage of autotest?
Normal usage of autotest is 'autotest-local <some control file>' or 'autotest-local run <testname>' (where testname is from 'autotest-local list')
But that is running autotest locally/offline. I can get it to work with beaker tests, but I have to tweak some env variables to fake it (not ideal). So run now it has to run automated, which isn't ideal either.
At the moment, it's impossible for even the core Beaker team to help with this (let alone anyone else that may be interested), as we don't even know how to get set up to run it.
Yeah, I was trying to figure out a workflow that I wanted to run autotest with.
Initially I wanted to run tests locally for developers and through automation for QE folks. I can run autotest tests for developers and beaker tests for QE folks. Ideally I need to mix and match in both directions. :-/
It shouldn't be hard to do, just need to hack up some glue code to detect and run things properly.
I'd also like to submit a talk on Beaker to the Continuous Integration miniconf at LCA, and that becomes a lot more compelling when I can point to a post about *how* to run autotest inside Beaker, rather than just to the pull request that made it possible.
Yeah, my stuff needs more work. And more thought. Perhaps some feedback about workflows you had in mind, might help target this a little more.
I'll keep poking at it this week.
Cheers, Don
On 11/05/2013 07:51 AM, Don Zickus wrote:
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
So playing with things a little more..
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own?
Unfortunately, I see autotest-0.15 has not been packaged with Fedora 20 for some reason.
Lucas,
Why is Fedora using a version of autotest from a year ago instead of something far more recent????
So git clone https://github.com/autotest/autotest.git
will get you the latest autotest.
python setup.py build python setup.py install
(though I had to disable version=version.get_version in setup.py and comment out the BEAKER_RECIPE_ID and BEAKER_LAB_CONTROLLER_URL hacks in client/harness_beaker.py)
will install the autotest pieces
Then as long as you have in your `env` BEAKER_RECIPE_ID and BEAKER_LAB_CONTROLLER_URL defined and AUTOTEST_HARNESS=beaker setup as meta data
autotest-local bootstrap
will request BEAKER_RECIPE_ID xml from BEAKER_LAB_CONTROLLER_URL, convert it into a control file for autotest to consume and start running tests (and reporting them to beaker).
OK, that sounds reasonable (especially once the RPM packaging is brought up to date)
- What do you need to set as the alternative harness in
http://beaker-project.org/docs/alternative-harnesses/#selecting-an-alternati...
I need 'AUTOTEST_HARNESS=beaker' set.
OK, in order to get the (ks_meta="harness=<package-or-URL-to-yum-install>") recipe attribute to work properly, it looks like we may also need a "autotest-beaker-harness" RPM that:
1. Depends on autotest itself 2. Sets "AUTOTEST_HARNESS=beaker" in the system environment variables 3. Arranges for "autotest-local bootstrap" to run on the next reboot
Otherwise you'll need to write a Beaker task that installs beah just to bootstrap autotest, and installing beah is one of the things using an alternative harness is supposed to avoid.
? 3. What's known to work, what's still quirky? (e.g. reboot)
reboot doesn't work for some reason. I don't think running actual autotest-tests work. I have only run with beaker tests through a job. And also running locally doesn't work (even though the tests are locally installed [unless you hack up some recipe id workarounds..]).
We're going to have to get the reboot glitch sorted in order to interoperate with the alternative harness selection mechanism.
- What's different (if anything) compared to normal usage of autotest?
Normal usage of autotest is 'autotest-local <some control file>' or 'autotest-local run <testname>' (where testname is from 'autotest-local list')
But that is running autotest locally/offline. I can get it to work with beaker tests, but I have to tweak some env variables to fake it (not ideal). So run now it has to run automated, which isn't ideal either.
The current recommendation for local beah testing is to assume offline execution if JOBID isn't set: http://beaker-project.org/docs/user-guide/multihost.html#standalone-executio...
Something similar may work here: if AUTOTEST_HARNESS=beaker is set, but the BEAKER_* environment variables are missing, perhaps there could be some way to provide the recipe definition XML file directly rather than needing to retrieve it from the server?
At the moment, it's impossible for even the core Beaker team to help with this (let alone anyone else that may be interested), as we don't even know how to get set up to run it.
Yeah, I was trying to figure out a workflow that I wanted to run autotest with.
Initially I wanted to run tests locally for developers and through automation for QE folks. I can run autotest tests for developers and beaker tests for QE folks. Ideally I need to mix and match in both directions. :-/
It shouldn't be hard to do, just need to hack up some glue code to detect and run things properly.
I'd also like to submit a talk on Beaker to the Continuous Integration miniconf at LCA, and that becomes a lot more compelling when I can point to a post about *how* to run autotest inside Beaker, rather than just to the pull request that made it possible.
Yeah, my stuff needs more work. And more thought. Perhaps some feedback about workflows you had in mind, might help target this a little more.
I'll keep poking at it this week.
Hopefully the above helps - it sounds to me like a way to provide an XML definition directly for local testing + the "autotest-beaker-harness" RPM to integrate with the alternative harness selection mechanism are the key next steps.
Then on to the fun part of getting non-Beaker autotest tests running inside Beaker :)
Cheers, Nick.
On 11/05/2013 12:00 AM, Nick Coghlan wrote:
On 11/05/2013 07:51 AM, Don Zickus wrote:
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
So playing with things a little more..
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own?
Unfortunately, I see autotest-0.15 has not been packaged with Fedora 20 for some reason.
Lucas,
Why is Fedora using a version of autotest from a year ago instead of something far more recent????
Because it is a big PITA to get Fedora to approve new package reviewers.
It is been months that cleber is dilligently going from review to review to finally become one, but it usually takes 20 days every iteration. At this staggering rate, he'll become a Fedora package mantainer by 2016. Seriously, I've been thinking "ok, now the package maintainership thing will be sorted out any minute now" for the past 4 months.
Now once he becomes a package maintainer, up to date packages will start to show up.
On Tue, Nov 05, 2013 at 01:00:15AM -0200, Lucas Meneghel Rodrigues wrote:
On 11/05/2013 12:00 AM, Nick Coghlan wrote:
On 11/05/2013 07:51 AM, Don Zickus wrote:
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
So playing with things a little more..
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own?
Unfortunately, I see autotest-0.15 has not been packaged with Fedora 20 for some reason.
Lucas,
Why is Fedora using a version of autotest from a year ago instead of something far more recent????
Because it is a big PITA to get Fedora to approve new package reviewers.
It is been months that cleber is dilligently going from review to review to finally become one, but it usually takes 20 days every iteration. At this staggering rate, he'll become a Fedora package mantainer by 2016. Seriously, I've been thinking "ok, now the package maintainership thing will be sorted out any minute now" for the past 4 months.
Now once he becomes a package maintainer, up to date packages will start to show up.
Odd. Maybe that is for new packages. For existing packages it has taken me less than a week to get approved to become a maintainer (with commit privileges) for a package.
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Cheers, Don
On 11/05/2013 12:11 PM, Don Zickus wrote:
On Tue, Nov 05, 2013 at 01:00:15AM -0200, Lucas Meneghel Rodrigues wrote:
On 11/05/2013 12:00 AM, Nick Coghlan wrote:
On 11/05/2013 07:51 AM, Don Zickus wrote:
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
So playing with things a little more..
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own?
Unfortunately, I see autotest-0.15 has not been packaged with Fedora 20 for some reason.
Lucas,
Why is Fedora using a version of autotest from a year ago instead of something far more recent????
Because it is a big PITA to get Fedora to approve new package reviewers.
It is been months that cleber is dilligently going from review to review to finally become one, but it usually takes 20 days every iteration. At this staggering rate, he'll become a Fedora package mantainer by 2016. Seriously, I've been thinking "ok, now the package maintainership thing will be sorted out any minute now" for the past 4 months.
Now once he becomes a package maintainer, up to date packages will start to show up.
Odd. Maybe that is for new packages. For existing packages it has taken me less than a week to get approved to become a maintainer (with commit privileges) for a package.
That is for new packages, because in order to be accepted as a packager, you *have* to package a new package, which I find puzzling. I mean, what if I want to assume an already existing package that was orphaned?
So Cleber, in order to become a maintainer, had to present a new software to package (arc, the new autotest rpc client that he wrote), and he's been since then trying to get all the *shoulds* and whatnot. He took 2 months writing a system to generate the man pages for arc, and other requests. You can see a summary of what is going on here:
https://bugzilla.redhat.com/show_bug.cgi?id=985967
It seems now that the package reviewers want him to participate in other package reviews to demonstrate 'he knows the rules', so while he doesn't post links to said package reviews he chimed in, they'll keep the request on hold.
Don,
Thanks for your help. It's certainly needed and much appreciatted.
But, I had to take this week as PTO because of some unforeseen events, so I won't be able to follow up in a timely manner. Next week I'll get to you and hopefully we'll expedite this and get it done.
I'll ping you early next week.
Thanks a lot!
----- Don Zickus dzickus@redhat.com wrote:
On Tue, Nov 05, 2013 at 01:00:15AM -0200, Lucas Meneghel Rodrigues wrote:
On 11/05/2013 12:00 AM, Nick Coghlan wrote:
On 11/05/2013 07:51 AM, Don Zickus wrote:
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
So playing with things a little more..
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own?
Unfortunately, I see autotest-0.15 has not been packaged with Fedora 20 for some reason.
Lucas,
Why is Fedora using a version of autotest from a year ago instead of something far more recent????
Because it is a big PITA to get Fedora to approve new package reviewers.
It is been months that cleber is dilligently going from review to review to finally become one, but it usually takes 20 days every iteration. At this staggering rate, he'll become a Fedora package mantainer by 2016. Seriously, I've been thinking "ok, now the package maintainership thing will be sorted out any minute now" for the past 4 months.
Now once he becomes a package maintainer, up to date packages will start to show up.
Odd. Maybe that is for new packages. For existing packages it has taken me less than a week to get approved to become a maintainer (with commit privileges) for a package.
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Cheers, Don
On Thu, Nov 07, 2013 at 04:52:32AM -0500, Cleber Rosa wrote:
Don,
Thanks for your help. It's certainly needed and much appreciatted.
But, I had to take this week as PTO because of some unforeseen events, so I won't be able to follow up in a timely manner. Next week I'll get to you and hopefully we'll expedite this and get it done.
I'll ping you early next week.
I just requested acls on that package in the meantime. I will look to see if I can sponsor you (or find someone who can :-) ).
Cheers, Don
Thanks a lot!
----- Don Zickus dzickus@redhat.com wrote:
On Tue, Nov 05, 2013 at 01:00:15AM -0200, Lucas Meneghel Rodrigues wrote:
On 11/05/2013 12:00 AM, Nick Coghlan wrote:
On 11/05/2013 07:51 AM, Don Zickus wrote:
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
So playing with things a little more..
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own?
Unfortunately, I see autotest-0.15 has not been packaged with Fedora 20 for some reason.
Lucas,
Why is Fedora using a version of autotest from a year ago instead of something far more recent????
Because it is a big PITA to get Fedora to approve new package reviewers.
It is been months that cleber is dilligently going from review to review to finally become one, but it usually takes 20 days every iteration. At this staggering rate, he'll become a Fedora package mantainer by 2016. Seriously, I've been thinking "ok, now the package maintainership thing will be sorted out any minute now" for the past 4 months.
Now once he becomes a package maintainer, up to date packages will start to show up.
Odd. Maybe that is for new packages. For existing packages it has taken me less than a week to get approved to become a maintainer (with commit privileges) for a package.
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Cheers, Don
On 11/05/2013 11:11 AM, Don Zickus wrote:
On Tue, Nov 05, 2013 at 01:00:15AM -0200, Lucas Meneghel Rodrigues wrote:
On 11/05/2013 12:00 AM, Nick Coghlan wrote:
On 11/05/2013 07:51 AM, Don Zickus wrote:
On Tue, Oct 29, 2013 at 01:33:33PM +1000, Nick Coghlan wrote:
Ping :)
Even a cursory blog post somewhere (or a reply to this email!) explaining to people how to *experiment* with this would be helpful. At the moment I'm having to point people at the merged pull request to say "Yes, in theory, you can use the upcoming version of autotest as a Beaker test harness", but I have zero resources to give them on how to actually run it (because I don't actually know myself).
So playing with things a little more..
- How do you get the experimental version of autotest to play with? Is
there a pre-built RPM somewhere that people can use, or do they have to build their own?
Unfortunately, I see autotest-0.15 has not been packaged with Fedora 20 for some reason.
Lucas,
Why is Fedora using a version of autotest from a year ago instead of something far more recent????
Because it is a big PITA to get Fedora to approve new package reviewers.
It is been months that cleber is dilligently going from review to review to finally become one, but it usually takes 20 days every iteration. At this staggering rate, he'll become a Fedora package mantainer by 2016. Seriously, I've been thinking "ok, now the package maintainership thing will be sorted out any minute now" for the past 4 months.
Now once he becomes a package maintainer, up to date packages will start to show up.
Odd. Maybe that is for new packages. For existing packages it has taken me less than a week to get approved to become a maintainer (with commit privileges) for a package.
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Don,
Now would be a good time to get some help here.
Please check the package review BZ at:
https://bugzilla.redhat.com/show_bug.cgi?id=985967
Thanks.
Cheers, Don _______________________________________________ Beaker-devel mailing list Beaker-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
On Thu, Dec 05, 2013 at 03:38:58PM -0300, Cleber Rosa wrote:
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Don,
Now would be a good time to get some help here.
Please check the package review BZ at:
Dumb question. What are you trying to achieve with that bugzilla? Are you just trying to become a developer with commit access to the autotest package? Or something bigger like a generic package maintainer?
The above bugzilla seems to imply the latter hence all the pushback. The former should be easier as all it needs is approval from Martin (though I requested approval a month ago and have heard nothing).
Cheers, Don
On 12/05/2013 04:57 PM, Don Zickus wrote:
On Thu, Dec 05, 2013 at 03:38:58PM -0300, Cleber Rosa wrote:
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Don,
Now would be a good time to get some help here.
Please check the package review BZ at:
Dumb question. What are you trying to achieve with that bugzilla? Are you just trying to become a developer with commit access to the autotest package? Or something bigger like a generic package maintainer?
The above bugzilla seems to imply the latter hence all the pushback. The former should be easier as all it needs is approval from Martin (though I requested approval a month ago and have heard nothing).
Cleber is not a Fedora package maintainer, that's why he's requested to become one, otherwise, he can't assume autotest, or so we thought.
Don,
First thing I tried was asking Martin the same thing. He told me that it did not depend on him, and that I'd have to go though the process of becoming a maintainer.
Having walked all those miles, even if becoming a general package maintainer wasn't really necessary, I think it's the one to go because we do have other packages (such as python-arc itself) that we want to have in Fedora.
Cheers, CR.
----- Original Message -----
From: "Don Zickus" dzickus@redhat.com To: "Cleber Rosa" crosa@redhat.com Cc: "Beaker development" beaker-devel@lists.fedorahosted.org, "Lucas Meneghel Rodrigues" lmr@redhat.com Sent: Thursday, December 5, 2013 3:57:47 PM Subject: Re: [Beaker-devel] Docs/overview for using autotest in Beaker?
On Thu, Dec 05, 2013 at 03:38:58PM -0300, Cleber Rosa wrote:
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Don,
Now would be a good time to get some help here.
Please check the package review BZ at:
Dumb question. What are you trying to achieve with that bugzilla? Are you just trying to become a developer with commit access to the autotest package? Or something bigger like a generic package maintainer?
The above bugzilla seems to imply the latter hence all the pushback. The former should be easier as all it needs is approval from Martin (though I requested approval a month ago and have heard nothing).
Cheers, Don
On Thu, Dec 05, 2013 at 02:43:35PM -0500, Cleber Rosa wrote:
Don,
First thing I tried was asking Martin the same thing. He told me that it did not depend on him, and that I'd have to go though the process of becoming a maintainer.
So talking with fedora folks, all you have to do is request commit acls for project autotest-framework. Martin is the owner, so he has the power to approve/reject that request.
Having walked all those miles, even if becoming a general package maintainer wasn't really necessary, I think it's the one to go because we do have other packages (such as python-arc itself) that we want to have in Fedora.
As for python-arc, it should be a new package, so all it has to be is approved and then you can do whatever you want with it. I can't imagine _every_ Fedora maintainer having to get the 'provenpackager' status.
I know I didn't, I just got sponsered by a Red Hat person to get into the Fedora database. After that a few mouse clicks is all that is needed to allow me to commit to packages.
We should ping Martin again to see what the hold up is.
Cheers, Don
Cheers, CR.
----- Original Message -----
From: "Don Zickus" dzickus@redhat.com To: "Cleber Rosa" crosa@redhat.com Cc: "Beaker development" beaker-devel@lists.fedorahosted.org, "Lucas Meneghel Rodrigues" lmr@redhat.com Sent: Thursday, December 5, 2013 3:57:47 PM Subject: Re: [Beaker-devel] Docs/overview for using autotest in Beaker?
On Thu, Dec 05, 2013 at 03:38:58PM -0300, Cleber Rosa wrote:
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Don,
Now would be a good time to get some help here.
Please check the package review BZ at:
Dumb question. What are you trying to achieve with that bugzilla? Are you just trying to become a developer with commit access to the autotest package? Or something bigger like a generic package maintainer?
The above bugzilla seems to imply the latter hence all the pushback. The former should be easier as all it needs is approval from Martin (though I requested approval a month ago and have heard nothing).
Cheers, Don
On Thu, 5 Dec 2013 16:05:11 -0500 Don Zickus dzickus@redhat.com wrote:
On Thu, Dec 05, 2013 at 02:43:35PM -0500, Cleber Rosa wrote:
Don,
First thing I tried was asking Martin the same thing. He told me that it did not depend on him, and that I'd have to go though the process of becoming a maintainer.
So talking with fedora folks, all you have to do is request commit acls for project autotest-framework. Martin is the owner, so he has the power to approve/reject that request.
Well, not quite. It depends on whether or not you're already a packager. If you are, then it's as simple as adding the acls. If you're not, the primary maintainer needs to file a FESCo ticket requesting that the person be added as a packager.
Having walked all those miles, even if becoming a general package maintainer wasn't really necessary, I think it's the one to go because we do have other packages (such as python-arc itself) that we want to have in Fedora.
As for python-arc, it should be a new package, so all it has to be is approved and then you can do whatever you want with it. I can't imagine _every_ Fedora maintainer having to get the 'provenpackager' status.
Not sure how provenpackagers fit into this discussion. There aren't that many people with provenpackager powers and that's how it should be.
I know I didn't, I just got sponsered by a Red Hat person to get into the Fedora database. After that a few mouse clicks is all that is needed to allow me to commit to packages.
The three ways that I can think of off hand to become a packager: 1. get sponsored when submitting a package review 2. be added as a co-maintainer to an existing package 3. take over a just-orphaned packager with a sponsor's help
We should ping Martin again to see what the hold up is.
I don't pretend to have all the context here, but I imagine that it just fell of his radar. You may have better luck waiting until F20 is released. We're very much in "all hands on deck" mode trying to limit final to a 1 week slip.
Tim
Cheers, Don
Cheers, CR.
----- Original Message -----
From: "Don Zickus" dzickus@redhat.com To: "Cleber Rosa" crosa@redhat.com Cc: "Beaker development" beaker-devel@lists.fedorahosted.org, "Lucas Meneghel Rodrigues" lmr@redhat.com Sent: Thursday, December 5, 2013 3:57:47 PM Subject: Re: [Beaker-devel] Docs/overview for using autotest in Beaker?
On Thu, Dec 05, 2013 at 03:38:58PM -0300, Cleber Rosa wrote:
Cleber, can you point me to some of the hold-ups on getting you approved? Being in Westford, I might be able to poke people directly to move things along quicker. 20 days seems awfully long and would be a big turn-off to get new Fedora contributors/maintainers (something I assumed Fedora wanted to avoid).
Don,
Now would be a good time to get some help here.
Please check the package review BZ at:
Dumb question. What are you trying to achieve with that bugzilla? Are you just trying to become a developer with commit access to the autotest package? Or something bigger like a generic package maintainer?
The above bugzilla seems to imply the latter hence all the pushback. The former should be easier as all it needs is approval from Martin (though I requested approval a month ago and have heard nothing).
Cheers, Don
Beaker-devel mailing list Beaker-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
On Tue, Nov 05, 2013 at 12:00:59PM +1000, Nick Coghlan wrote:
will request BEAKER_RECIPE_ID xml from BEAKER_LAB_CONTROLLER_URL, convert it into a control file for autotest to consume and start running tests (and reporting them to beaker).
OK, that sounds reasonable (especially once the RPM packaging is brought up to date)
- What do you need to set as the alternative harness in
http://beaker-project.org/docs/alternative-harnesses/#selecting-an-alternati...
I need 'AUTOTEST_HARNESS=beaker' set.
OK, in order to get the (ks_meta="harness=<package-or-URL-to-yum-install>") recipe attribute to work properly, it looks like we may also need a "autotest-beaker-harness" RPM that:
Yeah, Bill explained (and showed) how that works. So now I have the right variables. Though I was caught by surprise when my tasks timed out because nothing communicated to beaker that distribution-install was successfull. ;-)
- Depends on autotest itself
- Sets "AUTOTEST_HARNESS=beaker" in the system environment variables
- Arranges for "autotest-local bootstrap" to run on the next reboot
Yeah, I will sort this out a little later. Using Bill's idea of the %post section is probably a good start for now.
Otherwise you'll need to write a Beaker task that installs beah just to bootstrap autotest, and installing beah is one of the things using an alternative harness is supposed to avoid.
? 3. What's known to work, what's still quirky? (e.g. reboot)
reboot doesn't work for some reason. I don't think running actual autotest-tests work. I have only run with beaker tests through a job. And also running locally doesn't work (even though the tests are locally installed [unless you hack up some recipe id workarounds..]).
We're going to have to get the reboot glitch sorted in order to interoperate with the alternative harness selection mechanism.
Yup. :-)
- What's different (if anything) compared to normal usage of autotest?
Normal usage of autotest is 'autotest-local <some control file>' or 'autotest-local run <testname>' (where testname is from 'autotest-local list')
But that is running autotest locally/offline. I can get it to work with beaker tests, but I have to tweak some env variables to fake it (not ideal). So run now it has to run automated, which isn't ideal either.
The current recommendation for local beah testing is to assume offline execution if JOBID isn't set: http://beaker-project.org/docs/user-guide/multihost.html#standalone-executio...
Something similar may work here: if AUTOTEST_HARNESS=beaker is set, but the BEAKER_* environment variables are missing, perhaps there could be some way to provide the recipe definition XML file directly rather than needing to retrieve it from the server?
Well, we still have /root/RECIPE.txt and LABC_URL. Combine those pieces and one can figure out how to get the xml file in an automated way.
At the moment, it's impossible for even the core Beaker team to help with this (let alone anyone else that may be interested), as we don't even know how to get set up to run it.
Yeah, I was trying to figure out a workflow that I wanted to run autotest with.
Initially I wanted to run tests locally for developers and through automation for QE folks. I can run autotest tests for developers and beaker tests for QE folks. Ideally I need to mix and match in both directions. :-/
It shouldn't be hard to do, just need to hack up some glue code to detect and run things properly.
I'd also like to submit a talk on Beaker to the Continuous Integration miniconf at LCA, and that becomes a lot more compelling when I can point to a post about *how* to run autotest inside Beaker, rather than just to the pull request that made it possible.
Yeah, my stuff needs more work. And more thought. Perhaps some feedback about workflows you had in mind, might help target this a little more.
I'll keep poking at it this week.
Hopefully the above helps - it sounds to me like a way to provide an XML definition directly for local testing + the "autotest-beaker-harness" RPM to integrate with the alternative harness selection mechanism are the key next steps.
Then on to the fun part of getting non-Beaker autotest tests running inside Beaker :)
Yeah, getting there. For now how about a beaker patch?
Cheers, Don
--------------------8<---------------- From: Don Zickus dzickus@redhat.com Date: Fri, 8 Nov 2013 14:03:34 -0500 Subject: [PATCH] Tasks: make reservesys always exit 0
Different harnesses have a different environment than what reservesys was coded for. As a result, the task may fail for something silly
/usr/bin/killall rhts-test-runner.sh rhts-test-runner.sh: no process found
because the test harness I am using, doesn't run rhts-test-runner.sh.
I do not believe there is any reason to have reservesys fail. The test was built to always succeed. Force that success with an 'exit 0' at the end.
Signed-off-by: Don Zickus dzickus@redhat.com --- Tasks/distribution/reservesys/runtest.sh | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/Tasks/distribution/reservesys/runtest.sh b/Tasks/distribution/reservesys/runtest.sh index 3a79b90..b4353e3 100755 --- a/Tasks/distribution/reservesys/runtest.sh +++ b/Tasks/distribution/reservesys/runtest.sh @@ -242,3 +242,6 @@ RprtRslt $TEST PASS 0
# stop rhts service, So that reserve workflow works with test reboot support. STOPRHTS + +# different harness may cause this script to fail, always return zero +exit 0
On Fri, Nov 08, 2013 at 02:38:20PM -0500, Don Zickus wrote:
Hopefully the above helps - it sounds to me like a way to provide an XML definition directly for local testing + the "autotest-beaker-harness" RPM to integrate with the alternative harness selection mechanism are the key next steps.
Then on to the fun part of getting non-Beaker autotest tests running inside Beaker :)
Yeah, getting there. For now how about a beaker patch?
bump.
Cheers, Don
--------------------8<---------------- From: Don Zickus dzickus@redhat.com Date: Fri, 8 Nov 2013 14:03:34 -0500 Subject: [PATCH] Tasks: make reservesys always exit 0
Different harnesses have a different environment than what reservesys was coded for. As a result, the task may fail for something silly
/usr/bin/killall rhts-test-runner.sh rhts-test-runner.sh: no process found
because the test harness I am using, doesn't run rhts-test-runner.sh.
I do not believe there is any reason to have reservesys fail. The test was built to always succeed. Force that success with an 'exit 0' at the end.
Signed-off-by: Don Zickus dzickus@redhat.com
Tasks/distribution/reservesys/runtest.sh | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
diff --git a/Tasks/distribution/reservesys/runtest.sh b/Tasks/distribution/reservesys/runtest.sh index 3a79b90..b4353e3 100755 --- a/Tasks/distribution/reservesys/runtest.sh +++ b/Tasks/distribution/reservesys/runtest.sh @@ -242,3 +242,6 @@ RprtRslt $TEST PASS 0
# stop rhts service, So that reserve workflow works with test reboot support. STOPRHTS
+# different harness may cause this script to fail, always return zero
+exit 0
1.7.1
Beaker-devel mailing list Beaker-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/beaker-devel
On 11/16/2013 12:35 AM, Don Zickus wrote:
On Fri, Nov 08, 2013 at 02:38:20PM -0500, Don Zickus wrote:
Hopefully the above helps - it sounds to me like a way to provide an XML definition directly for local testing + the "autotest-beaker-harness" RPM to integrate with the alternative harness selection mechanism are the key next steps.
Then on to the fun part of getting non-Beaker autotest tests running inside Beaker :)
Yeah, getting there. For now how about a beaker patch?
bump.
Added to Gerrit: http://gerrit.beaker-project.org/2507
Patches that aren't sent to Gerrit (even small ones) are highly likely to be lost.
Cheers, Nick.
On Mon, Nov 18, 2013 at 04:24:49PM +1000, Nick Coghlan wrote:
On 11/16/2013 12:35 AM, Don Zickus wrote:
On Fri, Nov 08, 2013 at 02:38:20PM -0500, Don Zickus wrote:
Hopefully the above helps - it sounds to me like a way to provide an XML definition directly for local testing + the "autotest-beaker-harness" RPM to integrate with the alternative harness selection mechanism are the key next steps.
Then on to the fun part of getting non-Beaker autotest tests running inside Beaker :)
Yeah, getting there. For now how about a beaker patch?
bump.
Added to Gerrit: http://gerrit.beaker-project.org/2507
Patches that aren't sent to Gerrit (even small ones) are highly likely to be lost.
Sorry about that. I wasn't used to the process.
Cheers, Don
beaker-devel@lists.fedorahosted.org