Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
This is the test plan we'll use to determine if rawhide is 'broken' or not. Eventually this will evolve into a set of automated tests and public reports about rawhide.. but first I want to be sure we're testing the right things.
Remember, right now this isn't about *how* we're going to test, just *what* we're going to test. We'll get to *how* in a little bit.
Please give it a look - especially the Pass/Fail criteria and Test Cases - to make sure we've covered the critical parts for making sure Rawhide isn't broken.
Thanks!
-w
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
Er. *here* it is:
https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
-w
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
Er. *here* it is:
I can think of one thing, though it may (or may not, I'm not sure) be hard to test in an automated way: does input work?
AFAICT none of the other tests imply this, but a system that passes all the other tests but into which you can't type anything isn't much use :)
On Thu, 2009-06-18 at 14:25 -0700, Adam Williamson wrote:
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
Er. *here* it is:
I can think of one thing, though it may (or may not, I'm not sure) be hard to test in an automated way: does input work?
AFAICT none of the other tests imply this, but a system that passes all the other tests but into which you can't type anything isn't much use :)
Very true - but very hard to automate without lego-mindstorms-style hackery.
I think probably 'israwhidebroken.com' (or its sprititual equivalent) will need to have a box for "Other warnings" to allow people to make note of problems like these.
Of course, that means it needs to allow user submissions, which makes the entire web thing a lot more complex. Alas.
-w
On Thu, 2009-06-18 at 17:33 -0400, Will Woods wrote:
AFAICT none of the other tests imply this, but a system that passes all the other tests but into which you can't type anything isn't much use :)
Very true - but very hard to automate without lego-mindstorms-style hackery.
How about the Homer Simpson solution?
On Thu, 2009-06-18 at 14:41 -0700, Adam Williamson wrote:
On Thu, 2009-06-18 at 17:33 -0400, Will Woods wrote:
AFAICT none of the other tests imply this, but a system that passes all the other tests but into which you can't type anything isn't much use :)
Very true - but very hard to automate without lego-mindstorms-style hackery.
How about the Homer Simpson solution?
"Stupid bird! I never should have left you in charge!"
Heh. Anyway, I've added a functional test requirement for mouse/keyboard input. Thanks for the idea.
-w
On Fri, 2009-06-19 at 11:39 -0400, Will Woods wrote:
On Thu, 2009-06-18 at 14:41 -0700, Adam Williamson wrote:
On Thu, 2009-06-18 at 17:33 -0400, Will Woods wrote:
AFAICT none of the other tests imply this, but a system that passes all the other tests but into which you can't type anything isn't much use :)
Very true - but very hard to automate without lego-mindstorms-style hackery.
How about the Homer Simpson solution?
"Stupid bird! I never should have left you in charge!"
Heh. Anyway, I've added a functional test requirement for mouse/keyboard input. Thanks for the idea.
There's this: http://www.wiebetech.com/products/MouseJiggler.php but I can't think of a way to make a useful automated test from it (it's meant more for computer forensics against encrypted machines than automated tests).
Dave
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Will Woods wrote:
On Thu, 2009-06-18 at 14:25 -0700, Adam Williamson wrote:
I can think of one thing, though it may (or may not, I'm not sure) be hard to test in an automated way: does input work?
Very true - but very hard to automate without lego-mindstorms-style hackery.
I'm not sure that is true, at least for USB input. Couldn't you connect one computer's USB port to another's and have the first simulate a HID keyboard connected to the second?
smithj
On Jun 18, 2009, at 2:33 PM, Will Woods wwoods@redhat.com wrote:
On Thu, 2009-06-18 at 14:25 -0700, Adam Williamson wrote:
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
Er. *here* it is:
I can think of one thing, though it may (or may not, I'm not sure) be hard to test in an automated way: does input work?
AFAICT none of the other tests imply this, but a system that passes all the other tests but into which you can't type anything isn't much use :)
Very true - but very hard to automate without lego-mindstorms-style hackery.
I think probably 'israwhidebroken.com' (or its sprititual equivalent) will need to have a box for "Other warnings" to allow people to make note of problems like these.
Of course, that means it needs to allow user submissions, which makes the entire web thing a lot more complex. Alas.
-w
Why not have user reporting done with fedorahosted trac instance or a new bz comp and xlmrpc off of that. As long a comp keyboard/mouse-brk has an open bug fail the test. If is I hw we done care too much about leave it but change the status to a non blocking state.
Regards, Brennan Ashton
On Thu, Jun 18, 2009 at 1:23 PM, Will Woodswwoods@redhat.com wrote:
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
Er. *here* it is:
https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
-w
Should test cases for both encrypted and unencrypted roots be included?
tom
On Fri, 2009-06-19 at 09:16 -0700, Tom London wrote:
On Thu, Jun 18, 2009 at 1:23 PM, Will Woodswwoods@redhat.com wrote:
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
Er. *here* it is:
https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
-w
Should test cases for both encrypted and unencrypted roots be included?
Yes, but not here. This plan outlines a *very* basic check to see whether Rawhide is even worth testing further.
-w
On Fri, 2009-06-19 at 12:27 -0400, Will Woods wrote:
On Fri, 2009-06-19 at 09:16 -0700, Tom London wrote:
On Thu, Jun 18, 2009 at 1:23 PM, Will Woodswwoods@redhat.com wrote:
On Thu, 2009-06-18 at 16:23 -0400, Will Woods wrote:
Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
Er. *here* it is:
https://fedoraproject.org/wiki/QA:Rawhide_Acceptance_Test_Plan
-w
Should test cases for both encrypted and unencrypted roots be included?
Yes, but not here. This plan outlines a *very* basic check to see whether Rawhide is even worth testing further.
Tests for [un]encrypted root can be found in the installer test plan Liam posted earlier.
https://www.redhat.com/archives/fedora-test-list/2009-June/msg00529.html
Thanks, James
On 06/18/2009 08:23 PM, Will Woods wrote:
Hi folks,
Here's the first draft of a plan describing Rawhide Acceptance Testing.
This is the test plan we'll use to determine if rawhide is 'broken' or not. Eventually this will evolve into a set of automated tests and public reports about rawhide.. but first I want to be sure we're testing the right things.
Remember, right now this isn't about *how* we're going to test, just *what* we're going to test. We'll get to *how* in a little bit.
Please give it a look - especially the Pass/Fail criteria and Test Cases
- to make sure we've covered the critical parts for making sure Rawhide
isn't broken.
Thanks!
HEY!!!
Let's not forget to check if an succesful ( wired ) network connection can be established... ;)
JBG