proposal: deprecate using raise error.TestFail

Kamil Paral kparal at redhat.com
Wed Sep 8 11:41:09 UTC 2010


Hi,

currently many tests use this code in its test objects:

  raise error.TestFail
-or-
  raise error.TestWarn

to end a test that "failed" (from our perspective). It then creates a few
ugly exceptions in the output, see here, even if everything went just fine
(from autotest perspective):

http://autoqa.fedoraproject.org/results/250-autotest/qa03.c.fedoraproject.org/debug/client.DEBUG

and finally it reports:

09/08 12:48:13 INFO |       job:1254| 		FAIL	rpmlint	rpmlint	timestamp=1283950093	localtime=Sep 08 12:48:13	
09/08 12:48:13 INFO |       job:1254| 	END FAIL	rpmlint	rpmlint	timestamp=1283950093	localtime=Sep 08 12:48:13	

(which is also visible in autotest web frontend).

However, since we now use jskladan's patchset defining AutoQATest parent 
class (using self.result variable), I no longer see a reason to raise those 
exceptions. Quite the contrary.
Those exception don't even match our result states, which we have a much richer
set (passed, failed, info, aborted, crashed and maybe some other).

I'd like to propose that we stop using those autotest-internal TestFail 
and TestWarn exceptions.

What will change:
1. We no longer see red/purple boxes in autotest frontend.
2. The AutoQATest parent class will be adjusted to automatically output 
the test result after a test has finished.
3. Tests will just need to store proper result in self.result variable,
that's all. The methods may be then ended in a standard manner (return
or no statement).

What will be the benefits:
1. We will know that if we see a traceback it's a problem. We won't have to
examine in detail whether this traceback is "good" or "bad".
2. It will allow us to report any crashed test object (throwing an exception)
to the mailing list. We have already started to do that and it's great:
https://fedorahosted.org/pipermail/autoqa-results/2010-September/thread.html
But currently it works only for those tests that haven't filled self.result
before the crash occurred. We need to catch all exceptions.

What do you think?


More information about the autoqa-devel mailing list