proposal: deprecate using raise error.TestFail

Kamil Paral kparal at redhat.com
Fri Sep 10 09:07:29 UTC 2010


----- "Will Woods" <wwoods at redhat.com> wrote:

> On Wed, 2010-09-08 at 15:03 -0400, James Laska wrote:
> > On Wed, 2010-09-08 at 07:41 -0400, Kamil Paral wrote:
> > > 
> > > I'd like to propose that we stop using those autotest-internal
> TestFail 
> > > and TestWarn exceptions.
> > > 
> > > What will change:
> > > 1. We no longer see red/purple boxes in autotest frontend.
> > 
> > Hah, I was thinking the same thing when inspecting the autotest tko
> > results viewer.  It's hard to distinguish between a successful test
> run
> > that found problems, vs a test case failure.
> 
> Yeah - I think this would clean things up significantly for us, and
> is
> therefore a pretty great idea.
> 
> But I wish I knew why the autotest developers originally decided to
> use
> these exceptions rather than return values from the tests - surely
> there
> was some good reason for that design, and I'd like to know what we're
> losing by discarding it.

I'm looking into autotest/client/common_lib/error.py and I see there a
lot of exception classes - much more than I expected. It seems that
there are exception types for all kinds of events. But those events are
related more to job execution itself, rather than some testcase semantics.
Currently we mix those two things.

The reason why it is done by exceptions rather than method return values
is I think that it's just easier - you can wrap the whole test object
in try-except block and catch any problem. It will also end any further
methods, so normally when your raise TestFail exception in run_once, then
postprocess_iteration is no longer executed (in standard autotest 
environment, not in our case).

If we catch all exceptions in our AutoQATest class, it doesn't mean we
can no longer raise exceptions, it just means we shouldn't raise them
for tests that ended correctly but we deem them as failing some testcase
(which is a quite different semantics). So for example when I run some
external script and that script crashes (and there is no point in 
continuing), I can still raise TestFail. That means that we will catch
that exception and automatically report this test run as CRASHED. Which
is quite cool I think.


More information about the autoqa-devel mailing list