On Thu, 2009-04-02 at 11:45 -0700, Adam Williamson wrote:
On Thu, 2009-04-02 at 11:11 -0700, Adam Williamson wrote:
> > c) make the test cases as automated as possible. Offer sending
smolt
> > profile (smoltGUI) and run test cases directly from the desktop.
Also We
> > should add a link (or the document itself) about how to test what
and
> > why to the desktop. If it's possible for the test generated
results
> > should also be posted automagically to the results page, or, if
tests
> > fail, bug reports could be created by bugbuddy.
>
> These would all be nice, but we *are* running on a one-week cycle
here,
> it may be hard to get it all done. Assistance welcome. =) it's
pretty
> hard to automate some elements of some test cases, but some could
> certainly be done with scripts etc.
Oh, I forgot to mention another consideration here: I'm trying to
write
these test cases to be pretty future-proof, so we could use them
virtually unmodified for F12, F13, F14...after all, the URLs don't
have
expire-by dates and the test cases will still be hanging around in 10
years most likely. I suspect scripts are liable to go stale faster
than instructions.
My initial concern was the lack of defined test cases. Thankfully that
problem is slowly going away (see
https://fedoraproject.org/wiki/Category:Test_Cases). I'm always in
favor of automation where possible, but I try to make sure we know what
we automate before we dive into the shell.
However, with the test case "library" expanding ... it isn't a bad idea
to begin thinking as scripts/automation comes online, where/how will we
store these tests. Depending on the test target, automation may already
exist. So leveraging any off-the-shelf test automation is always a
"good thing." Some of these questions will naturally tie into the
beaker project as that matures, but there's no reason we can't
discuss/debate these issues now.
Thanks,
James