Future Requirements/Desires for Fedora Test Automation

Tim Flink tflink at redhat.com
Wed Jan 16 23:29:00 UTC 2013


On Wed, 16 Jan 2013 00:06:49 -0200
Lucas Meneghel Rodrigues <lmr at redhat.com> wrote:

> On 01/15/2013 09:25 PM, Tim Flink wrote:
> > I don't think this particular conversation ever made it very far and
> > IIRC, hasn't been on the list yet but I want to get it started
> > before FUDCon NA this weekend. I've cc'd David and Matthew because
> > I've talked with them about Fedora test automation recently and
> > they might have input.
> >
> > In my opinion (I suspect that other people feel similarly), AutoQA
> > in it's current form is not capable of meeting the test automation
> > needs for Fedora; mostly because we don't have a clear path towards
> > external tests and it seems pretty clear that the current devs
> > (myself included) don't have the bandwidth to add any more tests to
> > the current set.
> >
> > There has been some casual conversation about looking into switching
> > over to using Beaker [1] so that we can leverage some of the tests
> > currently being used by various groups within Red Hat instead of
> > having to rewrite them for AutoQA/Autotest. However, I don't want
> > this to sound like 'autotest is bad'. I sincerely doubt that there
> > is a single framework/runner out there which will 100% satisfy all
> > of our needs and I'm just looking to re-evaluate what we want from
> > test automation before deciding how we get there.
> 
> As one would expect, I'd jump right away into the discussion.

Despite the fact that I said that I wanted to _not_ get into framework
and implementation specifics in this thread twice in the part of the
email you didn't quote. Oh well, I got an email from a beaker dev, too
so I can't fault you too much :-P

I just wanted to start the conversation with what we wanted to have for
Fedora test automation so that there was a basis for comparison before
we dove into 'autotest does X', 'beaker does Y' and so on.

> One thing about beaker is that the test harness (Beah) has some 
> limitations that drive people to look into autotest frequently (I
> talk to people inside the company interested in it). The folks at
> beaker made some experiments using the autotest client as one of the
> possible harnesses inside their framework, but I haven't heard much
> from them after the initial patches.
> 
> It is true that we have scarce resources. My team has in fact 4
> people (not all of us allocated full time) and we spend much of the
> time writing and reviewing virtualization tests, our main
> attribution. So it's understandable that you want to change.

In my mind, it's not so much of a "autotest won't work for us" as a
"we're not using much of what autotest can do and I'm not sure our
needs are all that compatible without quite a bit of work". I sincerely
doubt that we're going to get all of what we want/need in fedora
without at least submitting code to an existing framework, though so
the desired direction of any upstream would certainly be a factor.

> I'm not sure whether I was actually called into this discussion, so
> I'll refrain to comment on the requirements. I still think that
> autotest provides at least a part of what is required.

Yeah, I was hoping to talk to you about autotest this weekend since
we're both going to be at FUDCon NA. I don't pretend to understand all
of what autotest can do and figured you would have more insight.

Tim
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <https://lists.fedorahosted.org/pipermail/autoqa-devel/attachments/20130116/b8348989/attachment.sig>


More information about the autoqa-devel mailing list