From: "Dan Callaghan" <dcallagh(a)redhat.com>
To: "beaker-devel" <beaker-devel(a)lists.fedorahosted.org>
Sent: Thursday, August 1, 2013 10:00:18 AM
Subject: Re: [Beaker-devel] Test framework for lshw
Excerpts from Nick Coghlan's message of 2013-08-01 09:57:14 +1000:
> On 07/31/2013 09:34 PM, Amit Saha wrote:
> > However, with this approach I think the difficulty is to run tests for
> > all archs (that we have in testdata)
> > on any arch. That is, I cannot run test the functionality of lshw with
> > arm test data on a x86_64
> > system and vice-versa. The reason being, lshw uses compiler macros
> > heavily during the compilation process
> > and hence the lshw executable has to be built on the arch on which it is
> > to be tested. [1]
> >
> > [1]
http://sourceforge.net/p/predef/wiki/Architectures/
> >
> > Thoughts and suggestions?
>
> Given the arch specific binaries, wouldn't we have the same architecture
> dependence problem with any other means of injecting test data for the
> "wrong" platform?
>
> This means I'm still inclined to run with the chroot based testing.
> Local testing during development would cover x86_64, and then we can
> spin up a job in beaker-devel to make sure it works across all supported
> architectures.
Beaker is probably overkill for this. The test suite should only run
tests for arch it is running on, then we should set up the tests to run
in %check during RPM build. That way all the tests will be run when Brew
builds for every arch.
The downside is that the only way to run all tests is to submit a Brew
scratch build (which is still lighter weight than a Beaker job though).
I think I like this idea. I will next attempt to add a Makefile target to run
the tests so that we can put something like 'make test' or 'make check' in
%check.
Best,
Amit.