On Fri, Feb 21, 2014 at 9:50 AM, Daniel P. Berrange <berrange(a)redhat.com> wrote:
On Fri, Feb 21, 2014 at 09:38:56AM -0500, Josh Boyer wrote:
> > That being said, a lot of packages in Fedora are simply that: packaged
> > upstreams. Many (most?) package maintainers are not developers of that
> > package and as such are probably not equipped to add tests to their
> > systems.
> >
> > A better case here would be to find a way to identify those packages
> > whose upstreams have tests that are not being run in %check. That
> > probably *would* be considered a bug.
>
> Unless there's a decree from FESCo or FPC about requiring this, it's
> going to be up to the maintainer as to whether using %check to run
> testsuites is required. A lot of testsuites require external network
> access and won't work when run under koji. Also, it increases build
> time and can bloat BuildRequires. It is *not* a clear-cut bug.
>
> Personally, I don't think %check is a good idea at all.
I think the benefit depends on the level of patching the Fedora maintainer
is doing. If they are shipping just vanilla upstream tar.gz then they can
have a moderate level of confidence in the functionality of their package
without tests, since you can assume upstream ran their test before release.
Running the test suite would however confirm that the package has not
been broken due to a change in an new version of an external library/tool
it depends on, so is a pretty good idea to enable on balance.
If the maintainer is including any non-trivial patches that I think that
enabling %check should almost be mandatory to ensure they are not causing
regressions through their patches.
I don't disagree that testsuites should be run. I just disagree they
should be run in %check. It's a poor replacement for things that
should be run via AutoQA or some other test framework after the build.
josh