Hey all--
So there's a rather spirited discussion perl packaging discussion going on over at https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=237883 about Test::Pod::Coverage module tests, and their importance.
The canonical approach is to insist on all tests being enabled that possibly can -- even to the point of requiring other modules be packaged to enable these tests. "Test as much as possible."
Test::Pod::Coverage tests don't actually test the functionality of the module. Further, they can't tell you if that documentation is any good, or even plain wrong. We also have a practice of disabling this test out of hand if it fails for whatever reason... Further, usually upstream takes the step of explicitly requiring this test be enabled -- e.g. via TEST_POD=1 or some other mechanism.
Documentation coverage is a good Kwalitee indication (as they say), not always an indication of good quality -- and certainly not a consistent, reliable indication of that.
My opinion is that we ought to not mandate the use of Pod coverage tests, simply because for our purposes it doesn't really matter what their result is. If they're present, we should conditionalize the tests (e.g. %_with_pod_tests magic or some such), but not insist on them by default.
Either way, we ought to document this in Packaging/Perl -- which I took a quick (and probably dirty) pass at reworking under PackagingDrafts/Perl.
-Chris
On 4/27/07, Chris Weyl cweyl@alumni.drew.edu wrote:
My opinion is that we ought to not mandate the use of Pod coverage tests, simply because for our purposes it doesn't really matter what their result is. If they're present, we should conditionalize the tests (e.g. %_with_pod_tests magic or some such), but not insist on them by default.
So, to follow up to myself here, given the lack of comments I'm inclined to approve 237883 without insisting on Test::Pod::Coverage due to T::P::C not testing the _functionality_ of the package. One last chance to scream :)
-Chris
On Fri, 2007-05-04 at 16:12 -0700, Chris Weyl wrote:
On 4/27/07, Chris Weyl cweyl@alumni.drew.edu wrote:
My opinion is that we ought to not mandate the use of Pod coverage tests, simply because for our purposes it doesn't really matter what their result is. If they're present, we should conditionalize the tests (e.g. %_with_pod_tests magic or some such), but not insist on them by default.
So, to follow up to myself here, given the lack of comments I'm inclined to approve 237883 without insisting on Test::Pod::Coverage due to T::P::C not testing the _functionality_ of the package. One last chance to scream :)
I'd rather include it. Simply because the Pod coverage tests don't test anything useful today doesn't mean they won't in the future, and by letting it slide now, we increase the chance that it will slide by unnoticed in the future.
But I'm not really torn up either way. I'd rather say "test everything, useful or not" to make sure we cover everything than trying to judge usefulness of tests on a test by test basis. Blanket policies help me sleep easier at night. :)
~spot
On Fri, 2007-05-04 at 16:12 -0700, Chris Weyl wrote:
On 4/27/07, Chris Weyl cweyl@alumni.drew.edu wrote:
My opinion is that we ought to not mandate the use of Pod coverage tests, simply because for our purposes it doesn't really matter what their result is. If they're present, we should conditionalize the tests (e.g. %_with_pod_tests magic or some such), but not insist on them by default.
So, to follow up to myself here, given the lack of comments I'm inclined to approve 237883 without insisting on Test::Pod::Coverage due to T::P::C not testing the _functionality_ of the package.
You don't want to know about the bugs and deficits your packages suffer from?
<biting sarcasm> But then let's be consequent and mandate disabling all testsuites, avoids a lot of fuzz and lets the distro appear "free of bugs". <biting sarcasm/>
One last chance to scream :)
YELL!
Ralf
"RC" == Ralf Corsepius rc040203@freenet.de writes:
RC> You don't want to know about the bugs and deficits your packages RC> suffer from?
Well, to play devil's advocate, if we're to consider lack of documentation coverage a bug and block inclusion of packages due to those bugs, then we shouldn't even have a kernel.
Of course we should run test suites, and we should of course block packages when those test suites fail but are expected to pass. But blocking due to lack of documentation coverage is pushing things a bit beyond the bounds of reason.
- J<
On Mon, 2007-05-07 at 18:04 -0500, Jason L Tibbitts III wrote:
"RC" == Ralf Corsepius rc040203@freenet.de writes:
RC> You don't want to know about the bugs and deficits your packages RC> suffer from?
Well, to play devil's advocate, if we're to consider lack of documentation coverage a bug and block inclusion of packages due to those bugs, then we shouldn't even have a kernel.
This is not about "lack of docs", but about packages failing their their own testsuites, because packages ship with broken testsuites.
A "reasonable" upstream being conscious about their weaknesses/deficits ("known bugs") would change their testsuite to "degrade gracefully" (e.g. to complain only).
Of course we should run test suites, and we should of course block packages when those test suites fail but are expected to pass. But blocking due to lack of documentation coverage is pushing things a bit beyond the bounds of reason.
Well, I guess you are aware that a perl-module without sufficiently complete docs (pods) is almost useless.
Ralf
perl-devel@lists.fedoraproject.org
-
Chris Weyl -
Jason L Tibbitts III -
Ralf Corsepius -
Tom Callaway