On Mon, Jan 7, 2019 at 3:28 PM Mark Reynolds <mreynolds(a)redhat.com> wrote:
On 1/6/19 11:05 PM, William Brown wrote:
> Hi,
>
> When developing I find that I only end up running the basic test + the test case for
the feature or fix that I’m developing. I think it would be good to have a few more to run
as “sanity” tests on our laptops before we do the full CI. What would be a good range of
the suites that we should run on builds during development?
Good question. I think the answer is really all of them :-) But
seriously this is something we should look into. So we "started"
creating acceptance tests in each test suite, but they are spread out
and not in a generalized location (essentially hard to discover). And
we only have "acceptance" tests in replication, plugins, and GER.
We are also trying to port everything to the suites, and not use
individual ticket tests anymore. This is a long road to getting this
all done, but we should consider expanding the basic suite to include
acceptance tests for all areas (maybe have a script that just points to,
and runs, the other suites' acceptance tests).
In downstream tests we have
different tiers. Tier0 runs after build is
done (basic tests like installation, package sanity, etc). Tier1 is
integration tests, Tier2 and Tier3 -- stress and long duration tests.
We should leverage py.test markers [1] and do the same in upstream
tests. Different test suites and individual tests can be marked (or
tagged), so they would be executed on request. Like run all the
replication tests (not only from replication test suite, but even from
others that use replicated topology).
Some test suites consume a lot of time to run, but they cover a
feature that is rarely used. They should me marked as tier3 for
example and execute only in full CI runs.
[1]
https://docs.pytest.org/en/latest/example/markers.html
Mark
>
> Thanks,
>
> --
> Sincerely,
>
> William
> _______________________________________________
> 389-devel mailing list -- 389-devel(a)lists.fedoraproject.org
> To unsubscribe send an email to 389-devel-leave(a)lists.fedoraproject.org
> Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
> List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
> List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproje...
_______________________________________________
389-devel mailing list -- 389-devel(a)lists.fedoraproject.org
To unsubscribe send an email to 389-devel-leave(a)lists.fedoraproject.org
Fedora Code of Conduct:
https://getfedora.org/code-of-conduct.html
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/389-devel@lists.fedoraproje...
--
Viktor