Hello Python packagers,
since there is no upstream supported universal test invocation for Python (`python setup.py test` is deprecated and the de-facto-standard `tox` doesn't always do what we want in RPM's %check and/or is not always used by upstreams) and a lot of upstreams use pytest, I'd like to propose a %pytest macros, for standardizing the way pytest is run.
Usage:
%check %pytest
Or with options passed to pyets:
%check %pytest -v -m "not network" tests/
(Here ends the tl;dr version.)
Why not just use `pytest` or `%{python3} -m pytest` instead of `%pytest`?
We want to *test the code we ship*. In the general case this involves:
1. Prepending sys.path with %{buildroot}%{python3_sitelib} (and/or sitearch) 2. Prepending $PATH with %{buildroot}%{_bindir} 3. Ensuring $PWD is not in sys.path
So the idea is (untested):
%pytest_cmd /usr/bin/pytest
%pytest %{expand: PYTHONPATH="${PYTHONPATH:-%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}}" \ PATH="%{buildroot}%{_bindir}:$PATH" \ %pytest_cmd }
Where PYTHONPATH solves (1), PATH solves (2) and using `/usr/bin/pytest` over `%{python3} -m pytest` kinda solves (3).
I say "kinda" because using `python -m pytest` explicitly adds $PWD to sys.path, but using `pytest` doesn't always prevent it.
https://docs.pytest.org/en/latest/usage.html#calling-pytest-through-python-m...
I've been trying to completely solve (3) in my head in a general way, but it always involves `cd`ing to a temporary directory before running the tests, but plenty of test suites cannot handle that gracefully.
Unfortunately Python interpreter doesn't have a flag that says "don't add current directory to sys.path, but respect $PYTHONPATH". In the long term, we might propose to add it.
Where to have the macro defined? (Skip this part if not interested.)
I was thinking:
a) pytest package b) pytest subpackage (required if rpm-build) c) python3-rpm-macros d) python-rpm-macros e) pyproject-rpm-macros
And here is the pros and cons:
a+) self contained, can change with upstream if needed a+) always present with pytest a-) pytest must co-own macro dir or depend on rpm-build
b+) same as a+, but not a- b-) additional layer of complexity
c+) easy c+) the macro doesn't need to require pytest, can be used with arbitrary command c+) can be generalized to non-pytest later if desired c-) when pytest CLI changes significantly, we need to change it in different package (but we are not using pytets CLI now at all, just Python/Shell environment variables) c~) technically not always present with pytest, but always present with python3-devel
d+-) same as (c) d-) the macro uses python3 specific things
e+) the new fancy macros are there e-) the other macros from there (actually pyproject related) are not required for this e-) not always present with pytest nor python3-devel
Hence, I'd go with (c) python3-rpm-macros.
On Mon, 27 Apr 2020, Miro Hrončok wrote:
Hello Python packagers,
since there is no upstream supported universal test invocation for Python (`python setup.py test` is deprecated and the de-facto-standard `tox` doesn't always do what we want in RPM's %check and/or is not always used by upstreams) and a lot of upstreams use pytest, I'd like to propose a %pytest macros, for standardizing the way pytest is run.
Usage:
%check %pytest
Or with options passed to pyets:
%check %pytest -v -m "not network" tests/
(Here ends the tl;dr version.)
Why not just use `pytest` or `%{python3} -m pytest` instead of `%pytest`?
We want to *test the code we ship*. In the general case this involves:
- Prepending sys.path with %{buildroot}%{python3_sitelib} (and/or sitearch)
- Prepending $PATH with %{buildroot}%{_bindir}
- Ensuring $PWD is not in sys.path
So the idea is (untested):
%pytest_cmd /usr/bin/pytest
%pytest %{expand: PYTHONPATH="${PYTHONPATH:-%{buildroot}%{python3_sitearch}:%{buildroot}%{python3_sitelib}}" \ PATH="%{buildroot}%{_bindir}:$PATH" \ %pytest_cmd }
Where PYTHONPATH solves (1), PATH solves (2) and using `/usr/bin/pytest` over `%{python3} -m pytest` kinda solves (3).
I say "kinda" because using `python -m pytest` explicitly adds $PWD to sys.path, but using `pytest` doesn't always prevent it.
https://docs.pytest.org/en/latest/usage.html#calling-pytest-through-python-m...
I've been trying to completely solve (3) in my head in a general way, but it always involves `cd`ing to a temporary directory before running the tests, but plenty of test suites cannot handle that gracefully.
Unfortunately Python interpreter doesn't have a flag that says "don't add current directory to sys.path, but respect $PYTHONPATH". In the long term, we might propose to add it.
I like the idea!
I have run into the issue of not wanting $PWD in sys.path before as well, so it would be great to solve that.
The only other thing that I might suggest is to set PYTHONDONTWRITEBYTECODE=1 so we don't get pytest bytecode written out.
Scott
On 27. 04. 20 15:47, Scott Talbert wrote:
On Mon, 27 Apr 2020, Miro Hrončok wrote:
Hello Python packagers,
since there is no upstream supported universal test invocation for Python (`python setup.py test` is deprecated and the de-facto-standard `tox` doesn't always do what we want in RPM's %check and/or is not always used by upstreams) and a lot of upstreams use pytest, I'd like to propose a %pytest macros, for standardizing the way pytest is run.
Usage:
%check %pytest
Or with options passed to pyets:
%check %pytest -v -m "not network" tests/
I like the idea!
I have run into the issue of not wanting $PWD in sys.path before as well, so it would be great to solve that.
Will pu(n)t it to my far future TODO.
The only other thing that I might suggest is to set PYTHONDONTWRITEBYTECODE=1 so we don't get pytest bytecode written out.
Good point, will do. Thanks.
On 27. 04. 20 11:34, Miro Hrončok wrote:
Hello Python packagers,
since there is no upstream supported universal test invocation for Python (`python setup.py test` is deprecated and the de-facto-standard `tox` doesn't always do what we want in RPM's %check and/or is not always used by upstreams) and a lot of upstreams use pytest, I'd like to propose a %pytest macros, for standardizing the way pytest is run.
Usage:
%check %pytest
Or with options passed to pyets:
%check %pytest -v -m "not network" tests/
(Here ends the tl;dr version.)
Why not just use `pytest` or `%{python3} -m pytest` instead of `%pytest`?
We want to *test the code we ship*. In the general case this involves:
1. Prepending sys.path with %{buildroot}%{python3_sitelib} (and/or sitearch) 2. Prepending $PATH with %{buildroot}%{_bindir} 3. Ensuring $PWD is not in sys.path
So the idea is...
https://src.fedoraproject.org/rpms/python-rpm-macros/pull-request/55
python-devel@lists.fedoraproject.org
-
Miro Hrončok -
Scott Talbert