Hello folks.
In https://gitlab.com/redhat/centos-stream/rpms/pyproject-rpm-macros/-/merge_re... I discovered a problem with the STI tests.
Before adding the standard-test-source import_role, the STI test would run for approximately *20 minutes*, succeed and the testing farm artifacts page would list all the tests (green).
After adding the standard-test-source thing, the STI tests run for approximately *5 minutes*, succeed and the testing farm artifacts page lists only one item with a cryptic name (such as git-refs-merge-requests-18-headaan6h3u3/tests/tests.yml). I believe the STI tests have actually not run at all and the job reported success.
I've applied my change in isolation at https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/400 -- it behaves the same (both on zuul's rpm-sti-test and Fedora CI - dist-git test.
The added construct is copied from the python-tox package, where it worked 2 months ago, see e.g. the zuul results of https://src.fedoraproject.org/rpms/python-tox/pull-request/124
However now it misbehaves there as well, see the results of https://src.fedoraproject.org/rpms/python-tox/pull-request/126
Specifically, compare:
https://artifacts.dev.testing-farm.io/b02f4e61-5517-4796-91d4-feaad434ce41/ ok https://artifacts.dev.testing-farm.io/30af69cb-eff7-4f6d-804a-d5533685f3e9/ bad
What is going on here? I've been debugging this for ~2 hours and I am totally lost. The nice green SUCCESS thing now scares me because it it is meaningless :(
Ack, we will try to add this a blocker for the next release.
It is indeed weird ...
/M
On Thu, Jun 1, 2023 at 1:07 PM Miro Hrončok mhroncok@redhat.com wrote:
Hello folks.
In
https://gitlab.com/redhat/centos-stream/rpms/pyproject-rpm-macros/-/merge_re... I discovered a problem with the STI tests.
Before adding the standard-test-source import_role, the STI test would run for approximately *20 minutes*, succeed and the testing farm artifacts page would list all the tests (green).
After adding the standard-test-source thing, the STI tests run for approximately *5 minutes*, succeed and the testing farm artifacts page lists only one item with a cryptic name (such as git-refs-merge-requests-18-headaan6h3u3/tests/tests.yml). I believe the STI tests have actually not run at all and the job reported success.
I've applied my change in isolation at https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/400 -- it behaves the same (both on zuul's rpm-sti-test and Fedora CI - dist-git test.
The added construct is copied from the python-tox package, where it worked 2 months ago, see e.g. the zuul results of https://src.fedoraproject.org/rpms/python-tox/pull-request/124
However now it misbehaves there as well, see the results of https://src.fedoraproject.org/rpms/python-tox/pull-request/126
Specifically, compare:
https://artifacts.dev.testing-farm.io/b02f4e61-5517-4796-91d4-feaad434ce41/ ok https://artifacts.dev.testing-farm.io/30af69cb-eff7-4f6d-804a-d5533685f3e9/ bad
What is going on here? I've been debugging this for ~2 hours and I am totally lost. The nice green SUCCESS thing now scares me because it it is meaningless :(
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
Hi,
I did not forget, we have found the issue. Ansible-core 2.15 breaks STI :(, moving back to 2.14 resolves the problem.
We are deploying the fix, will let you know once it is there.
Best regards, /M
On Thu, Jun 1, 2023 at 1:26 PM Miroslav Vadkerti mvadkert@redhat.com wrote:
Ack, we will try to add this a blocker for the next release.
It is indeed weird ...
/M
On Thu, Jun 1, 2023 at 1:07 PM Miro Hrončok mhroncok@redhat.com wrote:
Hello folks.
In
https://gitlab.com/redhat/centos-stream/rpms/pyproject-rpm-macros/-/merge_re... I discovered a problem with the STI tests.
Before adding the standard-test-source import_role, the STI test would run for approximately *20 minutes*, succeed and the testing farm artifacts page would list all the tests (green).
After adding the standard-test-source thing, the STI tests run for approximately *5 minutes*, succeed and the testing farm artifacts page lists only one item with a cryptic name (such as git-refs-merge-requests-18-headaan6h3u3/tests/tests.yml). I believe the STI tests have actually not run at all and the job reported success.
I've applied my change in isolation at https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/400 -- it behaves the same (both on zuul's rpm-sti-test and Fedora CI - dist-git test.
The added construct is copied from the python-tox package, where it worked 2 months ago, see e.g. the zuul results of https://src.fedoraproject.org/rpms/python-tox/pull-request/124
However now it misbehaves there as well, see the results of https://src.fedoraproject.org/rpms/python-tox/pull-request/126
Specifically, compare:
https://artifacts.dev.testing-farm.io/b02f4e61-5517-4796-91d4-feaad434ce41/ ok
https://artifacts.dev.testing-farm.io/30af69cb-eff7-4f6d-804a-d5533685f3e9/ bad
What is going on here? I've been debugging this for ~2 hours and I am totally lost. The nice green SUCCESS thing now scares me because it it is meaningless :(
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
-- Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252 Remote Czech Republic :: Red Hat Czech s.r.o
This should be now resolved, we hot-patched it in current production.
HTH, /M
On Tue, Jun 6, 2023 at 5:27 PM Miroslav Vadkerti mvadkert@redhat.com wrote:
Hi,
I did not forget, we have found the issue. Ansible-core 2.15 breaks STI :(, moving back to 2.14 resolves the problem.
We are deploying the fix, will let you know once it is there.
Best regards, /M
On Thu, Jun 1, 2023 at 1:26 PM Miroslav Vadkerti mvadkert@redhat.com wrote:
Ack, we will try to add this a blocker for the next release.
It is indeed weird ...
/M
On Thu, Jun 1, 2023 at 1:07 PM Miro Hrončok mhroncok@redhat.com wrote:
Hello folks.
In
https://gitlab.com/redhat/centos-stream/rpms/pyproject-rpm-macros/-/merge_re... I discovered a problem with the STI tests.
Before adding the standard-test-source import_role, the STI test would run for approximately *20 minutes*, succeed and the testing farm artifacts page would list all the tests (green).
After adding the standard-test-source thing, the STI tests run for approximately *5 minutes*, succeed and the testing farm artifacts page lists only one item with a cryptic name (such as git-refs-merge-requests-18-headaan6h3u3/tests/tests.yml). I believe the STI tests have actually not run at all and the job reported success.
I've applied my change in isolation at https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/400 -- it behaves the same (both on zuul's rpm-sti-test and Fedora CI - dist-git test.
The added construct is copied from the python-tox package, where it worked 2 months ago, see e.g. the zuul results of https://src.fedoraproject.org/rpms/python-tox/pull-request/124
However now it misbehaves there as well, see the results of https://src.fedoraproject.org/rpms/python-tox/pull-request/126
Specifically, compare:
https://artifacts.dev.testing-farm.io/b02f4e61-5517-4796-91d4-feaad434ce41/ ok
https://artifacts.dev.testing-farm.io/30af69cb-eff7-4f6d-804a-d5533685f3e9/ bad
What is going on here? I've been debugging this for ~2 hours and I am totally lost. The nice green SUCCESS thing now scares me because it it is meaningless :(
-- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok
-- Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252 Remote Czech Republic :: Red Hat Czech s.r.o
-- Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252 Remote Czech Republic :: Red Hat Czech s.r.o
On 06. 06. 23 21:35, Miroslav Vadkerti wrote:
This should be now resolved, we hot-patched it in current production.
Fix verified in in Fedora, c9s test in progress.
Thanks.
What can be done to avoid this in the future? Tests that aren't executed yet report success are quite dangerous.
On Tue Jun 6, 2023 at 17:27 +0200, Miroslav Vadkerti wrote:
Hi,
I did not forget, we have found the issue. Ansible-core 2.15 breaks STI :(, moving back to 2.14 resolves the problem.
What ansible-core change broke STI? Can you report the issue upstream?