CI June 2023

ci@lists.fedoraproject.org

3 participants
3 discussions

[release] Testing Farm 2023-05.1 with tmt-1.23
by Miroslav Vadkerti 23 Jun '23

23 Jun '23

Hi, Testing Farm release 2023-05.1 <https://docs.testing-farm.io/general/0.1/releases/2023-05.1.html> is deployed 🎉 (TFT-1871 <https://issues.redhat.com/browse/TFT-1871>). *🎮 Upgrades* - tmt <https://tmt.readthedocs.io/en/stable/> updated from 1.22 to 1.24 <https://github.com/teemtee/tmt/releases/tag/1.24.0>, see also release notes for 1.23 <https://github.com/teemtee/tmt/releases/tag/1.23.0> - Artemis upgraded to v0.0.57 <https://gitlab.com/testing-farm/artemis/-/tags/v0.0.57>, improving hardware requirements <https://docs.testing-farm.io/general/0.1/test-request.html#hardware> *🎯 Highlights* - Testing Farm CLI <https://docs.testing-farm.io/general/0.1/cli.html> reserve command available, for easy provisioning of a single machine from Testing Farm. (TFT-2073 <https://issues.redhat.com/browse/TFT-2073>) (DevConf mini-demo <https://youtu.be/nS0Z1OilOas?t=18474>) - Non-RHEL teams are onboarding to Testing Farm, see for example strizmi-kafk2a-operator <https://github.com/strimzi/strimzi-kafka-operator/blob/main/systemtest/tmt/…>. (fedoramagazine blog post <https://fedoramagazine.org/how-to-use-testing-farm-outside-of-rhel/>) - Support for customizing global timeout for a request. Default is 12 hours. (TFT-1763 <https://issues.redhat.com/browse/TFT-1763>) - Support for private GIT repositories, the username and password can be passed directly in the git url and are properly treated as secrets. ( TFT-1656 <https://issues.redhat.com/browse/TFT-1656>) - Support for tests filtering via the API request. (TFT-1993 <https://issues.redhat.com/browse/TFT-1993>) (docs <https://docs.testing-farm.io/general/0.1/test-request.html#tmt>) - Support for installing only certain packages from a repository. ( TFT-2070 <https://issues.redhat.com/browse/TFT-2070>) - Support for merge request refs for GitLab and GitHub. (TFT-1578 <https://issues.redhat.com/browse/TFT-1578>) - Metrics for average queue time available for Red Hat and Public Ranch available. (TFT-1764 <https://issues.redhat.com/browse/TFT-1764>) (link <http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?orgId=1&refresh=1m> ) - Kickstart support for Beaker provisioner, on par with kickstart support in the Beaker XML. (TFT-1395 <https://issues.redhat.com/browse/TFT-1395>) (thanks to Martin Litwora / sst_upgrades) - Support for different location of tmt metadata root in the git repositories. (TFT-1706 <https://issues.redhat.com/browse/TFT-1706>) - Artemis can now detect spot instances pre-maturely terminated. ( TFT-1888 <https://issues.redhat.com/browse/TFT-1888>) - TESTING_FARM_REQUEST_ID environment variable exposed to tests. ( TFT-2044 <https://issues.redhat.com/browse/TFT-2044>) (Thanks to Lukas Kotek / KVM QE) - Support for tuning Beaker watchdog timeout in Artemis. (TFT-1949 <https://issues.redhat.com/browse/TFT-1949>) (Thanks to Martin Litwora / sst_upgrades) - Testing Farm CLI run command available, to easily run small shell snippets in Testing Farm. (TFT-2019 <https://issues.redhat.com/browse/TFT-2019>) *🐞 Bugfixes* - Excluding of packages for repository and copr build artifacts properly works now. (TFT-977 <https://issues.redhat.com/browse/TFT-977>) - Compose and arch are not part of the plan name anymore in the JUnit XML. Oculus Improved to show compose and arch below the plan name. ( TFT-1810 <https://issues.redhat.com/browse/TFT-1810>) - Multiple fixes for dnf5 which landed in Fedora Rawhide. Zuul PR testing disabled due to dnf5/issues/549 <https://github.com/rpm-software-management/dnf5/issues/549>. - Non-printable characters in results-junit.xml are now properly handled (TFT-2039 <https://issues.redhat.com/browse/TFT-2039>). - Plans with non-standard characters supported. (TFT-1881 <https://issues.redhat.com/browse/TFT-1881>) - Test machine's hostname, FQDN and reverse DNS records are now properly resolvable. (TFT-1987 <https://issues.redhat.com/browse/TFT-1987>) - Improved stability of aarch64 in AWS by using non-spot instances. ( TFT-2080 <https://issues.redhat.com/browse/TFT-2080>) *🔍 Statistics* - Error rate ~ 3.69% in May <http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=12&orgId=…> - ~ 56k testing requests in May <http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=8&orgId=1…> - Average queued time ~24 second in Public <http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=26&orgId=…>, ~5 minutes in Red Hat <http://metrics.osci.redhat.com/d/NnHWU1dnz/testing-farm?viewPanel=27&orgId=…> in th last month <https://issues.redhat.com/browse/TFT-1761> <https://issues.redhat.com/browse/TFT-1761>On behalf of the Testing Farm Team, Miroslav Vadkerti -- Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252 Remote Czech Republic :: Red Hat Czech s.r.o

1 1

Fedora-Rawhide testing blocked in Testing Farm
by Miroslav Vadkerti 22 Jun '23

22 Jun '23

Hi, Fedora Rawhide testing blocked on dnf5 update, which broke Testing Farm. We are trying to hotpatch the problems to bring back testing against Rawhide. ETA 2 hours. For updates watch: https://status.testing-farm.io/issues/2023-06-21-rawhide-outage/ Best regards, /M -- Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252 Remote Czech Republic :: Red Hat Czech s.r.o

1 2

Please help me: STI tests with standard-test-source succeed without actually running
by Miro Hrončok 06 Jun '23

06 Jun '23

Hello folks. In https://gitlab.com/redhat/centos-stream/rpms/pyproject-rpm-macros/-/merge_r… I discovered a problem with the STI tests. Before adding the standard-test-source import_role, the STI test would run for approximately *20 minutes*, succeed and the testing farm artifacts page would list all the tests (green). After adding the standard-test-source thing, the STI tests run for approximately *5 minutes*, succeed and the testing farm artifacts page lists only one item with a cryptic name (such as git-refs-merge-requests-18-headaan6h3u3/tests/tests.yml). I believe the STI tests have actually not run at all and the job reported success. I've applied my change in isolation at https://src.fedoraproject.org/rpms/pyproject-rpm-macros/pull-request/400 -- it behaves the same (both on zuul's rpm-sti-test and Fedora CI - dist-git test. The added construct is copied from the python-tox package, where it worked 2 months ago, see e.g. the zuul results of https://src.fedoraproject.org/rpms/python-tox/pull-request/124 However now it misbehaves there as well, see the results of https://src.fedoraproject.org/rpms/python-tox/pull-request/126 Specifically, compare: https://artifacts.dev.testing-farm.io/b02f4e61-5517-4796-91d4-feaad434ce41/ ok https://artifacts.dev.testing-farm.io/30af69cb-eff7-4f6d-804a-d5533685f3e9/ bad What is going on here? I've been debugging this for ~2 hours and I am totally lost. The nice green SUCCESS thing now scares me because it it is meaningless :( -- Miro Hrončok -- Phone: +420777974800 IRC: mhroncok

3 5

2025

2024

2023

2022

2021

2020

2019

2018

2017

CI June 2023