On Fri, 2023-08-04 at 15:40 +0200, Dan HorĂ¡k wrote:
Hi,
seems there is an issue with the test results presented in bodhi, please
see
https://bodhi.fedoraproject.org/updates/FEDORA-2023-94f22746e1
bodhi thinks fedora-ci.koji-build.rpmdeplint.functional failed (it's
"red"), but when I click on the test in bodhi testing farm say
"passed".
What can be wrong?
Well, Bodhi just gets the result from resultsdb. And the result there
is definitely "failed":
https://resultsdb.fedoraproject.org/results/41380875
Fedora CI files that result via ci-resultsdb-listener:
https://pagure.io/ci-resultsdb-listener
I think the message that triggered that result was this one:
https://apps.fedoraproject.org/datagrepper/v2/id?id=0bb07f00-a232-48e9-a6...
which is actually an 'error' message, not a 'failed' message. resultsdb
does - as of fairly recently - have an ERROR outcome, at least in the
Fedora deployment, but it did not have one for a long time; ci-
resultsdb-listener specifically uses the FAILED outcome for .error
results, which we could perhaps improve now ERROR is available.
The message doesn't tell us exactly what the error was, just
"Infrastructure Failure". But this is basically what happened - some
kind of system error occurred during/after the test run which somehow
caused CI to publish a message indicating the test errored out, even
though (as you saw) the logs indicate it passed. From there the details
of how we report CI results to resultsdb turned it into a "failed"
result.
CCing the ci@ list - someone from there should be able to look more
closely into what went on here and if anything can be improved (beyond
making ci-resultsdb-listener use the ERROR outcome, now we have it).
Practically speaking, we should be able to just re-run the tests and it
should come out good. I'll bonk that button now.
--
Adam Williamson (he/him/his)
Fedora QA
Fedora Chat: @adamwill:fedora.im | Mastodon: @adamw(a)fosstodon.org
https://www.happyassassin.net