On Fri, 2023-08-04 at 15:40 +0200, Dan Horák wrote:

Hi,

seems there is an issue with the test results presented in bodhi, please see https://bodhi.fedoraproject.org/updates/FEDORA-2023-94f22746e1

bodhi thinks fedora-ci.koji-build.rpmdeplint.functional failed (it's "red"), but when I click on the test in bodhi testing farm say "passed".

What can be wrong?

Well, Bodhi just gets the result from resultsdb. And the result there is definitely "failed":

https://resultsdb.fedoraproject.org/results/41380875

Fedora CI files that result via ci-resultsdb-listener: https://pagure.io/ci-resultsdb-listener

I think the message that triggered that result was this one: https://apps.fedoraproject.org/datagrepper/v2/id?id=0bb07f00-a232-48e9-a64f-...

which is actually an 'error' message, not a 'failed' message. resultsdb does - as of fairly recently - have an ERROR outcome, at least in the Fedora deployment, but it did not have one for a long time; ci- resultsdb-listener specifically uses the FAILED outcome for .error results, which we could perhaps improve now ERROR is available.

The message doesn't tell us exactly what the error was, just "Infrastructure Failure". But this is basically what happened - some kind of system error occurred during/after the test run which somehow caused CI to publish a message indicating the test errored out, even though (as you saw) the logs indicate it passed. From there the details of how we report CI results to resultsdb turned it into a "failed" result.

CCing the ci@ list - someone from there should be able to look more closely into what went on here and if anything can be improved (beyond making ci-resultsdb-listener use the ERROR outcome, now we have it).

Practically speaking, we should be able to just re-run the tests and it should come out good. I'll bonk that button now.