BTW if somebody is going to tell me that I should not look at Jenkins console, then if there is this quite nice report [1], with the error [2], first thing I'd like to know is how to reproduce this locally. But there is no such information at all. On might just guess if the annocheck was executed by rpminspect or not, if the RPMs were installed or not, etc. How to reproduce [3], that is even less clear to me.

Vít

[1] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pip...

[2] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pip...

[3] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pip...

Dne 10. 01. 22 v 10:05 Vít Ondruch napsal(a):

Dne 07. 01. 22 v 21:46 Miroslav Vadkerti napsal(a):

...

Hi,

On Fri, Jan 7, 2022 at 6:43 PM Vít Ondruch vondruch@redhat.com wrote:

Hi everybody and sorry for the rant, but I must vent somewhere.

As usual, sorry for the bad experience, I guess the problems boils down to missing proper documentation for the test exposed to the end-user :(

Eh, don't start me on documentation ;) The rpminspect-pipeline [5] contains non.

But I still think that I should not need any documentation. The logs should be comprehensive on itself and should contain every information.

Actually, I have to correct my statement. The problem with current logs is that they targets the CI developer instead of targeting CI users. I understand that the CI is complex and CI developers needs to know some information, but I think this approach should be reconsidered. The CI should be done for CI users.

...

To run it exactly as CI does you would need to run:

$ git clone https://github.com/fedora-ci/rpminspect-pipeline    $ cd rpminspect-pipeline    $ tmt run -e PREVIOUS_TAG=f36-updates -e TASK_ID='79796333' -e RPMINSPECT_PROFILE_NAME=rawhide -e REPOSITORY_URL=https://src.fedoraproject.org/rpms/ruby -e CONFIG_BRANCH=rawhide -e DEFAULT_RELEASE_STRING=fc36 -e GIT_COMMIT=5d3688479ab299cbd5657f9a503463a04d963d65 -e DEBUG=off

I am looking once again at the Jenkins console [4] and the only thing from your list I can see there is the first line. And that is the first and last useful information in the log. Why the remaining lines are not there?

But also, I'd like to see in the log how `tmt` executes the rpminspect. After all, I am not working with Koji build locally and what not.

Last but not least, seeing something as a `rpminspect-image`, this seems to be over-engineered. If I submitted change to that project, I'd expect that simple merge would propagate my changes to CI. But that is probably not true, because the new image would need to be built. Or if CI is using such image, then it should also build it.

BTW neither the rpminspect-image is mentioned in the log.

...

We should expose this information in all `tmt` based tests, sorry it is still not there :( it is this RH issue https://issues.redhat.com/browse/TFT-641 https://issues.redhat.com/browse/TFT-641

Now why is running rpminspect in CI more complex than just run`rpminspect-fedora OLD_BUILD NEW_BUILD`, I guess that is the question to @Michal Srb mailto:msrb@redhat.com or @Cantrell, David mailto:dcantrell@redhat.com ...I do not exactly know

Best regards, /M

PS: command to run for rpminspect for Fedora is `rpminspect-fedora` not plain `rpminspect`.

Why to remember this, when I can look at how CI does it, right?

BTW while this might put rpminspect into the spotlight, a bit earlier I tried to understand how rpmlint [0] used to be executed and the experience was not really better.

Vít

[0] https://bodhi.fedoraproject.org/updates/FEDORA-2020-0d62cd7900

...

So I am preparing updated Ruby package. I know that there is
rpminspect executed at some point and the results are available
in Bodhi [1]. Therefore, I'd like to execute similar test on my
machine prior pushing anything anywhere. So lets give rpminspect
a shot:

~~~

$ rpminspect
rpminspect: Please specify a configuration file using '-c' or
supply ./rpminspect.yaml

~~~

Hm, bummer. So how is this configured in Bodhi? Lets look at
Jenkins [2]. Hm, nice, but how is it executed? No clue.

So lets try the main task [3]. Ha, console [4], there must be the
"rpminspect" command, right? But no, there is non. There is just
11 pages of nonsense.

So looking once again, ok, kubernetes, there is used some
container. Lets check what the rpminspect-pipeline stands for,
opening its repository [5]. FMF, that sounds familiar, lets open
the rpminspect.fmf [6]. Hm, /usr/local/bin/rpminspect_runner.sh,
what is that? How can I get it? Ah, there is image repository and
the file [7], which as again more then 5 pages of gibberish.
There finally some `rpminspect` call on line 155 [8], but then I
could continue with localization of the config file.

This is really awful and I suffer every time I have to do
something like this. I am really frustrated. I really don't
understand, why the command can't be visible as far as at the
console [4].

Comparing e.g. with GH actions, I have opened cpython repository
for the first time in my life and on 5 clicks, I know the test
suite is executed via `make buildbottest TESTOPTS="-j4 -uall,-cpu"`.

Do I want to try to test rpminspect at this moment? No, No, No
and no again, because this is not CI, this is just torture. This
is so intimidating.

So, would it be so big problem to have the several rpminspect
calls which are certainly done somewhere at some point visible on
the Jenkins console [4]?

/rant


Vít


P.S. Once again sorry and please no offense.


[1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-62cd39359e

[2]
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/61952/testReport/(root)/tests/

[3]
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/61952/

[4]
https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pipeline/job/master/61952/console

[5] https://github.com/fedora-ci/rpminspect-pipeline

[6]
https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runner.sh

[7]
https://github.com/fedora-ci/rpminspect-pipeline/blob/master/rpminspect.fmf

[8]
https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runner.sh#L155

_______________________________________________
CI mailing list -- ci@lists.fedoraproject.org
To unsubscribe send an email to ci-leave@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines:
https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org
Do not reply to spam on the list, report it:
https://pagure.io/fedora-infrastructure

-- Miroslav Vadkerti :: Senior Principal QE :: Testing Farm / Linux QE IRC mvadkert #tft #tmt #osci :: Mobile +420 773 944 252 Remote Czech Republic :: Red Hat Czech s.r.o

---

CI mailing list --ci@lists.fedoraproject.org To unsubscribe send an email toci-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org Do not reply to spam on the list, report it:https://pagure.io/fedora-infrastructure

On Mon, Jan 10, 2022 at 10:50:49AM +0100, Vít Ondruch wrote:

Dne 10. 01. 22 v 10:30 Michal Srb napsal(a):

...

Hi there,

On Fri, Jan 7, 2022 at 6:43 PM Vít Ondruch vondruch@redhat.com wrote:

Hi everybody and sorry for the rant, but I must vent somewhere.

So I am preparing updated Ruby package. I know that there is rpminspect executed at some point and the results are available in Bodhi [1]. Therefore, I'd like to execute similar test on my machine prior pushing anything anywhere. So lets give rpminspect a shot:

$ rpminspect
rpminspect: Please specify a configuration file using '-c' or
supply ./rpminspect.yaml

Hm, bummer. So how is this configured in Bodhi? Lets look at Jenkins [2]. Hm, nice, but how is it executed? No clue.

So lets try the main task [3]. Ha, console [4], there must be the "rpminspect" command, right? But no, there is non. There is just 11 pages of nonsense.

So looking once again, ok, kubernetes, there is used some container. Lets check what the rpminspect-pipeline stands for, opening its repository [5]. FMF, that sounds familiar, lets open the rpminspect.fmf [6]. Hm, /usr/local/bin/rpminspect_runner.sh, what is that? How can I get it? Ah, there is image repository and the file [7], which as again more then 5 pages of gibberish. There finally some `rpminspect` call on line 155 [8], but then I could continue with localization of the config file.

This is really awful and I suffer every time I have to do something like this. I am really frustrated. I really don't understand, why the command can't be visible as far as at the console [4].

Comparing e.g. with GH actions, I have opened cpython repository for the first time in my life and on 5 clicks, I know the test suite is executed via `make buildbottest TESTOPTS="-j4 -uall,-cpu"`.

Yes, this is how CI usually works. You have some scripts that you want to execute in some environment and maybe, at the end, you want to retrieve/archive a few test-specific files/directories that you know were created during the run. Easy-peasy.

We had trouble with the "environment" part in the past, when OpenStack was not very reliable. But instead of fixing just that one problem (credit card would do the trick), we went further and introduced more opinionated layers. In hindsight, I think we went a little overboard.

Do I want to try to test rpminspect at this moment? No, No, No and no again, because this is not CI, this is just torture. This is so intimidating.

So, would it be so big problem to have the several rpminspect calls which are certainly done somewhere at some point visible on the Jenkins console [4]?

The problem is that Jenkins is actually not running anything -- it delegates everything. So maintainers will never find any useful information in Jenkins logs. Jenkins doesn't really know what rpminspect command will be executed several layers down. And it currently doesn't have access to the raw output of the test run, the only thing that is returned back through all the layers is the captured stdout/stderr from tmt-executed tests (so we cannot just dump the command to a file in a lower layer and then print it to the Jenkins console later -- that file wouldn't be available in Jenkins). The situation will be much better when the whole workdir is available to the upper layers (Miro shared a JIRA link in the other email in this thread).

However, you're not completely out of luck here. rpminspect prints the command that was used to produce the test results as part of its "diagnostics" inspection [1]. And there you can see:

Details: /usr/bin/rpminspect -c /usr/share/rpminspect/fedora.yaml --format=json --release=fc36 --profile=rawhide ruby-3.0.3-154.fc36 ruby-3.0.3-155.fc36

Oh, wow. Please, can this be extracted somewhere visible? Nice place would be right bellow the "Test Result : tests" in the main landing page. Actually, I would not mind to have the whole "diagnostics" section visible on the first sight.

Also, having no real experiences with rpminspect, I might just guess that diagnostics might be the first section reported by rpminspect, so I'd suggest against alphabetical sorting of the results.

When you run rpminspect locally at the command line, 'diagnostics' is the first result displayed for the reason you indicate.

I added 'diagnostics' to the rpminspect output to help me understand how CI ran rpminspect because it was hard to debug bug reports.

Vít

...

That is the command that actually runs in CI.

Thanks, Michal

/rant

Vít

P.S. Once again sorry and please no offense.

[1] https://bodhi.fedoraproject.org/updates/FEDORA-2021-62cd39359e

[2] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pip...

[3] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pip...

[4] https://osci-jenkins-1.ci.fedoraproject.org/job/fedora-ci/job/rpminspect-pip...

[5] https://github.com/fedora-ci/rpminspect-pipeline

[6] https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runner....

[7] https://github.com/fedora-ci/rpminspect-pipeline/blob/master/rpminspect.fmf

[8] https://github.com/fedora-ci/rpminspect-image/blob/master/rpminspect_runner....

---

CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure

---

CI mailing list --ci@lists.fedoraproject.org To unsubscribe send an email toci-leave@lists.fedoraproject.org Fedora Code of Conduct:https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines:https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives:https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org Do not reply to spam on the list, report it:https://pagure.io/fedora-infrastructure

---

CI mailing list -- ci@lists.fedoraproject.org To unsubscribe send an email to ci-leave@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/ci@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure