CI November 2019

ci@lists.fedoraproject.org

10 participants
5 discussions

ELF checks in rpminspect + using annocheck
by David Cantrell 17 Dec '19

17 Dec '19

In today's Fedora CI meeting, we discussed an annocheck gate. rpminspect has annocheck functionality on its roadmap. bookwar asked that I post a message explaining what rpminspect does for ELF checks and what is planned. So here we go! What is rpminspect? ------------------- https://github.com/rpminspect/rpminspect The important things to know for the context of this email are that rpminspect allows us to run tests against Koji build output as well as compare two Koji builds and report changes. rpminspect itself is a command line tool. What does rpminspect do with ELF files? --------------------------------------- A number of tests are performed on ELF files. The basic design here is inherited from rpminspect's ancestor--rpmdiff (the internal Red Hat service, not the other rpmdiff). These tests are: * Check for execstack in ELF shared objects and executables * Check for TEXTREL sections in ELF shared objects, which means the build lacked -fPIC * Check that '-z relro -z now' was used for linking * Check that -D_FORTIFY_SOURCE was used when compiling * Check for use of any forbidden IPv6 networking functions (list in rpminspect.conf) * In ELF .a libraries, check to see that the object members were built with -fPIC In cases where rpminspect is comparing two builds, the above tests extend a bit and the messages report whether or not a desired feature was lost in the after build or an undesired feature was gained. What is planned for ELF in rpminspect? -------------------------------------- I am still reviewing rpminspect's ancestor for test ideas that make sense to port forward to the grand world of CI. Right now that involves looking at the following things that rpmdiff still does: * Runs elflint when comparing two builds and reports a regression if the output differs between the before and after builds. * Runs annocheck when comparing two builds and filters its results up for reporting. * Reports executables that gain or lose PIE. * In shared libraries, gathers symbols in before and after builds and reports what was gained or lost for ABI verification. Of these, the annocheck one is the most interesting to me and most relevant I feel. It's also easy to add to the current battery of ELF tests in rpminspect. It makes sense to me in rpminspect because we gain the ability to compare results between builds in addition to validating single builds against our policy. None of these planned ELF tests are particularly difficult. I already did the difficult ones. I have bumped up these tests in my own priority list for rpminspect so we can get that functionality out there faster. Questions or Thoughts? ---------------------- Have other ideas or questions? Change in prioritization? Want to help with code or the integration test suite for rpminspect? Let me know. The integration test suite is written in Python and I could really use contributions there to work through various permutations of build types and errors. The more people contributing there, the better. This is one reason I have not introduced new inspections recently because I have been working on building out the integration test suite for all existing inspections. Thanks, -- David Cantrell <dcantrell(a)redhat.com> Red Hat, Inc. | Boston, MA | EST5EDT

2 2

Distgit CI/CD workflow with Zuul
by Fabien Boucher 21 Nov '19

21 Nov '19

Hello, I would like to introduce a CI/CD workflow for Fedora distgits around Pull Requests my team and I have started to develop. We chose to use Zuul [1] (the CI engine) to implement this workflow because we have strong experience with it and some of its features could help a lot in the packaging context. To list some of them, Zuul is cross-repository aware, meaning that dependencies between Pull-Requests can be defined (the job workspace is populated with the dependent changes). Zuul provides a way to share artifacts between jobs meaning a parent job can do a scratch build on Koji and child jobs can use the built artifacts (rpms) to run various validations. Also Zuul comes with a component called Nodepool that helps handle VMs or containers to manage F30/31/Rawhide nodes for jobs. A more exhaustive list of features is available here [2]. For a couple of months, we have been experimenting with packaging and validation jobs for Fedora's distgits. The idea is to build a flexible packaging workflow for packagers willing to use Pull Requests on Pagure. The Pull Request system offers a place to wire Zuul jobs with the PR life cycle. Indeed, jobs are triggered based on the Pagure Pull Request status (PR opened/changed/merged). At the moment, the most complete workflow we have implemented is as follows: - When a PR is opened, Zuul runs a Koji scratch build job, then in parallel, it runs a linter, rpminspect, and test (embedded functional tests/STI) jobs. - When a PR is approved, (when the metadata tag "gateit" is set by one of the distgit admins) and the CI status is green, Zuul merges the PR. - When the PR is closed and merged, Zuul runs the regular Koji build. Another great feature of Zuul is how changes to CI configuration can be done. In fact, anyone can propose changes via Pull Request to a Zuul job, a project pipeline template, ... and see that change run without affecting the rest of the CI. This makes the CI more robust and more user friendly as everyone can be involved in the CI configuration. To give a bit of insight about what my team and I are doing: we maintain https://softwarefactory-project.io and https://review.rdoproject.org that both use Zuul to provide the CI for more than 1800 repositories (~95% of distgits). Our instance of Zuul runs (among others) the Pagure driver [3] for pagure.io and src.fedoraproject.org. For the moment, just three distgits are configured to let Zuul manage the packaging workflow via PR, but we are looking for early adopters to experiment and help improve the jobs and workflows. The process to attach a distgit on Zuul is described here [4]. Having Zuul manage the PR workflow is not incompatible/conflicting with a regular direct push workflow. We would be really happy to help anyone willing to be involved :) Fabien [1]: https://zuul-ci.org/ [2]: https://fedoraproject.org/wiki/Zuul-based-ci#What_is_Zuul.2FNodepool [3]: https://zuul-ci.org/docs/zuul/admin/drivers/pagure.html [4]: https://fedoraproject.org/wiki/Zuul-based-ci#How_to_Zuul_attach_a_Pagure_re…

7 18

Fwd: Jenkins upgrade to change from fedmsg to fedora-messaging
by Jim Bair 14 Nov '19

14 Nov '19

I realized reply-all didn't include ci after I sent it. Sorry! -Jim ---------- Forwarded message --------- From: Jim Bair <jbair(a)redhat.com> Date: Thu, Nov 14, 2019 at 3:43 PM Subject: Re: Jenkins upgrade to change from fedmsg to fedora-messaging To: Development discussions related to Fedora <devel(a)lists.fedoraproject.org > Thank you for the clarification, Pierre! I meant to do that yesterday, but I got sidetracked. =) The upgrade went well this morning and all jobs appear to be running smoothly. We will make the change to fedora-messaging sometime next week, most likely on either Monday or Tuesday. Once we've setup a time for it, I'll reply back here with the chosen time. Thanks! -Jim On Thu, Nov 14, 2019 at 2:28 AM Pierre-Yves Chibon <pingou(a)pingoured.fr> wrote: > On Wed, Nov 13, 2019 at 05:34:40PM +0000, Sérgio Basto wrote: > > BTW , Jenkins was retired from rawhide [1]. > > I tried built Jenkins on rawhide but build failed with a lot of > missing > > packages that are BuildRequires. > > [1] > https://src.fedoraproject.org/rpms/jenkins/c/9e8c8fe1695dbf811e101ef829c51f… > > This email was about the jenkins instance running at: > https://jenkins-continuous-infra.apps.ci.centos.org/ which is used for > running > the tests gating package updates. > I believe it's a jenkins instance deployed in openshift and I'm fairly > sure it > does not rely on RPM packages. > > So your email and Jim's are both be correct, they just speak about > different > things. > > Pierre > > > On Wed, 2019-11-13 at 09:17 -0600, Jim Bair wrote: > > > > Hello Fedora CI and Devel Teams! > > The CI team will be performing a Jenkins upgrade and jms-messaging > > plugin upgrade this Thursday to allow us to switch from fedmsg to > > fedora-messaging. While performing this upgrade, dist-git tests > will be > > temporarily unavailable. When things are back to normal, we will > > reply-all here. > > Much like our last maintenance window, the intent is to upgrade the > > software but not actually change from fedmsg to fedora-messaging > just > > yet. If you have any questions or concerns, by all means please > reach > > out. > > Thank you! > > -Jim Bair > _______________________________________________ > devel mailing list -- devel(a)lists.fedoraproject.org > To unsubscribe send an email to devel-leave(a)lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org >

1 0

Jenkins upgrade to change from fedmsg to fedora-messaging
by Jim Bair 13 Nov '19

13 Nov '19

Hello Fedora CI and Devel Teams! The CI team will be performing a Jenkins upgrade and jms-messaging plugin upgrade this Thursday to allow us to switch from fedmsg to fedora-messaging. While performing this upgrade, dist-git tests will be temporarily unavailable. When things are back to normal, we will reply-all here. Much like our last maintenance window, the intent is to upgrade the software but not actually change from fedmsg to fedora-messaging just yet. If you have any questions or concerns, by all means please reach out. Thank you! -Jim Bair

1 0

Jenkins plugin upgrade to change from fedmsg to fedora-messaging
by Jim Bair 07 Nov '19

07 Nov '19

Hello Fedora CI and Devel Teams! The CI team will be performing a Jenkins plugin upgrade this Thursday to allow us to switch from fedmsg to fedora-messaging. While performing this upgrade, dist-git tests will be temporarily unavailable. When the upgrade is complete and things appear good, we will reply-all here. Please note that this is simply an upgrade to the plugin itself; while it allows the use of the fedora-messaging bus, we will continue to use fedmsg after the upgrade to validate plugin stability. If all goes well, our plan is to switch to fedora-messaging next week (tentatively on Wednesday the 13th). If you have any questions or concerns, by all means please reach out to me. Thank you! -Jim Bair

1 1

2024

2023

2022

2021

2020

2019

2018

2017

CI November 2019