koji-devel February 2020

koji-devel@lists.fedorahosted.org

7 participants
5 discussions

koji-tools usage
by Tomas Kopecek 27 Feb '20

27 Feb '20

Hi, question for all: how are you using koji-usage (if you even know about their existence)? This is a bunch of random script with various level of maturity living in pagure [1]. They are nor packaged, nor distributed anywhere. I've gone through old PRs there and seen one from Pavol Babincak proposing setup.py. Maybe it makes sense for someone to have it installed in virtualenv, maybe it also makes sense to provide some basic spec file (that was my PR proposing splitting these tools to cli plugins and rest). Not sure about ad hoc PyPi distribution. We don't plan to give it much better shape, but for some more mature scripts it would make sense to promote them to basic koji (blinking in koji-ssl-admin direction). I'll add some paragraph about this possibility to README.md there. [1] https://pagure.io/koji-tools -- Tomas Kopecek <tkopecek(a)redhat.com> Release Engineering Development, RedHat

5 9

Teaching Koji to build Spack package caches
by Foraker, Jim 27 Feb '20

27 Feb '20

I've been working on a project that is investigating using Spack (https://spack.io/) to deploy a common development environment across HPC centers. Like many other ports collections, Spack began as a build system but has developed its own packaging format, referred to as build caches. I've made a prototype pass at using koji plugins to generate these build caches, which has worked well enough to demonstrate that we can share the infrastructure and build functional caches, but has of course spawned some questions. 1) Looking at the Content Generator docs, the model workflow only use Koji as a convenient repository for artifacts and metadata built entirely outside of Koji. Is this API still the right one to use if the builds are being done via a builder plug-in? Should I then call CGImport() from the build task and use a post-import hub callback for hub-side finalization, or does it make more sense to add a completion call on the hub, much like it's done for existing builds? 2) Content Generators seem to still require an RPM-style NVR uniqifier. Spack relies only on package names and a cryptographic hash of the sources, dependencies, and build options -- somewhat akin to Nix. My plan is to use the hash as the package version, and the release as a serial number, in case the same package hash needs to be rebuilt. This seems like it should work, but I'm a bit worried that it's non-standard. How closely are Content Generators expected to follow RPM NVR conventions, such as small string lengths, and monotonically-increasing versions? 3) There are a number of classes/functions that currently live in kojid or kojihub.py, which seem like they're broadly useful to plugins. In particular, I would like to lean on the BuildRoot and BaseBuildTask classes from kojid, and the Host and Task classes and get_tag(), get_user(), get_build_target(), and make_task() functions from kojihub.py. Would it make sense to move these into the koji library? 4) Has any thought been given to how the web interface could be made more extensible? While it mostly works with plugin-provided task types, things aren't perfect. For instance, they do not show up in the "Method" filter, and there is no "Watch task" support. Thanks, Jim

3 2

Meeting Notes for Koji Community Meeting on Feb 12, 2020
by Yuli Wang 13 Feb '20

13 Feb '20

*Meeting docs: *https://pad.mozilla.org/p/KojiFeb2020 *Meeting Time:* Feb 12 2020 10:00-11:00AM US/ET *Where: * https://bluejeans.com/1265288041 *Attendees* - Dennis Gregorovic, Jana Cupova, Tomas Kopecek, Brendan Reilly, Neal Gompa, Chris O'Brien, Ken Dreyer, Lisa Smith, Jiri Kriz, Mike McLean, Yuli Wang, Michal Faruga, Yuming Zhu, Nicholas Burr, Andrew Hills, Jonas Brauer, Pierre-Yves Chibon (pingou), Kevin Fenzi *Meeting summary:* 8 upstream koji users besides koji/brew development team joined this meeting. We shared latest Koji release and collected some feedback. then shared next koji release planning, talked if move sidetag plugin to koji etc. more detail please check following notes. *Meeting notes: * - Koji 1.20 released - https://pagure.io/koji/roadmap/1.20/ - https://docs.pagure.org/koji/release_notes/release_notes_1.20/ - Feedback or retrospective - Fedora - bulk uploads have been a huge performance improvement - smooth release / upgrade - Kudos to team on getting so many fixes in - additional dist-repo / zchunk configuration per tag is requested (for magaei) - https://pagure.io/koji/pull-request/1993 - Koji1.20.1 - bugfix - Proposed issues so far - - https://pagure.io/koji/roadmap/1.20.1/ - Koji1.21 - Proposed issues so far - - https://pagure.io/koji/roadmap/1.21 - 54 issues planned now, plan release on Apr. - MBS plans progress? - Red Hat's koji team (brew team) is going to take on development of MBS. - Others you want to talk: - sidetag plugin - do we want it moved into koji itself? - Neal: yes! magaei would really like this. lots of potential with the sidetag plugin - Mike: keeping it as a plugin sets the right expectation that this is _a_ solution but not necessarily _the_solution Last Meeting(Nov 20, 2019) action items review: - Follow up with Brew QE and Ken to see how we can collaborate on Koji integration testing - Jana: Brew QE did connect with Ken and has plans for using playbooks in the future - add koji-devel to calendar invite for future community meetings - Yuli update: koji-devel@ was added in the calendar, got replying: "Your message to koji-devel(a)lists.fedorahosted.org awaits moderator approval" - Yuli sent out email this Monday for the meeting heads up - tkopecek: We can set it as a title in #koji IRC channel +- week before - [kdreyer] that's a good idea - Still needed - Make sure that koji.build works - moving the hosting and domain registration to Red Hat is in progress and should happen soon - once that is done, can figure out how to best use koji.huild Thanks, Yuli

1 0

Upcoming Koji Community Meeting on Feb 12, 2020
by Yuli Wang 10 Feb '20

10 Feb '20

This is Koji Community Meeting Reminder: *Meeting docs: *https://pad.mozilla.org/p/KojiFeb2020 *Meeting Time:* Feb 12 2020 10:00-11:00AM US/ET *Where: * https://bluejeans.com/1265288041 *Proposed Agenda:* - Koji update: - Koji 1.20 released - https://pagure.io/koji/roadmap/1.20/ - Feedback or retrospective - Koji1.20.1 - Proposed issues so far - - https://pagure.io/koji/roadmap/1.20.1/ - MBS plans progress? - Others you want to talk: Feel free to add more topics you want to talk. See you this Wed. Thanks, Yuli

1 0

koji 1.20 dist-repo --split-debuginfo not working?
by Pat Riehecky 04 Feb '20

04 Feb '20

I'm running a fairly simple koji dist-repo and trying to get --split-debuginfo to actually split the debuginfo packages out. As near as I can tell, the debuginfo bits are getting correctly identified by koji.is_debuginfo, but the repo split bits are not actually using that value to route the packages. Am I running it wrongly to try and get the debuginfo into a separate repo? Pat Commands and Output: $ rpm -qa |grep koji koji-hub-plugins-1.20.0-1.el7.noarch koji-utils-1.20.0-1.el7.noarch python2-koji-web-1.20.0-1.el7.noarch koji-builder-plugins-1.20.0-1.el7.noarch koji-builder-1.20.0-1.el7.noarch python2-koji-cli-plugins-1.20.0-1.el7.noarch koji-1.20.0-1.el7.noarch koji-web-1.20.0-1.el7.noarch python2-koji-1.20.0-1.el7.noarch python2-koji-hub-1.20.0-1.el7.noarch python2-koji-hub-plugins-1.20.0-1.el7.noarch koji-hub-1.20.0-1.el7.noarch $ koji dist-repo --split-debuginfo --with-src --comps=/mnt/koji/group.xml TESTTAG abcd1372 tree . . ├── groups │ └── comps.xml ├── repo.json ├── src │ ├── kojipkgs │ ├── Packages │ │ └── b │ │ └── bzip2-1.0.6-26.el8.src.rpm │ ├── pkglist │ └── repodata │ ├── 69d3f220ec4bf74427c8a3e32670e12df7cba1b5b1e83820068c0bc62886cef9-comps.xml │ ├── 7da792eb29362d8fff6f97964596af0ce15f778db1e5fc7bebe8819b5816a768-other.sqlite.bz2 │ ├── 84046ce528b4dc422db3a08fbd3f862d0f43cb9846ac2259ac18fea2bc77ac44-primary.xml.gz │ ├── 9056a74894ec7df26cdf8f35d001e9f7abbd7c71dced9d574ad6d5f1e19df0fc-comps.xml.gz │ ├── b468f6f2bd29a1c61766b1b0b6fd99ac3db6321032e7b43cc9c88d533320d591-primary.sqlite.bz2 │ ├── b5d47d64298d8be1ff451bc17a4a6ddbbb4e8c1acbbe10544195d8216cfe24d0-other.xml.gz │ ├── b8470f8d1e5a356a439113c777dd84e39636ebfbabe140493f90e0058babb1c3-filelists.xml.gz │ ├── c8306769295fbd06c043ea78218f5f78dec5df072246f594128086c614650718-filelists.sqlite.bz2 │ └── repomd.xml └── x86_64 ├── debug │ ├── pkglist │ └── repodata │ ├── 3c61f948284e705ff9dcc68809f2a7d77272639a8f979b4b8127180523c40e3c-other.xml.gz │ ├── 69d3f220ec4bf74427c8a3e32670e12df7cba1b5b1e83820068c0bc62886cef9-comps.xml │ ├── 79cab7b59d4c9c9dc110cfc9a4cc2b63d702499d14f08a8d03f663ca125b240a-primary.sqlite.bz2 │ ├── 9056a74894ec7df26cdf8f35d001e9f7abbd7c71dced9d574ad6d5f1e19df0fc-comps.xml.gz │ ├── c2f18df7a8c4ec934fc857cc7a768f362734a02a8bb29056c317d38127c04d7d-other.sqlite.bz2 │ ├── d03057f39e93561a4a0f73e4424bb7fa4c6a5408fbe69ea51866cc4e95d57e2d-filelists.sqlite.bz2 │ ├── e4bd8be07a6c0a594021b9f749b80a2ea386a3fc9f1b7ba3ef6fd361da7f57bc-primary.xml.gz │ ├── e90cb6ef58252cf788c1e0681e111079975d457d18cc4aa44acb29de64d55a51-filelists.xml.gz │ └── repomd.xml ├── kojipkgs ├── Packages │ └── b │ ├── bzip2-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-debuginfo-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-debugsource-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-devel-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-libs-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-libs-debuginfo-1.0.6-26.el8.x86_64.rpm │ └── bzip2-static-1.0.6-26.el8.x86_64.rpm ├── pkglist └── repodata ├── 1612d2c3e6d4a19ec0c09956eb863067f684016747d135f98d5914ac45e19fdd-filelists.sqlite.bz2 ├── 1d15b7f4fba94d5e689b3e501a14c9b52d1b319f0ad9b3bc98ec0bc962d39bac-primary.sqlite.bz2 ├── 269dbc6d9eb6d9d636e113bf8729cc553414c0cd547ae4d629eebac099d4f6f7-other.sqlite.bz2 ├── 349a9f8497f319ae7b82fe17c8823fdb37c33abe1128f764d3f413385efd294f-primary.xml.gz ├── 69d3f220ec4bf74427c8a3e32670e12df7cba1b5b1e83820068c0bc62886cef9-comps.xml ├── 9056a74894ec7df26cdf8f35d001e9f7abbd7c71dced9d574ad6d5f1e19df0fc-comps.xml.gz ├── be8e4d00683213a20af62d002dfb9571b3aea039593840e376525be41fafb720-other.xml.gz ├── e4587a590847dd722b0a464e0dee955891dca47cfbe3da516ad41b9d9cf97caf-filelists.xml.gz └── repomd.xml 11 directories, 42 files VS $ koji dist-repo --with-src --comps=/mnt/koji/group.xml TESTTAG abcd1372 . ├── groups │ └── comps.xml ├── repo.json ├── src │ ├── kojipkgs │ ├── Packages │ │ └── b │ │ └── bzip2-1.0.6-26.el8.src.rpm │ ├── pkglist │ └── repodata │ ├── 69d3f220ec4bf74427c8a3e32670e12df7cba1b5b1e83820068c0bc62886cef9-comps.xml │ ├── 7da792eb29362d8fff6f97964596af0ce15f778db1e5fc7bebe8819b5816a768-other.sqlite.bz2 │ ├── 84046ce528b4dc422db3a08fbd3f862d0f43cb9846ac2259ac18fea2bc77ac44-primary.xml.gz │ ├── 9056a74894ec7df26cdf8f35d001e9f7abbd7c71dced9d574ad6d5f1e19df0fc-comps.xml.gz │ ├── b468f6f2bd29a1c61766b1b0b6fd99ac3db6321032e7b43cc9c88d533320d591-primary.sqlite.bz2 │ ├── b5d47d64298d8be1ff451bc17a4a6ddbbb4e8c1acbbe10544195d8216cfe24d0-other.xml.gz │ ├── b8470f8d1e5a356a439113c777dd84e39636ebfbabe140493f90e0058babb1c3-filelists.xml.gz │ ├── c8306769295fbd06c043ea78218f5f78dec5df072246f594128086c614650718-filelists.sqlite.bz2 │ └── repomd.xml └── x86_64 ├── kojipkgs ├── Packages │ └── b │ ├── bzip2-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-debuginfo-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-debugsource-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-devel-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-libs-1.0.6-26.el8.x86_64.rpm │ ├── bzip2-libs-debuginfo-1.0.6-26.el8.x86_64.rpm │ └── bzip2-static-1.0.6-26.el8.x86_64.rpm ├── pkglist └── repodata ├── 26953b8ee14f7d64b67dd949270d6b422e13f28d829bf86304a649569876edaf-primary.xml.gz ├── 4c002ec51430fb431a749819c94dfad396668ce73d33a234e7f42a5c2b8eeaaa-filelists.sqlite.bz2 ├── 4edcf979fb001b2e3afdd9bf434386f951641be20a7c65a157f65a89f8f175ee-other.sqlite.bz2 ├── 58a923a54e63474f849f615d2d00ecd62bfa431c422afd43254dc44c85899d9d-filelists.xml.gz ├── 69d3f220ec4bf74427c8a3e32670e12df7cba1b5b1e83820068c0bc62886cef9-comps.xml ├── 9056a74894ec7df26cdf8f35d001e9f7abbd7c71dced9d574ad6d5f1e19df0fc-comps.xml.gz ├── c2fc12efda1dfe044ad26b2f8aa0b40a85895b79902536517a2dedd105a8fd1e-primary.sqlite.bz2 ├── ca85c8e259e26a20c167784c30d6c03b7111c334e3d083e039619c504feec557-other.xml.gz └── repomd.xml 9 directories, 32 files -- Pat Riehecky Fermi National Accelerator Laboratory www.fnal.gov www.scientificlinux.org

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

koji-devel February 2020