buildsys February 2013

buildsys@lists.fedoraproject.org

7 participants
9 discussions

Koji-1.7.1-1.fc17 repository toplink failure
by Anthony Joseph Messina 04 Jul '13

04 Jul '13

After upgrading to koji-1.7.1 on my private hub and builders, my builders are no longer able to use the repositories they create when building subsequent dependent packages. When attempting to install locally built packages, the following error is issued in the root.log: error was [Errno 2] Local file does not exist: /tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/... The repos are created successfully: $ /usr/bin/createrepo -vd -o /tmp/koji/tasks/9129/19129/repo -i /mnt/koji/repos/f17-build/3627/x86_64/pkglist -g /mnt/koji/repos/f17- build/3627/groups/comps.xml --update --skip-stat /mnt/koji/repos/f17- build/3627/x86_64 Scanning old repo data Spawning worker 0 with 36 pkgs Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-devel-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-libs-2.6.1-3.fc17.1.x86_64.rpm ... ... I believe the issue is related to the following commit: https://fedorahosted.org/koji/changeset/da6dd0b1832e9929565a1ab03ab844aaced… I'm not sure why when building packages, the builders are inserting '/tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink' instead of topdir or topurl, such as '/mnt/koji' or 'http://example.com/kojifiles' Right now I'm working on selectively reverting that commit to see if I can build packages against other local packages. Any assistance is appreciated. Thanks. -Anthony -- Anthony - http://messinet.com - http://messinet.com/~amessina/gallery 8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

4 13

[koji patch] koji-shadow - show top broken packages instead of top brokendep packages
by Karsten Hopp 28 Feb '13

28 Feb '13

Currently koji-shadows prints out the top 5 brokendep packages, which is kind of useless as they don't need fixing. With this patchit prints out the top 10 broken packages, sorted by the number of packages they are blocking from being built. Secondary arch maintainers will know immediately which packages they should probably fix first without having to trace them down in the output. Karsten

1 0

[releng PATCH 1/4] parametrize the arch and tag list
by Dan Horák 28 Feb '13

28 Feb '13

The secondary arch to run sync on and the list of tags need to be provided on the command line now. --- scripts/sync-tagged-primary.py | 130 ++++++++++++++++++++------------------- 1 files changed, 67 insertions(+), 63 deletions(-) diff --git a/scripts/sync-tagged-primary.py b/scripts/sync-tagged-primary.py index ab66ac6..1b89755 100755 --- a/scripts/sync-tagged-primary.py +++ b/scripts/sync-tagged-primary.py @@ -1,4 +1,5 @@ #!/usr/bin/python +# -*- coding: utf-8 -*- # # synd-tagged-primary.py - A utility to sync tagged packages in primary koji # to a secondary arch @@ -7,6 +8,7 @@ # # Authors: # Dennis Gilmore <ausil(a)fedoraproject.org> +# Dan Horák <dan(a)danny.cz> # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -28,12 +30,15 @@ import tempfile import shutil import rpm -# Set some variables -# Some of these could arguably be passed in as args. -tags = ['f17-updates', 'f17-updates-testing', - 'f18-updates', 'f18-updates-testing'] # tag to check in koji +# get architecture and tags from command line +if len(sys.argv) > 2: + arch = sys.argv[1] + tags = sys.argv[2:] +else: + print("Synchronize tagged packages between primary and secondary koji") + print("Usage: %s <arch> <tag> ...<tag-N>" % sys.argv[0]) + exit(0) -arches = ['arm', 'ppc', 's390'] # Should probably set these from a koji config file SERVERCA = os.path.expanduser('~/.fedora-server-ca.cert') @@ -64,61 +69,60 @@ def rpmvercmp ((e1, v1, r1), (e2, v2, r2)): return -1 -for arch in arches: - print "=== Working on arch: %s ====" % arch - # Create a koji session - kojisession = koji.ClientSession('https://koji.fedoraproject.org/kojihub') - seckojisession = koji.ClientSession('https://%s.koji.fedoraproject.org/kojihub' % arch ) - seckojisession.ssl_login(CLIENTCERT, CLIENTCA, SERVERCA) - - for tag in tags: - print "=== Working on tag: %s ====" % tag - secblocked = [] # holding for blocked pkgs - totag = [] - tountag = [] - pripkgnvrs = [] - secpkgnvrs = [] - - pripkgs = getTagged(kojisession, tag) - secpkgs = getTagged(seckojisession, tag) - - for pkg in pripkgs: - pripkgnvrs.append(pkg['nvr']) - for pkg in secpkgs: - secpkgnvrs.append(pkg['nvr']) - - for pkg in pripkgs: - if pkg['nvr'] not in secpkgnvrs: - secpkg = seckojisession.getBuild(pkg['nvr']) - # see if we have the build on secondary koji and make sure its complete - if not secpkg is None and secpkg['state'] == 1 : - totag.append(pkg['nvr']) - print "need to tag %s" % pkg['nvr'] - - for pkg in secpkgs: - if pkg['nvr'] not in pripkgnvrs: - # make sure we have had a build of the package on primary tagged into the tag - pripkg = kojisession.tagHistory(tag=tag, package=pkg['name']) - if pripkg == []: - # if the package only exists on secondary let it be - print "Secondary arch only package %s" % pkg['nvr'] - # secondary arch evr is higher than primary untag ours - elif pripkg[0]['active'] == None: - # get the latest build from primary in the tag - pripkg = kojisession.listTagged(tag, latest=True, package=pkg['name']) - if pripkg == [] or rpmvercmp((str(pkg['epoch']), pkg['version'], pkg['release']), (str(pripkg[0]['epoch']), pripkg[0]['version'], pripkg[0]['release'])) == 1: - tountag.append(pkg['nvr']) - print "need to untag %s" % pkg['nvr'] - - seckojisession.multicall = True - for pkg in totag: - print "Tagging: Arch: %s Tag: %s Package: %s" % (arch, tag, pkg) - seckojisession.tagBuildBypass(tag, pkg) - - for pkg in tountag: - print "Untagging: Arch: %s Tag: %s Package: %s" % (arch, tag, pkg) - seckojisession.untagBuildBypass(tag, pkg) - - listings = seckojisession.multiCall() - - seckojisession.logout() +print "=== Working on arch: %s ====" % arch +# Create a koji session +kojisession = koji.ClientSession('https://koji.fedoraproject.org/kojihub') +seckojisession = koji.ClientSession('https://%s.koji.fedoraproject.org/kojihub' % arch ) +seckojisession.ssl_login(CLIENTCERT, CLIENTCA, SERVERCA) + +for tag in tags: + print "=== Working on tag: %s ====" % tag + secblocked = [] # holding for blocked pkgs + totag = [] + tountag = [] + pripkgnvrs = [] + secpkgnvrs = [] + + pripkgs = getTagged(kojisession, tag) + secpkgs = getTagged(seckojisession, tag) + + for pkg in pripkgs: + pripkgnvrs.append(pkg['nvr']) + for pkg in secpkgs: + secpkgnvrs.append(pkg['nvr']) + + for pkg in pripkgs: + if pkg['nvr'] not in secpkgnvrs: + secpkg = seckojisession.getBuild(pkg['nvr']) + # see if we have the build on secondary koji and make sure its complete + if not secpkg is None and secpkg['state'] == 1 : + totag.append(pkg['nvr']) + print "need to tag %s" % pkg['nvr'] + + for pkg in secpkgs: + if pkg['nvr'] not in pripkgnvrs: + # make sure we have had a build of the package on primary tagged into the tag + pripkg = kojisession.tagHistory(tag=tag, package=pkg['name']) + if pripkg == []: + # if the package only exists on secondary let it be + print "Secondary arch only package %s" % pkg['nvr'] + # secondary arch evr is higher than primary untag ours + elif pripkg[0]['active'] == None: + # get the latest build from primary in the tag + pripkg = kojisession.listTagged(tag, latest=True, package=pkg['name']) + if pripkg == [] or rpmvercmp((str(pkg['epoch']), pkg['version'], pkg['release']), (str(pripkg[0]['epoch']), pripkg[0]['version'], pripkg[0]['release'])) == 1: + tountag.append(pkg['nvr']) + print "need to untag %s" % pkg['nvr'] + + seckojisession.multicall = True + for pkg in totag: + print "Tagging: Arch: %s Tag: %s Package: %s" % (arch, tag, pkg) + seckojisession.tagBuildBypass(tag, pkg) + + for pkg in tountag: + print "Untagging: Arch: %s Tag: %s Package: %s" % (arch, tag, pkg) + seckojisession.untagBuildBypass(tag, pkg) + + listings = seckojisession.multiCall() + +seckojisession.logout() -- 1.7.7.6

2 4

[releng PATCH] add a wrapper script that calls the sync script with arches and tags
by Dan Horák 22 Feb '13

22 Feb '13

--- scripts/sync-tagged-primary | 10 ++++++++++ 1 files changed, 10 insertions(+), 0 deletions(-) create mode 100755 scripts/sync-tagged-primary diff --git a/scripts/sync-tagged-primary b/scripts/sync-tagged-primary new file mode 100755 index 0000000..ba4a37f --- /dev/null +++ b/scripts/sync-tagged-primary @@ -0,0 +1,10 @@ +#!/bin/sh + +tags_updates="f17-updates f18-updates" +tags_testing="f17-updates-testing f18-updates-testing" + +for arch in arm ppc s390 +do + ./sync-tagged-primary.py $arch $tags_updates + ./sync-tagged-primary.py $arch $tags_testing +done -- 1.7.7.6

1 0

[releng PATCH 1/3] parametrize the arch and tag list
by Dan Horák 22 Feb '13

22 Feb '13

The secondary arch to run sync on and the list of tags need to be provided on the command line now. --- scripts/sync-tagged-primary.py | 130 ++++++++++++++++++++------------------- 1 files changed, 67 insertions(+), 63 deletions(-) diff --git a/scripts/sync-tagged-primary.py b/scripts/sync-tagged-primary.py index fc37680..1b89755 100755 --- a/scripts/sync-tagged-primary.py +++ b/scripts/sync-tagged-primary.py @@ -1,4 +1,5 @@ #!/usr/bin/python +# -*- coding: utf-8 -*- # # synd-tagged-primary.py - A utility to sync tagged packages in primary koji # to a secondary arch @@ -7,6 +8,7 @@ # # Authors: # Dennis Gilmore <ausil(a)fedoraproject.org> +# Dan Horák <dan(a)danny.cz> # # This program is free software; you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by @@ -28,12 +30,15 @@ import tempfile import shutil import rpm -# Set some variables -# Some of these could arguably be passed in as args. -tags = ['f17-updates', 'f17-updates-testing', 'f18', - 'f18-updates', 'f18-updates-testing'] # tag to check in koji +# get architecture and tags from command line +if len(sys.argv) > 2: + arch = sys.argv[1] + tags = sys.argv[2:] +else: + print("Synchronize tagged packages between primary and secondary koji") + print("Usage: %s <arch> <tag> ...<tag-N>" % sys.argv[0]) + exit(0) -arches = ['arm', 'ppc', 's390'] # Should probably set these from a koji config file SERVERCA = os.path.expanduser('~/.fedora-server-ca.cert') @@ -64,61 +69,60 @@ def rpmvercmp ((e1, v1, r1), (e2, v2, r2)): return -1 -for arch in arches: - print "=== Working on arch: %s ====" % arch - # Create a koji session - kojisession = koji.ClientSession('https://koji.fedoraproject.org/kojihub') - seckojisession = koji.ClientSession('https://%s.koji.fedoraproject.org/kojihub' % arch ) - seckojisession.ssl_login(CLIENTCERT, CLIENTCA, SERVERCA) - - for tag in tags: - print "=== Working on tag: %s ====" % tag - secblocked = [] # holding for blocked pkgs - totag = [] - tountag = [] - pripkgnvrs = [] - secpkgnvrs = [] - - pripkgs = getTagged(kojisession, tag) - secpkgs = getTagged(seckojisession, tag) - - for pkg in pripkgs: - pripkgnvrs.append(pkg['nvr']) - for pkg in secpkgs: - secpkgnvrs.append(pkg['nvr']) - - for pkg in pripkgs: - if pkg['nvr'] not in secpkgnvrs: - secpkg = seckojisession.getBuild(pkg['nvr']) - # see if we have the build on secondary koji and make sure its complete - if not secpkg is None and secpkg['state'] == 1 : - totag.append(pkg['nvr']) - print "need to tag %s" % pkg['nvr'] - - for pkg in secpkgs: - if pkg['nvr'] not in pripkgnvrs: - # make sure we have had a build of the package on primary tagged into the tag - pripkg = kojisession.tagHistory(tag=tag, package=pkg['name']) - if pripkg == []: - # if the package only exists on secondary let it be - print "Secondary arch only package %s" % pkg['nvr'] - # secondary arch evr is higher than primary untag ours - elif pripkg[0]['active'] == None: - # get the latest build from primary in the tag - pripkg = kojisession.listTagged(tag, latest=True, package=pkg['name']) - if pripkg == [] or rpmvercmp((str(pkg['epoch']), pkg['version'], pkg['release']), (str(pripkg[0]['epoch']), pripkg[0]['version'], pripkg[0]['release'])) == 1: - tountag.append(pkg['nvr']) - print "need to untag %s" % pkg['nvr'] - - seckojisession.multicall = True - for pkg in totag: - print "Tagging: Arch: %s Tag: %s Package: %s" % (arch, tag, pkg) - seckojisession.tagBuildBypass(tag, pkg) - - for pkg in tountag: - print "Untagging: Arch: %s Tag: %s Package: %s" % (arch, tag, pkg) - seckojisession.untagBuildBypass(tag, pkg) - - listings = seckojisession.multiCall() - - seckojisession.logout() +print "=== Working on arch: %s ====" % arch +# Create a koji session +kojisession = koji.ClientSession('https://koji.fedoraproject.org/kojihub') +seckojisession = koji.ClientSession('https://%s.koji.fedoraproject.org/kojihub' % arch ) +seckojisession.ssl_login(CLIENTCERT, CLIENTCA, SERVERCA) + +for tag in tags: + print "=== Working on tag: %s ====" % tag + secblocked = [] # holding for blocked pkgs + totag = [] + tountag = [] + pripkgnvrs = [] + secpkgnvrs = [] + + pripkgs = getTagged(kojisession, tag) + secpkgs = getTagged(seckojisession, tag) + + for pkg in pripkgs: + pripkgnvrs.append(pkg['nvr']) + for pkg in secpkgs: + secpkgnvrs.append(pkg['nvr']) + + for pkg in pripkgs: + if pkg['nvr'] not in secpkgnvrs: + secpkg = seckojisession.getBuild(pkg['nvr']) + # see if we have the build on secondary koji and make sure its complete + if not secpkg is None and secpkg['state'] == 1 : + totag.append(pkg['nvr']) + print "need to tag %s" % pkg['nvr'] + + for pkg in secpkgs: + if pkg['nvr'] not in pripkgnvrs: + # make sure we have had a build of the package on primary tagged into the tag + pripkg = kojisession.tagHistory(tag=tag, package=pkg['name']) + if pripkg == []: + # if the package only exists on secondary let it be + print "Secondary arch only package %s" % pkg['nvr'] + # secondary arch evr is higher than primary untag ours + elif pripkg[0]['active'] == None: + # get the latest build from primary in the tag + pripkg = kojisession.listTagged(tag, latest=True, package=pkg['name']) + if pripkg == [] or rpmvercmp((str(pkg['epoch']), pkg['version'], pkg['release']), (str(pripkg[0]['epoch']), pripkg[0]['version'], pripkg[0]['release'])) == 1: + tountag.append(pkg['nvr']) + print "need to untag %s" % pkg['nvr'] + + seckojisession.multicall = True + for pkg in totag: + print "Tagging: Arch: %s Tag: %s Package: %s" % (arch, tag, pkg) + seckojisession.tagBuildBypass(tag, pkg) + + for pkg in tountag: + print "Untagging: Arch: %s Tag: %s Package: %s" % (arch, tag, pkg) + seckojisession.untagBuildBypass(tag, pkg) + + listings = seckojisession.multiCall() + +seckojisession.logout() -- 1.7.7.6

1 2

HFS-related issue with pungi/lorax building fc17 spin
by James Milne 20 Feb '13

20 Feb '13

Hi all, I'm having an issue with making an ISO with pungi. My build environment is constructed with mock. Pungi resolves package dependencies and builds a repository successfully, but lorax fails at the end making the ISO image. My problem appears to be related to making a Mac-compatible HFS boot image. Any ideas what is missing here? rebuilding initramfs images rebuilding boot/initramfs-3.3.4-5.fc17.x86_64.img populating output tree and building boot images running x86.tmpl umount: /tmp/lorax.imgutils.NFTlGE: target is busy. (In some cases useful info about processes that use the device is found by lsof(8) or fuser(1)) Traceback (most recent call last): File "/usr/sbin/mkefiboot", line 139, in <module> opt.diskname) File "/usr/sbin/mkefiboot", line 38, in mkmacboot mkhfsimg(None, outfile, label=label, graft=graft) File "/usr/lib/python2.7/site-packages/pylorax/imgutils.py", line 300, in mkhfsimg mkfsargs=["-v", label], graft=graft) File "/usr/lib/python2.7/site-packages/pylorax/imgutils.py", line 283, in mkfsimage do_grafts(graft, mnt, preserve) File "/usr/lib/python2.7/site-packages/pylorax/imgutils.py", line 196, in __exit__ umount(self.mnt) File "/usr/lib/python2.7/site-packages/pylorax/imgutils.py", line 110, in umount os.rmdir(mnt) OSError : [Errno 16] Device or resource busy: '/tmp/lorax.imgutils.NFTlGE' WARNING: Forcibly unmounting '/var/lib/mock/flipos-2-x86_64/root/tmp/lorax.imgutils.NFTlGE' from chroot. Finish: chroot pungi --nosource --nodebuginfo -G -C -B --name FlipOS -c /usr/share/flipos/flipos2.ks ERROR: Command failed. See logs for output. # pungi --nosource --nodebuginfo -G -C -B --name FlipOS -c /usr/share/flipos/flipos2.ks Regards James Milne FilmLight Ltd.

2 2

[koji PATCH resend] Strip leading and trailing whitespace from search terms.
by Chi Zhang 20 Feb '13

20 Feb '13

Signed-off-by: Chi Zhang <czhang(a)twitter.com> --- www/kojiweb/index.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/www/kojiweb/index.py b/www/kojiweb/index.py index 0505c9c..c9f8cc1 100644 --- a/www/kojiweb/index.py +++ b/www/kojiweb/index.py @@ -2160,7 +2160,7 @@ def search(environ, start=None, order='name'): form = environ['koji.form'] if form.has_key('terms') and form['terms']: - terms = form['terms'].value + terms = form['terms'].value.strip() type = form['type'].value match = form['match'].value values['terms'] = terms -- 1.7.11.7

1 0

[koji PATCH] add '--logfile <filename>' parameter for better logging (koji-shadow)
by Karsten Hopp 13 Feb '13

13 Feb '13

Here's one of the patches to koji-shadow that I'm using now for quite a while. koji-shadow prints everything to the console and it's up to the secondary arch maintainer to redirect it into a file. This patch adds a --logfile parameter where it logs everything into a file and also prints to the console (similar to '... | tee logfile') Karsten

1 0

Initialize a mock chroot environment variable
by Anthony Joseph Messina 02 Feb '13

02 Feb '13

Is it possible to initialize a mock chroot environment variable with the return from the evaulation of a simple script when calling a koji build? Something like: /etc/mock/site-defaults.cfg: config_opts['environment']['MYVAR'] = "/usr/local/sbin/myscript.sh" Where /usr/local/sbin/myscript.sh contains something like: #!/usr/bin/bash file="/tmp/$RANDOM.txt" touch "$file" # Created outside the chroot echo "$file" # $file passed as the MYVAR environment variable If not, might you suggest another way of accomplishing this task? Thanks in advance. -A -- Anthony - http://messinet.com - http://messinet.com/~amessina/gallery 8F89 5E72 8DF0 BCF0 10BE 9967 92DC 35DC B001 4A4E

2 2

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys February 2013