[PATCH] srpm-excluded-arch.py enhancement
by Jakub Cajka
Hello,
I have prepared enhancement of srpm-excluded-arch.py based on sharkcz's ideas(if I'm not mistaken :) ). I have moved this script from optparse to argparse, added possibility to run this script over multiple paths/directories(for example updates and release, etc.) and prepare list of excluded packages based on most recent ENVR. Also is possible to issue warning(to stderr) if package have multiple versions present and state of exclusion have changed(to warn about possible unintentional changes). Command syntax should be the same as original version, except switch -w, which adds up mentioned warnings(outputted to stderr), and --path accepts more than one path now, expansion is done up to 3 levels using python glob.
I hope it brings desired improvements( and no bugs :)), looking forward for any feedback,... Patch follows.
Best regards,
Jakub
>From b91af25ac1a89b4892cc1ffb4af60346cdb40eca Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jakub=20=C4=8Cajka?= <jcajka(a)redhat.com>
Date: Mon, 23 Feb 2015 15:57:48 +0100
Subject: [PATCH] srpm-excluded-arch.py can check multiple paths
---
scripts/srpm-excluded-arch.py | 97 ++++++++++++++++++++++++++++++-------------
1 file changed, 68 insertions(+), 29 deletions(-)
diff --git a/scripts/srpm-excluded-arch.py b/scripts/srpm-excluded-arch.py
index 2125517..718f7c2 100755
--- a/scripts/srpm-excluded-arch.py
+++ b/scripts/srpm-excluded-arch.py
@@ -3,39 +3,54 @@
# srpm-exlcuded-arch: is a tool to give you a list of packge names that
# are excluded on the given arches. access to a srpm tree is needed.
#
-# Copyright (C) 2008-2013 Red Hat, Inc.
+# Copyright (C) 2008-2015 Red Hat, Inc.
# SPDX-License-Identifier: GPL-2.0+
#
# Authors:
# Dennis Gilmore <dennis(a)ausil.us>
+# Jakub Cajka <jcajka(a)redhat.com>
import rpm
import os
import sys
-import optparse
+import argparse
import glob
+import types
-OptionParser = optparse.OptionParser
-usage = "%prog [options]"
-parser = OptionParser(usage=usage)
-parser.add_option("-a", "--arches",
- help="space or command separated list of arches to check for")
-parser.add_option("--path", default='./',
- help="path to dir with srpms, default current directory")
-(options, args) = parser.parse_args()
-arches = options.arches
-if arches == None:
- print "You must pass arches to check for in."
- sys.exit()
-else:
- if arches.find(',') == -1:
- arches = arches.split(' ')
- else:
- arches = arches.split(',')
-
-srpm_path = options.path
-srpms = glob.glob('%s/*.rpm' % srpm_path)
-pkglist = []
+def rpmvercmp((e1, v1, r1), (e2, v2, r2)):
+ """find out which build is newer"""
+ rc = rpm.labelCompare((e1, v1, r1), (e2, v2, r2))
+ if rc == 1:
+ #first evr wins
+ return 1
+ elif rc == 0:
+ #same evr
+ return 0
+ else:
+ #second evr wins
+ return -1
+
+parser = argparse.ArgumentParser()
+parser.add_argument('-a', '--arches', nargs='+', required=True)
+parser.add_argument('-w', '--withcheck', action='store_true')
+parser.add_argument('--path', nargs='+', default='./')
+args = parser.parse_args()
+
+# be previous version compatible i.e. -a "arch arch ..."
+if len(args.arches) == 1 and type(args.arches[0]) is types.StringType:
+ if args.arches[0].find(',') == -1:
+ args.arches = args.arches[0].split(' ')
+ else:
+ args.arches = args.arches[0].split(',')
+
+arches = args.arches
+srpms = []
+
+for srpm_path in args.path:
+ srpms += glob.glob('%s/*/*/*.rpm' % srpm_path)
+ srpms += glob.glob('%s/*/*.rpm' % srpm_path)
+ srpms += glob.glob('%s/*.rpm' % srpm_path)
+pkglist = {}
for srpm in srpms:
"""Return the rpm header."""
@@ -52,14 +67,38 @@ for srpm in srpms:
if arch not in hdr[rpm.RPMTAG_EXCLUSIVEARCH]:
if arch not in ExcludeArch:
ExcludeArch.append(arch)
- if ExcludeArch == arches:
- pkgname = hdr[rpm.RPMTAG_NAME]
- if pkgname not in pkglist:
- pkglist.append(pkgname)
- #print "Excluding: %s" % pkgname
+
+ pkgname = hdr[rpm.RPMTAG_NAME]
+ pkgevr = ('0'if not hdr[rpm.RPMTAG_EPOCH] else str(hdr[rpm.RPMTAG_EPOCH]),
+ hdr[rpm.RPMTAG_VERSION],
+ hdr[rpm.RPMTAG_RELEASE])
+ if not pkgname in pkglist:
+ pkglist[pkgname] = []
+ excluded = set(ExcludeArch) == set(arches)
+
+ pkglist[pkgname].append({'evr':pkgevr, 'excluded':excluded})
output = ""
+warning = ""
+
for pkg in pkglist:
- output += pkg + " "
+ pkglist[pkg].sort(cmp=rpmvercmp, key=lambda x: x['evr'], reverse=True)
+ if pkglist[pkg][0]['excluded']:
+ output += pkg + " "
+
+ if args.withcheck:
+ last = None
+ change = ""
+ for srpm in pkglist[pkg]:
+ if last and last['excluded'] != srpm['excluded']:
+ change += str(srpm)+'->'+str(last)
+ last = srpm
+ if change:
+ warning += pkg + " " + change + "\n"
+
+if args.withcheck and warning:
+ warning = "WARNING: Fellowing packages have changed their exclude state:\n"+warning
+ sys.stderr.write(warning)
print output
+
--
1.9.3
8 years
FAD proposal
by Paul W. Frields
One of Matthew Miller's goals for Fedora is to diversify our release
deliverables. This is one reason the Fedora Engineering team is
bringing on a full-time person soon to work on release tools. We want
to contribute to the Fedora release engineering effort too (as opposed
to simply asking existing folks to do more). :-)
I would expect over the first month or two in that role, our release
tools person will likely be:
* looking at the field of deliverables needed, across not just the
products but also at containers and Project Atomic needs
* getting familiar with any current release processes as needed
* discussion with stakeholders in the project, including rel-eng,
mattdm, infrastructure, and others, on this list and elsewhere about
requirements
During this time, I'd expect there will be some decisions on process,
architecture, etc. To kickstart development, I'd like to set up a FAD
where key community members, including developers and release
engineers, can start (or continue) working on actual code and
supporting infrastructure.
I started a wiki page here for interested people to sign up:
https://fedoraproject.org/wiki/FAD_Release_Tools_and_Infrastructure_2015
There are some proposed dates listed on the wiki page. None have been
fully decided yet. I have preliminary approval from the FPL, and I'm
awaiting OSAS input on budget availability, and any time restrictions
from their POV. There is a good chance we will do this at the Red Hat
Tower (HQ) in Raleigh, NC USA. The event will not be restricted to
USA people, although we'll have to select a roster of people to meet
goals.
There's a specific section in the page for you to write your name and
any restrictions on dates. This will help us make the best
date/location plan.
https://fedoraproject.org/wiki/FAD_Release_Tools_and_Infrastructure_2015#...
--
Paul W. Frields http://paul.frields.org/
gpg fingerprint: 3DA6 A0AC 6D58 FEC4 0233 5906 ACDB C937 BD11 3717
http://redhat.com/ - - - - http://pfrields.fedorapeople.org/
The open source story continues to grow: http://opensource.com
8 years
#6017: please give kushal permissions to do scratch cloud image builds in koji
by Fedora Release Engineering
#6017: please give kushal permissions to do scratch cloud image builds in koji
----------------------------+------------------------
Reporter: mattdm | Owner: rel-eng@…
Type: task | Status: new
Milestone: Fedora 21 Beta | Component: koji
Keywords: | Blocked By:
Blocking: |
----------------------------+------------------------
If I remember right (and I admit I remember only vaguely) there are
special permissions required to build images in koji. Please give these to
kushal so he can create scratch images to test things out. Thank you!
--
Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6017>
Fedora Release Engineering <http://fedorahosted.org/rel-eng>
Release Engineering for the Fedora Project
8 years, 1 month
#6040: system to run regular orphaned packages status reports from
by Fedora Release Engineering
#6040: system to run regular orphaned packages status reports from
-----------------------------+------------------------
Reporter: till | Owner: rel-eng@…
Type: task | Status: new
Milestone: Fedora 20 Final | Component: koji
Keywords: meeting | Blocked By:
Blocking: |
-----------------------------+------------------------
Currently I am creating status reports for orphaned packages from
autosign01, but this should move to its own system.
Requirements:
- run cron jobs
- send e-mails
- fast access to koji and pkgdb
- access to mash repos
- possible in the future: access to a directory served by a web server for
HTML based status reports
--
Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6040>
Fedora Release Engineering <http://fedorahosted.org/rel-eng>
Release Engineering for the Fedora Project
8 years, 1 month
#5939: f21-kde koji tag/target
by Fedora Release Engineering
#5939: f21-kde koji tag/target
---------------------+------------------------
Reporter: rdieter | Owner: rel-eng@…
Type: task | Status: new
Milestone: | Component: koji
Keywords: | Blocked By:
Blocking: |
---------------------+------------------------
Please create a f21-kde koji tag/target.
kde-sig will continue to have monthly kde4 and kde frameworks 5 updates to
work on, so having this till be greatly beneficial.
--
Ticket URL: <https://fedorahosted.org/rel-eng/ticket/5939>
Fedora Release Engineering <http://fedorahosted.org/rel-eng>
Release Engineering for the Fedora Project
8 years, 1 month
#1107: auto-cleanup rawhide trees
by Fedora Release Engineering
#1107: auto-cleanup rawhide trees
-------------------------+--------------------------------------------------
Reporter: notting | Owner: rel-eng(a)lists.fedoraproject.org
Type: enhancement | Status: new
Milestone: | Component: koji
Keywords: |
-------------------------+--------------------------------------------------
Rawhide tree expiry is manual at the moment, unless I missed something.
--
Ticket URL: <https://fedorahosted.org/rel-eng/ticket/1107>
Fedora Release Engineering <http://fedorahosted.org/rel-eng>
Release Engineering for the Fedora Project
8 years, 1 month
#6119: AtomicHost rel-eng integration
by Fedora Release Engineering
#6119: AtomicHost rel-eng integration
-----------------------------+------------------------
Reporter: walters | Owner: rel-eng@…
Type: task | Status: new
Milestone: Fedora 21 Final | Component: koji
Keywords: | Blocked By:
Blocking: |
-----------------------------+------------------------
https://fedoraproject.org/wiki/Changes/AtomicHost currently uses rpm-
ostree-toolbox:
https://github.com/projectatomic/rpm-ostree-toolbox/commits/
to generate:
- Installer
* Uses lorax
* But includes ostree content embedded
* Uses fedora-productimg-atomic
- Cloud images
* Uses ImageFactory and spin-kickstarts
- PXE-to-Live:
* Uses livemedia-creator
It runs everything (including ksflatten notably) inside automatically-
updated Docker containers. This is more like what rel-eng tends to do
with Mock, except Docker has a *much* simpler API for things like creating
bind mounts and such.
There are a range of implementation strategies here for integration. We
could run just the "installer" and "liveimage" -toolbox tasks on a compose
box, but continue using Koji for the cloud images as we are today.
(-toolbox does cloud images via ImageFactory locally so I can conveniently
test changes *before* uploading them)
--
Ticket URL: <https://fedorahosted.org/rel-eng/ticket/6119>
Fedora Release Engineering <http://fedorahosted.org/rel-eng>
Release Engineering for the Fedora Project
8 years, 1 month