koji's mergerepos does not consider epoch tag
by Atanas Zhelev
Hi all,
i'm using external repository with various vendor supplied rpm's in koji. One
of those rpm's had it's versioning schema changed and it uses epoch tag now.
After regeneration of the build tags koji does not see the new package. It
looks like mergerepos ignores the epoch tag. The "old" rpm which actually has
a higher version number if epoch is not considered is included instead.
Can you offer any advice?
Thanks
--
BR
Atanas
9 years, 2 months
docker-rpm-builder stable released - build native RPMs through docker
by Alan Franzoni
Hello,
I had given a shot to this project some months ago; after a bit of
refactoring, it's now usable.
(to the Fedora/Redhat people: if you feel I'm hijacking this ml with
my announcements, just let me know; since it's a low-traffic ml and
highly relevant to the topic, I thought it the best place where to
place such announcements)
https://github.com/alanfranz/docker-rpm-builder
RPM packages for Centos/RHEL and Fedora are available now.
Key features:
- fast RPM building without chroot unpacking/file copy/etc
- templating support for specfiles
- interactive mode when a build fails
- build straight from a directory without an intermediate source RPM
(optional: you can still rebuild a .src.rpm)
- works on any host Linux distro that supports docker and has Python 2.7
- leverages existing RPM toolchain, you don't need to learn a new language
- root filesystem is fully writeable during the build
If anybody happens to try it, let me know what you think about it.
I'll be doing a few benchmarks in the next weeks.
Bye!
--
My development blog: ollivander.franzoni.eu . @franzeur on Twitter
contact me at public(a)[mysurname].eu
9 years, 2 months
Fw: Mock, autosigning
by Clark Williams
Second send, since I managed to use the old buildsys list address in my
previous forward of this message. Sigh...
I've forwarded Julian's proposed plugin for mock to perform signing
since he wasn't sure where best to send it. I have *not* reviewed this
patch yet but will be doing so in the near future. Help reviewing from
anyone with signing expertise in the Fedora infrastructure would be
appreciated.
Clark
Begin forwarded message:
Date: Tue, 10 Feb 2015 01:30:03 +0000
From: Julien Ballet <lta(a)fb.com>
To: "clark.williams(a)gmail.com" <clark.williams(a)gmail.com>
Subject: Mock, autosigning
Hi,
I’ve been wandering around the different mock pages (wiki/trac), and I
haven’t found any indications about the preferred way to submit patches
for mock. The wiki page tells to go to Trac, and trac doesn’t say
anything :) So please forgive me for looking at git logs and reaching
out to the most recent/active contributor :)
I’ve been working on a small patch to mock that adds automatic signing
of built package. It also introduces a small api change, so please let
me know if it’s wrong and if you have a better idea :) Let me also know
if there’s a better place to post patches :)
Regards,
Lta.
9 years, 2 months
How should mock behave when DNF is not available?
by Miroslav Suchý
I am working on switching rawhide targets to DNF. However on EL* is DNF
not available.
Therefore you will not be able to build rawhide packages on EL7- using
standard mock configuration.
I see several options to offer to user:
1) Discourage user from doing that by providing no hints how to
workaround it.
2) Tell him that DNF is not available and he must run mock with "--yum"
option. And warn him that he may get different depsolving.
3) Tell him that DNF is not available and fall back automatically to yum
and warn him that he may get different depsolving.
Unless somebody will have better idea I would go with (2).
Ideas? Comments?
Mirek
9 years, 2 months
[PATCH] add missed permission check in RootExports.wrapperRPM
by Chenxiong Qi
From: Chenxiong Qi <cqi(a)redhat.com>
according to the docstring and the behavior in other similar methods,
permission should be checked against current session before increasing
priority. RootExports.wrapperRPM has no such step.
The process of adjusting priority gets refactored to avoid duplicated code.
---
hub/kojihub.py | 60 +++++++++++++++++++++----------------------------------
1 files changed, 23 insertions(+), 37 deletions(-)
diff --git a/hub/kojihub.py b/hub/kojihub.py
index 92fc889..61fb498 100644
--- a/hub/kojihub.py
+++ b/hub/kojihub.py
@@ -7002,6 +7002,20 @@ class RootExports(object):
context.session.assertPerm('admin')
return make_task('restartHosts', [], priority=priority)
+ def _adjust_priority(self, priority):
+ """Adjust priority relative to the default priority
+
+ priority: the amount to increase (or decrease) the task priority,
+ relative to the default priority; higher values mean lower
+ priority. Only admins have the right to specify a negative
+ priority here, otherwise, ActionNotAllowed will be raised.
+ returns final priority
+ """
+ if priority < 0:
+ if not context.session.hasPerm('admin'):
+ raise koji.ActionNotAllowed, 'only admins may create high-priority tasks'
+ return koji.PRIO_DEFAULT + priority
+
def build(self, src, target, opts=None, priority=None, channel=None):
"""Create a build task
@@ -7015,10 +7029,7 @@ class RootExports(object):
opts = {}
taskOpts = {}
if priority:
- if priority < 0:
- if not context.session.hasPerm('admin'):
- raise koji.ActionNotAllowed, 'only admins may create high-priority tasks'
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
+ taskOpts['priority'] = self._adjust_priority(priority)
if channel:
taskOpts['channel'] = channel
return make_task('build',[src, target, opts],**taskOpts)
@@ -7040,10 +7051,7 @@ class RootExports(object):
opts = {}
taskOpts = {}
if priority:
- if priority < 0:
- if not context.session.hasPerm('admin'):
- raise koji.ActionNotAllowed, 'only admins may create high-priority tasks'
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
+ taskOpts['priority'] = self._adjust_priority(priority)
if channel:
taskOpts['channel'] = channel
@@ -7067,10 +7075,7 @@ class RootExports(object):
opts = {}
taskOpts = {}
if priority:
- if priority < 0:
- if not context.session.hasPerm('admin'):
- raise koji.ActionNotAllowed, 'only admins may create high-priority tasks'
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
+ taskOpts['priority'] = self._adjust_priority(priority)
if channel:
taskOpts['channel'] = channel
@@ -7111,7 +7116,7 @@ class RootExports(object):
taskOpts = {}
if priority:
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
+ taskOpts['priority'] = self._adjust_priority(priority)
taskOpts['channel'] = channel
return make_task('wrapperRPM', [url, build_target, build, None, opts], **taskOpts)
@@ -7132,10 +7137,7 @@ class RootExports(object):
raise koji.GenericError, "Maven support not enabled"
taskOpts = {}
if priority:
- if priority < 0:
- if not context.session.hasPerm('admin'):
- raise koji.ActionNotAllowed, 'only admins may create high-priority tasks'
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
+ taskOpts['priority'] = self._adjust_priority(priority)
if channel:
taskOpts['channel'] = channel
@@ -7166,10 +7168,7 @@ class RootExports(object):
opts = {}
taskOpts = {}
if priority:
- if priority < 0:
- if not context.session.hasPerm('admin'):
- raise koji.ActionNotAllowed, 'only admins may create high-priority tasks'
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
+ taskOpts['priority'] = self._adjust_priority(priority)
if channel:
taskOpts['channel'] = channel
@@ -7190,12 +7189,7 @@ class RootExports(object):
taskOpts = {'channel': img_type}
taskOpts['arch'] = arch
if priority:
- if priority < 0:
- if not context.session.hasPerm('admin'):
- raise koji.ActionNotAllowed, \
- 'only admins may create high-priority tasks'
-
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
+ taskOpts['priority'] = self._adjust_priority(priority)
return make_task(img_type, [name, version, arch, target, ksfile, opts], **taskOpts)
@@ -7208,12 +7202,8 @@ class RootExports(object):
context.session.assertPerm('image')
taskOpts = {'channel': 'image'}
if priority:
- if priority < 0:
- if not context.session.hasPerm('admin'):
- raise koji.ActionNotAllowed, \
- 'only admins may create high-priority tasks'
+ taskOpts['priority'] = self._adjust_priority(priority)
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
if not opts.has_key('scratch') and not opts.has_key('indirection_template_url'):
raise koji.ActionNotAllowed, 'Non-scratch builds must provide url for the indirection template'
@@ -7228,12 +7218,8 @@ class RootExports(object):
context.session.assertPerm('image')
taskOpts = {'channel': 'image'}
if priority:
- if priority < 0:
- if not context.session.hasPerm('admin'):
- raise koji.ActionNotAllowed, \
- 'only admins may create high-priority tasks'
+ taskOpts['priority'] = self._adjust_priority(priority)
- taskOpts['priority'] = koji.PRIO_DEFAULT + priority
if not opts.has_key('scratch') and not opts.has_key('ksurl'):
raise koji.ActionNotAllowed, 'Non-scratch builds must provide ksurl'
--
1.7.1
9 years, 2 months
[PATCH 1/4] Prepare for the new sources file format
by Mathieu Bridon
From: Mathieu Bridon <bochecha(a)daitauha.fr>
The current format for the 'sources file is as follows:
$hash $filename
We're eventually going to move to:
$hashtype $hash $filename
This commit just prepares us for that, allowing to parse both formats.
If the 'sources' file doesn't contain the $hashtype, it is assumed to be
the one defined in the config file as lookasidehash.
---
src/pyrpkg/__init__.py | 10 +++++++++-
src/pyrpkg/sources.py | 9 +++++----
test/test_sources.py | 2 +-
3 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/src/pyrpkg/__init__.py b/src/pyrpkg/__init__.py
index 5c4dbc5..f9606fe 100644
--- a/src/pyrpkg/__init__.py
+++ b/src/pyrpkg/__init__.py
@@ -1568,7 +1568,15 @@ class Commands(object):
# Default to putting the files where the module is
if not outdir:
outdir = self.path
- for (csum, file) in self._read_sources():
+
+ for entry in self._read_sources():
+ if len(entry) == 2:
+ csum, file = entry
+ hashtype = self.lookasidehash
+
+ else:
+ hashtype, csum, file = entry
+
# See if we already have a valid copy downloaded
outfile = os.path.join(outdir, file)
if os.path.exists(outfile):
diff --git a/src/pyrpkg/sources.py b/src/pyrpkg/sources.py
index c4263ad..8d8b277 100644
--- a/src/pyrpkg/sources.py
+++ b/src/pyrpkg/sources.py
@@ -42,10 +42,11 @@ def _parse_line(line):
stripped_line = line.strip()
if not stripped_line:
return []
- entries = stripped_line.split(' ', 1)
- if len(entries) != 2:
- raise ValueError("Malformed line: %r." % line)
- return entries
+ entries = stripped_line.split(' ')
+ if len(entries) in (2, 3):
+ return entries
+
+ raise ValueError("Malformed line: %r." % line)
def _format_line(entry):
diff --git a/test/test_sources.py b/test/test_sources.py
index 997ab83..71b49fb 100644
--- a/test/test_sources.py
+++ b/test/test_sources.py
@@ -14,7 +14,7 @@ class formatLineTestCase(unittest.TestCase):
def test_wrong_number_of_fields(self):
WRONG_ENTRIES = [
('foo'),
- ('foo', 'bar', 'foo'),
+ ('foo', 'bar', 'foo', 'bar'),
]
for entry in WRONG_ENTRIES:
self.assertRaises(ValueError, sources._format_line, entry)
--
2.1.0
9 years, 2 months
status of signed rawhide
by Miroslav Suchý
What is the status of signing packages in rawhide?
Is every package now signed?
I am asking whether I should put gpgcheck=1 in mock configs.
Mirek
9 years, 2 months
Mock and DNF - the plan
by Miroslav Suchý
Hi,
today we [1] had a post-DevConf meeting and discussed the best approach to switch Mock to use DNF as default.
The resolution is that:
* just after F22 git branching (which should happen tomorrow) I will change fedora-rawhide-*.cfg files in mock to use
DNF. For everything else YUM will be still used. And YUM will be general default.
* Michael McLean will adjust builroot tags in koji code to allow setting of additional keys/values and it will be used
by Koji to generate mock config which will either use DNF or YUM.
* The koji.fedoraproject.org will not be changed right now, but likely in 1-3 months.
* During this transient period local mock builds for rawhide can differ from koji builds due different depsolving. I
will prepare announcement for devel mailing list so people are aware of this and encourage people to report any problem
they experience. Hopefully if there will be some problems we will have time to address them before the new code land in
Koji.
This change only affects future rawhide (Fedora 23) targets and it will not affect older Fedora and EPELs.
[1] me, Dennis Gilmore, Peter Robinson, Jan Zeleny, Michael Mraka, Michael McLean and Daniel Mach
--
Miroslav Suchy, RHCE, RHCDS
Red Hat, Senior Software Engineer, #brno, #devexp, #fedora-buildsys
9 years, 2 months
[PATCH] simplify the statistics of packagesbyuser by a single XMLRPC call
by Chenxiong Qi
From: Chenxiong Qi <cqi(a)redhat.com>
Original implementation sends multiple times of calls to listPackages to gather
the number of packages of each user, then multiple database queries and
unnecessary operations in listPackages that are not related to the statistics
itself will happen repeatedly while iterating users.
This patch simplifies this process by providing a higher level API, which
requires only one database query against the tag_packages table, and any client
is able to get the statistics data by a single XMLRPC call.
---
hub/kojihub.py | 12 ++++++++++++
www/kojiweb/index.py | 7 ++-----
2 files changed, 14 insertions(+), 5 deletions(-)
diff --git a/hub/kojihub.py b/hub/kojihub.py
index 92fc889..d5e9aa0 100644
--- a/hub/kojihub.py
+++ b/hub/kojihub.py
@@ -9332,6 +9332,18 @@ class RootExports(object):
values=locals(), opts=queryOpts)
return query.iterate()
+ def reportPackagesByUser(self):
+ '''Report of packages by user
+
+ Return a mapping from user.id to the number of packages
+ '''
+ fields = ('user_id', 'packages_count')
+ sql = '''
+SELECT owner, COUNT(package_id) FROM tag_packages
+GROUP BY owner ORDER BY owner'''
+ rows = _multiRow(sql, None, fields)
+ return dict(((str(row['user_id']), row['packages_count']) for row in rows))
+
class BuildRoot(object):
diff --git a/www/kojiweb/index.py b/www/kojiweb/index.py
index ff15a5f..279e6fc 100644
--- a/www/kojiweb/index.py
+++ b/www/kojiweb/index.py
@@ -1919,13 +1919,10 @@ def packagesbyuser(environ, start=None, order=None):
maxPackages = 1
users = server.listUsers()
+ data = server.reportPackagesByUser()
- server.multicall = True
for user in users:
- server.count('listPackages', userID=user['id'], with_dups=True)
- packageCounts = server.multiCall()
-
- for user, [numPackages] in zip(users, packageCounts):
+ numPackages = data.get(str(user['id']), 0)
user['packages'] = numPackages
if numPackages > maxPackages:
maxPackages = numPackages
--
1.7.1
9 years, 2 months
[PATCH] simplify the statistics of packagesbyuser by a single XMLRPC call
by Chenxiong Qi
From: Chenxiong Qi <cqi(a)redhat.com>
Original implementation sends multiple times of calls to listPackages to gather
the number of packages of each user, then multiple database queries and
unnecessary operations in listPackages that are not related to the statistics
itself will happen repeatedly while iterating users.
This patch simplifies this process by providing a higher level API, which
requires only one database query against the tag_packages table, and any client
is able to get the statistics data by a signle XMLRPC call.
---
.gitignore | 2 ++
hub/kojihub.py | 12 ++++++++++++
www/kojiweb/index.py | 7 ++-----
3 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/.gitignore b/.gitignore
index 4857165..3a8896f 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,3 +1,5 @@
*.pyc
*.pyo
tests/test.py
+*.swp
+*~
diff --git a/hub/kojihub.py b/hub/kojihub.py
index 92fc889..d5e9aa0 100644
--- a/hub/kojihub.py
+++ b/hub/kojihub.py
@@ -9332,6 +9332,18 @@ class RootExports(object):
values=locals(), opts=queryOpts)
return query.iterate()
+ def reportPackagesByUser(self):
+ '''Report of packages by user
+
+ Return a mapping from user.id to the number of packages
+ '''
+ fields = ('user_id', 'packages_count')
+ sql = '''
+SELECT owner, COUNT(package_id) FROM tag_packages
+GROUP BY owner ORDER BY owner'''
+ rows = _multiRow(sql, None, fields)
+ return dict(((str(row['user_id']), row['packages_count']) for row in rows))
+
class BuildRoot(object):
diff --git a/www/kojiweb/index.py b/www/kojiweb/index.py
index ff15a5f..279e6fc 100644
--- a/www/kojiweb/index.py
+++ b/www/kojiweb/index.py
@@ -1919,13 +1919,10 @@ def packagesbyuser(environ, start=None, order=None):
maxPackages = 1
users = server.listUsers()
+ data = server.reportPackagesByUser()
- server.multicall = True
for user in users:
- server.count('listPackages', userID=user['id'], with_dups=True)
- packageCounts = server.multiCall()
-
- for user, [numPackages] in zip(users, packageCounts):
+ numPackages = data.get(str(user['id']), 0)
user['packages'] = numPackages
if numPackages > maxPackages:
maxPackages = numPackages
--
1.7.1
9 years, 2 months