[PATCH] use createrepo_c instead of createrepo to make repos if it is available
by Dennis Gilmore
Signed-off-by: Dennis Gilmore <dennis(a)ausil.us>
---
builder/kojid | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/builder/kojid b/builder/kojid
index 3a92ec3..4e53fb5 100755
--- a/builder/kojid
+++ b/builder/kojid
@@ -4252,7 +4252,11 @@ class CreaterepoTask(BaseTaskHandler):
def create_local_repo(self, rinfo, arch, pkglist, groupdata, oldrepo):
koji.ensuredir(self.outdir)
- cmd = ['/usr/bin/createrepo', '-vd', '-o', self.outdir]
+ if os.path.isfile('/usr/bin/createrepo_c'):
+ cmd = ['/usr/bin/createrepo_c']
+ else:
+ cmd = ['/usr/bin/createrepo']
+ cmd.extend(['-vd', '-o', self.outdir])
if pkglist is not None:
cmd.extend(['-i', pkglist])
if os.path.isfile(groupdata):
--
2.4.0
8 years, 1 month
koji and mergerepo_c
by Thomas
Hi Folks,
Quick question, would you consider to use mergerepo_c with option
--all [1] in koji ?
I am happy to work on a patch if it would be accepted. Maybe, we can
keep default behavior and enable with an option on the tag.
My use case is to be able to ship different release built against fix
external repo package that may not be latest version
(Buildrequires:pkg = 1.0.0 while external repo has already 1.1.0)
[1]:
--all
Include all packages with the same name and arch if version or
release is different. If used --method argument is ignored!
--
Thomas
8 years, 2 months
[PATCH] Drop tables and functions only if exist
by Pavol Babincak
At the time of first initialization of Koji database tables and
functions don't exist. This isn't really a problem as SQL commands later
in the script create them.
For non-existent tables or functions psql prints NOTICE instead of an
ERROR. Therefore one can check if database was initialized correctly
with ON_ERROR_STOP set (see psql(1)):
psql returns 3 if an error occurred in a script and the variable
ON_ERROR_STOP was set.
---
docs/schema.sql | 70 ++++++++++++++++++++++++++++-----------------------------
1 file changed, 35 insertions(+), 35 deletions(-)
diff --git a/docs/schema.sql b/docs/schema.sql
index 56418c9..7a46f7f 100644
--- a/docs/schema.sql
+++ b/docs/schema.sql
@@ -1,56 +1,56 @@
-- vim:noet:sw=8
-- still needs work
-DROP TABLE build_notifications;
+DROP TABLE IF EXISTS build_notifications;
-DROP TABLE log_messages;
+DROP TABLE IF EXISTS log_messages;
-DROP TABLE buildroot_listing;
-DROP TABLE image_listing;
+DROP TABLE IF EXISTS buildroot_listing;
+DROP TABLE IF EXISTS image_listing;
-DROP TABLE rpminfo;
-DROP TABLE image_builds;
-DROP TABLE image_archives;
+DROP TABLE IF EXISTS rpminfo;
+DROP TABLE IF EXISTS image_builds;
+DROP TABLE IF EXISTS image_archives;
-DROP TABLE group_package_listing;
-DROP TABLE group_req_listing;
-DROP TABLE group_config;
-DROP TABLE groups;
+DROP TABLE IF EXISTS group_package_listing;
+DROP TABLE IF EXISTS group_req_listing;
+DROP TABLE IF EXISTS group_config;
+DROP TABLE IF EXISTS groups;
-DROP TABLE tag_listing;
-DROP TABLE tag_packages;
+DROP TABLE IF EXISTS tag_listing;
+DROP TABLE IF EXISTS tag_packages;
-DROP TABLE buildroot;
-DROP TABLE repo;
+DROP TABLE IF EXISTS buildroot;
+DROP TABLE IF EXISTS repo;
-DROP TABLE build_target_config;
-DROP TABLE build_target;
+DROP TABLE IF EXISTS build_target_config;
+DROP TABLE IF EXISTS build_target;
-DROP TABLE tag_config;
-DROP TABLE tag_inheritance;
-DROP TABLE tag;
+DROP TABLE IF EXISTS tag_config;
+DROP TABLE IF EXISTS tag_inheritance;
+DROP TABLE IF EXISTS tag;
-DROP TABLE build;
+DROP TABLE IF EXISTS build;
-DROP TABLE task;
+DROP TABLE IF EXISTS task;
-DROP TABLE host_channels;
-DROP TABLE host;
+DROP TABLE IF EXISTS host_channels;
+DROP TABLE IF EXISTS host;
-DROP TABLE channels;
-DROP TABLE package;
+DROP TABLE IF EXISTS channels;
+DROP TABLE IF EXISTS package;
-DROP TABLE user_groups;
-DROP TABLE user_perms;
-DROP TABLE permissions;
+DROP TABLE IF EXISTS user_groups;
+DROP TABLE IF EXISTS user_perms;
+DROP TABLE IF EXISTS permissions;
-DROP TABLE sessions;
-DROP TABLE users;
+DROP TABLE IF EXISTS sessions;
+DROP TABLE IF EXISTS users;
-DROP TABLE event_labels;
-DROP TABLE events;
-DROP FUNCTION get_event();
-DROP FUNCTION get_event_time(INTEGER);
+DROP TABLE IF EXISTS event_labels;
+DROP TABLE IF EXISTS events;
+DROP FUNCTION IF EXISTS get_event();
+DROP FUNCTION IF EXISTS get_event_time(INTEGER);
BEGIN WORK;
--
1.9.3
8 years, 3 months
[PATCH] Don't retry on SSL failures
by Mathieu Bridon
From: Mathieu Bridon <bochecha(a)daitauha.fr>
With the current code, trying to SSL-login with a bad certificate will
just make it look like the client code is hanging.
That's because it tries and tries again, silently, until it reaches it's
maximum retry limit.
But in the case of an SSL error, such as an expired client cert, there's
really no point in retrying.
---
koji/__init__.py | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/koji/__init__.py b/koji/__init__.py
index 58971da..9ebe83a 100644
--- a/koji/__init__.py
+++ b/koji/__init__.py
@@ -57,7 +57,7 @@ import xmlrpclib
import xml.sax
import xml.sax.handler
from xmlrpclib import loads, dumps, Fault
-#import OpenSSL.SSL
+import OpenSSL
import zipfile
def _(args):
@@ -1938,6 +1938,9 @@ class ClientSession(object):
except (SystemExit, KeyboardInterrupt):
#(depending on the python version, these may or may not be subclasses of Exception)
raise
+ except OpenSSL.SSL.Error as e:
+ # There's no point in retrying this
+ raise
except Exception, e:
self._close_connection()
if not self.logged_in:
--
2.4.3
8 years, 4 months
[PATCH] Check that opts is not None before accessing.
by Ralph Bean
This should fix the periodic 500 error we get from koji-web in Fedora
Infra's staging environment.
Signed-off-by: Ralph Bean <rbean(a)redhat.com>
---
www/lib/kojiweb/util.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/www/lib/kojiweb/util.py b/www/lib/kojiweb/util.py
index 89a5ddb..6610beb 100644
--- a/www/lib/kojiweb/util.py
+++ b/www/lib/kojiweb/util.py
@@ -466,7 +466,7 @@ def taskScratchClass(task_object):
request = task_object['request']
if len(request) >= 3:
opts = request[2]
- if opts.get('scratch'):
+ if opts and opts.get('scratch'):
return "scratch"
return ""
--
2.1.0
8 years, 4 months
[PATCH] Improved support for koji profiles.
by Daniel Mach
---
cli/koji | 137 ++++++++++++-------------------------------------------
docs/profiles | 57 +++++++++++++++++++++++
koji/__init__.py | 130 ++++++++++++++++++++++++++++++++++++++++++++++++++++
3 files changed, 215 insertions(+), 109 deletions(-)
create mode 100644 docs/profiles
diff --git a/cli/koji b/cli/koji
index f521d6d..29f26bb 100755
--- a/cli/koji
+++ b/cli/koji
@@ -186,88 +186,9 @@ def get_options():
list_commands()
parser.error('Unknown command: %s' % args[0])
assert False
- # load local config
- defaults = {
- 'server' : 'http://localhost/kojihub',
- 'weburl' : 'http://localhost/koji',
- 'topurl' : None,
- 'pkgurl' : None,
- 'topdir' : '/mnt/koji',
- 'max_retries' : None,
- 'retry_interval': None,
- 'anon_retry' : None,
- 'offline_retry' : None,
- 'offline_retry_interval' : None,
- 'keepalive' : True,
- 'timeout' : None,
- 'use_fast_upload': False,
- 'poll_interval': 5,
- 'krbservice': 'host',
- 'cert': '~/.koji/client.crt',
- 'ca': '~/.koji/clientca.crt',
- 'serverca': '~/.koji/serverca.crt',
- 'authtype': None
- }
- #note: later config files override earlier ones
- configs = koji.config_directory_contents('/etc/koji.conf.d')
- if os.access('/etc/koji.conf', os.F_OK):
- configs.append('/etc/koji.conf')
- if options.configFile:
- fn = os.path.expanduser(options.configFile)
- if os.path.isdir(fn):
- contents = koji.config_directory_contents(fn)
- if not contents:
- parser.error("No config files found in directory: %s" % fn)
- configs.extend(contents)
- else:
- if not os.access(fn, os.F_OK):
- parser.error("No such file: %s" % fn)
- configs.append(fn)
- else:
- user_config_dir = os.path.expanduser("~/.koji/config.d")
- configs.extend(koji.config_directory_contents(user_config_dir))
- fn = os.path.expanduser("~/.koji/config")
- if os.access(fn, os.F_OK):
- configs.append(fn)
- got_conf = False
- for configFile in configs:
- f = open(configFile)
- config = ConfigParser.ConfigParser()
- config.readfp(f)
- f.close()
- if config.has_section(options.profile):
- got_conf = True
- for name, value in config.items(options.profile):
- #note the defaults dictionary also serves to indicate which
- #options *can* be set via the config file. Such options should
- #not have a default value set in the option parser.
- if defaults.has_key(name):
- if name in ('anon_retry', 'offline_retry', 'keepalive', 'use_fast_upload'):
- defaults[name] = config.getboolean(options.profile, name)
- elif name in ('max_retries', 'retry_interval',
- 'offline_retry_interval', 'poll_interval', 'timeout'):
- try:
- defaults[name] = int(value)
- except ValueError:
- parser.error("value for %s config option must be a valid integer" % name)
- assert False
- else:
- defaults[name] = value
- if configs and not got_conf:
- warn("Warning: no configuration for profile name: %s" % options.profile)
- for name, value in defaults.iteritems():
- if getattr(options, name, None) is None:
- setattr(options, name, value)
- dir_opts = ('topdir', 'cert', 'ca', 'serverca')
- for name in dir_opts:
- # expand paths here, so we don't have to worry about it later
- value = os.path.expanduser(getattr(options, name))
- setattr(options, name, value)
-
- #honor topdir
- if options.topdir:
- koji.BASEDIR = options.topdir
- koji.pathinfo.topdir = options.topdir
+
+ defaults = koji.read_config(options.profile, user_config=options.configFile)
+ options._update_loose(defaults.__dict__)
#pkgurl is obsolete
if options.pkgurl:
@@ -1503,11 +1424,11 @@ def linked_upload(localfile, path, name=None):
try:
if name is None:
name = os.path.basename(localfile)
- dest_dir = os.path.join(koji.pathinfo.work(), path)
+ dest_dir = os.path.join(koji_profile_module.pathinfo.work(), path)
dst = os.path.join(dest_dir, name)
koji.ensuredir(dest_dir)
# fix uid/gid to keep httpd happy
- st = os.stat(koji.pathinfo.work())
+ st = os.stat(koji_profile_module.pathinfo.work())
os.chown(dest_dir, st.st_uid, st.st_gid)
print "Linking rpm to: %s" % dst
os.link(localfile, dst)
@@ -2018,7 +1939,7 @@ def handle_prune_signed_copies(options, session, args):
for sig in sigs:
sigkey = sig['sigkey']
by_sig.setdefault(sigkey, []).append(rpminfo)
- builddir = koji.pathinfo.build(binfo)
+ builddir = koji_profile_module.pathinfo.build(binfo)
build_files = 0
build_space = 0
if not by_sig and options.debug:
@@ -2028,7 +1949,7 @@ def handle_prune_signed_copies(options, session, args):
archdirs = {}
sigdirs = {}
for rpminfo in rpms:
- signedpath = "%s/%s" % (builddir, koji.pathinfo.signed(rpminfo, sigkey))
+ signedpath = "%s/%s" % (builddir, koji_profile_module.pathinfo.signed(rpminfo, sigkey))
try:
st = os.lstat(signedpath)
except OSError:
@@ -2285,8 +2206,8 @@ def handle_list_signed(options, session, args):
binfo = session.getBuild(rinfo['build_id'])
build_idx[binfo['id']] = binfo
binfo['name'] = binfo['package_name']
- builddir = koji.pathinfo.build(binfo)
- signedpath = "%s/%s" % (builddir, koji.pathinfo.signed(rinfo, sigkey))
+ builddir = koji_profile_module.pathinfo.build(binfo)
+ signedpath = "%s/%s" % (builddir, koji_profile_module.pathinfo.signed(rinfo, sigkey))
if not os.path.exists(signedpath):
if options.debug:
print "No copy: %s" % signedpath
@@ -2492,7 +2413,6 @@ def anon_handle_latest_build(options, session, args):
if len(args) < 2:
parser.error(_("A tag name and package name must be specified"))
assert False
- pathinfo = koji.PathInfo()
for pkg in args[1:]:
if options.arch:
@@ -2503,7 +2423,7 @@ def anon_handle_latest_build(options, session, args):
for x in data:
z = x.copy()
x['name'] = builds_hash[x['build_id']]['package_name']
- x['path'] = os.path.join(pathinfo.build(x), pathinfo.rpm(z))
+ x['path'] = os.path.join(koji_profile_module.pathinfo.build(x), koji_profile_module.pathinfo.rpm(z))
fmt = "%(path)s"
else:
fmt = "%(name)s-%(version)s-%(release)s.%(arch)s"
@@ -2515,11 +2435,11 @@ def anon_handle_latest_build(options, session, args):
if options.paths:
if options.type == 'maven':
for x in data:
- x['path'] = pathinfo.mavenbuild(x)
+ x['path'] = koji_profile_module.pathinfo.mavenbuild(x)
fmt = "%(path)-40s %(tag_name)-20s %(maven_group_id)-20s %(maven_artifact_id)-20s %(owner_name)s"
else:
for x in data:
- x['path'] = pathinfo.build(x)
+ x['path'] = koji_profile_module.pathinfo.build(x)
fmt = "%(path)-40s %(tag_name)-20s %(owner_name)s"
else:
if options.type == 'maven':
@@ -2597,7 +2517,6 @@ def anon_handle_list_tagged(options, session, args):
parser.error(_("Only one package name may be specified"))
assert False
activate_session(session)
- pathinfo = koji.PathInfo()
package = None
if len(args) > 1:
package = args[1]
@@ -2630,14 +2549,14 @@ def anon_handle_list_tagged(options, session, args):
build_idx = dict([(b['id'],b) for b in builds])
for rinfo in data:
build = build_idx[rinfo['build_id']]
- builddir = pathinfo.build(build)
+ builddir = koji_profile_module.pathinfo.build(build)
if options.sigs:
sigkey = rinfo['sigkey']
- signedpath = os.path.join(builddir, pathinfo.signed(rinfo, sigkey))
+ signedpath = os.path.join(builddir, koji_profile_module.pathinfo.signed(rinfo, sigkey))
if os.path.exists(signedpath):
rinfo['path'] = signedpath
else:
- rinfo['path'] = os.path.join(builddir, pathinfo.rpm(rinfo))
+ rinfo['path'] = os.path.join(builddir, koji_profile_module.pathinfo.rpm(rinfo))
fmt = "%(path)s"
data = [x for x in data if x.has_key('path')]
else:
@@ -2649,11 +2568,11 @@ def anon_handle_list_tagged(options, session, args):
if options.paths:
if options.type == 'maven':
for x in data:
- x['path'] = pathinfo.mavenbuild(x)
+ x['path'] = koji_profile_module.pathinfo.mavenbuild(x)
fmt = "%(path)-40s %(tag_name)-20s %(maven_group_id)-20s %(maven_artifact_id)-20s %(owner_name)s"
else:
for x in data:
- x['path'] = pathinfo.build(x)
+ x['path'] = koji_profile_module.pathinfo.build(x)
fmt = "%(path)-40s %(tag_name)-20s %(owner_name)s"
else:
if options.type == 'maven':
@@ -2721,7 +2640,6 @@ def anon_handle_list_untagged(options, session, args):
opts = {}
if package:
opts['name'] = package
- pathinfo = koji.PathInfo()
data = session.untaggedBuilds(**opts)
if options.show_references:
@@ -2741,7 +2659,7 @@ def anon_handle_list_untagged(options, session, args):
#data = [x for x in data if not refs.has_key(x['id'])]
if options.paths:
for x in data:
- x['path'] = pathinfo.build(x)
+ x['path'] = koji_profile_module.pathinfo.build(x)
fmt = "%(path)s"
else:
fmt = "%(name)s-%(version)s-%(release)s"
@@ -3053,9 +2971,9 @@ def anon_handle_rpminfo(options, session, args):
print "External Repository: %(name)s [%(id)i]" % repo
print "External Repository url: %(url)s" % repo
else:
- print "RPM Path: %s" % os.path.join(koji.pathinfo.build(buildinfo), koji.pathinfo.rpm(info))
+ print "RPM Path: %s" % os.path.join(koji_profile_module.pathinfo.build(buildinfo), koji_profile_module.pathinfo.rpm(info))
print "SRPM: %(epoch)s%(name)s-%(version)s-%(release)s [%(id)d]" % buildinfo
- print "SRPM Path: %s" % os.path.join(koji.pathinfo.build(buildinfo), koji.pathinfo.rpm(buildinfo))
+ print "SRPM Path: %s" % os.path.join(koji_profile_module.pathinfo.build(buildinfo), koji_profile_module.pathinfo.rpm(buildinfo))
print "Built: %s" % time.strftime('%a, %d %b %Y %H:%M:%S %Z', time.localtime(info['buildtime']))
print "Payload: %(payloadhash)s" %info
print "Size: %(size)s" %info
@@ -3127,23 +3045,23 @@ def anon_handle_buildinfo(options, session, args):
if maven_archives:
print "Maven archives:"
for archive in maven_archives:
- print os.path.join(koji.pathinfo.mavenbuild(info), koji.pathinfo.mavenfile(archive))
+ print os.path.join(koji_profile_module.pathinfo.mavenbuild(info), koji_profile_module.pathinfo.mavenfile(archive))
win_archives = session.listArchives(buildID=info['id'], type='win')
if win_archives:
print "Windows archives:"
for archive in win_archives:
- print os.path.join(koji.pathinfo.winbuild(info), koji.pathinfo.winfile(archive))
+ print os.path.join(koji_profile_module.pathinfo.winbuild(info), koji_profile_module.pathinfo.winfile(archive))
rpms = session.listRPMs(buildID=info['id'])
image_info = session.getImageBuild(info['id'])
img_archives = session.listArchives(buildID=info['id'], type='image')
if img_archives:
print 'Image archives:'
for archive in img_archives:
- print os.path.join(koji.pathinfo.imagebuild(info), archive['filename'])
+ print os.path.join(koji_profile_module.pathinfo.imagebuild(info), archive['filename'])
if rpms:
print "RPMs:"
for rpm in rpms:
- print os.path.join(koji.pathinfo.build(info), koji.pathinfo.rpm(rpm))
+ print os.path.join(koji_profile_module.pathinfo.build(info), koji_profile_module.pathinfo.rpm(rpm))
if options.changelog:
changelog = session.getChangelogEntries(info['id'])
if changelog:
@@ -4261,7 +4179,7 @@ def _printTaskInfo(session, task_id, level=0, recurse=True, verbose=True):
files = session.listTaskOutput(task_id)
logs = [filename for filename in files if filename.endswith('.log')]
output = [filename for filename in files if not filename.endswith('.log')]
- files_dir = '%s/%s' % (koji.pathinfo.work(), koji.pathinfo.taskrelpath(task_id))
+ files_dir = '%s/%s' % (koji_profile_module.pathinfo.work(), koji_profile_module.pathinfo.taskrelpath(task_id))
owner = session.getUser(info['owner'])['name']
@@ -6124,9 +6042,8 @@ def anon_handle_download_build(options, session, args):
if not suboptions.topurl:
print "You must specify --topurl to download images"
return 1
- pi = koji.PathInfo(topdir=suboptions.topurl)
for archive in archives:
- url = '%s/%s' % (pi.imagebuild(info), archive['filename'])
+ url = '%s/%s' % (pathinfo.imagebuild(info), archive['filename'])
urls.append((url, archive['filename']))
else:
# can't happen
@@ -6628,7 +6545,9 @@ def activate_session(session):
if __name__ == "__main__":
global options
+ global koji_profile_module
options, command, args = get_options()
+ koji_profile_module = koji.get_profile_module(options.profile)
logger = logging.getLogger("koji")
handler = logging.StreamHandler(sys.stderr)
diff --git a/docs/profiles b/docs/profiles
new file mode 100644
index 0000000..2f21723
--- /dev/null
+++ b/docs/profiles
@@ -0,0 +1,57 @@
+=============
+Koji Profiles
+=============
+This document describes how to work with koji profiles.
+
+
+Command Line Interface
+======================
+Koji client allows connecting to multiple koji instances from CLI
+by using profiles. The default profile is given by executable file name,
+which is 'koji'.
+
+To change koji profile, you can:
+* run koji with --profile=$profile_name argument
+* change executable file name by symlinking $profile_name -> koji
+
+
+Configuration Files
+===================
+Configuration files are located in following locations:
+* /etc/koji.conf
+* /etc/koji.conf.d/*.conf
+* ~/.koji/config.d/*.conf
+* user-specified config
+Koji reads them all, looking for [$profile_name] sections.
+
+
+Using Koji Profiles in Python
+=============================
+Instead of using koji module directly,
+get profile specific module by calling:
+>>> mod = koji.get_profile_module($profile_name)
+
+This module is clone of koji module with additional
+profile specific tweaks.
+
+Profile configuration is available via:
+>>> mod.config
+
+
+Example
+-------
+import koji
+
+fedora_koji = koji.get_profile_module("koji")
+ppc_koji = koji.get_profile_module("ppc-koji")
+
+for i in (fedora_koji, ppc_koji):
+ print "PROFILE: %s" % i.config.profile
+ for key, value in sorted(i.config.__dict__.items()):
+ print " %s = %s" % (key, value)
+ print
+
+
+TODO
+====
+* consider using pyxdg for user config locations
diff --git a/koji/__init__.py b/koji/__init__.py
index c0d5a30..dae52fc 100644
--- a/koji/__init__.py
+++ b/koji/__init__.py
@@ -28,13 +28,16 @@ except ImportError:
sys.stderr.write("Warning: Could not install krbV module. Kerberos support will be disabled.\n")
sys.stderr.flush()
import base64
+import ConfigParser
import datetime
import errno
from fnmatch import fnmatch
import httplib
+import imp
import logging
import logging.handlers
from koji.util import md5_constructor
+import optparse
import os
import os.path
import pwd
@@ -1457,6 +1460,133 @@ def config_directory_contents(dir_name):
return configs
+CONFIG_DEFAULTS = {
+ 'server': 'http://localhost/kojihub',
+ 'weburl': 'http://localhost/koji',
+ 'topurl': None,
+ 'pkgurl': None,
+ 'topdir': '/mnt/koji',
+ 'max_retries': None,
+ 'retry_interval': None,
+ 'anon_retry': None,
+ 'offline_retry': None,
+ 'offline_retry_interval': None,
+ 'keepalive': True,
+ 'timeout': None,
+ 'use_fast_upload': False,
+ 'poll_interval': 5,
+ 'krbservice': 'host',
+ 'cert': '~/.koji/client.crt',
+ 'ca': '~/.koji/clientca.crt',
+ 'serverca': '~/.koji/serverca.crt',
+ 'authtype': None
+}
+
+
+INT_OPTIONS = ['max_retries', 'retry_interval', 'offline_retry_interval', 'poll_interval', 'timeout']
+BOOL_OPTIONS = ['anon_retry', 'offline_retry', 'keepalive', 'use_fast_upload']
+PATH_OPTIONS = ['topdir', 'cert', 'ca', 'serverca']
+
+
+def config_directory_contents(dir_name):
+ configs = []
+ try:
+ conf_dir_contents = os.listdir(dir_name)
+ except OSError, exception:
+ if exception.errno != errno.ENOENT:
+ raise
+ else:
+ for name in sorted(conf_dir_contents):
+ if not name.endswith('.conf'):
+ continue
+ config_full_name = os.path.join(dir_name, name)
+ configs.append(config_full_name)
+ return configs
+
+
+def read_config(profile_name, user_config=None):
+ result = CONFIG_DEFAULTS.copy()
+ for option in CONFIG_DEFAULTS:
+ if option in PATH_OPTIONS:
+ result[option] = os.path.expanduser(result[option])
+
+ configs = []
+
+ # main config
+ configs.append("/etc/koji.conf")
+
+ # conf.d
+ configs.extend(config_directory_contents("/etc/koji.conf.d"))
+
+ # user config
+ configs.append(os.path.expanduser("~/.koji/config"))
+
+ # user conf.d
+ configs.extend(config_directory_contents(os.path.expanduser("~/.koji/conf.d")))
+
+ # TODO: read configs via xdg.BaseDirectory.load_config_path("koji")
+
+ # user config specified in runtime
+ if user_config is not None:
+ configs.append(user_config)
+
+ # read configs in particular order, use the last value found
+ for config_path in configs:
+ if not os.access(config_path, os.F_OK):
+ continue
+ config = ConfigParser.SafeConfigParser()
+ config.readfp(open(config_path, "r"))
+
+ if profile_name not in config.sections():
+ continue
+
+ # check for invalid options
+ invalid_options = []
+ for option in config.options(profile_name):
+ if option not in result:
+ invalid_options.append(option)
+
+ if invalid_options:
+ raise ValueError("Invalid options: %s" % ", ".join(invalid_options))
+
+ for option in config.options(profile_name):
+ if option in BOOL_OPTIONS:
+ result[option] = config.getboolean(profile_name, option)
+ elif option in INT_OPTIONS:
+ result[option] = config.getint(profile_name, option)
+ else:
+ result[option] = config.get(profile_name, option)
+ if option in PATH_OPTIONS:
+ result[option] = os.path.expanduser(result[option])
+
+ result["profile"] = profile_name
+
+ # convert dict to optparse Values
+ options = optparse.Values(result)
+ return options
+
+
+config = read_config("koji")
+
+
+def get_profile_module(profile_name, config=None):
+ """
+ Create module for a koji instance.
+ Override profile specific module attributes:
+ * BASEDIR
+ * config
+ * pathinfo
+ """
+ if config is None:
+ config = read_config(profile_name)
+ mod = imp.new_module("_koji__%s" % profile_name)
+ mod.__dict__.update(globals())
+ mod.config = config
+ mod.BASEDIR = config.topdir
+ mod.pathinfo = PathInfo(topdir=config.topdir)
+ return mod
+
+
class PathInfo(object):
# ASCII numbers and upper- and lower-case letter for use in tmpdir()
ASCII_CHARS = [chr(i) for i in range(48, 58) + range(65, 91) + range(97, 123)]
--
2.4.3
8 years, 4 months
Koji 2.0 planning
by Mike McLean
It's been eight years since koji 1.0. In that time Koji has grown a lot,
but always incrementally and with great care to avoid breaking
compatibility. Over the years, we've found numerous things that we
wanted to add or change, but that we dismissed as too big, too
complicated, or too invasive.
Bumping the major release number gives use the freedom to shake things
up a bit. Koji 2.0 is about making major changes, otherwise it would
just be koji 1.13.
Koji 2.0 will take some time. We will maintain Koji 1.x until 2.0 is
sufficiently stable. Some features (e.g. content generators) will also
appear in 1.x.
The list below is probably not complete, it is ambitious, and it is
certainly open to discussion. If you are a Koji user, or otherwise
invested in Koji, then you are encouraged to join in. I expect there
will be a lot to say, so I've created a new mailing list devoted
specifically to Koji development.
https://lists.fedorahosted.org/mailman/listinfo/koji-devel
Also, apologies for the tense summary listing. I can certain expound on
any of these as necessary. I hope this suffices to start some conversation.
= High level goals =
• better documentation
• more community involvement
• refactor/modernize code
• more modular design
∘ content generators
∘ broader, better plugin framework
• better support for different types of build processes
• better support for for different types build output
• make hard-wired restrictions more configurable
• easier to deploy
• better qa process
• better release process
= Highlights/Major changes =
• python3 support
∘ the bulk of the code will target python 2.6 + python-six
∘ we'll create a basic client lib for older systems (e.g rhel5
clients/builders)
• drop xmlrpc in favor a json based rpc
• build namespaces
∘ allow for use cases that require multiple builds of the same NVR
(or NVRA)
• refactor task scheduling
• extend content generator support
∘ content generators will land in 1.x fairly soon, but in 2.0 they
will be more integral
∘ refactor kojid to use content generator calls
∘ (possibly) tighter integration in the db
• unify handling of rpms with other build types in the db
∘ e.g. unify rpminfo and archiveinfo tables
• support different ways of building (possibly via content generators)
• utilize jsonb fields in postgres
• modular auth
∘ make it easier to add new auth mechanisms
∘ support openid auth
• improve plugins
∘ make the plugin framework cleaner and more pythonic
∘ support plugins in cli and web ui
• improve/update web ui
∘ drop cheetah templates in favor of python-jinja2
∘ more parity with cli
∘ history browsing
∘ landing page search or history
∘ support plugins
• change how tag pkglists (and blocking) work
• refactor package ownership
• refactor uploads
• more flexible gc
= Yet more changes =
• store all task requests by named args
∘ (for ease of inspection)
• get rid of tagBuild tasks
• drop odd event refererences in favor of timestamps
• streamlined cli options
• marker files for many things on disk
• more history data
• drop modpython support
• policy code
∘ more robust syntax
‣ test negation
‣ OR
‣ parentheses
‣ quoted strings
∘ multiple result policies (non-terminal actions)
∘ all-matches policy (needed for scheduler?)
∘ break action (breaks out of nesting)
∘ stop action (halts processing of an all-matches policy)
8 years, 5 months
pxe install error
by jefby
Hello,everyone
When i use the pxe method to install the system for ARM64 machine , i
found some error when anaconda install the packages,the fault is like
below , how to fix it ?? why ?
Thanks very much
find_file: stat /proc/device-tree/chosen/bootpath, No such file or directory
anaconda 21.48.21-1 for Fedora 21 (pre-release) started.
* installation log files are stored in /tmp during the installation
art
Traceback (most recent call last):
File "/sbin/anaconda", line 1295, in <module>
setupDisplay(anaconda, opts, addon_paths)
File "/sbin/anaconda", line 749, in setupDisplay
if not nm_is_connecting() and not nm_is_connected():
File "/usr/lib64/python2.7/site-packages/pyanaconda/nm.py", line
156, in nm_is_connecting
return nm_state() == NetworkManager.State.CONNECTING
File "/usr/lib64/python2.7/site-packages/pyanaconda/nm.py", line
128, in nm_state
prop = _get_property("/org/freedesktop/NetworkManager", "State")
File "/usr/lib64/python2.7/site-packages/pyanaconda/nm.py", line
106, in _get_property
proxy = _get_proxy(object_path=object_path,
interface_name="org.freedesktop.DBus.Properties")
File "/usr/lib64/python2.7/site-packages/pyanaconda/nm.py", line 94,
in _get_proxy
cancellable)
GLib.Error: g-io-error-quark: Error calling StartServiceByName for
org.freedesktop.NetworkManager: Timeout was reached (24)
Pane is dead
File "/usr/lib64/python2.7/site-packages/pyanaconda/nm.py", line
106, in _get_property
proxy = _get_proxy(object_path=object_path,
interface_name="org.freedesktop.DBus.Properties")
File "/usr/lib64/python2.7/site-packages/pyanaconda/nm.py", line 94,
in _get_proxy
cancellable)
GLib.Error: g-io-error-quark: Error calling StartServiceByName for
org.freedesktop.NetworkManager: Timeout was reached (24)
Pane is dead
jefby
8 years, 5 months
Cannot login from custom spins without repair
by Ed I
Hi folks,
I logged a ticket on the RedHat Bugzilla here
https://bugzilla.redhat.com/show_bug.cgi?id=1230385, and since then I
have had no additional clues in trying to resolve this problem.
I'm hoping someone can shed some insight on how I can determine the
cause. You can see a full description of that problem in the ticket,
but in brief, I have what seems to be an SELinux problem with my
custom spins that requires a policy relabel before I can login for the
first time.
I have the feeling the issue is that I am missing a package in my
Kickstart, but I cannot determine which, or if that is actually the
problem. I don't have a problem when I build a spin with GNOME or
XFCE, only when I omit the desktop environment for a simple window
manager, such as XMonad.
If anyone cares to look at that Kickstart file, would you be able to
spot if there is a missing package that would get rid of this problem?
Thank you!
Ed
8 years, 5 months
getting koji metadata from (or, failing that, into) packages
by Matthew Miller
It strikes me as useful for a number of things to get the build tag and
SCM URL from an RPM without searching in koji. Is this currently
possible in some way I just don't know? If not, might it be something
we could add in the future (possibly including extending rpm)?
--
Matthew Miller
<mattdm(a)fedoraproject.org>
Fedora Project Leader
8 years, 5 months