buildsys January 2008

buildsys@lists.fedoraproject.org

19 participants
23 discussions

Koji server installation
by Degremont, Guillaume 28 May '08

28 May '08

Hello, I am currently working on deploying a small build system based on mock/koji The system is very simple, just one server hosting the builds and koji (another buidl system may be added later, but that is not the issue at thje present time). I am having some troubles deploying koji. The only document I have found is the ServerHowTo (http://fedoraproject.org/wiki/Koji/ServerHowTo) I retrieved the koji, koji-hub, koji-web and koji-utils packages and installd them successfully on my server. I configured koji to use SSL following the guidelines. Not being a SSL expert, I think I did not do any error, but it was tricky since filenames change between the certificate creation section and the kojihub/kojweb/kojid configuration sections. and I configured all 4 servers (kojihub, kojiweb, kojira, kojid) to be hosted on the same server, named murray. However, when I try to use koji, I get the following error: [koji@murray ~]$ koji add-user userTest Kerberos authentication failed: 'No credentials cache found' (-1765328189) [koji@murray ~]$ I have modified the /etc/koji.conf (though it is not mentioned in the How To) as follows, to ensure it will use SSL: [root@murray ~]# more /etc/koji.conf [koji] ;configuration for koji cli tool ;url of XMLRPC server server = http://murray.mysite.hp.com/kojihub ;url of web interface weburl = http://murray.mysite.hp.com/koji ;url of package download site pkgurl = http://murray.mysite.hp.com/packages ;path to the koji top directory topdir = /mnt/koji ;configuration for SSL athentication ;client certificate cert = /etc/kojiweb/clients/certs/koji.cert ;certificate of the CA that issued the client certificate ca = /etc/kojiweb/clients/koji_ca_cert.crt ;certificate of the CA that issued the HTTP server certificate serverca = /etc/kojiweb/clients/koji_ca_cert.crt koji_ca_cert.crt being the ca certificate I generated and koji.cert a certificate I generated for the koji user. This is my first problem. Can anyone help me on this ? My other problem is with the servers. I configured my apache and started it to have the kojihub and kojiweb started. I then want to perform some add--user, add-host commands. But I get the message "unable to connect to server". [root@murray ~]# koji --noauth add-host murray.mysite.hp.com i386 x86_64 Error: Unable to connect to server With the following logs from httpd: [Thu Nov 15 10:28:51 2007] [error] [client X.X.X.X] PythonHandler kojixmlrpc: Traceback (most recent call last): [Thu Nov 15 10:28:51 2007] [error] [client X.X.X.X] PythonHandler kojixmlrpc: File "/usr/lib/python2.4/site-packages/mod_python/apache.py", line 299, in HandlerDispatch\n result = object(req) [Thu Nov 15 10:28:51 2007] [error] [client X.X.X.X] PythonHandler kojixmlrpc: File "/usr/share/koji-hub/kojixmlrpc.py", line 278, in handler\n context.cnx = koji.db.connect(opts.get("KojiDebug",False)) [Thu Nov 15 10:28:51 2007] [error] [client X.X.X.X] PythonHandler kojixmlrpc: File "/usr/lib/python2.4/site-packages/koji/db.py", line 128, in connect\n conn = pgdb.connect(**opts) [Thu Nov 15 10:28:51 2007] [error] [client X.X.X.X] PythonHandler kojixmlrpc: File "/usr/lib/python2.4/site-packages/pgdb.py", line 383, in connect\n dbtty, dbuser, dbpasswd) [Thu Nov 15 10:28:51 2007] [error] [client X.X.X.X] PythonHandler kojixmlrpc: InternalError: could not connect to server: Connection refused\n\tIs the server running on host "murray.mysite.hp.com" and accepting\n\tTCP/IP connections on port 5432?\n Do you know what it comes from ? I can supply my other conf files, if needed. But I strictly followed the Howto nstructions for the configuration files. Reagrds, Guillaume Degremont PS: for security related reasons, I replaced the ipaddress with X.X.X.X and changed the hostname and fully qualified domain name with dummy ones ^^

2 1

mock 0.9 backport to F7/F8 -- Feb 1
by Michael E Brown 13 Mar '08

13 Mar '08

All mock users, The mock maintainers (Clark, Jesse, me) will upgrade mock in F7/F8 to current 0.9 on/around Feb 1. The mock 0.9 branch has brewed in rawhide since early Dec, and so far it looks good. The 0.9 branch is now being used on the official build systems, so if there were any major problems, we would expect to have hit them by now. The *only* difference between 0.8.<latest> and 0.9.<latest> at this point is that we have dropped the old mock setuid wrapper and now use the consolehelper subsystem. For this, you will notice new /etc/pam.d/mock, /etc/consolehelper/mock files which configure mock. The default config is set up to operate exactly the same as the old 0.8 branch: ie. you must be a member of the 'mock' group to run mock. Additionally, with consolehelper comes one new feature: if you are not in the 'mock' group, you will be prompted to enter the root password and it will run. This means you can run mock without worrying about any pre-setup. -- Michael

5 6

Killing build jobs - mock python processes survive
by Michael Schwendt 03 Feb '08

03 Feb '08

Does that work successfully in koji? Here, the mock Python processes survive the kill signal. Only the parent mock process terminates. The other ones keep building until the job succeeds or fails.

2 3

Both a .buildstamp and .buildstamp_1 ?!?
by Peter Åstrand 29 Jan '08

29 Jan '08

Installation of our nightly build of our custom distribution failed with the error message: "The OURDIST installation tree in that directory does not seem to match your boot media." After some serious debugging, I found out the problem: In minstg2.img, there were *two* buildstamps: -rw-r--r-- 1 root root 61 27 jan 12.51 squashfs-root/.buildstamp -rw-r--r-- 1 root root 61 28 jan 12.52 squashfs-root/.buildstamp_1 Only the second one matches the initrd buildstamp. Any idea why this happens? My guess is that this is some kind of intermittent problem, but those gives me a bad feeling... Best regards, --- Peter Åstrand ThinLinc Chief Developer Cendio AB http://www.cendio.se Wallenbergs gata 4 583 30 Linköping Phone: +46-13-21 46 00

3 6

New upstream release of Koji (and a refresh of Fedora builders?)
by Jesse Keating 25 Jan '08

25 Jan '08

I'd like to do a new upstream release of koji soon, to take advantage of createrepo improvements and to have a release with some of the bugfixes that I've found since our refresh to RHEL5/new mock. The builders are locally patched for the bugfixes but I'd rather have them in a package. Any objections to doing a release next week and scheduling an evening to roll out upgrades? A small outage will be needed on the hub, and then the builders can have a rolling outage. There are no significant changes that would require much coordination. -- Jesse Keating Fedora -- All my bits are free, are yours?

5 7

[PATCH] set the current working directory in the chroot
by Mike Bonnet 24 Jan '08

24 Jan '08

This patch allows you to set the current working directory (in the chroot) before running a command with --chroot. This avoids the need to pass shell snippets ('cd /some/path && /run/cmd') to mock when running a command that expects to executed from a certain directory. It's useful when using --copyin to setup the environment before running a command. >From e4071d1d41a62ccf4461dfab958f9325edf30c97 Mon Sep 17 00:00:00 2001 From: Mike Bonnet <mikeb(a)redhat.com> Date: Thu, 24 Jan 2008 17:09:06 -0500 Subject: [PATCH] optionally set the current working directory (in the chroot) before running command with --chroot --- docs/mock.1 | 3 +++ py/mock.py | 8 ++++++-- py/mock/util.py | 13 +++++++++---- 3 files changed, 18 insertions(+), 6 deletions(-) diff --git a/docs/mock.1 b/docs/mock.1 index 38c3233..531d117 100644 --- a/docs/mock.1 +++ b/docs/mock.1 @@ -140,6 +140,9 @@ Fail build if rpmbuild takes longer than 'timeout' seconds \fB\-\-unpriv\fR Drop privileges before running command when using --chroot .TP +\fB\-\-cwd=\fR\fIDIR\fP +Change to the specified directory (relative to the chroot) before running command when using --chroot +.TP \fB\-q\fR, \fB\-\-quiet\fR Be quiet. .TP diff --git a/py/mock.py b/py/mock.py index f422a33..d5afbbe 100755 --- a/py/mock.py +++ b/py/mock.py @@ -152,6 +152,10 @@ def command_parse(config_opts): " seconds ") parser.add_option("--unpriv", action="store_true", default=False, help="Drop privileges before running command when using --chroot") + parser.add_option("--cwd", action="store", default=None, + metavar="DIR", + help="Change to the specified directory (relative to the chroot)" + " before running command when using --chroot") # verbosity parser.add_option("-v", "--verbose", action="store_const", const=2, @@ -536,9 +540,9 @@ def main(ret): chroot._mountall() if options.unpriv: chroot.doChroot(args, shell=shell, - uid=chroot.chrootuid, gid=chroot.chrootgid) + uid=chroot.chrootuid, gid=chroot.chrootgid, cwd=options.cwd) else: - chroot.doChroot(args, shell=shell) + chroot.doChroot(args, shell=shell, cwd=options.cwd) finally: chroot._umountall() diff --git a/py/mock/util.py b/py/mock/util.py index f93f98b..65cc995 100644 --- a/py/mock/util.py +++ b/py/mock/util.py @@ -201,6 +201,10 @@ def condChroot(chrootPath): os.chroot(chrootPath) uid.setresuid(saved['ruid'], saved['euid']) +def condChdir(cwd): + if cwd is not None: + os.chdir(cwd) + def condDropPrivs(uid, gid): if gid is not None: os.setregid(gid, gid) @@ -245,12 +249,12 @@ def logOutput(fds, logger, returnOutput=1, start=0, timeout=0): # The "Not-as-complicated" version # decorate(traceLog()) -def do(command, shell=False, chrootPath=None, timeout=0, raiseExc=True, returnOutput=0, uid=None, gid=None, personality=None, *args, **kargs): +def do(command, shell=False, chrootPath=None, cwd=None, timeout=0, raiseExc=True, returnOutput=0, uid=None, gid=None, personality=None, *args, **kargs): logger = kargs.get("logger", getLog()) output = "" start = time.time() - preexec = ChildPreExec(personality, chrootPath, uid, gid) + preexec = ChildPreExec(personality, chrootPath, cwd, uid, gid) try: child = None logger.debug("Executing command: %s" % command) @@ -292,9 +296,10 @@ def do(command, shell=False, chrootPath=None, timeout=0, raiseExc=True, returnOu return output class ChildPreExec(object): - def __init__(self, personality, chrootPath, uid, gid): + def __init__(self, personality, chrootPath, cwd, uid, gid): self.personality = personality self.chrootPath = chrootPath + self.cwd = cwd self.uid = uid self.gid = gid @@ -303,4 +308,4 @@ class ChildPreExec(object): condPersonality(self.personality) condChroot(self.chrootPath) condDropPrivs(self.uid, self.gid) - + condChdir(self.cwd) -- 1.5.3.3

2 1

[PATCH] add --unpriv option to drop privileges when running a command with --chroot
by Mike Bonnet 24 Jan '08

24 Jan '08

This patch adds a --unpriv option that will cause privileges to be dropped before running a command with --chroot. This can be used to more closely simulate the environment used when running rpmbuilds.

2 3

[PATCH] do mounts when running mock --chroot
by Mike Bonnet 22 Jan '08

22 Jan '08

Here's a small patch to handle mounting things into the chroot when using the --chroot command, the same way we do when using --rebuild.

2 1

highmem kernel build failed?
by Joey Baltrusch 19 Jan '08

19 Jan '08

I am a noob to Linux, but I've been able to follow the FC8 script at http://fedoraproject.org/wiki/Docs/CustomKernel?highlight=%28kernel%29 For the kernel config, I used the kernel-2.6.23.9-i686.config After modifying the highmem switch successfully using the script (it looks good in .config), I followed the script down to the rpmbuild command. I used the "Build all flavors" format, and got this response at the end. I assume I'm missing something from my Dev environment, it's the default from the FC8 i386 DVD. RPM Build Errorsbad exit status from /var/tmp/rmp-tmp.19630 I'm building linux on a asus P5K SE with 8Gig ram, and I'm going to use Blender, so I want to maximize my memory usage. Any suggestions are much appreciated! -- Want an e-mail address like mine? Get a free e-mail account today at www.mail.com!

2 1

plague: Job waited too long for repo to unlock. Killing it...
by Michael Schwendt 18 Jan '08

18 Jan '08

If in a failed job.log you see the message Job waited too long for repo to unlock. Killing it... please notify me. It's a problem in the plague server code that results in a denial of service for subsequent build jobs. I have a traceback from Dec 28th, but in the context of the source code it doesn't make sense yet (because a few lines earlier the code ensures that the files to be copied exist and are readable). Buildsys runs a slightly modified version that adds a bit more debug output in this area. [...] 37683 (grads/ppc): Build result files - [ 'state.log', 'grads-1.9b4-21.el5.src.rpm', 'grads-debuginfo-1.9b4-21.el5.ppc.rpm', 'grads-1.9b4-21.el5.ppc.rpm', 'root.log', 'build.log', 'job.log' ] 37683 (grads/x86_64): Build result files - [ 'root.log', 'build.log', 'grads-debuginfo-1.9b4-21.el5.x86_64.rpm', 'grads-1.9b4-21.el5.x86_64.rpm', 'grads-1.9b4-21.el5.src.rpm', 'state.log', 'job.log' ] 37683 (grads/i386): Build result files - [ 'build.log', 'state.log', 'grads-1.9b4-21.el5.i386.rpm', 'job.log', 'root.log', 'grads-debuginfo-1.9b4-21.el5.i386.rpm', 'grads-1.9b4-21.el5.src.rpm' ] Repo 'fedora-5-epel': updating repository metadata... Exception in thread Repo: fedora-5-epel: Traceback (most recent call last): File "/usr/lib64/python2.3/threading.py", line 436, in __bootstrap self.run() File "/usr/share/plague/server/Repo.py", line 233, in run self._update_repo() File "/usr/share/plague/server/Repo.py", line 137, in _update_repo shutil.copy(src, file_in_dst) File "/usr/lib64/python2.3/shutil.py", line 72, in copy copymode(src, dst) File "/usr/lib64/python2.3/shutil.py", line 49, in copymode st = os.stat(src) OSError: [Errno 2] No such file or directory: '/srv/rpmbuild/server_work/fedora-5-epel/37683-grads-1.9b4-21.el5/x86_64/grads-1.9b4-21.el5.x86_64.rpm' 37683 (grads): Job finished.

3 8

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

buildsys January 2008