It's been a while since I played with the koji buildsystem so since I've a spare box in the backroom doing nothing and I'm between projects, I thought I'd have a shot at getting a standalone system up and running using a clean minimal fc13 image.
koji-1.4.0-2.fc13.noarch postgresql-8.4.4-1.fc13.x86_64 httpd-2.2.16-1.fc13.x86_64 mock-1.1.3-1.fc13.noarch
I've a fairly large chain of scripts that automatically sets up the initial system, ie the users, the postgresql schema's (and upgrades), the ssl configs, certificate files, the config file changes, firewall changes, preloading the repo's and other tweaks so that I can easily trash the system and rebuild it from scratch without getting mired in the lengthy set-up procedure and that has worked fine for fc11/fc12
In fc13, when koji starts creating the FS structures for the chroot under /var/lib/mock, it will attempt to create a /var/cache/mock/<tag-X-Y>/filesystems file which it can't do since /var/cache/mock/<tag-X-Y> was never created. The cheap way out of that is to set config_opts['cache_topdir'] = '/var/lib/mock' instead of /var/cache/mock. That option didn't need fiddling with in fc12 so I'm unsure if this was a change in koji's code base or something unrelated.
The following is the product of being unable to email a notification because it can't connect to the mail host... I can think of dozens of reasons why this could occur but it may be better to stick in an exception to queue them up internally for retry or at least spit out 'Hey! I can't talk to your mailhost!" and fail the BuildNotify task
Traceback (most recent call last): File "/usr/sbin/kojid", line 1437, in runTask response = (handler.run(),) File "/usr/sbin/kojid", line 1513, in run return self.handler(*self.params,**self.opts) File "/usr/sbin/kojid", line 3651, in handler *server = smtplib.SMTP(options.smtphost)* File "/usr/lib64/python2.6/smtplib.py", line 239, in __init__ (code, msg) = self.connect(host, port) File "/usr/lib64/python2.6/smtplib.py", line 295, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib64/python2.6/smtplib.py", line 273, in _get_socket return socket.create_connection((port, host), timeout) File "/usr/lib64/python2.6/socket.py", line 514, in create_connection raise error, msg error: [Errno 111] Connection refused
The main problem I keep hitting, is mainly with noarch packages. BuildError: error building package (arch noarch), mock exited with status 1; see build.log for more information however when I go to look all I see in the log is the single line Mock Version: 1.1.3 The root.log doesn't show anything interesting either
however if I manually poke the rpm through with mock, it works. ie [root@whitebishop ~]# mock -r koji/el5-ga-build-9-44 --rebuild ~/buildsys-macros-5-2.ovs.src.rpm
[root@whitebishop ~]# ls -l /var/lib/mock/el5-ga-build-9-44/result total 48 -rw-rw-r--. 1 root mock 2593 Aug 19 11:57 build.log -rw-r--r--. 1 root mock 2283 Aug 19 11:57 buildsys-macros-5-2.ovs.noarch.rpm -rw-r--r--. 1 root mock 2147 Aug 19 11:57 buildsys-macros-5-2.ovs.src.rpm -rw-rw-r--. 1 root mock 32152 Aug 19 11:57 root.log -rw-rw-r--. 1 root mock 193 Aug 19 11:57 state.log
[root@whitebishop ~]# cat /var/lib/mock/el5-ga-build-9-44/result/build.log Mock Version: 1.1.3 ENTER do(['bash', '--login', '-c', 'rpmbuild -bs --target i386 --nodeps builddir/build/SPECS/buildsys-macros.spec'], False, '/var/lib/mock/el5-ga-build-9-44/root/', None, 86400, True, 0, 0, 484, None, logger=<mock.trace_decorator.getLog object at 0x1f2abd0>) Executing command: ['bash', '--login', '-c', 'rpmbuild -bs --target i386 --nodeps builddir/build/SPECS/buildsys-macros.spec'] warning: Could not canonicalize hostname: whitebishop Building target platforms: i386 Building for target i386 Wrote: /builddir/build/SRPMS/buildsys-macros-5-2.ovs.src.rpm Child returncode was: 0 LEAVE do -->
I can't reason out why there would be any difference unless it's passing --arch noarch.. in which case [root@whitebishop ~]# mock -r koji/el5-ga-build-9-44 --arch noarch --rebuild ~/buildsys-macros-5-2.ovs.src.rpm ERROR: Cannot build target noarch on arch x86_64 But then I'd expect to have the error line in the build.log which I don't... so I'm a bit flummoxed if this is what has happened or not.
Ideas?
Phil =--=
On 08/19/2010 07:05 AM, Bryce wrote:
In fc13, when koji starts creating the FS structures for the chroot under /var/lib/mock, it will attempt to create a /var/cache/mock/<tag-X-Y>/filesystems file which it can't do since /var/cache/mock/<tag-X-Y> was never created. The cheap way out of that is to set config_opts['cache_topdir'] = '/var/lib/mock' instead of /var/cache/mock. That option didn't need fiddling with in fc12 so I'm unsure if this was a change in koji's code base or something unrelated.
I've never encountered this problem before. Fedora is running mock-1.1.1, is using no such workaround, and does not seem to see this issue. I don't see anything in the diff that would explain your issue, though I guess it wouldn't hurt to try 1.1.1 and see if that changes anything.
Have you by chance set something site-defaults.cfg? If you have, that might might be tickling a bug in mock that I'm not seeing.
The following is the product of being unable to email a notification because it can't connect to the mail host... I can think of dozens of reasons why this could occur but it may be better to stick in an exception to queue them up internally for retry or at least spit out 'Hey! I can't talk to your mailhost!" and fail the BuildNotify task
Traceback (most recent call last): File "/usr/sbin/kojid", line 1437, in runTask response = (handler.run(),) File "/usr/sbin/kojid", line 1513, in run return self.handler(*self.params,**self.opts) File "/usr/sbin/kojid", line 3651, in handler *server = smtplib.SMTP(options.smtphost)* File "/usr/lib64/python2.6/smtplib.py", line 239, in __init__ (code, msg) = self.connect(host, port) File "/usr/lib64/python2.6/smtplib.py", line 295, in connect self.sock = self._get_socket(host, port, self.timeout) File "/usr/lib64/python2.6/smtplib.py", line 273, in _get_socket return socket.create_connection((port, host), timeout) File "/usr/lib64/python2.6/socket.py", line 514, in create_connection raise error, msg error: [Errno 111] Connection refused
Yep, this should be handled better
The main problem I keep hitting, is mainly with noarch packages. BuildError: error building package (arch noarch), mock exited with status 1; see build.log for more information however when I go to look all I see in the log is the single line Mock Version: 1.1.3 The root.log doesn't show anything interesting either
however if I manually poke the rpm through with mock, it works. ie [root@whitebishop ~]# mock -r koji/el5-ga-build-9-44 --rebuild ~/buildsys-macros-5-2.ovs.src.rpm
[root@whitebishop ~]# ls -l /var/lib/mock/el5-ga-build-9-44/result total 48 -rw-rw-r--. 1 root mock 2593 Aug 19 11:57 build.log -rw-r--r--. 1 root mock 2283 Aug 19 11:57 buildsys-macros-5-2.ovs.noarch.rpm -rw-r--r--. 1 root mock 2147 Aug 19 11:57 buildsys-macros-5-2.ovs.src.rpm -rw-rw-r--. 1 root mock 32152 Aug 19 11:57 root.log -rw-rw-r--. 1 root mock 193 Aug 19 11:57 state.log
[root@whitebishop ~]# cat /var/lib/mock/el5-ga-build-9-44/result/build.log Mock Version: 1.1.3 ENTER do(['bash', '--login', '-c', 'rpmbuild -bs --target i386 --nodeps builddir/build/SPECS/buildsys-macros.spec'], False, '/var/lib/mock/el5-ga-build-9-44/root/', None, 86400, True, 0, 0, 484, None, logger=<mock.trace_decorator.getLog object at 0x1f2abd0>) Executing command: ['bash', '--login', '-c', 'rpmbuild -bs --target i386 --nodeps builddir/build/SPECS/buildsys-macros.spec'] warning: Could not canonicalize hostname: whitebishop Building target platforms: i386 Building for target i386 Wrote: /builddir/build/SRPMS/buildsys-macros-5-2.ovs.src.rpm Child returncode was: 0 LEAVE do -->
I can't reason out why there would be any difference unless it's passing --arch noarch.. in which case [root@whitebishop ~]# mock -r koji/el5-ga-build-9-44 --arch noarch --rebuild ~/buildsys-macros-5-2.ovs.src.rpm ERROR: Cannot build target noarch on arch x86_64 But then I'd expect to have the error line in the build.log which I don't... so I'm a bit flummoxed if this is what has happened or not.
koji determines the arch(s) primarily from the initial srpm it creates. It reads the buildarchs, exclusivearch, and excludearch values from that srpm and attempts to honor them. However this is for determining which buildArch tasks to fire off, not for passing to mock.
When kojid handles a noarch buildArch task, it picks a valid actual arch to generate the buildroot, since there is no such thing as a "noarch" buildroot. This is the arch that is passed to mock (via the custom mock config file that kojid writes, not the command line). I don't believe koji will ever pass a --arch arg to mock.
Is it possible that you're simply missing "BuildArch: noarch" in your spec?
On 23/08/10 20:06, Mike McLean wrote:
On 08/19/2010 07:05 AM, Bryce wrote:
In fc13, when koji starts creating the FS structures for the chroot under /var/lib/mock, it will attempt to create a /var/cache/mock/<tag-X-Y>/filesystems file which it can't do since /var/cache/mock/<tag-X-Y> was never created. The cheap way out of that is to set config_opts['cache_topdir'] = '/var/lib/mock' instead of /var/cache/mock. That option didn't need fiddling with in fc12 so I'm unsure if this was a change in koji's code base or something unrelated.
I've never encountered this problem before. Fedora is running mock-1.1.1, is using no such workaround, and does not seem to see this issue. I don't see anything in the diff that would explain your issue, though I guess it wouldn't hurt to try 1.1.1 and see if that changes anything.
Have you by chance set something site-defaults.cfg? If you have, that might might be tickling a bug in mock that I'm not seeing.
Okies let me try it all again from scratch (this is going to take quite a while on this box so apologies for the very late reply and bad humour <> brought on by the long waits between stages) . Installing a minimal *fc13* x86_64 ISO in a vm.... <much disk chundering> yum update fc13.. <at the mercy of the UK BT NOTwork,...> allocate a virtual disk for to hold the koji system <even more disk IO...> copy over the src and i386/x86_64 rpms directories to be loaded. <the disk controller wants a pay rise..> make copy of the 'clean' fc13 image so I don't have to go through all that hassle again 8/ <more disk IO ... Zzz.> setup koji (users/postgres, httpd, ssl, koji* conf files, directories & permissions, ...) [root@localhost buildsys]# rpm -q mock koji httpd postgresql mock-1.1.4-1.fc13.noarch koji-1.4.0-2.fc13.noarch httpd-2.2.16-1.fc13.x86_64 postgresql-8.4.4-1.fc13.x86_64
Just check that mock's site defaults aren't modified from it's default [root@koji ~]# rpm -qfV /etc/mock/site-defaults.cfg [root@koji ~]#
check that koji it does what it says on the tin, (ie run simple koji commands successfully, poke at web interface etc) <looks fine> . import the src and i386/x86_64 rpms <I'm going to die of old age!,... > make another snaphot backup of the vm <I enjoy the pain, really> Run threading script to create all the interdependencies between tags <I'm so glad that's all in a script (140 odd lines)> . Finally build src.rpm that creates i386/x86_64 binary <*FAIL*>
<investigate> :
The build/root and state logs are basically empty. Nothing of use can be gleaned from them.
[root@builder ~]# cd /etc/mock/koji [root@builder koji]# ls el5-ga-build-1-6.cfg
Hurm,.. ok then lets run the init phase manually.
[root@builder koji]# mock -r koji/el5-ga-build-1-6 init INFO: mock.py version 1.1.4 starting... State Changed: init plugins INFO: selinux enabled State Changed: start State Changed: lock buildroot State Changed: clean State Changed: init State Changed: lock buildroot Mock Version: 1.1.4 INFO: Mock Version: 1.1.4 Traceback (most recent call last): File "/usr/sbin/mock", line 748, in <module> main(retParams) File "/usr/sbin/mock", line 616, in main chroot.init() File "<peak.util.decorators.rewrap wrapping mock.backend.init at 0x01EF58C0>", line 3, in init File "/usr/lib/python2.6/site-packages/mock/trace_decorator.py", line 70, in trace result = func(*args, **kw) File "/usr/lib/python2.6/site-packages/mock/backend.py", line 195, in init self._callHooks('initfailed') File "<peak.util.decorators.rewrap wrapping mock.backend._callHooks at 0x01F51C08>", line 3, in _callHooks File "/usr/lib/python2.6/site-packages/mock/trace_decorator.py", line 70, in trace result = func(*args, **kw) File "/usr/lib/python2.6/site-packages/mock/backend.py", line 609, in _callHooks hook() File "<peak.util.decorators.rewrap wrapping selinux._selinuxPostBuildHook at 0x01F64758>", line 3, in _selinuxPostBuildHook File "/usr/lib/python2.6/site-packages/mock/trace_decorator.py", line 70, in trace result = func(*args, **kw) File "/usr/lib/python2.6/site-packages/mock/plugins/selinux.py", line 67, in _selinuxPostBuildHook os.unlink(self.filesystems) OSError: [Errno 2] No such file or directory: '*/var/cache/mock*/el5-ga-build-1-6/filesystems' [root@builder koji]# ls -l /var/cache/mock total 0
Right ho,.. it's DEFINITELY looking for files in /var/cache/mock. The only place this is referenced is in /etc/mock/site-defaults.cfg as being the DEFAULT value so it's not specifically set but mock is filling in the value anyway (note it's #ed out in the cfg file)
[root@builder koji]# grep /var/cache/mock /etc/mock/site-defaults.cfg *#* config_opts['cache_topdir'] = '/var/cache/mock'
rpm claims the site-defaults.cfg file is in the default downloaded state, so not modified. As stated before, if I quickly edit this variable and point it at /var/lib/mock instead I can get a successful build. Maybe this is something that mock-1.1.1 never filled in the blank for.
The main problem I keep hitting, is mainly with noarch packages. BuildError: error building package (arch noarch), mock exited with status 1; see build.log for more information however when I go to look all I see in the log is the single line Mock Version: 1.1.3 The root.log doesn't show anything interesting either
however if I manually poke the rpm through with mock, it works. ie [root@whitebishop ~]# mock -r koji/el5-ga-build-9-44 --rebuild ~/buildsys-macros-5-2.ovs.src.rpm
[root@whitebishop ~]# ls -l /var/lib/mock/el5-ga-build-9-44/result total 48 -rw-rw-r--. 1 root mock 2593 Aug 19 11:57 build.log -rw-r--r--. 1 root mock 2283 Aug 19 11:57 buildsys-macros-5-2.ovs.noarch.rpm -rw-r--r--. 1 root mock 2147 Aug 19 11:57 buildsys-macros-5-2.ovs.src.rpm -rw-rw-r--. 1 root mock 32152 Aug 19 11:57 root.log -rw-rw-r--. 1 root mock 193 Aug 19 11:57 state.log
[root@whitebishop ~]# cat /var/lib/mock/el5-ga-build-9-44/result/build.log Mock Version: 1.1.3 ENTER do(['bash', '--login', '-c', 'rpmbuild -bs --target i386 --nodeps builddir/build/SPECS/buildsys-macros.spec'], False, '/var/lib/mock/el5-ga-build-9-44/root/', None, 86400, True, 0, 0, 484, None, logger=<mock.trace_decorator.getLog object at 0x1f2abd0>) Executing command: ['bash', '--login', '-c', 'rpmbuild -bs --target i386 --nodeps builddir/build/SPECS/buildsys-macros.spec'] warning: Could not canonicalize hostname: whitebishop Building target platforms: i386 Building for target i386 Wrote: /builddir/build/SRPMS/buildsys-macros-5-2.ovs.src.rpm Child returncode was: 0 LEAVE do -->
I can't reason out why there would be any difference unless it's passing --arch noarch.. in which case [root@whitebishop ~]# mock -r koji/el5-ga-build-9-44 --arch noarch --rebuild ~/buildsys-macros-5-2.ovs.src.rpm ERROR: Cannot build target noarch on arch x86_64 But then I'd expect to have the error line in the build.log which I don't... so I'm a bit flummoxed if this is what has happened or not.
koji determines the arch(s) primarily from the initial srpm it creates. It reads the buildarchs, exclusivearch, and excludearch values from that srpm and attempts to honor them. However this is for determining which buildArch tasks to fire off, not for passing to mock.
When kojid handles a noarch buildArch task, it picks a valid actual arch to generate the buildroot, since there is no such thing as a "noarch" buildroot. This is the arch that is passed to mock (via the custom mock config file that kojid writes, not the command line). I don't believe koji will ever pass a --arch arg to mock.
Is it possible that you're simply missing "BuildArch: noarch" in your spec?
actually...
Buildarch: noarch
is rpm so mean as to be picky about BuildArch: vs Buildarch: ?? I did change it to be studdlycapped but no difference. I'll dig into this deeper tomorrow (real work getting piled up..)
Phil =--=
Mike McLean wrote:
I've never encountered this problem before. Fedora is running mock-1.1.1, is using no such workaround, and does not seem to see this issue. I don't see anything in the diff that would explain your issue, though I guess it wouldn't hurt to try 1.1.1 and see if that changes anything.
I am seeing something similar under F12. After updating from mock-1.0.7 to mock-1.0.11 I could no longer build packages. If I manually ran mock, I would see a similar error about not being able to create directories in /var/cache/mock.
Have you by chance set something site-defaults.cfg? If you have, that might might be tickling a bug in mock that I'm not seeing.
I have nothing custom on my build server. I had to revert to mock-1.0.7. I can build packages again. I diff'd the two site-defaults.cfg between 1.0.7 and 1.0.11 and the only difference were comments.
buildsys@lists.fedoraproject.org