[BUG] Koji and Lustre
by Thomas Guthmann
Hey guys,
We found a bug(?) in listTaskOutput (/usr/share/koji-hub/kojihub.py)
when used with a Lustre filesystem. This function parses all attributes
of every file of a build and is used when you want to display
build.log/root.log through the web interface. Everything returned by
listTaskOutput is returned through XML-RPC and as a result we had this :
An error has occurred while processing your request.
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
Traceback (most recent call last):
File "/usr/share/koji-web/lib/kojiweb/publisher.py", line 16, in
publish_object
return old_publish_object(req, object)
File "/usr/lib64/python2.4/site-packages/mod_python/publisher.py",
line 412, in publish_object
return publish_object(req,util.apply_fs_data(object, req.form,
req=req))
File "/usr/lib64/python2.4/site-packages/mod_python/util.py", line
439, in apply_fs_data
return object(**args)
File "/usr/share/koji-web/scripts/index.py", line 649, in getfile
output = server.listTaskOutput(taskID, stat=True)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1468,
in __call__
return self.__func(self.__name,args,opts)
File "/usr/lib/python2.4/site-packages/koji/__init__.py", line 1718,
in _callMethod
raise err
Fault: <Fault 1: 'exceptions.OverflowError: long int exceeds XML-RPC
limits'>
The issue comes from the st_dev value gathered by getStat (stat). In
Lustre this value can be very high and that's why it complains. To fix
that, we used the same condition than st_size, we cast the value as a
string. See attached patch.
Example (see 'Device:' the value after the '/'):
# stat build.log
File: `build.log'
Size: 106021 Blocks: 208 IO Block: 2097152 regular file
Device: e04ae70eh/3763005198d Inode: 721664 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 48/ apache) Gid: ( 48/ apache)
Access: 2010-07-13 10:44:40.000000000 +1000
Modify: 2010-07-12 12:54:15.000000000 +1000
Change: 2010-07-12 12:54:52.000000000 +1000
# That's what is read by listTaskOutput
relfilename=build.log ATTR=st_atime GETATTR=1278981616
relfilename=build.log ATTR=st_blksize GETATTR=2097152
relfilename=build.log ATTR=st_blocks GETATTR=208
relfilename=build.log ATTR=st_ctime GETATTR=1278903292
relfilename=build.log ATTR=st_dev GETATTR=3763005198 <----- /!\
relfilename=build.log ATTR=st_gid GETATTR=48
relfilename=build.log ATTR=st_ino GETATTR=721664
relfilename=build.log ATTR=st_mode GETATTR=33188
relfilename=build.log ATTR=st_mtime GETATTR=1278903255
relfilename=build.log ATTR=st_nlink GETATTR=1
relfilename=build.log ATTR=st_rdev GETATTR=0
relfilename=build.log ATTR=st_size GETATTR=106021
relfilename=build.log ATTR=st_uid GETATTR=48
Hope it helps, lost a good amount of time on that one :)
Cheers,
Thomas
9 years, 4 months
Strange mock build failure due to typo
by Paul Howarth
Hi,
today I have been preparing an update to perl-Math-Pari and came across
a very strange build failure whilst doing a local mock build on a Fedora
13 x86_64 host. My package built successfully on x86_64 but when I tried
to build for i386, the build failed during %setup but without any
diagnostics. The SRPM was installed but no attempt to install its build
requirements was made. The root.log showed an exit status of 0 for all
commands that had been run.
After much experimentation bisecting the changes I had made, I
discovered that a typo in the changelog entry was the culprit: I had set
the year to 2100 instead of 2010. So it would appear that somewhere in
the mock/yum/rpm stack there may be a year 2038 problem waiting to bite
us (though I suspect there may not be too many 32-bit builds happening
by then).
Seriously though, it would be nice to have better diagnostics for this
and perhaps an rpmlint check for changelog entries in the future?
Paul.
9 years, 12 months
[PATCH 1/3] Install build deps with yum-builddep.
by Ville Skyttä
No longer need to screen-scrape resolvedep and feed that to yum
install, and we have a chance to get BuildConflicts handing "for free"
(when RHBZ #614191 is done in yum(-builddep)).
---
mock.spec.in | 2 +-
py/mock/backend.py | 33 ++++++++++++++++++++++-----------
2 files changed, 23 insertions(+), 12 deletions(-)
diff --git a/mock.spec.in b/mock.spec.in
index e2580ca..0949975 100644
--- a/mock.spec.in
+++ b/mock.spec.in
@@ -18,7 +18,7 @@ Source: https://fedorahosted.org/mock/attachment/wiki/MockTarballs/%{name}-%{ver
URL: http://fedoraproject.org/wiki/Projects/Mock
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
BuildArch: noarch
-Requires: python >= 2.4, yum >= 2.4, tar, pigz, python-ctypes, python-decoratortools, usermode
+Requires: python >= 2.4, yum >= 2.4, yum-utils >= 1.1.9, tar, pigz, python-ctypes, python-decoratortools, usermode
Requires: createrepo
Requires(pre): shadow-utils
BuildRequires: python-devel
diff --git a/py/mock/backend.py b/py/mock/backend.py
index ba0ad14..81a98d0 100644
--- a/py/mock/backend.py
+++ b/py/mock/backend.py
@@ -69,6 +69,7 @@ class Root(object):
self.chroot_file_contents = config['files']
self.chroot_setup_cmd = config['chroot_setup_cmd']
self.yum_path = '/usr/bin/yum'
+ self.yum_builddep_path = '/usr/bin/yum-builddep'
self.macros = config['macros']
self.more_buildreqs = config['more_buildreqs']
self.cache_topdir = config['cache_topdir']
@@ -444,23 +445,28 @@ class Root(object):
"""figure out deps from srpm. call yum to install them"""
try:
self.uidManager.becomeUser(0, 0)
+
+ def _yum_and_check(cmd):
+ output = self._yum(cmd, returnOutput=1)
+ for line in output.split('\n'):
+ if line.lower().find('No Package found for'.lower()) != -1:
+ raise mock.exception.BuildError, "Bad build req: %s. Exiting." % line
+
+ # first, install pre-existing deps and configured additional ones
arg_string = self.preExistingDeps
for hdr in mock.util.yieldSrpmHeaders(srpms, plainRpmOk=1):
# get text buildreqs
- a = mock.util.requiresTextFromHdr(hdr)
- b = mock.util.getAddtlReqs(hdr, self.more_buildreqs)
- for item in mock.util.uniqReqs(a, b):
+ for item in mock.util.getAddtlReqs(hdr, self.more_buildreqs):
arg_string = arg_string + " '%s'" % item
-
- # everything exists, okay, install them all.
- # pass build reqs (as strings) to installer
if arg_string != "":
- output = self._yum('resolvedep %s' % arg_string, returnOutput=1)
- for line in output.split('\n'):
- if line.lower().find('No Package found for'.lower()) != -1:
- raise mock.exception.BuildError, "Bad build req: %s. Exiting." % line
+ # everything exists, okay, install them all.
+ # pass build reqs (as strings) to installer
+ _yum_and_check('resolvedep %s' % arg_string)
# nothing made us exit, so we continue
self._yum('install %s' % arg_string, returnOutput=1)
+
+ # install actual build dependencies
+ _yum_and_check("builddep '%s'" % "' '".join(srpms))
finally:
self.uidManager.restorePrivs()
@@ -676,7 +682,12 @@ class Root(object):
if not self.online:
cmdOpts = "-C"
- cmd = '%s --installroot %s %s %s' % (self.yum_path, self.makeChrootPath(), cmdOpts, cmd)
+ # invoke yum-builddep instead of yum if cmd is builddep
+ exepath = self.yum_path
+ if cmd.startswith("builddep "):
+ exepath = self.yum_builddep_path
+ cmd = cmd[len("builddep "):]
+ cmd = '%s --installroot %s %s %s' % (exepath, self.makeChrootPath(), cmdOpts, cmd)
self.root_log.debug(cmd)
output = ""
try:
--
1.7.2.3
10 years, 1 month
Koji policies
by ed_pollard@us.ibm.com
I am attempting to setup koji in a test environment and would like to
allow users to add packages to tags without having admin rights. I found
http://fedoraproject.org/wiki/Koji/Policies
but have been unable to configure a policy for my purposes. Any help would
be greatly appreciated.
------
Edward Pollard
ed_pollard(a)us.ibm.com
10 years, 2 months
Mock alternating /etc/php.ini file
by David Hrbáč
Hi,
I have the following scenario. I'm trying to build some php-pear package
with Mock for CentOS 4.x. PHP settings within php.ini has memory_limit =
8MB. This memory limit is too low to build the package. I don't want to
change php package itself. I have tried:
a) package with sed to replace memory setting - fails with access denied
on /etc/php.ini
b) package containing own /etc/php.ini - conflicts with php-common
Any ideas? Thanks,
David Hrbáč
10 years, 3 months
cannot open Providename index using db3 - Invalid argument (22)
by Vijay N. Majagaonkar
Hi,
I am running out of strange problem at the time of creating ISO, ISO created
successfully but it failed at the time of OS installation and this what i
found in root.log file
I am not able to under stand the what is the problem "cannot open
Providename index using db3 - Invalid argument (22)"
can someone please give me some point or direction to fix this,
[Log]
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Building
isolinux directory
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: mkdosfs 2.11 (12
Mar 2005)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: rpmdb:
/var/lib/rpm/Providename: unsupported hash version: 9
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error: cannot
open Providename index using db3 - Invalid argument (22)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: rpmdb:
/var/lib/rpm/Providename: unsupported hash version: 9
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error: cannot
open Providename index using db3 - Invalid argument (22)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
http://koji.fedora.com/packages/kernel/2.6.18/164.el5/x86_64/kernel-xen-2...:
[Errno 12] Timeout: <urlopen error timed out>
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Trying other
mirror.
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Traceback (most
recent call last):
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 293, in ?
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: util =
YumDownloader()
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 42, in __init__
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: self.main()
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 80, in main
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
self.downloadPackages(opts)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/bin/yumdownloader", line 214, in downloadPackages
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: path =
repo.getPackage(download)
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 849, in getPackage
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: cache=cache
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: File
"/usr/lib/python2.4/site-packages/yum/yumRepo.py", line 811, in _getFile
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: raise
Errors.RepoError, errstr
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall:
yum.Errors.RepoError: failed to retrieve
kernel/2.6.18/164.el5/x86_64/kernel-xen-2.6.18-164.el5.x86_64.rpm from
anaconda-extrarepo-1
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: error was [Errno
12] Timeout: <urlopen error timed out>
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: kernel
(kernel-xen) doesn't exist for x86_64. skipping
DEBUG util.py:256: /usr/lib/anaconda-runtime/buildinstall: Building
minstg.img
[/Log]
Thanks for the help
V!jay
10 years, 3 months
Mock Error
by Narendra Sisodiya
creating 32 bit iso file on 64 bit machine using pungi and mock. but
following error I am getting.
[root@narendra mock]# mock -r fedora-13-i386 --clean
INFO: mock.py version 1.1.5 starting...
State Changed: init plugins
INFO: selinux enabled
State Changed: start
State Changed: lock buildroot
State Changed: clean
[root@narendra mock]# mock -r fedora-13-i386 --init
INFO: mock.py version 1.1.5 starting...
State Changed: init plugins
INFO: selinux enabled
State Changed: start
State Changed: lock buildroot
State Changed: clean
State Changed: init
State Changed: lock buildroot
Mock Version: 1.1.5
INFO: Mock Version: 1.1.5
INFO: enabled root cache
State Changed: unpacking root cache
INFO: enabled yum cache
State Changed: cleaning yum metadata
INFO: enabled ccache
State Changed: running yum
ERROR: Command failed. See logs for output.
# ['/usr/sbin/groupadd', '-g', '478', 'mockbuild']
[root@narendra mock]#
--
┌─────────────────────────┐
│ Narendra Sisodiya
│ http://narendrasisodiya.com
└─────────────────────────┘
10 years, 3 months
Pungi Error
by Narendra Sisodiya
I am running fedora 13 64 bit system
I am trying to run pungi.
$pungi -c test.ks
I am getting this error
/usr/lib/python2.6/site-packages/pykickstart/parser.py:673:
DeprecationWarning: %packages does not end with %end. This syntax has been
deprecated. It may be removed from future releases, which will result in a
fatal error from kickstart. Please modify your kickstart file to use this
updated syntax.
warnings.warn(_("%s does not end with %%end. This syntax has been
deprecated. It may be removed from future releases, which will result in a
fatal error from kickstart. Please modify your kickstart file to use this
updated syntax.") % "%packages", DeprecationWarning)
Warning: Reusing existing destination directory.
Traceback (most recent call last):
File "/usr/bin/pungi", line 216, in <module>
main()
File "/usr/bin/pungi", line 86, in main
mypungi._inityum() # initialize the yum object for things that need it
File "/usr/lib/python2.6/site-packages/pypungi/__init__.py", line 166, in
_inityum
del self.ayum.prerepoconf
AttributeError: prerepoconf
--
┌─────────────────────────┐
│ Narendra Sisodiya
│ http://narendrasisodiya.com
└─────────────────────────┘
10 years, 3 months
Fwd: Creating Install media of fedora.
by Narendra Sisodiya
What are the tools available to install media. I want to create custom
fedora 13 install DVD (not live). Also, which fedora version or ubuntu
version you recommend to do this task,
--
┌─────────────────────────┐
│ Narendra Sisodiya
│ http://narendrasisodiya.com
└─────────────────────────┘
10 years, 3 months