Extras build system back up
by Dan Williams
Hi,
We're back up and using the latest plague code. yum update your
plague-client and you should be good to go.
Dan
16 years, 7 months
adding packages to mock chroot
by David Smith
When you create a mock chroot, all the packages you want to install in
it need to be available from a yum repostory. I've got the situation
where I'm trying to bootstrap a bit, so, I'd like to be able to:
- build a chroot
- build a small set of packages
- add those packages to the chroot to be used by future builds
Off the top of my head, possibilities include:
(1) creating a base chroot, building the rpms, add the new packages to a
yum repo, creating a second chroot that contains the new packages
(2) creating the base chroot, building the rpms, editing the
buildroots.xml file and adding in the new packages, updating the yum
repo, then run "mock init" again (hoping that the new packages would get
added to the chroot)
Are there better ways of doing this?
Thanks.
--
David Smith
dsmith(a)redhat.com
Red Hat, Inc.
http://www.redhat.com
256.217.0141 (direct)
256.837.0057 (fax)
17 years, 3 months
bugzilla #164441 (mock-helper and basedir)
by Clark Williams
Hi all,
I've been looking at how to change mock so that the build chroot's can
be placed at arbitrary locations (as opposed to the current
hardcoded /var/lib/mock). The main reason I want to do this is that I
build target root filesystems for embedded targets in our compile farm
and we use a big NFS filesystem as our main storage. I can access the
build directory and our cross-toolchains from any system in our farm and
nothing that's specific to a build is on any farm system (I can
kickstart it at will and not lose anything). Yes, we've successfully
redirected /var/lib/mock to the NFS storage with a mount --bind but I
just find that kinda scary. I'm just not very confident in remounting an
NFS filesystem to a local filesystem. The bottom line is that I'd like
to remove dependencies on the build system's local filesystem and move
it all to the NFS storage.
So, the problem seems to be how to modify mock-helper so that it allows
the move, yet doesn't become a handy-dandy-attack-point for some cracker
wannabe.
Can we say that the following is true?
"The mock-helper program always begins execution *outside* of a chroot
and is the only mechanism mock uses to *enter* a chroot."
If so, then we can check to insure that the intended chroot directory:
1. Is not '/'
2. Is not a symlink to '/'
3. Is not in a list of special directories (/bin, /sbin, /lib, etc.)
Are we "safe" if we insure that whatever command is executed by
mock-helper is executed in a chroot directory that does not contain a
system critical component?
Clark
--
Clark Williams <williams(a)redhat.com>
17 years, 3 months
kmod arch, ExlusiveArch, and buildsys?
by Warren Togami
Hi Dan,
I am supposed to ask you about your opinion about how to ask buildsys to
build for specific archs for a kernel module. I believe that we should
use ExclusiveArch with explicit listed archs, and buildsys should loop
through each listed arch and build each. This would also work for
multi-arch performance intensive userspace packages similar to what we
do in glibc or openssl in Core. [1] Contributors may also list any
other arch like s390x or alpha in ExclusiveArch, and buildsys simply
ignores it if it isn't supported in Extras.
I believe this is the best approach to take for Extras, because Core has
been similar to this for many years. Do you have any opinion on this,
or know of a better way to handle this?
Here is some related discussion:
<scop> the problem with exclusivearch is that let's say you list i586
i686 x86_64 ppc and someone wants to rebuild it for alpha -> won't
build, even if the package/software would work
<warren> scop, you can include other archs in the list, buildsys can
simply ignore Extras non-supported archs.
<scop> warren, it's abuse of the tag nevertheless
<warren> scop, I'd personally disagree, but either way this is very
simple and it already behaves similar to this in Core literally forever.
<warren> more importantly the simplicity of this allows us to move
forward sooner
<warren> (assuming dcbw doesn't have a better idea)
<scop> that I agree with
Warren Togami
wtogami(a)redhat.com
[1]
However we should disallow this, unless someone proves through
benchmarks that there is a huge benefit of publishing both i386 and i686
packages and our tools support selection of the right arch directly.
Currently I believe it doesn't.
17 years, 3 months
Re: Sugg: Improve for mock (caching)
by Enrico Scholz
skvidal(a)phy.duke.edu (seth vidal) writes:
>> The speedup gained by caching is not (primarily) related to the
>> network connection but to the rpm installation and disk-IO. On the
>> fedora.us server we used caching also and the initial buildroot was
>> created in <1min (afair).
>>
>> Local tests at that time showed, that the initial buildroot can be
>> created in 20 seconds when a tmpfs will be used instead of a real disk
>> with ext3.
>
> So the numbers I'm curious about that you may have:
> - remote repos - but lan-like connection times
> - local-disk repos - chroot creation - all on ext3
> - tmpfs targets for the chroot creation and repos
>
> do you have anything like that?
ok, a little bit late, but I implemented caching now so I can provide
some real numbers:
Traditional 'mock init' on a tmpfs takes
| Init buildroot...
|
| real 1m37.445s
| user 1m15.329s
| sys 0m8.009s
tar'ing this filesystem, and extracting it takes
| Extracting cached buildroot...
|
| real 0m1.266s
| user 0m0.188s
| sys 0m0.944s
Both operations are resulting into the same root-fs; the executed code
was:
| mock=( env http_proxy= mock --resultdir="$results" -r "$MOCK_ROOT" )
| mock_yum=( env http_proxy= /usr/sbin/mock-helper yum --installroot "$BUILD_DIR" )
|
| function initBuildroot() {
| "${mock[@]}" clean >/dev/null
|
| if test -e "$CACHE_NAME"; then
| echo "Extracting cached buildroot..."
| time /usr/sbin/mock-helper uncache "$MOCK_ROOT"
|
| echo "Updating metadata..."
| "${mock_yum[@]}" check-update || :
| else
| lock
| echo "Init buildroot..."
| time "${mock[@]}" init >/dev/null
| unlock
|
| echo "Cleaning up root..."
| "${mock_yum[@]}" clean packages headers
| echo "Creating cache..."
| /usr/sbin/mock-helper cache "$MOCK_ROOT"
| fi
| }
The 'cache' and 'uncache' operations of 'mock-helper' were added by me
and are simple 'tar' wrappers. The resulting tarball is uncompressed and
280MB sized, both the rootfs and the tarball are located in the tmpfs.
Environment:
mock-machine: Intel Celeron 2.80GHz, 1500MB RAM
repository-server: PII 266MHz, 196MB RAM, http-access to repo (Apache2)
100Mb/s LAN; 'wget' download rate between repo and mock machine appr. 7-8MB/s
Enrico
17 years, 3 months
Kernel-Modules in the buildsys
by Thorsten Leemhuis
Hi all!
The kernel-module discussion on fedora-extras list was quite successful
and we have a kernel-module scheme now where nobody complained loudly
(at least until now). Two examples that are based on the latest results
from the discussion are found in extras-cvs in the packages lirc-kmod
and thinkpad-kmod.
What we don't have yet: Support for the that kernel-module scheme in the
buildsystem. What's needed? Roughly this:
- we need to get the kernel-verrel to the spec-file/rpmbuild somehow
- we need to get the kernel-variants to the spec-file/rpmbuild somehow
Example: To rebuild lirc-kmod for the kernel 2.6.15-1.1831_FC4 and
2.6.15-1.1831_FC4smp on i686 one would run this command with the current
scheme:
$ rpmbuild --rebuild lirc-kmod-0.8.0-2.2.6.15_1.1831_FC4.src.rpm
--define 'kver 2.6.15-1.1831_FC4' --define 'kvariants "" smp ' --target
i686
So guys, how to we teach that the buildsystem? Do we handle that in
mock? Or in plague? As a yum-plugin? I don't know plague and mock to
much, so any help/suggestions appreciated!
/me looks at the command and wonders why we use "kver" for kversion --
we either should use "kver" and "kvars" or "kversion" and "kvariants"
--
Thorsten Leemhuis <fedora(a)leemhuis.info>
17 years, 3 months
x86_64-devel koffice builds fail randomly (on random builders)
by Andreas Bierfert
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Hi,
when I pushed koffice 1.4.90-{1,2} everything worked fine. Not there was a bug
with some desktop file and I fixed it by adding an %exclude to the file section
(so no changes in the sources or the build part or the alike) but now the build
keeps failing randomly. See the logs from try 2-5 (lost the first one).
Any help would be very much welcome...
- - Andreas
This is from the 2nd try:
creating libkiviocommon_la.all_cpp.cpp ...
/bin/sh ../../libtool --silent --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I.
- -I. -I../.. -I./ui -Iui -I./config -Iconfig -I./kiviosdk -I./tools
- -I../../lib/kofficeui -I../../lib/kofficeui -I../../lib/kofficecore
- -I../../lib/kofficecore -I../../lib/store -I../../lib/store -I../../lib/kwmf
- -I../../lib/kwmf -I../../lib/kopalette -I../../lib/kopalette
- -I../../lib/kopalette -I../../lib/kopalette -I../../lib/kotext
- -I../../lib/kotext -I../../lib/kformula -I/usr/include/kde
- -I/usr/lib64/qt-3.3/include -I. -DQT_THREAD_SUPPORT -D_REENTRANT
- -Wno-long-long -Wundef -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align
- -Wconversion -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall
- -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4
- -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute
- -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common
- -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT -DQT_NO_TRANS
LATION -DHAVE_KNEWSTUFF -c -o libkiviocommon_la.all_cpp.lo
libkiviocommon_la.all_cpp.cpp
/bin/sh ../../libtool --silent --tag=CXX --mode=link g++ -Wno-long-long -Wundef
- -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion
- -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall
- -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4
- -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute
- -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common
- -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT
- -DQT_NO_TRANSLATION -DHAVE_KNEWSTUFF -o libkiviocommon.la -rpath /usr/lib64
- -L/usr/lib64/qt-3.3/lib -L/usr/lib64 -L/usr/lib64 -no-undefined
- -Wl,--no-undefined -Wl,--allow-shlib-undefined libkiviocommon_la.all_cc.lo
libkiviocommon_la.all_cpp.lo ../../kivio/kiviopart/tools/libtools.la
../../kivio/kiviopart/kiviosdk/libkiviosdk.la
../../kivio/kiviopart/config/libkivioconfig.la ../../kivio/kiviopart/ui/libui.la
../../lib/kofficeui/libkofficeui.la ../../lib/kopainter/libkopainter.la ../../lib
/kopalette/libkopalette.la ../../lib/kotext/libkotext.la -lpython2.4 -ldl -ldl
-L/usr/lib64
: : No such file or directory
: : No such file or directory
libtool: link: `/usr/lib64/libkutils.la' is not a valid libtool archive
make[4]: *** [libkiviocommon.la] Error 1
make[4]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kivio/kiviopart'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kivio/kiviopart'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kivio'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90'
make: *** [all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.25214 (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.25214 (%build)
3rd try:
/bin/sh ../libtool --silent --tag=CXX --mode=link g++ -Wno-long-long -Wundef
- -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion
- -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall
- -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4
- -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute
- -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common
- -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT
- -DQT_NO_TRANSLATION -DHAVE_KNEWSTUFF -o kpresenter -no-undefined
- -L/usr/lib64/qt-3.3/lib -L/usr/lib64 -L/usr/lib64 kpresenter.la.o
libkdeinit_kpresenter.la
libtool: link: `/usr/lib64/libkdecore.la' is not a valid libtool archive
make[3]: *** [kpresenter] Error 1
make[3]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kpresenter'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kpresenter'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90'
make: *** [all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.96654 (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.96654 (%build)
4th try:
/bin/sh ../../../libtool --silent --tag=CXX --mode=link g++ -Wno-long-long
- -Wundef -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion
- -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall
- -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4
- -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute
- -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common
- -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT
- -DQT_NO_TRANSLATION -DHAVE_KNEWSTUFF -fexceptions -o krossruby.la -rpath
/usr/lib64/kde3 -module -lruby -L/usr/lib64/qt-3.3/lib -L/usr/lib64
- -L/usr/lib64 -avoid-version -module -no-undefined -Wl,--no-undefined
- -Wl,--allow-shlib-undefined -version-info 1:0:0 krossruby_la.all_cpp.lo
- -lqt-mt -lz -lpng -lz -lm -lXext -lX11 -lSM -lICE -lpthread -lkdecore
../api/libkrossapi.la ../main/libkrossmain.la
libtool: link: `/usr/lib64/libkio.la' is not a valid libtool archive
make[4]: *** [krossruby.la] Error 1
make[4]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/lib/kross/ruby'
make[3]: *** [all-recursive] Error 1
make[3]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/lib/kross'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/lib'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90'
make: *** [all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.10097 (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.10097 (%build)
5th try:
bin/sh ../libtool --silent --tag=CXX --mode=link g++ -Wno-long-long -Wundef
- -ansi -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -Wcast-align -Wconversion
- -Wchar-subscripts -Wall -W -Wpointer-arith -O2 -O2 -g -pipe -Wall
- -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4
- -m64 -mtune=generic -Wformat-security -Wmissing-format-attribute
- -Wno-non-virtual-dtor -fno-exceptions -fno-check-new -fno-common
- -DQT_CLEAN_NAMESPACE -DQT_NO_ASCII_CAST -DQT_NO_STL -DQT_NO_COMPAT
- -DQT_NO_TRANSLATION -DHAVE_KNEWSTUFF -o libkdgantt.la
- -L/usr/lib64/qt-3.3/lib -L/usr/lib64 -L/usr/lib64 -no-undefined
- -Wl,--no-undefined -Wl,--allow-shlib-undefined libkdgantt_la.all_cpp.lo -lkdeui
libtool: link: `/usr/lib64/libkdefx.la' is not a valid libtool archive
make[2]: *** [libkdgantt.la] Error 1
make[2]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90/kdgantt'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/koffice-1.4.90'
make: *** [all] Error 2
error: Bad exit status from /var/tmp/rpm-tmp.40707 (%build)
RPM build errors:
Bad exit status from /var/tmp/rpm-tmp.40707 (%build)
- --
Andreas Bierfert | http://awbsworld.de | GPG: C58CF1CB
andreas.bierfert(a)lowlatency.de | http://lowlatency.de | signed/encrypted
phone: +49 2402 102373 | cell: +49 172 9789968 | mail preferred
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org
iD8DBQFD6wANQEQyPsWM8csRAsPMAJwIzdk8K2zR9AfJRkWnpkhwtN206wCfXBwJ
sc6WBZYbexp7WyxRLduvyJc=
=GFC8
-----END PGP SIGNATURE-----
17 years, 3 months
pthread_sigmask module
by Jeff Sheltren
Hi, I am having some problems with the build server hanging (ie.
there are no jobs running, but when I submit a job it sits with
status 'waiting')...
Since this is running on EL4 (python 2.3.4), I'm getting this message
when I start up plague-builder:
WARNING: You might want to install the pthread_sigmask module.
Python versions earlier than 2.4 have problems with signals and
spawned processes.
I'm guessing this is related to my problem :) So the question is,
where can I find this module?
Thanks,
Jeff
17 years, 3 months