Hi all,
I'm working on updating libgda + libgnomedb to 3.0.x for F-8 / future rawhide (more on that later) and yesterday evening before going to bed I queued a build of libgda-3.0.1, which completed successfully. This morning I build libgnomedb-3.0.0, which correctly got its buildroot populated with the fresh libgda-3.0.0 . So next I tried to build gnumeric against the new libgda + libgnomedb, but populating the root failed with:
Executing /usr/sbin/mock-helper yum --installroot /var/lib/mock/dist-f8-build-5517-1020/root resolvedep 'intltool' 'pygtk2-devel
= 2.6.0' 'libgnomeprintui22-devel >= 2.8.2' 'libgnomedb-devel >= 3.0.0'
'python-devel' 'scrollkeeper' 'gettext' 'automake' 'desktop-file-utils >= 0.9' 'libgnomeui-devel >= 2.4.0' 'libgsf-gnome-devel >= 1.13.2' 'goffice-devel >= 0.2.0' 'libtool' 'autoconf' 'guile-devel' 0:intltool-0.35.5-3.fc7.x86_64 0:pygtk2-devel-2.10.4-2.fc7.x86_64 0:libgnomeprintui22-devel-2.18.0-1.fc7.x86_64 1:libgnomedb-devel-1.9.100-15.fc7.x86_64 0:python-devel-2.5-12.fc7.x86_64 0:scrollkeeper-0.3.14-11.fc7.x86_64 0:gettext-0.16.1-8.fc7.x86_64 0:automake-1.10-5.noarch 0:desktop-file-utils-0.12-4.fc7.x86_64 0:libgnomeui-devel-2.18.1-2.fc7.x86_64 0:libgsf-gnome-devel-1.14.3-4.fc7.x86_64 0:goffice-devel-0.2.2-1.fc7.x86_64 0:libtool-1.5.22-11.fc7.x86_64 0:autoconf-2.61-8.fc7.noarch 5:guile-devel-1.8.1-3.fc7.x86_64
Notice how it says its going to include libgnomedb-devel-1.9.100, not 3.0.0
and then: Error: Missing Dependency: libgda-3.so.3()(64bit) is needed by package libgnomedb
Which figures, as that is the old 1.9.100 libgda .so
For the full log see: http://koji.fedoraproject.org/koji/getfile?taskID=18756&name=root.log
So why is koji finding the new libgda, but not the new libgnomedb, is some kinda sync script running for this at regular intervals, or ... ?
Regards,
Hans
Hans de Goede wrote, at 05/28/2007 05:12 PM +9:00:
Hi all,
I'm working on updating libgda + libgnomedb to 3.0.x for F-8 / future rawhide (more on that later) and yesterday evening before going to bed I queued a build of libgda-3.0.1, which completed successfully. This morning I build libgnomedb-3.0.0, which correctly got its buildroot populated with the fresh libgda-3.0.0 . So next I tried to build gnumeric against the new libgda + libgnomedb, but populating the root failed with:
Executing /usr/sbin/mock-helper yum --installroot
1:libgnomedb-devel-1.9.100-15.fc7.x86_64
Notice how it says its going to include libgnomedb-devel-1.9.100, not 3.0.0
and then: Error: Missing Dependency: libgda-3.so.3()(64bit) is needed by package libgnomedb
Which figures, as that is the old 1.9.100 libgda .so
For the full log see: http://koji.fedoraproject.org/koji/getfile?taskID=18756&name=root.log
From my experience, perhaps you have to wait for one hour or so...
Mamoru
On Mon, 28 May 2007 10:12:56 +0200, Hans de Goede wrote:
Hi all,
I'm working on updating libgda + libgnomedb to 3.0.x for F-8 / future rawhide (more on that later) and yesterday evening before going to bed I queued a build of libgda-3.0.1, which completed successfully. This morning I build libgnomedb-3.0.0, which correctly got its buildroot populated with the fresh libgda-3.0.0 . So next I tried to build gnumeric against the new libgda + libgnomedb, but populating the root failed with:
Executing /usr/sbin/mock-helper yum --installroot /var/lib/mock/dist-f8-build-5517-1020/root resolvedep 'intltool' 'pygtk2-devel
= 2.6.0' 'libgnomeprintui22-devel >= 2.8.2' 'libgnomedb-devel >= 3.0.0'
'python-devel' 'scrollkeeper' 'gettext' 'automake' 'desktop-file-utils >= 0.9' 'libgnomeui-devel >= 2.4.0' 'libgsf-gnome-devel >= 1.13.2' 'goffice-devel >= 0.2.0' 'libtool' 'autoconf' 'guile-devel' 0:intltool-0.35.5-3.fc7.x86_64 0:pygtk2-devel-2.10.4-2.fc7.x86_64 0:libgnomeprintui22-devel-2.18.0-1.fc7.x86_64 1:libgnomedb-devel-1.9.100-15.fc7.x86_64 0:python-devel-2.5-12.fc7.x86_64 0:scrollkeeper-0.3.14-11.fc7.x86_64 0:gettext-0.16.1-8.fc7.x86_64 0:automake-1.10-5.noarch 0:desktop-file-utils-0.12-4.fc7.x86_64 0:libgnomeui-devel-2.18.1-2.fc7.x86_64 0:libgsf-gnome-devel-1.14.3-4.fc7.x86_64 0:goffice-devel-0.2.2-1.fc7.x86_64 0:libtool-1.5.22-11.fc7.x86_64 0:autoconf-2.61-8.fc7.noarch 5:guile-devel-1.8.1-3.fc7.x86_64
Notice how it says its going to include libgnomedb-devel-1.9.100, not 3.0.0
and then: Error: Missing Dependency: libgda-3.so.3()(64bit) is needed by package libgnomedb
Which figures, as that is the old 1.9.100 libgda .so
For the full log see: http://koji.fedoraproject.org/koji/getfile?taskID=18756&name=root.log
So why is koji finding the new libgda, but not the new libgnomedb, is some kinda sync script running for this at regular intervals, or ... ?
The newRepo task was not finished yet when you submitted your build request: http://koji.fedoraproject.org/koji/taskinfo?taskID=18751
Michael Schwendt wrote:
On Mon, 28 May 2007 10:12:56 +0200, Hans de Goede wrote:
Hi all,
I'm working on updating libgda + libgnomedb to 3.0.x for F-8 / future rawhide (more on that later) and yesterday evening before going to bed I queued a build of libgda-3.0.1, which completed successfully. This morning I build libgnomedb-3.0.0, which correctly got its buildroot populated with the fresh libgda-3.0.0 . So next I tried to build gnumeric against the new libgda + libgnomedb, but populating the root failed with:
Executing /usr/sbin/mock-helper yum --installroot /var/lib/mock/dist-f8-build-5517-1020/root resolvedep 'intltool' 'pygtk2-devel
= 2.6.0' 'libgnomeprintui22-devel >= 2.8.2' 'libgnomedb-devel >= 3.0.0'
'python-devel' 'scrollkeeper' 'gettext' 'automake' 'desktop-file-utils >= 0.9' 'libgnomeui-devel >= 2.4.0' 'libgsf-gnome-devel >= 1.13.2' 'goffice-devel >= 0.2.0' 'libtool' 'autoconf' 'guile-devel' 0:intltool-0.35.5-3.fc7.x86_64 0:pygtk2-devel-2.10.4-2.fc7.x86_64 0:libgnomeprintui22-devel-2.18.0-1.fc7.x86_64 1:libgnomedb-devel-1.9.100-15.fc7.x86_64 0:python-devel-2.5-12.fc7.x86_64 0:scrollkeeper-0.3.14-11.fc7.x86_64 0:gettext-0.16.1-8.fc7.x86_64 0:automake-1.10-5.noarch 0:desktop-file-utils-0.12-4.fc7.x86_64 0:libgnomeui-devel-2.18.1-2.fc7.x86_64 0:libgsf-gnome-devel-1.14.3-4.fc7.x86_64 0:goffice-devel-0.2.2-1.fc7.x86_64 0:libtool-1.5.22-11.fc7.x86_64 0:autoconf-2.61-8.fc7.noarch 5:guile-devel-1.8.1-3.fc7.x86_64
Notice how it says its going to include libgnomedb-devel-1.9.100, not 3.0.0
and then: Error: Missing Dependency: libgda-3.so.3()(64bit) is needed by package libgnomedb
Which figures, as that is the old 1.9.100 libgda .so
For the full log see: http://koji.fedoraproject.org/koji/getfile?taskID=18756&name=root.log
So why is koji finding the new libgda, but not the new libgnomedb, is some kinda sync script running for this at regular intervals, or ... ?
The newRepo task was not finished yet when you submitted your build request: http://koji.fedoraproject.org/koji/taskinfo?taskID=18751
Ah thats new (and a regression compared to plague), who / what starts this task and when / how often?
Regards,
Hans
On Monday 28 May 2007 09:22:55 Hans de Goede wrote:
Ah thats new (and a regression compared to plague), who / what starts this task and when / how often?
The only 'regression' is that it is slightly slower than plague. With Plague you had to wait for a repo recreation as well, it just was a bit faster. With Koji, there are far more packages involved and it takes longer. But that's why there are things like chain-build so that it manages the wait for the package to show up in the repo for you.
Repo recreation is automatic, it starts when a package build completes (or rather is queued up if there is already one in progress I think).
Jesse Keating wrote:
On Monday 28 May 2007 09:22:55 Hans de Goede wrote:
Ah thats new (and a regression compared to plague), who / what starts this task and when / how often?
The only 'regression' is that it is slightly slower than plague. With Plague you had to wait for a repo recreation as well, it just was a bit faster.
Ah I see, well with plague it was so fast I never hit the problem :)
With Koji, there are far more packages involved and it takes longer. But that's why there are things like chain-build so that it manages the wait for the package to show up in the repo for you.
And how do I do a chain build?
Thanks & Regards,
Hans
On Tue, 29 May 2007 10:03:20 +0200, Hans de Goede wrote:
Jesse Keating wrote:
On Monday 28 May 2007 09:22:55 Hans de Goede wrote:
Ah thats new (and a regression compared to plague), who / what starts this task and when / how often?
The only 'regression' is that it is slightly slower than plague. With Plague you had to wait for a repo recreation as well, it just was a bit faster.
Ah I see, well with plague it was so fast I never hit the problem :)
That is because plague only ran createrepo for the small and single needsign repo, whereas kojira builds a complete set of arch-specific repositores from scratch (using hardlinks) and then runs createrepo for each of them.
With Koji, there are far more packages involved and it takes longer. But that's why there are things like chain-build so that it manages the wait for the package to show up in the repo for you.
And how do I do a chain build?
koji chain-build --help
On Tuesday 29 May 2007 05:03:46 Michael Schwendt wrote:
That is because plague only ran createrepo for the small and single needsign repo, whereas kojira builds a complete set of arch-specific repositores from scratch (using hardlinks) and then runs createrepo for each of them.
It doesn't do it from scratch per-se. Createrepo now has the ability to just 'update' existing repodata with changes rather than creating it all from scratch again.
On Tue, 29 May 2007 08:04:17 -0400, Jesse Keating wrote:
That is because plague only ran createrepo for the small and single needsign repo, whereas kojira builds a complete set of arch-specific repositores from scratch (using hardlinks) and then runs createrepo for each of them.
It doesn't do it from scratch per-se. Createrepo now has the ability to just 'update' existing repodata with changes rather than creating it all from scratch again.
Still it takes ~20 minutes for dist-f8-build (~3 min per repo plus the time it takes to fill the repos with package links): http://koji.fedoraproject.org/koji/taskinfo?taskID=19563
On Tuesday 29 May 2007 08:28:45 Michael Schwendt wrote:
Still it takes ~20 minutes for dist-f8-build (~3 min per repo plus the time it takes to fill the repos with package links): http://koji.fedoraproject.org/koji/taskinfo?taskID=19563
Yep. Some thought is going into how to speed this up, since the packages/ dir is available there may not be a need to make hardlinks, and since most if not all the repodata is already in the koji DB it should be easy enough to dump contents directly into repodata files using some API, and reference the packages where they live. Could reduce the repo creation time significantly. However not something we could have done by the merge time.
On Tue, 2007-05-29 at 14:28 +0200, Michael Schwendt wrote:
On Tue, 29 May 2007 08:04:17 -0400, Jesse Keating wrote:
That is because plague only ran createrepo for the small and single needsign repo, whereas kojira builds a complete set of arch-specific repositores from scratch (using hardlinks) and then runs createrepo for each of them.
It doesn't do it from scratch per-se. Createrepo now has the ability to just 'update' existing repodata with changes rather than creating it all from scratch again.
Still it takes ~20 minutes for dist-f8-build (~3 min per repo plus the time it takes to fill the repos with package links): http://koji.fedoraproject.org/koji/taskinfo?taskID=19563
"JK" == Jesse Keating jkeating@redhat.com writes:
JK> koji chain-build --help
People keep saying this, but frankly: usage: koji chain-build [options] target URL [URL2 [:] URL3 [:] URL4 ...] doesn't help all that much, since it's not at all obvious what URL is supposed to be.
If I have pkgA v1.2-3 and pkgB v6.7-8 to build on the devel branch, and pkgB depends on pkgA, does this work?
koji chain-build \ 'cvs://cvs.fedoraproject.org/cvs/pkgs?rpms/pkgA/devel#pkgA-1_2-3_fc8' \ 'cvs://cvs.fedoraproject.org/cvs/pkgs?rpms/pkgB/devel#pkgB-6_7-8_fc8'
If there's really no simpler way to do that then that is really painful to figure out.
Jason L Tibbitts III wrote:
If I have pkgA v1.2-3 and pkgB v6.7-8 to build on the devel branch, and pkgB depends on pkgA, does this work?
koji chain-build \ 'cvs://cvs.fedoraproject.org/cvs/pkgs?rpms/pkgA/devel#pkgA-1_2-3_fc8' \ 'cvs://cvs.fedoraproject.org/cvs/pkgs?rpms/pkgB/devel#pkgB-6_7-8_fc8'
Yes, used it myself this morning (though I typo'd the url...).
-- Rex
"JLT" == Jason L Tibbitts tibbs@math.uh.edu writes:
JLT> OK, cool, let's hack on Makefile.common a bit....
OK, here's a nasty first-pass hack. Apply this to Makefile.common, then do "make chain-queue" in each CVS branch directory where you would normally do "make build". (As with building, the directories must be tagged and such.) Then after your last chain-queue, run "make chain-go". In fact, you probably have to apply this to all of your Makefile.common files, unless you have them all linked together or something. And you'll have to patch after tagging and such, else the tagging will fail and screw all sorts of nice things up. It doesn't seem very easy to have local Makefile.common changes.
"make chain-restart" will clear out the queue.
I did a basic test by chain-building two of my packages, but they don't really depend on each other so it's not a complete test.
- J<
On 29 May 2007 17:46:13 -0500, Jason L Tibbitts III wrote:
"JLT" == Jason L Tibbitts writes:
JLT> OK, cool, let's hack on Makefile.common a bit....
OK, here's a nasty first-pass hack.
When I tried out chain-builds with a local koji test-server, the command accepted normal _tags_ in the same way as "koji build" does. It was not necessary to enter long cvs urls. Do I remember that incorrectly?
On Tue, 2007-05-29 at 17:46 -0500, Jason L Tibbitts III wrote:
"JLT" == Jason L Tibbitts tibbs@math.uh.edu writes:
JLT> OK, cool, let's hack on Makefile.common a bit....
OK, here's a nasty first-pass hack. Apply this to Makefile.common, then do "make chain-queue" in each CVS branch directory where you would normally do "make build". (As with building, the directories must be tagged and such.) Then after your last chain-queue, run "make chain-go". In fact, you probably have to apply this to all of your Makefile.common files, unless you have them all linked together or something. And you'll have to patch after tagging and such, else the tagging will fail and screw all sorts of nice things up. It doesn't seem very easy to have local Makefile.common changes.
"make chain-restart" will clear out the queue.
I did a basic test by chain-building two of my packages, but they don't really depend on each other so it's not a complete test.
I just checked in an alternate chain-build implementation to Makefile.common, based on a target we were using internally (and have tested rather extensively). Update your common/ directories and run "make help" to see the chain-build usage.
You specify the packages that the current package depends on using the CHAIN= parameter to "make chain-build". The packages specified in the CHAIN= parameter will be checked out into a temp directory and "make cvsurl" will be called to get their CVS URL (this will reference the latest tag that was applied to the package on the current branch, and that tag must not have been built in Koji already). The CVS URLs from each CHAIN= package and the current package will be used to generate the appropriate koji command-line to build each package in order (the current package will be built last, and should not appear in the CHAIN= parameter).
If you want the Koji client to exit immediately after launching the task, you can also pass BUILD_FLAGS="--nowait" to make, as with a normal "make build".
Please give it a try, and let me know if you run into any issues.
Thanks, Mike
On Thu, 2007-05-31 at 15:32 -0400, Mike Bonnet wrote:
I just checked in an alternate chain-build implementation to Makefile.common, based on a target we were using internally (and have tested rather extensively). Update your common/ directories and run "make help" to see the chain-build usage.
You specify the packages that the current package depends on using the CHAIN= parameter to "make chain-build". The packages specified in the CHAIN= parameter will be checked out into a temp directory and "make cvsurl" will be called to get their CVS URL (this will reference the latest tag that was applied to the package on the current branch, and that tag must not have been built in Koji already). The CVS URLs from each CHAIN= package and the current package will be used to generate the appropriate koji command-line to build each package in order (the current package will be built last, and should not appear in the CHAIN= parameter).
What do you do if you have several packages that have a common dependency?
For instance: bzrtools Requires: bzr >= %{majorver} bzr < %{nextver} bzr-gtk Requires: bzr >= %{majorver} bzr < %{nextver}
Does cd bzr-gtk/devel ; make chain-build CHAIN='bzr bzrtools' build bzr, then bzrtools when bzr is ready, then bzr-gtk when bzrtools is ready? Or does it attempt to make bzr and bzr-gtk independently followed by bzr-gtk?
Related to this is specifying more than one package.
foo Requires: bar, bar Requires: baz.
An update of baz requires an update of bar which requires an update of foo. Does cd foo/devel ; make chain-build CHAIN='baz bar' do the right thing?
Thanks, -Toshio
On Fri, 2007-06-01 at 15:09 -0700, Toshio Kuratomi wrote:
On Thu, 2007-05-31 at 15:32 -0400, Mike Bonnet wrote:
I just checked in an alternate chain-build implementation to Makefile.common, based on a target we were using internally (and have tested rather extensively). Update your common/ directories and run "make help" to see the chain-build usage.
You specify the packages that the current package depends on using the CHAIN= parameter to "make chain-build". The packages specified in the CHAIN= parameter will be checked out into a temp directory and "make cvsurl" will be called to get their CVS URL (this will reference the latest tag that was applied to the package on the current branch, and that tag must not have been built in Koji already). The CVS URLs from each CHAIN= package and the current package will be used to generate the appropriate koji command-line to build each package in order (the current package will be built last, and should not appear in the CHAIN= parameter).
What do you do if you have several packages that have a common dependency?
For instance: bzrtools Requires: bzr >= %{majorver} bzr < %{nextver} bzr-gtk Requires: bzr >= %{majorver} bzr < %{nextver}
Does cd bzr-gtk/devel ; make chain-build CHAIN='bzr bzrtools' build bzr, then bzrtools when bzr is ready, then bzr-gtk when bzrtools is ready? Or does it attempt to make bzr and bzr-gtk independently followed by bzr-gtk?
Related to this is specifying more than one package.
foo Requires: bar, bar Requires: baz.
An update of baz requires an update of bar which requires an update of foo. Does cd foo/devel ; make chain-build CHAIN='baz bar' do the right thing?
You want to run "make chain-build" from the last package in the chain. This makes sense because you will have needed to bump the revision in the spec and run "make tag" for each of the previous packages in the chain already. So, if foo BuildRequires bar and bar BuildRequires baz:
cd foo/devel <bump spec and commit> make tag cd ../../bar/devel <bump spec and commit> make tag cd ../../baz/devel <bump spec and commit> make tag make chain-build BUILD="foo bar"
Which will build foo, wait until it appears in the repo to build bar, and then wait until bar appears in the repo to build baz.
I'm going to see about rolling ajax's chain-build-expert patch in, for finer control of build groupings.
On Thu, 2007-05-31 at 15:32 -0400, Mike Bonnet wrote:
I just checked in an alternate chain-build implementation to Makefile.common, based on a target we were using internally (and have tested rather extensively). Update your common/ directories and run "make help" to see the chain-build usage.
You specify the packages that the current package depends on using the CHAIN= parameter to "make chain-build". The packages specified in the CHAIN= parameter will be checked out into a temp directory and "make cvsurl" will be called to get their CVS URL (this will reference the latest tag that was applied to the package on the current branch, and that tag must not have been built in Koji already). The CVS URLs from each CHAIN= package and the current package will be used to generate the appropriate koji command-line to build each package in order (the current package will be built last, and should not appear in the CHAIN= parameter).
This appears to stick a : between every target in the CHAIN, which means the build system will stall between every one. Would it be possible to get like a 'make expert-chain' that lets you specify the stall points explicitly?
Otherwise, if I need to rebuild all X drivers for a new server ABI, I'll be waiting on ~40 stall points.
- ajax
On Mon, 2007-06-04 at 09:56 -0400, Adam Jackson wrote:
This appears to stick a : between every target in the CHAIN, which means the build system will stall between every one. Would it be possible to get like a 'make expert-chain' that lets you specify the stall points explicitly?
In fact, rather than just ask for it, here's a patch that does what I want.
- ajax
On Tue, 2007-06-05 at 08:24 -0400, Adam Jackson wrote:
On Mon, 2007-06-04 at 09:56 -0400, Adam Jackson wrote:
This appears to stick a : between every target in the CHAIN, which means the build system will stall between every one. Would it be possible to get like a 'make expert-chain' that lets you specify the stall points explicitly?
In fact, rather than just ask for it, here's a patch that does what I want.
This patch doesn't seem to have made it's way into Makefile.common. Should I file a bug?
- ajax
On Tuesday 05 June 2007 08:24:22 Adam Jackson wrote:
On Mon, 2007-06-04 at 09:56 -0400, Adam Jackson wrote:
This appears to stick a : between every target in the CHAIN, which means the build system will stall between every one. Would it be possible to get like a 'make expert-chain' that lets you specify the stall points explicitly?
In fact, rather than just ask for it, here's a patch that does what I want.
- ajax
Couple things.
Should there be a .PHONY target for chain-build-expert? Can you add a section in the help for this new target?
On 29 May 2007 14:15:46 -0500, Jason L Tibbitts III wrote:
"JK" == Jesse Keating writes:
JK> koji chain-build --help
People keep saying this, but frankly: usage: koji chain-build [options] target URL [URL2 [:] URL3 [:] URL4 ...] doesn't help all that much, since it's not at all obvious what URL is supposed to be.
Same as with: koji build --help
It's missing better documentation.