We've installed koji (1.2.5) on F8 and successfully got the buildroots generating and scratch builds of src.rpms working.
I'm trying to build packages from our git repo. I gave it the url to our repo:
git://git.fedorahosted.org/git?spacewalk.git#HEAD
But it complains about not having a spec file. Which makes sense as there is no spec file at the top of the tree. The spec file I want to build is actually located in the web directory.
I've tried several versions of the url with no luck. It seems that koji expects the scm repository house a single package which is not the case for us.
Has anyone else run into a similar situation? Is there a way to specify the subdirectory to look in after doing the git clone?
Jesus M. Rodriguez wrote:
I've tried several versions of the url with no luck. It seems that koji expects the scm repository house a single package which is not the case for us.
There are innumerable ways that one might build an srpm from an SCM checkout. Koji can only support a limited set. Basically, your SCM layout must support koji, not the other way around.
Mike McLean wrote:
Jesus M. Rodriguez wrote:
I've tried several versions of the url with no luck. It seems that koji expects the scm repository house a single package which is not the case for us.
There are innumerable ways that one might build an srpm from an SCM checkout. Koji can only support a limited set. Basically, your SCM layout must support koji, not the other way around.
So am I correct in my assumption that koji expects the scm repository to house a single package?
Is there a way to build from src.rpm without doing scratch builds? /me couldn't find anything in the docs about this.
Thanks jesus
-- jesus m. rodriguez | jesusr@redhat.com sr. software engineer | irc: zeus red hat network | 919.754.4413 (w) rhce # 805008586930012 | 919.623.0080 (c) +-------------------------------------------+ | "Those who cannot learn from history | | are doomed to repeat it." | | -- George Santayana | +-------------------------------------------+
On Wed, 2008-07-23 at 14:14 -0400, Jesus M. Rodriguez wrote:
Mike McLean wrote:
Jesus M. Rodriguez wrote:
I've tried several versions of the url with no luck. It seems that koji expects the scm repository house a single package which is not the case for us.
There are innumerable ways that one might build an srpm from an SCM checkout. Koji can only support a limited set. Basically, your SCM layout must support koji, not the other way around.
So am I correct in my assumption that koji expects the scm repository to house a single package?
Is there a way to build from src.rpm without doing scratch builds? /me couldn't find anything in the docs about this.
If you are an admin user in the koji system then you can build from srpm without doing scratch...but mere mortals aren't allowed that privilege ;-)
Jesus M. Rodriguez wrote:
So am I correct in my assumption that koji expects the scm repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm' issued from the checkout will create a single source rpm. There is not enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the different srpms? Is there a real reason that they are all in one module?
Is there a way to build from src.rpm without doing scratch builds? /me couldn't find anything in the docs about this.
Some of my current work in progress will allow this to be configurable. Look for this to land in the next few weeks.
At present, as Doug points out, there is an exception for admins.
On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
Jesus M. Rodriguez wrote:
So am I correct in my assumption that koji expects the scm repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm' issued from the checkout will create a single source rpm. There is not enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the different srpms? Is there a real reason that they are all in one module?
We generate different srpms with the Makefile that exists in each subdirectory. For example, if you want an srpm for the web module cd spacewalk.git/web; make srpm
Want one for the java code, then cd spacewalk.git/java.
With SVN it seems completely doable because I could give the path to repo/spacewalk/java and have koji checkout just that directory. Seems to be a limitation of GIT in this case (or we're using it wrong) :)
In theory, if there was a way I could tell koji to pull from this git repo, and cd into this directory, you'll find the spec file and Makefile there :)
Is there a real reason for them to be all in one module? Because this is how we had it in SVN, then switched to git. :)
Is there a way to build from src.rpm without doing scratch builds? /me couldn't find anything in the docs about this.
Some of my current work in progress will allow this to be configurable. Look for this to land in the next few weeks.
At present, as Doug points out, there is an exception for admins.
For our purposes, we might go the route of the admins.
On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
Jesus M. Rodriguez wrote:
So am I correct in my assumption that koji expects the scm repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm' issued from the checkout will create a single source rpm. There is not enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the different srpms? Is there a real reason that they are all in one module?
We generate different srpms with the Makefile that exists in each subdirectory. For example, if you want an srpm for the web module cd spacewalk.git/web; make srpm
Want one for the java code, then cd spacewalk.git/java.
With SVN it seems completely doable because I could give the path to repo/spacewalk/java and have koji checkout just that directory. Seems to be a limitation of GIT in this case (or we're using it wrong) :)
Well, git was intentionally written to be basically a state machine of the entire project. So, housing more than one project in a single git repo isn't wrong, but it does tie your various projects together at the state machine level. This is why you can't checkout just one subdirectory from git, you have to grab the entire project. I think I heard mumblings of someone wanting to make subprojects work nicely in git, but I haven't heard if it ever got off the ground.
Regardless though, koji won't work with what your current SCM layout without modification to the koji code.
On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote:
On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
Jesus M. Rodriguez wrote:
So am I correct in my assumption that koji expects the scm repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm' issued from the checkout will create a single source rpm. There is not enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the different srpms? Is there a real reason that they are all in one module?
We generate different srpms with the Makefile that exists in each subdirectory. For example, if you want an srpm for the web module cd spacewalk.git/web; make srpm
Want one for the java code, then cd spacewalk.git/java.
With SVN it seems completely doable because I could give the path to repo/spacewalk/java and have koji checkout just that directory. Seems to be a limitation of GIT in this case (or we're using it wrong) :)
Well, git was intentionally written to be basically a state machine of the entire project. So, housing more than one project in a single git repo isn't wrong, but it does tie your various projects together at the state machine level. This is why you can't checkout just one subdirectory from git, you have to grab the entire project. I think I heard mumblings of someone wanting to make subprojects work nicely in git, but I haven't heard if it ever got off the ground.
Regardless though, koji won't work with what your current SCM layout without modification to the koji code.
While that's true, I'm not sure we're definitely handling things correctly in the git case. The git support in Koji hasn't gotten a lot of testing, and there are a lot of ways to setup a git repo. However, I think that multiple projects in the same repo is probably something we want to support (since we support it in cvs and svn).
The SCM URL you pass to Koji has a format of:
scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier
With git we combine the /path/to/repo and the path/to/module into a single URL we pass to "git clone", essentially collapsing them both into a full module path (this is also what we do with svn). Would it be more appropriate to treat the /path/to/repo as the location of the git repo which we would pass to "git clone", and /path/to/module as a subdirectory in the repo, which we would use as the base directory of the build, and look there for a .spec file?
On Wed, 2008-07-23 at 23:51 -0400, Mike Bonnet wrote:
On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote:
On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
Jesus M. Rodriguez wrote:
So am I correct in my assumption that koji expects the scm repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm' issued from the checkout will create a single source rpm. There is not enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the different srpms? Is there a real reason that they are all in one module?
We generate different srpms with the Makefile that exists in each subdirectory. For example, if you want an srpm for the web module cd spacewalk.git/web; make srpm
Want one for the java code, then cd spacewalk.git/java.
With SVN it seems completely doable because I could give the path to repo/spacewalk/java and have koji checkout just that directory. Seems to be a limitation of GIT in this case (or we're using it wrong) :)
Well, git was intentionally written to be basically a state machine of the entire project. So, housing more than one project in a single git repo isn't wrong, but it does tie your various projects together at the state machine level. This is why you can't checkout just one subdirectory from git, you have to grab the entire project. I think I heard mumblings of someone wanting to make subprojects work nicely in git, but I haven't heard if it ever got off the ground.
Regardless though, koji won't work with what your current SCM layout without modification to the koji code.
While that's true, I'm not sure we're definitely handling things correctly in the git case. The git support in Koji hasn't gotten a lot of testing, and there are a lot of ways to setup a git repo. However, I think that multiple projects in the same repo is probably something we want to support (since we support it in cvs and svn).
Just because these are things that are done in cvs and svn doesn't necessarily mean we want to do them in git ;-) Amongst other things, if you put multiple projects into a single repo, then to build *any* of those projects, you have to clone the *entire* repo. In addition, you already have to deal with the fact that you are cloning instead of checking out like you do with cvs. In other words, the overhead of cloning a huge repo grows much more rapidly for git than for cvs, so the pain of a multi-project repo is going to be felt much more acutely.
The SCM URL you pass to Koji has a format of:
scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier
With git we combine the /path/to/repo and the path/to/module into a single URL we pass to "git clone", essentially collapsing them both into a full module path (this is also what we do with svn). Would it be more appropriate to treat the /path/to/repo as the location of the git repo which we would pass to "git clone", and /path/to/module as a subdirectory in the repo, which we would use as the base directory of the build, and look there for a .spec file?
I could certainly see that as being a valid interpretation of the url string.
On Wed, Jul 23, 2008 at 11:51:36PM -0400, Mike Bonnet wrote:
On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote:
On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
Jesus M. Rodriguez wrote:
So am I correct in my assumption that koji expects the scm repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm' issued from the checkout will create a single source rpm. There is not enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the different srpms? Is there a real reason that they are all in one module?
We generate different srpms with the Makefile that exists in each subdirectory. For example, if you want an srpm for the web module cd spacewalk.git/web; make srpm
Want one for the java code, then cd spacewalk.git/java.
With SVN it seems completely doable because I could give the path to repo/spacewalk/java and have koji checkout just that directory. Seems to be a limitation of GIT in this case (or we're using it wrong) :)
Well, git was intentionally written to be basically a state machine of the entire project. So, housing more than one project in a single git repo isn't wrong, but it does tie your various projects together at the state machine level. This is why you can't checkout just one subdirectory from git, you have to grab the entire project. I think I heard mumblings of someone wanting to make subprojects work nicely in git, but I haven't heard if it ever got off the ground.
Regardless though, koji won't work with what your current SCM layout without modification to the koji code.
While that's true, I'm not sure we're definitely handling things correctly in the git case. The git support in Koji hasn't gotten a lot of testing, and there are a lot of ways to setup a git repo. However, I think that multiple projects in the same repo is probably something we want to support (since we support it in cvs and svn).
The SCM URL you pass to Koji has a format of:
scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier
With git we combine the /path/to/repo and the path/to/module into a single URL we pass to "git clone", essentially collapsing them both into a full module path (this is also what we do with svn). Would it be more appropriate to treat the /path/to/repo as the location of the git repo which we would pass to "git clone", and /path/to/module as a subdirectory in the repo, which we would use as the base directory of the build, and look there for a .spec file?
Yep, that would actually work in our case. Because then I could do the following:
git://git.fedorahosted.org/git/spacewalk.git?web#HEAD
web would contain the .spec file.
On Thu, 2008-07-24 at 09:32 -0400, Jesus Rodriguez wrote:
On Wed, Jul 23, 2008 at 11:51:36PM -0400, Mike Bonnet wrote:
On Wed, 2008-07-23 at 23:06 -0400, Doug Ledford wrote:
On Wed, 2008-07-23 at 22:27 -0400, Jesus Rodriguez wrote:
On Wed, Jul 23, 2008 at 09:50:13PM -0400, Mike McLean wrote:
Jesus M. Rodriguez wrote:
So am I correct in my assumption that koji expects the scm repository to house a single package?
Yes. Furthermore, koji assumes that the simple command 'make srpm' issued from the checkout will create a single source rpm. There is not enough information passed to koji for it to handle anything otherwise.
In your SCM layout with multiple packages, how do you generate the different srpms? Is there a real reason that they are all in one module?
We generate different srpms with the Makefile that exists in each subdirectory. For example, if you want an srpm for the web module cd spacewalk.git/web; make srpm
Want one for the java code, then cd spacewalk.git/java.
With SVN it seems completely doable because I could give the path to repo/spacewalk/java and have koji checkout just that directory. Seems to be a limitation of GIT in this case (or we're using it wrong) :)
Well, git was intentionally written to be basically a state machine of the entire project. So, housing more than one project in a single git repo isn't wrong, but it does tie your various projects together at the state machine level. This is why you can't checkout just one subdirectory from git, you have to grab the entire project. I think I heard mumblings of someone wanting to make subprojects work nicely in git, but I haven't heard if it ever got off the ground.
Regardless though, koji won't work with what your current SCM layout without modification to the koji code.
While that's true, I'm not sure we're definitely handling things correctly in the git case. The git support in Koji hasn't gotten a lot of testing, and there are a lot of ways to setup a git repo. However, I think that multiple projects in the same repo is probably something we want to support (since we support it in cvs and svn).
The SCM URL you pass to Koji has a format of:
scheme://[user@]host/path/to/repo?path/to/module#revision_or_tag_identifier
With git we combine the /path/to/repo and the path/to/module into a single URL we pass to "git clone", essentially collapsing them both into a full module path (this is also what we do with svn). Would it be more appropriate to treat the /path/to/repo as the location of the git repo which we would pass to "git clone", and /path/to/module as a subdirectory in the repo, which we would use as the base directory of the build, and look there for a .spec file?
Yep, that would actually work in our case. Because then I could do the following:
git://git.fedorahosted.org/git/spacewalk.git?web#HEADweb would contain the .spec file.
I've just committed a change to Koji git to support this style of repo setup. The module path (everything between the ? and #) is now considered a subdirectory of the repo, and Koji will expect to find the relevant Makefile and specfile there. Building from the previous repo layout (where the top-level directory contains the Makefile and specfile) can be achieved by leaving empty or omitting the module path, like so:
git://git.fedorahosted.org/git/koji?#a9a82be4 git://git.fedorahosted.org/git/koji#a9a82be4
The two examples above are functionally identical. I've tested these changes against both the Spacewalk and Koji git repos, and they seem to be working as expected. Let me know if you run into problems.
"Jesus M. Rodriguez" jesusr@redhat.com writes:
git://git.fedorahosted.org/git?spacewalk.git#HEAD
But it complains about not having a spec file. Which makes sense as there is no spec file at the top of the tree.
We solved a similar issue (building multiple branches from same sources) by passing some information to make; e.g. see
http://ensc.de/fedora/0007-pass-some-information-to-the-make-srpm-task.patch
'make' extracts then the branch-id out of the scm revision and set e.g. correct %dist.
Should be easy to adapt for your case.
Enrico
buildsys@lists.fedoraproject.org
-
Doug Ledford -
Enrico Scholz -
jesus rodriguez -
Mike Bonnet -
Mike McLean