After upgrading to koji-1.7.1 on my private hub and builders, my builders are no longer able to use the repositories they create when building subsequent dependent packages. When attempting to install locally built packages, the following error is issued in the root.log:
error was [Errno 2] Local file does not exist: /tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/...
The repos are created successfully: $ /usr/bin/createrepo -vd -o /tmp/koji/tasks/9129/19129/repo -i /mnt/koji/repos/f17-build/3627/x86_64/pkglist -g /mnt/koji/repos/f17- build/3627/groups/comps.xml --update --skip-stat /mnt/koji/repos/f17- build/3627/x86_64 Scanning old repo data Spawning worker 0 with 36 pkgs Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-devel-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-libs-2.6.1-3.fc17.1.x86_64.rpm ... ...
I believe the issue is related to the following commit: https://fedorahosted.org/koji/changeset/da6dd0b1832e9929565a1ab03ab844aaceda...
I'm not sure why when building packages, the builders are inserting
'/tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink'
instead of topdir or topurl, such as
'/mnt/koji' or 'http://example.com/kojifiles'
Right now I'm working on selectively reverting that commit to see if I can build packages against other local packages.
Any assistance is appreciated. Thanks. -Anthony
Anthony Messina píše v Ne 02. 12. 2012 v 03:56 -0600:
After upgrading to koji-1.7.1 on my private hub and builders, my builders are
is it upgrade from 1.7.0 or 1.6?
Dan
no longer able to use the repositories they create when building subsequent dependent packages. When attempting to install locally built packages, the following error is issued in the root.log:
error was [Errno 2] Local file does not exist: /tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/...
The repos are created successfully: $ /usr/bin/createrepo -vd -o /tmp/koji/tasks/9129/19129/repo -i /mnt/koji/repos/f17-build/3627/x86_64/pkglist -g /mnt/koji/repos/f17- build/3627/groups/comps.xml --update --skip-stat /mnt/koji/repos/f17- build/3627/x86_64 Scanning old repo data Spawning worker 0 with 36 pkgs Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-devel-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-libs-2.6.1-3.fc17.1.x86_64.rpm ... ...
I believe the issue is related to the following commit: https://fedorahosted.org/koji/changeset/da6dd0b1832e9929565a1ab03ab844aaceda...
I'm not sure why when building packages, the builders are inserting
'/tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink'
instead of topdir or topurl, such as
'/mnt/koji' or 'http://example.com/kojifiles'
Right now I'm working on selectively reverting that commit to see if I can build packages against other local packages.
Any assistance is appreciated. Thanks. -Anthony
-- buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
On Sunday, December 02, 2012 11:01:10 AM Dan Horák wrote:
Anthony Messina píše v Ne 02. 12. 2012 v 03:56 -0600:
After upgrading to koji-1.7.1 on my private hub and builders, my builders are
is it upgrade from 1.7.0 or 1.6?
Dan
This was an upgrade from 1.7.0, previously stable on F17. -A
On Sunday, December 02, 2012 03:56:55 AM Anthony Messina wrote:
After upgrading to koji-1.7.1 on my private hub and builders, my builders are no longer able to use the repositories they create when building subsequent dependent packages. When attempting to install locally built packages, the following error is issued in the root.log:
error was [Errno 2] Local file does not exist: /tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/...
The repos are created successfully: $ /usr/bin/createrepo -vd -o /tmp/koji/tasks/9129/19129/repo -i /mnt/koji/repos/f17-build/3627/x86_64/pkglist -g /mnt/koji/repos/f17- build/3627/groups/comps.xml --update --skip-stat /mnt/koji/repos/f17- build/3627/x86_64 Scanning old repo data Spawning worker 0 with 36 pkgs Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-devel-2.6.1-3.fc17.1.x86_64.rpm Worker 0: reading toplink/packages/dahdi-tools/2.6.1/3.fc17.1/x86_64/dahdi- tools-libs-2.6.1-3.fc17.1.x86_64.rpm ... ...
I believe the issue is related to the following commit: https://fedorahosted.org/koji/changeset/da6dd0b1832e9929565a1ab03ab844aaceda f755
I'm not sure why when building packages, the builders are inserting
'/tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink'
instead of topdir or topurl, such as
'/mnt/koji' or 'http://example.com/kojifiles'
Right now I'm working on selectively reverting that commit to see if I can build packages against other local packages.
Any assistance is appreciated. Thanks. -Anthony
If I manually edit the files to 'revert' this commit (https://fedorahosted.org/koji/changeset/da6dd0b1832e9929565a1ab03ab844aaceda...), builds resume as expected.
Though that's probably not the 'solution.'
-A
On 12/02/2012 04:56 AM, Anthony Messina wrote:
After upgrading to koji-1.7.1 on my private hub and builders, my builders are no longer able to use the repositories they create when building subsequent dependent packages. When attempting to install locally built packages, the following error is issued in the root.log:
error was [Errno 2] Local file does not exist: /tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/...
are you using external repos?
how do you have your /mnt/koji mounts set up?
On Monday, December 03, 2012 03:52:29 PM Mike McLean wrote:
On 12/02/2012 04:56 AM, Anthony Messina wrote:
After upgrading to koji-1.7.1 on my private hub and builders, my builders are no longer able to use the repositories they create when building subsequent dependent packages. When attempting to install locally built packages, the following error is issued in the root.log:
error was [Errno 2] Local file does not exist: /tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/...
are you using external repos?
how do you have your /mnt/koji mounts set up?
I am using external repos. Those appear to be fine. It's the previous builds from Koji itself that seem to not be accessible.
~]$ koji list-external-repos External repo name URL ------------------------- ---------------------------------------- f17-external-repo file:///mnt/koji/mirrors/fedora/linux/releases/17/Everything/$arch/os/ f17-external-repo-rpmfusion-free file:///mnt/koji/mirrors/rpmfusion/free/fedora/releases/17/Everything/$arch/os/ el6-external-repo-centos file:///mnt/koji/mirrors/centos/6/os/$arch/ el6-external-repo-epel file:///mnt/koji/mirrors/epel/6/$arch/ el6-external-repo-centos-updates file:///mnt/koji/mirrors/centos/6/updates/$arch/ f17-external-repo-updates file:///mnt/koji/mirrors/fedora/linux/updates/17/$arch/ f17-external-repo-rpmfusion-free-updates file:///mnt/koji/mirrors/rpmfusion/free/fedora/updates/17/$arch/ f18-external-repo-updates file:///mnt/koji/mirrors/fedora/linux/updates/18/$arch/ f18-external-repo file:///mnt/koji/mirrors/fedora/linux/development/18/$arch/os/ f18-external-repo-rpmfusion-free-updates file:///mnt/koji/mirrors/rpmfusion/free/fedora/updates/18/$arch/
/mntkoji is accessible to my builders via NFSv4 at /mnt/koji
~]# ls -l /mnt/koji total 28 drwxr-xr-x. 10 apache apache 4096 Dec 2 10:36 mash drwxr-xr-x. 6 root root 4096 Oct 2 2011 mirrors drwxr-xr-x. 37 apache apache 4096 Jul 28 23:09 packages drwxr-xr-x. 9 apache apache 4096 Mar 5 2012 repos drwxr-xr-x. 3 apache apache 4096 Dec 5 2010 scratch drwxr-xr-x. 34 apache apache 4096 Jul 28 22:44 sources drwxr-xr-x. 5 apache apache 4096 Dec 5 2010 work
On 12/03/2012 05:05 PM, Anthony Messina wrote:
I am using external repos. Those appear to be fine. It's the previous builds from Koji itself that seem to not be accessible.
It's the merging of external and internal repos that is causing the problem. The internal packages are not being merged correctly. This is what causes the bogus file paths.
I have managed to replicate the issue on a test system.
On Monday, December 03, 2012 05:08:44 PM Mike McLean wrote:
On 12/03/2012 05:05 PM, Anthony Messina wrote:
I am using external repos. Those appear to be fine. It's the previous builds from Koji itself that seem to not be accessible.
It's the merging of external and internal repos that is causing the problem. The internal packages are not being merged correctly. This is what causes the bogus file paths.
I have managed to replicate the issue on a test system.
That's excellent news, in a way. Thanks for your speedy take up of this issue. -A
On Monday, December 03, 2012 04:20:47 PM Anthony Messina wrote:
Signed by amessina@messinet.com. Show Details
On Monday, December 03, 2012 05:08:44 PM Mike McLean wrote:
On 12/03/2012 05:05 PM, Anthony Messina wrote:
I am using external repos. Those appear to be fine. It's the previous builds from Koji itself that seem to not be accessible.
It's the merging of external and internal repos that is causing the problem. The internal packages are not being merged correctly. This is what causes the bogus file paths.
I have managed to replicate the issue on a test system.
That's excellent news, in a way. Thanks for your speedy take up of this issue. -A
This issue is fixed with koji-1.7.1-2.
Thanks. -A
On 01/24/2013 11:23 AM, Anthony Messina wrote:
On Monday, December 03, 2012 04:20:47 PM Anthony Messina wrote:
Signed by amessina@messinet.com. Show Details
On Monday, December 03, 2012 05:08:44 PM Mike McLean wrote:
On 12/03/2012 05:05 PM, Anthony Messina wrote:
I am using external repos. Those appear to be fine. It's the previous builds from Koji itself that seem to not be accessible.
It's the merging of external and internal repos that is causing the problem. The internal packages are not being merged correctly. This is what causes the bogus file paths.
I have managed to replicate the issue on a test system.
That's excellent news, in a way. Thanks for your speedy take up of this issue. -A
This issue is fixed with koji-1.7.1-2.
Well, worked around in Fedora's build. I don't have a proper fix yet. It looks like it will require a significant rework of koji's mergerepos script.
Hi list,
Updated to koji-1.8.0-1 over here.
Koji repos reference local package URLs under the /kojifiles directory. Since the previous version installed here, symlinks named 'toplink' have been placed in generated repos' top level directories. The symlink is included package URL paths, and without 'FollowSymLinks' added to Apache's /kojifiles directory options, package URLs break.
Of course the Apache 'FollowSymLinks' behavior is disabled by default because of security implications. There may be a more secure configuration, but we're not worried about that in our environment. If others don't chime in with advice for more secure options, do your homework when security is a consideration in your environment!
The line in question in the provided koji-hub httpd.conf example:
https://git.fedorahosted.org/cgit/koji/tree/hub/httpd.conf?id=da6dd0b1832e99...
Thanks-
John
On 06/15/2013 02:39 PM, John Morris wrote:
Hi list,
Updated to koji-1.8.0-1 over here.
Koji repos reference local package URLs under the /kojifiles directory. Since the previous version installed here, symlinks named 'toplink' have been placed in generated repos' top level directories. The symlink is included package URL paths, and without 'FollowSymLinks' added to Apache's /kojifiles directory options, package URLs break.
Using toplink allows koji to create yum repos without a fixed baseurl (rpm paths remain relative to the repo itself). This makes koji's repos much more sane.
Of course the Apache 'FollowSymLinks' behavior is disabled by default because of security implications. There may be a more secure configuration, but we're not worried about that in our environment. If others don't chime in with advice for more secure options, do your homework when security is a consideration in your environment!
Actually, FollowSymlinks is the default in the current apache httpd. http://httpd.apache.org/docs/current/mod/core.html#options
""" FollowSymLinks The server will follow symbolic links in this directory. This is the default setting. """
This may be new in 2.4.
The line in question in the provided koji-hub httpd.conf example:
https://git.fedorahosted.org/cgit/koji/tree/hub/httpd.conf?id=da6dd0b1832e99...
Thanks-
John
buildsys mailing list buildsys@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/buildsys
On 06/18/2013 11:03 PM, Mike McLean wrote:
On 06/15/2013 02:39 PM, John Morris wrote:
Hi list,
Updated to koji-1.8.0-1 over here.
Koji repos reference local package URLs under the /kojifiles directory. Since the previous version installed here, symlinks named 'toplink' have been placed in generated repos' top level directories. The symlink is included package URL paths, and without 'FollowSymLinks' added to Apache's /kojifiles directory options, package URLs break.
Using toplink allows koji to create yum repos without a fixed baseurl (rpm paths remain relative to the repo itself). This makes koji's repos much more sane.
Of course the Apache 'FollowSymLinks' behavior is disabled by default because of security implications. There may be a more secure configuration, but we're not worried about that in our environment. If others don't chime in with advice for more secure options, do your homework when security is a consideration in your environment!
Actually, FollowSymlinks is the default in the current apache httpd. http://httpd.apache.org/docs/current/mod/core.html#options
""" FollowSymLinks The server will follow symbolic links in this directory. This is the default setting. """
This may be new in 2.4.
You nailed it!
We're on el6 over here (can't afford the upgrading effort at the frequency of Fedora releases) with apache 2.2. The same docs for 2.2 omit that statement about the default:
https://httpd.apache.org/docs/2.2/mod/core.html#options
I must have come across this before, since the 'latest' links would have broken too. My original config used 'SymLinksIfOwnerMatch'. This broke with the 'topdir' links because our top-level koji directory was owned by root, not apache.
As for the security question, it says it right there: symlinks are potentially insecure even when these options are omitted.
Thanks, Mike.
John
On 12/02/2012 04:56 AM, Anthony Messina wrote:
After upgrading to koji-1.7.1 on my private hub and builders, my builders are no longer able to use the repositories they create when building subsequent dependent packages. When attempting to install locally built packages, the following error is issued in the root.log:
error was [Errno 2] Local file does not exist: /tmp/koji/tasks/9129/19129/repo_3627_premerge/toplink/packages/...
...
I believe the issue is related to the following commit: https://fedorahosted.org/koji/changeset/da6dd0b1832e9929565a1ab03ab844aaceda...
It appears this change breaks external repos. Your reversion should be a safe workaround until I can get a fix out.
buildsys@lists.fedoraproject.org