On Tue, Feb 25, 2014 at 09:16:19AM -0500, Konstantin Ryabitsev wrote:
On Tue, Feb 25, 2014 at 5:53 AM, Pierre-Yves Chibon
<pingou(a)pingoured.fr>
wrote:
grok-dumb-pull seems to be the way to go in the current environment.
However I
have a couple of questions:
- has there been any thoughts on trying to get grokmirror included in
github as
one of the hooks they provide by default? This would reduce their
bandwith
usage for all the project that, like us, would like to mirror their
github
repo elsewhere.
I've not approached them for anything like this, but I would imagine a
single manifest file would not work at all for all of github's
repositories. The largest collection we currently manage with grokmirror
is 5,500 repositories and though it does admirably well, it's getting to
the point where parsing/writing the manifest file is taking upwards of a
second. They could probably generate a manifest file per user, but not a
single manifest for all the repositories they host.
Per-user or per-organization might already just do the job. Tbh, I was even
thinking of a per-project manifest.
- One thing that isn't clear to me is how the manifest.js is
made
available. It
is just provided at a specified URL or is it in fact included in the
git repo?
It's made available outside git repositories as a simple http download.
This way we can make use of an extremely lightweight HTTP with
"if-newer-than" header and bail out early if the remote manifest hasn't
changed.
So, in theory, it should be duable for github, might as simple as
https://github.com/fedora-infra/fedocal/manifest where they have
https://github.com/fedora-infra/fedocal/branches now.
Food for thoughts I guess :)
Thanks for the feedbacks!
Pierre