On Oct 7, 2011, at 2:42 AM, Petr Pisar wrote:
On 2011-10-06, Jesse Keating <jkeating(a)j2solutions.net> wrote:
> On Oct 5, 2011, at 11:27 PM, Petr Pisar wrote:
>>
>> I've written an ultimate heavy-parallel rebuilding tool. (Actually it's
>> so much parallel that Fedora infrustructure, git repositories namely,
>> spontaneously fails.) It's packaged in `perl-Fedora-Rebuild' package,
>> there is sample executable `rebuildperl'. Currently it's
>> under-documented, not optimized and with some internal bugs. But I will
>> develop it more because we need it for each year Perl rebuild, so the
>> tool will improve.
>
> What failure did you create, and was there a ticket about this? If
> you're able to create a failure with load I'm VERY interested in this.
>
The job was like `git pull && rpmdev-bump && git commit && git
push' run
on already cloned repository. We configured SSH client to use
auto-closing master connnection and when we run the job in more threads
we got errors like unexpected SSH session close by remote (probably
after timing out gitolite or what Fedora uses behind the SSH).
We found by experiments the `more threads' is more than 6 (or 8,
I can't remeber now).
Marcela asked on #fedora-admin whether there was some limit set on the
server site but nobody could confirm or deny it.
Ok, that's strange because I've done very similar with the mass rebuild script I
have and I was not able to create any sort of noticeable load on the server. Certainly
does sound like an ssh issue though.
Might have gone quicker if you pull via git:// and then only push via ssh:// reducing your
ssh handshakes by half.
- jlk