On Tue, Nov 25, 2014 at 07:24:32AM -0700, Kevin Fenzi wrote:
> The OOM killer on bapp02 has terminated a few mirrormanager
crawler
> processes. It seems it needs more memory or the number of parallel
> crawlers has to be further limited.
Well, it's got 16GB now... I can bump it to 24 without too much
trouble. Will of course need a freeze break...
We are currently doing 60 threads. We could cut it down, but I guess
I'd say lets try more memory first.
> Another good idea would be to limit the duration of the rsync crawls
> in /etc/mirrormanager/prod.cfg to maybe one day (--timeout=86400) to
> avoid stale rsync processes
Good idea. Whats the config directive there?
Seems we are not setting it at all currently.
I just found out that the crawler is not configurable. I was thinking
about update-master-directory-list.
To add a timeout to the rsync crawls it would be
necessary to add it directly to /usr/share/mirrormanager/server/crawler_perhost:497
So maybe something for the mirrormanager rewrite.
Adrian