On Tue, 27 Jan 2015 10:29:23 -0600
Michael Cronenworth <mike(a)cchtml.com> wrote:
On 01/26/2015 09:46 AM, Kevin Fenzi wrote:
> I think we added the Crawl-delay several years ago when we were
> having storage issues. We could definitely try removing it and see
> if things improve.
10 seconds may be on the high side, but you may still want to keep it
and lower it to 1 to 5 seconds. Bing Bot hits my sites like it's
running a benchmark.
Ok. I changed it to 1. Should sync out over the next little while...
We can see if it makes any difference.