[Koha] Help! Software Error

[Paul A]

At 01:13 PM 6/3/2015 +1200, Robin Sheat wrote:
>Paul A schreef op di 02-06-2015 om 19:48 [-0400]:
> > How do you "slow it down"? To the best of my knowledge it's an on/off
> > choice. See
> > 
> <https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt>.
>
>https://en.wikipedia.org/wiki/Robots_exclusion_standard#Crawl-delay_directive
>
>PS: no need to CC me in replies, I'm on the mailing list.

Robin -- thanks (and apologies for the unintended Cc, end of a long day, etc).

This was specifically for Google, so I had a look at the wiki article and 
did some more digging. It seems that at least Bing and Yahoo (neither of 
whom appear to abuse our servers) support the Crawl-Delay, but Google does 
not, they have a differing method explained at 
<https://support.google.com/webmasters/answer/48620?hl=en>.

I'll have to look more closely, but it may not be quite what I'm looking 
for as it involves their proprietorial "webmaster tools" rather than robots.txt

Best -- Paul