[Koha] Help! Software Error

Paul A paul.a at navalmarinearchive.com
Wed Jun 3 11:48:22 NZST 2015


At 11:30 AM 6/3/2015 +1200, Robin Sheat wrote:
>Paul A schreef op di 02-06-2015 om 09:27 [-0400]:
> > And I must admit that "google is getting my goat."  Woodpeckering may
> > be a
> > fact of modern life, but at several million entries per day, there's
> > got to
> > be a limit.
>
>Use robots.txt to slow down its crawl rate. Some crawlers don't obey
>that, but google does.

How do you "slow it down"? To the best of my knowledge it's an on/off 
choice. See 
<https://developers.google.com/webmasters/control-crawl-index/docs/robots_txt>.

tnx and br -- p.



More information about the Koha mailing list