[Koha] Koha slowed down by Google indexing?!

[Mark Alexander]

> When I searched for who is 66.249.64.32 I saw this IP addresse belongs
> to Google.

This does seem to be the Google indexer:

  % nslookup 66.249.64.32
  ...
  32.64.249.66.in-addr.arpa	name = crawl-66-249-64-32.googlebot.com.

I haven't seen this problem (yet), but perhaps that is because I have
a /usr/share/koha/opac/htdocs/robots.txt containing this:

Crawl-delay: 60

User-agent: *
Disallow: /

User-agent: Googlebot
Disallow: /cgi-bin/koha/opac-search.pl
Disallow: /cgi-bin/koha/opac-showmarc.pl
Disallow: /cgi-bin/koha/opac-detailprint.pl
Disallow: /cgi-bin/koha/opac-ISBDdetail.pl
Disallow: /cgi-bin/koha/opac-MARCdetail.pl
Disallow: /cgi-bin/koha/opac-reserve.pl
Disallow: /cgi-bin/koha/opac-export.pl
Disallow: /cgi-bin/koha/opac-detail.pl
Disallow: /cgi-bin/koha/opac-authoritiesdetail.pl