[Koha] Koha slowed down by Google indexing?!

[Hugo Agud]

Hi

Yes this is annoying issue with boots, this is google but there are plenty
of them...

You should use robots.txt propertly, but If I am not wrong with Google it
is more effective go to google webmaster web and modify the googleboot
behaviour with your koha installarion
You should also use a koha-sitemap.. depending on the version is out of the
box functionality
Perhaps you may think on use ufw or even ufw + fail2ban

Some times bots are nightmare

2017-05-03 13:49 GMT+02:00 Mark Alexander <marka at pobox.com>:

> > When I searched for who is 66.249.64.32 I saw this IP addresse belongs
> > to Google.
>
> This does seem to be the Google indexer:
>
>   % nslookup 66.249.64.32
>   ...
>   32.64.249.66.in-addr.arpa     name = crawl-66-249-64-32.googlebot.com.
>
> I haven't seen this problem (yet), but perhaps that is because I have
> a /usr/share/koha/opac/htdocs/robots.txt containing this:
>
> Crawl-delay: 60
>
> User-agent: *
> Disallow: /
>
> User-agent: Googlebot
> Disallow: /cgi-bin/koha/opac-search.pl
> Disallow: /cgi-bin/koha/opac-showmarc.pl
> Disallow: /cgi-bin/koha/opac-detailprint.pl
> Disallow: /cgi-bin/koha/opac-ISBDdetail.pl
> Disallow: /cgi-bin/koha/opac-MARCdetail.pl
> Disallow: /cgi-bin/koha/opac-reserve.pl
> Disallow: /cgi-bin/koha/opac-export.pl
> Disallow: /cgi-bin/koha/opac-detail.pl
> Disallow: /cgi-bin/koha/opac-authoritiesdetail.pl
> _______________________________________________
> Koha mailing list  http://koha-community.org
> Koha at lists.katipo.co.nz
> https://lists.katipo.co.nz/mailman/listinfo/koha
>



-- 

*Hugo Agud - Orex Digital *

*www.orex.es <http://www.orex.es>*


<http://www.orex.es/>    [image: www.orex.es/koha] <http://www.orex.es/koha>
   [image: www.orex.es/vufind] <http://www.orex.es/vufind>
<http://www.orex.es/omeka>


Director

Calle Sant Joaquin,117, 2º-3ª · 08922 Santa Coloma de Gramanet - Tel: 933
856 138   hagud at orex.es · http://www.orex.es/



No imprima este mensaje a no ser que sea necesario. Una tonelada de papel
implica la tala de 15 árboles y el consumo de 250.000 litros de agua.



Aviso de confidencialidad
Este mensaje contiene información que puede ser CONFIDENCIAL y/o de USO
RESTRINGIDO. Si usted no es el receptor deseado del mensaje (ni
está autorizado a recibirlo por el remitente), no está autorizado a copiar,
reenviar o divulgar el mensaje o su contenido. Si ha recibido este mensaje
por error, por favor, notifíquenoslo inmediatamente y bórrelo de su sistema.