[Koha] Block web crawlers Bots

[Fridolin SOMERS]

Hi,

We use syntax :

User-agent: *
Disallow: /

I don't know if it has same results.

Note it make take some days to get out of bots scope.

If you have an Nginx reverse proxy, you may use
https://github.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker

Best regards,

Le 27/01/2022 à 19:04, JITHIN N a écrit :
> My server memory usage is increasing by some web search engine bots by
> access of OPAC. I tried put
> 
> *robots.txt in *
> 
> on /usr/share/koha/opac/htdocs
> 
> As
> 
> *User-agent: **
> 
> *Disallow: /*
> 
> But still, my server is getting loaded by bots. The apache log file shows
> bots names like petalbot, Googlebot, AhrefsBot etc. How can I block all
> these bots' access to my OPAC pages?
> 
> With Regards
> 
> Jithin N
> 
> *System Information*
> 
> Koha 21.05
> 
> Deian 10
> _______________________________________________
> 
> Koha mailing list  http://koha-community.org
> Koha at lists.katipo.co.nz
> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

-- 
Fridolin SOMERS <fridolin.somers at biblibre.com>
Software and system maintainer 🦄
BibLibre, France