This has happened to us several times now. Once I recall was the Baidu web crawler, but also other .cn crawlers. It also happened to us a couple of years ago when we were still on Unicorn. ByWater's support has fixed it with robot.txt and by blocking some ports (TNX!). Frankly, if it keeps happening (seems to be fixed for now) I will probably just drop .cn at the firewall. Greg ------------------------------ On 03/07/2012 10:36 AM, dschust@tx.rr.com wrote:
I had something similar happen in November and the robot.txt resolved it for me at that time. -- David Schuster Library Technology Coordinator Plano ISD
---- Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 29/02/2012 01:00, Agnes Rivers-Moore a écrit :
Dear friends That is problem #1
Here is problem #2 We have also noticed that opac-search.pl and mysqld is causing heavy loading. Since a few days/week, we (BibLibre) face the same problem, on various Koha : on our hosting server (20+ libraries), some of our libraries that have their own server, small or large.
After investigating a lot, we think it's because something happened with web crawler : we suddenly have *a lot* of queries coming from googlebot, bingbot and baidu (the chinese search engine). Plus some dark-net bots that try to exploit some security holes that are fixed (like reading /etc/password with a ../../../../../etc/password), but in case this Koha is not uptodate that could work.
How did we fix that ? quick fix = drop packet at firewall level. We will have to investigate for a more long-term fix (not sure a robot.txt will be enough, but not sure it won't too)
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
-- Greg Lawson Rolling Hills Consolidated Library 1912 N. Belt Highway St. Joseph, MO 64506 816-232-5479