Dear friends We have six libraries running Koha 3.00.06. The system runs Redhat, with 6 virtual Xen servers. Problem #1 From time to time, at various times of day one or more of the Koha systems will slow down and stop. We can watch the CPU graph on the virtual machine manager - it usually drops to less than 1% use for a few minutes. At this point we have librarians staring at unresponsive Koha screens - connecting... connecting... The virtual server desktop is unresponsive even at the physical server. We cannot connect with a VNC client or browse in. This is very frustrating when the CPU graph implies that the server is apparently not busy. Then the CPU use will ramp rapidly up to 12.4% use (which is pretty much full capacity on our virtual server setup). It stays there - and although it may have recovered once or twice from this, usually we have to kill the server. Sometimes we force the VM to shut down, and then restart it, and can see zebra searches happening with Top. However we may still be unable to log in with VNC or connect from local client browsers. This is really difficult to track down because we have not found associated error messages, and by the time we get to them, the servers seem to be showing low load, but we may be missing some initiating event. That is problem #1 Here is problem #2 We have also noticed that opac-search.pl and mysqld is causing heavy loading. Within the virtual server we have multiple cpus and the load is shared between them, However, often opac-search.pl is around 30%, and mysqld alone is using close to half of the virtual server's CPU capacity. The records we have in this single site library do not have large numbers of items attached to any one biblio record. There was a discussion in March about something like this a year ago, but I can't see a resolution. Does anyone have any ideas on either of these issues? Hoping someone from the developer group might have some ideas. Agnes -- CEO/ Chief Librarian Hanover Public Library. 451 10th Ave, Hanover, ON N4N 2P1 Tel: 519-364-1420 Email: arm@hanover.ca Please consider the environment before printing this message or attachments. Support your library! Please consider making a donation, joining the Friends of the Library, or remembering us in your will. -- This email was Anti Virus checked by Astaro Security Gateway. http://www.astaro.com
Le 29/02/2012 01:00, Agnes Rivers-Moore a écrit :
Dear friends That is problem #1
Here is problem #2 We have also noticed that opac-search.pl and mysqld is causing heavy loading.
Since a few days/week, we (BibLibre) face the same problem, on various Koha : on our hosting server (20+ libraries), some of our libraries that have their own server, small or large. After investigating a lot, we think it's because something happened with web crawler : we suddenly have *a lot* of queries coming from googlebot, bingbot and baidu (the chinese search engine). Plus some dark-net bots that try to exploit some security holes that are fixed (like reading /etc/password with a ../../../../../etc/password), but in case this Koha is not uptodate that could work. How did we fix that ? quick fix = drop packet at firewall level. We will have to investigate for a more long-term fix (not sure a robot.txt will be enough, but not sure it won't too) -- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08
I had something similar happen in November and the robot.txt resolved it for me at that time. -- David Schuster Library Technology Coordinator Plano ISD ---- Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 29/02/2012 01:00, Agnes Rivers-Moore a écrit :
Dear friends That is problem #1
Here is problem #2 We have also noticed that opac-search.pl and mysqld is causing heavy loading.
Since a few days/week, we (BibLibre) face the same problem, on various Koha : on our hosting server (20+ libraries), some of our libraries that have their own server, small or large.
After investigating a lot, we think it's because something happened with web crawler : we suddenly have *a lot* of queries coming from googlebot, bingbot and baidu (the chinese search engine). Plus some dark-net bots that try to exploit some security holes that are fixed (like reading /etc/password with a ../../../../../etc/password), but in case this Koha is not uptodate that could work.
How did we fix that ? quick fix = drop packet at firewall level. We will have to investigate for a more long-term fix (not sure a robot.txt will be enough, but not sure it won't too)
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
This has happened to us several times now. Once I recall was the Baidu web crawler, but also other .cn crawlers. It also happened to us a couple of years ago when we were still on Unicorn. ByWater's support has fixed it with robot.txt and by blocking some ports (TNX!). Frankly, if it keeps happening (seems to be fixed for now) I will probably just drop .cn at the firewall. Greg ------------------------------ On 03/07/2012 10:36 AM, dschust@tx.rr.com wrote:
I had something similar happen in November and the robot.txt resolved it for me at that time. -- David Schuster Library Technology Coordinator Plano ISD
---- Paul Poulain <paul.poulain@biblibre.com> wrote:
Le 29/02/2012 01:00, Agnes Rivers-Moore a écrit :
Dear friends That is problem #1
Here is problem #2 We have also noticed that opac-search.pl and mysqld is causing heavy loading. Since a few days/week, we (BibLibre) face the same problem, on various Koha : on our hosting server (20+ libraries), some of our libraries that have their own server, small or large.
After investigating a lot, we think it's because something happened with web crawler : we suddenly have *a lot* of queries coming from googlebot, bingbot and baidu (the chinese search engine). Plus some dark-net bots that try to exploit some security holes that are fixed (like reading /etc/password with a ../../../../../etc/password), but in case this Koha is not uptodate that could work.
How did we fix that ? quick fix = drop packet at firewall level. We will have to investigate for a more long-term fix (not sure a robot.txt will be enough, but not sure it won't too)
-- Paul POULAIN http://www.biblibre.com Expert en Logiciels Libres pour l'info-doc Tel : (33) 4 91 81 35 08 _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
-- Greg Lawson Rolling Hills Consolidated Library 1912 N. Belt Highway St. Joseph, MO 64506 816-232-5479
Koha OPAC indexing by crawler can slow down the server, and, at the end, the whole catalog isn't indexed. A solution, for webmaster, could be to use this tool: https://metacpan.org/module/koha-sitemap Installation (by sysadmin): cpan Koha::Contrib::Tamil How to use it: perldoc koha-sitemap More details on sitemaps principle here: http://www.sitemaps.org
participants (5)
-
Agnes Rivers-Moore -
dschust@tx.rr.com -
Frédéric Demians -
G. Laws -
Paul Poulain