High CPU usage, slow OPAC/Staff interface, with memcache
I am running Koha 24.05.03.000 on Ubuntu 24.04 with 8 vCPU cores and 32 GB of RAM. Recently I found that the VPS was running at 100% CPU usage and discovered that memcache was not running. The high CPU usage would cause it to crash. To solve this, I started memcache and confirmed it was running and followed the instructions at https://wiki.koha-community.org/wiki/Koha_Tuning_Guide. It seemed to run better for a short time. But it is back up to 100% usage, which slows it down considerably. When I run top to see what is using the CPU, it shows many opac-search.pl running on many processes. Is that normal? Are there other things I can do to reign in CPU usage? Daniel
Hi Daniel Have you checked apache logs? It may be a crawl robot that is indexing your system Regards, Alvaro |----------------------------------------------------------------------------------------| Stay safe / Cuídate/ Reste sécurisé *7* Switch off as you go / Apaga lo que no usas / Débranchez au fur et à mesure. *q *Recycle always / Recicla siempre / Recyclez toujours P Print only if absolutely necessary / Imprime solo si es necesario / Imprimez seulement si nécessaire Le jeu. 15 août 2024 à 15:27, Daniel Owens <dcowens76@gmail.com> a écrit :
I am running Koha 24.05.03.000 on Ubuntu 24.04 with 8 vCPU cores and 32 GB of RAM.
Recently I found that the VPS was running at 100% CPU usage and discovered that memcache was not running. The high CPU usage would cause it to crash. To solve this, I started memcache and confirmed it was running and followed the instructions at https://wiki.koha-community.org/wiki/Koha_Tuning_Guide. It seemed to run better for a short time. But it is back up to 100% usage, which slows it down considerably. When I run top to see what is using the CPU, it shows many opac-search.pl running on many processes. Is that normal? Are there other things I can do to reign in CPU usage?
Daniel
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Thank you for responding, Alvaro. I hadn't recently checked apache logs. I had been checking the apache error logs to see why apache was shutting down. But that seems to have been resolved with memcache. But the CPU issue remains. I am guessing you mean the access log? I don't see much there. But maybe I am looking in the wrong place: sudo tail -f /var/log/apache2/access.log If I find the right place, am I looking for a single IP that is accessing Koha repeatedly? Daniel On 8/15/24 3:48 PM, Alvaro Cornejo wrote:
Hi Daniel
Have you checked apache logs? It may be a crawl robot that is indexing your system
Regards,
Alvaro
|----------------------------------------------------------------------------------------| Stay safe / Cuídate/ Reste sécurisé */7/*Switch off as you go / Apaga lo que no usas / Débranchez au fur et à mesure. *q *Recycle always / Recicla siempre / Recyclez toujours PPrint only if absolutely necessary / Imprime solo si es necesario / Imprimez seulement si nécessaire
Le jeu. 15 août 2024 à 15:27, Daniel Owens <dcowens76@gmail.com> a écrit :
I am running Koha 24.05.03.000 on Ubuntu 24.04 with 8 vCPU cores and 32 GB of RAM.
Recently I found that the VPS was running at 100% CPU usage and discovered that memcache was not running. The high CPU usage would cause it to crash. To solve this, I started memcache and confirmed it was running and followed the instructions at https://wiki.koha-community.org/wiki/Koha_Tuning_Guide. It seemed to run better for a short time. But it is back up to 100% usage, which slows it down considerably. When I run top to see what is using the CPU, it shows many opac-search.pl <http://opac-search.pl> running on many processes. Is that normal? Are there other things I can do to reign in CPU usage?
Daniel
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Hi, Yes, the idea is to identify who is calling the opac-search.pl. The opac-search.pl is not a service, therefore "someone" needs to run it. Unless there is something that put it in a sort of endless loop or it hungs up Hope helps, Alvaro Le ven. 16 août 2024, 16:09, Daniel Owens <dcowens76@gmail.com> a écrit :
Thank you for responding, Alvaro.
I hadn't recently checked apache logs. I had been checking the apache error logs to see why apache was shutting down. But that seems to have been resolved with memcache.
But the CPU issue remains. I am guessing you mean the access log? I don't see much there. But maybe I am looking in the wrong place: sudo tail -f /var/log/apache2/access.log
If I find the right place, am I looking for a single IP that is accessing Koha repeatedly?
Daniel On 8/15/24 3:48 PM, Alvaro Cornejo wrote:
Hi Daniel
Have you checked apache logs? It may be a crawl robot that is indexing your system
Regards,
Alvaro
|----------------------------------------------------------------------------------------| Stay safe / Cuídate/ Reste sécurisé *7* Switch off as you go / Apaga lo que no usas / Débranchez au fur et à mesure. *q *Recycle always / Recicla siempre / Recyclez toujours P Print only if absolutely necessary / Imprime solo si es necesario / Imprimez seulement si nécessaire
Le jeu. 15 août 2024 à 15:27, Daniel Owens <dcowens76@gmail.com> a écrit :
I am running Koha 24.05.03.000 on Ubuntu 24.04 with 8 vCPU cores and 32 GB of RAM.
Recently I found that the VPS was running at 100% CPU usage and discovered that memcache was not running. The high CPU usage would cause it to crash. To solve this, I started memcache and confirmed it was running and followed the instructions at https://wiki.koha-community.org/wiki/Koha_Tuning_Guide. It seemed to run better for a short time. But it is back up to 100% usage, which slows it down considerably. When I run top to see what is using the CPU, it shows many opac-search.pl running on many processes. Is that normal? Are there other things I can do to reign in CPU usage?
Daniel
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Daniel, if you use plack, check your access logs with: sudo tail -f /var/log/koha/YOURKOHASITE/plack.log or sudo grep opac-search /var/log/koha/YOURSITE/plack.log | more search the list archives, there was a discussion a few weeks back about this, and ways to use fail2ban to help with bots that fail to care about robots.txt On 8/16/24 15:09, Daniel Owens wrote:
Thank you for responding, Alvaro.
I hadn't recently checked apache logs. I had been checking the apache error logs to see why apache was shutting down. But that seems to have been resolved with memcache.
But the CPU issue remains. I am guessing you mean the access log? I don't see much there. But maybe I am looking in the wrong place: sudo tail -f /var/log/apache2/access.log
If I find the right place, am I looking for a single IP that is accessing Koha repeatedly?
Daniel
On 8/15/24 3:48 PM, Alvaro Cornejo wrote:
Hi Daniel
Have you checked apache logs? It may be a crawl robot that is indexing your system
Regards,
Alvaro
|----------------------------------------------------------------------------------------|
Stay safe / Cuídate/ Reste sécurisé */7/*Switch off as you go / Apaga lo que no usas / Débranchez au fur et à mesure. *q *Recycle always / Recicla siempre / Recyclez toujours PPrint only if absolutely necessary / Imprime solo si es necesario / Imprimez seulement si nécessaire
Le jeu. 15 août 2024 à 15:27, Daniel Owens <dcowens76@gmail.com> a écrit :
I am running Koha 24.05.03.000 on Ubuntu 24.04 with 8 vCPU cores and 32 GB of RAM.
Recently I found that the VPS was running at 100% CPU usage and discovered that memcache was not running. The high CPU usage would cause it to crash. To solve this, I started memcache and confirmed it was running and followed the instructions at https://wiki.koha-community.org/wiki/Koha_Tuning_Guide. It seemed to run better for a short time. But it is back up to 100% usage, which slows it down considerably. When I run top to see what is using the CPU, it shows many opac-search.pl <http://opac-search.pl> running on many processes. Is that normal? Are there other things I can do to reign in CPU usage?
Daniel
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
-- Hector Gonzalez cacho@genac.org
Hi! Den 16.08.2024 23:09, skrev Daniel Owens:
Thank you for responding, Alvaro.
I hadn't recently checked apache logs. I had been checking the apache error logs to see why apache was shutting down. But that seems to have been resolved with memcache.
But the CPU issue remains. I am guessing you mean the access log? I don't see much there. But maybe I am looking in the wrong place: sudo tail -f /var/log/apache2/access.log
If you used the Debian packages to install Koha there will be dedicated logs for each instance. Have a look in /var/log/koha/<your instance name>/ There will be logs there both for Plack and Apache. Best regards, Magnus Enger Libriotech
On 8/19/24 1:06 AM, Magnus Enger wrote:
Hi!
Den 16.08.2024 23:09, skrev Daniel Owens:
Thank you for responding, Alvaro.
I hadn't recently checked apache logs. I had been checking the apache error logs to see why apache was shutting down. But that seems to have been resolved with memcache.
But the CPU issue remains. I am guessing you mean the access log? I don't see much there. But maybe I am looking in the wrong place: sudo tail -f /var/log/apache2/access.log
If you used the Debian packages to install Koha there will be dedicated logs for each instance.
Have a look in /var/log/koha/<your instance name>/
There will be logs there both for Plack and Apache.
Best regards, Magnus Enger Libriotech
Excellent, thanks for this. Daniel
Hi Daniel, I agree with Alvaro's suggestion. We recently had the same problem (and the same script opac-search.pl taking up all the CPU). A look through the apache log files showed very frequent accesses by a number of bots. I put a robots.txt file at the top level of the site and the problem went away. We did notice that it takes time for the bots to notice the file so it took a day or so for the load to really fall back. Hope this helps Chris Brown On Thu, Aug 15, 2024 at 9:26 PM Daniel Owens <dcowens76@gmail.com> wrote:
I am running Koha 24.05.03.000 on Ubuntu 24.04 with 8 vCPU cores and 32 GB of RAM.
Recently I found that the VPS was running at 100% CPU usage and discovered that memcache was not running. The high CPU usage would cause it to crash. To solve this, I started memcache and confirmed it was running and followed the instructions at https://wiki.koha-community.org/wiki/Koha_Tuning_Guide. It seemed to run better for a short time. But it is back up to 100% usage, which slows it down considerably. When I run top to see what is using the CPU, it shows many opac-search.pl running on many processes. Is that normal? Are there other things I can do to reign in CPU usage?
Daniel
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
participants (5)
-
Alvaro Cornejo -
Chris Brown -
Daniel Owens -
Hector Gonzalez Jaime -
Magnus Enger