Carlos, I think that web crawler (Google, MS, and other) are querying your Koha. The following robots.txt file will prevent them to search your catalogue.

User-agent: * Disallow:/cgi-bin/koha/opac-search.pl Disallow:/cgi-bin/koha/opac-export.pl Disallow:/cgi-bin/koha/opac-showmarc.pl Disallow:/cgi-bin/koha/opac-ISBDdetail.pl Disallow:/cgi-bin/koha/opac-MARCdetail.pl

Cheers, Eric On 2014-08-13 11:28, Carlos Rodrigo Cordova Sandoval wrote:

Thanks Tomas,

I send the command output.

http://pastebin.com/Xk22B3Yd

-Carlos +569 6668 32 94

2014-08-13 10:57 GMT-04:00 Tomas Cohen Arazi <tomascohen@gmail.com>:

...

Maybe you have a IO bottleneck (thinking of that mysql cpu usage). How many simultaneous connections do u havE? (netstat)

On Wed, Aug 13, 2014 at 11:52 AM, Carlos Rodrigo Cordova Sandoval < carlos.cordovas@gmail.com> wrote:

...

Hi Friends,

have a problem with our koha 3.16.02 installed on debian 64bit 7x and has 45,000 records and 157,000 items, our koha uses 100% of cpu. view image.

http://snag.gy/xWIXP.jpg

and optimize mysql, apache and memcache activated according to wiki http://wiki.koha-community.org/wiki/Koha_Tuning_Guide

the features of the server are: Processor: 4 core Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz RAM: 6GB

thank you very much.

-Carlos _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

-- Tomás Cohen Arazi Prosecretaría de Informática Universidad Nacional de Córdoba ✆ +54 351 5353750 ext 13168 GPG: B76C 6E7C 2D80 551A C765 E225 0A27 2EA1 B2F3 C15F

_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

any other suggestions :) -Carlos +569 6668 32 94 2014-08-13 12:09 GMT-04:00 Carlos Rodrigo Cordova Sandoval < carlos.cordovas@gmail.com>:

Thank you very much, we have already blocked the bots. Even the cpu is at 100%

Carlos El 13/08/2014 11:53, "Eric Bégin" <eric.begin@inlibro.com> escribió:

Carlos,

...

I think that web crawler (Google, MS, and other) are querying your Koha.

The following robots.txt file will prevent them to search your catalogue.

User-agent: * Disallow:/cgi-bin/koha/opac-search.pl Disallow:/cgi-bin/koha/opac-export.pl Disallow:/cgi-bin/koha/opac-showmarc.pl Disallow:/cgi-bin/koha/opac-ISBDdetail.pl Disallow:/cgi-bin/koha/opac-MARCdetail.pl

Cheers,

Eric

On 2014-08-13 11:28, Carlos Rodrigo Cordova Sandoval wrote:

Thanks Tomas,

I send the command output. http://pastebin.com/Xk22B3Yd

-Carlos +569 6668 32 94

2014-08-13 10:57 GMT-04:00 Tomas Cohen Arazi <tomascohen@gmail.com> <tomascohen@gmail.com>:

Maybe you have a IO bottleneck (thinking of that mysql cpu usage). How many simultaneous connections do u havE? (netstat)

On Wed, Aug 13, 2014 at 11:52 AM, Carlos Rodrigo Cordova Sandoval <carlos.cordovas@gmail.com> wrote:

Hi Friends,

have a problem with our koha 3.16.02 installed on debian 64bit 7x and has 45,000 records and 157,000 items, our koha uses 100% of cpu. view image. http://snag.gy/xWIXP.jpg

and optimize mysql, apache and memcache activated according to wikihttp://wiki.koha-community.org/wiki/Koha_Tuning_Guide

the features of the server are: Processor: 4 core Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz RAM: 6GB

thank you very much.

-Carlos _______________________________________________ Koha mailing list http://koha-community.orgKoha@lists.katipo.co.nzhttp://lists.katipo.co.nz/ma...

-- Tomás Cohen Arazi Prosecretaría de Informática Universidad Nacional de Córdoba ✆ +54 351 5353750 ext 13168 GPG: B76C 6E7C 2D80 551A C765 E225 0A27 2EA1 B2F3 C15F

_______________________________________________ Koha mailing list http://koha-community.orgKoha@lists.katipo.co.nzhttp://lists.katipo.co.nz/ma...

Si lo recupera. http://biblioteca.udd.cl/robots.txt -Carlos +569 6668 32 94 2014-08-13 14:50 GMT-04:00 Tomas Cohen Arazi <tomascohen@gmail.com>:

Are you able to retrieve the robots.txt file in your top-level OPAC domain?

I once set it using Alias /robots.txt "path/to/robots.txt"

in the OPAC vhost definition.

On Wed, Aug 13, 2014 at 3:49 PM, Carlos Rodrigo Cordova Sandoval < carlos.cordovas@gmail.com> wrote:

...

any other suggestions :)

-Carlos +569 6668 32 94

2014-08-13 12:09 GMT-04:00 Carlos Rodrigo Cordova Sandoval < carlos.cordovas@gmail.com>:

Thank you very much, we have already blocked the bots. Even the cpu is at

...

100%

Carlos El 13/08/2014 11:53, "Eric Bégin" <eric.begin@inlibro.com> escribió:

Carlos,

...

I think that web crawler (Google, MS, and other) are querying your Koha.

The following robots.txt file will prevent them to search your catalogue.

User-agent: * Disallow:/cgi-bin/koha/opac-search.pl Disallow:/cgi-bin/koha/opac-export.pl Disallow:/cgi-bin/koha/opac-showmarc.pl Disallow:/cgi-bin/koha/opac-ISBDdetail.pl Disallow:/cgi-bin/koha/opac-MARCdetail.pl

Cheers,

Eric

On 2014-08-13 11:28, Carlos Rodrigo Cordova Sandoval wrote:

Thanks Tomas,

I send the command output. http://pastebin.com/Xk22B3Yd

-Carlos +569 6668 32 94

2014-08-13 10:57 GMT-04:00 Tomas Cohen Arazi <tomascohen@gmail.com> <tomascohen@gmail.com>:

Maybe you have a IO bottleneck (thinking of that mysql cpu usage). How many simultaneous connections do u havE? (netstat)

On Wed, Aug 13, 2014 at 11:52 AM, Carlos Rodrigo Cordova Sandoval <carlos.cordovas@gmail.com> wrote:

Hi Friends,

have a problem with our koha 3.16.02 installed on debian 64bit 7x and has 45,000 records and 157,000 items, our koha uses 100% of cpu. view image. http://snag.gy/xWIXP.jpg

and optimize mysql, apache and memcache activated according to wikihttp://wiki.koha-community.org/wiki/Koha_Tuning_Guide

the features of the server are: Processor: 4 core Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz RAM: 6GB

thank you very much.

-Carlos _______________________________________________ Koha mailing list http://koha-community.orgKoha@lists.katipo.co.nzhttp://lists.katipo.co.nz/ma...

-- Tomás Cohen Arazi Prosecretaría de Informática Universidad Nacional de Córdoba ✆ +54 351 5353750 ext 13168 GPG: B76C 6E7C 2D80 551A C765 E225 0A27 2EA1 B2F3 C15F

_______________________________________________ Koha mailing list http://koha-community.orgKoha@lists.katipo.co.nzhttp://lists.katipo.co.nz/ma...

-- Tomás Cohen Arazi Prosecretaría de Informática Universidad Nacional de Córdoba ✆ +54 351 5353750 ext 13168 GPG: B76C 6E7C 2D80 551A C765 E225 0A27 2EA1 B2F3 C15F

At 11:24 AM 8/14/2014 +1200, Robin Sheat wrote:

Carlos Rodrigo Cordova Sandoval schreef op wo 13-08-2014 om 10:52 [-0400]:

...

have a problem with our koha 3.16.02 installed on debian 64bit 7x and has 45,000 records and 157,000 items, our koha uses 100% of cpu. view image.

It's hard to tell for sure, but it really just looks like you're seeing a lot of traffic. Have you tried running stats over your apache logs to see what's going on there? If you set apache to log things with a better logformat than default[0], then you'll get a lot more data about who and what is hitting your system. It might be you're being spidered by a badly behaving bot, or it might just be you're so popular that you need to invest in more hardware

FYI, we had to deny at our border routers: ShenZhen Sunris Deny 202.46.32.0-202.46.63.255 Beijing Baidu Deny 180.76.0.0-180.76.255.255 NCICNET-NET Deny 175.180.0.0-175.183.255.255 CHINACACHE-1 Deny 69.28.48.0-69.28.63.255 XeraCom IT AB Deny 89.160.60.192-89.160.60.223 For constant woodpeckering despite asking them "nicely" in robots.txt -- it's a crying shame, but... Best -- Paul

thank you very much to all the solicion step to separate the database on a dedicated server. Thank you very much. -Carlos +569 6668 32 94 2014-08-14 12:43 GMT-04:00 Carlin, William <WCarlin@washoecounty.us>:

Hello Chad,

Haven't tried yet...

Best wishes and think good thoughts,

William Carlin Systems Washoe County Library System

-----Original Message----- From: Koha [mailto:koha-bounces@lists.katipo.co.nz] On Behalf Of Robin Sheat Sent: Wednesday, August 13, 2014 4:25 PM To: koha@lists.katipo.co.nz Subject: Re: [Koha] Koha used 100% cpu

Carlos Rodrigo Cordova Sandoval schreef op wo 13-08-2014 om 10:52 [-0400]:

...

have a problem with our koha 3.16.02 installed on debian 64bit 7x and has 45,000 records and 157,000 items, our koha uses 100% of cpu. view image.

It's hard to tell for sure, but it really just looks like you're seeing a lot of traffic. Have you tried running stats over your apache logs to see what's going on there? If you set apache to log things with a better logformat than default[0], then you'll get a lot more data about who and what is hitting your system. It might be you're being spidered by a badly behaving bot, or it might just be you're so popular that you need to invest in more hardware (and/or help the Plack support move forward so that Koha is a lot lighter on CPU.)

As good as htop is, the one thing it's not showing is the distribution of wait states, regular top will say something like:

Cpu(s): 38.3%us, 3.8%sy, 0.0%ni, 56.4%id, 0.9%wa, 0.0%hi, 0.6%si, 0.0%st

if the 'wa' one is high, that usually means things are stuck in IO wait, which is common if you're on a virtual machine and something else starts using all your disk bandwidth. It can also mean that it's just got so much going on that mysql is always reading the disk, telling these situations apart just requires experience.

If you do find that mysql is doing too much work, especially IO — and looking at htop I think it might be — make sure you've tuned it to have enough RAM to hold your entire database in memory. That can make a huge difference. There's a mysql tuner script out there that might help[1].

[0] I add:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\""

to the bottom of /etc/koha/apache-shared.conf, that helps a lot. I should really make that default for the packages.

[1] https://launchpad.net/mysql-tuning-primer or http://mysqltuner.com/ I think it's one of these.

-- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5FA7 4B49 1E4D CAA4 4C38 8505 77F5 B724 F871 3BDF

_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha