[Koha] Koha slowed down by Google indexing?!

[Michael Kuhn]

Hi

We're on Debian GNU/Linux 8 running Koha 16.11.04.

Today the library complained that Koha was acting very slow - a search 
in the OPAC takes about 10 to 30 seconds. With command "top" the 
following can be seen:

  top - 13:06:31 up  2:44,  3 users,  load average: 3.14, 2.54, 2.41
  Tasks: 162 total,   5 running, 157 sleeping,   0 stopped,   0 zombie
  %Cpu0  : 95.7 us,  4.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 
si,  0.0 st
  %Cpu1  : 95.7 us,  4.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.3 
si,  0.0 st
  KiB Mem:   4049620 total,  2377480 used,  1672140 free,   218872 buffers
  KiB Swap:  4189180 total,     6632 used,  4182548 free.   813584 
cached Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ 
COMMAND
  19678 phsh-ko+  20   0  224672  83616   8996 R  43.2  2.1   0:01.59 
opac-searc+
  19672 phsh-ko+  20   0  393696 148372  12304 R  41.2  3.7   0:03.09 
opac-searc+
  19682 phsh-ko+  20   0  211028  69912   8860 R  28.9  1.7   0:00.87 
opac-searc+
  19684 phsh-ko+  20   0  168824  42320   8496 R  14.3  1.0   0:00.43 
opac-searc+
  ...

Most of the time, "opac-search.pl" is eating up 95-100% of the CPU and 
more than one process can be seen at the same time. Eventually I found 
many lines like the following in the file "opac-error.log":

  [Wed May 03 13:13:10.796794 2017] [cgi:error] [pid 20053] [client 
66.249.64.32:39077] AH01215: [Wed May  3 13:13:10 2017] opac-detail.pl: 
Use of uninitialized value in subroutine entry at 
/usr/share/perl5/URI/Escape.pm line 184.
  [Wed May 03 13:13:17.600461 2017] [cgi:error] [pid 20479] [client 
66.249.64.32:50544] AH01215: [Wed May  3 13:13:17 2017] 
opac-ISBDdetail.pl: Use of uninitialized value in subroutine entry at 
/usr/share/perl5/URI/Escape.pm line 184.

When I searched for who is 66.249.64.32 I saw this IP addresse belongs 
to Google.

When I added some functionality to show the execution time of 
"opac-search.pl" I got many more lines like the following, all of them 
indicating to IP address 66.249.64.32:

  [Wed May 03 13:10:01.030216 2017] [cgi:error] [pid 20053] [client 
66.249.64.32:39077] AH01215: START: 1493809799.15058 / END: 
1493809801.02982 = 1.87923502922058
  [Wed May 03 13:10:04.426503 2017] [cgi:error] [pid 20147] [client 
104.154.58.95:33362] AH01215: START: 1493809799.83011 / END: 
1493809804.42624 = 4.59613108634949
  [Wed May 03 13:10:06.473646 2017] [cgi:error] [pid 20053] [client 
66.249.64.32:39077] AH01215: START: 1493809805.1977 / END: 
1493809806.47357 = 1.27586483955383

I'm not sure what this means. Is this Google indexing our library 
catalog and thus slowing everything down?

Do you have experienced similar behaviour or can someone please explain 
what's happening?

Best wishes: Michael