[Koha] Problems with the facebook web crawler

[Nigel Titley]

Is anyone else getting problems with the facebook web crawler hammering 
their OPAC search function?

This has been happening on and off for a couple of months but set in 
with a vengeance a couple of days ago. The crawler is hitting us with 
many OPAC search queries, beyond the capacity of our system to respond.

robots.txt is being ignored

I started by blocking facebook's entire IPv6 range as the queries were 
all coming in over IPv6. They responded by switching to IPv4 and because 
they have a number of blocks it wasn't practical to block each and every 
one of them.

I've temporarily switched off OPAC entirely and the system has returned 
to normal and I can at least perform intranet functions but this is 
obviously non-ideal.

Does anyone have any thoughts on this?

I'm running 22.05.13.000 on Ubuntu.

Thanks

Nigel