[Koha] n-Tier deployment of Koha

[dcook at prosentient.com.au]

Hi Admire,

You'll probably have more luck asking on the dev listserv: koha-devel at lists.koha-community.org. However, I'll try to offer some insight here. I'm also CCing in Mengu who has probably the biggest horizontally scaled out Koha in the world. 

Off the top of my head, technically you could have a load balancer in front of  multiple Koha application servers. You'd want to make sure that the cronjobs are only running on 1 application server though. You also would probably want to use an external Elasticsearch cluster since Zebra would be a bottleneck. In theory, you could use a multi-master MySQL/MariaDB database cluster, but I don't know if anyone has done that (and I'd be concerned about the "eventual consistency" of multi-master setups). If you wanted to a master-replica cluster, you'd need to patch Koha to allow for read-only connections (for things like the Reports module); at the moment, Koha can only use 1 read/write database connection. You'd also need to move your RabbitMQ server to an external server which is problematic with the koha-common package at the moment (but there are ways). (Note: I may be missing another horizontal scaling issue, but these are what come to mind at the moment. You might need to use a network share for some file system resources in /var/koha/* but I can't recall off the top of my head.)

It would be much easier to scale vertically. I'd suggest you do some analysis to see where your key pain points are.
- Out of the box, Koha only uses 2 Starman/Plack workers, so if it's a case where your Koha is overwhelmed by legitimate traffic, I'd recommend adding more Starman/Plack workers via koha-conf.xml (if you have the CPU and RAM to spare). 
- If your Koha is slow, you might want to look at your database's I/O performance as well. Using quality SSDs can make a huge difference for database driven applications. (Note also that having MySQL/MariaDB on an external server is a common practice but technically speaking it does add some latency to database transactions which you wouldn't have if you ran MySQL/MariaDB on the same server and used Unix sockets. For very large Koha databases, we've noticed that having an external DB server adds too much latency.)
- If you're getting a lot of traffic, you should check to see how much traffic is from bots. You might need to implement something like fail2ban to block bots that put too much load on your system. Typically, I find when Koha is overwhelmed, it's due to bots. (You can experiment with a robots.txt file to help with this but it's not a completely effective solution.)

There's lots of things to explore. 

In the long-term I'd love to make Koha easier to scale horizontally (and ideally deploy as a load-balanced containerized application) but there's work to do and not much demand for it from libraries, so I don't think it's a priority for devs at the moment, unfortunately. 

Anyway, I hope that helps! 

David Cook
Senior Software Engineer
Prosentient Systems
Suite 7.03
6a Glen St
Milsons Point NSW 2061
Australia

Office: 02 9212 0899
Online: 02 8005 0595

-----Original Message-----
Message: 10
Date: Wed, 27 Jul 2022 16:28:12 +0100
From: Admire Mutsikiwa <amutsikiwa at uzlib.uz.ac.zw>
To: koha at lists.katipo.co.nz
Subject: [Koha] n-Tier deployment of Koha
Message-ID:
	<CAD4nctbymX-UfziZr67h-4p4O0GBPapecRCXfc=obHtmiFt0VQ at mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"

My Koha implementation is sometimes overwhelmed  as it is a two-tier
implementation with a database on one server and a Koha application on
another server.  I am wondering if it is possible to have a 3 or 4 tier
deployment architecture, with for instance, having a load balancer, koha
application server and a clustered MySQL/Mariadb implementation.

Kind Regards,

-- 



*The information transmitted by this email is intended only for the 
person or entity to which it is addressed. This email may contain 
proprietary, business-confidential, and/or privileged material. If you 
are 
not the intended recipient of this message, be aware that any use, 
review, 
retransmission, distribution, reproduction or any action taken 
in reliance 
upon this message is strictly prohibited. If you received 
this in error, 
please contact the sender and delete the material from 
all computers*