[Koha] MySQL database server, dedicated vs virtual

Tue Dec 6 04:33:16 NZDT 2016

At 02:02 PM 12/4/2016 +0000, Marcel de Rooy wrote:
>Please note:
>Staying at Koha 3.8 is not recommended. The current release (16.11 == 
>3.26) is already 9(!) release cycles behind..
>And I would not advise others to do so.

Marcel -- I don't know how you could read my comments as a "recommendation" 
or "advice" to stay with Koha 3.8.24. I mentioned, in response to RG's post 
"... was about 60-80% slower... ", that we had done some lengthy analysis 
of search speed (which I carried out with the assistance of the the Koha 
community and openly made available) and made the disappointing decision to 
remain with that version, under our specific circumstances. And I concluded 
with "YMMV" which stands for "your mileage might vary."

Perhaps I should have been clearer: Koha was our indisputable choice for an 
OPAC (3.4.x) back in 2011; we are not a lending library, so the undoubted 
enhancements for lending libraries are irrelevant to our needs; the 
corollary is that if such enhancements have negative effects on our 
requirements, we very definitely look at them (sandbox level) but will not 
use them in production (i.e. on the public web interface.)

Our production servers are all Ubuntu LTS based: this is a two to five year 
cycle; we do not have the budget or necessary volunteer hours to review 
monthly or bi-monthly changes (except security concerns); our overriding 
concern is total, hands-off stability; I have no idea if we are the only 
Koha-based library that uses a similar LTS approach.

Koha 3.8.24 was released 5/29/2014, corresponding to the Ubuntu 14 LTS 
cycle. We recently upgraded two of our production servers to Ubuntu 16 LTS 
for other databases, but unless I am mistaken Koha 16.11 does not install 
(or at least not easily in our experience, and we have tried) on that o/s. 
So we maintained an additional, dedicated 14 server (with three years 
remaining Ubuntu support to 2019) for the OPAC when we followed the Ubuntu 
cycle. Again, this was disappointing, and again this represents only our 
very specific needs.

So, when I say "your mileage might vary" I mean exactly that -- we probably 
are atypical of many Koha users, but are extremely happy and proud of our 
Koha OPAC -- so, if it wasn't clear, let me state that our analysis of 
search speed is *not* any sort of recommendation.

Best regards -- Paul

>Marcel
>
>
>________________________________
>Van: Koha <koha-bounces at lists.katipo.co.nz> namens Paul A 
><paul.a at navalmarinearchive.com>
>Verzonden: vrijdag 2 december 2016 21:53
>Aan: koha
>Onderwerp: Re: [Koha] MySQL database server, dedicated vs virtual
>
>Roger,
>
>We also looked into the "search performance" last year. We are not a
>lending library, so cataloguing and OPAC are our primary concerns. Please
>see <http://navalmarinearchive.com/z_koha/> for the detailed tech analysis
>and conclusions.
>
>You mention below that "Koha does not benefit a lot from multiple CPU cores
>since each CGI request is typically processed by one CPU except for the
>Zebra searches and database queries which run as separate processes."
>
>I talked to Intel about CPU core cross-threading as it was a very obvious,
>high-load, show-stopper with Zebra. The Zebra author never replied to my
>queries. Intel were not optimistic, as kernel (and software) were not their
>bailliwick. I do not know if Plack or Elastic have worked around this.
>
>These are hardware (perhaps software usage of core capability), not Koha,
>restrictions. Tweaking memcached can have appreciable benefits. But the
>bottom line remains that if a single CPU core hits 101-104%, search
>functions descend into the "swimming upstream in treacle" world.
>
>We made the very conscious (but disappointing) decision to stay with Koha
>3.8.24 based on our test results. I do spend quite some time testing later
>versions at "sandbox" level, but have not been able to reproduce "older"
>search speed.
>
>YMMV -- Paul
>Production: Koha 3.8.24 on 14.04.2 LTS (GNU/Linux 3.13.0-43-generic
>x86_64), 8-core I7 processors, 64 Gigs RAM, all SSD drives.
>
>
>
>
>At 06:47 PM 12/2/2016 +0100, Roger Grossmann wrote:
>
> >Hi Tobias,
> >
> >a few month ago we did comparisons measuring the Koha-OPAC-Search
> >performance with plack and memcached enabled using version 3.22.
> >Our tests were not scientifically organised. We wanted to gather
> >experience running Koha in different environments. We used the same MySQL
> >database and Koha version in all environments. We did two types of OPAC
> >searches on a limited collection of about 50.000 titles: an 'a*'-search of
> >the complete word index and some special searches with fixed title search
> >terms.
> >We tested the following environments:
> >
> >1) Full Koha installation on a cloud provider hosted VM (the performance
> >of VMs of the specific hoster were high ranked in published comparisons):
> >Debian 8 on a cloud hosted system with virtual 2CPUs, 8 GB RAM, 256 SSD
> >2) Full Koha installation on a physical server: Debian 8 on a well
> >equipped physical machine with 64 GB RAM, 4 processors, 250 GB SSD
> >3) Full Koha installation on a kvm-VM on a physical server: Debian 8 on a
> >well equipped machine with 64 GB RAM, 4 core CPUs, kvm VM with Debian 8
> >4) Koha running on a physical machine like 2) and 3) with a separate
> >physical DB Server. Koha and DB server were connected through a 1 GB 
> network.
> >
> >Our findings were the following:
> >No wonder, environment 2 was the fastest. Running Koha on a physical
> >server with the database on the same machine brings optimal performance.
> >As a surprise, environment 3 had a performance closed to environment 2
> >with a difference of max. 5% performance loss. The kvm virtualiser seems
> >to add very little overhead. Advantages of environment 3 are beside the
> >high performance that it is possible to copy and save the VM doing updates
> >and so on. We did not test running multiple kvm-VMs on the same host.
> >Rather than running instances in separate VMs, Koha's concept of managing
> >instances makes the use of one host very efficient. Running multiple
> >instances in one machine requires less maintenance than running multiple
> >VMs each with one Koha instance.
> >Environment 1 was about 60-80% slower than environment 2. We got varying
> >results probably depending on the load of the provider based environment.
> >Server requests in environment 4 took double the time of environment 1. It
> >seemed that the network latency and transmission of data between DB Server
> >and Koha is a bottleneck here.
> >
> >We can recommend option 3. It brings an optimum of performance with the
> >flexibility of a virtualised environment. For a single Koha instance you
> >probably do not need more than 2 CPUs cores. Typically you can improve the
> >database performance with an optimised database configuration
> >(specifically the page buffer if you have enough RAM). So using >=16 GB
> >RAM should be well equipped from my perspective.
> >Key parameters to speed up Koha were in our tests the hard disk speed (use
> >SSDs) and CPU speed. Koha does not benefit a lot from multiple CPU cores
> >since each CGI request is typically processed by one CPU except for the
> >Zebra searches and database queries which run as separate processes.
> >Additional CPU cores can be useful if the load is high on the server when
> >processing many parallel requests.
> >
> >Doing searches, the Zebra indexer impacts the speed very much. In our
> >tests Zebra took about 50% or more of the request time. I assume that the
> >planned change to Elasticsearch will result in performance improvements in
> >the future.
> >
> >Best regards,
> >Roger
> >
> >--
> >LMSCloud GmbH
> >Roger GroÃmann - GeschÃÂ¤ftsfÃÂ¼hrer
> >Konrad-Zuse-Platz 8 - D-81829 MÃÂ¼nchen
> >e roger.grossmann at lmscloud.de
> >w www.lmscloud.de<http://www.lmscloud.de>
> >
> > > Am 01.12.2016 um 18:03 schrieb tobias carlsson
> > <tobias.carlsson at bitlabbet.se>:
> > >
> > >
> > >
> > > Hi!
> > >
> > > I'm currently working with a Koha implementation in a public library in
> > > southern Sweden. Our installation (it's not live yet) is currently on a
> > > virtual machine running Ubuntu Server 14.04 LTS and the version of Koha
> > > is 3.22. The MySQL database also resides on the same virtual server.
> > >
> > > I've tried some at home - on a Koha test server running under Debian
> > > with the MySQL database on a machine running Ubuntu Server 14.04 (with a
> > > Core i3 4170 @3,7 GHz (stock), with a quite slow Western Digital Red.
> > > The difference between running on "bare metal" and virtual seem rather
> > > striking. Is there anyone out there who have moved from virtual running
> > > databases to physical? What can you tell us about this?
> > >
> > > I know that a database server should run with good I/O performance and a
> > > fast CPU and lots of RAM and so on, but since I'm a librarian, not an
> > > engineer (that sounded familiar...), I have limited experience of
> > > dealing with servers in a large scale and I'm no expert on databases.
> > >
> > > I'm aware of database optimizations and the use of plack and memcached,
> > > but running the database on a "bare metal" server with a high clock,
> > > made me start thinking of how we should deal with the database. It seems
> > > to me that many libraries use virtual servers for everything, even
> > > databases and that it's common to run Koha and server on the same
> > > (virtual) instance.
> > >
> > > Please share your experiences and thoughts about this.
> > >
> > > Best regards,
> > >
> > > Tobias Carlsson
> > > Systems Librarian and IT Technician
> > >
> > > Currently working with Koha at Vaggeryds bibliotek (Vaggeryd public
> > > library) in Sweden.
> > > Working as a Systems Librarian at Gislaveds bibliotek.
> > >
> > >
> > > _______________________________________________
> > > Koha mailing list  http://koha-community.org
> > > Koha at lists.katipo.co.nz
> > > https://lists.katipo.co.nz/mailman/listinfo/koha
>
>Koha --Discussion list for the Koha Library 
>system<https://lists.katipo.co.nz/mailman/listinfo/koha>
>lists.katipo.co.nz
>To see the collection of prior postings to the list, visit the Koha 
>Archives. Using the Koha Mailing List: To post a message to all the list ...
>
>
>
> >
> >_______________________________________________
> >Koha mailing list  http://koha-community.org
>
>Official Website of Koha Library Software<http://koha-community.org/>
>koha-community.org
>Koha Library Software The world's first free and open source library 
>system Koha is a fully featured, scalable library management system. 
>Development is sponsored by ...
>
>
>
> >Koha at lists.katipo.co.nz
> >https://lists.katipo.co.nz/mailman/listinfo/koha
>
>Koha --Discussion list for the Koha Library 
>system<https://lists.katipo.co.nz/mailman/listinfo/koha>
>lists.katipo.co.nz
>To see the collection of prior postings to the list, visit the Koha 
>Archives. Using the Koha Mailing List: To post a message to all the list ...
>
>
>
>
>---
>Maritime heritage and history, preservation and conservation,
>research and education through the written word and the arts.
><http://NavalMarineArchive.com> and <http://UltraMarine.ca>
>
>Canadian Society of Marine Artists<http://ultramarine.ca/>
>ultramarine.ca
>The Canadian Society of Marine Artists is a not-for-profit non-profit 
>administered by the Naval Marine Archive and is dedicated to Canadian 
>maritime art promotion ...
>
>
>
>[http://navalmarinearchive.com/ims/victory_2015.jpg]<http://navalmarinearchive.com/>
>
>Naval Marine Archive - The Canadian Collection<http://navalmarinearchive.com/>
>navalmarinearchive.com
>Naval Marine Archive - The Canadian Collection is a non-profit Canadian 
>charity dedicated to maritime history, conservation, marine research, 
>nautical education ...
>
>
>
>
>_______________________________________________
>Koha mailing list  http://koha-community.org
>
>Official Website of Koha Library Software<http://koha-community.org/>
>koha-community.org
>Koha Library Software The world's first free and open source library 
>system Koha is a fully featured, scalable library management system. 
>Development is sponsored by ...
>
>
>
>Koha at lists.katipo.co.nz
>https://lists.katipo.co.nz/mailman/listinfo/koha
>
>Koha --Discussion list for the Koha Library 
>system<https://lists.katipo.co.nz/mailman/listinfo/koha>
>lists.katipo.co.nz
>To see the collection of prior postings to the list, visit the Koha 
>Archives. Using the Koha Mailing List: To post a message to all the list ...
>
>
>
>_______________________________________________
>Koha mailing list  http://koha-community.org
>Koha at lists.katipo.co.nz
>https://lists.katipo.co.nz/mailman/listinfo/koha

---
Maritime heritage and history, preservation and conservation,
research and education through the written word and the arts.
<http://NavalMarineArchive.com> and <http://UltraMarine.ca>