[Koha] Major, multiple problems with our Koha 17.05 system.
r.delahunty at arts.ac.uk
Fri Oct 13 00:29:17 NZDT 2017
VERY long sorry, but our Koha system is virtually unusable and we need help!
We are experiencing dozens of outages daily (both OPAC and intranet, more often only OPAC) with our Koha 17.05.01.000 (Debian 3.2.89-2, perl 5.020002, mysql 14.14, apache 2.4.10). Searches can take 15+ seconds. While university staff provide first tier support, we don't have server access, and the second tier and hosting is via an external company. Of course we are working with that company to have them identify the cause, but I have decided to ask the community of any user on 17.05 has the problems we have.
Our system is quite extensively customised, with major work done to the OPAC at https://libsearch.arts.ac.uk<https://libsearch.arts.ac.uk/>. We have some auto-renewal functionality we funded working, while we work on more improvements (we are NOT using the template toolkit, relying on workarounds to offer reasonably useful auto-renewal features).
Our problems started in the summer vacation soon after the upgrade, where Koha returns functionality became unstable. It was taking 8 seconds to return one item. Our support company advised that it was the AddReturn (and, in particular, MarkIssueReturned) which was the pinch point. Seems some code to resolve the auto_increment bug works slower in 17.05 and our support company was correcting that with a view (I understand) to adding it to Master.
Things improved a little, but with the start of term almost 2 weeks ago we are encountering so many problems...
"We were alerted to issues at 4.22pm. and found that the plack server was no longer responding. We restarted plack and then apache and the service restored. We are trying to get to the bottom of what caused this issue. It appears to be completely separate from the high CPU problem that we are also seeing."
I understand more resources have been allocated to mySQL and additional plack helpers have been allocated (whatever that means).
We are heavy users of SIP (80% of transactions are via self issue). But there are so many problems with the returns unit that they are almost unusable. Front line staff are fed up! The sorters and the kiosks configured for returns (as well as issue) fail repeatedly dozens of times daily (ERR_SIP_COM_RECV). Reconnection does happen- takes about 40seconds or so, leading to queues. And the items have NOT come off the account and flow into the exceptions bin. At the same time the intranet report 500 server errors. I am pretty sure most of our problems are related to returns code, but could well be wrong/ oversimplifying things.
Our support company advised:
"We have added extra logging and can see that some checkin requests via the SIP2 protocol are causing the sipserver process to abort. The book is returned within Koha but as the process returns no information to the SIP2 client therefore the unit responds with an error. After waiting for the timeout period it then will reconnect to the server. We are adding further debug to the logs to work out why the sipserver process is aborting. It does seem to be a special subset of the checkin requests. I'll update you when we have more news."
And more problems relating to total outages (with proxy errors 502 relating to invalid responses from an upstream server):
"There appears to be a number of issues you are experiencing. Friday you had high CPU usage which is different to today's issues which we think are down to the apache webserver not restarting correctly. I will double check first thing tomorrow that the overnight apache restart has occurred properly. We are hoping the database connections change may address the high CPU issue. We will monitor closely for the errors we have seen in the apache error logs tomorrow."
We are a university with 20,000 students across 6 colleges, and right now inductions and training in OPAC use is going on with new students. These continual problems are deeply embarrassing, and faith in the Koha system is dropping rapidly. Has any other Koha 17.05 user (especially one heavily reliant on SIP) experienced anything like this? Has anyone out there any suggestions?
University of the Arts London
This email and any attachments are intended solely for the addressee and may contain confidential information. If you are not the intended recipient of this email and/or its attachments you must not take any action based upon them and you must not copy or show them to anyone. Please send the email back to us and immediately and permanently delete it and its attachments. Where this email is unrelated to the business of University of the Arts London or of any of its group companies the opinions expressed in it are the opinions of the sender and do not necessarily constitute those of University of the Arts London (or the relevant group company). Where the sender's signature indicates that the email is sent on behalf of UAL Short Courses Limited the following also applies: UAL Short Courses Limited is a company registered in England and Wales under company number 02361261. Registered Office: University of the Arts London, 272 High Holborn, London WC1V 7EY
More information about the Koha