[Koha] [Koha-devel] What's on in Koha devel #5 - Special hk16

Sun Oct 9 01:25:18 NZDT 2016

Thanks for the overview Jonathan! I know I for one have the react
discussion as a top priority for the jacket.

Kyle

Sent from my phone. Please excuse my brevity.

On Oct 8, 2016 2:20 PM, "Jonathan Druart" <
jonathan.druart at bugs.koha-community.org> wrote:

Hello librarians and developers,

I send a "What's on in Koha devel" email to koha-devel list each
month. But this one is a bit special as I will focus on the different
subjects you may heard of if you attend the hackfest in Marseille next
week.
It can be helpful for both librarians and developers!

A lot of things are going on in the Koha ecosystem so it might be hard
to follow all the discussions or to jump into one.

So my attempted goal here is to compile the topics currently "à la
mode" for Koha developers.
In this email I will go a bit more into details than usual, to let you
the opportunity to better understand the themes and to ask questions
next week if you want to know more. I hope it will help you to choose
the subject you want to be involved in during the hackfest.

= Sandboxes =

Sandboxes have been developed with the aim to help people without any
technical skills to test patches submitted on the Koha bugtracker.
You will find all the information you want on the dedicated wiki page
https://wiki.koha-community.org/wiki/Sandboxes

= Refactoring =
During the last 3 (more?) years we have integrated DBIx::Class (a Perl
ORM, Object-Relationnal Mapping) into Koha, for several (sometimes
disputed) reasons.
To take advantage of it, we are using it as much as possible through a
home-made object module called Koha::Object.
For the last year, a lot of legacy code has been rewritten and moved
out of the C4 namespace to the new Koha namespace. For instance
Koha::Virtualshelves replaced C4::VirtualShelves::Page (bug 14544),
Koha::Libraries replaces C4::Branch (bug 15293).
At the moment, the job focusses on moving the legacy authorised values
from the C4::Koha module to Koha::AuthorisedValues (bug 15799) and the
patrons/borrowers/users/members code from C4::Members to Koha::Patrons
(bug 16846).
For an overview of this refactoring work, please have a look at bug
15449 and its scary dependency graph:
https://bugs.koha-community.org/bugzilla3/showdependencygraph.cgi?id=15449

Another refactoring work is about moving the biblioitems.marcxml
content out of the biblioitems table. The idea is to create another
table (biblio_metadata) to add the ability to store a record in
different formats. The direct and major addition of this move would be
to bring performance speed. See the comment 1 of bug 17196 for more
information.

= Speed improvements =

During the last 2 releases, we have made a lot of speed improvements.
That has been achieve with the consolidation and the stabilisation of
our caching system and plack integration.
If you are running a recent version of Koha (3.22, 16.05 or later) you
should set them up correctly to fully enjoy the improvement. To be
fair I must say that we have had to concentrate our efforts on these
points because of our previous technical decisions (mainly related to
DBIx::Class).
Two main playgrounds: Plack and Memcached, that I will explain now.

== Plack ==
I bet you already have heard of Plack, because Koha developers have
been talking about it for ages.
Basically it is just an interface between the web server (Apache,
starman, etc.) and the perl application (Koha).
When a user hits the a Koha url, a lot of files (Perl modules) are
compiled. In CGI mode, this compilation step is done for every
request. Using Plack they are compiled only once. The code will be put
in RAM and other requests won't need to process all the modules again.
It is an advantage since we are using DBIx::Class more and more and
its schema is heavy to load. With Plack it is only loaded once.
Koha is now considered as stable under Plack for the last versions of
Koha, so you must use it.
To know about known bugs, you can follow bug 7172 which gather Plack
related known bugs together. At this time there is only one patch not
yet pushed, bug 17392 (ping QAer!).

Note that a timeout issue seems related to Plack and is not yet resolved,
see:
Bug 16714 - Unexpected logout with "IP address change" (with
SessionRestrictionByIP set)

== Caching system ==
The way we are caching "stuffs" in Koha has really been improved
recently and will continue to!
Important steps have been done in this area.
We are using Memcached - a memory caching system - to store big bunch
of data that we do not want to recalculate or retrieve from the
database everytime.
For instance:
 - the sysprefs. A lot of them are retrieved from the database for each
request
 - the biblio frameworks
 - the holidays
 - ... a lot of other things could/will be cached!

A big step has been done when we decided to introduce a 2-level
caching mechanism (bug 16044): we have now a in-memory L1 cache (flush
at every request) and a L2 cache (Memcached).
To understand how this mechanisms are useful, imagine a simple
scenario: launch a search which will return 20 results.
For each result we need to know the MARC bibliographic framework of
the record. Say they are all using the default frameworkcode. Without
any caching mechanism, we retrieved from the database the whole
structure, 20 times, once per result.
With our new caching mechanism, the information for the default MARC
bibliographic framework will be retrieved on the first request, then
put in both L1 and L2 cache. The 19 other results will retrieve it
from the L1 cache.
If the same or another user does a search, the framework info will be
retrieved from the L2 cache for the first result, and the L1 cache
will be populated. The other results will retrieve it from the L1
cache.
Not sure this is clear, but catch me if you need more details :)

All of that to say that if you are a developer, there are 3 other bugs
with ideas to improve again the caching mechanism. They are all of
them in discussion:
Bug 16140 - Only clear L1 cache when needed
Bug 16079 - Retrieving system preferences from database via DBIx is
not fast enough
Bug 15341 - Performance - Retrieve all sysprefs at once

One which would be nice to have is:
Bug 17261 - Add memcached configuration info to about.pl
It will permit to display the memcached configuration in the about page.

= MySQL 5.7 compatibility =
If you are interested in using Koha with MySQL 5.7 (the default
version for the last Ubuntu 16.04), you should take a look at bug
17258 and its dependencies.
Koha is not ready at all for the new default sql_mode configuration of
this version of MySQL (STRICT_TRANS_TABLES).

= Security issues =
A lot of security issues (CSRF and XSS) have been fixed for the last 4
months, and almost all of these fixes have been backported to stable
releases.
They are all reported under the 2 following omnibus:
Bug 17096 - [OMNIBUS] CSRF protections
Bug 14568 - [OMNIBUS] XSS in Staff Client
Only one known bug is waiting to be QAed (but 17365).

= Elastic search =
There are no big new features pushed since the first big push.
But a few bug fixes and enhancement are waiting to be QAed. You can
find them on the dependency graph of bug 12478.
Note that there are 2 known bugs, without patches:

Bug 16660 - Elasticsearch broken if OpacSuppression is activated
Bug 17373 - Elasticsearch - Authority mappings are not defined for UNIMARC
Are there some UNIMARC users around? :)

= RESTful API =
Tons of patches have been submitted on bugzilla, but only few got
attention from signoffers.
If you are interested in testing of them, search for "rest api".

= ReactJS =
React is a JS library aimed to easier DOM manipulations.
This topic is a recurrent one and developers involved in this
discussion will be present at the hackfest.
They should organise a discussion to reach a consensus once and for
all. If accepted, it would be good to define guidelines and write
complete examples.

= Koha plugin system =
It seems that people are curious about our plugin system.
It would be good to revive the discussion on new and more powerful
plugin system.
See the discussion on the koha-devel list at
http://lists.koha-community.org/pipermail/koha-devel/2016-May/042673.html

= Transactions & exceptions =
It would be good to see some developers organise a brainstorming
session on the different topics that Tomas raised on his email to
koha-devel a few weeks ago.
http://lists.koha-community.org/pipermail/koha-devel/2016-
September/043032.html

= Mana =
Paul presented us Mana
(https://lists.katipo.co.nz/pipermail/koha/2016-July/045739.html) at
the beginning of July.
Morgan finished her internship and will be there are the hackfest to
show us what she developed.
Testers and QAers would certainly be welcomed to test and review this
feature.

= Hea =
Hea is a website (http://hea.koha-community.org) collecting usage
statistics from different Koha installations around the world.
This feature is not enabled by default but is very useful for
developers and other people loving statistics (so librarians isn't
it?).
If it is not done yet, you should enable this feature to join the 496
libraries already registered!
One enhancement is developed and waiting for testers:
14608 - HEA : add possibility of sharing usage statistics in
Administration page and Web installer

Hope to see you ready to fix, translate, test or QA patches (and drink
beers when everything is done) this week!

Cheers,
Jonathan
_______________________________________________
Koha-devel mailing list
Koha-devel at lists.koha-community.org
http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/