Hello librarians and developers, I send a "What's on in Koha devel" email to koha-devel list each month. But this one is a bit special as I will focus on the different subjects you may heard of if you attend the hackfest in Marseille next week. It can be helpful for both librarians and developers! A lot of things are going on in the Koha ecosystem so it might be hard to follow all the discussions or to jump into one. So my attempted goal here is to compile the topics currently "à la mode" for Koha developers. In this email I will go a bit more into details than usual, to let you the opportunity to better understand the themes and to ask questions next week if you want to know more. I hope it will help you to choose the subject you want to be involved in during the hackfest. = Sandboxes = Sandboxes have been developed with the aim to help people without any technical skills to test patches submitted on the Koha bugtracker. You will find all the information you want on the dedicated wiki page https://wiki.koha-community.org/wiki/Sandboxes = Refactoring = During the last 3 (more?) years we have integrated DBIx::Class (a Perl ORM, Object-Relationnal Mapping) into Koha, for several (sometimes disputed) reasons. To take advantage of it, we are using it as much as possible through a home-made object module called Koha::Object. For the last year, a lot of legacy code has been rewritten and moved out of the C4 namespace to the new Koha namespace. For instance Koha::Virtualshelves replaced C4::VirtualShelves::Page (bug 14544), Koha::Libraries replaces C4::Branch (bug 15293). At the moment, the job focusses on moving the legacy authorised values from the C4::Koha module to Koha::AuthorisedValues (bug 15799) and the patrons/borrowers/users/members code from C4::Members to Koha::Patrons (bug 16846). For an overview of this refactoring work, please have a look at bug 15449 and its scary dependency graph: https://bugs.koha-community.org/bugzilla3/showdependencygraph.cgi?id=15449 Another refactoring work is about moving the biblioitems.marcxml content out of the biblioitems table. The idea is to create another table (biblio_metadata) to add the ability to store a record in different formats. The direct and major addition of this move would be to bring performance speed. See the comment 1 of bug 17196 for more information. = Speed improvements = During the last 2 releases, we have made a lot of speed improvements. That has been achieve with the consolidation and the stabilisation of our caching system and plack integration. If you are running a recent version of Koha (3.22, 16.05 or later) you should set them up correctly to fully enjoy the improvement. To be fair I must say that we have had to concentrate our efforts on these points because of our previous technical decisions (mainly related to DBIx::Class). Two main playgrounds: Plack and Memcached, that I will explain now. == Plack == I bet you already have heard of Plack, because Koha developers have been talking about it for ages. Basically it is just an interface between the web server (Apache, starman, etc.) and the perl application (Koha). When a user hits the a Koha url, a lot of files (Perl modules) are compiled. In CGI mode, this compilation step is done for every request. Using Plack they are compiled only once. The code will be put in RAM and other requests won't need to process all the modules again. It is an advantage since we are using DBIx::Class more and more and its schema is heavy to load. With Plack it is only loaded once. Koha is now considered as stable under Plack for the last versions of Koha, so you must use it. To know about known bugs, you can follow bug 7172 which gather Plack related known bugs together. At this time there is only one patch not yet pushed, bug 17392 (ping QAer!). Note that a timeout issue seems related to Plack and is not yet resolved, see: Bug 16714 - Unexpected logout with "IP address change" (with SessionRestrictionByIP set) == Caching system == The way we are caching "stuffs" in Koha has really been improved recently and will continue to! Important steps have been done in this area. We are using Memcached - a memory caching system - to store big bunch of data that we do not want to recalculate or retrieve from the database everytime. For instance: - the sysprefs. A lot of them are retrieved from the database for each request - the biblio frameworks - the holidays - ... a lot of other things could/will be cached! A big step has been done when we decided to introduce a 2-level caching mechanism (bug 16044): we have now a in-memory L1 cache (flush at every request) and a L2 cache (Memcached). To understand how this mechanisms are useful, imagine a simple scenario: launch a search which will return 20 results. For each result we need to know the MARC bibliographic framework of the record. Say they are all using the default frameworkcode. Without any caching mechanism, we retrieved from the database the whole structure, 20 times, once per result. With our new caching mechanism, the information for the default MARC bibliographic framework will be retrieved on the first request, then put in both L1 and L2 cache. The 19 other results will retrieve it from the L1 cache. If the same or another user does a search, the framework info will be retrieved from the L2 cache for the first result, and the L1 cache will be populated. The other results will retrieve it from the L1 cache. Not sure this is clear, but catch me if you need more details :) All of that to say that if you are a developer, there are 3 other bugs with ideas to improve again the caching mechanism. They are all of them in discussion: Bug 16140 - Only clear L1 cache when needed Bug 16079 - Retrieving system preferences from database via DBIx is not fast enough Bug 15341 - Performance - Retrieve all sysprefs at once One which would be nice to have is: Bug 17261 - Add memcached configuration info to about.pl It will permit to display the memcached configuration in the about page. = MySQL 5.7 compatibility = If you are interested in using Koha with MySQL 5.7 (the default version for the last Ubuntu 16.04), you should take a look at bug 17258 and its dependencies. Koha is not ready at all for the new default sql_mode configuration of this version of MySQL (STRICT_TRANS_TABLES). = Security issues = A lot of security issues (CSRF and XSS) have been fixed for the last 4 months, and almost all of these fixes have been backported to stable releases. They are all reported under the 2 following omnibus: Bug 17096 - [OMNIBUS] CSRF protections Bug 14568 - [OMNIBUS] XSS in Staff Client Only one known bug is waiting to be QAed (but 17365). = Elastic search = There are no big new features pushed since the first big push. But a few bug fixes and enhancement are waiting to be QAed. You can find them on the dependency graph of bug 12478. Note that there are 2 known bugs, without patches: Bug 16660 - Elasticsearch broken if OpacSuppression is activated Bug 17373 - Elasticsearch - Authority mappings are not defined for UNIMARC Are there some UNIMARC users around? :) = RESTful API = Tons of patches have been submitted on bugzilla, but only few got attention from signoffers. If you are interested in testing of them, search for "rest api". = ReactJS = React is a JS library aimed to easier DOM manipulations. This topic is a recurrent one and developers involved in this discussion will be present at the hackfest. They should organise a discussion to reach a consensus once and for all. If accepted, it would be good to define guidelines and write complete examples. = Koha plugin system = It seems that people are curious about our plugin system. It would be good to revive the discussion on new and more powerful plugin system. See the discussion on the koha-devel list at http://lists.koha-community.org/pipermail/koha-devel/2016-May/042673.html = Transactions & exceptions = It would be good to see some developers organise a brainstorming session on the different topics that Tomas raised on his email to koha-devel a few weeks ago. http://lists.koha-community.org/pipermail/koha-devel/2016-September/043032.h... = Mana = Paul presented us Mana (https://lists.katipo.co.nz/pipermail/koha/2016-July/045739.html) at the beginning of July. Morgan finished her internship and will be there are the hackfest to show us what she developed. Testers and QAers would certainly be welcomed to test and review this feature. = Hea = Hea is a website (http://hea.koha-community.org) collecting usage statistics from different Koha installations around the world. This feature is not enabled by default but is very useful for developers and other people loving statistics (so librarians isn't it?). If it is not done yet, you should enable this feature to join the 496 libraries already registered! One enhancement is developed and waiting for testers: 14608 - HEA : add possibility of sharing usage statistics in Administration page and Web installer Hope to see you ready to fix, translate, test or QA patches (and drink beers when everything is done) this week! Cheers, Jonathan