[Koha] [Discussion Tech] Cache handling in koha

[Tomas Cohen Arazi]

Hi, I'm just starting a thread to discuss this topic so we make some
decisions and we can continue improving Koha in this aspect. As far as
I can remember there are a few issues with this:

1) The use of ENV vs. koha-conf.xml vs. DB/systempreferences to store
memcached configuration.

I introduced the use of ENV (from Apache vhost definition) for storing
memcached servers configuration data on Bug 6193. This was done for
being able to cache the koha-conf.xml configuration variables (prior
to that, memcached info was saved in koha-conf.xml itself and thus
not-cacheable). When coding that I realized that memcached was being
initialized everywhere we needed it. And namespaces where not always
consistent (KOHA and koha alternatively, this might not be a problem
if this happens to be case insensitive, anyway, it looked bad to me).

I then exported the object memcached and the ismemcached variable in
C4::Context. This first patch (koha-conf.xml in memcache, and
memcached object available for every other lib that includes
C4::Context) has been pushed.

Then there's a second patch, that removes several memcached
configuration validation and  initialization code from the rest of the
libs, and reuses that variables from C4::Context. I tested it to work
flawlessly, but things around it halted when other proposals raised.

I like mi approach. I like how the resulting code looked, I wont hide
it. But I don't need it to be accepted at all, maybe the other
proposals are far better. The only thing I missed on this cache thing
was more feedback.

Back to the ENV use I think we must make a desicion:
 - Do we want to cache koha-conf.xml ?
If the answer is no, then we can revert that patch. And we are done.
If the answer is yes, we can deal with it is storing the settings in
the database, but this will make it difficult to memoize the function
reading. Or leave it as it is now.

I don't agree it is a problem to move those settings to the vhost
definition, it could be even moved into a separate file with an
include sentence in the vhost definition for convenience
(/etc/koha/memcached.conf and Include in vhost definition...)

2) Memoization in Koha
Do we plan to use memoization in the mid/long-term? Or are we thinking
of a more abstract set of 'cache' utilities which provide an interface
to memcached?

Using memoization is nice, clean and simple. It has the disadvantage
that caching is done in an implicit way, and there are hidden problems
(read side-effects) that have to be taken into account. For example,
someone writes a method, other memoizes it, but doesn't know that a
third method relies on data returned by the first mehotd being
'online'. As-is, memoize_memcached doesn't provide a proper way of
invalidating data automatically. As said in a previous email: we will
end up hooking all the data writing methods so they invalidate all the
memcached store, or those methods the writer knows have to do with the
current. Almost like making it explicit, but just for writers. In
which case it is better to implement the second option, making
everything explicit:

The second approach means we have to hook every function to handle its
caching, and cache invalidation. Its advantage is that being explicit,
but requieres a lot of work and we will end up having business logic
mixed with cache handling code, which is not a problem per-se. Just
thinking.

If we are planning to go on with memoization, I made a proposal on IRC
(which had zero responses) to add memoize_memcached an array of
references to data invalidation methods, so we don't have to hook
every method. The Memoize::Memcached dev told me to take over the lib
as he isn't maintaining it nor working on perl anymore. I'd like to do
that, but I halted my work on that until it is more clear that it is
not a waste of time.

Ok. I'd like to re-read this words I wrote, but I'm just back from
holidays and my bosses need a lot of answers for a lot of stuff I
didn't finish last year, right now, so...

Regards
To+