[Koha] zebraqueue and cron jobs

[Tomas Cohen Arazi]

On Wed, Nov 3, 2010 at 2:22 PM, Peter  Huerter <pete.huerter at gmail.com> wrote:
> Chris Cormack <chris at ...> writes:
> .. snip ..
>> Do NOT use the zebraqueue_daemon.
>> The cron job is a far safer option, the daemon is due to be removed
>> (or completely rewritten), it has memory leaks.
>>
>> Chris
>>
>
> Is this covered by the following bug report?
>
> http://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=2453
>
> If so, then perhaps we can continue to use the daemon if we stay away from large
> MARC records(?).  Using a Cron job to update the index may confuse our
> volunteers that do data entry (because of the lag, even though we set if for 1
> minute).
>
> If not, can you please point me to the bug report that covers this problem with
> zebraqueue so that we can track it?

If you read the thread I mentioned and google for more info
(zebrauqeue+daemon+problem) you will find people saying it is flawed.
I asked (without response) if the problem had to do with atomicity on
the operations so I could put my hands on: it's a very known and
documented pattern...

With that in mind, as cron doesn't serialize operations (if an
operation takes too much a new process will be launched on deadline) I
supposed that creating a new daemon that calls the *trusted*
rebuild_zebra routines in a serialized fashion, with configurable
intervals of whatever time one wants should have been both harmless
(not adding big things that might lead to insolvable or undiscoverable
errors) and usefull: non-dev users are very confused about zebraqueue
daemon and is not clear what needs to be fixed.
I think there are efforts to change the zebra indexing engine in favor
of solr, but I think the users need more clear info on the current
state of zebra use in koha, and zebraqueue in particular:

1) We should remove zebraqueue_daemon references on the docs, and the
svn tree to avoid confusions, and make ir clear that a cron has to be
set in any case
or
2) We have to create a bug that explains in more detail the problems
around zebraqueue_daemon so they can be fixed (I heard about memory
leaks this time?), and temporarily do (1) too.
or else

exactly
To+

PD: Yes, I'm a South Park fan.