[Koha] BiblioAddsAuthorities: SimpleSearch failing (previously "bulk BiblioAddsAuthorities?")
pete.huerter at gmail.com
Thu Mar 17 06:20:08 NZDT 2011
Just following up ...
I finally got a script to work that loops through all biblios calling
It is very slow however since it calls rebuild_zebra.pl every time a new
authority is added (it took 10 hours to link ~12000 biblios). But for our
purposes it does what we want really well. It extends some
existing/fantastic Koha code to a batch job for a task (adding authorities)
that is otherwise non-trivial (as far as I can tell). We manage our own
To make the script work there were 3 technical issues:
1) Call rebuild_zebra.pl every time a new authority is added. This is
required for SimpleSearch to be able to find a recently added authority. I
used perl fork/exec/waitpid for this.
2) Re-indexing zebra was not enough to make SimpleSearch "see" recently
added authorities. The connection to zebra had to be reset. For this I
used the set_context, and restore_context interface. It appears that when
you have an active Context, and even if you rebuild_zebra, that context
encapsulates a stale Zebra index. So you need to create a new Context in
which to do your SimpleSearch, and then recycle that one (so you don't run
out of system resources).
3) For a long running script it appears that set_context, and
restore_context does not cleanup all open file handles, so I had to work
around a system error "no files left" or something like that. To do that I
used a shameless hack. I wrote a .csh script to call my perl script so that
it processes only 100 biblios at a time. That way when perl exits each
time, the system resources (stale file handles, etc.) are recycled. Of
course it is possible that there is a problem with my script, but the hack
worked around it and I was in a hurry.
I'd be happy to share the script with anyone who is interested. I'm not
sure what the policy is on refactoring Koha code and posting it. It only
works for MARC21 format - not sure what if anything would need to be done
Thanks for all of your help,
btw. this is one of the first perl script I have ever written, so it is not
slick if you know what I mean :)
View this message in context: http://koha.1045719.n5.nabble.com/bulk-BiblioAddsAuthorities-tp3378000p3789499.html
Sent from the Koha - Discuss mailing list archive at Nabble.com.
More information about the Koha