[Koha] BiblioAddsAuthorities: SimpleSearch failing (previously "bulk BiblioAddsAuthorities?")

[Peter Huerter]

Just following up ...

I finally got a script to work that loops through all biblios calling
BiblioAddsAuthorities.

It is very slow however since it calls rebuild_zebra.pl every time a new
authority is added (it took 10 hours to link ~12000 biblios).  But for our
purposes it does what we want really well.  It extends some
existing/fantastic Koha code to a batch job for a task (adding authorities)
that is otherwise non-trivial (as far as I can tell).  We manage our own
authorities in-house.

To make the script work there were 3 technical issues:

1) Call rebuild_zebra.pl every time a new authority is added.  This is
required for SimpleSearch to be able to find a recently added authority.  I
used perl fork/exec/waitpid for this.

2) Re-indexing zebra was not enough to make SimpleSearch "see" recently
added authorities.  The connection to zebra had to be reset.  For this I
used the set_context, and restore_context interface.  It appears that when
you have an active Context, and even if you rebuild_zebra, that context
encapsulates a stale Zebra index.  So you need to create a new Context in
which to do your SimpleSearch, and then recycle that one (so you don't run
out of system resources).

3) For a long running script it appears that set_context, and
restore_context does not cleanup all open file handles, so I had to work
around a system error "no files left" or something like that.  To do that I
used a shameless hack.  I wrote a .csh script to call my perl script so that
it processes only 100 biblios at a time.  That way when perl exits each
time, the system resources (stale file handles, etc.) are recycled.  Of
course it is possible that there is a problem with my script, but the hack
worked around it and I was in a hurry.

I'd be happy to share the script with anyone who is interested.  I'm not
sure what the policy is on refactoring Koha code and posting it.  It only
works for MARC21 format - not sure what if anything would need to be done
for UNIMARC.

Thanks for all of your help,

Pete.

btw. this is one of the first perl script I have ever written, so it is not
slick if you know what I mean :)


--
View this message in context: http://koha.1045719.n5.nabble.com/bulk-BiblioAddsAuthorities-tp3378000p3789499.html
Sent from the Koha - Discuss mailing list archive at Nabble.com.