Re: [Koha] Staging of import stalls
Ed, I also had issues getting zebra working initially as we had most of the biblios in Unicode Marathi.So while I was playing with zebra we used nozebra to get the services off the ground and then switched to zebra once we had solved the zebra search of Unicode Marathi. 1) Turn off noZebra from system preferences. For testing Zebra you can run it from command line. sudo -u ${KOHA_USER} zebrasrv -f /etc/koha/koha-conf.xml adjust the path to koha-conf.xml if different. The first time I ran this it refused to run as it was not finding some directories. I had to create those and change ownerships to koha user. I forget exact paths but a look at the output will tell you. Once its running, do your bulk import by staging and then importing. After that in a separate terminal run misc/migration_tools/rebuild_zebra.pl -b -w This should build your index and then you should be able to search from opac & stff client. 2) Once you have the search going you can run the zebra server and zebraqueue in daemon mode and have them start on system boot. Here is how to do it on Ubuntu, adjust according to your OS. sudo ln -s ${SCRIPT_DIR}/koha-zebra-ctl.sh /etc/init.d/koha-zebra-daemon ${SCRIPT_DIR} is /usr/share/koha/bin/ by default in a standard install. sudo update-rc.d koha-zebra-daemon defaults sudo /etc/init.d/koha-zebra-daemon start Zebraqueue Daemon: sudo ln -s ${SCRIPT_DIR}/koha-zebraqueue-ctl.sh /etc/init.d/koha-zebraqueue-daemon sudo update-rc.d koha-zebraqueue-daemon defaults sudo /etc/init.d/koha-zebraqueue-daemon start Thats it. the zebraqueue will take care of indexing any subsequent additions you make. If you bulk import any more biblios run rebiuld_zebra.pl again as zebraqueue is slower in indexing. Also run rebuild_zebra.pl for your authorities. The daemons will log in /var/log/koha/ Have a look at files in there if something doesn't work out. -- Regards, Koustubha Kale Anant Corporation Contact Details : Address : 103, Armaan Residency, R. W Sawant Road, Nr. Golden Dyes Naka, Thane (w), Maharashtra, India, Pin : 400601. TeleFax : +91-22-21720108, +91-22-21720109 Mobile : +919820715876 Website : http://www.anantcorp.com Original Message: ----------------- From: Roche III, Edward edward_roche@solanco.k12.pa.us Date: Sat, 28 Feb 2009 22:33:19 -0500 To: kmkale@anantcorp.com, dearden@sarsf.org, koha@lists.katipo.co.nz Subject: RE: [Koha] Staging of import stalls how would you get the zebra server running as well as the daemans? I originally tries to use zebra but it was giving me issues so I decide to go the nozebra route. Thanks Ed Edward Roche Technology Support/Network Admin Solanco School District edward_roche@solanco.k12.pa.us 717.786.2151 The greatest mistake you can make in life is to continually be afraid you will make one. Elbert Hubbard -----Original Message----- From: Koustubha Kale [mailto:kmkale@anantcorp.com] Sent: Sat 2/28/2009 2:31 AM To: Doug Dearden Cc: Roche III, Edward; koha@lists.katipo.co.nz Subject: Re: [Koha] Staging of import stalls ED, Do try Doug's suggestion. I have also faced this problem. removing the problem record solves the stall. Also I have staged and imported more than 87000 records in one go, so the batch process does work well. In case you are not using zebra, one trick i observed was to have the zebra server and queue daemons running during the import and put nozebra to off even if you are not using zebra. this speeds up the staging and import by orders of magnitude. after the import is over you can run the rebuild_nozebra.pl to rebuild the index after turning on nozebra setting. This little trick resulted in hours of time saving. -- Regards, Koustubha Kale Anant Corporation Contact Details : Address : 103, Armaan Residency, R. W Sawant Road, Nr. Golden Dyes Naka, Thane (w), Maharashtra, India, Pin : 400601. TeleFax : +91-22-21720108, +91-22-21720109 Mobile : +919820715876 Website : http://www.anantcorp.com Doug Dearden wrote:
Hi Ed,
I had the same problem. Take a look at the last import (Home > Tools > Manage Staged MARC Records > Batch
Figure out the last record in the batch that went in, then edit the import file you are using and remove the record immediately after that one in the file. It has some kind of problem. Try again and see if you get further. It is either that record, or the last one that actually worked, I can't remember exactly which.
Also, there is a command line tool that you can use to import. You will find it at /usr/share/koha/bin/migration_tools/bulkmarcimport.pl
From the migration tools subdirectory enter: perl bulkmarcimport.pl and you will get the syntax and parameter info. It has a test mode you can run against the file.
Best,
Doug
"Roche III, Edward" <edward_roche@solanco.k12.pa.us> 2/27/2009 9:20 AM
Good Morning All
I am trying stage my MARC import data but it keeps stalling. I broke it down into 4 individual files of about 4000 records each. But it still stalls at about 15%. Has anyone had this issue? Can it be done from the terminal instead of through the browser? Every time I start the process it pins my cpu at 95 to 100% which is why it is probably stalling but not sure why.
My Config
----------------------------------
Ubuntu 8.1
1.5gb memory
Pentium 4 - 2.5ghz dual core
Koha 3.0
NoZebra
Basic LAMP install
Any thoughts would be great.
Thanks
Ed
Edward J. Roche
Network Administrator
Solanco School District
717.786.2151 x2437
"At the End of the Day, Did You Play to Win or Not to Lose?"
_______________________________________________ Koha mailing list Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
-------------------------------------------------------------------- mail2web LIVE Free email based on Microsoft® Exchange technology - http://link.mail2web.com/LIVE
participants (1)
-
kmkale@anantcorp.com