[Koha] Incomplete imports

Mon Jun 8 17:34:25 NZST 2009

Dear Justin,

I recently experienced maybe a similar import scenario: in the end I 
failed due to some unprocessable MARC records, that passed the MARC 
Test (Marc Editor), but froze the koha import process mysteriously 
without any proper error message. They were hiding approximately 20 to 
60 records upfront(!) of the last processed record and all had some 
special characters in the 240 field.

Best regards,
Marc

koha-request at lists.katipo.co.nz schrieb:
> Send Koha mailing list submissions to
> 	koha at lists.katipo.co.nz
> 
> To subscribe or unsubscribe via the World Wide Web, visit
> 	http://lists.katipo.co.nz/mailman/listinfo/koha
> or, via email, send a message with subject or body 'help' to
> 	koha-request at lists.katipo.co.nz
> 
> You can reach the person managing the list at
> 	koha-owner at lists.katipo.co.nz
> 
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Koha digest..."
> 
> 
> Today's Topics:
> 
>    1. Incomplete imports (Justin Aquadro)
>    2. Re: Incomplete imports (Cab Vinton)
>    3. Re: Incomplete imports (Chris Cormack)
> 
> 
> ----------------------------------------------------------------------
> 
> Message: 1
> Date: Sat, 06 Jun 2009 23:24:51 -0400
> From: Justin Aquadro <jaquadro at gmail.com>
> Subject: [Koha] Incomplete imports
> To: koha at lists.katipo.co.nz
> Message-ID: <4A2B3303.3080603 at gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
> 
> Greetings all,
> 
> We are looking into using Koha in our school system to replace the aging 
> Winnebago Spectrum.  I've installed 3.0.1 on a test server to play 
> with.  I've become hung up on the imports however.  Taking an export of 
> ~15000 records from Spectrum, I swapped the 852 fields to 952 in 
> MarcEdit according to the wiki instructions, and staged the import 
> without problem.  However when I started importing the staged records, 
> it made it through 160 records and just stopped, no error messages or 
> any kind of feedback from the import page.  Now I have a partially 
> complete import that I can't resume or roll back.  The same thing 
> happened when I tried breaking the import into batches of 1000, the 
> script just stopped processing after an arbitrary number of records.
> 
> Can anyone offer some insight into what's causing these frozen, 
> incomplete imports?  Since there's no errors, I'm at a loss.  Also, how 
> can I cleanly remove these frozen batches from the staging area?
> 
> Many thanks,
> Justin.
> 
> P.S. I apologize in advance if this message was accidentally dispatched 
> twice.
> 
> 
> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4136 (20090606) __________
> 
> The message was checked by ESET NOD32 Antivirus.
> 
> http://www.eset.com
> 
> 
> 
> 
> ------------------------------
> 
> Message: 2
> Date: Sun, 7 Jun 2009 09:24:20 -0400
> From: Cab Vinton <bibliwho at gmail.com>
> Subject: Re: [Koha] Incomplete imports
> To: Justin Aquadro <jaquadro at gmail.com>
> Cc: koha at lists.katipo.co.nz
> Message-ID:
> 	<835777770906070624j6278f551rb554e87cad215441 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> One issue that can result in unsuccessful imports is records coming in
> as MARC-8 rather than UTF8 format.
> 
> MarcBreaker will let you translate the records from the former to the latter.
> 
> We learned the hard way that the records coming from our state's union
> catalog are in the wrong format.
> 
> Hope this helps,
> 
> Cab Vinton, Director
> Sanbornton Public Library
> Sanbornton, NH
> 
> 
> On Sat, Jun 6, 2009 at 11:24 PM, Justin Aquadro<jaquadro at gmail.com> wrote:
>> Greetings all,
>>
>> We are looking into using Koha in our school system to replace the aging
>> Winnebago Spectrum. ?I've installed 3.0.1 on a test server to play
>> with. ?I've become hung up on the imports however. ?Taking an export of
>> ~15000 records from Spectrum, I swapped the 852 fields to 952 in
>> MarcEdit according to the wiki instructions, and staged the import
>> without problem. ?However when I started importing the staged records,
>> it made it through 160 records and just stopped, no error messages or
>> any kind of feedback from the import page. ?Now I have a partially
>> complete import that I can't resume or roll back. ?The same thing
>> happened when I tried breaking the import into batches of 1000, the
>> script just stopped processing after an arbitrary number of records.
>>
>> Can anyone offer some insight into what's causing these frozen,
>> incomplete imports? ?Since there's no errors, I'm at a loss. ?Also, how
>> can I cleanly remove these frozen batches from the staging area?
>>
>> Many thanks,
>> Justin.
>>
>> P.S. I apologize in advance if this message was accidentally dispatched
>> twice.
>>
>>
>> __________ Information from ESET NOD32 Antivirus, version of virus signature database 4136 (20090606) __________
>>
>> The message was checked by ESET NOD32 Antivirus.
>>
>> http://www.eset.com
>>
>>
>> _______________________________________________
>> Koha mailing list
>> Koha at lists.katipo.co.nz
>> http://lists.katipo.co.nz/mailman/listinfo/koha
>>
> 
> 
> ------------------------------
> 
> Message: 3
> Date: Mon, 8 Jun 2009 08:20:47 +1200
> From: Chris Cormack <chris at bigballofwax.co.nz>
> Subject: Re: [Koha] Incomplete imports
> To: Justin Aquadro <jaquadro at gmail.com>
> Cc: koha at lists.katipo.co.nz
> Message-ID:
> 	<606aab810906071320s72387f7et5250c69b0696f549 at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
> 
> 2009/6/7 Justin Aquadro <jaquadro at gmail.com>:
>> Greetings all,
>>
>> We are looking into using Koha in our school system to replace the aging
>> Winnebago Spectrum. ?I've installed 3.0.1 on a test server to play
>> with. ?I've become hung up on the imports however. ?Taking an export of
>> ~15000 records from Spectrum, I swapped the 852 fields to 952 in
>> MarcEdit according to the wiki instructions, and staged the import
>> without problem. ?However when I started importing the staged records,
>> it made it through 160 records and just stopped, no error messages or
>> any kind of feedback from the import page. ?Now I have a partially
>> complete import that I can't resume or roll back. ?The same thing
>> happened when I tried breaking the import into batches of 1000, the
>> script just stopped processing after an arbitrary number of records.
>>
>> Can anyone offer some insight into what's causing these frozen,
>> incomplete imports? ?Since there's no errors, I'm at a loss. ?Also, how
>> can I cleanly remove these frozen batches from the staging area?
>>
> Hi Justin
> 
> This sounds a lot like bug 2926
> http://bugs.koha.org/cgi-bin/bugzilla3/show_bug.cgi?id=2926
> 
> Which as it happens the awesome Galen Charlton (our fearless release
> manager) has just fixed.
> 
> http://git.koha.org/cgi-bin/gitweb.cgi?p=Koha;a=commit;h=da51de184c1179fd725f0e341c9186b7f5cd585e
> 
> Chris
> 
> 
> ------------------------------
> 
> _______________________________________________
> Koha mailing list
> Koha at lists.katipo.co.nz
> http://lists.katipo.co.nz/mailman/listinfo/koha
> 
> 
> End of Koha Digest, Vol 44, Issue 17
> ************************************
> 