bulkmarcimport.pl AND -match= fails when trying to import (bibliographic) records
Hello there! This is my first message to the list, nice to meet you all. I am a Systems Librarian at FCEN, Universidad de Buenos Aires. This is the thing: I am trying to import some bibliographic records into KOHA with BULKMARCIMPORT.PL I first imported the records with bulkmarcimport.pl perl /usr/share/koha/bin/migration_tools/bulkmarcimport.pl -biblios -v -fk -all -commit 1000 -file=/home/nrucks/20recs.mrc (file /home/nrucks/20recs.mrc contains 20 marc bibliographic records) So far so good, everything is OK, I can see the 20 new records reflected on the website, on MYSQL database and in zebra indexes (rebuilt with koha-rebuild-zebra). Then I made some modifications to the original 20 records and tried to import then again with BULKMARCIMPORT.PL , only this time using the -match= option, because I don't want to ADD records, but REPLACE them with corrections. Field 035 of each record contains and keeps a unique identifier like the following: 035 $a (Ag-BiblioFCEN)000000002 perl /usr/share/koha/bin/migration_tools/bulkmarcimport.pl -biblios -v -fk -all -commit 1000 -file=/home/nrucks/20recs-modified.mrc -match=035 (file /home/nrucks/20recs-modified.mrc has 20 marc bibliographic records, slightly modified from 20recs.mrc ) But this always fails... I get this for each of the 20 records: << .Use of uninitialized value $recorddata in pattern match (m//) at /usr/share/koha/bin/migration_tools/bulkmarcimport.pl line 582, <GEN46> line 1. unable to search the database for duplicates : No query entered at /usr/share/koha/bin/migration_tools/bulkmarcimport.pl line 290, <GEN46> line 1.
I tried some variants like -match=035a, with same results. Note that the marc records are OK: I did try the same through the website, and worked fine: records were correctly REPLACED. http://.../cgi-bin/koha/tools/stage-marc-import.pl http://.../cgi-bin/koha/tools/manage-marc-import.pl <select name="overlay_action" id="overlay_action"> <option value="replace" selected="selected">Reemplace un registro existente con uno nuevo</option> But I plan to do this on a larger scale, and anyway, bulkmarcimport.pl should work. Any recommendations as to the syntax of -match option? , or conditions of zebra index? Or options to use another field (-i ISBN will not do since many of our books do not have it)? Or even some kind of hardcode workaround to BULKMARCIMPORT.PL (meaning "use THIS FIELD to check if record already exists"), or any other things I am obviously not taking into account? Thanks in advance! • VERSIONS of my installed KOHA (as stated in http:/.../about.pl): Versión de Koha: 17.05.02.000 Versión del SO ('uname -a'): Linux koha 2.6.32-44-pve #1 SMP Sun Mar 13 17:00:54 CET 2016 x86_64 Versión de Perl: 5.020002 Versión de MySQL: mysql Ver 14.14 Distrib 5.5.57, for debian-linux-gnu (x86_64) using readline 6.3 Versión de Apache: Server version: Apache/2.4.10 (Debian) Versión de Zebra: Zebra 2.0.59 (C) 1994-2014, Nicolas Rucks Biblioteca Central UBA-FCEN "Machines should not be friendly. Machines should be obedient."
Hello again! Maybe this too easy, or too difficult, maybe I am missing a point. As I could manage to understand, bulkmarcimport.pl requieres "-match=" with a comma (,) (From bulkmarcimport.pl DESCRIPTION:) -match=<FIELD> FIELD matchindex,fieldtomatch Could anybody show me a concrete example of "matchindex" value? how should the command be? Meanwhile, I tried this perl /usr/share/koha/bin/migration_tools/bulkmarcimport.pl -biblios -v -fk -all -commit 1000 -file=/home/nrucks/20recs-modified.mrc -match=9001,035a and got, for each of the 20 records, this: CCL parsing error (10014) Unknown qualifier ZOOM for query: 9001="(Ag-BiblioFCEN)000000020" at /usr/share/koha/lib/C4/Search.pm line 275. unable to search the database for duplicates : CCL parsing error (10014) Unknown qualifier ZOOM at /usr/share/koha/bin/migration_tools/patriciobulkmarcimport.pl line 290. Any help much appreciated Thanks and happy Easter. Nicolas Rucks Biblioteca Central UBA-FCEN "Machines should not be friendly. Machines should be obedient." ----- Mensaje original ----- De: "Nicolas Rucks" <nrucks@bl.fcen.uba.ar> Para: koha@lists.katipo.co.nz Enviados: Jueves, 22 de Marzo 2018 17:29:45 Asunto: [Koha] bulkmarcimport.pl AND -match= fails when trying to import (bibliographic) records Hello there! This is my first message to the list, nice to meet you all. I am a Systems Librarian at FCEN, Universidad de Buenos Aires. This is the thing: I am trying to import some bibliographic records into KOHA with BULKMARCIMPORT.PL I first imported the records with bulkmarcimport.pl perl /usr/share/koha/bin/migration_tools/bulkmarcimport.pl -biblios -v -fk -all -commit 1000 -file=/home/nrucks/20recs.mrc (file /home/nrucks/20recs.mrc contains 20 marc bibliographic records) So far so good, everything is OK, I can see the 20 new records reflected on the website, on MYSQL database and in zebra indexes (rebuilt with koha-rebuild-zebra). Then I made some modifications to the original 20 records and tried to import then again with BULKMARCIMPORT.PL , only this time using the -match= option, because I don't want to ADD records, but REPLACE them with corrections. Field 035 of each record contains and keeps a unique identifier like the following: 035 $a (Ag-BiblioFCEN)000000002 perl /usr/share/koha/bin/migration_tools/bulkmarcimport.pl -biblios -v -fk -all -commit 1000 -file=/home/nrucks/20recs-modified.mrc -match=035 (file /home/nrucks/20recs-modified.mrc has 20 marc bibliographic records, slightly modified from 20recs.mrc ) But this always fails... I get this for each of the 20 records: << .Use of uninitialized value $recorddata in pattern match (m//) at /usr/share/koha/bin/migration_tools/bulkmarcimport.pl line 582, <GEN46> line 1. unable to search the database for duplicates : No query entered at /usr/share/koha/bin/migration_tools/bulkmarcimport.pl line 290, <GEN46> line 1.
I tried some variants like -match=035a, with same results. Note that the marc records are OK: I did try the same through the website, and worked fine: records were correctly REPLACED. http://.../cgi-bin/koha/tools/stage-marc-import.pl http://.../cgi-bin/koha/tools/manage-marc-import.pl <select name="overlay_action" id="overlay_action"> <option value="replace" selected="selected">Reemplace un registro existente con uno nuevo</option> But I plan to do this on a larger scale, and anyway, bulkmarcimport.pl should work. Any recommendations as to the syntax of -match option? , or conditions of zebra index? Or options to use another field (-i ISBN will not do since many of our books do not have it)? Or even some kind of hardcode workaround to BULKMARCIMPORT.PL (meaning "use THIS FIELD to check if record already exists"), or any other things I am obviously not taking into account? Thanks in advance! • VERSIONS of my installed KOHA (as stated in http:/.../about.pl): Versión de Koha: 17.05.02.000 Versión del SO ('uname -a'): Linux koha 2.6.32-44-pve #1 SMP Sun Mar 13 17:00:54 CET 2016 x86_64 Versión de Perl: 5.020002 Versión de MySQL: mysql Ver 14.14 Distrib 5.5.57, for debian-linux-gnu (x86_64) using readline 6.3 Versión de Apache: Server version: Apache/2.4.10 (Debian) Versión de Zebra: Zebra 2.0.59 (C) 1994-2014, Nicolas Rucks Biblioteca Central UBA-FCEN "Machines should not be friendly. Machines should be obedient." _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
On Wed, Mar 28, 2018 at 04:58:30PM -0300, Nicolas Rucks wrote:
Hello again! Maybe this too easy, or too difficult, maybe I am missing a point. As I could manage to understand, bulkmarcimport.pl requieres "-match=" with a comma (,)
(From bulkmarcimport.pl DESCRIPTION:) -match=<FIELD> FIELD matchindex,fieldtomatch
Could anybody show me a concrete example of "matchindex" value? how should the command be?
The part before the comma has to be a field name, not a Z39.50 BIB-1 USE attribute number. So, for example, using ISBNs would be like this: -match=isbn,020a I figured out the field name for 035 $a once but have forgotten. control-number, maybe: -match=control-number,035a No, that doesn't seem to be it. Argh. I'll dig around in the Z39.50 config files. Paul. -- Paul Hoffman <paul@flo.org> Software Manager Fenway Library Organization 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
On Wed, Mar 28, 2018 at 04:35:30PM -0400, Paul Hoffman wrote:
On Wed, Mar 28, 2018 at 04:58:30PM -0300, Nicolas Rucks wrote:
Hello again! Maybe this too easy, or too difficult, maybe I am missing a point. As I could manage to understand, bulkmarcimport.pl requieres "-match=" with a comma (,)
(From bulkmarcimport.pl DESCRIPTION:) -match=<FIELD> FIELD matchindex,fieldtomatch
Could anybody show me a concrete example of "matchindex" value? how should the command be?
The part before the comma has to be a field name, not a Z39.50 BIB-1 USE attribute number. So, for example, using ISBNs would be like this:
-match=isbn,020a
I figured out the field name for 035 $a once but have forgotten. control-number, maybe:
-match=control-number,035a
No, that doesn't seem to be it. Argh. I'll dig around in the Z39.50 config files.
It's Other-control-number: -match Other-control-number,035a Or at least I can search using that field name to find things by OCLC number. On the other hand, that appears to index 035 $z as well as 035 $a -- that could be a problem, since $z contains cancelled or invalid numbers. Here's how I found it: # fgrep 035 /etc/koha/zebradb/*/*/biblios/record.abs /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035$a Other-control-number /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035$z Other-control-number /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035 Other-control-number /etc/koha/zebradb/marc_defs/normarc/biblios/record.abs:#melm 035 Local-number,Identifier-standard Paul. -- Paul Hoffman <paul@flo.org> Software Manager Fenway Library Organization 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number)
BIG YEAY! Thanks Paul!, it did work. The 20 records were properly replaced (as far as I quickly checked). perl /usr/share/koha/bin/migration_tools/bulkmarcimport.pl -biblios -v -fk -all -commit 1000 -file=/home/nrucks/20recs-modified.mrc -match Other-control-number,035a Nicolas Rucks Biblioteca Central UBA-FCEN Argentina "Machines should not be friendly. Machines should be obedient." (Working with an unfriendly yet obedient machine) ----- Mensaje original ----- De: "Paul Hoffman" <paul@flo.org> Para: koha@lists.katipo.co.nz Enviados: Miércoles, 28 de Marzo 2018 18:04:47 Asunto: Re: [Koha] bulkmarcimport.pl AND -match= fails when trying to import (bibliographic) records On Wed, Mar 28, 2018 at 04:35:30PM -0400, Paul Hoffman wrote:
On Wed, Mar 28, 2018 at 04:58:30PM -0300, Nicolas Rucks wrote:
Hello again! Maybe this too easy, or too difficult, maybe I am missing a point. As I could manage to understand, bulkmarcimport.pl requieres "-match=" with a comma (,)
(From bulkmarcimport.pl DESCRIPTION:) -match=<FIELD> FIELD matchindex,fieldtomatch
Could anybody show me a concrete example of "matchindex" value? how should the command be?
The part before the comma has to be a field name, not a Z39.50 BIB-1 USE attribute number. So, for example, using ISBNs would be like this:
-match=isbn,020a
I figured out the field name for 035 $a once but have forgotten. control-number, maybe:
-match=control-number,035a
No, that doesn't seem to be it. Argh. I'll dig around in the Z39.50 config files.
It's Other-control-number: -match Other-control-number,035a Or at least I can search using that field name to find things by OCLC number. On the other hand, that appears to index 035 $z as well as 035 $a -- that could be a problem, since $z contains cancelled or invalid numbers. Here's how I found it: # fgrep 035 /etc/koha/zebradb/*/*/biblios/record.abs /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035$a Other-control-number /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035$z Other-control-number /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035 Other-control-number /etc/koha/zebradb/marc_defs/normarc/biblios/record.abs:#melm 035 Local-number,Identifier-standard Paul. -- Paul Hoffman <paul@flo.org> Software Manager Fenway Library Organization 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number) _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
Nicolas, Paul: great to see you figured the issue. But for the record: indexes generation is defined in /etc/koha/zebradb/marc_defs/{marc flavour}/biblios/biblio-koha-indexdefs.xml and not (anymore) in record.abs. I mention this because record.abs might not be accurate a few years back to now. Kind regards El jue., 29 mar. 2018 a las 1:14, Nicolas Rucks (<nrucks@bl.fcen.uba.ar>) escribió:
BIG YEAY! Thanks Paul!, it did work. The 20 records were properly replaced (as far as I quickly checked).
perl /usr/share/koha/bin/migration_tools/bulkmarcimport.pl -biblios -v -fk -all -commit 1000 -file=/home/nrucks/20recs-modified.mrc -match Other-control-number,035a
Nicolas Rucks Biblioteca Central UBA-FCEN Argentina
"Machines should not be friendly. Machines should be obedient." (Working with an unfriendly yet obedient machine)
----- Mensaje original ----- De: "Paul Hoffman" <paul@flo.org> Para: koha@lists.katipo.co.nz Enviados: Miércoles, 28 de Marzo 2018 18:04:47 Asunto: Re: [Koha] bulkmarcimport.pl AND -match= fails when trying to import (bibliographic) records
On Wed, Mar 28, 2018 at 04:35:30PM -0400, Paul Hoffman wrote:
On Wed, Mar 28, 2018 at 04:58:30PM -0300, Nicolas Rucks wrote:
Hello again! Maybe this too easy, or too difficult, maybe I am missing a point. As I could manage to understand, bulkmarcimport.pl requieres "-match=" with a comma (,)
(From bulkmarcimport.pl DESCRIPTION:) -match=<FIELD> FIELD matchindex,fieldtomatch
Could anybody show me a concrete example of "matchindex" value? how should the command be?
The part before the comma has to be a field name, not a Z39.50 BIB-1 USE attribute number. So, for example, using ISBNs would be like this:
-match=isbn,020a
I figured out the field name for 035 $a once but have forgotten. control-number, maybe:
-match=control-number,035a
No, that doesn't seem to be it. Argh. I'll dig around in the Z39.50 config files.
It's Other-control-number:
-match Other-control-number,035a
Or at least I can search using that field name to find things by OCLC number. On the other hand, that appears to index 035 $z as well as 035 $a -- that could be a problem, since $z contains cancelled or invalid numbers.
Here's how I found it:
# fgrep 035 /etc/koha/zebradb/*/*/biblios/record.abs /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035$a Other-control-number /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035$z Other-control-number /etc/koha/zebradb/marc_defs/marc21/biblios/record.abs:melm 035 Other-control-number /etc/koha/zebradb/marc_defs/normarc/biblios/record.abs:#melm 035 Local-number,Identifier-standard
Paul.
-- Paul Hoffman <paul@flo.org> Software M <https://maps.google.com/?q=-+%0APaul+Hoffman++%0ASoftware+M&entry=gmail&source=g> anager Fenway Library Organization 550 Huntington Ave. Boston, MA 02115 (617) 442-2384 (FLO main number) _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
-- Tomás Cohen Arazi Theke Solutions (https://theke.io <http://theke.io/>) ✆ +54 9351 3513384 GPG: B2F3C15F
participants (3)
-
Nicolas Rucks -
Paul Hoffman -
Tomas Cohen Arazi