Having problem using UNIMARC in Z39.50 target
Hi, all. I'm wondering whether anyone else is using a UNIMARC Z39.50 target and having a similar problem. We're running Koha 3.22.04.000. I've set up a target for Bibliothèque nationale de France using the following settings. Hostname: z3950.bnf.fr Port: 2211 Database: TOUT-UTF8 Userid: Z3950 Password: Z3950_BNF Syntax: UNIMARC Encoding: utf8 Record type: Bibliographic I got the configuration information from <http://www.bnf.fr/fr/professionnels/recuperation_donnees_bnf_boite_outils/a.profil_z3950_bnf.html>. In the Koha cataloging module, I can retrieve records from BnF via Z39.50, but Koha doesn't display the results correctly. Most of the columns are blank. The author column contains data from the 100 field. In UNIMARC the 100 field contains general processing data, not the author. It appears to me that even though the target is configured as UNIMARC, Koha is trying to display the results as MARC21. Is this a bug in Koha, or is there something else I need to do? I get the same results when I try the ByWater Solutions demo system. BTW, here's a copy of my YAZ session. Z> open tcp:z3950.bnf.fr:2211 /TOUT-UTF8 Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : DBV-OSI II UNIX API Name : Frontal Z3950 BnF Version: Version 1.0 Options: search present delSet scan namedResultSets Z> find @attr 1=4 history Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 15583, setno 1 records returned: 0 Z> format UNIMARC Z> show 3 Sent presentRequest (3+1). Records: 1 Record type: Unimarc 00493nam 22000733n 450 100 $a s 1873u y0fre 50 200 1 $a The History of Napoleon III, Emperor of the French, including a brie f narrative of all the most important events which have occurred in Europe since the fall of Napoleon I until the overthrow of the Second Empire and the death o f Napoleon III, by John S. C. Abbott... with illustrations 700 | $a Abbott $b John Stevens Cabot $f 1805-1877 $4 070 801 0 $a FR $b FR-751131015 $c s 1873 $g AFNOR nextResultSetPosition = 4 -- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
You can't mix UNIMARC and MARC21 in one Koha. It sounds like you are trying to load a UNIMARC record into a MARC21 framework. Koha can't do that conversion for you (in fact nothing can in a lossless way yet) Does this sound like what is happening? Chris On 1 July 2016 8:20:27 am NZST, Andy Boze <boze.1@nd.edu> wrote:
Hi, all. I'm wondering whether anyone else is using a UNIMARC Z39.50 target and having a similar problem. We're running Koha 3.22.04.000.
I've set up a target for Bibliothèque nationale de France using the following settings.
Hostname: z3950.bnf.fr Port: 2211 Database: TOUT-UTF8 Userid: Z3950 Password: Z3950_BNF Syntax: UNIMARC Encoding: utf8 Record type: Bibliographic
I got the configuration information from <http://www.bnf.fr/fr/professionnels/recuperation_donnees_bnf_boite_outils/a.profil_z3950_bnf.html>.
In the Koha cataloging module, I can retrieve records from BnF via Z39.50, but Koha doesn't display the results correctly. Most of the columns are blank. The author column contains data from the 100 field. In UNIMARC the 100 field contains general processing data, not the author. It appears to me that even though the target is configured as UNIMARC, Koha is trying to display the results as MARC21.
Is this a bug in Koha, or is there something else I need to do? I get the same results when I try the ByWater Solutions demo system. BTW, here's a copy of my YAZ session.
Z> open tcp:z3950.bnf.fr:2211 /TOUT-UTF8 Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : DBV-OSI II UNIX API Name : Frontal Z3950 BnF Version: Version 1.0 Options: search present delSet scan namedResultSets Z> find @attr 1=4 history Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 15583, setno 1 records returned: 0 Z> format UNIMARC Z> show 3 Sent presentRequest (3+1). Records: 1 Record type: Unimarc 00493nam 22000733n 450 100 $a s 1873u y0fre 50 200 1 $a The History of Napoleon III, Emperor of the French, including a brie f narrative of all the most important events which have occurred in Europe since the fall of Napoleon I until the overthrow of the Second Empire and the death o f Napoleon III, by John S. C. Abbott... with illustrations 700 | $a Abbott $b John Stevens Cabot $f 1805-1877 $4 070 801 0 $a FR $b FR-751131015 $c s 1873 $g AFNOR
nextResultSetPosition = 4
-- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
-- Sent from my Android device with K-9 Mail. Please excuse my brevity.
Ah, OK, that makes sense. I was assuming that Koha could do some sort of magic to translate from one format to the other. I'm not a cataloger, so I don't deal with this sort of thing much. Thanks for replying! On 6/30/2016 4:32 PM, Chris Cormack wrote:
You can't mix UNIMARC and MARC21 in one Koha. It sounds like you are trying to load a UNIMARC record into a MARC21 framework.
Koha can't do that conversion for you (in fact nothing can in a lossless way yet)
Does this sound like what is happening?
Chris
On 1 July 2016 8:20:27 am NZST, Andy Boze <boze.1@nd.edu> wrote:
Hi, all. I'm wondering whether anyone else is using a UNIMARC Z39.50 target and having a similar problem. We're running Koha 3.22.04.000.
I've set up a target for Bibliothèque nationale de France using the following settings.
Hostname: z3950.bnf.fr Port: 2211 Database: TOUT-UTF8 Userid: Z3950 Password: Z3950_BNF Syntax: UNIMARC Encoding: utf8 Record type: Bibliographic
I got the configuration information from <http://www.bnf.fr/fr/professionnels/recuperation_donnees_bnf_boite_outils/a.profil_z3950_bnf.html>.
In the Koha cataloging module, I can retrieve records from BnF via Z39.50, but Koha doesn't display the results correctly. Most of the columns are blank. The author column contains data from the 100 field. In UNIMARC the 100 field contains general processing data, not the author. It appears to me that even though the target is configured as UNIMARC, Koha is trying to display the results as MARC21.
Is this a bug in Koha, or is there something else I need to do? I get the same results when I try the ByWater Solutions demo system. BTW, here's a copy of my YAZ session.
Z> open tcp:z3950.bnf.fr:2211 /TOUT-UTF8 Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : DBV-OSI II UNIX API Name : Frontal Z3950 BnF Version: Version 1.0 Options: search present delSet scan namedResultSets Z> find @attr 1=4 history Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 15583, setno 1 records returned: 0 Z> format UNIMARC Z> show 3 Sent presentRequest (3+1). Records: 1 Record type: Unimarc 00493nam 22000733n 450 100 $a s 1873u y0fre 50 200 1 $a The History of Napoleon III, Emperor of the French, including a brie f narrative of all the most important events which have occurred in Europe since the fall of Napoleon I until the overthrow of the Second Empire and the death o f Napoleon III, by John S. C. Abbott... with illustrations 700 | $a Abbott $b John Stevens Cabot $f 1805-1877 $4 070 801 0 $a FR $b FR-751131015 $c s 1873 $g AFNOR
nextResultSetPosition = 4
-- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
-- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
Hi, in 3.22 you can set the path to an xslt stylesheet in Z39.50 target configuration; So if you can find an xslt UNIMARC to MARC21 you will be able to import unimarc into your koha. See https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16488 Laurence Le 30/06/2016 22:45, Andy Boze a écrit :
Ah, OK, that makes sense. I was assuming that Koha could do some sort of magic to translate from one format to the other. I'm not a cataloger, so I don't deal with this sort of thing much. Thanks for replying!
On 6/30/2016 4:32 PM, Chris Cormack wrote:
You can't mix UNIMARC and MARC21 in one Koha. It sounds like you are trying to load a UNIMARC record into a MARC21 framework.
Koha can't do that conversion for you (in fact nothing can in a lossless way yet)
Does this sound like what is happening?
Chris
On 1 July 2016 8:20:27 am NZST, Andy Boze <boze.1@nd.edu> wrote:
Hi, all. I'm wondering whether anyone else is using a UNIMARC Z39.50 target and having a similar problem. We're running Koha 3.22.04.000.
I've set up a target for Bibliothèque nationale de France using the following settings.
Hostname: z3950.bnf.fr Port: 2211 Database: TOUT-UTF8 Userid: Z3950 Password: Z3950_BNF Syntax: UNIMARC Encoding: utf8 Record type: Bibliographic
I got the configuration information from <http://www.bnf.fr/fr/professionnels/recuperation_donnees_bnf_boite_outils/a.profil_z3950_bnf.html>.
In the Koha cataloging module, I can retrieve records from BnF via Z39.50, but Koha doesn't display the results correctly. Most of the columns are blank. The author column contains data from the 100 field. In UNIMARC the 100 field contains general processing data, not the author. It appears to me that even though the target is configured as UNIMARC, Koha is trying to display the results as MARC21.
Is this a bug in Koha, or is there something else I need to do? I get the same results when I try the ByWater Solutions demo system. BTW, here's a copy of my YAZ session.
Z> open tcp:z3950.bnf.fr:2211 /TOUT-UTF8 Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : DBV-OSI II UNIX API Name : Frontal Z3950 BnF Version: Version 1.0 Options: search present delSet scan namedResultSets Z> find @attr 1=4 history Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 15583, setno 1 records returned: 0 Z> format UNIMARC Z> show 3 Sent presentRequest (3+1). Records: 1 Record type: Unimarc 00493nam 22000733n 450 100 $a s 1873u y0fre 50 200 1 $a The History of Napoleon III, Emperor of the French, including a brie f narrative of all the most important events which have occurred in Europe since the fall of Napoleon I until the overthrow of the Second Empire and the death o f Napoleon III, by John S. C. Abbott... with illustrations 700 | $a Abbott $b John Stevens Cabot $f 1805-1877 $4 070 801 0 $a FR $b FR-751131015 $c s 1873 $g AFNOR
nextResultSetPosition = 4
-- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
-- ------------------------ Laurence Rault Responsable Pôle Projets BibLibre http://www.biblibre.com laurence.rault@biblibre.com t. 09 65 22 10 71 p. 06 27 85 46 49
Thanks. I was wondering whether something like that might be a possibility. I tried using the stylesheet referred to in the bug report. As I expected, it didn't work, but it gives me a starting point to investigate further. On 7/1/2016 2:17 AM, Laurence Rault wrote:
Hi,
in 3.22 you can set the path to an xslt stylesheet in Z39.50 target configuration;
So if you can find an xslt UNIMARC to MARC21 you will be able to import unimarc into your koha. See https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16488
Laurence
Le 30/06/2016 22:45, Andy Boze a écrit :
Ah, OK, that makes sense. I was assuming that Koha could do some sort of magic to translate from one format to the other. I'm not a cataloger, so I don't deal with this sort of thing much. Thanks for replying!
On 6/30/2016 4:32 PM, Chris Cormack wrote:
You can't mix UNIMARC and MARC21 in one Koha. It sounds like you are trying to load a UNIMARC record into a MARC21 framework.
Koha can't do that conversion for you (in fact nothing can in a lossless way yet)
Does this sound like what is happening?
Chris
On 1 July 2016 8:20:27 am NZST, Andy Boze <boze.1@nd.edu> wrote:
Hi, all. I'm wondering whether anyone else is using a UNIMARC Z39.50 target and having a similar problem. We're running Koha 3.22.04.000.
I've set up a target for Bibliothèque nationale de France using the following settings.
Hostname: z3950.bnf.fr Port: 2211 Database: TOUT-UTF8 Userid: Z3950 Password: Z3950_BNF Syntax: UNIMARC Encoding: utf8 Record type: Bibliographic
I got the configuration information from <http://www.bnf.fr/fr/professionnels/recuperation_donnees_bnf_boite_outils/a.profil_z3950_bnf.html>.
In the Koha cataloging module, I can retrieve records from BnF via Z39.50, but Koha doesn't display the results correctly. Most of the columns are blank. The author column contains data from the 100 field. In UNIMARC the 100 field contains general processing data, not the author. It appears to me that even though the target is configured as UNIMARC, Koha is trying to display the results as MARC21.
Is this a bug in Koha, or is there something else I need to do? I get the same results when I try the ByWater Solutions demo system. BTW, here's a copy of my YAZ session.
Z> open tcp:z3950.bnf.fr:2211 /TOUT-UTF8 Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : DBV-OSI II UNIX API Name : Frontal Z3950 BnF Version: Version 1.0 Options: search present delSet scan namedResultSets Z> find @attr 1=4 history Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 15583, setno 1 records returned: 0 Z> format UNIMARC Z> show 3 Sent presentRequest (3+1). Records: 1 Record type: Unimarc 00493nam 22000733n 450 100 $a s 1873u y0fre 50 200 1 $a The History of Napoleon III, Emperor of the French, including a brie f narrative of all the most important events which have occurred in Europe since the fall of Napoleon I until the overthrow of the Second Empire and the death o f Napoleon III, by John S. C. Abbott... with illustrations 700 | $a Abbott $b John Stevens Cabot $f 1805-1877 $4 070 801 0 $a FR $b FR-751131015 $c s 1873 $g AFNOR
nextResultSetPosition = 4
-- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
-- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
In case anyone is interested, I'm now able to transform UNIMARC into MARC21 via Z39.50. I found that MarcEdit includes an XSLT file that works with Koha. I asked the author of MarcEdit, Terry Reese, if he would mind me sharing the information. He has kindly uploaded the file to GitHub at <https://github.com/reeset/unimarc>, from where it can be copied or downloaded. The transformation follows the UNIMARC to MARC 21 Conversion Specifications at <https://www.loc.gov/marc/unimarctomarc21.html>. Just place the file in the koha/intranet/htdocs/intranet-tmpl/prog/en/xslt directory and note it in the setup for the Z39.50 server. On 7/1/2016 3:17 PM, Andy Boze wrote:
Thanks. I was wondering whether something like that might be a possibility. I tried using the stylesheet referred to in the bug report. As I expected, it didn't work, but it gives me a starting point to investigate further.
On 7/1/2016 2:17 AM, Laurence Rault wrote:
Hi,
in 3.22 you can set the path to an xslt stylesheet in Z39.50 target configuration;
So if you can find an xslt UNIMARC to MARC21 you will be able to import unimarc into your koha. See https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16488
Laurence
Le 30/06/2016 22:45, Andy Boze a écrit :
Ah, OK, that makes sense. I was assuming that Koha could do some sort of magic to translate from one format to the other. I'm not a cataloger, so I don't deal with this sort of thing much. Thanks for replying!
On 6/30/2016 4:32 PM, Chris Cormack wrote:
You can't mix UNIMARC and MARC21 in one Koha. It sounds like you are trying to load a UNIMARC record into a MARC21 framework.
Koha can't do that conversion for you (in fact nothing can in a lossless way yet)
Does this sound like what is happening?
Chris
On 1 July 2016 8:20:27 am NZST, Andy Boze <boze.1@nd.edu> wrote:
Hi, all. I'm wondering whether anyone else is using a UNIMARC Z39.50 target and having a similar problem. We're running Koha 3.22.04.000.
I've set up a target for Bibliothèque nationale de France using the following settings.
Hostname: z3950.bnf.fr Port: 2211 Database: TOUT-UTF8 Userid: Z3950 Password: Z3950_BNF Syntax: UNIMARC Encoding: utf8 Record type: Bibliographic
I got the configuration information from <http://www.bnf.fr/fr/professionnels/recuperation_donnees_bnf_boite_outils/a.profil_z3950_bnf.html>.
In the Koha cataloging module, I can retrieve records from BnF via Z39.50, but Koha doesn't display the results correctly. Most of the columns are blank. The author column contains data from the 100 field. In UNIMARC the 100 field contains general processing data, not the author. It appears to me that even though the target is configured as UNIMARC, Koha is trying to display the results as MARC21.
Is this a bug in Koha, or is there something else I need to do? I get the same results when I try the ByWater Solutions demo system. BTW, here's a copy of my YAZ session.
Z> open tcp:z3950.bnf.fr:2211 /TOUT-UTF8 Connecting...OK. Sent initrequest. Connection accepted by v3 target. ID : DBV-OSI II UNIX API Name : Frontal Z3950 BnF Version: Version 1.0 Options: search present delSet scan namedResultSets Z> find @attr 1=4 history Sent searchRequest. Received SearchResponse. Search was a success. Number of hits: 15583, setno 1 records returned: 0 Z> format UNIMARC Z> show 3 Sent presentRequest (3+1). Records: 1 Record type: Unimarc 00493nam 22000733n 450 100 $a s 1873u y0fre 50 200 1 $a The History of Napoleon III, Emperor of the French, including a brie f narrative of all the most important events which have occurred in Europe since the fall of Napoleon I until the overthrow of the Second Empire and the death o f Napoleon III, by John S. C. Abbott... with illustrations 700 | $a Abbott $b John Stevens Cabot $f 1805-1877 $4 070 801 0 $a FR $b FR-751131015 $c s 1873 $g AFNOR
nextResultSetPosition = 4
-- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
-- Andy Boze, Associate Librarian University of Notre Dame 208A Hesburgh Library (574) 631-8708
Dear all, I have just uploaded to BZ a big update of the original UNIMARC->MARC21 xsl that was currently available. You may find the file in the appropriate BZ case (https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=16488) and test it for yourself. You must copy the file to koha-tmpl/intranet-tmpl/prog/en/xslt/ And then put "UNIMARC2MARC21.xsl" in "XSLT File(s) for transforming results:" textfield of a UNIMARC Z39.50 source that you wish to translate to MARC21 The new version has better handling of Unimarc-100 (MARC21-008) field, and supports the translation of far more UNIMARC fields. Almost all of the code was based on the (almost 15-year-old!) conversion guidelines by LOC, so some fields/subfields might now be deprecated. Even worse, important new fields might be missing from the translation. (856 field was such an example, so by reading more recent docs, I think I translated it properly). Warnings: - The work is nowhere near complete, but it's better (and more complete) that what was currently available online. - My knowledge of UNIMARC is very-very limited. - The xsl was not thoroughly tested and the time I had to spend on it was limited. Expect to find errors :) Known issues: Although the xsl /seems/ to work fine for me, there are certain encoding issues when using it to IMPORT utf8 UNIMARC records to Koha. Displaying of UTF8 records in the Koha Z39.50 result list is OK, but importing them to cataloging is not. It's possible that it's an issue of Koha code and not related to this xsl. (I will open a BZ case soon). Best regards, Theodoros Theodoropoulos
participants (4)
-
Andy Boze -
Chris Cormack -
Laurence Rault -
Theodoros Theodoropoulos