Multiple problems in MARC download and import
We're using Koha 19.05.02 I have a txt file of biblio numbers that need to all have the contents of one field changed. 477 numbers (no dupes, I checked). I then went through the process of exporting these records. When I open the file in MarcEdit, it says I have 225 records. I don't know what happened to the other 252. I decided to go through the rest of the process, just to see if it worked as expected (this is why we have a test site). I edited the file, saved it as mrc and tried to import it to Koha. Where I got a very large error message saying nothing imported SOFTWARE ERROR, contact your webmaster. Unfortunately, the tech half of the team is away at the moment. . . so any help would be appreciated. I'm especially curious as to why I uploaded a list of 477 biblio numbers but I got only 225 records in return. I'm now going to experiment with batch editing within Koha. . . wish me luck! Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999
Hi Elaine This might happen when you have a linux txt file and open it with a windows editor or vice-versa. This is usually due to windows and linux using different line terminators characters [CR] vs [CR]+[LF]. You might also need to verify that, when doing your replacement, you didn't added/removed special characters or spaces. You can do some test with just a few items to see if you find the issue. Regards Alvaro |-----------------------------------------------------------------------------------------------------------------| Envíe y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier celular y Nextel en el Perú, México y en mas de 180 paises. Use aplicaciones 2 vias via SMS y GPRS online Visitenos en www.perusms.com Le lun. 19 août 2019 à 12:38, Elaine Bradtke <eb@efdss.org> a écrit :
We're using Koha 19.05.02 I have a txt file of biblio numbers that need to all have the contents of one field changed. 477 numbers (no dupes, I checked). I then went through the process of exporting these records. When I open the file in MarcEdit, it says I have 225 records. I don't know what happened to the other 252.
I decided to go through the rest of the process, just to see if it worked as expected (this is why we have a test site). I edited the file, saved it as mrc and tried to import it to Koha. Where I got a very large error message saying nothing imported SOFTWARE ERROR, contact your webmaster.
Unfortunately, the tech half of the team is away at the moment. . . so any help would be appreciated. I'm especially curious as to why I uploaded a list of 477 biblio numbers but I got only 225 records in return.
I'm now going to experiment with batch editing within Koha. . . wish me luck! Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999 _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
It was all done on Windows 10, using Libre Office and MarcEdit. I've had trouble before with uploads failing because I usually use a Mac, and it has a similar problem with line terminators. I thought I'd be safer on Windows. Hah. I'll look again at the file, but all I did was change http: to https: Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999 On Mon, Aug 19, 2019 at 1:28 PM Alvaro Cornejo <cornejo.alvaro@gmail.com> wrote:
Hi Elaine
This might happen when you have a linux txt file and open it with a windows editor or vice-versa. This is usually due to windows and linux using different line terminators characters [CR] vs [CR]+[LF].
You might also need to verify that, when doing your replacement, you didn't added/removed special characters or spaces.
You can do some test with just a few items to see if you find the issue.
Regards
Alvaro
|-----------------------------------------------------------------------------------------------------------------| Envíe y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier celular y Nextel en el Perú, México y en mas de 180 paises. Use aplicaciones 2 vias via SMS y GPRS online Visitenos en www.perusms.com
Le lun. 19 août 2019 à 12:38, Elaine Bradtke <eb@efdss.org> a écrit :
We're using Koha 19.05.02 I have a txt file of biblio numbers that need to all have the contents of one field changed. 477 numbers (no dupes, I checked). I then went through the process of exporting these records. When I open the file in MarcEdit, it says I have 225 records. I don't know what happened to the other 252.
I decided to go through the rest of the process, just to see if it worked as expected (this is why we have a test site). I edited the file, saved it as mrc and tried to import it to Koha. Where I got a very large error message saying nothing imported SOFTWARE ERROR, contact your webmaster.
Unfortunately, the tech half of the team is away at the moment. . . so any help would be appreciated. I'm especially curious as to why I uploaded a list of 477 biblio numbers but I got only 225 records in return.
I'm now going to experiment with batch editing within Koha. . . wish me luck! Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999 _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
Hi Elaine You could run unix2dos on the file (http://dos2unix.sourceforge.net/), or if you open the file in Notepad++ (https://notepad-plus-plus.org/) you can then go to 'Edit > EOL Conversion > Windows' and save the document. With kind regards from the Dalton McCaughey Library Team Carlos Lopez Dalton McCaughey Library | 29 College Crescent, Parkville, VICTORIA 3052 Ph: 03 9340 8888 ext.1 | library@dml.vic.edu.au | library.dmlibrary.org.au -----Original Message----- From: Koha <koha-bounces@lists.katipo.co.nz> On Behalf Of Elaine Bradtke Sent: Tuesday, 20 August 2019 9:35 AM Cc: koha <koha@lists.katipo.co.nz> Subject: Re: [Koha] Multiple problems in MARC download and import It was all done on Windows 10, using Libre Office and MarcEdit. I've had trouble before with uploads failing because I usually use a Mac, and it has a similar problem with line terminators. I thought I'd be safer on Windows. Hah. I'll look again at the file, but all I did was change http: to https: Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999 On Mon, Aug 19, 2019 at 1:28 PM Alvaro Cornejo <cornejo.alvaro@gmail.com> wrote:
Hi Elaine
This might happen when you have a linux txt file and open it with a windows editor or vice-versa. This is usually due to windows and linux using different line terminators characters [CR] vs [CR]+[LF].
You might also need to verify that, when doing your replacement, you didn't added/removed special characters or spaces.
You can do some test with just a few items to see if you find the issue.
Regards
Alvaro
|-----------------------------------------------------------------------------------------------------------------| Envíe y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier celular y Nextel en el Perú, México y en mas de 180 paises. Use aplicaciones 2 vias via SMS y GPRS online Visitenos en www.perusms.com
Le lun. 19 août 2019 à 12:38, Elaine Bradtke <eb@efdss.org> a écrit :
We're using Koha 19.05.02 I have a txt file of biblio numbers that need to all have the contents of one field changed. 477 numbers (no dupes, I checked). I then went through the process of exporting these records. When I open the file in MarcEdit, it says I have 225 records. I don't know what happened to the other 252.
I decided to go through the rest of the process, just to see if it worked as expected (this is why we have a test site). I edited the file, saved it as mrc and tried to import it to Koha. Where I got a very large error message saying nothing imported SOFTWARE ERROR, contact your webmaster.
Unfortunately, the tech half of the team is away at the moment. . . so any help would be appreciated. I'm especially curious as to why I uploaded a list of 477 biblio numbers but I got only 225 records in return.
I'm now going to experiment with batch editing within Koha. . . wish me luck! Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999 _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha -- Click here to report this message as spam: https://console.mailguard.com.au/ras/1WlL9msK4Y/3ENKksR4Kfg8qPdlVfxsJm/2.3
Line endings are not the problem. Just as an experiment, I tried to upload the original MARC export from Koha it responds: Software error: /usr/share/koha/intranet/cgi-bin/tools/stage-marc So I'm thinking it's something wrong with our Koha. Does this mean there's something missing? Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999 On Mon, Aug 19, 2019 at 10:18 PM Carlos Lopez <clopez@dml.vic.edu.au> wrote:
Hi Elaine
You could run unix2dos on the file (http://dos2unix.sourceforge.net/), or if you open the file in Notepad++ (https://notepad-plus-plus.org/) you can then go to 'Edit > EOL Conversion > Windows' and save the document.
With kind regards from the Dalton McCaughey Library Team
Carlos Lopez
Dalton McCaughey Library | 29 College Crescent, Parkville, VICTORIA 3052 Ph: 03 9340 8888 ext.1 | library@dml.vic.edu.au | library.dmlibrary.org.au
-----Original Message----- From: Koha <koha-bounces@lists.katipo.co.nz> On Behalf Of Elaine Bradtke Sent: Tuesday, 20 August 2019 9:35 AM Cc: koha <koha@lists.katipo.co.nz> Subject: Re: [Koha] Multiple problems in MARC download and import
It was all done on Windows 10, using Libre Office and MarcEdit. I've had trouble before with uploads failing because I usually use a Mac, and it has a similar problem with line terminators. I thought I'd be safer on Windows. Hah. I'll look again at the file, but all I did was change http: to https: Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999
On Mon, Aug 19, 2019 at 1:28 PM Alvaro Cornejo <cornejo.alvaro@gmail.com> wrote:
Hi Elaine
This might happen when you have a linux txt file and open it with a windows editor or vice-versa. This is usually due to windows and linux using different line terminators characters [CR] vs [CR]+[LF].
You might also need to verify that, when doing your replacement, you didn't added/removed special characters or spaces.
You can do some test with just a few items to see if you find the issue.
Regards
Alvaro
|-----------------------------------------------------------------------------------------------------------------|
Envíe y Reciba Datos y mensajes de Texto (SMS) hacia y desde cualquier celular y Nextel en el Perú, México y en mas de 180 paises. Use aplicaciones 2 vias via SMS y GPRS online Visitenos en www.perusms.com
Le lun. 19 août 2019 à 12:38, Elaine Bradtke <eb@efdss.org> a écrit :
We're using Koha 19.05.02 I have a txt file of biblio numbers that need to all have the contents of one field changed. 477 numbers (no dupes, I checked). I then went through the process of exporting these records. When I open the file in MarcEdit, it says I have 225 records. I don't know what happened to the other 252.
I decided to go through the rest of the process, just to see if it worked as expected (this is why we have a test site). I edited the file, saved it as mrc and tried to import it to Koha. Where I got a very large error message saying nothing imported SOFTWARE ERROR, contact your webmaster.
Unfortunately, the tech half of the team is away at the moment. . . so any help would be appreciated. I'm especially curious as to why I uploaded a list of 477 biblio numbers but I got only 225 records in return.
I'm now going to experiment with batch editing within Koha. . . wish me luck! Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site)
Registered Company No. 297142 Charity Registered in England and Wales No. 305999 _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz https://lists.katipo.co.nz/mailman/listinfo/koha --
Click here to report this message as spam: https://console.mailguard.com.au/ras/1WlL9msK4Y/3ENKksR4Kfg8qPdlVfxsJm/2.3
Hi Elaine
Line endings are not the problem.
Just as an experiment, I tried to upload the original MARC export from Koha it responds: Software error: /usr/share/koha/intranet/cgi-bin/tools/stage-marc
So I'm thinking it's something wrong with our Koha. Does this mean there's something missing?
You might try to import your original MARC export into one of the publicly available Koha demo installations at https://koha-community.org/demo/ Like this you might see if this is a general problem or if it's one that only occurs with your own installation. Best wishes: Michael -- Geschäftsführer · Diplombibliothekar BBS, Informatiker eidg. Fachausweis Admin Kuhn GmbH · Pappelstrasse 20 · 4123 Allschwil · Schweiz T 0041 (0)61 261 55 61 · E mik@adminkuhn.ch · W www.adminkuhn.ch
We fixed the import problem. It was a configuration problem. Now the question is, why are the exports not exporting all the biblios I request. a list of 400+ biblio numbers gives me a download of 200+, 300+ biblio numbers results in 100+ records. They are mostly, *but not all*, missing from the end. It's the ones that are missing seemingly randomly in the middle of the of the run that puzzles me. I exported a .csv file from a Koha report, and did not open it on my local machine. I turned it right around and used it as a list to export bib records. I have also tried opening it and checking it in Libre Office. The result is the same. Elaine Bradtke VWML English Folk Dance and Song Society | http://www.efdss.org Cecil Sharp House, 2 Regent's Park Road, London NW1 7AY Tel +44 (0) 20 7485 2206 (This number is for the English Folk Dance and Song Society in London, England. If you wish to phone me personally, send an e-mail first. I work off site) -------------------------------------------------------------------------- Registered Company No. 297142 Charity Registered in England and Wales No. 305999 On Fri, Aug 23, 2019 at 3:58 AM Michael Kuhn <mik@adminkuhn.ch> wrote:
Hi Elaine
Line endings are not the problem.
Just as an experiment, I tried to upload the original MARC export from Koha it responds: Software error: /usr/share/koha/intranet/cgi-bin/tools/stage-marc
So I'm thinking it's something wrong with our Koha. Does this mean there's something missing?
You might try to import your original MARC export into one of the publicly available Koha demo installations at https://koha-community.org/demo/
Like this you might see if this is a general problem or if it's one that only occurs with your own installation.
Best wishes: Michael -- Geschäftsführer · Diplombibliothekar BBS, Informatiker eidg. Fachausweis Admin Kuhn GmbH · Pappelstrasse 20 · 4123 Allschwil · Schweiz T 0041 (0)61 261 55 61 · E mik@adminkuhn.ch · W www.adminkuhn.ch
participants (4)
-
Alvaro Cornejo -
Carlos Lopez -
Elaine Bradtke -
Michael Kuhn