Re: [Koha] Background job / Staging MARC import stuck at 0%
Hi Cindy, apologies for the double-response: I had unsubscribed from the koha list so my response didn't get added to the thread. I wanted to remedy that, and also say that, because I had unsubscribed, I missed Mengü's suggestion of updating manage-marc-import.pl. I am eager to try that fix and will report back later today or tomorrow with my findings. Here's my original response to Cindy before re-subscribing to to the koha list: Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.) However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha. Thanks for your suggestion! Bests, Michael On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown@gmail.com> wrote:
Hi Cindy,
Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.)
However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha.
Thanks for your suggestion!
Bests, Michael
On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <cmurdock@ccfls.org> wrote:
Hi Michael,
Can you confirm your koha-conf.xml contains a message broker section, like this?
<message_broker> <hostname>localhost</hostname> <port>61613</port> <username>guest</username> <password>guest</password> <vhost></vhost> </message_broker>
I had the same problem after upgrading from 22.05 to 22.11; I'm guessing 22.05 must have had some defaults set in the code that allowed the message broker to start automatically without this section whereas 22.11 must not. I was missing this section and added it and was able to complete imports after restarting Koha.
I'm still having the issue with zombie background jobs (I'm afraid to test rolling back the code on a production server) but I made a hacky bash script to run as a cron job to check for them and kill them off so at least our staff can do batch imports without issue, but that's another matter.
HTH! Cindy
----------------------------------------------------------- Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS https://meadvillelibrary.org | https://ccfls.org
Please report tech support issues in Mantis: https://mantis.ccfls.org
On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <michael8rown@gmail.com> wrote:
Greetings:
My name is Michael Brown and I am a professional cataloger and SirsiDynix System Admin at the Texas State Library & Archives in Austin (20+ years now). I have been using Koha on Arch Linux in my home library for about a year now. I am migrating my home server to AlmaLinux and I'm having a problem.
I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a MARC file for import gets stuck at 0%. On screen, I am able to select the file for import (bib.mrc), review the profile options (but I don't change any defaults), and then click on "Stage for import" at the bottom. Next screen reads:
The job has been enqueued! It will be processed as soon as possible. 0% View detail of the enqueued job
After a few seconds, it changes to (and then hangs at):
The job has been enqueued! It will be processed as soon as possible. 0% Not started View detail of the enqueued job
Clicking on "View detail of the enqueued job" I see:
Details of job #22
Job ID: 22 Status: New Progress: 0 / 0 Type: Staged MARC records for import Queued: 03/02/2023 05:42 Started: Ended:
Report Detailed messages Return to the job list
The corresponding entry in mariadb is:
id 22 status new progress NULL size 0 borrowernumber 1 type stage_marc_for_import queue Name of the queue the job is sent to long_tasks data {"encoding":"UTF-8","comments":"","basket_id":null... context JSON-serialized context information for the job {"flags":1,"branch":"ALMA","interface":"intranet",... enqueued_on 2023-03-02 05:42:33 started_on NULL ended_on NULL
(If you need to see the full entries for "data" and "context", please let me know.)
tmp, koha_upload, and lock directories have been tweaked and fine tuned. I was getting early warnings about them not being set in koha-conf.xml so I created them (and set correct permissions) and I can see the uploaded file (for job 22 the name is 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload function appears to be working.
I am getting no apache errors and no other on-screen diagnostics.
I have Koha 22.05.02.000 running on Arch Linux that imports this file just fine. Similarly, I have Koha latest running on a Debian VM that can import this file just fine, too.
What am I missing?
Details of my system:
Koha version: 22.11.03.000 Rosalie OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC: /usr/share/koha/lib /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 /var/lib/koha/plugins MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux (x86_64) using EditLine wrapper Apache version: Server version: Apache/2.4.53 (AlmaLinux) Server built: Jul 20 2022 00:00:00 Memcached: Servers: 127.0.0.1:11211 | Namespace: KOHA | Status: running. | Config read from: koha-conf.xml Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. SHA1 ID: ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status: Running Message broker: Using RabbitMQ Date and time: 03/01/2023 16:20 Time zone: Used: America/Chicago | Config: Undefined | Environment (TZ): Undefined
Thanks, Michael _______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Hi Michael, Could you check the following things: 1) if your rabbitmq-server is running ? 1.1) if yes, then check if the cluster of nodes in rabbit exist , the command is: rabbitmqctl cluster_status 1.1.1) if not shows you something in json format you can try to start it with command rabbitmqctl start_app 1.2 ) If no , start it and check the logs from your rabbitmq-server at /var/log/rabbitmq/ and check if the service is start it normally . 2) check the worker-output.log on your koha if koha write there something and have koha connection with rabbit server . Best Regards, George -----Original Message----- From: Koha <koha-bounces@lists.katipo.co.nz> On Behalf Of Michael Brown Sent: Wednesday, May 10, 2023 3:50 PM To: Cindy Murdock Ames <cmurdock@ccfls.org> Cc: koha@lists.katipo.co.nz Subject: Re: [Koha] Background job / Staging MARC import stuck at 0% Hi Cindy, apologies for the double-response: I had unsubscribed from the koha list so my response didn't get added to the thread. I wanted to remedy that, and also say that, because I had unsubscribed, I missed Mengü's suggestion of updating manage-marc-import.pl. I am eager to try that fix and will report back later today or tomorrow with my findings. Here's my original response to Cindy before re-subscribing to to the koha list: Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.) However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha. Thanks for your suggestion! Bests, Michael On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown@gmail.com> wrote:
Hi Cindy,
Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.)
However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha.
Thanks for your suggestion!
Bests, Michael
On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <cmurdock@ccfls.org> wrote:
Hi Michael,
Can you confirm your koha-conf.xml contains a message broker section, like this?
<message_broker> <hostname>localhost</hostname> <port>61613</port> <username>guest</username> <password>guest</password> <vhost></vhost> </message_broker>
I had the same problem after upgrading from 22.05 to 22.11; I'm guessing 22.05 must have had some defaults set in the code that allowed the message broker to start automatically without this section whereas 22.11 must not. I was missing this section and added it and was able to complete imports after restarting Koha.
I'm still having the issue with zombie background jobs (I'm afraid to test rolling back the code on a production server) but I made a hacky bash script to run as a cron job to check for them and kill them off so at least our staff can do batch imports without issue, but that's another matter.
HTH! Cindy
----------------------------------------------------------- Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS https://meadvillelibrary.org | https://ccfls.org
Please report tech support issues in Mantis: https://mantis.ccfls.org
On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <michael8rown@gmail.com> wrote:
Greetings:
My name is Michael Brown and I am a professional cataloger and SirsiDynix System Admin at the Texas State Library & Archives in Austin (20+ years now). I have been using Koha on Arch Linux in my home library for about a year now. I am migrating my home server to AlmaLinux and I'm having a problem.
I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a MARC file for import gets stuck at 0%. On screen, I am able to select the file for import (bib.mrc), review the profile options (but I don't change any defaults), and then click on "Stage for import" at the bottom. Next screen reads:
The job has been enqueued! It will be processed as soon as possible. 0% View detail of the enqueued job
After a few seconds, it changes to (and then hangs at):
The job has been enqueued! It will be processed as soon as possible. 0% Not started View detail of the enqueued job
Clicking on "View detail of the enqueued job" I see:
Details of job #22
Job ID: 22 Status: New Progress: 0 / 0 Type: Staged MARC records for import Queued: 03/02/2023 05:42 Started: Ended:
Report Detailed messages Return to the job list
The corresponding entry in mariadb is:
id 22 status new progress NULL size 0 borrowernumber 1 type stage_marc_for_import queue Name of the queue the job is sent to long_tasks data {"encoding":"UTF-8","comments":"","basket_id":null... context JSON-serialized context information for the job {"flags":1,"branch":"ALMA","interface":"intranet",... enqueued_on 2023-03-02 05:42:33 started_on NULL ended_on NULL
(If you need to see the full entries for "data" and "context", please let me know.)
tmp, koha_upload, and lock directories have been tweaked and fine tuned. I was getting early warnings about them not being set in koha-conf.xml so I created them (and set correct permissions) and I can see the uploaded file (for job 22 the name is 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload function appears to be working.
I am getting no apache errors and no other on-screen diagnostics.
I have Koha 22.05.02.000 running on Arch Linux that imports this file just fine. Similarly, I have Koha latest running on a Debian VM that can import this file just fine, too.
What am I missing?
Details of my system:
Koha version: 22.11.03.000 Rosalie OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC: /usr/share/koha/lib /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 /var/lib/koha/plugins MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux (x86_64) using EditLine wrapper Apache version: Server version: Apache/2.4.53 (AlmaLinux) Server built: Jul 20 2022 00:00:00 Memcached: Servers: 127.0.0.1:11211 | Namespace: KOHA | Status: running. | Config read from: koha-conf.xml Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. SHA1 ID: ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status: Running Message broker: Using RabbitMQ Date and time: 03/01/2023 16:20 Time zone: Used: America/Chicago | Config: Undefined | Environment (TZ): Undefined
Thanks, Michael _______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
I tried Mengü's suggestion of editing manage-marc-import.pl, to no avail. Still hangs at enqueuement; no messages, no errors, simply hangs and spins my CPU into a frenzy. Responses to George's message below: On Wed, May 10, 2023 at 8:34 AM George Veranis <gveranis@dataly.gr> wrote:
Hi Michael,
Could you check the following things:
1) if your rabbitmq-server is running ?
Yes.
1.1) if yes, then check if the cluster of nodes in rabbit exist , the command is: rabbitmqctl cluster_status
Cluster status of node rabbit@kiko ... Basics Cluster name: rabbit@kiko Disk Nodes rabbit@kiko Running Nodes rabbit@kiko Versions rabbit@kiko: RabbitMQ 3.9.21 on Erlang 24.3.4.2 Maintenance status Node: rabbit@kiko, status: not under maintenance Alarms (none) Network Partitions (none) Listeners Node: rabbit@kiko, interface: [::], port: 61613, protocol: stomp, purpose: STOMP Node: rabbit@kiko, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@kiko, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0 Feature flags Flag: implicit_default_bindings, state: enabled Flag: maintenance_mode_status, state: enabled Flag: quorum_queue, state: enabled Flag: stream_queue, state: enabled Flag: user_limits, state: enabled Flag: virtual_host_metadata, state: enabled
1.1.1) if not shows you something in json format you can try to start it with command rabbitmqctl start_app
N/A, rabbitmq is already running.
1.2 ) If no , start it and check the logs from your rabbitmq-server at /var/log/rabbitmq/ and check if the service is start it normally .
N/A, rabbitmq is already running.
2) check the worker-output.log on your koha if koha write there something and have koha connection with rabbit server .
I cannot find a file called "worker-output.log" anywhere in my Alma installation.
Best Regards, George
-----Original Message----- From: Koha <koha-bounces@lists.katipo.co.nz> On Behalf Of Michael Brown Sent: Wednesday, May 10, 2023 3:50 PM To: Cindy Murdock Ames <cmurdock@ccfls.org> Cc: koha@lists.katipo.co.nz Subject: Re: [Koha] Background job / Staging MARC import stuck at 0%
Hi Cindy, apologies for the double-response: I had unsubscribed from the koha list so my response didn't get added to the thread. I wanted to remedy that, and also say that, because I had unsubscribed, I missed Mengü's suggestion of updating manage-marc-import.pl. I am eager to try that fix and will report back later today or tomorrow with my findings. Here's my original response to Cindy before re-subscribing to to the koha list:
Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.)
However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha.
Thanks for your suggestion!
Bests, Michael
On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown@gmail.com> wrote:
Hi Cindy,
Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.)
However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha.
Thanks for your suggestion!
Bests, Michael
On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <cmurdock@ccfls.org> wrote:
Hi Michael,
Can you confirm your koha-conf.xml contains a message broker section, like this?
<message_broker> <hostname>localhost</hostname> <port>61613</port> <username>guest</username> <password>guest</password> <vhost></vhost> </message_broker>
I had the same problem after upgrading from 22.05 to 22.11; I'm guessing 22.05 must have had some defaults set in the code that allowed the message broker to start automatically without this section whereas 22.11 must not. I was missing this section and added it and was able to complete imports after restarting Koha.
I'm still having the issue with zombie background jobs (I'm afraid to test rolling back the code on a production server) but I made a hacky bash script to run as a cron job to check for them and kill them off so at least our staff can do batch imports without issue, but that's another matter.
HTH! Cindy
----------------------------------------------------------- Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS https://meadvillelibrary.org | https://ccfls.org
Please report tech support issues in Mantis: https://mantis.ccfls.org
On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <michael8rown@gmail.com> wrote:
Greetings:
My name is Michael Brown and I am a professional cataloger and SirsiDynix System Admin at the Texas State Library & Archives in Austin (20+ years now). I have been using Koha on Arch Linux in my home library for about a year now. I am migrating my home server to AlmaLinux and I'm having a problem.
I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a MARC file for import gets stuck at 0%. On screen, I am able to select the file for import (bib.mrc), review the profile options (but I don't change any defaults), and then click on "Stage for import" at the bottom. Next screen reads:
The job has been enqueued! It will be processed as soon as possible. 0% View detail of the enqueued job
After a few seconds, it changes to (and then hangs at):
The job has been enqueued! It will be processed as soon as possible. 0% Not started View detail of the enqueued job
Clicking on "View detail of the enqueued job" I see:
Details of job #22
Job ID: 22 Status: New Progress: 0 / 0 Type: Staged MARC records for import Queued: 03/02/2023 05:42 Started: Ended:
Report Detailed messages Return to the job list
The corresponding entry in mariadb is:
id 22 status new progress NULL size 0 borrowernumber 1 type stage_marc_for_import queue Name of the queue the job is sent to long_tasks data {"encoding":"UTF-8","comments":"","basket_id":null... context JSON-serialized context information for the job {"flags":1,"branch":"ALMA","interface":"intranet",... enqueued_on 2023-03-02 05:42:33 started_on NULL ended_on NULL
(If you need to see the full entries for "data" and "context", please let me know.)
tmp, koha_upload, and lock directories have been tweaked and fine tuned. I was getting early warnings about them not being set in koha-conf.xml so I created them (and set correct permissions) and I can see the uploaded file (for job 22 the name is 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload function appears to be working.
I am getting no apache errors and no other on-screen diagnostics.
I have Koha 22.05.02.000 running on Arch Linux that imports this file just fine. Similarly, I have Koha latest running on a Debian VM that can import this file just fine, too.
What am I missing?
Details of my system:
Koha version: 22.11.03.000 Rosalie OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC: /usr/share/koha/lib /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 /var/lib/koha/plugins MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux (x86_64) using EditLine wrapper Apache version: Server version: Apache/2.4.53 (AlmaLinux) Server built: Jul 20 2022 00:00:00 Memcached: Servers: 127.0.0.1:11211 | Namespace: KOHA | Status: running. | Config read from: koha-conf.xml Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. SHA1 ID: ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status: Running Message broker: Using RabbitMQ Date and time: 03/01/2023 16:20 Time zone: Used: America/Chicago | Config: Undefined | Environment (TZ): Undefined
Thanks, Michael _______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
The “worker-output.log” is normally under /var/log/koha/instance_name_of_koha/ From: Michael Brown <michael8rown@gmail.com> Sent: Saturday, May 13, 2023 4:21 PM To: George Veranis <gveranis@dataly.gr> Cc: Cindy Murdock Ames <cmurdock@ccfls.org>; koha@lists.katipo.co.nz Subject: Re: [Koha] Background job / Staging MARC import stuck at 0% I tried Mengü's suggestion of editing manage-marc-import.pl <http://manage-marc-import.pl> , to no avail. Still hangs at enqueuement; no messages, no errors, simply hangs and spins my CPU into a frenzy. Responses to George's message below: On Wed, May 10, 2023 at 8:34 AM George Veranis <gveranis@dataly.gr <mailto:gveranis@dataly.gr> > wrote: Hi Michael, Could you check the following things: 1) if your rabbitmq-server is running ? Yes. 1.1) if yes, then check if the cluster of nodes in rabbit exist , the command is: rabbitmqctl cluster_status Cluster status of node rabbit@kiko ... Basics Cluster name: rabbit@kiko Disk Nodes rabbit@kiko Running Nodes rabbit@kiko Versions rabbit@kiko: RabbitMQ 3.9.21 on Erlang 24.3.4.2 Maintenance status Node: rabbit@kiko, status: not under maintenance Alarms (none) Network Partitions (none) Listeners Node: rabbit@kiko, interface: [::], port: 61613, protocol: stomp, purpose: STOMP Node: rabbit@kiko, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@kiko, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0 Feature flags Flag: implicit_default_bindings, state: enabled Flag: maintenance_mode_status, state: enabled Flag: quorum_queue, state: enabled Flag: stream_queue, state: enabled Flag: user_limits, state: enabled Flag: virtual_host_metadata, state: enabled 1.1.1) if not shows you something in json format you can try to start it with command rabbitmqctl start_app N/A, rabbitmq is already running. 1.2 ) If no , start it and check the logs from your rabbitmq-server at /var/log/rabbitmq/ and check if the service is start it normally . N/A, rabbitmq is already running. 2) check the worker-output.log on your koha if koha write there something and have koha connection with rabbit server . I cannot find a file called "worker-output.log" anywhere in my Alma installation. Best Regards, George -----Original Message----- From: Koha <koha-bounces@lists.katipo.co.nz <mailto:koha-bounces@lists.katipo.co.nz> > On Behalf Of Michael Brown Sent: Wednesday, May 10, 2023 3:50 PM To: Cindy Murdock Ames <cmurdock@ccfls.org <mailto:cmurdock@ccfls.org> > Cc: koha@lists.katipo.co.nz <mailto:koha@lists.katipo.co.nz> Subject: Re: [Koha] Background job / Staging MARC import stuck at 0% Hi Cindy, apologies for the double-response: I had unsubscribed from the koha list so my response didn't get added to the thread. I wanted to remedy that, and also say that, because I had unsubscribed, I missed Mengü's suggestion of updating manage-marc-import.pl <http://manage-marc-import.pl> . I am eager to try that fix and will report back later today or tomorrow with my findings. Here's my original response to Cindy before re-subscribing to to the koha list: Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.) However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha. Thanks for your suggestion! Bests, Michael On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown@gmail.com <mailto:michael8rown@gmail.com> > wrote:
Hi Cindy,
Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.)
However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha.
Thanks for your suggestion!
Bests, Michael
On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <cmurdock@ccfls.org <mailto:cmurdock@ccfls.org> > wrote:
Hi Michael,
Can you confirm your koha-conf.xml contains a message broker section, like this?
<message_broker> <hostname>localhost</hostname> <port>61613</port> <username>guest</username> <password>guest</password> <vhost></vhost> </message_broker>
I had the same problem after upgrading from 22.05 to 22.11; I'm guessing 22.05 must have had some defaults set in the code that allowed the message broker to start automatically without this section whereas 22.11 must not. I was missing this section and added it and was able to complete imports after restarting Koha.
I'm still having the issue with zombie background jobs (I'm afraid to test rolling back the code on a production server) but I made a hacky bash script to run as a cron job to check for them and kill them off so at least our staff can do batch imports without issue, but that's another matter.
HTH! Cindy
----------------------------------------------------------- Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS https://meadvillelibrary.org | https://ccfls.org
Please report tech support issues in Mantis: https://mantis.ccfls.org
On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <michael8rown@gmail.com <mailto:michael8rown@gmail.com> > wrote:
Greetings:
My name is Michael Brown and I am a professional cataloger and SirsiDynix System Admin at the Texas State Library & Archives in Austin (20+ years now). I have been using Koha on Arch Linux in my home library for about a year now. I am migrating my home server to AlmaLinux and I'm having a problem.
I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a MARC file for import gets stuck at 0%. On screen, I am able to select the file for import (bib.mrc), review the profile options (but I don't change any defaults), and then click on "Stage for import" at the bottom. Next screen reads:
The job has been enqueued! It will be processed as soon as possible. 0% View detail of the enqueued job
After a few seconds, it changes to (and then hangs at):
The job has been enqueued! It will be processed as soon as possible. 0% Not started View detail of the enqueued job
Clicking on "View detail of the enqueued job" I see:
Details of job #22
Job ID: 22 Status: New Progress: 0 / 0 Type: Staged MARC records for import Queued: 03/02/2023 05:42 Started: Ended:
Report Detailed messages Return to the job list
The corresponding entry in mariadb is:
id 22 status new progress NULL size 0 borrowernumber 1 type stage_marc_for_import queue Name of the queue the job is sent to long_tasks data {"encoding":"UTF-8","comments":"","basket_id":null... context JSON-serialized context information for the job {"flags":1,"branch":"ALMA","interface":"intranet",... enqueued_on 2023-03-02 05:42:33 started_on NULL ended_on NULL
(If you need to see the full entries for "data" and "context", please let me know.)
tmp, koha_upload, and lock directories have been tweaked and fine tuned. I was getting early warnings about them not being set in koha-conf.xml so I created them (and set correct permissions) and I can see the uploaded file (for job 22 the name is 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload function appears to be working.
I am getting no apache errors and no other on-screen diagnostics.
I have Koha 22.05.02.000 running on Arch Linux that imports this file just fine. Similarly, I have Koha latest running on a Debian VM that can import this file just fine, too.
What am I missing?
Details of my system:
Koha version: 22.11.03.000 Rosalie OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC: /usr/share/koha/lib /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 /var/lib/koha/plugins MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux (x86_64) using EditLine wrapper Apache version: Server version: Apache/2.4.53 (AlmaLinux) Server built: Jul 20 2022 00:00:00 Memcached: Servers: 127.0.0.1:11211 <http://127.0.0.1:11211> | Namespace: KOHA | Status: running. | Config read from: koha-conf.xml Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. SHA1 ID: ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status: Running Message broker: Using RabbitMQ Date and time: 03/01/2023 16:20 Time zone: Used: America/Chicago | Config: Undefined | Environment (TZ): Undefined
Thanks, Michael _______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz <mailto:Koha@lists.katipo.co.nz> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz <mailto:Koha@lists.katipo.co.nz> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Hi Michael, Did you also restart some services after changing the script including memcache? Sometimes it is necessary but not remember for this case. You should also look https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33412 Mengü Yazıcıoğlu Devinim Yazılım Eğitim Danışmanlık Reşit Galip Cad. No: 29/6 Çankaya/ANKARA Tel: (312) 446 86 68 Cep: (532) 701 90 27 On 13.05.2023 16:21, Michael Brown wrote:
I tried Mengü's suggestion of editing manage-marc-import.pl, to no avail. Still hangs at enqueuement; no messages, no errors, simply hangs and spins my CPU into a frenzy. Responses to George's message below:
On Wed, May 10, 2023 at 8:34 AM George Veranis <gveranis@dataly.gr> wrote:
Hi Michael,
Could you check the following things:
1) if your rabbitmq-server is running ?
Yes.
1.1) if yes, then check if the cluster of nodes in rabbit exist , the command is: rabbitmqctl cluster_status
Cluster status of node rabbit@kiko ... Basics
Cluster name: rabbit@kiko
Disk Nodes rabbit@kiko
Running Nodes rabbit@kiko
Versions rabbit@kiko: RabbitMQ 3.9.21 on Erlang 24.3.4.2
Maintenance status
Node: rabbit@kiko, status: not under maintenance
Alarms
(none)
Network Partitions
(none)
Listeners
Node: rabbit@kiko, interface: [::], port: 61613, protocol: stomp, purpose: STOMP Node: rabbit@kiko, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication Node: rabbit@kiko, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0
Feature flags
Flag: implicit_default_bindings, state: enabled Flag: maintenance_mode_status, state: enabled Flag: quorum_queue, state: enabled Flag: stream_queue, state: enabled Flag: user_limits, state: enabled Flag: virtual_host_metadata, state: enabled
1.1.1) if not shows you something in json format you can try to start it with command rabbitmqctl start_app
N/A, rabbitmq is already running.
1.2 ) If no , start it and check the logs from your rabbitmq-server at /var/log/rabbitmq/ and check if the service is start it normally .
N/A, rabbitmq is already running.
2) check the worker-output.log on your koha if koha write there something and have koha connection with rabbit server .
I cannot find a file called "worker-output.log" anywhere in my Alma installation.
Best Regards, George
-----Original Message----- From: Koha <koha-bounces@lists.katipo.co.nz> On Behalf Of Michael Brown Sent: Wednesday, May 10, 2023 3:50 PM To: Cindy Murdock Ames <cmurdock@ccfls.org> Cc: koha@lists.katipo.co.nz Subject: Re: [Koha] Background job / Staging MARC import stuck at 0%
Hi Cindy, apologies for the double-response: I had unsubscribed from the koha list so my response didn't get added to the thread. I wanted to remedy that, and also say that, because I had unsubscribed, I missed Mengü's suggestion of updating manage-marc-import.pl. I am eager to try that fix and will report back later today or tomorrow with my findings. Here's my original response to Cindy before re-subscribing to to the koha list:
Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.)
However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha.
Thanks for your suggestion!
Bests, Michael
On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown@gmail.com> wrote:
Hi Cindy,
Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.)
However, it doesn't help; I still cannot import batch bib records in Koha 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem. Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha.
Thanks for your suggestion!
Bests, Michael
On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <cmurdock@ccfls.org> wrote:
Hi Michael,
Can you confirm your koha-conf.xml contains a message broker section, like this?
<message_broker> <hostname>localhost</hostname> <port>61613</port> <username>guest</username> <password>guest</password> <vhost></vhost> </message_broker>
I had the same problem after upgrading from 22.05 to 22.11; I'm guessing 22.05 must have had some defaults set in the code that allowed the message broker to start automatically without this section whereas 22.11 must not. I was missing this section and added it and was able to complete imports after restarting Koha.
I'm still having the issue with zombie background jobs (I'm afraid to test rolling back the code on a production server) but I made a hacky bash script to run as a cron job to check for them and kill them off so at least our staff can do batch imports without issue, but that's another matter. HTH! Cindy
----------------------------------------------------------- Cindy Murdock Ames IT Services Director Meadville Public Library | CCFLS https://meadvillelibrary.org | https://ccfls.org
Please report tech support issues in Mantis: https://mantis.ccfls.org
On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <michael8rown@gmail.com> wrote:
Greetings:
My name is Michael Brown and I am a professional cataloger and SirsiDynix System Admin at the Texas State Library & Archives in Austin (20+ years now). I have been using Koha on Arch Linux in my home library for about a year now. I am migrating my home server to AlmaLinux and I'm having a problem.
I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a MARC file for import gets stuck at 0%. On screen, I am able to select the file for import (bib.mrc), review the profile options (but I don't change any defaults), and then click on "Stage for import" at the bottom. Next screen reads:
The job has been enqueued! It will be processed as soon as possible. 0% View detail of the enqueued job
After a few seconds, it changes to (and then hangs at):
The job has been enqueued! It will be processed as soon as possible. 0% Not started View detail of the enqueued job
Clicking on "View detail of the enqueued job" I see:
Details of job #22
Job ID: 22 Status: New Progress: 0 / 0 Type: Staged MARC records for import Queued: 03/02/2023 05:42 Started: Ended:
Report Detailed messages Return to the job list
The corresponding entry in mariadb is:
id 22 status new progress NULL size 0 borrowernumber 1 type stage_marc_for_import queue Name of the queue the job is sent to long_tasks data {"encoding":"UTF-8","comments":"","basket_id":null... context JSON-serialized context information for the job {"flags":1,"branch":"ALMA","interface":"intranet",... enqueued_on 2023-03-02 05:42:33 started_on NULL ended_on NULL
(If you need to see the full entries for "data" and "context", please let me know.)
tmp, koha_upload, and lock directories have been tweaked and fine tuned. I was getting early warnings about them not being set in koha-conf.xml so I created them (and set correct permissions) and I can see the uploaded file (for job 22 the name is 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload function appears to be working.
I am getting no apache errors and no other on-screen diagnostics.
I have Koha 22.05.02.000 running on Arch Linux that imports this file just fine. Similarly, I have Koha latest running on a Debian VM that can import this file just fine, too.
What am I missing?
Details of my system:
Koha version: 22.11.03.000 Rosalie OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1 SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC: /usr/share/koha/lib /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 /var/lib/koha/plugins MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux (x86_64) using EditLine wrapper Apache version: Server version: Apache/2.4.53 (AlmaLinux) Server built: Jul 20 2022 00:00:00 Memcached: Servers: 127.0.0.1:11211 | Namespace: KOHA | Status: running. | Config read from: koha-conf.xml Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. SHA1 ID: ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status: Running Message broker: Using RabbitMQ Date and time: 03/01/2023 16:20 Time zone: Used: America/Chicago | Config: Undefined | Environment (TZ): Undefined
Thanks, Michael _______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
_______________________________________________
Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
participants (3)
-
George Veranis -
Mengu Yazicioglu -
Michael Brown