[Koha] Background job / Staging MARC import stuck at 0%

George Veranis gveranis at dataly.gr
Tue May 16 05:59:14 NZST 2023


The “worker-output.log” is normally under /var/log/koha/instance_name_of_koha/

 

From: Michael Brown <michael8rown at gmail.com> 
Sent: Saturday, May 13, 2023 4:21 PM
To: George Veranis <gveranis at dataly.gr>
Cc: Cindy Murdock Ames <cmurdock at ccfls.org>; koha at lists.katipo.co.nz
Subject: Re: [Koha] Background job / Staging MARC import stuck at 0%

 

I tried Mengü's suggestion of editing manage-marc-import.pl <http://manage-marc-import.pl> , to no avail. Still hangs at enqueuement; no messages, no errors, simply hangs and spins my CPU into a frenzy. Responses to George's message below:

 

On Wed, May 10, 2023 at 8:34 AM George Veranis <gveranis at dataly.gr <mailto:gveranis at dataly.gr> > wrote:

Hi Michael, 

Could you check the following things: 

1) if your rabbitmq-server is running ? 

 

Yes.

 

1.1) if yes, then check if the cluster of nodes in rabbit exist , the command is: 
rabbitmqctl cluster_status

 

Cluster status of node rabbit at kiko ...
Basics

Cluster name: rabbit at kiko

Disk Nodes
rabbit at kiko

Running Nodes
rabbit at kiko

Versions
rabbit at kiko: RabbitMQ 3.9.21 on Erlang 24.3.4.2

Maintenance status

Node: rabbit at kiko, status: not under maintenance

Alarms

(none)

Network Partitions

(none)

Listeners

Node: rabbit at kiko, interface: [::], port: 61613, protocol: stomp, purpose: STOMP
Node: rabbit at kiko, interface: [::], port: 25672, protocol: clustering, purpose: inter-node and CLI tool communication
Node: rabbit at kiko, interface: [::], port: 5672, protocol: amqp, purpose: AMQP 0-9-1 and AMQP 1.0

Feature flags

Flag: implicit_default_bindings, state: enabled
Flag: maintenance_mode_status, state: enabled
Flag: quorum_queue, state: enabled
Flag: stream_queue, state: enabled
Flag: user_limits, state: enabled
Flag: virtual_host_metadata, state: enabled

 

 

1.1.1) if not shows you something in json format you can try to start it with command 
rabbitmqctl start_app 

N/A, rabbitmq is already running.

 

1.2 ) If no , start it and check the logs from your rabbitmq-server at /var/log/rabbitmq/ and check if the service is start it normally . 

N/A, rabbitmq is already running.

 

2) check the worker-output.log on your koha if koha write there something and have koha connection with rabbit server . 

I cannot find a file called "worker-output.log" anywhere in my Alma installation.

 

Best Regards, 
George 


-----Original Message-----
From: Koha <koha-bounces at lists.katipo.co.nz <mailto:koha-bounces at lists.katipo.co.nz> > On Behalf Of Michael Brown
Sent: Wednesday, May 10, 2023 3:50 PM
To: Cindy Murdock Ames <cmurdock at ccfls.org <mailto:cmurdock at ccfls.org> >
Cc: koha at lists.katipo.co.nz <mailto:koha at lists.katipo.co.nz> 
Subject: Re: [Koha] Background job / Staging MARC import stuck at 0%

 Hi Cindy, apologies for the double-response: I had unsubscribed from the koha list so my response didn't get added to the thread. I wanted to remedy that, and also say that, because I had unsubscribed, I missed Mengü's suggestion of updating manage-marc-import.pl <http://manage-marc-import.pl> . I am eager to try that fix and will report back later today or tomorrow with my findings. Here's my original response to Cindy before re-subscribing to to the koha list:

Yes, my koha-conf.xml does contain that block because I added it during the initial setup. (You're right, it wasn't there originally.)

However, it doesn't help; I still cannot import batch bib records in Koha
22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my cpu through the roof until I navigate away from that page. I've gotten one off-list suggestion that upgrading to the newest Koha fixes the problem.
Unfortunately for me, I don't really have time for that right now, so I have (at least temporarily) abandoned my efforts with Koha.

Thanks for your suggestion!

Bests,
Michael

On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown at gmail.com <mailto:michael8rown at gmail.com> > wrote:

> Hi Cindy,
>
> Yes, my koha-conf.xml does contain that block because I added it 
> during the initial setup. (You're right, it wasn't there originally.)
>
> However, it doesn't help; I still cannot import batch bib records in 
> Koha
> 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends 
> my cpu through the roof until I navigate away from that page. I've 
> gotten one off-list suggestion that upgrading to the newest Koha fixes the problem.
> Unfortunately for me, I don't really have time for that right now, so 
> I have (at least temporarily) abandoned my efforts with Koha.
>
> Thanks for your suggestion!
>
> Bests,
> Michael
>
> On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <cmurdock at ccfls.org <mailto:cmurdock at ccfls.org> >
> wrote:
>
>> Hi Michael,
>>
>> Can you confirm your koha-conf.xml contains a message broker section, 
>> like this?
>>
>> <message_broker>
>>    <hostname>localhost</hostname>
>>    <port>61613</port>
>>    <username>guest</username>
>>    <password>guest</password>
>>    <vhost></vhost>
>>  </message_broker>
>>
>> I had the same problem after upgrading from 22.05 to 22.11; I'm 
>> guessing
>> 22.05 must have had some defaults set in the code that allowed the 
>> message broker to start automatically without this section whereas 22.11 must not.
>> I was missing this section and added it and was able to complete 
>> imports after restarting Koha.
>>
>> I'm still having the issue with zombie background jobs (I'm afraid to 
>> test rolling back the code on a production server) but I made a hacky 
>> bash script to run as a cron job to check for them and kill them off 
>> so at least our staff can do batch imports without issue, but that's another matter.
>>
>> HTH!
>> Cindy
>>
>>
>> -----------------------------------------------------------
>> Cindy Murdock Ames
>> IT Services Director
>> Meadville Public Library | CCFLS
>> https://meadvillelibrary.org | https://ccfls.org
>>
>> Please report tech support issues in Mantis:  
>> https://mantis.ccfls.org
>>
>>
>> On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <michael8rown at gmail.com <mailto:michael8rown at gmail.com> >
>> wrote:
>>
>>> Greetings:
>>>
>>> My name is Michael Brown and I am a professional cataloger and 
>>> SirsiDynix System Admin at the Texas State Library & Archives in 
>>> Austin (20+ years now). I have been using Koha on Arch Linux in my 
>>> home library for about a year now. I am migrating my home server to 
>>> AlmaLinux and I'm having a problem.
>>>
>>> I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a 
>>> MARC file for import gets stuck at 0%. On screen, I am able to 
>>> select the file for import (bib.mrc), review the profile options 
>>> (but I don't change any defaults), and then click on "Stage for 
>>> import" at the bottom. Next screen
>>> reads:
>>>
>>>     The job has been enqueued! It will be processed as soon as possible.
>>>     0%
>>>     View detail of the enqueued job
>>>
>>> After a few seconds, it changes to (and then hangs at):
>>>
>>>     The job has been enqueued! It will be processed as soon as possible.
>>>     0% Not started
>>>     View detail of the enqueued job
>>>
>>> Clicking on "View detail of the enqueued job" I see:
>>>
>>> Details of job #22
>>>
>>>     Job ID: 22
>>>     Status:    New
>>>     Progress:    0 / 0
>>>     Type:    Staged MARC records for import
>>>     Queued:    03/02/2023 05:42
>>>     Started:
>>>     Ended:
>>>
>>> Report
>>> Detailed messages
>>> Return to the job list
>>>
>>> The corresponding entry in mariadb is:
>>>
>>> id 22
>>> status new
>>> progress NULL
>>> size 0
>>> borrowernumber 1
>>> type stage_marc_for_import
>>> queue Name of the queue the job is sent to long_tasks data 
>>> {"encoding":"UTF-8","comments":"","basket_id":null...
>>> context JSON-serialized context information for the job 
>>> {"flags":1,"branch":"ALMA","interface":"intranet",...
>>> enqueued_on 2023-03-02 05:42:33
>>> started_on NULL
>>> ended_on NULL
>>>
>>> (If you need to see the full entries for "data" and "context", 
>>> please let me know.)
>>>
>>> tmp, koha_upload, and lock directories have been tweaked and fine tuned.
>>> I
>>> was getting early warnings about them not being set in koha-conf.xml 
>>> so I created them (and set correct permissions) and I can see the 
>>> uploaded file (for job 22 the name is 
>>> 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload 
>>> function appears to be working.
>>>
>>> I am getting no apache errors and no other on-screen diagnostics.
>>>
>>> I have Koha 22.05.02.000 running on Arch Linux that imports this 
>>> file just fine. Similarly, I have Koha latest running on a Debian VM 
>>> that can import this file just fine, too.
>>>
>>> What am I missing?
>>>
>>> Details of my system:
>>>
>>> Koha version: 22.11.03.000 Rosalie
>>> OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1 
>>> SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl 
>>> interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC: 
>>> /usr/share/koha/lib
>>> /usr/local/lib64/perl5/5.32
>>> /usr/local/share/perl5/5.32
>>> /usr/lib64/perl5/vendor_perl
>>> /usr/share/perl5/vendor_perl
>>> /usr/lib64/perl5
>>> /usr/share/perl5
>>> /var/lib/koha/plugins
>>> MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux 
>>> (x86_64) using EditLine wrapper Apache version: Server version: 
>>> Apache/2.4.53 (AlmaLinux) Server built:
>>> Jul
>>> 20 2022 00:00:00
>>> Memcached: Servers: 127.0.0.1:11211 <http://127.0.0.1:11211>  | Namespace: KOHA | Status:
>>> running. |
>>> Config read from: koha-conf.xml
>>> Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free 
>>> software, covered by the GNU General Public License, and you are 
>>> welcome to change it and/or distribute copies of it under certain 
>>> conditions. SHA1
>>> ID:
>>> ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status: 
>>> Running Message broker: Using RabbitMQ Date and time: 03/01/2023 
>>> 16:20 Time zone: Used: America/Chicago | Config: Undefined | 
>>> Environment (TZ):
>>> Undefined
>>>
>>> Thanks,
>>> Michael
>>> _______________________________________________
>>>
>>> Koha mailing list  http://koha-community.org Koha at lists.katipo.co.nz <mailto:Koha at lists.katipo.co.nz> 
>>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>>
>>
_______________________________________________

Koha mailing list  http://koha-community.org Koha at lists.katipo.co.nz <mailto:Koha at lists.katipo.co.nz> 
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha



More information about the Koha mailing list