[Koha] Background job / Staging MARC import stuck at 0%
Katrin Fischer
katrin.fischer.83 at web.de
Tue May 30 22:51:24 NZST 2023
Hi Anna,
we recently saw the import stuck at 'orange' when the background job for
processing the jobs was missing. In adminstration > jobs the import
remained with status 'new' indefinitely. In our case a restart of
koha-common fixed it.
Hope this helps,
Katrin
On 30.05.23 05:05, Anna Shields wrote:
> Michael,
>
> Did you ever have any luck resolving this?
>
> I'm having a similar issue in which my imports get stuck at 0% on the
> staging step. I, too, seem to have a functioning RabbitMQ, and none of the
> suggestions I saw in this thread seemed to make a difference.
>
> I'm now on v22.11.06-3, and updating hasn't seemed to make any difference,
> either.
>
> Anna
>
> On Tue, May 16, 2023 at 7:01 PM <koha-request at lists.katipo.co.nz> wrote:
>
>> Hi Michael,
>>
>> Did you also restart some services after changing the script including
>> memcache?
>>
>> Sometimes it is necessary but not remember for this case.
>>
>> You should also look
>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33412
>>
>> Mengü Yazıcıoğlu
>> Devinim Yazılım Eğitim Danışmanlık
>> Reşit Galip Cad. No: 29/6
>> Çankaya/ANKARA
>>
>> Tel: (312) 446 86 68
>> Cep: (532) 701 90 27
>>
>> On 13.05.2023 16:21, Michael Brown wrote:
>>> I tried Mengü's suggestion of editing manage-marc-import.pl, to no
>> avail.
>>> Still hangs at enqueuement; no messages, no errors, simply hangs and
>> spins
>>> my CPU into a frenzy. Responses to George's message below:
>>>
>>> On Wed, May 10, 2023 at 8:34 AM George Veranis <gveranis at dataly.gr>
>> wrote:
>>>> Hi Michael,
>>>>
>>>> Could you check the following things:
>>>>
>>>> 1) if your rabbitmq-server is running ?
>>>>
>>> Yes.
>>>
>>>
>>>> 1.1) if yes, then check if the cluster of nodes in rabbit exist , the
>>>> command is:
>>>> rabbitmqctl cluster_status
>>>>
>>> Cluster status of node rabbit at kiko ...
>>> Basics
>>>
>>> Cluster name: rabbit at kiko
>>>
>>> Disk Nodes
>>> rabbit at kiko
>>>
>>> Running Nodes
>>> rabbit at kiko
>>>
>>> Versions
>>> rabbit at kiko: RabbitMQ 3.9.21 on Erlang 24.3.4.2
>>>
>>> Maintenance status
>>>
>>> Node: rabbit at kiko, status: not under maintenance
>>>
>>> Alarms
>>>
>>> (none)
>>>
>>> Network Partitions
>>>
>>> (none)
>>>
>>> Listeners
>>>
>>> Node: rabbit at kiko, interface: [::], port: 61613, protocol: stomp,
>> purpose:
>>> STOMP
>>> Node: rabbit at kiko, interface: [::], port: 25672, protocol: clustering,
>>> purpose: inter-node and CLI tool communication
>>> Node: rabbit at kiko, interface: [::], port: 5672, protocol: amqp, purpose:
>>> AMQP 0-9-1 and AMQP 1.0
>>>
>>> Feature flags
>>>
>>> Flag: implicit_default_bindings, state: enabled
>>> Flag: maintenance_mode_status, state: enabled
>>> Flag: quorum_queue, state: enabled
>>> Flag: stream_queue, state: enabled
>>> Flag: user_limits, state: enabled
>>> Flag: virtual_host_metadata, state: enabled
>>>
>>>
>>>
>>>> 1.1.1) if not shows you something in json format you can try to start it
>>>> with command
>>>> rabbitmqctl start_app
>>>>
>>>> N/A, rabbitmq is already running.
>>>> 1.2 ) If no , start it and check the logs from your rabbitmq-server at
>>>> /var/log/rabbitmq/ and check if the service is start it normally .
>>>>
>>>> N/A, rabbitmq is already running.
>>>> 2) check the worker-output.log on your koha if koha write there
>> something
>>>> and have koha connection with rabbit server .
>>>>
>>>> I cannot find a file called "worker-output.log" anywhere in my Alma
>>> installation.
>>>
>>>
>>>> Best Regards,
>>>> George
>>>>
>>>>
>>>> -----Original Message-----
>>>> From: Koha <koha-bounces at lists.katipo.co.nz> On Behalf Of Michael Brown
>>>> Sent: Wednesday, May 10, 2023 3:50 PM
>>>> To: Cindy Murdock Ames <cmurdock at ccfls.org>
>>>> Cc: koha at lists.katipo.co.nz
>>>> Subject: Re: [Koha] Background job / Staging MARC import stuck at 0%
>>>>
>>>> Hi Cindy, apologies for the double-response: I had unsubscribed from
>> the
>>>> koha list so my response didn't get added to the thread. I wanted to
>> remedy
>>>> that, and also say that, because I had unsubscribed, I missed Mengü's
>>>> suggestion of updating manage-marc-import.pl. I am eager to try that
>> fix
>>>> and will report back later today or tomorrow with my findings. Here's my
>>>> original response to Cindy before re-subscribing to to the koha list:
>>>>
>>>> Yes, my koha-conf.xml does contain that block because I added it during
>>>> the initial setup. (You're right, it wasn't there originally.)
>>>>
>>>> However, it doesn't help; I still cannot import batch bib records in
>> Koha
>>>> 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my
>>>> cpu through the roof until I navigate away from that page. I've gotten
>> one
>>>> off-list suggestion that upgrading to the newest Koha fixes the problem.
>>>> Unfortunately for me, I don't really have time for that right now, so I
>>>> have (at least temporarily) abandoned my efforts with Koha.
>>>>
>>>> Thanks for your suggestion!
>>>>
>>>> Bests,
>>>> Michael
>>>>
>>>> On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown at gmail.com>
>>>> wrote:
>>>>
>>>>> Hi Cindy,
>>>>>
>>>>> Yes, my koha-conf.xml does contain that block because I added it
>>>>> during the initial setup. (You're right, it wasn't there originally.)
>>>>>
>>>>> However, it doesn't help; I still cannot import batch bib records in
>>>>> Koha
>>>>> 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends
>>>>> my cpu through the roof until I navigate away from that page. I've
>>>>> gotten one off-list suggestion that upgrading to the newest Koha fixes
>>>> the problem.
>>>>> Unfortunately for me, I don't really have time for that right now, so
>>>>> I have (at least temporarily) abandoned my efforts with Koha.
>>>>>
>>>>> Thanks for your suggestion!
>>>>>
>>>>> Bests,
>>>>> Michael
>>>>>
>>>>> On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <cmurdock at ccfls.org>
>>>>> wrote:
>>>>>
>>>>>> Hi Michael,
>>>>>>
>>>>>> Can you confirm your koha-conf.xml contains a message broker section,
>>>>>> like this?
>>>>>>
>>>>>> <message_broker>
>>>>>> <hostname>localhost</hostname>
>>>>>> <port>61613</port>
>>>>>> <username>guest</username>
>>>>>> <password>guest</password>
>>>>>> <vhost></vhost>
>>>>>> </message_broker>
>>>>>>
>>>>>> I had the same problem after upgrading from 22.05 to 22.11; I'm
>>>>>> guessing
>>>>>> 22.05 must have had some defaults set in the code that allowed the
>>>>>> message broker to start automatically without this section whereas
>>>> 22.11 must not.
>>>>>> I was missing this section and added it and was able to complete
>>>>>> imports after restarting Koha.
>>>>>>
>>>>>> I'm still having the issue with zombie background jobs (I'm afraid to
>>>>>> test rolling back the code on a production server) but I made a hacky
>>>>>> bash script to run as a cron job to check for them and kill them off
>>>>>> so at least our staff can do batch imports without issue, but that's
>>>> another matter.
>>>>>> HTH!
>>>>>> Cindy
>>>>>>
>>>>>>
>>>>>> -----------------------------------------------------------
>>>>>> Cindy Murdock Ames
>>>>>> IT Services Director
>>>>>> Meadville Public Library | CCFLS
>>>>>> https://meadvillelibrary.org | https://ccfls.org
>>>>>>
>>>>>> Please report tech support issues in Mantis:
>>>>>> https://mantis.ccfls.org
>>>>>>
>>>>>>
>>>>>> On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <michael8rown at gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Greetings:
>>>>>>>
>>>>>>> My name is Michael Brown and I am a professional cataloger and
>>>>>>> SirsiDynix System Admin at the Texas State Library & Archives in
>>>>>>> Austin (20+ years now). I have been using Koha on Arch Linux in my
>>>>>>> home library for about a year now. I am migrating my home server to
>>>>>>> AlmaLinux and I'm having a problem.
>>>>>>>
>>>>>>> I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a
>>>>>>> MARC file for import gets stuck at 0%. On screen, I am able to
>>>>>>> select the file for import (bib.mrc), review the profile options
>>>>>>> (but I don't change any defaults), and then click on "Stage for
>>>>>>> import" at the bottom. Next screen
>>>>>>> reads:
>>>>>>>
>>>>>>> The job has been enqueued! It will be processed as soon as
>>>> possible.
>>>>>>> 0%
>>>>>>> View detail of the enqueued job
>>>>>>>
>>>>>>> After a few seconds, it changes to (and then hangs at):
>>>>>>>
>>>>>>> The job has been enqueued! It will be processed as soon as
>>>> possible.
>>>>>>> 0% Not started
>>>>>>> View detail of the enqueued job
>>>>>>>
>>>>>>> Clicking on "View detail of the enqueued job" I see:
>>>>>>>
>>>>>>> Details of job #22
>>>>>>>
>>>>>>> Job ID: 22
>>>>>>> Status: New
>>>>>>> Progress: 0 / 0
>>>>>>> Type: Staged MARC records for import
>>>>>>> Queued: 03/02/2023 05:42
>>>>>>> Started:
>>>>>>> Ended:
>>>>>>>
>>>>>>> Report
>>>>>>> Detailed messages
>>>>>>> Return to the job list
>>>>>>>
>>>>>>> The corresponding entry in mariadb is:
>>>>>>>
>>>>>>> id 22
>>>>>>> status new
>>>>>>> progress NULL
>>>>>>> size 0
>>>>>>> borrowernumber 1
>>>>>>> type stage_marc_for_import
>>>>>>> queue Name of the queue the job is sent to long_tasks data
>>>>>>> {"encoding":"UTF-8","comments":"","basket_id":null...
>>>>>>> context JSON-serialized context information for the job
>>>>>>> {"flags":1,"branch":"ALMA","interface":"intranet",...
>>>>>>> enqueued_on 2023-03-02 05:42:33
>>>>>>> started_on NULL
>>>>>>> ended_on NULL
>>>>>>>
>>>>>>> (If you need to see the full entries for "data" and "context",
>>>>>>> please let me know.)
>>>>>>>
>>>>>>> tmp, koha_upload, and lock directories have been tweaked and fine
>>>> tuned.
>>>>>>> I
>>>>>>> was getting early warnings about them not being set in koha-conf.xml
>>>>>>> so I created them (and set correct permissions) and I can see the
>>>>>>> uploaded file (for job 22 the name is
>>>>>>> 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload
>>>>>>> function appears to be working.
>>>>>>>
>>>>>>> I am getting no apache errors and no other on-screen diagnostics.
>>>>>>>
>>>>>>> I have Koha 22.05.02.000 running on Arch Linux that imports this
>>>>>>> file just fine. Similarly, I have Koha latest running on a Debian VM
>>>>>>> that can import this file just fine, too.
>>>>>>>
>>>>>>> What am I missing?
>>>>>>>
>>>>>>> Details of my system:
>>>>>>>
>>>>>>> Koha version: 22.11.03.000 Rosalie
>>>>>>> OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1
>>>>>>> SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl
>>>>>>> interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC:
>>>>>>> /usr/share/koha/lib
>>>>>>> /usr/local/lib64/perl5/5.32
>>>>>>> /usr/local/share/perl5/5.32
>>>>>>> /usr/lib64/perl5/vendor_perl
>>>>>>> /usr/share/perl5/vendor_perl
>>>>>>> /usr/lib64/perl5
>>>>>>> /usr/share/perl5
>>>>>>> /var/lib/koha/plugins
>>>>>>> MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux
>>>>>>> (x86_64) using EditLine wrapper Apache version: Server version:
>>>>>>> Apache/2.4.53 (AlmaLinux) Server built:
>>>>>>> Jul
>>>>>>> 20 2022 00:00:00
>>>>>>> Memcached: Servers: 127.0.0.1:11211 | Namespace: KOHA | Status:
>>>>>>> running. |
>>>>>>> Config read from: koha-conf.xml
>>>>>>> Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free
>>>>>>> software, covered by the GNU General Public License, and you are
>>>>>>> welcome to change it and/or distribute copies of it under certain
>>>>>>> conditions. SHA1
>>>>>>> ID:
>>>>>>> ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status:
>>>>>>> Running Message broker: Using RabbitMQ Date and time: 03/01/2023
>>>>>>> 16:20 Time zone: Used: America/Chicago | Config: Undefined |
>>>>>>> Environment (TZ):
>>>>>>> Undefined
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Michael
>>>>>>> _______________________________________________
>>>>>>>
>>>>>>> Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz
>>>>>>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>>>>>>
>>>> _______________________________________________
>>>>
>>>> Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz
>>>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>>>
>>>>
>>> _______________________________________________
>>>
>>> Koha mailing list http://koha-community.org
>>> Koha at lists.katipo.co.nz
>>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>
>> ------------------------------
>>
>> Subject: Digest Footer
>>
>> _______________________________________________
>> Koha mailing list
>> Koha at lists.katipo.co.nz
>> https://lists.katipo.co.nz/mailman/listinfo/koha
>>
>>
>> ------------------------------
>>
>> End of Koha Digest, Vol 211, Issue 15
>> *************************************
>>
> _______________________________________________
>
> Koha mailing list http://koha-community.org
> Koha at lists.katipo.co.nz
> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
More information about the Koha
mailing list