[Koha] Background job / Staging MARC import stuck at 0%
Mengu Yazicioglu
mengu at devinim.com.tr
Wed May 17 08:35:48 NZST 2023
Hi Michael,
Did you also restart some services after changing the script including
memcache?
Sometimes it is necessary but not remember for this case.
You should also look
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33412
Mengü Yazıcıoğlu
Devinim Yazılım Eğitim Danışmanlık
Reşit Galip Cad. No: 29/6
Çankaya/ANKARA
Tel: (312) 446 86 68
Cep: (532) 701 90 27
On 13.05.2023 16:21, Michael Brown wrote:
> I tried Mengü's suggestion of editing manage-marc-import.pl, to no avail.
> Still hangs at enqueuement; no messages, no errors, simply hangs and spins
> my CPU into a frenzy. Responses to George's message below:
>
> On Wed, May 10, 2023 at 8:34 AM George Veranis <gveranis at dataly.gr> wrote:
>
>> Hi Michael,
>>
>> Could you check the following things:
>>
>> 1) if your rabbitmq-server is running ?
>>
> Yes.
>
>
>> 1.1) if yes, then check if the cluster of nodes in rabbit exist , the
>> command is:
>> rabbitmqctl cluster_status
>>
> Cluster status of node rabbit at kiko ...
> Basics
>
> Cluster name: rabbit at kiko
>
> Disk Nodes
> rabbit at kiko
>
> Running Nodes
> rabbit at kiko
>
> Versions
> rabbit at kiko: RabbitMQ 3.9.21 on Erlang 24.3.4.2
>
> Maintenance status
>
> Node: rabbit at kiko, status: not under maintenance
>
> Alarms
>
> (none)
>
> Network Partitions
>
> (none)
>
> Listeners
>
> Node: rabbit at kiko, interface: [::], port: 61613, protocol: stomp, purpose:
> STOMP
> Node: rabbit at kiko, interface: [::], port: 25672, protocol: clustering,
> purpose: inter-node and CLI tool communication
> Node: rabbit at kiko, interface: [::], port: 5672, protocol: amqp, purpose:
> AMQP 0-9-1 and AMQP 1.0
>
> Feature flags
>
> Flag: implicit_default_bindings, state: enabled
> Flag: maintenance_mode_status, state: enabled
> Flag: quorum_queue, state: enabled
> Flag: stream_queue, state: enabled
> Flag: user_limits, state: enabled
> Flag: virtual_host_metadata, state: enabled
>
>
>
>> 1.1.1) if not shows you something in json format you can try to start it
>> with command
>> rabbitmqctl start_app
>>
>> N/A, rabbitmq is already running.
>
>> 1.2 ) If no , start it and check the logs from your rabbitmq-server at
>> /var/log/rabbitmq/ and check if the service is start it normally .
>>
>> N/A, rabbitmq is already running.
>
>> 2) check the worker-output.log on your koha if koha write there something
>> and have koha connection with rabbit server .
>>
>> I cannot find a file called "worker-output.log" anywhere in my Alma
> installation.
>
>
>> Best Regards,
>> George
>>
>>
>> -----Original Message-----
>> From: Koha <koha-bounces at lists.katipo.co.nz> On Behalf Of Michael Brown
>> Sent: Wednesday, May 10, 2023 3:50 PM
>> To: Cindy Murdock Ames <cmurdock at ccfls.org>
>> Cc: koha at lists.katipo.co.nz
>> Subject: Re: [Koha] Background job / Staging MARC import stuck at 0%
>>
>> Hi Cindy, apologies for the double-response: I had unsubscribed from the
>> koha list so my response didn't get added to the thread. I wanted to remedy
>> that, and also say that, because I had unsubscribed, I missed Mengü's
>> suggestion of updating manage-marc-import.pl. I am eager to try that fix
>> and will report back later today or tomorrow with my findings. Here's my
>> original response to Cindy before re-subscribing to to the koha list:
>>
>> Yes, my koha-conf.xml does contain that block because I added it during
>> the initial setup. (You're right, it wasn't there originally.)
>>
>> However, it doesn't help; I still cannot import batch bib records in Koha
>> 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends my
>> cpu through the roof until I navigate away from that page. I've gotten one
>> off-list suggestion that upgrading to the newest Koha fixes the problem.
>> Unfortunately for me, I don't really have time for that right now, so I
>> have (at least temporarily) abandoned my efforts with Koha.
>>
>> Thanks for your suggestion!
>>
>> Bests,
>> Michael
>>
>> On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown at gmail.com>
>> wrote:
>>
>>> Hi Cindy,
>>>
>>> Yes, my koha-conf.xml does contain that block because I added it
>>> during the initial setup. (You're right, it wasn't there originally.)
>>>
>>> However, it doesn't help; I still cannot import batch bib records in
>>> Koha
>>> 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends
>>> my cpu through the roof until I navigate away from that page. I've
>>> gotten one off-list suggestion that upgrading to the newest Koha fixes
>> the problem.
>>> Unfortunately for me, I don't really have time for that right now, so
>>> I have (at least temporarily) abandoned my efforts with Koha.
>>>
>>> Thanks for your suggestion!
>>>
>>> Bests,
>>> Michael
>>>
>>> On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <cmurdock at ccfls.org>
>>> wrote:
>>>
>>>> Hi Michael,
>>>>
>>>> Can you confirm your koha-conf.xml contains a message broker section,
>>>> like this?
>>>>
>>>> <message_broker>
>>>> <hostname>localhost</hostname>
>>>> <port>61613</port>
>>>> <username>guest</username>
>>>> <password>guest</password>
>>>> <vhost></vhost>
>>>> </message_broker>
>>>>
>>>> I had the same problem after upgrading from 22.05 to 22.11; I'm
>>>> guessing
>>>> 22.05 must have had some defaults set in the code that allowed the
>>>> message broker to start automatically without this section whereas
>> 22.11 must not.
>>>> I was missing this section and added it and was able to complete
>>>> imports after restarting Koha.
>>>>
>>>> I'm still having the issue with zombie background jobs (I'm afraid to
>>>> test rolling back the code on a production server) but I made a hacky
>>>> bash script to run as a cron job to check for them and kill them off
>>>> so at least our staff can do batch imports without issue, but that's
>> another matter.
>>>> HTH!
>>>> Cindy
>>>>
>>>>
>>>> -----------------------------------------------------------
>>>> Cindy Murdock Ames
>>>> IT Services Director
>>>> Meadville Public Library | CCFLS
>>>> https://meadvillelibrary.org | https://ccfls.org
>>>>
>>>> Please report tech support issues in Mantis:
>>>> https://mantis.ccfls.org
>>>>
>>>>
>>>> On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <michael8rown at gmail.com>
>>>> wrote:
>>>>
>>>>> Greetings:
>>>>>
>>>>> My name is Michael Brown and I am a professional cataloger and
>>>>> SirsiDynix System Admin at the Texas State Library & Archives in
>>>>> Austin (20+ years now). I have been using Koha on Arch Linux in my
>>>>> home library for about a year now. I am migrating my home server to
>>>>> AlmaLinux and I'm having a problem.
>>>>>
>>>>> I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a
>>>>> MARC file for import gets stuck at 0%. On screen, I am able to
>>>>> select the file for import (bib.mrc), review the profile options
>>>>> (but I don't change any defaults), and then click on "Stage for
>>>>> import" at the bottom. Next screen
>>>>> reads:
>>>>>
>>>>> The job has been enqueued! It will be processed as soon as
>> possible.
>>>>> 0%
>>>>> View detail of the enqueued job
>>>>>
>>>>> After a few seconds, it changes to (and then hangs at):
>>>>>
>>>>> The job has been enqueued! It will be processed as soon as
>> possible.
>>>>> 0% Not started
>>>>> View detail of the enqueued job
>>>>>
>>>>> Clicking on "View detail of the enqueued job" I see:
>>>>>
>>>>> Details of job #22
>>>>>
>>>>> Job ID: 22
>>>>> Status: New
>>>>> Progress: 0 / 0
>>>>> Type: Staged MARC records for import
>>>>> Queued: 03/02/2023 05:42
>>>>> Started:
>>>>> Ended:
>>>>>
>>>>> Report
>>>>> Detailed messages
>>>>> Return to the job list
>>>>>
>>>>> The corresponding entry in mariadb is:
>>>>>
>>>>> id 22
>>>>> status new
>>>>> progress NULL
>>>>> size 0
>>>>> borrowernumber 1
>>>>> type stage_marc_for_import
>>>>> queue Name of the queue the job is sent to long_tasks data
>>>>> {"encoding":"UTF-8","comments":"","basket_id":null...
>>>>> context JSON-serialized context information for the job
>>>>> {"flags":1,"branch":"ALMA","interface":"intranet",...
>>>>> enqueued_on 2023-03-02 05:42:33
>>>>> started_on NULL
>>>>> ended_on NULL
>>>>>
>>>>> (If you need to see the full entries for "data" and "context",
>>>>> please let me know.)
>>>>>
>>>>> tmp, koha_upload, and lock directories have been tweaked and fine
>> tuned.
>>>>> I
>>>>> was getting early warnings about them not being set in koha-conf.xml
>>>>> so I created them (and set correct permissions) and I can see the
>>>>> uploaded file (for job 22 the name is
>>>>> 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload
>>>>> function appears to be working.
>>>>>
>>>>> I am getting no apache errors and no other on-screen diagnostics.
>>>>>
>>>>> I have Koha 22.05.02.000 running on Arch Linux that imports this
>>>>> file just fine. Similarly, I have Koha latest running on a Debian VM
>>>>> that can import this file just fine, too.
>>>>>
>>>>> What am I missing?
>>>>>
>>>>> Details of my system:
>>>>>
>>>>> Koha version: 22.11.03.000 Rosalie
>>>>> OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1
>>>>> SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl
>>>>> interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC:
>>>>> /usr/share/koha/lib
>>>>> /usr/local/lib64/perl5/5.32
>>>>> /usr/local/share/perl5/5.32
>>>>> /usr/lib64/perl5/vendor_perl
>>>>> /usr/share/perl5/vendor_perl
>>>>> /usr/lib64/perl5
>>>>> /usr/share/perl5
>>>>> /var/lib/koha/plugins
>>>>> MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux
>>>>> (x86_64) using EditLine wrapper Apache version: Server version:
>>>>> Apache/2.4.53 (AlmaLinux) Server built:
>>>>> Jul
>>>>> 20 2022 00:00:00
>>>>> Memcached: Servers: 127.0.0.1:11211 | Namespace: KOHA | Status:
>>>>> running. |
>>>>> Config read from: koha-conf.xml
>>>>> Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free
>>>>> software, covered by the GNU General Public License, and you are
>>>>> welcome to change it and/or distribute copies of it under certain
>>>>> conditions. SHA1
>>>>> ID:
>>>>> ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status:
>>>>> Running Message broker: Using RabbitMQ Date and time: 03/01/2023
>>>>> 16:20 Time zone: Used: America/Chicago | Config: Undefined |
>>>>> Environment (TZ):
>>>>> Undefined
>>>>>
>>>>> Thanks,
>>>>> Michael
>>>>> _______________________________________________
>>>>>
>>>>> Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz
>>>>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>>>>
>> _______________________________________________
>>
>> Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz
>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>
>>
> _______________________________________________
>
> Koha mailing list http://koha-community.org
> Koha at lists.katipo.co.nz
> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
More information about the Koha
mailing list