[Koha] Background job / Staging MARC import stuck at 0%
Himanshu Aggarwal
rtr.himanshu at gmail.com
Tue May 30 16:51:36 NZST 2023
>From where are you importing the files?
On Tue, May 30, 2023 at 8:36 AM Anna Shields <apcshields at gmail.com> wrote:
> Michael,
>
> Did you ever have any luck resolving this?
>
> I'm having a similar issue in which my imports get stuck at 0% on the
> staging step. I, too, seem to have a functioning RabbitMQ, and none of the
> suggestions I saw in this thread seemed to make a difference.
>
> I'm now on v22.11.06-3, and updating hasn't seemed to make any difference,
> either.
>
> Anna
>
> On Tue, May 16, 2023 at 7:01 PM <koha-request at lists.katipo.co.nz> wrote:
>
> > Hi Michael,
> >
> > Did you also restart some services after changing the script including
> > memcache?
> >
> > Sometimes it is necessary but not remember for this case.
> >
> > You should also look
> > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=33412
> >
> > Mengü Yazıcıoğlu
> > Devinim Yazılım Eğitim Danışmanlık
> > Reşit Galip Cad. No: 29/6
> > Çankaya/ANKARA
> >
> > Tel: (312) 446 86 68
> > Cep: (532) 701 90 27
> >
> > On 13.05.2023 16:21, Michael Brown wrote:
> > > I tried Mengü's suggestion of editing manage-marc-import.pl, to no
> > avail.
> > > Still hangs at enqueuement; no messages, no errors, simply hangs and
> > spins
> > > my CPU into a frenzy. Responses to George's message below:
> > >
> > > On Wed, May 10, 2023 at 8:34 AM George Veranis <gveranis at dataly.gr>
> > wrote:
> > >
> > >> Hi Michael,
> > >>
> > >> Could you check the following things:
> > >>
> > >> 1) if your rabbitmq-server is running ?
> > >>
> > > Yes.
> > >
> > >
> > >> 1.1) if yes, then check if the cluster of nodes in rabbit exist , the
> > >> command is:
> > >> rabbitmqctl cluster_status
> > >>
> > > Cluster status of node rabbit at kiko ...
> > > Basics
> > >
> > > Cluster name: rabbit at kiko
> > >
> > > Disk Nodes
> > > rabbit at kiko
> > >
> > > Running Nodes
> > > rabbit at kiko
> > >
> > > Versions
> > > rabbit at kiko: RabbitMQ 3.9.21 on Erlang 24.3.4.2
> > >
> > > Maintenance status
> > >
> > > Node: rabbit at kiko, status: not under maintenance
> > >
> > > Alarms
> > >
> > > (none)
> > >
> > > Network Partitions
> > >
> > > (none)
> > >
> > > Listeners
> > >
> > > Node: rabbit at kiko, interface: [::], port: 61613, protocol: stomp,
> > purpose:
> > > STOMP
> > > Node: rabbit at kiko, interface: [::], port: 25672, protocol: clustering,
> > > purpose: inter-node and CLI tool communication
> > > Node: rabbit at kiko, interface: [::], port: 5672, protocol: amqp,
> purpose:
> > > AMQP 0-9-1 and AMQP 1.0
> > >
> > > Feature flags
> > >
> > > Flag: implicit_default_bindings, state: enabled
> > > Flag: maintenance_mode_status, state: enabled
> > > Flag: quorum_queue, state: enabled
> > > Flag: stream_queue, state: enabled
> > > Flag: user_limits, state: enabled
> > > Flag: virtual_host_metadata, state: enabled
> > >
> > >
> > >
> > >> 1.1.1) if not shows you something in json format you can try to start
> it
> > >> with command
> > >> rabbitmqctl start_app
> > >>
> > >> N/A, rabbitmq is already running.
> > >
> > >> 1.2 ) If no , start it and check the logs from your rabbitmq-server at
> > >> /var/log/rabbitmq/ and check if the service is start it normally .
> > >>
> > >> N/A, rabbitmq is already running.
> > >
> > >> 2) check the worker-output.log on your koha if koha write there
> > something
> > >> and have koha connection with rabbit server .
> > >>
> > >> I cannot find a file called "worker-output.log" anywhere in my Alma
> > > installation.
> > >
> > >
> > >> Best Regards,
> > >> George
> > >>
> > >>
> > >> -----Original Message-----
> > >> From: Koha <koha-bounces at lists.katipo.co.nz> On Behalf Of Michael
> Brown
> > >> Sent: Wednesday, May 10, 2023 3:50 PM
> > >> To: Cindy Murdock Ames <cmurdock at ccfls.org>
> > >> Cc: koha at lists.katipo.co.nz
> > >> Subject: Re: [Koha] Background job / Staging MARC import stuck at 0%
> > >>
> > >> Hi Cindy, apologies for the double-response: I had unsubscribed from
> > the
> > >> koha list so my response didn't get added to the thread. I wanted to
> > remedy
> > >> that, and also say that, because I had unsubscribed, I missed Mengü's
> > >> suggestion of updating manage-marc-import.pl. I am eager to try that
> > fix
> > >> and will report back later today or tomorrow with my findings. Here's
> my
> > >> original response to Cindy before re-subscribing to to the koha list:
> > >>
> > >> Yes, my koha-conf.xml does contain that block because I added it
> during
> > >> the initial setup. (You're right, it wasn't there originally.)
> > >>
> > >> However, it doesn't help; I still cannot import batch bib records in
> > Koha
> > >> 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends
> my
> > >> cpu through the roof until I navigate away from that page. I've gotten
> > one
> > >> off-list suggestion that upgrading to the newest Koha fixes the
> problem.
> > >> Unfortunately for me, I don't really have time for that right now, so
> I
> > >> have (at least temporarily) abandoned my efforts with Koha.
> > >>
> > >> Thanks for your suggestion!
> > >>
> > >> Bests,
> > >> Michael
> > >>
> > >> On Tue, May 9, 2023 at 4:40 PM Michael Brown <michael8rown at gmail.com>
> > >> wrote:
> > >>
> > >>> Hi Cindy,
> > >>>
> > >>> Yes, my koha-conf.xml does contain that block because I added it
> > >>> during the initial setup. (You're right, it wasn't there originally.)
> > >>>
> > >>> However, it doesn't help; I still cannot import batch bib records in
> > >>> Koha
> > >>> 22.11 the way I could in Koha 22.05. The enqueuement hangs and sends
> > >>> my cpu through the roof until I navigate away from that page. I've
> > >>> gotten one off-list suggestion that upgrading to the newest Koha
> fixes
> > >> the problem.
> > >>> Unfortunately for me, I don't really have time for that right now, so
> > >>> I have (at least temporarily) abandoned my efforts with Koha.
> > >>>
> > >>> Thanks for your suggestion!
> > >>>
> > >>> Bests,
> > >>> Michael
> > >>>
> > >>> On Tue, May 9, 2023 at 1:37 PM Cindy Murdock Ames <
> cmurdock at ccfls.org>
> > >>> wrote:
> > >>>
> > >>>> Hi Michael,
> > >>>>
> > >>>> Can you confirm your koha-conf.xml contains a message broker
> section,
> > >>>> like this?
> > >>>>
> > >>>> <message_broker>
> > >>>> <hostname>localhost</hostname>
> > >>>> <port>61613</port>
> > >>>> <username>guest</username>
> > >>>> <password>guest</password>
> > >>>> <vhost></vhost>
> > >>>> </message_broker>
> > >>>>
> > >>>> I had the same problem after upgrading from 22.05 to 22.11; I'm
> > >>>> guessing
> > >>>> 22.05 must have had some defaults set in the code that allowed the
> > >>>> message broker to start automatically without this section whereas
> > >> 22.11 must not.
> > >>>> I was missing this section and added it and was able to complete
> > >>>> imports after restarting Koha.
> > >>>>
> > >>>> I'm still having the issue with zombie background jobs (I'm afraid
> to
> > >>>> test rolling back the code on a production server) but I made a
> hacky
> > >>>> bash script to run as a cron job to check for them and kill them off
> > >>>> so at least our staff can do batch imports without issue, but that's
> > >> another matter.
> > >>>> HTH!
> > >>>> Cindy
> > >>>>
> > >>>>
> > >>>> -----------------------------------------------------------
> > >>>> Cindy Murdock Ames
> > >>>> IT Services Director
> > >>>> Meadville Public Library | CCFLS
> > >>>> https://meadvillelibrary.org | https://ccfls.org
> > >>>>
> > >>>> Please report tech support issues in Mantis:
> > >>>> https://mantis.ccfls.org
> > >>>>
> > >>>>
> > >>>> On Thu, Mar 2, 2023 at 8:51 AM Michael Brown <
> michael8rown at gmail.com>
> > >>>> wrote:
> > >>>>
> > >>>>> Greetings:
> > >>>>>
> > >>>>> My name is Michael Brown and I am a professional cataloger and
> > >>>>> SirsiDynix System Admin at the Texas State Library & Archives in
> > >>>>> Austin (20+ years now). I have been using Koha on Arch Linux in my
> > >>>>> home library for about a year now. I am migrating my home server to
> > >>>>> AlmaLinux and I'm having a problem.
> > >>>>>
> > >>>>> I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a
> > >>>>> MARC file for import gets stuck at 0%. On screen, I am able to
> > >>>>> select the file for import (bib.mrc), review the profile options
> > >>>>> (but I don't change any defaults), and then click on "Stage for
> > >>>>> import" at the bottom. Next screen
> > >>>>> reads:
> > >>>>>
> > >>>>> The job has been enqueued! It will be processed as soon as
> > >> possible.
> > >>>>> 0%
> > >>>>> View detail of the enqueued job
> > >>>>>
> > >>>>> After a few seconds, it changes to (and then hangs at):
> > >>>>>
> > >>>>> The job has been enqueued! It will be processed as soon as
> > >> possible.
> > >>>>> 0% Not started
> > >>>>> View detail of the enqueued job
> > >>>>>
> > >>>>> Clicking on "View detail of the enqueued job" I see:
> > >>>>>
> > >>>>> Details of job #22
> > >>>>>
> > >>>>> Job ID: 22
> > >>>>> Status: New
> > >>>>> Progress: 0 / 0
> > >>>>> Type: Staged MARC records for import
> > >>>>> Queued: 03/02/2023 05:42
> > >>>>> Started:
> > >>>>> Ended:
> > >>>>>
> > >>>>> Report
> > >>>>> Detailed messages
> > >>>>> Return to the job list
> > >>>>>
> > >>>>> The corresponding entry in mariadb is:
> > >>>>>
> > >>>>> id 22
> > >>>>> status new
> > >>>>> progress NULL
> > >>>>> size 0
> > >>>>> borrowernumber 1
> > >>>>> type stage_marc_for_import
> > >>>>> queue Name of the queue the job is sent to long_tasks data
> > >>>>> {"encoding":"UTF-8","comments":"","basket_id":null...
> > >>>>> context JSON-serialized context information for the job
> > >>>>> {"flags":1,"branch":"ALMA","interface":"intranet",...
> > >>>>> enqueued_on 2023-03-02 05:42:33
> > >>>>> started_on NULL
> > >>>>> ended_on NULL
> > >>>>>
> > >>>>> (If you need to see the full entries for "data" and "context",
> > >>>>> please let me know.)
> > >>>>>
> > >>>>> tmp, koha_upload, and lock directories have been tweaked and fine
> > >> tuned.
> > >>>>> I
> > >>>>> was getting early warnings about them not being set in
> koha-conf.xml
> > >>>>> so I created them (and set correct permissions) and I can see the
> > >>>>> uploaded file (for job 22 the name is
> > >>>>> 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so the actual upload
> > >>>>> function appears to be working.
> > >>>>>
> > >>>>> I am getting no apache errors and no other on-screen diagnostics.
> > >>>>>
> > >>>>> I have Koha 22.05.02.000 running on Arch Linux that imports this
> > >>>>> file just fine. Similarly, I have Koha latest running on a Debian
> VM
> > >>>>> that can import this file just fine, too.
> > >>>>>
> > >>>>> What am I missing?
> > >>>>>
> > >>>>> Details of my system:
> > >>>>>
> > >>>>> Koha version: 22.11.03.000 Rosalie
> > >>>>> OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1
> > >>>>> SMP PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64 Perl
> > >>>>> interpreter: /usr/bin/perl Perl version: 5.032001 Perl @INC:
> > >>>>> /usr/share/koha/lib
> > >>>>> /usr/local/lib64/perl5/5.32
> > >>>>> /usr/local/share/perl5/5.32
> > >>>>> /usr/lib64/perl5/vendor_perl
> > >>>>> /usr/share/perl5/vendor_perl
> > >>>>> /usr/lib64/perl5
> > >>>>> /usr/share/perl5
> > >>>>> /var/lib/koha/plugins
> > >>>>> MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux
> > >>>>> (x86_64) using EditLine wrapper Apache version: Server version:
> > >>>>> Apache/2.4.53 (AlmaLinux) Server built:
> > >>>>> Jul
> > >>>>> 20 2022 00:00:00
> > >>>>> Memcached: Servers: 127.0.0.1:11211 | Namespace: KOHA | Status:
> > >>>>> running. |
> > >>>>> Config read from: koha-conf.xml
> > >>>>> Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free
> > >>>>> software, covered by the GNU General Public License, and you are
> > >>>>> welcome to change it and/or distribute copies of it under certain
> > >>>>> conditions. SHA1
> > >>>>> ID:
> > >>>>> ac40f289672405a299436d73c1532f9906774cc6 Using ICU Zebra status:
> > >>>>> Running Message broker: Using RabbitMQ Date and time: 03/01/2023
> > >>>>> 16:20 Time zone: Used: America/Chicago | Config: Undefined |
> > >>>>> Environment (TZ):
> > >>>>> Undefined
> > >>>>>
> > >>>>> Thanks,
> > >>>>> Michael
> > >>>>> _______________________________________________
> > >>>>>
> > >>>>> Koha mailing list http://koha-community.org
> Koha at lists.katipo.co.nz
> > >>>>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
> > >>>>>
> > >> _______________________________________________
> > >>
> > >> Koha mailing list http://koha-community.org Koha at lists.katipo.co.nz
> > >> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
> > >>
> > >>
> > > _______________________________________________
> > >
> > > Koha mailing list http://koha-community.org
> > > Koha at lists.katipo.co.nz
> > > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
> >
> >
> > ------------------------------
> >
> > Subject: Digest Footer
> >
> > _______________________________________________
> > Koha mailing list
> > Koha at lists.katipo.co.nz
> > https://lists.katipo.co.nz/mailman/listinfo/koha
> >
> >
> > ------------------------------
> >
> > End of Koha Digest, Vol 211, Issue 15
> > *************************************
> >
> _______________________________________________
>
> Koha mailing list http://koha-community.org
> Koha at lists.katipo.co.nz
> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>
--
Assistant Librarian,
Confucius Institute Scholar 13-14,
R.D. National College,
Bandra (W.)
More information about the Koha
mailing list