[Koha] Background job / Staging MARC import stuck at 0%

Michael Brown michael8rown at gmail.com
Sat Mar 4 11:48:55 NZDT 2023 

Thanks, Eric. Still no luck even though I downgraded several packages to
the Koha required versions:

     JSON::Validator => 5.08
     Mojolicious => 8.12
     Mojolicious::Plugin::OAuth2 => 2.02
     Mojolicious::Plugin::OpenAPI => 5.05
     Mojolicious::Plugin::RenderFile => 0.12

background_jobs.data =
{"parse_items":"1","marc_modification_template":null,"nomatch_action":"create_new","encoding":"UTF-8","record_type":"biblio","matcher_id":"","vendor_id":null,"basket_id":null,"filepath":"/var/lib/koha/alma/tmp/koha_upload/b63453b06ba1fa2ca63a76e64e4909bb_l.mrc","comments":"","filename":"l.mrc","item_action":"always_add","format":"ISO2709","overlay_action":"replace"}

Still nothing being logged, but app.pl is really lighting up the CPU:

top - 16:38:47 up 5 min,  1 user,  load average: 1.32, 1.04, 0.50
Tasks: 237 total,   2 running, 235 sleeping,   0 stopped,   0 zombie
%Cpu(s): 26.3 us,  1.9 sy,  0.0 ni, 71.5 id,  0.0 wa,  0.2 hi,  0.1 si,
 0.0 st
MiB Mem :   5827.8 total,   2597.2 free,   2334.9 used,   1492.9 buff/cache
MiB Swap:   1331.0 total,   1331.0 free,      0.0 used.   3492.9 avail Mem

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+
COMMAND
   3936 apache    20   0  127888 121296  19480 R  99.3   2.0   0:04.01
app.pl
   1592 michael   20   0 4325576 206032 112492 S   3.3   3.5   0:15.64
gnome-shell
   1179 michael   20   0  521988  92176  57912 S   3.0   1.5   0:13.63 Xorg

   2217 michael   20   0  691664  50664  38408 S   1.3   0.8   0:03.49
gnome-terminal-
   3599 michael   20   0  226528   4204   3332 R   0.7   0.1   0:00.48 top

    783 memcach+  20   0  418036  11400   4360 S   0.3   0.2   0:00.12
memcached
    786 rabbitmq  20   0 2848520 127696  65352 S   0.3   2.1   0:09.08
beam.smp

It will remain at about that level until I close the window.

Michael

On Fri, Mar 3, 2023 at 12:56 PM Eric Phetteplace <ephetteplace at cca.edu>
wrote:

> I would absolutely think that the wrong version of JSON::Validator could
> cause a problem here. Try to match Koha's versions in the cpanfile,
> probably `cpanm --installdeps .` from the root but maybe that's not wise
> with a package installation.
>
> Best,
>
> ERIC PHETTEPLACE Systems Librarian, Libraries (he/him)
>
> ephetteplace at cca.edu
>
>
> *CCA is situated on the traditional unceded lands of the **Chochenyo and
> Ramaytush Ohlone** peoples.*
>
> Black-owned bookstores in Oakland: Ashay by the Bay
> <https://ashaybythebay.com/>, Marcus Books
> <https://www.facebook.com/marcus.books/>
>
> :(){ :|: & };:
>
>
> On Fri, Mar 3, 2023 at 5:58 AM Michael Brown <michael8rown at gmail.com>
> wrote:
>
>> Hi Jonathan,
>>
>> Thanks for the suggestion; nothing much to report, though.
>>
>>     tail -f /var/log/koha/*log
>>
>> showed nothing at all when I staged "l.mrc" this morning.
>>
>> As for the background_job.data suggestion, it took me a minute to
>> understand what you were referring to and by the time I figured it out I
>> had already left home, so I cannot access my server right now. However, I
>> did copy the background_job.data entry for one of the uploads I had
>> performed yesterday. Here it is:
>>
>>
>> {"encoding":"UTF-8","comments":"","basket_id":null,"record_type":"biblio","vendor_id":null,"matcher_id":"","parse_items":"1","item_action":"always_add","format":"ISO2709","marc_modification_template":null,"filepath":"/var/lib/koha/alma/tmp/koha_upload/2287629673fb980ad4102f62ebeaa1b9_bib.mrc","nomatch_action":"create_new","filename":"bib.mrc","overlay_action":"replace"}
>>
>> Is that helpful? If not, I will pull the entry from the test I made this
>> morning while watching the logs.
>>
>> When I was first starting out with Koha (on Arch Linux about a year ago) I
>> was having issues that I eventually traced to JSON Validator and
>> Mojolicious perl modules. They were both much newer than those required by
>> Koha, and when I downgraded those two modules, things started working. But
>> my problem back then was not with uploads; it was with the display of
>> borrowers. Patrons were not loading on-screen and an Inspect > Console
>> message gave me the idea to downgrade. It worked after that.
>>
>> Could this possibly be a similar problem? Should I try force-downgrading
>> JSON::Validator and/or Mojolicious? Is there any debugging I can do of the
>> Koha::REST::V1 module? As I reported yesterday, app.pl shows
>> continual-constant CPU usage as long as that enqueuement page is open.
>> Once
>> I navigate away from that enqueuement page, the app.pl process quits, CPU
>> usage returns to normal.
>>
>> Michael
>>
>> On Fri, Mar 3, 2023 at 12:55 AM Jonathan Druart <
>> jonathan.druart at bugs.koha-community.org> wrote:
>>
>> > Hello Michael,
>> > I have just tried on 22.11.03 and it seems to work for me.
>> > Can you try to open a console, watch the logs: tail -f
>> > /var/log/koha/$KOHA_INSTANCE/*.log
>> > Import a file.
>> > Do you see something in the logs?
>> >
>> > Otherwise look at the background_jobs.data JSON, you may see an error in
>> > it. But if the status is still "new" I am not expecting an error there.
>> >
>> >
>> > Le jeu. 2 mars 2023 à 14:50, Michael Brown <michael8rown at gmail.com> a
>> > écrit :
>> >
>> > > Greetings:
>> > >
>> > > My name is Michael Brown and I am a professional cataloger and
>> SirsiDynix
>> > > System Admin at the Texas State Library & Archives in Austin (20+
>> years
>> > > now). I have been using Koha on Arch Linux in my home library for
>> about a
>> > > year now. I am migrating my home server to AlmaLinux and I'm having a
>> > > problem.
>> > >
>> > > I am running Koha 22.11.03.000 Rosalie on AlmaLinux 9.1. Staging a
>> MARC
>> > > file for import gets stuck at 0%. On screen, I am able to select the
>> file
>> > > for import (bib.mrc), review the profile options (but I don't change
>> any
>> > > defaults), and then click on "Stage for import" at the bottom. Next
>> > screen
>> > > reads:
>> > >
>> > >     The job has been enqueued! It will be processed as soon as
>> possible.
>> > >     0%
>> > >     View detail of the enqueued job
>> > >
>> > > After a few seconds, it changes to (and then hangs at):
>> > >
>> > >     The job has been enqueued! It will be processed as soon as
>> possible.
>> > >     0% Not started
>> > >     View detail of the enqueued job
>> > >
>> > > Clicking on "View detail of the enqueued job" I see:
>> > >
>> > > Details of job #22
>> > >
>> > >     Job ID: 22
>> > >     Status:    New
>> > >     Progress:    0 / 0
>> > >     Type:    Staged MARC records for import
>> > >     Queued:    03/02/2023 05:42
>> > >     Started:
>> > >     Ended:
>> > >
>> > > Report
>> > > Detailed messages
>> > > Return to the job list
>> > >
>> > > The corresponding entry in mariadb is:
>> > >
>> > > id 22
>> > > status new
>> > > progress NULL
>> > > size 0
>> > > borrowernumber 1
>> > > type stage_marc_for_import
>> > > queue Name of the queue the job is sent to long_tasks
>> > > data {"encoding":"UTF-8","comments":"","basket_id":null...
>> > > context JSON-serialized context information for the job
>> > > {"flags":1,"branch":"ALMA","interface":"intranet",...
>> > > enqueued_on 2023-03-02 05:42:33
>> > > started_on NULL
>> > > ended_on NULL
>> > >
>> > > (If you need to see the full entries for "data" and "context", please
>> let
>> > > me know.)
>> > >
>> > > tmp, koha_upload, and lock directories have been tweaked and fine
>> tuned.
>> > I
>> > > was getting early warnings about them not being set in koha-conf.xml
>> so I
>> > > created them (and set correct permissions) and I can see the uploaded
>> > file
>> > > (for job 22 the name is 2287629673fb980ad4102f62ebeaa1b9_bib.mrc), so
>> the
>> > > actual upload function appears to be working.
>> > >
>> > > I am getting no apache errors and no other on-screen diagnostics.
>> > >
>> > > I have Koha 22.05.02.000 running on Arch Linux that imports this file
>> > just
>> > > fine. Similarly, I have Koha latest running on a Debian VM that can
>> > import
>> > > this file just fine, too.
>> > >
>> > > What am I missing?
>> > >
>> > > Details of my system:
>> > >
>> > > Koha version: 22.11.03.000 Rosalie
>> > > OS version ('uname -a'): Linux alma 5.14.0-162.12.1.el9_1.x86_64 #1
>> SMP
>> > > PREEMPT_DYNAMIC Mon Jan 23 14:51:52 EST 2023 x86_64
>> > > Perl interpreter: /usr/bin/perl
>> > > Perl version: 5.032001
>> > > Perl @INC: /usr/share/koha/lib
>> > > /usr/local/lib64/perl5/5.32
>> > > /usr/local/share/perl5/5.32
>> > > /usr/lib64/perl5/vendor_perl
>> > > /usr/share/perl5/vendor_perl
>> > > /usr/lib64/perl5
>> > > /usr/share/perl5
>> > > /var/lib/koha/plugins
>> > > MySQL version: mysql Ver 15.1 Distrib 10.5.16-MariaDB, for Linux
>> (x86_64)
>> > > using EditLine wrapper
>> > > Apache version: Server version: Apache/2.4.53 (AlmaLinux) Server
>> built:
>> > Jul
>> > > 20 2022 00:00:00
>> > > Memcached: Servers: 127.0.0.1:11211 | Namespace: KOHA | Status:
>> > running. |
>> > > Config read from: koha-conf.xml
>> > > Zebra version: Zebra 2.2.7 (C) 1994-2023, Index Data Zebra is free
>> > > software, covered by the GNU General Public License, and you are
>> welcome
>> > to
>> > > change it and/or distribute copies of it under certain conditions.
>> SHA1
>> > ID:
>> > > ac40f289672405a299436d73c1532f9906774cc6 Using ICU
>> > > Zebra status: Running
>> > > Message broker: Using RabbitMQ
>> > > Date and time: 03/01/2023 16:20
>> > > Time zone: Used: America/Chicago | Config: Undefined | Environment
>> (TZ):
>> > > Undefined
>> > >
>> > > Thanks,
>> > > Michael
>> > > _______________________________________________
>> > >
>> > > Koha mailing list  http://koha-community.org
>> > > Koha at lists.katipo.co.nz
>> > > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>> > >
>> > _______________________________________________
>> >
>> > Koha mailing list  http://koha-community.org
>> > Koha at lists.katipo.co.nz
>> > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>> >
>> _______________________________________________
>>
>> Koha mailing list  http://koha-community.org
>> Koha at lists.katipo.co.nz
>> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>
>

More information about the Koha mailing list