Export Data tool fails (only tool that fails)- ver 23.11.06
Hello, (I started a similar question on a thread titled "Export page failing to load and related error". Posting again with additional information and to bump the message into new views) *Issue:* The "Export Data" tool will not load. I go to *More* > *Cataloging* > *Export catalog data* and the page immediately fails. (server details at the bottom of the message) The error message is a standard message: "*Service Unavailable:* The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later." All other tools in the Koha GUI work. The export is the only one failing. Does anyone know how to better troubleshoot this issue? *Troubleshooting Steps Tried:* - Upgrade my server to 16GB of RAM. Results: same error - With koha-shell, run the command "/usr/share/koha/bin/export_records.pl --filename=JVK-manual-export". Results: the shell displays all the records, but I cannot find a file that was generated. *Background:* My team uses one instance of Koha to build catalogs that will eventually be sent to remote library locations Africa. I can access the database directly and run a count query on the "items" table. It shows we have *63,426* unique items in the catalog. We have multiple branches to organize all the items. The goal is to export the catalog data per branch library and then send the MARC file to the remote libraries so the librarian can import them into their local instance of Koha. We've been doing this for a few years, so the item records continue to grow. *Koha server details:* Koha version: 23.11.06.000 Server OS: Debian 11 RAM: 8GB CPU: 2 Storage: 160 GB Running on Amazon Lightsail Sincerely, Evan Stewart Africa's Hope, USA
Hi Evan. It's a long time since I've dealt with Apache server errors and the 503 error you are getting, but here are some things to look at: - As you have tried with plenty of server memory, it's not likely to be that. It may be something to do with how Apache is configured - including settings like the Timeout, KeepAlive, MaxKeepAliveRequests, KeepAliveTimeout settings. - I would look at the Apache server logs to see if there is something more in there about the cause of the issue. - If your server is behind a firewall, proxy servers, or caches (something I don't know much about) then that may be contributing as well. It may also be a Koha bug, but 63,000 items is not that large, and I couldn't find a bug about performance issues when exporting the catalog. For running the misc/export_records.pl script from the command line, then outputting to the terminal instead of saving to the file, this may be a file permission error (well, that's what my searching says). In my local development environment testing, it was saving the file to where I ran the command line from. So maybe changing the path of the file to somewhere where the user you are running the command with has permission may work. I'm not a system admin, so hopefully more experienced Koha system administrators can share what they would try to troubleshoot the issues you are having. David Nind New Zealand
HI David, Thank you. I'll look into the Apache settings as well. That reminds me of a similar issue I had with a PHP application and the problem was PHP execution time settings. Also, I located an error message in the log. Looks like I'm getting a permission denied error for exports.pl. *[cgid:error] [pid 554389:tid 554389] (13)Permission denied: [client 172.56.200.129:42349 <http://172.56.200.129:42349>] AH01257: stderr from /usr/share/koha/intranet/cgi-bin/tools/export.pl <http://export.pl>: unable to connect to cgi daemon after multiple tries, referer: http://Ipaddress:8081/cgi-bin/koha/tools/inventory.pl <http://Ipaddress:8081/cgi-bin/koha/tools/inventory.pl>* When I checked the permission for the file, they are the same as other files in the "... /cgi-bin/tools" directory. The file inventory.pl permissions are the same as export.pl. The inventory page works on the front end. -Evan On Fri, Jul 19, 2024 at 3:50 PM David Nind <david@davidnind.com> wrote:
Hi Evan.
It's a long time since I've dealt with Apache server errors and the 503 error you are getting, but here are some things to look at:
- As you have tried with plenty of server memory, it's not likely to be that. It may be something to do with how Apache is configured - including settings like the Timeout, KeepAlive, MaxKeepAliveRequests, KeepAliveTimeout settings.
- I would look at the Apache server logs to see if there is something more in there about the cause of the issue.
- If your server is behind a firewall, proxy servers, or caches (something I don't know much about) then that may be contributing as well.
It may also be a Koha bug, but 63,000 items is not that large, and I couldn't find a bug about performance issues when exporting the catalog.
For running the misc/export_records.pl script from the command line, then outputting to the terminal instead of saving to the file, this may be a file permission error (well, that's what my searching says).
In my local development environment testing, it was saving the file to where I ran the command line from. So maybe changing the path of the file to somewhere where the user you are running the command with has permission may work.
I'm not a system admin, so hopefully more experienced Koha system administrators can share what they would try to troubleshoot the issues you are having.
David Nind New Zealand
All, I checked the Apache config settings. I have: Timeout 300 (seconds) KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 5 (seconds) -Evan On Fri, Jul 19, 2024 at 4:18 PM Evan Stewart <evanstewart406@gmail.com> wrote:
HI David,
Thank you. I'll look into the Apache settings as well. That reminds me of a similar issue I had with a PHP application and the problem was PHP execution time settings.
Also, I located an error message in the log. Looks like I'm getting a permission denied error for exports.pl.
*[cgid:error] [pid 554389:tid 554389] (13)Permission denied: [client 172.56.200.129:42349 <http://172.56.200.129:42349>] AH01257: stderr from /usr/share/koha/intranet/cgi-bin/tools/export.pl <http://export.pl>: unable to connect to cgi daemon after multiple tries, referer: http://Ipaddress:8081/cgi-bin/koha/tools/inventory.pl <http://Ipaddress:8081/cgi-bin/koha/tools/inventory.pl>*
When I checked the permission for the file, they are the same as other files in the "... /cgi-bin/tools" directory. The file inventory.pl permissions are the same as export.pl. The inventory page works on the front end.
-Evan
On Fri, Jul 19, 2024 at 3:50 PM David Nind <david@davidnind.com> wrote:
Hi Evan.
It's a long time since I've dealt with Apache server errors and the 503 error you are getting, but here are some things to look at:
- As you have tried with plenty of server memory, it's not likely to be that. It may be something to do with how Apache is configured - including settings like the Timeout, KeepAlive, MaxKeepAliveRequests, KeepAliveTimeout settings.
- I would look at the Apache server logs to see if there is something more in there about the cause of the issue.
- If your server is behind a firewall, proxy servers, or caches (something I don't know much about) then that may be contributing as well.
It may also be a Koha bug, but 63,000 items is not that large, and I couldn't find a bug about performance issues when exporting the catalog.
For running the misc/export_records.pl script from the command line, then outputting to the terminal instead of saving to the file, this may be a file permission error (well, that's what my searching says).
In my local development environment testing, it was saving the file to where I ran the command line from. So maybe changing the path of the file to somewhere where the user you are running the command with has permission may work.
I'm not a system admin, so hopefully more experienced Koha system administrators can share what they would try to troubleshoot the issues you are having.
David Nind New Zealand
Update, I was able to export a .mrc file using the koha-shell. I needed to change the directory I was in when I entered the shell. In case someone else tries this method, I did: - sudo koha-shell instanceName - cd ~ - this will get back to koha-shell's home directory - /usr/share/koha/bin/export_records.pl --format marc --record-type bibs --filename my-file-name.mrc - After several minutes the export completed. - exit - ls /var/lib/koha/instanceName/ - the file was found in this directory The front end Export tool is still failing, but I'll see if I can use the terminal export for a work around. -Evan On Fri, Jul 19, 2024 at 4:27 PM Evan Stewart <evanstewart406@gmail.com> wrote:
All,
I checked the Apache config settings. I have:
Timeout 300 (seconds) KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 5 (seconds)
-Evan
On Fri, Jul 19, 2024 at 4:18 PM Evan Stewart <evanstewart406@gmail.com> wrote:
HI David,
Thank you. I'll look into the Apache settings as well. That reminds me of a similar issue I had with a PHP application and the problem was PHP execution time settings.
Also, I located an error message in the log. Looks like I'm getting a permission denied error for exports.pl.
*[cgid:error] [pid 554389:tid 554389] (13)Permission denied: [client 172.56.200.129:42349 <http://172.56.200.129:42349>] AH01257: stderr from /usr/share/koha/intranet/cgi-bin/tools/export.pl <http://export.pl>: unable to connect to cgi daemon after multiple tries, referer: http://Ipaddress:8081/cgi-bin/koha/tools/inventory.pl <http://Ipaddress:8081/cgi-bin/koha/tools/inventory.pl>*
When I checked the permission for the file, they are the same as other files in the "... /cgi-bin/tools" directory. The file inventory.pl permissions are the same as export.pl. The inventory page works on the front end.
-Evan
On Fri, Jul 19, 2024 at 3:50 PM David Nind <david@davidnind.com> wrote:
Hi Evan.
It's a long time since I've dealt with Apache server errors and the 503 error you are getting, but here are some things to look at:
- As you have tried with plenty of server memory, it's not likely to be that. It may be something to do with how Apache is configured - including settings like the Timeout, KeepAlive, MaxKeepAliveRequests, KeepAliveTimeout settings.
- I would look at the Apache server logs to see if there is something more in there about the cause of the issue.
- If your server is behind a firewall, proxy servers, or caches (something I don't know much about) then that may be contributing as well.
It may also be a Koha bug, but 63,000 items is not that large, and I couldn't find a bug about performance issues when exporting the catalog.
For running the misc/export_records.pl script from the command line, then outputting to the terminal instead of saving to the file, this may be a file permission error (well, that's what my searching says).
In my local development environment testing, it was saving the file to where I ran the command line from. So maybe changing the path of the file to somewhere where the user you are running the command with has permission may work.
I'm not a system admin, so hopefully more experienced Koha system administrators can share what they would try to troubleshoot the issues you are having.
David Nind New Zealand
Hi Evan. Glad you managed to get the export working with the command line script! Hopefully, others will help troubleshoot why the export tool in the staff interface is not working - whether Apache config settings need changing, if it is something else, or if it is a bug that needs fixing. David Nind On Sat, 20 Jul 2024 at 10:46, Evan Stewart <evanstewart406@gmail.com> wrote:
Update,
I was able to export a .mrc file using the koha-shell. I needed to change the directory I was in when I entered the shell. In case someone else tries this method, I did:
- sudo koha-shell instanceName - cd ~ - this will get back to koha-shell's home directory - /usr/share/koha/bin/export_records.pl --format marc --record-type bibs --filename my-file-name.mrc - After several minutes the export completed. - exit - ls /var/lib/koha/instanceName/ - the file was found in this directory
The front end Export tool is still failing, but I'll see if I can use the terminal export for a work around.
-Evan
On Fri, Jul 19, 2024 at 4:27 PM Evan Stewart <evanstewart406@gmail.com> wrote:
All,
I checked the Apache config settings. I have:
Timeout 300 (seconds) KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 5 (seconds)
-Evan
On Fri, Jul 19, 2024 at 4:18 PM Evan Stewart <evanstewart406@gmail.com> wrote:
HI David,
Thank you. I'll look into the Apache settings as well. That reminds me of a similar issue I had with a PHP application and the problem was PHP execution time settings.
Also, I located an error message in the log. Looks like I'm getting a permission denied error for exports.pl.
*[cgid:error] [pid 554389:tid 554389] (13)Permission denied: [client 172.56.200.129:42349 <http://172.56.200.129:42349>] AH01257: stderr from /usr/share/koha/intranet/cgi-bin/tools/export.pl <http://export.pl>: unable to connect to cgi daemon after multiple tries, referer: http://Ipaddress:8081/cgi-bin/koha/tools/inventory.pl <http://Ipaddress:8081/cgi-bin/koha/tools/inventory.pl>*
When I checked the permission for the file, they are the same as other files in the "... /cgi-bin/tools" directory. The file inventory.pl permissions are the same as export.pl. The inventory page works on the front end.
-Evan
On Fri, Jul 19, 2024 at 3:50 PM David Nind <david@davidnind.com> wrote:
Hi Evan.
It's a long time since I've dealt with Apache server errors and the 503 error you are getting, but here are some things to look at:
- As you have tried with plenty of server memory, it's not likely to be that. It may be something to do with how Apache is configured - including settings like the Timeout, KeepAlive, MaxKeepAliveRequests, KeepAliveTimeout settings.
- I would look at the Apache server logs to see if there is something more in there about the cause of the issue.
- If your server is behind a firewall, proxy servers, or caches (something I don't know much about) then that may be contributing as well.
It may also be a Koha bug, but 63,000 items is not that large, and I couldn't find a bug about performance issues when exporting the catalog.
For running the misc/export_records.pl script from the command line, then outputting to the terminal instead of saving to the file, this may be a file permission error (well, that's what my searching says).
In my local development environment testing, it was saving the file to where I ran the command line from. So maybe changing the path of the file to somewhere where the user you are running the command with has permission may work.
I'm not a system admin, so hopefully more experienced Koha system administrators can share what they would try to troubleshoot the issues you are having.
David Nind New Zealand
One thing to check is the staffClientBaseURL system preference (and also OPACBaseURL). (Clutching at straws here - if this doesn't match your actual server name, then it can cause issues for some things. I don't know if this affects the catalog export.)
Hi David (and any others who might have this problem), I stumbled across the answer (prayer works!). I checked to see if the "cgi" Apache module was installed and enabled. It wasn't! The module "cgid" was enabled, but not "cgi". Apparently, the export.pl script requires regular "cgi". To fix, I logged into the server terminal and ran the commands: - sudo a2enmod cgi - the output indicated that it was not already enabled. - sudo systemctl restart apache2 The odd thing is that I have the cgi module enable step in my server build documentation. I'm not sure why it didn't install then. I noticed that cgid was enabled (so maybe it took precedence when I originally attempted to enable "cgi"). Enabling the "cgi" Apache module fixed the problem. -Evan On Fri, Jul 19, 2024 at 5:17 PM David Nind <david@davidnind.com> wrote:
One thing to check is the staffClientBaseURL system preference (and also OPACBaseURL).
(Clutching at straws here - if this doesn't match your actual server name, then it can cause issues for some things. I don't know if this affects the catalog export.)
participants (2)
-
David Nind -
Evan Stewart