First off: we're on Ubuntu 12.04, using Debian packages, now running koha 3.10. This is my first experience with koha, and from what I can tell I must have done some things during the original setup as root instead of as the koha user. It's far enough in the past now that I'm not sure exactly what. In spite of this, search has been fully working until this week, with the caveat that I would have to manually run the rebuild_zebra.pl script. This had always involved using sudo to rebuild the indexes (one part of why I think I must have done something as root). That was all fine when the only materials that change are bulk imports (one import per item type) from our old system, done by me, but it's not going to fly long term, when my librarians need to be able to add records on their own, so this week I dug into why index updates were not happening automatically, and I found a simple permissions error. I updated the zebra export folders to give the koha user write access to the zebra folders, and now nothing shows up in search. When I watch the re-index process, I see a few additional folders that cause permissions issues under /var/lock. I can update permissions on these as well, but they are removed and re-created with the bad permissions every time the server reboots. Rare enough, but still a problem. *How can I fix this so that the lock folders get the correct permissions when they are recreated after reboot? Is this something I'll need to script? * Also watching the re-index process I see we have four cases where marc records that are too large for the spec that somehow made it into the system. I found and removed two of those, but I'm having a hard time finding the other two. I only know they're somewhere near the end of our biblios (should have larger biblio numbers), because they're up past 92000 out of 99769 during the import, but I can't see where exactly to pin them down. This brings up two questions: *How I can find the over-sized marc records?* and *How can I get koha to use handle these large items? *I know it's possible, because until this week I was able to search on those records. With just the two bad records, I can then watch zebra index those first 92000 or so records. After the exporting biblios phase, during the reindex phase, with extra verbosity set I do see this error: about bad marc records. I don't have exact message handy now, but I remember it was explicit about skipping everything after the error event. Everything else looks to have committed. In spite of this, all searches still come up empty. I tried re-indexing with the -x switch, in an attempt to use the xml format to get around the marc 99999 byte limit. This completes with no errors. However, I still don't see any search results. I know the zebra server is running. Joel Coehoorn Director of Information Technology York College, Nebraska 402.363.5603 jcoehoorn@york.edu *The mission of York College is to transform lives through Christ-centered education and to equip students for lifelong service to God, family, and society*
Op 23/07/13 06:22, Coehoorn, Joel schreef:
In spite of this, search has been fully working until this week, with the caveat that I would have to manually run the rebuild_zebra.pl script. This had always involved using sudo to rebuild the indexes (one part of why I think I must have done something as root).
With the packages, you don't need to touch any of this. It does it all for you and it manages the permissions appropriately. By using the "internal" commands directly, it is the case that, as you discovered, you broke things and the package driven functions fail now.
When I watch the re-index process, I see a few additional folders that cause permissions issues under /var/lock. I can update permissions on these as well, but they are removed and re-created with the bad permissions every time the server reboots. Rare enough, but still a problem.
What are you considering good permissions and bad permissions? Can you provide some actual examples? Also examples of the errors and what the current states are of the files it's complaining about, along with the exact commands you are running, including the command prompt. What you should be seeing is things like this (where the library name here is 'catalyst'): $ ls -ld /var/lock/koha/catalyst drwxr-xr-x 4 catalyst-koha catalyst-koha 4096 Sep 13 2012 /var/lock/koha/catalyst $ ls -lR /var/lock/koha/catalyst /var/lock/koha/catalyst: total 8 drwxr-xr-x 2 catalyst-koha catalyst-koha 4096 Feb 27 19:55 authorities drwxr-xr-x 2 catalyst-koha catalyst-koha 4096 Feb 27 19:55 biblios /var/lock/koha/catalyst/authorities: total 4 -rw-r--r-- 1 catalyst-koha catalyst-koha 5 Jul 18 10:22 zebrasrv.pid /var/lock/koha/catalyst/biblios: total 4 -rw-r--r-- 1 catalyst-koha catalyst-koha 5 Jul 18 10:22 zebrasrv.pid and similar for the stuff in /var/lib/koha. It must all be read/write libraryname-koha. You should never use rebuild-zebra. Instead you should use /usr/sbin/koha-rebuild-zebra.
*How can I fix this so that the lock folders get the correct permissions when they are recreated after reboot? Is this something I'll need to script? *
They are recreated after reboot with the correct permissions. This happens by running /usr/sbin/koha-create-dirs $( koha-list ) during boot.
*How can I get koha to use handle these large items? *I know it's possible, because until this week I was able to search on those records.
Stop doing stuff with rebuild-zebra.pl directly, and it'll just work. More specifically, the packages do XML indexing by default, and this is how these are handled.
With just the two bad records, I can then watch zebra index those first 92000 or so records. After the exporting biblios phase, during the reindex phase, with extra verbosity set I do see this error:
There's no error there. But try it with koha-rebuild-zebra, it might work better.
I tried re-indexing with the -x switch, in an attempt to use the xml format to get around the marc 99999 byte limit. This completes with no errors. However, I still don't see any search results. I know the zebra server is running.
Did you check the error logs (in /var/log/koha/libraryname) for zebra? -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D
Okay, I can /usr/sbin/koha_rebuild_zebra to run if I use the -f switch. This makes sense: whatever I did that broke this originally is preventing that from seeing incremental changes. However, this breaks my search again, and there are no results. To get search results back, I have to run the rebuild_zebra.pl script (I know, I know) with sudo. What could be going on here? Joel Coehoorn Director of Information Technology York College, Nebraska 402.363.5603 jcoehoorn@york.edu *The mission of York College is to transform lives through Christ-centered education and to equip students for lifelong service to God, family, and society* On Wed, Jul 24, 2013 at 7:25 AM, Robin Sheat <robin@catalyst.net.nz> wrote:
Op 23/07/13 06:22, Coehoorn, Joel schreef:
In spite of this, search has been fully working until this week, with the caveat that I would have to manually run the rebuild_zebra.pl script. This had always involved using sudo to rebuild the indexes (one part of why I think I must have done something as root).
With the packages, you don't need to touch any of this. It does it all for you and it manages the permissions appropriately. By using the "internal" commands directly, it is the case that, as you discovered, you broke things and the package driven functions fail now.
When I watch the re-index process, I see a few additional folders that cause permissions issues under /var/lock. I can update permissions on these as well, but they are removed and re-created with the bad permissions every time the server reboots. Rare enough, but still a problem.
What are you considering good permissions and bad permissions? Can you provide some actual examples? Also examples of the errors and what the current states are of the files it's complaining about, along with the exact commands you are running, including the command prompt.
What you should be seeing is things like this (where the library name here is 'catalyst'):
$ ls -ld /var/lock/koha/catalyst drwxr-xr-x 4 catalyst-koha catalyst-koha 4096 Sep 13 2012 /var/lock/koha/catalyst $ ls -lR /var/lock/koha/catalyst /var/lock/koha/catalyst: total 8 drwxr-xr-x 2 catalyst-koha catalyst-koha 4096 Feb 27 19:55 authorities drwxr-xr-x 2 catalyst-koha catalyst-koha 4096 Feb 27 19:55 biblios
/var/lock/koha/catalyst/authorities: total 4 -rw-r--r-- 1 catalyst-koha catalyst-koha 5 Jul 18 10:22 zebrasrv.pid
/var/lock/koha/catalyst/biblios: total 4 -rw-r--r-- 1 catalyst-koha catalyst-koha 5 Jul 18 10:22 zebrasrv.pid
and similar for the stuff in /var/lib/koha. It must all be read/write libraryname-koha. You should never use rebuild-zebra. Instead you should use /usr/sbin/koha-rebuild-zebra.
*How can I fix this so that the lock folders get the correct permissions when they are recreated after reboot? Is this something I'll need to script? *
They are recreated after reboot with the correct permissions. This happens by running /usr/sbin/koha-create-dirs $( koha-list ) during boot.
*How can I get koha to use handle these large items? *I know it's possible, because until this week I was able to search on those records.
Stop doing stuff with rebuild-zebra.pl directly, and it'll just work. More specifically, the packages do XML indexing by default, and this is how these are handled.
With just the two bad records, I can then watch zebra index those first 92000 or so records. After the exporting biblios phase, during the reindex phase, with extra verbosity set I do see this error:
There's no error there. But try it with koha-rebuild-zebra, it might work better.
I tried re-indexing with the -x switch, in an attempt to use the xml format to get around the marc 99999 byte limit. This completes with no errors. However, I still don't see any search results. I know the zebra server is running.
Did you check the error logs (in /var/log/koha/libraryname) for zebra?
-- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
Op 24/07/13 19:59, Coehoorn, Joel schreef:
Okay, I can /usr/sbin/koha_rebuild_zebra to run if I use the -f switch. This makes sense: whatever I did that broke this originally is preventing that from seeing incremental changes.
Yes, you'll need -f. I also recommend -v -v to get a bit more noise about what's happening. Incremental changes should be updated from a cron job that the packages install.
However, this breaks my search again, and there are no results. To get search results back, I have to run the rebuild_zebra.pl <http://rebuild_zebra.pl> script (I know, I know) with sudo. What could be going on here?
By doing a rebuild_zebra with sudo, you've gone and broken your permissions again, stopping incremental updating from working. So here's what I'd do: * shut down anything to do with zebrasrv. 'sudo service koha-common stop' and some checking with ps to make sure there's nothing still running. If there is, figure out where it's coming from and kill it. * delete /var/lock/koha * delete /var/run/koha * delete /var/lib/koha * run 'sudo koha-create-dirs library' * rebuild everything: 'sudo koha-rebuild-zebra -f -v -v library' * start up zebrasrv: 'sudo service koha-common start' I'm not sure why rebuilding with koha-rebuild-zebra would have stopped search from working. If that happens again, I'd suggest diagnosing it properly by running zebrasrv from the command line so you get to see all the information about what it's doing. Also note that as a result of deleting /var/lib/koha, you'll need to run 'sudo koha-email-enable library' to have email turned back on if you use that. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D
Hi Robin, I have copied your instructions to http://wiki.koha-community.org/wiki/Fix_zebra_permissions (Everybody, not Robin in particular:) Please feel free to add/edit things. -- Mirko
I had some other things to do this morning, but I can confirm now that this has worked for us. Thanks so much! Joel Coehoorn Director of Information Technology York College, Nebraska 402.363.5603 jcoehoorn@york.edu *The mission of York College is to transform lives through Christ-centered education and to equip students for lifelong service to God, family, and society* On Thu, Jul 25, 2013 at 6:01 AM, Robin Sheat <robin@catalyst.net.nz> wrote:
Op 24/07/13 19:59, Coehoorn, Joel schreef:
Okay, I can /usr/sbin/koha_rebuild_zebra to run if I use the -f switch. This makes sense: whatever I did that broke this originally is preventing that from seeing incremental changes.
Yes, you'll need -f. I also recommend -v -v to get a bit more noise about what's happening.
Incremental changes should be updated from a cron job that the packages install.
However, this breaks my search again, and there are no results. To get search results back, I have to run the rebuild_zebra.pl <http://rebuild_zebra.pl> script (I know, I know) with sudo. What could be going on here?
By doing a rebuild_zebra with sudo, you've gone and broken your permissions again, stopping incremental updating from working. So here's what I'd do:
* shut down anything to do with zebrasrv. 'sudo service koha-common stop' and some checking with ps to make sure there's nothing still running. If there is, figure out where it's coming from and kill it. * delete /var/lock/koha * delete /var/run/koha * delete /var/lib/koha * run 'sudo koha-create-dirs library' * rebuild everything: 'sudo koha-rebuild-zebra -f -v -v library' * start up zebrasrv: 'sudo service koha-common start'
I'm not sure why rebuilding with koha-rebuild-zebra would have stopped search from working. If that happens again, I'd suggest diagnosing it properly by running zebrasrv from the command line so you get to see all the information about what it's doing.
Also note that as a result of deleting /var/lib/koha, you'll need to run 'sudo koha-email-enable library' to have email turned back on if you use that.
-- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D
_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha
participants (3)
-
Coehoorn, Joel -
Mirko -
Robin Sheat