Hello Thanks for your help, Bernardo. I couldn't find a koha-zebrademon.err file on the server, does it always have that name? I have limited access to the server (and limited knowledge of linux :)), it could explain. I will ask the tech which logs he is looking at. Zebra server stops working anytime, not when we run rebuild-zebra. Sometimes I run rebuild-zebra and it stops three days after, sometimes it stops the same day. (and sorry for the hijack, I didn't know...) Cheers, Sonia. Date: Fri, 22 Jun 2012 07:48:07 -0300 Subject: Re: [Koha] Rebuild-zebra stops working very often... From: bgkriegel@gmail.com To: sossolapro@hotmail.com CC: koha@lists.katipo.co.nz Sonia, your zebra server stops when you do a rebuild or at any other moment? what logs are your responsible person looking at? Normally zebra will report any errors in /usr/share/koha/koha-zebrademon.err B. -- Bernardo Gonzalez Kriegelbgkriegel@gmail.com On Fri, Jun 22, 2012 at 3:53 AM, Sonia P. <sossolapro@hotmail.com> wrote: Dear all I have got some problems with rebuild-zebra which stops working very often, as a result our librarians can't find the newly entered items (the search indexes are not up to date). When I manually restart zebra, it solves the problem. But then it stops again running the next day, sooner or later. We just don't know why it stops working. The person responsible for the server said that << The logs suggested that the re-indexing was being run according to the schedule >> Do you have any idea why it would stop working? What should we check? Where could it come from? Many thanks for your help! Sonia. _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

Ahh that is not the zebra error log. But the rebuild zebra one. I think you may be confusing zebra with the job to rebuild the zebra indexes, they are 2 quite different things. However not being able to talk to the database is a fatal error for the rebuild zebra script. The database failing will also cause all sorts of other errors. I would get your tech people to look at your database set up. Something very bad is going wrong there. Chris "Sonia P." <sossolapro@hotmail.com> wrote: Dear all Thanks again for your help. Here is (below) the content of the zebra error file. Would you please have a look at it and tell me if there are things which could explain why zebra-rebuild stops working now and then? There is that DBIconnect error, but I am not sure it's related. And there is no time & day in the error file, so I can't check what happened when zebra rebuild doesn't work properly. (I have removed all the multiple duplicates...) Cheers, Sonia. Wide character in subroutine entry at /usr/share/perl5/MARC/Charset/Table.pm line 96. DBI connect('dbname=koha_library;host=localhost;port=3306','koha_library',...) failed: Can't connect to loc al MySQL server through socket '/var/run/mysqld/mysqld.sock' (111) at /usr/share/koha/lib/C4/Context.pm lin e 692 Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111) at /usr/share/koha/l ib/C4/Context.pm line 692. DBI connect('dbname=koha_library;host=localhost;port=3306','koha_library',...) failed: Can't connect to loc al MySQL server through socket '/var/run/mysqld/mysqld.sock' (111) at /usr/share/koha/lib/C4/Context.pm lin e 692 Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (111) at /usr/share/koha/l ib/C4/Context.pm line 692. DBI connect('dbname=koha_library;host=localhost;port=3306','koha_library',...) failed: Can't connect to loc al MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/koha/lib/C4/Context.pm line 692 Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2) at /usr/share/koha/lib /C4/Context.pm line 692. error retrieving biblio 12126 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 473. error retrieving biblio 11493 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 473. error retrieving biblio 13829 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 473. $ Wide character in subroutine entry at /usr/share/perl5/MARC/Charset/Table.pm line 96. $ DBI connect('dbname=koha_library;host=localhost;port=3306','koha_library',...) failed: Can't connect to local MySQL server through socket $ '/var/run/mysqld/mysqld.sock' (111) at /usr/share/koha/lib/C4/Context.pm line 692 error retrieving biblio 14703 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 473. $ Wide character in subroutine entry at /usr/share/perl5/MARC/Charset/Table.pm line 96. $ DBI connect('dbname=koha_library;host=localhost;port=3306','koha_library',...) failed: Can't connect to local MySQL server through socket $ '/var/run/mysqld/mysqld.sock' (111) at /usr/share/koha/lib/C4/Context.pm line 692 $ error retrieving biblio 14703 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 473. $ Wide character in subroutine entry at /usr/share/perl5/MARC/Charset/Table.pm line 96. $ DBI connect('dbname=koha_library;host=localhost;port=3306','koha_library',...) failed: Can't connect to local MySQL server through socket $ '/var/run/mysqld/mysqld.sock' (111) at /usr/share/koha/lib/C4/Context.pm line 692 $ error retrieving biblio 14703 at /usr/share/koha/bin/migration_tools/rebuild_zebra.pl line 473. _____________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

Op 25-06-12 02:44, Sonia P. schreef:

So I don't know what to do... We are stuck. I can't restart zebra every two hours.

When you say that you are restarting zebra, what _exactly_ are you doing? Because, as Chris says, there seems to be some confusion: restarting zebra should not affect whether rebuild-zebra is doing its job or not. Also, rebuild-zebra fires up every 15 minutes or so do do things, does them, and then stops. There is something in your explanation that doesn't make sense that needs to be figured out before any useful suggestions can be made.

Would reinstalling everything help? Wouldn't it take two weeks to set up and fix everything?

No idea, it would depend on what the problem was. In theory, if you follow exactly the same steps that you did last time, you will have the same problems. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D

Op 25-06-12 14:18, Sonia P. schreef:

Rebuild-zebra is supposed to run every 15 minutes for us, and sometimes it doesn't. I know, because then I am contacted by librarians who say they can't find books which they have just entered (or I check myself, I run a few mysql queries and check if I can find them on the website). So I know it's because the search indexes are not up to date. Then, I restart rebuild-zebra with the following: sudo koha-rebuild-zebra -v -f library

OK, just to clarify, that command doesn't restart anything. It just forces everything to be indexed from scratch, that's where the confusion was coming in. You should be getting sent emails if something is going wrong with the regular index procedure, but if local mail hasn't been properly configured on your server, you might not be seeing them. It would be worth putting an alias in for the root user so that it goes to a real person. To do this: 1. edit /etc/aliases 2. add a line like: "root: your@email.address", save 3. run the 'newaliases' command This'll mean that you get sent any errors when zebra rebuilds. To see what's there now, run 'sudo mail'. Type the number of the message to read it, 'q' will exit and '?' will provide a small amount of help. If there's something there, maybe it'll give a clue. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D

Hello I just realised that I have never thank you all for your help. We are still working on it. We still have that same problem... The nozebra parameter is set on "use". We have updated Koha, and it hasn't changed anything. When we run zebra-rebuild, we can see some very minor errors (with records which have been deleted long time ago obviously) but I don't think there is anything in there which would explain that thing. Cheers, Sonia.

Date: Mon, 25 Jun 2012 16:07:35 +0200 From: veron@veron.ch To: koha@lists.katipo.co.nz Subject: Re: [Koha] Rebuild-zebra stops working very often...

Sonja,

I just had similar symptoms - until I remembered that I had set preference NoZebra to "Don't use" for some testing purposes. It always should be set tu "Use".

It's a little chance, but maybe you might want to check this preference.

Marc

Am 25.06.2012 15:18, schrieb Sonia P.:

...

OK, thanks all for your help! I am sorry I don't know all the details and I am obviously not great with Koha.

Rebuild-zebra is supposed to run every 15 minutes for us, and sometimes it doesn't. I know, because then I am contacted by librarians who say they can't find books which they have just entered (or I check myself, I run a few mysql queries and check if I can find them on the website). So I know it's because the search indexes are not up to date. Then, I restart rebuild-zebra with the following: sudo koha-rebuild-zebra -v -f library And then the problem is solved, but not forever. It stops again later, the same day or another day, sometimes one week later. If I had more time to work on Koha, I could check exactly how often it happens and things like that, but I don't have time. :(

In << sudo tail /var/log/koha/ashs/zebra-error.log.1 >> is ashs the name of the library? On our server, the closest thing I could find is << sudo tail /var/log/koha/library/zebra-error.log >> and it doesn't give me any information. So, if I understand well "tail", it means that the file is empty. Same thing for zebra-output. When I look at the current processes (ps -aux), I can see that there is a zebrasrv process runing (it says << Mar26 2:02 zebrasrv -v none,fatal,warn -f /etc/koha/sites/library/koha-conf >>, March 26 being the day when we restarted the whole server). So all that probably means that there is no problem with Zebra, am I right?

With the errors I have reported previously (DBIconnect...), what is it that can't connect to MySQL? that perl script you describe, Chris? I understand it's not Zebra itself, is it rebuild-zebra?

Thanks a lot for all your explanations...

Cheers,

Sonia.

...

Date: Mon, 25 Jun 2012 10:20:57 +0100 From: robin@catalyst.net.nz To: koha@lists.katipo.co.nz Subject: Re: [Koha] Rebuild-zebra stops working very often...

Op 25-06-12 02:44, Sonia P. schreef:

...

So I don't know what to do... We are stuck. I can't restart zebra every two hours.

When you say that you are restarting zebra, what _exactly_ are you doing? Because, as Chris says, there seems to be some confusion: restarting zebra should not affect whether rebuild-zebra is doing its job or not. Also, rebuild-zebra fires up every 15 minutes or so do do things, does them, and then stops. There is something in your explanation that doesn't make sense that needs to be figured out before any useful suggestions can be made.

...

Would reinstalling everything help? Wouldn't it take two weeks to set up and fix everything?

No idea, it would depend on what the problem was. In theory, if you follow exactly the same steps that you did last time, you will have the same problems.

-- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D _______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

Thanks a lot, Robin. We will try that script and I will get back to you with the results. :)

* there's something else funny going on there, even if things weren't able to go into zebra, that shouldn't cause zebra to stop until you do a reindex. That's not something I've seen it do ever.

Well, I wouldn't mind having a very weird unexplainable problem (that would make me feel better for not having solved it in months...), but it's very likely that it will be something really silly at the end and I will feel even more miserable. :) As we have just installed the new version of Koha, and the problem is still there, I guess it means that the problem comes from the data itself not from our old Koha installation (?). I remember when we reinstalled Koha, the guy responsible for the installation, the server etc, told me that there were inconsistancies, data which have been probably erased from the database not properly. He asked me to get rid of these records in Koha, I mean using the Koha interface. But I couldn't find these records, so I told him, and then we moved to something else. So maybe that's really coming from there? Cheers, Sonia.

Op 20-07-12 09:29, Sonia P. schreef:

<< Can't locate C4/Context.pm in @INC (@INC contains: /etc/perl /usr/local/lib/perl/5.10.1 /usr/local/share/perl/5.10.1 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl.) at batchCheckNonIndexedBiblios.pl line 36. >>

In the script line 36 is << use C4::Context; >>

We have put the script here: /usr/share/koha/bin/migration_tools/batchCheckNonIndexedBiblios.pl

Does that error mean that things are not located where it should be?

You didn't show what command line you're using to run it, that's the most critical thing. However the problem is simply that you're not providing the path to where the C4 directory is. I don't know your system layout, but I'm going to guess: perl -I /usr/share/koha/lib /usr/share/koha/bin/migration_tools/batchCheckNonIndexedBiblios.pl (all one line) Also, you will have to put KOHA_CONF=/path/to/koha-conf.xml at the start of it, inserting the real path. -- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D

OK, after permissions have been changed... Here is the report (it looks long but it's quite empty so I copy-paste) : 1000 done 2000 done biblionumber 2315 not indexed 3000 done 4000 done biblionumber 5230 not indexed biblionumber 5234 not indexed 5000 done 6000 done 7000 done 8000 done 9000 done 10000 done 11000 done 12000 done 13000 done 14000 done 15000 done 16000 done 17000 done biblionumber 19992 not indexed biblionumber 19994 not indexed biblionumber 20030 not indexed 18000 done biblionumber 20177 not indexed biblionumber 20234 not indexed biblionumber 20316 not indexed biblionumber 20317 not indexed biblionumber 20330 not indexed biblionumber 20333 not indexed biblionumber 20384 not indexed biblionumber 20397 not indexed biblionumber 20398 not indexed biblionumber 20399 not indexed biblionumber 20400 not indexed biblionumber 20418 not indexed biblionumber 20446 not indexed biblionumber 20448 not indexed biblionumber 20491 not indexed biblionumber 20494 not indexed biblionumber 20495 not indexed biblionumber 20499 not indexed biblionumber 20500 not indexed biblionumber 20501 not indexed biblionumber 20502 not indexed biblionumber 20505 not indexed biblionumber 20590 not indexed biblionumber 20614 not indexed biblionumber 20615 not indexed biblionumber 20616 not indexed biblionumber 20617 not indexed biblionumber 20618 not indexed biblionumber 20626 not indexed biblionumber 20627 not indexed biblionumber 20628 not indexed biblionumber 20648 not indexed biblionumber 20677 not indexed biblionumber 20686 not indexed biblionumber 20849 not indexed biblionumber 20917 not indexed biblionumber 20921 not indexed biblionumber 21016 not indexed biblionumber 21039 not indexed biblionumber 21045 not indexed biblionumber 21097 not indexed biblionumber 21138 not indexed biblionumber 21141 not indexed biblionumber 21142 not indexed biblionumber 21145 not indexed 19000 done biblionumber 21810 not indexed biblionumber 21811 not indexed 20000 done biblionumber 22320 not indexed biblionumber 22338 not indexed biblionumber 22386 not indexed biblionumber 22583 not indexed 21000 done biblionumber 23506 not indexed biblionumber 23507 not indexed biblionumber 23508 not indexed 22000 done 23000 done 24000 done 25000 done 26000 done 27000 done 28000 done 29000 done 30000 done 31000 done 32000 done biblionumber 34273 not indexed biblionumber 34274 not indexed biblionumber 34275 not indexed biblionumber 34276 not indexed biblionumber 34277 not indexed biblionumber 34278 not indexed biblionumber 34279 not indexed biblionumber 34280 not indexed biblionumber 34281 not indexed biblionumber 34282 not indexed biblionumber 34283 not indexed biblionumber 34284 not indexed biblionumber 34285 not indexed biblionumber 34286 not indexed biblionumber 34287 not indexed biblionumber 34288 not indexed biblionumber 34289 not indexed biblionumber 34290 not indexed biblionumber 34291 not indexed biblionumber 34292 not indexed biblionumber 34293 not indexed biblionumber 34294 not indexed biblionumber 34295 not indexed biblionumber 34296 not indexed biblionumber 34297 not indexed biblionumber 34298 not indexed biblionumber 34299 not indexed biblionumber 34300 not indexed biblionumber 34301 not indexed biblionumber 34302 not indexed biblionumber 34303 not indexed biblionumber 34304 not indexed biblionumber 34305 not indexed biblionumber 34306 not indexed biblionumber 34307 not indexed biblionumber 34308 not indexed biblionumber 34309 not indexed biblionumber 34310 not indexed biblionumber 34311 not indexed biblionumber 34312 not indexed biblionumber 34313 not indexed biblionumber 34314 not indexed biblionumber 34315 not indexed biblionumber 34316 not indexed biblionumber 34317 not indexed biblionumber 34318 not indexed biblionumber 34319 not indexed biblionumber 34320 not indexed biblionumber 34321 not indexed biblionumber 34322 not indexed biblionumber 34323 not indexed biblionumber 34324 not indexed biblionumber 34325 not indexed biblionumber 34326 not indexed biblionumber 34327 not indexed biblionumber 34328 not indexed biblionumber 34329 not indexed biblionumber 34330 not indexed biblionumber 34331 not indexed biblionumber 34332 not indexed biblionumber 34333 not indexed biblionumber 34334 not indexed biblionumber 34335 not indexed biblionumber 34336 not indexed biblionumber 34337 not indexed biblionumber 34338 not indexed biblionumber 34339 not indexed biblionumber 34340 not indexed biblionumber 34341 not indexed biblionumber 34342 not indexed biblionumber 34343 not indexed biblionumber 34344 not indexed biblionumber 34345 not indexed biblionumber 34346 not indexed biblionumber 34347 not indexed biblionumber 34348 not indexed biblionumber 34349 not indexed biblionumber 34350 not indexed 138 bibliorecords not indexed I am afraid I don't know what to do from now on... The last bunch of records, from 34273, are the newly entered (these last days). What should I do? Thanks! Sonia.

Hello It seems that we can't use 'delete' commands in the Koha admin website (build SQL reports or so). How can I do that? I need to delete these records, I know the proper command, I just want to be allowed to do it. (I am now down to 20 faulty records, so I will just delete them from the database one by one, I know they are only present in the 'biblio' table) Cheers, Sonia.

From: sossolapro@hotmail.com To: koha@lists.katipo.co.nz Date: Sun, 22 Jul 2012 00:00:42 +0200 Subject: [Koha] SQL queries to delete records from the database?

Hi there

(re: rebuild-zebra stops working very often, which is not a proper title any more... :p)

I tried to edit a few records from these non-indexed records. As we speak another language, we have accent issues. So I need to make sure (again) that our librarians doesn't use any accent on the letters. So all the records which I have found from that list of non-indexed records have accent problems.

On some records, I can edit the content and remove the accents.

But for some records, I can't even edit the content, because then there is an error with Koha, the page doesn't show anything with something like that: << Software error: Can't call method "field" on an undefined value at /usr/share/koha/lib/C4/Items.pm line 1421.>> which is just a random error because it can't find the record anymore (it finds the records on certain pages, for instance if I want to see its history, but it can't find the record on any page where it should show its title). So no way to solve the problem using the Koha interface...

I guess I need a sql query to delete these records from the database, but something that would really delete everything related to that item (history, etc). Do you have any idea about that?

But then I just don't understand why these problem records prevent zebra from reindexing the newly entered records... :/

Anyway, I have done the koha-rebuild-zebra -v -f library command to reindex everything. See below the new report from that new perl script, we moved from 138 non-indexed records to 56 non-indexed records, just because I have modified a few records (5?) and because I have rerun rebuild-zebra (so it indexed the newly entered records which had no accent problems anyway).

Many thanks for your help! Would greatly appreciate your expertise about SQL queries to delete everything for faulty records.

Sonia.

############

1000 done 2000 done biblionumber 2315 not indexed 3000 done 4000 done biblionumber 5230 not indexed biblionumber 5234 not indexed 5000 done 6000 done 7000 done 8000 done 9000 done 10000 done 11000 done 12000 done 13000 done 14000 done 15000 done 16000 done 17000 done biblionumber 19992 not indexed biblionumber 20030 not indexed 18000 done biblionumber 20177 not indexed biblionumber 20234 not indexed biblionumber 20316 not indexed biblionumber 20317 not indexed biblionumber 20397 not indexed biblionumber 20398 not indexed biblionumber 20399 not indexed biblionumber 20400 not indexed biblionumber 20418 not indexed biblionumber 20446 not indexed biblionumber 20448 not indexed biblionumber 20491 not indexed biblionumber 20494 not indexed biblionumber 20495 not indexed biblionumber 20499 not indexed biblionumber 20500 not indexed biblionumber 20501 not indexed biblionumber 20502 not indexed biblionumber 20505 not indexed biblionumber 20590 not indexed biblionumber 20614 not indexed biblionumber 20615 not indexed biblionumber 20616 not indexed biblionumber 20617 not indexed biblionumber 20618 not indexed biblionumber 20626 not indexed biblionumber 20627 not indexed biblionumber 20628 not indexed biblionumber 20648 not indexed biblionumber 20677 not indexed biblionumber 20686 not indexed biblionumber 20849 not indexed biblionumber 20917 not indexed biblionumber 20921 not indexed biblionumber 21016 not indexed biblionumber 21039 not indexed biblionumber 21045 not indexed biblionumber 21097 not indexed biblionumber 21138 not indexed biblionumber 21141 not indexed biblionumber 21142 not indexed biblionumber 21145 not indexed 19000 done biblionumber 21810 not indexed biblionumber 21811 not indexed 20000 done biblionumber 22320 not indexed biblionumber 22338 not indexed biblionumber 22386 not indexed biblionumber 22583 not indexed 21000 done biblionumber 23506 not indexed biblionumber 23507 not indexed biblionumber 23508 not indexed 22000 done 23000 done 24000 done 25000 done 26000 done 27000 done 28000 done 29000 done 30000 done 31000 done 32000 done 56 bibliorecords not indexed

_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

So something like that? mysqldump -u kohauser -p kohalibrary > savedatabase230712.sql mysql -u username -p[password]use kohalibrary;DELETE FROM biblio WHERE biblionumber='2315'; (never done mysql things in commandline before...) Cheers, Sonia.

Date: Mon, 23 Jul 2012 15:42:59 +1200 From: chrisc@catalyst.net.nz To: sossolapro@hotmail.com CC: koha@lists.katipo.co.nz Subject: Re: [Koha] SQL queries to delete records from the database?

* Sonia P. (sossolapro@hotmail.com) wrote:

...

Hello

It seems that we can't use 'delete' commands in the Koha admin website (build SQL reports or so). How can I do that? I need to delete these records, I know the proper command, I just want to be allowed to do it.

Yes, that is because its a hugely dangerous thing to allow someone to type a delete into the web interface. Imagine, delete from issues;

For doing these kind of operations, you should do it in the mysql prompt on the commandline. After backing up the database of course.

Chris

...

(I am now down to 20 faulty records, so I will just delete them from the database one by one, I know they are only present in the 'biblio' table)

Cheers,

Sonia.

...

From: sossolapro@hotmail.com To: koha@lists.katipo.co.nz Date: Sun, 22 Jul 2012 00:00:42 +0200 Subject: [Koha] SQL queries to delete records from the database?

Hi there

(re: rebuild-zebra stops working very often, which is not a proper title any more... :p)

I tried to edit a few records from these non-indexed records. As we speak another language, we have accent issues. So I need to make sure (again) that our librarians doesn't use any accent on the letters. So all the records which I have found from that list of non-indexed records have accent problems.

On some records, I can edit the content and remove the accents.

But for some records, I can't even edit the content, because then there is an error with Koha, the page doesn't show anything with something like that: << Software error: Can't call method "field" on an undefined value at /usr/share/koha/lib/C4/Items.pm line 1421.>> which is just a random error because it can't find the record anymore (it finds the records on certain pages, for instance if I want to see its history, but it can't find the record on any page where it should show its title). So no way to solve the problem using the Koha interface...

I guess I need a sql query to delete these records from the database, but something that would really delete everything related to that item (history, etc). Do you have any idea about that?

But then I just don't understand why these problem records prevent zebra from reindexing the newly entered records... :/

Anyway, I have done the koha-rebuild-zebra -v -f library command to reindex everything. See below the new report from that new perl script, we moved from 138 non-indexed records to 56 non-indexed records, just because I have modified a few records (5?) and because I have rerun rebuild-zebra (so it indexed the newly entered records which had no accent problems anyway).

Many thanks for your help! Would greatly appreciate your expertise about SQL queries to delete everything for faulty records.

Sonia.

############

1000 done 2000 done biblionumber 2315 not indexed 3000 done 4000 done biblionumber 5230 not indexed biblionumber 5234 not indexed 5000 done 6000 done 7000 done 8000 done 9000 done 10000 done 11000 done 12000 done 13000 done 14000 done 15000 done 16000 done 17000 done biblionumber 19992 not indexed biblionumber 20030 not indexed 18000 done biblionumber 20177 not indexed biblionumber 20234 not indexed biblionumber 20316 not indexed biblionumber 20317 not indexed biblionumber 20397 not indexed biblionumber 20398 not indexed biblionumber 20399 not indexed biblionumber 20400 not indexed biblionumber 20418 not indexed biblionumber 20446 not indexed biblionumber 20448 not indexed biblionumber 20491 not indexed biblionumber 20494 not indexed biblionumber 20495 not indexed biblionumber 20499 not indexed biblionumber 20500 not indexed biblionumber 20501 not indexed biblionumber 20502 not indexed biblionumber 20505 not indexed biblionumber 20590 not indexed biblionumber 20614 not indexed biblionumber 20615 not indexed biblionumber 20616 not indexed biblionumber 20617 not indexed biblionumber 20618 not indexed biblionumber 20626 not indexed biblionumber 20627 not indexed biblionumber 20628 not indexed biblionumber 20648 not indexed biblionumber 20677 not indexed biblionumber 20686 not indexed biblionumber 20849 not indexed biblionumber 20917 not indexed biblionumber 20921 not indexed biblionumber 21016 not indexed biblionumber 21039 not indexed biblionumber 21045 not indexed biblionumber 21097 not indexed biblionumber 21138 not indexed biblionumber 21141 not indexed biblionumber 21142 not indexed biblionumber 21145 not indexed 19000 done biblionumber 21810 not indexed biblionumber 21811 not indexed 20000 done biblionumber 22320 not indexed biblionumber 22338 not indexed biblionumber 22386 not indexed biblionumber 22583 not indexed 21000 done biblionumber 23506 not indexed biblionumber 23507 not indexed biblionumber 23508 not indexed 22000 done 23000 done 24000 done 25000 done 26000 done 27000 done 28000 done 29000 done 30000 done 31000 done 32000 done 56 bibliorecords not indexed

_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha

-- Chris Cormack Catalyst IT Ltd. +64 4 803 2238 PO Box 11-053, Manners St, Wellington 6142, New Zealand

OK, thanks. Yes, I am 100% sure I want to do this. We have been dealing with that recurrent index problem for months now. I am despaired. I could try anything. :) (and deleting a few lines in the biblio table won't probably crash anything...) I don't know how to restore from a back-up but someone is here to deal with major issues like that, and I could learn. Sonia.

From: chris@bigballofwax.co.nz Date: Mon, 23 Jul 2012 16:00:21 +1200 Subject: Re: [Koha] SQL queries to delete records from the database? To: sossolapro@hotmail.com CC: koha@lists.katipo.co.nz

On 23 July 2012 15:57, Sonia P. <sossolapro@hotmail.com> wrote:

...

So something like that?

mysqldump -u kohauser -p kohalibrary > savedatabase230712.sql

mysql -u username -p[password]use kohalibrary;DELETE FROM biblio WHERE biblionumber='2315'; (never done mysql things in commandline before...) Cheers, Sonia.

Yes

If you are incredibly sure you want do this, and you are sure you know how to restore from a backup, and no one is using or has used Koha since you made that backup.

Chris

Thanks for your help, Marc. We don't use zebra-daemon, as I think it's deprecated (are we wrong?). But I still forward your message to our server guy, maybe it will give him some clues. Cheers, Sonia.

From: mtompset@hotmail.com To: sossolapro@hotmail.com CC: koha@lists.katipo.co.nz Subject: Re: [Koha] SQL queries to delete records from the database? Date: Mon, 23 Jul 2012 13:37:48 +0800

Greetings,

Did I hear someone say 'recurrent index problems'? I think this means I should repaste my indexing journey here. This was on an Ubuntu 10.04 LTS VM with only 512MB of memory. As such, your mileage may vary. And if there is something wrong in here, someone can comment and correct me.

--- BEGIN POST FROM ZEBRA LIST --- Greetings,

I am sharing this in hopes that this helps people who are frustrated trying to figure out why Zebra indexing is not working for them.

When I initially did a standard install, I ran the full reindex as root. This was a mistake! This created a whole bunch of files with root.root as the owner.group in the subdirectories of the /var/lib/koha/zebradb/ directory. I corrected this problem with:

adminuser$ sudo chown –R –v koha.koha /var/lib/koha/zebradb

This, however, did not solve the problem. It was then suggested that I restart the koha-zebra-daemon (Thanks to jcamins):

adminuser$ sudo service koha-zebra-daemon restart

This corrected the problem temporarily, and everything was running as koha (the crontab to do a full reindex, etc.)

adminuser$ sudo cat /etc/cron.d/koha [it had ‘koha’ in the ‘run as this user’ column] adminuser$ ps aux | grep zebra koha 11635 0.0 1.3 99632 7192 ? S 10:40 0:00 /usr/bin/zebrasrv -v none,fatal,warn -f /etc/koha/koha-conf.xml koha 12563 0.0 0.1 18296 584 ? Ss Jun29 0:00 daemon --name=koha-zebra-ctl.kohadata --errlog=/var/log/koha/koha-zebradaemon.err --stdout=/var/log/koha/koha-zebradaemon.log --output=/var/log/koha/koha-zebradaemon-output.log --verbose=1 --respawn --delay=30 -- /usr/bin/zebrasrv -v none,fatal,warn -f /etc/koha/koha-conf.xml adminuser 12722 0.0 0.1 6160 680 pts/0 S+ 11:49 0:00 grep --color=auto zebra

However, our VM was running low on memory (512MB total) and full indexes were triggering out of memory problems. I found that stopping the apache server, doing the full reindex, restarting the koha-zebra-daemon and then starting the apache server allowed for a successful reindex and searching to work properly.

adminuser$ sudo service apache2 stop adminuser$ sudo su – koha koha$ cd /usr/share/koha/bin/migration_tools koha$ echo $KOHA_CONF [this better be set correctly] koha$ echo $PERL5LIB [this better be set correctly] koha$ ./rebuild_zebra –b –a –r –v [long indexing/exporting output which could take several hours – DO THIS AFTER HOURS!! – unless it is currently broken] koha$ exit adminuser$ sudo service apache2 start

How did I know we were having memory problems? I opened a secondary window and watched while reindexing, and I noticed some errors in the logs adminuser$ free –m –s 1 [output snipped] ^C adminuser$ cd /var/log/koha adminuser$ grep memory * [you’ll recognize the problem when you see the output]

If you don’t have swap and can turn it on, do so. This is left as an exercise for sysadmin administrators.

I don’t know why I have to restart the koha-zebra-daemon after a full-reindex, but I’m hoping to get a memory bump to 1GB from 512MB, and then not have to stop apache. We sadly, can not turn on swap.

Anyways, I hope this helps someone somewhere. Also, if someone can explain why I need to restart the koha-zebra-daemon after a full reindex even as koha, that would be appreciated too. I was thinking that perhaps the daemon has filehandles open when the files are deleted, and so the filehandles it has are stale?

GPML, Mark Tompsett --- END POST FROM ZEBRA LIST ---

OK, I guess it wasn't a good idea... I all messed up the indexes or I don't know what (strange links in search results) and we need to restore the last back-up. No drama but very disappointed as it was my last idea. I am ready to resign or commit harakiri. (Bernardo, indeed we use zebra, sorry for the mistake with zebraqueue... That's the whole story of my life, zebra not working properly with us...) Cheers, Sonia. Date: Mon, 23 Jul 2012 10:13:47 -0300 Subject: Re: [Koha] SQL queries to delete records from the database? From: bgkriegel@gmail.com To: sossolapro@hotmail.com CC: koha@lists.katipo.co.nz Thanks for your help, Marc. We don't use zebra-daemon, as I think it's deprecated (are we wrong?). Yes, you are wrong. zebra-daemon is not deprecated. What is not recommended to use is zebraqueue. You have better results and more functionality using zebra than not using it. Bernardo

Yes, I did. And actually it looks like I didn't mess anything with my deletions this morning. I thought it was strange, but after reloading the back-up it looks the same. I guess I just didn't pay attention what the search results look like (in the item column). Anyway, I didn't delete these records for fun, I thought it would help solve our problem with reindexing, and it didn't. Just after uploading the database, I run a reindex and then I created a new record and then waited for zebra to reindex (every 15 minutes on the cronjob). But it didn't. I didn't check if it has run at all or if it hasn't reached the last records. I thought these faulty records were preventing zebra from reaching the last records... So, after that I just put back the old database (as I thought my SQL things were messing up with the search results). Yes, Bob, I guess a professional would solve the problem... I am doing this as a voluntary job, but it's taking too much time for me now and I am no good. We are paying someone for the server, so he helps me with all the technical things, but he doesn't know Koha especially. Very disappointing situation for me... Especially because I pushed them (when I was working for them) to change from their old library software to Koha. I feel responsible, though I still believe in Koha of course. Our organisation used to work with a Koha support company, but that wasn't working greatly. That's when I took over... (you will know everything :)) Thanks for your help! Cheers, Sonia.

Date: Tue, 24 Jul 2012 10:26:58 +0200 From: robin@catalyst.net.nz To: koha@lists.katipo.co.nz Subject: Re: [Koha] SQL queries to delete records from the database?

Op 24-07-12 00:57, Sonia P. schreef:

...

OK, I guess it wasn't a good idea... I all messed up the indexes or I don't know what (strange links in search results) and we need to restore the last back-up. No drama but very disappointed as it was my last idea. I am ready to resign or commit harakiri.

Did you run a full re-index? It's always very important to rebuild zebra after modifying the database directly.

Very important.

-- Robin Sheat Catalyst IT Ltd. ✆ +64 4 803 2204 GPG: 5957 6D23 8B16 EFAB FEF8 7175 14D3 6485 A99C EB6D

_______________________________________________ Koha mailing list http://koha-community.org Koha@lists.katipo.co.nz http://lists.katipo.co.nz/mailman/listinfo/koha