[Koha] Elasticsearch setup
Tom Hanstra
hanstra at nd.edu
Thu Mar 19 02:53:43 NZDT 2020
Yes, I can try downgrading.
Here is another error I saw which might be a clue:
No record passed at /usr/share/koha/lib/Koha/MetadataIterator.pm line 71.
Something went wrong reading record for biblio
Koha::Schema::Result::Biblio=HASH(0x112a7b28)->biblionumber: Can't bless
non-reference value at /usr/share/koha/lib/Koha/BiblioUtils.pm line 70.
For those using ElasticSearch, have you done a side-by-side comparison with
Zebra to make sure the results match up? Just curious about how well it
works. I'm hoping I'll especially gain in terms of speed.
Tom
On Wed, Mar 18, 2020 at 9:44 AM Jonathan Druart <
jonathan.druart at bugs.koha-community.org> wrote:
> I am not sure version 7 has been tested already. Could you try with ES6?
>
>
> Le mer. 18 mars 2020 à 14:30, Tom Hanstra <hanstra at nd.edu> a écrit :
>
>> curl http://localhost:9200
>> {
>> "name" : "london-test-20",
>> "cluster_name" : "elasticsearch",
>> "cluster_uuid" : "9tssH9MpQn2yTcUxf5xbzg",
>> "version" : {
>> "number" : "7.6.1",
>> "build_flavor" : "default",
>> "build_type" : "rpm",
>> "build_hash" : "aa751e09be0a5072e8570670309b1f12348f023b",
>> "build_date" : "2020-02-29T00:15:25.529771Z",
>> "build_snapshot" : false,
>> "lucene_version" : "8.4.0",
>> "minimum_wire_compatibility_version" : "6.8.0",
>> "minimum_index_compatibility_version" : "6.0.0-beta1"
>> },
>> "tagline" : "You Know, for Search"
>> }
>>
>> On Wed, Mar 18, 2020 at 4:03 AM Jonathan Druart <
>> jonathan.druart at bugs.koha-community.org> wrote:
>>
>>> Which version of elasticsearch did you install?
>>>
>>> Le mar. 17 mars 2020 à 21:33, Tom Hanstra <hanstra at nd.edu> a écrit :
>>> >
>>> > Well, that obviously is an important switch. Ant that takes me one step
>>> > closer. Thank you.
>>> >
>>> > Now, of course, I'm getting no results. Koha says that I did not
>>> specify
>>> > any search criteria and the logs show the following:
>>> >
>>> > [Tue Mar 17 20:28:53.076236 2020] [cgi:error] [pid 3868] [client
>>> > 127.0.0.1:41634] AH01215: [Tue Mar 17 20:28:53 2020] opac-search.pl:
>>> Use of
>>> > uninitialized value $f in pattern match (m//) at
>>> > /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
>>> line
>>> > 725.: /usr/share/koha/opac/cgi-bin/opac/opac-search.pl, referer:
>>> >
>>> https://london-test.library.nd.edu/cgi-bin/koha/opac-search.pl?idx=&q=bible
>>> > [Tue Mar 17 20:28:53.076368 2020] [cgi:error] [pid 3868] [client
>>> > 127.0.0.1:41634] AH01215: [Tue Mar 17 20:28:53 2020] opac-search.pl:
>>> Use of
>>> > uninitialized value $f in exists at
>>> > /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm
>>> line
>>> > 729.: /usr/share/koha/opac/cgi-bin/opac/opac-search.pl, referer:
>>> >
>>> https://london-test.library.nd.edu/cgi-bin/koha/opac-search.pl?idx=&q=bible
>>> > [Tue Mar 17 20:28:53.236107 2020] [cgi:error] [pid 3868] [client
>>> > 127.0.0.1:41634] AH01215: [Tue Mar 17 20:28:53 2020] opac-search.pl:
>>> > [Request] ** [http://localhost:9200]-[400]
>>> [illegal_argument_exception]
>>> > Text fields are not optimised for operations that require per-document
>>> > field data like aggregations and sorting, so these operations are
>>> disabled
>>> > by default. Please use a keyword field instead. Alternatively, set
>>> > fielddata=true on [location__facet] in order to load field data by
>>> > uninverting the inverted index. Note that this can use significant
>>> memory.,
>>> > called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__
>>> at
>>> > /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/Search.pm line 98.
>>> With
>>> > vars: {'body' => {'status' => 400,'error' => {'root_cause' =>
>>> [{'reason' =>
>>> > 'Text fields are not optimised for operations that require per-document
>>> > field data like aggregations and sorting, so these operations are
>>> disabled
>>> > by default. Please use a keyword field instead. Alternatively, set
>>> > fielddata=true on [location__facet] in order to load field data by
>>> > uninverting the inverted index. Note that this can use significant
>>> > memory.','type' => 'illegal_argument_exception'}],'grouped' => bless(
>>> > do{\\(my $o = 1)}, 'JSON::PP::Boolean' ),'phase' => 'query','reason' =>
>>> > 'all shards failed','type' =>
>>> > 'search_phase_execution_exception','caused_by' => {'reason' => 'Text
>>> fields
>>> > are not optimised for operations that require per-document field data
>>> like
>>> > aggregations and sorting, so these operations are disabled by default.
>>> > Please use a keyword field instead. Alternatively, set fielddata=true
>>> on
>>> > [location__facet] in order to load field data by uninverting the
>>> inverted
>>> > index. Note that this can use significant memory.','type' =>
>>> > 'illegal_argument_exception','caused_by' => {'reason' => 'Text fields
>>> are
>>> > not optimised for operations that require per-document field data like
>>> > aggregations and sorting, so these operations are disabled by default.
>>> > Please use a keyword field instead. Alternatively, set fielddata=true
>>> on
>>> > [location__facet] in order to load field data by uninverting the
>>> inverted
>>> > index. Note that this can use significant memory.','type' =>
>>> > 'illegal_argument_exception'}},'failed_shards' => [{'index' =>
>>> > 'koha_london_test_biblios','reason' => {'reason' => 'Text fields are
>>> not
>>> > optimised for operations that require per-document field data like
>>> > aggregations and sorting, so these operations are disabled by default.
>>> > Please use a keyword field instead. Alternatively, set fielddata=true
>>> on
>>> > [location__facet] in order to load field data by uninverting the
>>> inverted
>>> > index. Note that this can use significant memory.','type' =>
>>> > 'illegal_argument_exception'},'shard' => 0,'node' =>
>>> > 'a0gvjmfUSfepKAIckHdhVA'}]}},'request' => {'serialize' => 'std','path'
>>> =>
>>> > '/koha_london_test_biblios/_search','ignore' => [],'mime_type' =>
>>> > 'application/json','body' => {'aggregations' => {'ccode' => {'terms' =>
>>> > {'size' => '20','field' => 'ccode__facet'}},'location' => {'terms' =>
>>> > {'size' => '20','field' => 'location__facet'}},'title-series' =>
>>> {'terms'
>>> > => {'size' => '20','field' => 'title-series__facet'}},'subject' =>
>>> {'terms'
>>> > => {'size' => '20','field' => 'subject__facet'}},'author' => {'terms'
>>> =>
>>> > {'size' => '20','field' => 'author__facet'}},'su-geo' => {'terms' =>
>>> > {'size' => '20','field' => 'su-geo__facet'}},'ln' => {'terms' =>
>>> {'size' =>
>>> > '20','field' => 'ln__facet'}},'holdingbranch' => {'terms' => {'field'
>>> =>
>>> > 'holdingbranch__facet'}},'itype' => {'terms' => {'size' =>
>>> '20','field' =>
>>> > 'itype__facet'}}},'from' => 0,'query' => {'query_string' => {'fields'
>>> =>
>>> >
>>> ['record-source','local-number','Host-Item-Number','number-legal-deposit','identifier-standard','su-geo','ln','ff7-02','microform-generation','control-number','ta','notforloan','isbn','publisher','ff7-01','bio','itemnumber','ctype','ff7-00','location','subject','ff8-23','bgf-number','bib-level','ff8-29','title','material-type','issues','number-db','date-of-publication','place-of-publication','homebranch','author','issn','holdingbranch','ff7-01-02','number-natl-biblio','local-classification','bnb-card-number','llength','rtype','date-entered-on-file','pl','title-series','date-of-acquisition','date-time-last-modified','lf','copydate','koha-auth-number','itype'],'fuzziness'
>>> > => 'auto','query' => '(bible) AND suppress:false','default_operator' =>
>>> > 'AND','lenient' =>
>>> $VAR1->{'body'}{'error'}{'grouped'},'analyze_wildcard'
>>> > => $VAR1->{'body'}{'error'}{'grouped'}}},'size' => '30'},'qs' =>
>>> > {},'method' => 'GET'},'status_code' => 400}:
>>> > /usr/share/koha/opac/cgi-bin/opac/opac-search.pl, referer:
>>> >
>>> https://london-test.library.nd.edu/cgi-bin/koha/opac-search.pl?idx=&q=bible
>>> > [Tue Mar 17 20:28:53.236252 2020] [cgi:error] [pid 3868] [client
>>> > 127.0.0.1:41634] AH01215: [Tue Mar 17 20:28:53 2020] opac-search.pl:
>>> Use of
>>> > uninitialized value $error in concatenation (.) or string at
>>> > /usr/share/koha/opac/cgi-bin/opac/opac-search.pl line 642.:
>>> > /usr/share/koha/opac/cgi-bin/opac/opac-search.pl, referer:
>>> >
>>> https://london-test.library.nd.edu/cgi-bin/koha/opac-search.pl?idx=&q=bible
>>> >
>>> > So, I'm obviously missing something in either the way I've created the
>>> > index or in the way Koha is using it.
>>> >
>>> > Anyone seen something like this?
>>> >
>>> > Tom
>>> >
>>> > On Tue, Mar 17, 2020 at 3:05 PM Katrin Fischer <
>>> katrin.fischer.83 at web.de>
>>> > wrote:
>>> >
>>> > > Hi Tom,
>>> > >
>>> > > if everything is set up correctly you can switch back and forth using
>>> > > the searchengine system preference.
>>> > >
>>> > > Hope this helps,
>>> > >
>>> > > Katrin
>>> > >
>>> > > On 17.03.20 17:08, Tom Hanstra wrote:
>>> > > > I have managed to get elasticsearch running (thanks Thomas!) and
>>> it seems
>>> > > > to have indexed many of my records (though there were quite a few
>>> errors
>>> > > > and messages received during the indexing process).
>>> > > >
>>> > > > Now, how do I best check to see what is in the Elasticsearch
>>> index? Is
>>> > > > there a way to test or at least count the number of records
>>> indexed?
>>> > > >
>>> > > > And then how do I switch Koha from using Zebra to using
>>> Elasticsearch?
>>> > > Do
>>> > > > I have to "turn off" Zebra in some way and turn on Elasticsearch?
>>> Or is
>>> > > > that supposed to be switched automatically?
>>> > > >
>>> > > > Thanks in advance for any tips that might help!
>>> > > >
>>> > > > Tom
>>> > > >
>>> > > > On Mon, Mar 16, 2020 at 2:20 PM <mlist at opendoor.fr> wrote:
>>> > > >
>>> > > >> Hello,
>>> > > >>
>>> > > >> We use debian, but are installing from sources. Here are my notes
>>> about
>>> > > >> ElasticSearch:
>>> > > >>
>>> > > >> Get a recent version of Java
>>> > > >>
>>> > > >> Install ES from upstream repositoruies
>>> > > >>
>>> > > >> Configure Koha <-> ES via $KOHA_CONF
>>> > > >>
>>> > > >> Install lucene plugin:
>>> > > >>
>>> > > >> /usr/share/elasticsearch/bin/elasticsearch-plugin install
>>> analysis-icu
>>> > > >>
>>> > > >> Install following packages:
>>> > > >>
>>> > > >> libcatmandu-marc-perl
>>> > > >> libcatmandu-store-elasticsearch-perl
>>> > > >>
>>> > > >> systemctl enable elasticsearch
>>> > > >>
>>> > > >> restart server (or restart all koha related services)
>>> > > >>
>>> > > >> rebuild index:
>>> > > >>
>>> > > >> /home/koha/Current/bin/search_tools/rebuild_elasticsearch.pl -v
>>> > > >>
>>> > > >> /home/koha/Current/bin/search_tools/rebuild_elasticsearch.pl
>>> –delete
>>> > > -v
>>> > > >> -a -b -p 4
>>> > > >>
>>> > > >>
>>> > > >> Best regards
>>> > > >>
>>> > > >>
>>> > > >> On 2020-03-16 7:09 p.m., Tom Hanstra wrote:
>>> > > >>> In the documentation I'm reading about the use of elasticsearch,
>>> all
>>> > > >>> information is completely Debian specific, including both the
>>> use of
>>> > > >>> kohadevbox and debian software installations of the
>>> koha-elasticsearch
>>> > > >>> metapackage.
>>> > > >>>
>>> > > >>> For those of us who don't use Debian (yes, there are some of
>>> us), is
>>> > > >> there
>>> > > >>> any documentation of more complete ElasticSearch setup commands
>>> and
>>> > > >>> information of what is inside of the koha-elasticsearch
>>> metapackage and
>>> > > >>> other supporting software? I'd rather not have to reverse
>>> engineer
>>> > > those
>>> > > >>> pieces from Debian, but the documentation seems to assume
>>> installation
>>> > > >> has
>>> > > >>> one or the other of the above Debian based packages.
>>> > > >>>
>>> > > >>> I'm assuming that Koha can work with Elasticsearch on any Linux
>>> > > >>> distribution, with the right setup. If so, can someone help fill
>>> in the
>>> > > >>> gaps between Debian and the other Linuxes.
>>> > > >>>
>>> > > >>> Thanks,
>>> > > >>> Tom
>>> > > >>>
>>> > > >> --
>>> > > >> Thomas Constans
>>> > > >> Services en informatique libre
>>> > > >> T: 33(0)6 23 37 87 85
>>> > > >> @: https://opendoor.fr
>>> > > >> #: https://twitter.com/ThomasConstans
>>> > > >> gpg: 0xBA19745F80410541
>>> > > >>
>>> > > >> _______________________________________________
>>> > > >>
>>> > > >> Koha mailing list http://koha-community.org
>>> > > >> Koha at lists.katipo.co.nz
>>> > > >> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>> > > >>
>>> > > >
>>> > > _______________________________________________
>>> > >
>>> > > Koha mailing list http://koha-community.org
>>> > > Koha at lists.katipo.co.nz
>>> > > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>> > >
>>> >
>>> >
>>> > --
>>> > *Tom Hanstra*
>>> > *Sr. Systems Administrator*
>>> > hanstra at nd.edu
>>> > _______________________________________________
>>> >
>>> > Koha mailing list http://koha-community.org
>>> > Koha at lists.katipo.co.nz
>>> > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>>>
>>
>>
>> --
>> *Tom Hanstra*
>> *Sr. Systems Administrator*
>> hanstra at nd.edu
>>
>>
>>
--
*Tom Hanstra*
*Sr. Systems Administrator*
hanstra at nd.edu
More information about the Koha
mailing list