[Koha] Elasticsearch setup
Tom Hanstra
hanstra at nd.edu
Thu Mar 19 02:30:09 NZDT 2020
curl http://localhost:9200
{
"name" : "london-test-20",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "9tssH9MpQn2yTcUxf5xbzg",
"version" : {
"number" : "7.6.1",
"build_flavor" : "default",
"build_type" : "rpm",
"build_hash" : "aa751e09be0a5072e8570670309b1f12348f023b",
"build_date" : "2020-02-29T00:15:25.529771Z",
"build_snapshot" : false,
"lucene_version" : "8.4.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
On Wed, Mar 18, 2020 at 4:03 AM Jonathan Druart <
jonathan.druart at bugs.koha-community.org> wrote:
> Which version of elasticsearch did you install?
>
> Le mar. 17 mars 2020 à 21:33, Tom Hanstra <hanstra at nd.edu> a écrit :
> >
> > Well, that obviously is an important switch. Ant that takes me one step
> > closer. Thank you.
> >
> > Now, of course, I'm getting no results. Koha says that I did not specify
> > any search criteria and the logs show the following:
> >
> > [Tue Mar 17 20:28:53.076236 2020] [cgi:error] [pid 3868] [client
> > 127.0.0.1:41634] AH01215: [Tue Mar 17 20:28:53 2020] opac-search.pl:
> Use of
> > uninitialized value $f in pattern match (m//) at
> > /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm line
> > 725.: /usr/share/koha/opac/cgi-bin/opac/opac-search.pl, referer:
> >
> https://london-test.library.nd.edu/cgi-bin/koha/opac-search.pl?idx=&q=bible
> > [Tue Mar 17 20:28:53.076368 2020] [cgi:error] [pid 3868] [client
> > 127.0.0.1:41634] AH01215: [Tue Mar 17 20:28:53 2020] opac-search.pl:
> Use of
> > uninitialized value $f in exists at
> > /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/QueryBuilder.pm line
> > 729.: /usr/share/koha/opac/cgi-bin/opac/opac-search.pl, referer:
> >
> https://london-test.library.nd.edu/cgi-bin/koha/opac-search.pl?idx=&q=bible
> > [Tue Mar 17 20:28:53.236107 2020] [cgi:error] [pid 3868] [client
> > 127.0.0.1:41634] AH01215: [Tue Mar 17 20:28:53 2020] opac-search.pl:
> > [Request] ** [http://localhost:9200]-[400] [illegal_argument_exception]
> > Text fields are not optimised for operations that require per-document
> > field data like aggregations and sorting, so these operations are
> disabled
> > by default. Please use a keyword field instead. Alternatively, set
> > fielddata=true on [location__facet] in order to load field data by
> > uninverting the inverted index. Note that this can use significant
> memory.,
> > called from sub Search::Elasticsearch::Role::Client::Direct::__ANON__ at
> > /usr/share/koha/lib/Koha/SearchEngine/Elasticsearch/Search.pm line 98.
> With
> > vars: {'body' => {'status' => 400,'error' => {'root_cause' => [{'reason'
> =>
> > 'Text fields are not optimised for operations that require per-document
> > field data like aggregations and sorting, so these operations are
> disabled
> > by default. Please use a keyword field instead. Alternatively, set
> > fielddata=true on [location__facet] in order to load field data by
> > uninverting the inverted index. Note that this can use significant
> > memory.','type' => 'illegal_argument_exception'}],'grouped' => bless(
> > do{\\(my $o = 1)}, 'JSON::PP::Boolean' ),'phase' => 'query','reason' =>
> > 'all shards failed','type' =>
> > 'search_phase_execution_exception','caused_by' => {'reason' => 'Text
> fields
> > are not optimised for operations that require per-document field data
> like
> > aggregations and sorting, so these operations are disabled by default.
> > Please use a keyword field instead. Alternatively, set fielddata=true on
> > [location__facet] in order to load field data by uninverting the inverted
> > index. Note that this can use significant memory.','type' =>
> > 'illegal_argument_exception','caused_by' => {'reason' => 'Text fields are
> > not optimised for operations that require per-document field data like
> > aggregations and sorting, so these operations are disabled by default.
> > Please use a keyword field instead. Alternatively, set fielddata=true on
> > [location__facet] in order to load field data by uninverting the inverted
> > index. Note that this can use significant memory.','type' =>
> > 'illegal_argument_exception'}},'failed_shards' => [{'index' =>
> > 'koha_london_test_biblios','reason' => {'reason' => 'Text fields are not
> > optimised for operations that require per-document field data like
> > aggregations and sorting, so these operations are disabled by default.
> > Please use a keyword field instead. Alternatively, set fielddata=true on
> > [location__facet] in order to load field data by uninverting the inverted
> > index. Note that this can use significant memory.','type' =>
> > 'illegal_argument_exception'},'shard' => 0,'node' =>
> > 'a0gvjmfUSfepKAIckHdhVA'}]}},'request' => {'serialize' => 'std','path' =>
> > '/koha_london_test_biblios/_search','ignore' => [],'mime_type' =>
> > 'application/json','body' => {'aggregations' => {'ccode' => {'terms' =>
> > {'size' => '20','field' => 'ccode__facet'}},'location' => {'terms' =>
> > {'size' => '20','field' => 'location__facet'}},'title-series' => {'terms'
> > => {'size' => '20','field' => 'title-series__facet'}},'subject' =>
> {'terms'
> > => {'size' => '20','field' => 'subject__facet'}},'author' => {'terms' =>
> > {'size' => '20','field' => 'author__facet'}},'su-geo' => {'terms' =>
> > {'size' => '20','field' => 'su-geo__facet'}},'ln' => {'terms' => {'size'
> =>
> > '20','field' => 'ln__facet'}},'holdingbranch' => {'terms' => {'field' =>
> > 'holdingbranch__facet'}},'itype' => {'terms' => {'size' => '20','field'
> =>
> > 'itype__facet'}}},'from' => 0,'query' => {'query_string' => {'fields' =>
> >
> ['record-source','local-number','Host-Item-Number','number-legal-deposit','identifier-standard','su-geo','ln','ff7-02','microform-generation','control-number','ta','notforloan','isbn','publisher','ff7-01','bio','itemnumber','ctype','ff7-00','location','subject','ff8-23','bgf-number','bib-level','ff8-29','title','material-type','issues','number-db','date-of-publication','place-of-publication','homebranch','author','issn','holdingbranch','ff7-01-02','number-natl-biblio','local-classification','bnb-card-number','llength','rtype','date-entered-on-file','pl','title-series','date-of-acquisition','date-time-last-modified','lf','copydate','koha-auth-number','itype'],'fuzziness'
> > => 'auto','query' => '(bible) AND suppress:false','default_operator' =>
> > 'AND','lenient' => $VAR1->{'body'}{'error'}{'grouped'},'analyze_wildcard'
> > => $VAR1->{'body'}{'error'}{'grouped'}}},'size' => '30'},'qs' =>
> > {},'method' => 'GET'},'status_code' => 400}:
> > /usr/share/koha/opac/cgi-bin/opac/opac-search.pl, referer:
> >
> https://london-test.library.nd.edu/cgi-bin/koha/opac-search.pl?idx=&q=bible
> > [Tue Mar 17 20:28:53.236252 2020] [cgi:error] [pid 3868] [client
> > 127.0.0.1:41634] AH01215: [Tue Mar 17 20:28:53 2020] opac-search.pl:
> Use of
> > uninitialized value $error in concatenation (.) or string at
> > /usr/share/koha/opac/cgi-bin/opac/opac-search.pl line 642.:
> > /usr/share/koha/opac/cgi-bin/opac/opac-search.pl, referer:
> >
> https://london-test.library.nd.edu/cgi-bin/koha/opac-search.pl?idx=&q=bible
> >
> > So, I'm obviously missing something in either the way I've created the
> > index or in the way Koha is using it.
> >
> > Anyone seen something like this?
> >
> > Tom
> >
> > On Tue, Mar 17, 2020 at 3:05 PM Katrin Fischer <katrin.fischer.83 at web.de
> >
> > wrote:
> >
> > > Hi Tom,
> > >
> > > if everything is set up correctly you can switch back and forth using
> > > the searchengine system preference.
> > >
> > > Hope this helps,
> > >
> > > Katrin
> > >
> > > On 17.03.20 17:08, Tom Hanstra wrote:
> > > > I have managed to get elasticsearch running (thanks Thomas!) and it
> seems
> > > > to have indexed many of my records (though there were quite a few
> errors
> > > > and messages received during the indexing process).
> > > >
> > > > Now, how do I best check to see what is in the Elasticsearch index?
> Is
> > > > there a way to test or at least count the number of records indexed?
> > > >
> > > > And then how do I switch Koha from using Zebra to using
> Elasticsearch?
> > > Do
> > > > I have to "turn off" Zebra in some way and turn on Elasticsearch?
> Or is
> > > > that supposed to be switched automatically?
> > > >
> > > > Thanks in advance for any tips that might help!
> > > >
> > > > Tom
> > > >
> > > > On Mon, Mar 16, 2020 at 2:20 PM <mlist at opendoor.fr> wrote:
> > > >
> > > >> Hello,
> > > >>
> > > >> We use debian, but are installing from sources. Here are my notes
> about
> > > >> ElasticSearch:
> > > >>
> > > >> Get a recent version of Java
> > > >>
> > > >> Install ES from upstream repositoruies
> > > >>
> > > >> Configure Koha <-> ES via $KOHA_CONF
> > > >>
> > > >> Install lucene plugin:
> > > >>
> > > >> /usr/share/elasticsearch/bin/elasticsearch-plugin install
> analysis-icu
> > > >>
> > > >> Install following packages:
> > > >>
> > > >> libcatmandu-marc-perl
> > > >> libcatmandu-store-elasticsearch-perl
> > > >>
> > > >> systemctl enable elasticsearch
> > > >>
> > > >> restart server (or restart all koha related services)
> > > >>
> > > >> rebuild index:
> > > >>
> > > >> /home/koha/Current/bin/search_tools/rebuild_elasticsearch.pl -v
> > > >>
> > > >> /home/koha/Current/bin/search_tools/rebuild_elasticsearch.pl
> –delete
> > > -v
> > > >> -a -b -p 4
> > > >>
> > > >>
> > > >> Best regards
> > > >>
> > > >>
> > > >> On 2020-03-16 7:09 p.m., Tom Hanstra wrote:
> > > >>> In the documentation I'm reading about the use of elasticsearch,
> all
> > > >>> information is completely Debian specific, including both the use
> of
> > > >>> kohadevbox and debian software installations of the
> koha-elasticsearch
> > > >>> metapackage.
> > > >>>
> > > >>> For those of us who don't use Debian (yes, there are some of us),
> is
> > > >> there
> > > >>> any documentation of more complete ElasticSearch setup commands and
> > > >>> information of what is inside of the koha-elasticsearch
> metapackage and
> > > >>> other supporting software? I'd rather not have to reverse engineer
> > > those
> > > >>> pieces from Debian, but the documentation seems to assume
> installation
> > > >> has
> > > >>> one or the other of the above Debian based packages.
> > > >>>
> > > >>> I'm assuming that Koha can work with Elasticsearch on any Linux
> > > >>> distribution, with the right setup. If so, can someone help fill
> in the
> > > >>> gaps between Debian and the other Linuxes.
> > > >>>
> > > >>> Thanks,
> > > >>> Tom
> > > >>>
> > > >> --
> > > >> Thomas Constans
> > > >> Services en informatique libre
> > > >> T: 33(0)6 23 37 87 85
> > > >> @: https://opendoor.fr
> > > >> #: https://twitter.com/ThomasConstans
> > > >> gpg: 0xBA19745F80410541
> > > >>
> > > >> _______________________________________________
> > > >>
> > > >> Koha mailing list http://koha-community.org
> > > >> Koha at lists.katipo.co.nz
> > > >> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
> > > >>
> > > >
> > > _______________________________________________
> > >
> > > Koha mailing list http://koha-community.org
> > > Koha at lists.katipo.co.nz
> > > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
> > >
> >
> >
> > --
> > *Tom Hanstra*
> > *Sr. Systems Administrator*
> > hanstra at nd.edu
> > _______________________________________________
> >
> > Koha mailing list http://koha-community.org
> > Koha at lists.katipo.co.nz
> > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>
--
*Tom Hanstra*
*Sr. Systems Administrator*
hanstra at nd.edu
More information about the Koha
mailing list