[Correction.] Discussions of programming development issues are more appropriate for the koha-devel mailing list. I encourage people to subscribe to that list. The programming development issue spilled over on to this list as part of a discussion of development management which is more than merely a programming development issue. Correction to my earlier reply inline: On Mon, November 22, 2010 06:56, Thomas Dukleth wrote: [...]
On Sat, November 20, 2010 04:49, LAURENT Henri-Damien wrote:
Le 19/11/2010 22:41, Thomas Dukleth a écrit :
[...]
4. SIMPLESERVER.
Same for the Z3950 server which we wrote a wrapper using Net::Z3950::SimpleServer. This would allow ppl to expose their collection as a Z3950 server.
Net::Z39.50::SimpleServer has been on my list as one option which would need improvement to have features sufficiently comparable to Zebra as a Z39.50 Server. 'Simple' may be taken to mean that supporting complexity is an exercise left to the programmer using the tool. Implementations of SimpleServer generally only support use attributes because of the complexity of managing more. Below, Henri-Damien attests to the difficulties of parsing PQF reliably. SimpleServer does not support SRU. The documentation for SimpleServer is less complete than Zebra documentation. We also have much greater working knowledge of Zebra.
I will enquire with IndexData about what options there might be for adding SRU support to SimpleServer.
I try to be especially careful about being accurate. I easily falsified my claim that SimpleServer does not support SRU without any need to do anything more than examine the documentation more completely. My knowledge that Simple Server had not had SRU support at an important point in the past led me to be less careful than I almost always am before I posted my message. I did check before posting but I did not check well enough while giving my attention to what SimpleServer provides for managing PQF. Even with SRU support the basic problem of SimpleServer being simple remains. SimpleServer requires the implementing programmer to do much work in managing the complexity of PQF. The extent to which SimpleServer provides management of PQF merely changes one data model into another data model without necessarily reducing the complexity. Henri-Damien Laurent had thought it prudent to avoid such complexity when he considered undertaking the simpler task of merely writing PQF queries where Koha acts as a Z39.50 client. Writing PQF for a Z39.50 client is much easier than parsing PQF for a Z39.50 server. Something as simple as distinguishing a list of words to be matched from a phrase to be matched for SimpleServer requires the programmer to manage the complexity of PQF. Zebra takes care of the complexity of parsing PQF for us automatically where we only need to bother about using the configuration files appropriately. Zebra has some problems which would cost something to fix. With an appropriate investment of resources, SimpleServer could be a comparable alternative to Zebra as a Z39.50/SRU server for Koha when using Solr/Lucene for local indexing. As SimpleServer is written in Perl, we may consider that SimpleServer is a better long term option for a Z39.50/SRU server in Perl based Koha than C based Zebra when using Solr/Lucene. There are not any no cost options for replacing Zebra as a Z39.50/SRU server for Koha with a sufficiently comparable feature set when using Solr/Lucene for local indexing. Zebra has a very large feature set as a Z39.50/SRU server which we should not give up easily. The features may be overlooked by most people because Zebra is not nearly as well configured by default in Koha as it could be. Each of the options has its own advantages and disadvantages. We need to understand the details well and give appropriate weight to the various factors. I agree that it would be much easier to maintain one record indexing system at a high level than two and that factor should be weighted appropriately.
5. PREFIX QUERY FORMAT.
As I have stated previously, C4::Search ought to be rewritten in future using prefix query format (PQF) as the native language for Z39.50.
Well, Rewriting the whole Search With PQF would not be handy in many aspects. I have thought about that many times. And using PQF still appears to me not to be the solution. a) It is really a pain to maintain and analyse. b) Whenever you need some more feature in your search, you have to add some more qualifiers, and therefore provide a robust parser.
I agree that analysing PQF connectors is tricky in comparison to CCL connectors which Yaz converts to PQF. In my own work independent of Koha, I overcame difficulties by having the user interface display the PQF query which my code would generate.
I have code for writing PQF query sets which I had started in 2005 using PHP Yaz before Net::Z3950::ZOOM was available for Perl. My code supports the complete and I mean complete Bib-1 attribute set. I could port the code to Perl with a little effort, although, it would still need some work for more extensibility of the user controlled term sets.
Queries built using PQF can be tested by building the same query in CCL and sending it to Yaz for conversion to PQF as a comparison.
Writing a PQF parser to interpret incoming PQF queries for SimpleServer would be perhaps an order of magnitude more complex.
Continuing to support Common Command Language (CCL) is trivial because Yaz translates CCL to PQF as it does now for Koha.
We may need CCL support to work most effectively with CCL based Pazpar2. Basic functionality in Pazpar2 converts CCL to whatever query language is appropriate for each of configured target server. Pazpar2 documentation contains some incomplete descriptions of PQF options. [...] Thomas Dukleth Agogme 109 E 9th Street, 3D New York, NY 10003 USA http://www.agogme.com +1 212-674-3783