[Koha] Proposal to form Koha Technical Committee

[Thomas Dukleth]

Reply inline:


On Wed, November 10, 2010 15:54, Paul Poulain wrote:
> Le 09/11/2010 22:01, Chris Nighswonger a écrit :
>> On Tue, Nov 9, 2010 at 2:15 PM, Lenora Oftedahl <OFTL at critfc.org> wrote:
>>

1.  ADVISING THE DEVELOPMENT PROCESS.

>>> When the Release Manager is chosen and accepts the position, they
> understand what they are taking on, don't they?
>>>
>> Just to clarify: It is *not* the responsibility of the RM to vet RFCs
>> and pursue the critique, correction, and/or amending of them. The RM
>> simply wields the final say over what is finally actually pushed.
>>
> I fully agree with you. And that's why i'm proposing to have another
> structure/team/guy/whatever to work on strategic/long-term things.

I fully understand the need to have discussion, comment, and criticism of
prospective development plans to have a good sense of what the community
would favour.  Forming any ad hoc group for consultation is something
which anyone can do at any time.  The more open the group, the more
legitimacy its work would have to the rest of the Koha community.  A
select small group of interested people is also fine for obtaining a
perspective outside of one development organisation and would obtain
legitimacy by a demonstration of critical detailed examination of
development work.  Everyone benefits by having work examined by those
outside the self-reinforcing insularity of the organisation preparing some
work.

As I explained in my concurrence with David Lang, rigid long term
decisions are antithetical to the vibrancy of the open source development
model used in free software projects,
http://lists.katipo.co.nz/pipermail/koha/2010-November/026272.html . 
Rigid long term decisions are also a weakness of the development model
most often used for proprietary software.  We should not empower any group
to do more than advise the development process.

Whatever comment can be obtained from whomever one can obtain it, can
inform the development process of an organisation.  Favourable comments
and the absence of objections can be used to reassure development clients
of the likelihood of successful feature integration and help persuade the
release manager to include a development option as the best course even
when there is some conflict with other development.

If an issue is sufficiently important, a community consensus can be
obtained which a release manager would be reluctant to oppose.  We should
be careful not to impose an undue burden on anyone who has been willing to
do the work of release manager.

We should also be careful not to mistake the 'tyranny of the majority' for
a consensus which could clobber features used by a minority.  Remember
that everyone is a member of the minority for something.

The easiest means to ensuring cooperative development inclusion is careful
planning in advance and early wide consultation to avoid development
conflicts.  Developers need to be prepared to make appropriate changes any
time there is serious criticism of their work.  We want all the good work
relevant to Koha which people produce to be included.


2.  SOLR/LUCENE BASED RECORD INDEXING AND RETRIEVAL.

>
> To answer MJ question related to that question : I fully agree solR
> won't probably be included in 3.4 (not ready enough,...) But do we want
> it for 3.6 ?
> (to answer your comment : yes we have a contract, but i'm ready, if "the
> community" argue & decide it's a wrong idea, to reach the library and
> explain the problem. And find a solution) But atm, I have no clear
> decision taken, just a global feeling that "ppl think it's a good idea
> if we succeed to address the z3950 server problem".

I hope that BibLibre and others will take the following as the
constructive criticism it is intended to be.  Everyone should be reminded
that BibLibre and Paul Poulain's predecessor business have provided in my
opinion the greatest share of important features for Koha without which
most of us would not be using Koha.  Paul Poulain's business also did much
of the most difficult early development work for Zebra.  The prospect of
dropping Zebra support in favour of a Solr/Lucene support being developed
by BibLibre is a major factor which has led to discussions about how to
best manage development conflicts.

I think that Solr/Lucene will prove to be the best option for local record
indexing in Koha long term which it would not have been at the time Zebra
support was developed for Koha.  The disagreement from me at least is only
over how the consequences of such a change are managed.  An issue is
whether Zebra remains available for use as an option along side
Solr/Lucene at least until everyone is satisfied that Zebra can be
replaced without loss of capabilities.

Much of the capabilities which Zebra support provides are not being used
in Koha and we are comparing our own familiarity with Zebra difficulties
with a rosy ideal of what Solr/Lucene offers.  The first advantage of
Solr/Lucene is that it empowers less sophisticated users to control how
indexing is configured via a web interface.  A web interface could be
created for Zebra but that is not an existing feature.  I am for
everything which empowers users to more easily exercise control over their
software.

There is more than just the problem of losing very good Z39.50/SRU server
support which would follow from BibLibre's announced implementation of
Solr/Lucene for local indexing.  Z39.50 client support which was
undertaken for local indexing with Zebra could enable future Z39.50/SRU
client development without Zebra.  In rewriting searching for their own
testing branch of Koha for Solr/Lucene implementation, BibLibre have
removed valuable Z39.50 client support which could be used in future
features such as querying Z39.50 servers in addition to Solr/Lucene in the
OPAC and presenting a unified result set using Pazpar2.  Z39.50 client
copy cataloguing support is independent from C4::Search and is thus safe
but could not be improved using code from C4::Search if that code is gone.

As I have stated previously, C4::Search ought to be rewritten in future
using prefix query format (PQF) as the native language for Z39.50. 
Continuing to support Common Command Language (CCL) is trivial because Yaz
translates CCL to PQF as it does now for Koha.  In the context of
refactoring to support all options, future rewriting of C4::Search for
Zebra would be rewriting it in C4::Search::Zebra along with
C4::Search::Solr and C4::Search would become a neutral abstraction.  When
the opportunity to rewrite the Zebra code would arise, rewriting would be
much easier with known working code to use as a starting point, however
imperfects.

The Koha community, especially BibLibre, have a huge amount of work
invested in Zebra support and as I stress the development which underlies
Zebra support.  It would be a shame to loose the future benefit of that
investment for problems which all seem resolvable to me even if
Solr/Lucene is found to be a superior solution for local indexing as I
expect it will be in due time.

I continue to investigate JZKit for a Z39.50/SRU server from Knowledge
Integration and options from Index Data.  Today, I reminded Ian Ibbotson
from Knowledge Integration that I am still waiting for a direct reply to
some specific questions which I posed over two weeks ago about JZKit. 
Most of the details which I know about JZKit came from carefully examining
the source code but I do not want to rely upon that examination and other
clues alone in the absence of documentation.  I will present a full report
with a detailed examination of capabilities and options when I have
received a further response from those who know their code best.

I have raised all of these issues on the koha-devel list without much
specific response.  On the koha-devel list, MJ Ray has also more recently
asked for more details about Zebra problems in a form which could be
independently examined by others and he should have more complete replies
than he has had thus far,
http://lists.koha-community.org/pipermail/koha-devel/2010-November/034664.html
.

I have other comments to make about the prospective Solr/Lucene
implementation which I have not put forward but I do not wish to confuse
what I could improve later for Solr/Lucine support with what would be
extraordinarily difficult to improve if support would be removed for
Zebra.


3.  CONTINUING DISCUSSION.

> Do you think I
> should consider this as enough and go my way? I don't think so, because
> the customer want his sponsored work to be in official Koha.
>
> Does it mean I just should shut my mouth, candidate to be the RM for
> 3.6, and do what I want here in term of strategic decision ? Well, if
> ppl think it's how I should do, then I'll do. But that's not how I want
> to do it : community matter

Hooray for the customer wanting cooperation and not relegation to a
difficult to maintain fork of Koha.

I have great respect for everyone who takes the time to engage in
discussion, presents evidence in support, and argues for a position based
on evidence.  We only risk running aground when we do not take the time to
have discussions.

>
>> Clearly the RM should:
>>
>> 1. Be knowledgeable of RFCs which are proposed for the current version.
>>
> ++
>> 2. Lead by example in the matter of participation in the RFC process.
>>
> ++
>> However, to suggest that the RM bares the responsibility for doing the
>> leg-work in the RFC process will probably reduce the future candidate
>> pool of RM's due to fear of being worked to death.
>
> ++
>
> AND (as I already wrote), the RM deals with the next version, and
> sometimes things will be for the next-next version. Or the
> next-next-next. Or some things are related to long-term so must be
> addressed separately (ie : not just in a given version)
>
>> It is, in fact, the responsibility of the author of the RFC (or the
>> author's employer) to promote that RFC and keep it before the
>> community.
>>
>> And it is the responsibility of the *community* to vet, critique,
>> correct, and/or amend RFCs.
>>
>> Historically there have been failures on both parts. Things are
>> looking up, however.
>>
>
> Agreed. (I'm not trying to say I did everything well. But I try to learn
> from my mistakes, and I just want Koha to be always more and more
> reliable & sustainable & predictible)

We all have to make mistakes to learn from them and the more work one does
the more likely one is to make mistakes which are learning opportunities. 
I commend BibLibre for their hard work.

I hope for more than mere predictability for Koha.  I hope to be regularly
astonished by unpredictable improvements in Koha in which I would include
Solr/Lucene based indexing when it is well implemented.

>
>>> And if dates slip, well, gee, won't we be getting a better release
> due to the extra work?
>>>
>> Again looking at history, the type of slippage traditionally seen is
>> terrible for users, vendors, and clients of vendors. You cannot plan
>> around an ever-moving date.
> ++ and i'm 10000% happy with the time-based release for 3.4 & for
> futures versions hopefully.
>
> And, once again, I feel very uncomfortable when chris says : "it's the
> decision for 3.4, the 3.6RM will decide how he want to deal with it".
> I strongly disagree : it should a community "stable" decision a RM can't
> change just because he want to change it.

This point could just as easily be taken in the opposite way.  The
'community stable decision' in which everyone participated had been to
choose Zebra in 2005 and in this case the release manager is merely acting
to preserve people's options and not undue what had been the community
consensus or loose a good feature even if only a minority of users are
currently making use of it.

Some people in the community could decide that some new development should
be acceptable despite concerns about bugs and safety.

> That's a question of being
> sustainable & predictible (for both vendors and users)
>
>> People who might be considered crazy (like
>> me) and run the cutting edge code probably don't care. But not
>> everyone can afford those risks. This is a problem which can be and is
>> being addressed.
>>
> agreed
>>> Again, I think we are slipping into the commercial vendor mode where
> more voices are being added for no purpose other than to make noise.
>>>
>> I agree with this.
>>
> Not sure I understand well what's meaning here. (The fact is that many
> new features are developed by sponsored work and commercial vendors, so
> we have to deal with that ! )
>>> I think adding a SINGLE person to the release team of RFC manager to
> make comments and organize RFC's might help the RM, but that's up to the
> Release Manager to choose someone they can work with.
>>>
>> Really about the best we can get without *active* * community*
>> participation is an RFC cheerleader. I'm not sure what there is here
>> to manage.
>>
>> The process/workflow is simple:
>>
>> 1. Post your RFC on *both* the wiki and both lists.
>> 2. Bump the post to the lists if you (as the author) feel that there
>> is not enough vetting, etc. going on.
>> 3. If you (as the author) are actively promoting your RFC and it is
>> not getting enough discussion, feel free to complain loudly;
>> otherwise, don't complain.
>>
>
> Well, that's what I do : I complain loudly ;-)
>
>> I think the idea suggested in another thread of having a section of
>> the monthly news letter to highlight RFCs is a great idea. Perhaps it
>> could especially highlight those with fewer comments/discussion/etc.
>>
> ++ I think it's a great idea too !

The more publicity for RFCs the better.

The brief RFCs descriptions which people tend to write should not become
mere static documents as originally described by the developer.  They
should be expanded with implementation details when implementation options
are considered.


Thomas Dukleth
Agogme
109 E 9th Street, 3D
New York, NY  10003
USA
http://www.agogme.com
+1 212-674-3783