[Koha] Open source ballot design

Wed Oct 14 12:26:01 NZDT 2009

Okay - I consulted a group on IRC earlier and we've decided that this
survey will ask only 2 questions and so does not need to be reviewed -
there will be no misunderstanding. Once Nicolas has a chance to make
sure I translated all strings to French I will send out the link and
that will be that.

Nicole

On Tue, Oct 13, 2009 at 4:25 PM, Thomas Dukleth <kohalist at agogme.com> wrote:
> [Section 5.1 identifies an important voting preference problem which may
> be very likely to occur in our final poll.]
>
>
> Reply inline:
>
> On Tue, October 13, 2009 09:21, MJ Ray wrote:
>
> [...]
>
> 1.  INFORMED VOTING.
>
>>  Should we turn the wiki pages into
>> some sort of consultation information brochure?
>
> The wiki pages could be much easier to read in general but that is a
> different problem.  The wiki pages will do if we use them.  Before people
> initiate the ballot, they should be requested to examine the wiki pages
> and the mailing list archive to become an informed voter.
>
> [...]
>
> 2,  TAKING COMMENTS INTO ACCUNT.
>
>> I suspect statistics is less obvious to most of the community than
>> making a great Library Management System, but I think it's worth
>> posting the ballot design and taking comments into account.
>
> I am pleased that you are not opposed to some more input.  I trust that
> you advise Nicole Engard well about which comments are worth incorporating
> and which would cause problems.
>
>>
>> I beg everyone to check that any suggested changes have some grounding
>> in statistical theory before posting them, else the review process
>> probably won't work because we'll get overloaded.  I wrote
>> http://people.debian.org/~mjr/surveys.html a few years ago which might
>> interest some people.
>
> I really do not think that there will be very many comments overall.  I
> doubt that many comments would even reach the level of skewing statistics.
>
>
> 3.  IRC MEETING.
>
>>
>> [...]
>>> At Wednesday's IRC meeting it was announced, if I recall correctly, that
>>> two weeks would be taken to analyse the previous poll before a final
>>> ballot would be presented.  That period should give sufficient time for
>>> most if not all of my suggestions to have an opportunity to have some
>>> effect on the drafting of a final ballot.
>>
>> I feel that recollection is incorrect.  No particular time was
>> suggested, although I'd prefer it not to drag forever. Please check:
>> http://stats.workbuffer.org/irclog/koha/2009-10-07#i_316162
>
> Checking the log it is not clear to me what the two week period referred
> to in that meeting had been intended to cover.  Certainly there was a
> statement of giving time for analysis of the previous results, a two week
> period, and statements about informing the work for the next poll.  What
> people are commenting about precisely can be difficult to determine.  From
> the meeting I had expected that you would report some analysis of the
> previous poll to everyone which could be used to inform the writing of the
> final ballot.  Had I thought that there would be no open opportunity to
> inform the process I would complained loudly at the meeting.
>
> This is the trouble with ambiguous references and assuming that others
> understand the same intent which you mean to convey.  This is why the
> wording of the questions and options needs more eyes to find areas of
> misunderstanding.
>
>
> 4.  INFORMATION SET.
>
>>
>> So:
>>
>> 1. do we need information brochure(s) about the options?
>>    - who will produce it/them?
>>    - who will referee it/them to make sure it's fair?
>>    - how should it be distributed?
>
> We could include a simple neutral one sentence descriptive statements next
> to each option on the ballot.  Remember that koha.org has not had a
> prominent statement of Koha's origins with HLT for a few years.  Few
> librarians would know that SPI was established to provide a legal entity
> for free software projects starting with Debian, the largest GNU/Linux
> distribution.
>
> Otherwise, the wiki and mailing list record will have to do for anything
> more unless someone is volunteering to work extra hard.
>
>
> 5.  STATISTICAL PROBLEMS.
>
>>
>> 2. are there other trained statisicians in our community?
>
> You may be the only one but that does not mean that you have anticipated
> every problem and the best ways to resolve them.  You should inform Nicole
> from a broader base of input.
>
> I have read extensively on voting theory and maximising preferences years
> ago.  That does not make me a trained statistician but no one in our
> community with valuable input should be ignored.  We should strive to not
> let the patches fall on the floor.
>
> 5.1.  VOTING PROBLEM.
>
> The vote on first choices for forming a foundation in the previous survey
> had an equal division between the "independent foundation now" option and
> what had been presented as alternatives in the previous survey.
>
> Have you considered how we should address the possibility that an absolute
> majority may be in favour of a range of interim foundation choices but the
> largest first rank vote count may be for "wait to agree on an independent
> foundation"?
>
> Which foundation forming option should we choose?
>
> Hypothetical first rank vote.
>
> Horowhenua Library Trust (HLT) for an interim period:  35%
> Software Freedom Conservancy (Conservancy) for an interim period:  0%
> Software in the Public Interest (SPI) for an interim period:  20%
> Wait to agree on an independent foundation:  45%
> No opinion:  0%
>
> Hypothetical second rank vote.
>
> Horowhenua Library Trust (HLT) for interim an period:  60%
> Software Freedom Conservancy (Conservancy) for an interim period:  0%
> Software in the Public Interest (SPI) for an interim period:  35%
> Wait to agree on an independent foundation:  5%
> No opinion:  0%
>
> Hypothetical third rank vote.
>
> Horowhenua Library Trust (HLT) for an interim period:  5%
> Software Freedom Conservancy (Conservancy) for an interim period:  50%
> Software in the Public Interest (SPI) for an interim period:  40%
> Wait to agree on an independent foundation:  5%
> No opinion:  0%
> Hypothetical fourth rank vote.
>
> Horowhenua Library Trust (HLT) for an interim period:  5%
> Software Freedom Conservancy (Conservancy) for an interim period: 35%
> Software in the Public Interest (SPI) for an interim period:  5%
> Wait to agree on an independent foundation:  55%
> No opinion:  0%
>
> Comparing each option to the other as if many two option ballots had been
> cast could show a stronger weight for one preference in what would become
> the critical comparison.  Do you prefer HLT or "wait to agree on an
> independent foundation"?
>
>
> 5.1.1.  RUN-OFF ELECTION REMEDY.
>
> An easy remedy in the absence of an absolute majority is holding a run-off
> poll with a single two option choice.  Such a remedy would probably be the
> simplest to understand but it unfairly excludes other options which people
> may prefer without having a ballot of many options comparing each to the
> other.
>
> A ballot with many comparisons to the other can be difficult for people to
> understand.
>
> Any run-off election is an unnecessary extra poll to be conducted.
> However, some form of run-off election would be preferable to accepting a
> minority choice which merely happened to have more votes in a multi-option
> ballot.
>
>
> 5.1.2.  PURE RANKED PREFERENCE REMEDY.
>
> A ranked preference ballot could elicit that information but people should
> know that is how it would be used.  The Schulze method is a ranked
> preference analysis which might be used in comparing each choice to the
> other as if many ballots would be held with only two options,
> http://en.wikipedia.org/wiki/Schulze_method .
>
> This Schulze method is generally considered the fairest system for
> mathematically maximising people's preferences in voting.
>
> A modified Schulze method is used by the Debian project in voting but some
> of their modifications can distort the statistics to disfavour a majority
> vote in some circumstances.
>
>
> 5.1.3.  PARTIAL RANKED PREFERENCE REMEDY.
>
> Another way of treating this particular issue as it can be anticipated is
> to break the question down by type of choice.
>
> What type of foundation forming option should we use to start a foundation?
> A.  Form interim foundation while discussing an independent foundation
> following majority choice of interim home.  (Exclude option B.)  [  ]
> B.  Only wait to agree on an independent foundation. (Exclude option A.) [  ]
> C.  No opinion.  [  ]
>
> All the options including the "wait to agree on an independent foundation"
> option could be listed on the next page for ranking.  If an absolute
> majority would select A, then the largest preference for interim options
> would be selected.  The Schulze method could still be used for comparing
> the preferences.
>
>
> 5.1.4.  SIMPLEST REMEDY.
>
> A simple ranked ballot using the pure Schulze method would be easiest to
> understand and convey all preferences but it would need to be explained in
> advance how the rankings could work to reject an option which had no
> absolute majority but had the largest number of first rank preferences.
>
> 5.2.  OTHER PROBLEMS.
>
> There are other problems but I really did not even have time to present
> this one today.
>
>
> 6.  ANALYTICAL DISCUSSION.
>
>>
>> 3. how do we gather opinions on the survey design nicely?
>
> This should be a thoroughly dispassionate analytical discussion.
> Obviously something went wrong in a recent discussion which had started
> reasonably but over an issue where we have no direct control.  Having no
> direct control raises the sense of frustration.  We will all be on guard
> to avoid harshness in our discourse.
>
> Criticism should never be taken personally.  We have a great community
> which should not be afraid of discussion.  Contention and disagreement
> conducted in a civil tone helps solve problems.  Lack of discussion merely
> ignores problems without attempting to solve them.
>
> [...]
>
>
> Thomas Dukleth
> Agogme
> 109 E 9th Street, 3D
> New York, NY  10003
> USA
> http://www.agogme.com
> +1 212-674-3783
>
>
> _______________________________________________
> Koha mailing list
> Koha at lists.katipo.co.nz
> http://lists.katipo.co.nz/mailman/listinfo/koha
>