[Section 5.1 identifies an important voting preference problem which may be very likely to occur in our final poll.] Reply inline: On Tue, October 13, 2009 09:21, MJ Ray wrote: [...] 1. INFORMED VOTING.
Should we turn the wiki pages into some sort of consultation information brochure?
The wiki pages could be much easier to read in general but that is a different problem. The wiki pages will do if we use them. Before people initiate the ballot, they should be requested to examine the wiki pages and the mailing list archive to become an informed voter. [...] 2, TAKING COMMENTS INTO ACCUNT.
I suspect statistics is less obvious to most of the community than making a great Library Management System, but I think it's worth posting the ballot design and taking comments into account.
I am pleased that you are not opposed to some more input. I trust that you advise Nicole Engard well about which comments are worth incorporating and which would cause problems.
I beg everyone to check that any suggested changes have some grounding in statistical theory before posting them, else the review process probably won't work because we'll get overloaded. I wrote http://people.debian.org/~mjr/surveys.html a few years ago which might interest some people.
I really do not think that there will be very many comments overall. I doubt that many comments would even reach the level of skewing statistics. 3. IRC MEETING.
[...]
At Wednesday's IRC meeting it was announced, if I recall correctly, that two weeks would be taken to analyse the previous poll before a final ballot would be presented. That period should give sufficient time for most if not all of my suggestions to have an opportunity to have some effect on the drafting of a final ballot.
I feel that recollection is incorrect. No particular time was suggested, although I'd prefer it not to drag forever. Please check: http://stats.workbuffer.org/irclog/koha/2009-10-07#i_316162
Checking the log it is not clear to me what the two week period referred to in that meeting had been intended to cover. Certainly there was a statement of giving time for analysis of the previous results, a two week period, and statements about informing the work for the next poll. What people are commenting about precisely can be difficult to determine. From the meeting I had expected that you would report some analysis of the previous poll to everyone which could be used to inform the writing of the final ballot. Had I thought that there would be no open opportunity to inform the process I would complained loudly at the meeting. This is the trouble with ambiguous references and assuming that others understand the same intent which you mean to convey. This is why the wording of the questions and options needs more eyes to find areas of misunderstanding. 4. INFORMATION SET.
So:
1. do we need information brochure(s) about the options? - who will produce it/them? - who will referee it/them to make sure it's fair? - how should it be distributed?
We could include a simple neutral one sentence descriptive statements next to each option on the ballot. Remember that koha.org has not had a prominent statement of Koha's origins with HLT for a few years. Few librarians would know that SPI was established to provide a legal entity for free software projects starting with Debian, the largest GNU/Linux distribution. Otherwise, the wiki and mailing list record will have to do for anything more unless someone is volunteering to work extra hard. 5. STATISTICAL PROBLEMS.
2. are there other trained statisicians in our community?
You may be the only one but that does not mean that you have anticipated every problem and the best ways to resolve them. You should inform Nicole from a broader base of input. I have read extensively on voting theory and maximising preferences years ago. That does not make me a trained statistician but no one in our community with valuable input should be ignored. We should strive to not let the patches fall on the floor. 5.1. VOTING PROBLEM. The vote on first choices for forming a foundation in the previous survey had an equal division between the "independent foundation now" option and what had been presented as alternatives in the previous survey. Have you considered how we should address the possibility that an absolute majority may be in favour of a range of interim foundation choices but the largest first rank vote count may be for "wait to agree on an independent foundation"? Which foundation forming option should we choose? Hypothetical first rank vote. Horowhenua Library Trust (HLT) for an interim period: 35% Software Freedom Conservancy (Conservancy) for an interim period: 0% Software in the Public Interest (SPI) for an interim period: 20% Wait to agree on an independent foundation: 45% No opinion: 0% Hypothetical second rank vote. Horowhenua Library Trust (HLT) for interim an period: 60% Software Freedom Conservancy (Conservancy) for an interim period: 0% Software in the Public Interest (SPI) for an interim period: 35% Wait to agree on an independent foundation: 5% No opinion: 0% Hypothetical third rank vote. Horowhenua Library Trust (HLT) for an interim period: 5% Software Freedom Conservancy (Conservancy) for an interim period: 50% Software in the Public Interest (SPI) for an interim period: 40% Wait to agree on an independent foundation: 5% No opinion: 0% Hypothetical fourth rank vote. Horowhenua Library Trust (HLT) for an interim period: 5% Software Freedom Conservancy (Conservancy) for an interim period: 35% Software in the Public Interest (SPI) for an interim period: 5% Wait to agree on an independent foundation: 55% No opinion: 0% Comparing each option to the other as if many two option ballots had been cast could show a stronger weight for one preference in what would become the critical comparison. Do you prefer HLT or "wait to agree on an independent foundation"? 5.1.1. RUN-OFF ELECTION REMEDY. An easy remedy in the absence of an absolute majority is holding a run-off poll with a single two option choice. Such a remedy would probably be the simplest to understand but it unfairly excludes other options which people may prefer without having a ballot of many options comparing each to the other. A ballot with many comparisons to the other can be difficult for people to understand. Any run-off election is an unnecessary extra poll to be conducted. However, some form of run-off election would be preferable to accepting a minority choice which merely happened to have more votes in a multi-option ballot. 5.1.2. PURE RANKED PREFERENCE REMEDY. A ranked preference ballot could elicit that information but people should know that is how it would be used. The Schulze method is a ranked preference analysis which might be used in comparing each choice to the other as if many ballots would be held with only two options, http://en.wikipedia.org/wiki/Schulze_method . This Schulze method is generally considered the fairest system for mathematically maximising people's preferences in voting. A modified Schulze method is used by the Debian project in voting but some of their modifications can distort the statistics to disfavour a majority vote in some circumstances. 5.1.3. PARTIAL RANKED PREFERENCE REMEDY. Another way of treating this particular issue as it can be anticipated is to break the question down by type of choice. What type of foundation forming option should we use to start a foundation? A. Form interim foundation while discussing an independent foundation following majority choice of interim home. (Exclude option B.) [ ] B. Only wait to agree on an independent foundation. (Exclude option A.) [ ] C. No opinion. [ ] All the options including the "wait to agree on an independent foundation" option could be listed on the next page for ranking. If an absolute majority would select A, then the largest preference for interim options would be selected. The Schulze method could still be used for comparing the preferences. 5.1.4. SIMPLEST REMEDY. A simple ranked ballot using the pure Schulze method would be easiest to understand and convey all preferences but it would need to be explained in advance how the rankings could work to reject an option which had no absolute majority but had the largest number of first rank preferences. 5.2. OTHER PROBLEMS. There are other problems but I really did not even have time to present this one today. 6. ANALYTICAL DISCUSSION.
3. how do we gather opinions on the survey design nicely?
This should be a thoroughly dispassionate analytical discussion. Obviously something went wrong in a recent discussion which had started reasonably but over an issue where we have no direct control. Having no direct control raises the sense of frustration. We will all be on guard to avoid harshness in our discourse. Criticism should never be taken personally. We have a great community which should not be afraid of discussion. Contention and disagreement conducted in a civil tone helps solve problems. Lack of discussion merely ignores problems without attempting to solve them. [...] Thomas Dukleth Agogme 109 E 9th Street, 3D New York, NY 10003 USA http://www.agogme.com +1 212-674-3783