Back to previous post: Hugo IV: A New Hope

Go to Making Light's front page.

Forward to next post: Dysfunctional Families: Boundaries

Subscribe (via RSS) to this post's comment thread. (What does this mean? Here's a quick introduction.)

May 12, 2015

Discussing Specific Changes to the Hugo Nomination Election: A Post Not By Bruce Schneier
Posted by Abi Sutherland at 02:36 AM *

After a couple thousand posts, here’s the current proposal, summarized in this comment by Keith “Kilo” Watt.


- o0o -


[Remaining Issues “To-Do” List:]

  • Name of the system
  • How to handle ties
  • How to handle withdrawn nominations
  • Best way to present at the business meeting
  • The final formal proposal language itself

[Plain-Language Explanation of SDV-LPE]

Least Popular Elimination (formally called “single divisible vote with least popular eliminated” or SDV-LPE for short) is very simple and straightforward.

- You have one nomination “vote”, which we’ll call one “point” to avoid confusion.

- You can distribute that nomination “vote” among as many works as you feel are Hugo-worthy, and it will get divided among them equally. So, if you nominate two works, each gets half a point, if you nominate three works, each gets one third of a point, etc.

- All the points for each work from all the ballots submitted are added together, and the two works that got the least number of points are compared with each other. One of these works is the least popular and will be eliminated.

- For those works that are eligible to be eliminated, we compare the total number of nominations they each received (that is, the total number of times that work appeared on anyone’s nomination ballot). The work that received the fewest number of nominations is the least popular and now completely vanishes from the nomination process as though it never existed.

- We start over for the next round, and repeat the process, however, if one of your nominations was eliminated, you now have fewer works on your nomination ballot — so each one gets more points since you aren’t dividing your vote among as many works.

Ties (in Points):

- This isn’t really anything different. Basically, if there is a tie for least or second-least number of points, all of those works are eligible to be eliminated.

Ties (in Number of Nominations):

- If two or more works are tied for appearing on the fewest number of ballots, the tied works with the lowest point total will be eliminated. (In other words, if we’re tied for number of nominations, we’ll go back to comparing points for this case.)

- If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, then all members of the tie will be eliminated, unless that leaves the final ballot with fewer than five works; in that case, keep all the members of the tie. (In other words, if we’re tied in both comparisons, in general we’ll just eliminate all the tied works.)

Why does this help? In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn’t penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate -anything- you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don’t, that’s okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

How does this eliminate slates? It doesn’t, not completely — nor should it, as slate proponents have as much right to representation as any other voter. However, by nominating a full slate of works, they have weakened their votes by spreading their points out among the five works on the slate. Since the rest of fandom is slowly increasing the points given to their choices, it’s not possible for slates to control the -entire- ballot. It is very likely, however, that they will get some of the final ballots slots — and again, this is entirely fair and appropriate, since a large number of people are supporting those works. In the end, it is the final ballot that will determine the winner — and the voting system for the final winner is completely unchanged by this system. With this system, a significant fraction of the final ballot will have been determined by independent members of fandom.


[FAQ’s]

1. How does this system eliminate slate or bloc voting?

It doesn’t, exactly, nor should a work be automatically eliminated just because it appears on a slate. On the other hand, any slate which nominates a full set of five works will find that each of its nominations only count 1/5 as much. With a large enough support behind the slate (five times as much), the slate may still sweep a category; however, if that many voters support the slate, they arguably deserve to win, and no fair and unbiased system of nomination will prevent that. The answer is, simply, to increase the general pool of voters. Regardless, with SDV-LPE, slates will never receive a disproportionate share of the final ballot, as occurred in the 2015 Hugos.

2. What if there are multiple slates (slate wars, “parties”, etc.)?

As with a single slate, the more works that anyone nominates, the less their votes count for each work. The end result is that even multiple slates are unable to sweep the nominations.

3. What happens if a genuinely popular work is nominated by a group of unrelated people?

If it is genuinely popular, the system will still select that work for the final ballot.

4. What happens if a genuinely popular work also appears on a slate?

Even if it is on a slate, if the work garners support from individuals - particularly if they list it as their only nomination, or with just a few nominations - then the system will select that work for the final ballot.

5. Isn’t it true that any voting system can be gamed (or strategized, etc.)?

Yes, there is a theorem which proves that all voting systems must have inherent flaws. The objective is to choose a system whose flaws are not in an area of concern to the electorate.

6. What are SDV-LPE’s flaws?

In very rare cases, it is possible for eliminating both members of a tie to change the final ballot slightly from what it would be if the tie were broken so that only one member was eliminated. This situation was extremely rare using realistic data, so the statistical probability of how ties are handled affecting the results is extremely small.

7. What are SDV-LPE’s benefits?

Simply put, it reduces the power of bloc voting without eliminating the chance that works appearing on slates will make it to the final ballot. Conversely, it makes it very difficult for slates to prevent non-slate works from appearing on the ballot.

8. Couldn’t slates just recommend a single work for a candidate, and it will automatically appear on the final ballot?

Yes, that is certainly a viable possibility - it’s also completely fair. It does not force any other works off of the ballot, and the final Hugo winner is determined by the same voting process we have always had. Just appearing on the ballot isn’t a guarantee of winning a ballot. However, if a large section of fandom strongly believes that a work deserves a Hugo nomination, then it should, in fact, be represented on the ballot.

9. What happens with a large field with no stand-out favorites when a slate votes?

Even in this case, we were unable to find a simulation in which no non-slate works appeared on the final ballot. Slate works did receive a larger proportion of nomination slots than they did otherwise, however, again, this could be considered a fair and valid result. If there was no general favorite, then voters really had no collective preference.

10. Isn’t this system too complicated for the average voter to understand?

No, it’s actually quite simple and straightforward, both in terms of voter instructions and in how the system operates. Essentially, the total number of points for each work are totaled (and this will usually be the sum of fractional points). These points are used to determine our two candidates for elimination, since the voters felt the least strongly about them. We then look at the number times the two works appeared on any ballot. The work that appeared the least number of times must be the less popular of the two, so is eliminated. This process continues until the five finalists remain. Note that this is not that different from the STV process used in determining the final Hugo winner once the finalists have been selected.

11. I think we should just increase the number of nomination slots on the final ballot to (for example 6), and decrease the number of slots a voter can vote for to a smaller number (for example, 4). Wouldn’t that be simpler and easier?

Unfortunately, this simply means that the largest slate will receive four of the nominations and the next largest will receive the remaining two. It doesn’t solve the problem of forcing works off the ballot that had a chance to win the final election. Keep in mind that SDV-LPE will work with this (or most any other) change as well, so one does not preclude the other.

12. I think we should set up a committee to handle these situations as they occur. The committee would be empowered to add nomination slots or throw out slate-influenced ballots as required.

This could work. The problem is that now you have a small group of people who serve as literal gatekeepers to the Hugo Awards. In spite of the word on the Internet, this has never been the case in the past. Establishing it now means that those groups who believed it existed in the past will now be correct. Ultimately, human judgement is fallible. The fairness of a committee’s decisions will forever be subject to opinion. The end result is that the prestige of the Hugo Awards will forever be tarnished.

13. I think we should use [insert other mathematical voting system].

We considered essentially every applicable type of voting system currently in the literature, guided by two experts in the field. It should be kept in mind, however, that the goals and requirements for choosing a set of representatives in a political situation are different from those for choosing a set of Hugo finalists. Some of these systems do, in fact, have positive properties that speak for them. None of them were as simple or as intuitive as SDV-LPE, yet SDV-LPE meets all of the stated goals for a Hugo nomination system.

14. Won’t SDV-LPE be complicated to code and implement?

Actually, no. One of our non-experts coded a full simulator for the system in a matter of days. A full web-based app would not be much more difficult to handle.

15. Wasn’t this system just designed by Social Justice Warriors to block the Good Stuff?

It is true that much of the discussion for this system occurred on Teresa and Patrick Nielsen Hayden’s “Making Light” discussion board, and it is also true that groups such as the Sad Puppies and the Rabid Puppies consider TNH and PNH to be The Enemy, and therefore completely biased and not to be trusted. Other than serving as moderators, TNH and PNH had no real input in the discussions of the system, however. Those of us who worked on the system were very clear that our goal was not to keep the Sad/Rabid Puppies off of the Hugo ballot and that any system which specifically targets any type of work is inherently wrong and unfair. One of the members of the group is a retired US Naval officer, a combat veteran, a certified Navy marksman, a Christian, and considers Robert Heinlein to be the greatest science fiction author who has ever lived. In short, he is exactly the Puppies’ demographic. But any slate, of any sort, be it a Sad Puppy or a Happy Kitten of Social Justice, breaks the Hugo Award because a small percentage of voters can effectively prevent any other work from appearing on the final ballot. This is a major flaw in the Hugo nomination system, and it is a flaw that must be fixed if the integrity of the award is to be maintained. Politics should play no role whatsoever in whether a work is Hugo-worthy or not.


[Current Draft of Proposal Language]

Short Title: Change Nominations to Single Divisible Vote, Least Popular Elimination

Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.

3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.

3.8.3: Any nominations for “No Award” shall be disregarded.

3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.

3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.

3.8.6: The Committee shall move a nomination from another category to the work’s default category only if the member has made fewer than five (5) nominations in the default category.

3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.9 or 3.2.10, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category.

3.8.8 The final Award ballots shall list in each category the five eligible nominees as determined by successive rounds of elimination, where each nominator gets a single vote, divided equally among their nominations. For example, if only one work is nominated in the category, that nomination gets a full vote. If two works are nominated in a category, each nomination gets 1/2 of a vote. If three works are nominated in a category, each nomination gets 1/3 of a vote, etc.

3.8.8.1 In each round, the two works (or more, in the case of a tie) with the least total number of (fractional) votes will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots.

3.8.8.2 In the event that two or more works are tied for appearing on the fewest number of ballots, the tied works with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, then all members of that tie will be eliminated.

3.8.8.3 If (due to a tie) elimination would reduce the number of ballots to fewer than 5, then instead none of the works should be eliminated and all remaining works appear on the final ballot.

3.8.8.4 For the next round, votes are reassigned to each work as in 3.8.8. (Example: If a ballot contained five nominations originally and one of those nominations is eliminated, each of the four remaining nomination now gets a 1/4 of a vote)

Section 3.11 Tallying of Votes

3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, … places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes. During the same period a record of at least the last fifteen rounds of the selection process for each category shall also be published.

Submitted by: (insert members here, ideally including at least one attending member who will be at all the relevant WSFS meetings. A proposing member gets the privilege of speaking first to the pro side of the proposal)

Commentary: In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn’t penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate -anything- you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don’t, that’s okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.


- o0o -


Jameson Quinn offers:

I’ve created a public Google doc to hammer out a proposal text and FAQ for SDV-LPE.

There have been numerous proposals in this thread which do not fall under the SDV-LPE umbrella. In my opinion, they have turned out to be either off-topic or dead ends. This is not a criticism; several of the proposals were my own, and I thought and still think that they would have been good ideas. But it seems the community is very reluctant to experiment with anything that goes much beyond fixing the immediate problem, and I can respect that.

So I think the remaining order of business is to wrap up the SDV-LPE proposal, either using the doc above, or through other means.


- o0o -


Joshua Kronengold adds the following:

Re name, if we want an acronym that means something, how about:

One Notional Equally-divided Vote, with Oppositional Test Eliminations?

(ie, ONEVOTE)

I’ve always favored “rerun the vote with declined candidates eliminated, but only fill as needed” when people declined as a rule, since it ends up with the fairest overall result (and withdrawing Hugo nominations is a can of worms we never need open). I’d be fine (5 vs 10) with just taking the next item on the list.

Re tiebreakers: I’m reasonably fine with no tiebreaker (as the system was originally conceived), but also entirely fine with the current “just score” version, or using rank as a not-really-random (but close enough while being verifiable) tiebreaker. It’s probably best to just have “just score” as tiebreak despite potential issues, because it makes things much easier to independently verify and keeps things simple.

Comments on Discussing Specific Changes to the Hugo Nomination Election: A Post Not By Bruce Schneier :
#1 ::: Annie Y ::: (view all by) ::: May 12, 2015, 03:23 AM:

About withdrawn and not- accepted nominations an(because they are different things even if treated similarly) and works found ineligible after the ballot is officially declared - so essentially needing a replacement:

- Do not select 5 but instead select 15 or just run the whole elimination to the bitter end (if automated, this won't take that long). The top 5 get nominated (or more in case of ties); if someone withdraws/does not accept/is found ineligible - use the next on the list.

This way we do not need to rerun eliminations later on and a replacement is clear as soon as one is needed... and the top 15 are easily published from the Worldcon after the vote (whatever the number that is published is)...

Which brings the question - when top 15 are published, will it be based on points or based on nominations? Or maybe we need both?

That will be a bit unfair in case a work is pulled out for ineligibility (as it still can harm another) but makes it easier...

Just thinking aloud (and now copying in the other thread as the old one closed while I was typing)

#2 ::: Martin Wisse ::: (view all by) ::: May 12, 2015, 04:42 AM:

So if I nominate 1 candidate for best novel, I get one point and a slate voter who has nominated five candidates gets 1/5 point per nomination.

Okay, that makes sense. But it doesn't distinguish between people slate voting and those who naturally nominate five choices in a given category (and I could've nominated more this year). Not a problem if you assume slate voting isn't the issue, slate dominating is.

Which I'm not sure I agree with but is perhaps the best practical approach to take.

What I'm struggling with is the assumption that in lesser voted categories slate voters would be at a disadvantage because their candidates have less points. Is the assumption here that in such a situation the odds are high two slate noms would be in the least popular situation and either one of them or both are eliminated, freeing up space for non-slate votes?

#3 ::: influxus ::: (view all by) ::: May 12, 2015, 04:58 AM:

Thanks for doing this. It sounds really promising. Regarding faq 13 & 14, is it possible to release the simulations and code?

#4 ::: Brian Z ::: (view all by) ::: May 12, 2015, 05:01 AM:

Its 1970. TV won’t shut up about My Lai and Cambodia. You get up to go turn down the volume, bumping into the record player and making the first Black Sabbath album skip. Postman’s here. You forgot to lick the stamp on your ballot of Ubik by Philip K. Dick, The Black Corridor by Michael Moocock, The Andromeda Strain by Michael Crichton, Frank Herbert’s Dune Messiah and hey, it seemed kinda nifty at the time, Macroscope by Piers Anthony. Result: your five votes all go to Piers Anthony.

Not sure I can support this one.

#5 ::: influxus ::: (view all by) ::: May 12, 2015, 05:07 AM:

@2: The assumption is that slates dominate because other people's votes are too diluted across a wide field. As the works are successively eliminated everybody who still has books in the field finds their nominations converging towards a vote each on a work each. So, eventually a consensus should emerge that is capable of counter-balancing any slates. And if a consensus doesn't emerge, then it seems somewhat fair that the slate should win.

#6 ::: felice ::: (view all by) ::: May 12, 2015, 05:25 AM:

Annie Y @1: "- Do not select 5 but instead select 15 or just run the whole elimination to the bitter end (if automated, this won't take that long). The top 5 get nominated (or more in case of ties); if someone withdraws/does not accept/is found ineligible - use the next on the list. This way we do not need to rerun eliminations later on and a replacement is clear as soon as one is needed..."

The problem with that is if a popular work is withdrawn for whatever reason after the finalists are chosen, the votes of all the people who supported it are still weighted less for their other nominations, which means their second choices are less likely to be in 6th place waiting to replace it. Rerunning the process without the withdrawn work and taking the highest-ranked new work to fill the vacancy is the fairest practical option.

"Which brings the question - when top 15 are published, will it be based on points or based on nominations? Or maybe we need both?"

At a minimum, I think we'd need to publish the points and nominations for the round which contains 15 nominations; perhaps all the subsequent rounds that whittle it down to the final five, as well.

That does raise a small issue with “rerun the vote with declined candidates eliminated, but only fill as needed” - if rerunning after a withdrawal changes more than one finalist, which is possible (but probably uncommon), we'd need to publish both sets of data, and that will show who would have won a Hugo if the withdrawn work had been withdrawn before the initial selection of finalists. That could be messy. So maybe just take the next on the list is a better option, even if it's not as fair.

An example: if the winners are ABCDE, with F being the last eliminated, and B is withdrawn: the simple approach is to just promote F, for a revised ballot of ACDEF. With "rerun and fill as needed", the rerun could give us CAHDE, and a new final ballot of AHCDE (H filling the vacant spot left by B), which is fairer (the people who nominated B like H more than they like F). But the rerun could also give us CAHJK; with H as the highest ranked new result, we still get a new final ballot of AHCDE, but we know J and K should be there instead of D and E (because the votes from B were split between C, H, J, and K, improving the ranking for all of them). It's not fair to D and E to tell them "sorry, you're not finalists any more", but it's also not fair to tell J and K that they should have been finalists but aren't. Basically, withdrawing a work after the finalists have been notified is a bit of a nightmare.


Martin Wisse @2: "Okay, that makes sense. But it doesn't distinguish between people slate voting and those who naturally nominate five choices in a given category (and I could've nominated more this year). Not a problem if you assume slate voting isn't the issue, slate dominating is."

Slate voting is a problem, but trying to prevent it is a real can of worms. Slate dominating is a bigger problem, and one that it's easier to do something about. And it will probably lessen the incentive for slate voting as a bonus.

"What I'm struggling with is the assumption that in lesser voted categories slate voters would be at a disadvantage because their candidates have less points. Is the assumption here that in such a situation the odds are high two slate noms would be in the least popular situation and either one of them or both are eliminated, freeing up space for non-slate votes?"

Yes. But it's still possible for a slate to get multiple slots in a category with relatively few non-slate voters and few clear favourites (eg Short Story). SDV-LPE helps a lot, but it doesn't totally solve the slate problem on its own. But it's a good system even aside from the slate-fighting benefits.

Brian Z @4: "Result: your five votes all go to Piers Anthony."

Um... how?

#7 ::: Mike Scott ::: (view all by) ::: May 12, 2015, 05:32 AM:

Having struggled with my own proposal to change the threshold for eligibility to increase the number of nominees in categories where it looks like there's been slate activity, I support this proposal. It's better than mine. I would also lose the 5% rule, which is damaging the short story category since there are such a lot of short stories these days, but I believe there will be a separate proposal to do that. Sadly, I won't be at Sasquan to vote for it.

#8 ::: Annie Y ::: (view all by) ::: May 12, 2015, 05:33 AM:

felice@6

The problem with a rerun is calling F and telling them - "well, you are not nominated anymore because someone withdraw/we found someone else is ineligible". This is why I am for a straight "Pick the full order and stick to it". Yes - a popular choice withdrawal will move things around but do we really want to go down the "you are nominated, you are not" rigmarole? (and what are we doing if someone withdraws after the ballot is out - we change 3 places in the ballot because A decided they want out now? Or because it was found ineligible (that also can happen))? As fair as it is mathematically, it will cause way too much noise and headaches for any administrators.

#9 ::: Brian Z ::: (view all by) ::: May 12, 2015, 05:38 AM:

felice @6: Um... how?

if one of your nominations was eliminated, you now have fewer works on your nomination ballot — so each one gets more points

#10 ::: Annie Y ::: (view all by) ::: May 12, 2015, 05:42 AM:

Brian Z @9
But in your example, you did not mail your ballot at all? So you have 0 points to share around. How does a full point end up being for Anthony?

#11 ::: Brian Z ::: (view all by) ::: May 12, 2015, 05:51 AM:

Annie Y @ 10: the dog barked, jolting me out of my Toni Iommi trance, and I suddenly remembered that the stamps were sitting on the entry table. Phew!

#12 ::: felice ::: (view all by) ::: May 12, 2015, 06:06 AM:

Annie Y @8: "The problem with a rerun is calling F and telling them - "well, you are not nominated anymore because someone withdraw/we found someone else is ineligible". This is why I am for a straight "Pick the full order and stick to it". Yes - a popular choice withdrawal will move things around but do we really want to go down the "you are nominated, you are not" rigmarole? (and what are we doing if someone withdraws after the ballot is out - we change 3 places in the ballot because A decided they want out now? Or because it was found ineligible (that also can happen))? As fair as it is mathematically, it will cause way too much noise and headaches for any administrators."

Yeah, on reflection I think sticking with F is probably the only practical option. The "rerun and fill as needed" solution only delays the "you are nominated, you are not" rigmarole rather than avoiding it. Thanks for pointing that out!


Brian Z @9: "if one of your nominations was eliminated, you now have fewer works on your nomination ballot — so each one gets more points"

If Ubik, The Black Corridor, The Andromeda Strain, and Dune Messiah get eliminated before Macroscope, sure. That means more other people liked Macroscope; and it made the final ballot under the current system too, so you're hardly any worse off. If you really didn't want Macroscope to be a finalist, you shouldn't nominate it; but it's highly unlikely that leaving it off your ballot and only nominating the other four would make any difference (the last one left will be half a point higher up in the rankings, since your point is no longer split between it and Macroscope), and you'll probably end up with nothing you nominated on the final ballot. It is theoretically possible that that half point could mean the difference between 5th and 6th place for one of the other four, but it's quite improbable.

#13 ::: Annie Y ::: (view all by) ::: May 12, 2015, 06:15 AM:

Brian Z,

Great. So how does the point go to Anthony now with your ballot with 5 nominees? The only way to happen is if the other 4 get eliminated before Anthony - the same way as Anthony will get nominated under the current system only of these 4 are under it in the total ballots.

So... what am I missing in your example?

#14 ::: influxus ::: (view all by) ::: May 12, 2015, 06:32 AM:

Is it possible to handle withdrawals as follows?

Give authors a chance to withdraw when you reach, say, the final 13. Then if any do, rerun from scratch, call any new members of the final 13. When you have 13 willing nominees, finish the process.

#15 ::: Annie Y ::: (view all by) ::: May 12, 2015, 06:35 AM:

And how do you propose to deal with ineligibility found later? Or authors that decide that they changed their mind - for whatever reason?

#16 ::: influxus ::: (view all by) ::: May 12, 2015, 06:40 AM:

Annie Y,

As you suggested, keep it fixed once formally announced.

#17 ::: Brian Z ::: (view all by) ::: May 12, 2015, 06:47 AM:

felice @12

It is theoretically possible that that half point could mean the difference between 5th and 6th place for one of the other four, but it's quite improbable.

I hope it is improbable enough.

The Russians land a probe on Venus. WorldCon's done and "My Sweet Lord" is on the radio. Your next LoC goes "Ubik was trippy and all, but I mean, I'm glad I gave Piers Anthony all five of my votes. Maybe I'm the cat who got him onto the final ballot. He is every bit as good as Dick, Moorcock, and Herbert, probably better. If I could do it over I'd nominate Macroscope and nothing else. Piers Anthony is a genius and shouldn't have to compete with those has-beens. Next year I'll nominate only Ringworld. Larry Niven is practically the all-time greatest writer in history, second only to Piers Anthony." It is 2015 and you find that 'zine while cleaning your closet. Do you regret it?

#18 ::: Brian Z ::: (view all by) ::: May 12, 2015, 07:07 AM:

Self @ 17,

Come to think of it I remembered that wrong. The LoC actually said "Next year I will nominate only I Will Fear No Evil, since Mr. Heinlein literally can do no wrong."

#19 ::: albatross ::: (view all by) ::: May 12, 2015, 08:09 AM:

I love the way fandom's response to this mess is itself an SF trope--let's spin up a new technology that solves the problem.

#20 ::: J Thomas ::: (view all by) ::: May 12, 2015, 08:17 AM:

@6 felice

An example: if the winners are ABCDE, with F being the last eliminated, and B is withdrawn: the simple approach is to just promote F, for a revised ballot of ACDEF.

That is the simplest, but not much simpler and is unfair to the actual winners.

With "rerun and fill as needed", the rerun could give us CAHDE, and a new final ballot of AHCDE (H filling the vacant spot left by B), which is fairer (the people who nominated B like H more than they like F).

Yes. Of course the most likely result is that you get CAFDE and nothing is changed. But it could be different and then you get the better result.

But the rerun could also give us CAHJK; with H as the highest ranked new result, we still get a new final ballot of AHCDE, but we know J and K should be there instead of D and E (because the votes from B were split between C, H, J, and K, improving the ranking for all of them).

That could rarely happen.

It's not fair to D and E to tell them "sorry, you're not finalists any more",

But in fact they are not finalists any more. Better, when you call them to ask them whether they would decline, don't tell them they are finalists -- because after all, it isn't final. Ask them, "If it were to turn out that you were a finalist, would you accept?". Remind them that it would be premature to tell anyone they are a finalist, since it isn't true yet and might not work out. Then it isn't unfair to them in the rare case that it turns out otherwise.

but it's also not fair to tell J and K that they should have been finalists but aren't.

Yes.

Basically, withdrawing a work after the finalists have been notified is a bit of a nightmare.

It doesn't have to be. Just do the right thing, and it all works out.

That does raise a small issue with “rerun the vote with declined candidates eliminated, but only fill as needed” - if rerunning after a withdrawal changes more than one finalist, which is possible (but probably uncommon), we'd need to publish both sets of data, and that will show who would have won a Hugo if the withdrawn work had been withdrawn before the initial selection of finalists. That could be messy. So maybe just take the next on the list is a better option, even if it's not as fair.

If the winners of the last rerun are the winners, you can publish that version. Add a note that B withdrew. The more different versions you publish, the more people will be upset with the voting system after it's in use.

"If B had withdrawn 2 days earlier my favorite would have come in second, but instead it's off the ballot completely. What kind of crazy voting is this?"

Usually it won't be a problem because we'll get the same result with any of the three different methods.

But in the rare case that they're different, we do best by announcing the actual winners, and publish the numbers that made them the winners.

We can expect enough trouble already from people who look at those numbers.

"Hey, Slaughterhouse-five and Bug Jack Barron both got way more votes than Macroscope. How come Macroscope was a finalist and they weren't?"

"Practically everybody who voted for Bug Jack Barron also voted for Slaughterhouse-five and The Left Hand of Darkness. That proves they didn't care much about those, and they got part of what they wanted when The Left Hand of Darkness won. There weren't as many people who voted for Macroscope, but almost all of them voted for just that and nothing else. So it deserved to win."

It may take some people awhile to see that this is the best way.

#21 ::: junego ::: (view all by) ::: May 12, 2015, 08:31 AM:

The Google docs app isn't playing well with my iPad, so I'm putting my suggestions/votes/comments here.

I have been following this discussion almost from the beginning and want to commend and congratulate everyone who has participated. My non-expert opinion is that this is an understandable and viable fix for the nominating process. Not perfect, but probably good enough.

1) Suggest changing the explanation for the system from 'one vote (per category) that is divided' to '100 points (per category) that is divided' equally between all of a person's nominations in that category.

This is mathematically identical to SDV but may be more understandable/acceptable to the math or fraction phobic.

The current system allows up to 5 undifferentiated recommendations per category. I think translating that to 100 points per category divided equally is less jarring/more intuitive than reducing the whole category to 1 "vote" or point that is divided.

2) I also suggest that it would be better to de-emphasize "vote" language as much as possible and replace it with nomination, recommendation, choices, etc. Using the v-word may tend to stir subconscious cultural themes like "one person/one vote" combined with "hey, I don't want my vote DIVIDED!", which might add to resistance to the proposal. (I had this experience when changes were first proposed but had time to logic my way out of the knee-jerk response. There won't be as much time for people to get past this kind of thing at the business meeting.)

3) I may have missed it in the proposed explanation but in case I didn't, I suggest adding language that emphasizes that this proposal DOES NOT CHANGE what a member has to do to submit their nominations. It only changes how the counts are tallied.

Again, thanks to everyone for working so diligently on the proposal.

#22 ::: Brian Z ::: (view all by) ::: May 12, 2015, 08:37 AM:

J Thomas @ 20

"... There weren't as many people who voted for Macroscope, but almost all of them voted for just that and nothing else. So it deserved to win."

It may take some people awhile to see that this is the best way.

It may depend on what they see as the ultimate goal of the exercise.

#23 ::: Cassy B. ::: (view all by) ::: May 12, 2015, 09:12 AM:

junego @21, While I agree with all your points, I'd recommend 60 points-per-category-to-be-divided-equally, not 100. No fractions that way. One work on a ballot=60 points. Two works=30 points each. Three=20 points each. Four=15 points each. Five=12 points each.

Simple, no fractions, no rounding errors.

#24 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:20 AM:

@3: I am not the only person who has coded this up, but my code (and data) is visible at https://github.com/The-Center-for-Election-Science/HugoVotesim. I did it in R, the statistical computing environment. I realize that R is not at all the most accessible language for non-statisticians, and I'm sorry about that, but it would have taken more work to do bootstrapping in python.

I found:
1. SDV-LPE is reasonably similar to the current system; we can expect the nominee lists to overlap by about 4.5 out of 5 candidates on average.
2. If you break ties by points, then by previous round points, etc., you will essentially never have to fall back to eliminating all tied works, except early in the process for works with only a few votes each, which are going to be eliminated anyway. "Essentially never" means less than 1 in 1000 elections.
3. If somebody declines a nomination, running the system again from scratch will change one of the other 4 existing nominations only about 1 time in 100.

#25 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:38 AM:

@20: I don't think we should talk about Macroscope hypothetically "winning", but just about it getting a nomination. In the scenario you paint, it's pretty clear that The Left Hand of Darkness (or something else non-Xanthan) would get the actual Hugo. So Piers Anthony would get to go to the Hugo losers party, but that's all. (In fact, I suspect that Vonnegut and Spinrad would get invited too, because you can never have too many losers at the party, especially when we're talking losers like them.)

#26 ::: influxus ::: (view all by) ::: May 12, 2015, 09:39 AM:

Jameson Quinn @ 24:

Thank you, that's awesome.

Hey R is powerful, fit for purpose and free. I'm not complaining.

It would be good (for someone) to put a web app up (somewhere) if this proposal is getting broad acceptance as a viable reform.

#27 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:45 AM:

@23: Must... resist... bringing up my more complicated proposal...

Oh, what the hell.

From a voting-theoretic point of view, it would be even better if one work got 20 points, two got 8 points each, three got 4 each, four got 2 each, and 5 got 1 each. These numbers no longer add up to the same thing, but they approximate the Saint-Lagüe weights, and thus give better resistance to naive slate voting.

But that's hard to explain, so I think that your 60 point idea is better as a serious proposal.

#28 ::: J Thomas ::: (view all by) ::: May 12, 2015, 09:45 AM:

@23 Cassy B.

I'd recommend 60 points-per-category-to-be-divided-equally, not 100. No fractions that way. One work on a ballot=60 points. Two works=30 points each. Three=20 points each. Four=15 points each. Five=12 points each.

Simple, no fractions, no rounding errors.

That looks very good. My only concern is that while it works perfectly and it makes good sense, we are trying to come up with a description that will make sense to people who are not paying attention.

We have seen earlier that people who are not interested in the details prefer simple wrong explanations.

So our challenge is to find a simple overview which is not wrong, but which superficially looks so obvious that people don't notice that there are important details they are missing.

Maybe the important points to get across are:

1. It's entirely a process of elimination. We remove losers until there are no losers left to remove.

2. Each loser is selected because it had fewer votes than another nominee.

3. Nominees that have more shared ballots get more opportunities to be removed.

Since slates are maximally shared, they get lots of chances to be removed and it's harder for them to dominate the final results.

I'm not clear how to make a correct superficial short description of this, and I'm not certain it's what needs to be said.

#29 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:49 AM:

@24: "1. SDV-LPE is reasonably similar to the current system; we can expect the nominee lists to overlap by about 4.5 out of 5 candidates on average."

I meant, "on average with historical, mostly-slate-free voting patterns". If there are slates, the slates would get much less traction with SDV-LPE and so the overlap could be as small as 1.

#30 ::: Brian Z ::: (view all by) ::: May 12, 2015, 09:53 AM:

@ 25 Jameson Quinn

Spinrad would have been unable to get an invite to the losers' party whether Piers Anthony knocked him off the ballot or not, because it didn't exist yet.

If A Spell for Chameleon had knocked off George R.R. Martin's Dying of the Light in 1978, there might not be be a losers' party today.

#31 ::: J Thomas ::: (view all by) ::: May 12, 2015, 09:59 AM:

@25

In the scenario you paint, it's pretty clear that The Left Hand of Darkness (or something else non-Xanthan) would get the actual Hugo.

In reality, The Left Hand of Darkness did get the actual hugo, and Macroscope came in third. Slaughterhouse-Five and Bug Jack Barron came in fourth and fifth.

I think this was before Xanth. Piers Anthony put his heart into writing Macroscope and people didn't like it that much. Then he found he could crank out Xanth books and get a large steady income, and he settled for that.

I don't know whether any of those would get on the short list the new way, it would depend on the individual ballots.

#32 ::: junego ::: (view all by) ::: May 12, 2015, 10:11 AM:

Cassy B. @ 23
junego @21, While I agree with all your points, I'd recommend 60 points-per-category-to-be-divided-equally, not 100. No fractions that way. One work on a ballot=60 points. Two works=30 points each. Three=20 points each. Four=15 points each. Five=12 points each. Simple, no fractions, no rounding errors.

Sure, that works for me.

#33 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:17 AM:

Hi all, and welcome to the new voices!

I'm really glad to get some more input on the proposal and the system. I'm diligently taking notes on everyone's thoughts and suggestions. Completely optional, but if you want to make sure I don't miss a suggestion, you could put a header at the beginning of your message like [TIES], or [DELCINED NOMINATIONS] or whatever. Totally up to you, and probably not necessary for general conversation.

I wrote the code for the current simulator, but I'm not much of a web programmer or database specialist (I was a NASA engineer, so spacecraft programming I grok). I'd be glad to work with anyone who wants to make a web app though -- I can provide the algorithm if someone else can handle the back end. I don't think it's a high priority unless the proposal actually passes, though.

Until that time, I'd be glad to run any situations that people might be curious about. We now have the 1984 Hugo ballots that we can use as a base, plus some random data based on 2013. As Jameson mentioned, the system not actually all that sensitive to small changes -- usually you get the same results unless things were very close to begin with (which makes sense in any case). It is possible to manufacture unusual situations, so one of the previous discussions was engaged in how much we should worry about them. Jameson's simulations showed that those situations are quite rare indeed.

I'll respond to specific questions in another message.

Thanks,
Kilo

#34 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:22 AM:

For a name: how about Proportionally Allocated Pairwise Elimination (PAPE)? It's pronounceable, descriptive, and short. It's a bit technical-sounding, but that's not necessarily a bad thing.

Whichever name you favor, please vote on name, tiebreaker, and declined-nomination replacement procedure. We're using range voting for this poll, so rank each option 0-10. You can vote either here or there, as long as you give some recognizable nym with your vote there.

Here are the options again:

Name:
Single Divisible Vote, Least Popular (SDV-LP)
Single Divisible Vote, Least Popular Elimination (SDV-LPE)
Single Divisible Vote (SDV)
Least Popular Elimination (LPE)
Hybrid Divisible Vote
Quantum Divisible Vote
Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
Bottom Up Elimination (BUE, pronounced “byoo”)
One Notional Equally-divided Vote, with Oppositional Test Eliminations (ONEVOTE)
Proportionally Allocated Pairwise Elimination (PAPE)

My vote (edit with your ratings):

9: (SDV-LPE)
5: (SDV)
7: (LPE)
7: Hybrid Divisible Vote
3: Quantum Divisible Vote
5: (BUMPE)
5: (BUPE, pronounced “boopy”)
9: (BUE, pronounced “byoo”)
7: (ONEVOTE)
10: PAPE

....

Tiebreakers:

(T0) Eliminate all tied works (current version)
(T1) Break ties by score
(T1a) Scores from previous rounds may be used if the current round is not decisive
(T2) Random.
(T3) Earliest Ballot
(T4) Some other ranking form.

My vote:
10: T1a then T0.
5: anything else nonrandom
0: anything random

....

Declined nominations:

Options:


(DN1) Take the 6th place item.
(DN2) Recalculate the election without the declined entry and those are the new nominees. (Full recalculate)
(DN3) Recalculate the election without the declined entry, and add the first winner that is not yet on the ballot. (Incremental recalculate)

My vote:

10: DN1, 6th place
0: DN2, Full recalculate
5: DN3, Incremental recalculate

#35 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:27 AM:

ah, crap. I said this informal poll was using "range voting", I should have said "score voting". They're the same thing but the latter term is preferred these days. I sometimes forget.

#36 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:28 AM:

Regarding declined nominations:

For each round, the code reports the current surviving works, their current point total, and the number of nominations they received. It then reports which works were eliminated that round, along with their point total and number of nominations. It makes it pretty easy to see how the process went.

My intent for section 3.11.4 is to just publish the last fifteen rounds (which will generally also have the last 15 works that were elminated, barring ties). In this way everyonje can see how we eventually converged to a final set of works. I think this actually is easier and more informative than just listing the top 15 works as we do now. Of course, I personally think the whole text file of the process from beginning to end should be released, but that's up to the administrators and is what the "at least" language is for.

I don't see any inherent issues with just taking the works from the previous rounds. Some here think that is less fair, but consider that the voters' decisions were made assuming the declined work was actually an option. We have no way of knowing how they would have nominated if that work wasn't available, so I don't see any advantage to re-running the process, even in the very rare cases that it might make a difference.

As I read the current constitution, section 3.9.1 just says that the work is removed from the final ballot -- there is currently no explicit requirement to add the next work to the list, so I believe the administrators have the option to handle it however they want. Perhaps one of the experienced admins can confirm that. My thoughts are to just leave that part of the constitution unchanged. It's simpler, one less change to pass the business meeting, and pretty much solves all the issues. I would recommend to any Hugo admin to just take the work eliminated in the previous round, but it seems to me the constitution lets them do as they see fit.

Kilo

#37 ::: junego ::: (view all by) ::: May 12, 2015, 10:29 AM:

Jameson Quinn @ 27
@23: Must... resist... bringing up my more complicated proposal... Oh, what the hell.
From a voting-theoretic point of view, it would be even better if one work got 20 points, two got 8 points each, three got 4 each, four got 2 each, and 5 got 1 each. These numbers no longer add up to the same thing, but they approximate the Saint-Lagüe weights, and thus give better resistance to naive slate voting.

Yeah, I understand what your getting at but from a NOMINATOR standpoint I don't necessarily want to order my choices this early in the process. I'm mostly just recommending the coolest things I read/saw/heard last year. I could just nominate fewer items in the category to express preferences, but that doesn't satisfy your "theoretical" standpoint. 😋

But that's hard to explain, so I think that your 60 point idea is better as a serious proposal.

I don't think it's that hard to explain. IIRC, the main complaint in the previous discussion was more along the lines of my above. There is and was support for ranking nominations, but it seems to be less than those just wanting to nominate without.

#38 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:31 AM:

Jameson@34: By the way, my phone and tablet don't like Google docs much (it runs, just not in an easy-to-read fashion for aging eyes), so I would actually prefer that people post here, if that's okay. It makes it easier for me (and I would suspect others as well) to stay on top of what's happening. I do agree that it would streamline changes and make note-taking easier, but I'm often squeezing in time as I can.

Kilo

#39 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:33 AM:

junego@37: I think the concensus was that people want to rank the final voting, but that for nominations, we essentially want a controlled version of "throwing a name into a hat". Basically nominate anything that people think are Hugo worthy. The final voting then ranks which of those are the best.

Kilo

#40 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:40 AM:

@38: Would Google Docs work for you if we increased the font size? Those of us not on tablets could easily deal with a larger font.

#41 ::: DonBoy ::: (view all by) ::: May 12, 2015, 10:40 AM:

Delurking:

In FAQ 8, I think

Just appearing on the ballot isn’t a guarantee of winning a ballot.

is supposed to be

Just appearing on the ballot isn’t a guarantee of winning a Hugo.

#42 ::: Niall McAuley ::: (view all by) ::: May 12, 2015, 10:42 AM:

Keith, one wording issue. I think it would be better to re-order the first two points to emphasize that nominations are in the same form (for the nominator) as they are now. Many other possible systems require that nominators list nominations in preference order, and I think a strength of this proposal is that it does not:

- You can nominate as many works as you feel are Hugo-worthy, in the same way as you can nominate 5 works today, as an unsorted list.

- You have one nomination “vote”, which we’ll call one “point” to avoid confusion, and it will get divided among them equally. So, if you nominate two works, each gets half a point, if you nominate three works, each gets one third of a point, etc.

#43 ::: Jim Van Verth ::: (view all by) ::: May 12, 2015, 10:42 AM:

@20:

Having seen first-hand the stress that Hugo and Campbell nominees go through during awards season, I think calling people up and asking them "If you were given a nomination, would you accept?" is darn cruel, especially if you're that 1 in 100 person who got dropped off the list when the algorithm is re-run. Sure, it's rare, but it can happen.

As previously suggested, I rather see the algorithm run once to create a ranked list, and work down it when people decline. It may not be 100% mathematically fair, but it's kinder.

#44 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 10:43 AM:

[DECLINED NOMINATIONS] I strongly prefer the simple way of handling these: if ABCDE are nominated and then E withdraws, leave ABCD alone and promote F. My reason for preferring this is as follows:

The election (for nominees) is an algorithm that takes a pile of preferences in voters’ heads, translates those preferences into a pile of ballots, and translates those ballots into a ranked list of works in each category (with the first-eliminated candidates at the bottom of the list). Once that list is generated, the election is over and we should handle declined or disqualified nominees without regenerating the list.

It’s like, if Clinton and Bush run against each other in a Presidential election, and Clinton wins, but then says “screw this, I don’t want to be President after all”, the voters don’t get to run back to the polls and try again.

Also, if ABCDE are told they are the nominees, and E declines, and the algorithm is rerun, and D gets kicked off the revised ballot, then... I don’t want to see what happens when the authors of D and E happen to be in the same room.

#45 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:45 AM:

@Actually, it's 1 in 100 times, so that's 1 in 500 nominees. But yeah, I still agree with your point.

#46 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 10:55 AM:

I don't really have anything substantial to add, just a personal perspective: I have learning disabilities, thanks to neurological side effects of an autoimmune condition, and this is genuinely impenetrable to me. Not "this is a lot of work to understand", but "I am not able to grasp these details after any amount of effort in a short period of time". Maybe with years of repetition it would sink in, and maybe not.

(If I'd studied these things before that neurological damage developed, I could run on memories and suitably adaptive pathways. But I seem not able to learn new stuff involving a bunch of different kinds of math.)

So I'm in the position of having to go entirely on trust of others. That's scary and uncomfortable. Nothing about this is easy or good for my sense of security. I hope that when advocates work to round up support, you will remember that people like me exist, along with others who aren't going to quickly - or maybe ever - master the math for themselves, either.

I don't understand how this system doesn't end up rewarding single-choice voters throughout. It seems like that would be the sensible thing to do for anyone wanting Puppies-like domination: forget the "rule the nominations" part and just focus laser-like on a single pick in every category and carry the day. And it's deeply distressing that I can't make sense of any explanation about why that wouldn't happen, or why I shouldn't be bothered if it does.

#47 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 10:56 AM:

[WORDING] A phrase I would like to see, somewhere in the explanation or FAQs, is “the greatest good for the greatest number”. Relative to the current system, SDV-LPE increases the number of voters who can expect to see at least one of their preferences on the final Hugo ballot. (It decreases the number of voters who can expect to see more than one of their preferences on the ballot, but I think that’s a fair trade-off.)

#48 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 10:56 AM:

A phrase I would like to see, somewhere in the explanation or FAQs, is “the greatest good for the greatest number”. Relative to the current system, SDV-LPE increases the number of voters who can expect to see at least one of their preferences on the final Hugo ballot. (It decreases the number of voters who can expect to see more than one of their preferences on the ballot, but I think that’s a fair trade-off.)

#49 ::: junego ::: (view all by) ::: May 12, 2015, 10:59 AM:

Jameson Quinn @24

I found:
1. SDV-LPE is reasonably similar to the current system; we can expect the nominee lists to overlap by about 4.5 out of 5 candidates on average.
2. If you break ties by points, then by previous round points, etc., you will essentially never have to fall back to eliminating all tied works, except early in the process for works with only a few votes each, which are going to be eliminated anyway. "Essentially never" means less than 1 in 1000 elections.
3. If somebody declines a nomination, running the system again from scratch will change one of the other 4 existing nominations only about 1 time in 100.

Thank you for the stats.

I'm not sure what you mean at 2) about "break ties by points, then by previous round points"?

Your 3) finished convincing me that the new system should freeze the finalist list when the public announcement is made and any further withdrawals should be filled by either a) having frozen the entire list and taking next in line without recalculating or b) recalculating only for the "6th" spot. The nightmare (and "unfairness") of throwing a finalist off the list after the public announcement outweighs the "unfairness" of whoever might have replaced them if someone else hadn't withdrawn.

If the rule is clearly stated it will make the choices unambiguous. Well, hopefully...this is fandom after all 😆

#50 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 11:02 AM:

(sorry for the double-post above)

Re Bruce Baugh’s comment, I am wondering if someone with the appropriate skills can describe SDV-LPE in a graphic format, for the benefit of people who find pictures clearer than words.

#51 ::: Tim Illingworth ::: (view all by) ::: May 12, 2015, 11:07 AM:

Um. I was working through the proposed language to make it more consistent with the rest of the Constitution, and the first sentence of 3.8.8.2 puzzles me:

In the event that two or more works are tied for appearing on the fewest number of ballots, the tied works with the lowest point total will be eliminated.

But we're already looking at works with the same point total - that's how we picked them in 3.8.8.1. What am I missing?

#52 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:12 AM:

A diagram would definitely be helpful for some of us, yes. Thank you, Seth.

#53 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:12 AM:

@46: Thanks for adding your point of view.

The answer is "that wouldn't happen", not "that would actually be OK". The simplest explanation I can come up with is below. I realize it may not be simple enough. But explanation or no explanation, we have simulations to show that it wouldn't happen in any elections that are even vaguely Hugo-like. So I hope that if you can't trust the explanations, you can trust the sims, or trust the people trying to explain it.

Here's my explanation: Looking at actual Hugo ballots, we see that most voters only vote for 1 or at most 2 of the eventual nominees. In fact, the average is less than 1. That means that, by the end of the process, ordinary nonstrategic voters end up giving the same "points" to the works they like as your hypothetical bullet voters, even in the "matchup" (two lowest points) phase.

Furthermore, from the very beginning of the process, strategic and nonstrategic votes count equally in the "showdown" phase (eliminate whichever has fewest approvals). In fact. insofar as your hypothetical bullet voters had factional divisions, they would actually be at a disadvantage, not an advantage, over honest voters.

So bullet voters have no advantage when it counts, and even when it doesn't matter, they actually have disadvantages along with advantages.

I hope that some part of that is clear enough to help. If not, sorry; it's the best I can do.

#54 ::: junego ::: (view all by) ::: May 12, 2015, 11:14 AM:

Annie Y @8
felice@6
The problem with a rerun is calling F and telling them - "well, you are not nominated anymore because someone withdraw/we found someone else is ineligible". This is why I am for a straight "Pick the full order and stick to it". Yes - a popular choice withdrawal will move things around but do we really want to go down the "you are nominated, you are not" rigmarole? (and what are we doing if someone withdraws after the ballot is out - we change 3 places in the ballot because A decided they want out now? Or because it was found ineligible (that also can happen))? As fair as it is mathematically, it will cause way too much noise and headaches for any administrators.

I agree, Annie Y. Disenfranchising publicly announced finalists because :math!: would be a PR nightmare and would be perceived as completely unfair.

From Jameson Quinn's post @24 it would be an uncommon problem anyway, but if there's a clear rule about freezing (at least) the finalist list after the public announcement, if (and when) that uncommon problem crops up, there's clear criteria to resolve it.

#55 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:16 AM:

Jameson, that is helpful. I feel some nagging concern yet that I'll see if I can articulate later, but it helps.

#56 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:19 AM:

@51: we're looking at the two works with the lowest point totals, but one of them still may be lower.

@49: After eliminating ZYXW, say works U and V are tied with 10 points and 20 approvals each. Look at how many points they had before you eliminated W; say, 10.167 each. Then, look at how many they had before eliminating X. Aha! Now U is ahead with 10.667 to 10.167. So eliminate V. The point is this is almost certain to be decisive except in the very early eliminations (ie, Z vs Y), when both works are doomed anyway. So if this is not decisive, it's OK to just eliminate both.

#57 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:24 AM:

aak, my numbers were a bit backwards in the UV example @56. That is to say, in earlier rounds, they would have had fewer points, not more. The basic idea still stands.

#58 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:25 AM:

I will note one quick presentation thing, for the sake of folks who are dyslexic or have any of a wide variety of other learning impairments: single characters are much easier to confuse than any longer string of characters. If someone writes up examples, please consider doing it with 3- to 5-letter (or so) expansions. Alpha, Beta, Gamma, Delta, and so on are very, very much easier for many of us to reliably, accurately parse than ABCD.

#59 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:26 AM:

Bruce@46
I don't understand how this system doesn't end up rewarding single-choice voters throughout. It seems like that would be the sensible thing to do for anyone wanting Puppies-like domination: forget the "rule the nominations" part and just focus laser-like on a single pick in every category and carry the day. And it's deeply distressing that I can't make sense of any explanation about why that wouldn't happen, or why I shouldn't be bothered if it does.

It can indeed happen that if the slates focus on a single work, then they will probably get it on the final ballot. There are two reasons why that's okay. First, the whole point of this proposal is not to prevent slate works from getting on the final ballot. The purpose is to keep them from getting all five slots on the final ballot. When they get all five slots, they prevent any other work from having even a chance to win, and that's what is not fair about slates.

The second reason is that getting a work on the final ballot doesn't mean they will win the Hugo. The regular voting system is unchanged. They can present their one candidate (in this case), and the voters can decide if they like it better than the other candidates that were presented.

Now, if your concern is in the voting process itself, that's a separate issue and no change to the nominating system will affect that. However, I don't think that will be a problem either. The only reason that slates work is because fandom's interests are so varied. They end up nominating a lot of different works. But on the final ballot everyone is looking at the same five works, so that effect is gone. Slates will have a much harder time forcing a final win because the rest of fandom isn't as widely distributed anymore.

Does that help?

Kilo

#60 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:33 AM:

@59: "It can indeed happen that if the slates focus on a single work, then they will probably get it on the final ballot."

Yes. But it will not generally happen that a slate can get something on the ballot by bullet voting, unless they could also have gotten it on the ballot by voting more broadly.

There is one exception to this general rule. If there are two slates, and neither bullet votes, it is possible for the stronger slate to eliminate the weaker one entirely. In that case, the weaker slate could probably avoid that outcome by bullet voting. But this scenario is highly unlikely (less than 2% likely at worst) even in elections with 2 competing slates, so I think it's not worth worrying about.

#61 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:36 AM:

@60: ...and even if that scenario did occur, the bullet voting strategy would only restore the "fair" (proportional) result; it wouldn't give an unfair advantage.

#62 ::: Brian Z ::: (view all by) ::: May 12, 2015, 11:36 AM:

@59 Keith "Kilo" Watt

There are two reasons why that's OK.

Does everyone find this OK? I believe the purpose of the nomination process is to shortlist five works in each category that are identified as being among the best of the year by the most voters. The goal is to honor and reward excellence, not to feel fair or make as many people happy as possible.


#63 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:36 AM:

Kilo: That's bound to help someone else - I mean that; I love that kind of exposition - but I'm thinking at the moment of the final ballot, mostly. I don't like the idea of a cabal/slate being able to dominate the nominations, but I also don't really like the prospect of comparable effort being able to push one of the 5 (or 6, or whatever) final nominees to the win, either. I was unclear about that earlier, for which my apologies.

To be clearer, then:

So we have a final ballot. It's got 5 entries, give or take. If I'm understanding things, voting for 2 or more entries to be Best Thing spreads your point's worth of influence around, while voting for just 1 concentrates it all on that one pick. So if the United Jerks Of Flunkyvania focus on the one work in each category that Dear Leader has picked, it seems like they won't just get it on the final ballot, they'll likely get a win, unless there's some substantial strong convergence on a single other candidate by Everyone Else.

I'm not at all sure I like that. I'm not at all sure it's avoidable under any circumstances. I am sure that I can't assess the relative chances of such a thing working under the proposal versus under existing rules, nor if it's something I should be much concerned about.

#64 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:37 AM:

[DECLINED NOMINATIONS]

All: Is there a consensus to just take the work(s) that were eliminated in the previous round in the event of a declined or ineligible nomination? If so, do we want to explicitly add this to the proposal or leave the constitution as it is now and just make it a recommendation to the Hugo admins?

Kilo

#65 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 11:38 AM:

If someone writes up examples, please consider doing it with 3- to 5-letter (or so) expansions. Alpha, Beta, Gamma, Delta, and so on are very, very much easier for many of us to reliably, accurately parse than ABCD.

For a graphic presentation, would icons (rocket, blaster, propeller beanie, star, dragon) be more or less confusable?

#66 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:41 AM:

Seth, combining some good iconography with short text is probably ideal for reaching the widest range of folks.

#67 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 11:44 AM:

So if the United Jerks Of Flunkyvania focus on the one work in each category that Dear Leader has picked, it seems like they won't just get it on the final ballot, they'll likely get a win, unless there's some substantial strong convergence on a single other candidate by Everyone Else.

The final Hugo election involves preferential voting. If the majority of Hugo voters put the Flunkyvania choice at or near the bottom of their list of preferences, then it will lose, even if the United Jerks all rank it on top. (This is basically what happened to the Sad Puppy 2 candidates last year.)

#68 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 11:45 AM:

@64 [DECLINED NOMINATIONS] Best to make that rule explicit, I think.

#69 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:47 AM:

Seth@67: OK. This is the point where I'll wait for some exposition/examples that I may be able to follow, but I'm certainly willing to accept it as a true-looking answer.

#70 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:48 AM:

Bruce@63

Ah, so we're talking about the final election. This system doesn't change the final election at all. It's only for the nomination process, so the two are completely separate things. There aren't any votes to spread around under the current voting system.

Here's how the current voting system works once the final ballot is determined, hopefully someone will correct me if I've described something poorly.

In the current voting system (NOT the nomination system we are discussing in this thread), you take all five works on the ballot and you rank them order of preference (1 for your favorite, 2 for your next favorite, etc.) If you aren't familiar with one or more works on the final ballot, it's okay to just not rank them and leave them off. This doesn't give your higher-rank candidates any help, however. You also have the option of voting for "no award". This means that any work ranked above "no award" you consider to be Hugo-worthy, but anything below "no award" is not.

So, in short, what we're doing here won't change that. If slates voted for their candidate and put "no award" as their second choice, then the only thing they can do is say, "this one should win or nothing". Fandom can disagree, though, and there's nothing a slate can do about that.

Kilo

#71 ::: Vicki ::: (view all by) ::: May 12, 2015, 11:50 AM:

Bruce--

If I read this right, the proposal only changes how works get onto the ballot. The voting process after that is unchanged.

Once Alpha, Beta, Gamma, Delta, and Upsilon are on the list, everyone gets to rank them as we do now. Whether I vote "Alpha, Beta, Gamma, No Award, Upsilon, Delta" or just vote for Alpha and leave the rest of that section unranked, my whole vote goes to Alpha in the first round of counting.

#72 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:51 AM:

Kilo@70: Ah! Genuinely, I missed that this isn't supposed to change the final ballot round at all. That does make a lot of difference. Thank you.

#73 ::: P J Evans ::: (view all by) ::: May 12, 2015, 11:57 AM:

The current method of dealing with refused nominations, or withdrawn works, is to pick the next on the list.
felice needs to explain WTF we should run the entire process over again, when we have an ordered-by-popularity list already available.

(BTW: this isn't anything new. There was a nomination refused in 1984, as well as a nominee ruled ineligible, and both were handled in exactly this way.)

#74 ::: influxus ::: (view all by) ::: May 12, 2015, 11:58 AM:

@64 [DECLINED NOMINATIONS] Yes. This sounds sensible. @43 was very persuasive.

#75 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 12:02 PM:

OK. The open questions were naming, ties, withdrawals, and language/explanation/etc. It seems that we're coming to consensus on withdrawals. I think that my simulations show that we have a tiebreaker that works; to wit, use current round scores, then previous round scores, then the one before that, etc; and if you get all the way back to first round scores without breaking the tie (which would never happen in cases that matter), eliminate both.

So the remaining questions would be, name and language/explanation. Let's resolve naming. I like PAPE (Proportionally Allocated Pairwise Elimination). What do others think? (It seems my idea of a score voting poll is not catching on, so I'll drop it, unless somebody else picks it up.)

#76 ::: junego ::: (view all by) ::: May 12, 2015, 12:03 PM:

Here are my rankings for the proposed system names with 10 as highest and 0 as lowest:

0 - Single Divisible Vote, Least Popular (SDV-LP)
0- Single Divisible Vote, Least Popular Elimination (SDV-LPE)
0 - Single Divisible Vote (SDV)
2 - Least Popular Elimination (LPE)
0 - Hybrid Divisible Vote
0 - Quantum Divisible Vote
10 - Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
9 - Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
8 - Bottom Up Elimination (BUE, pronounced “byoo”)
0 - One Notional Equally-divided Vote, with Oppositional Test 0 0 - Eliminations (ONEVOTE)
5 - Proportionally Allocated Pairwise Elimination (PAPE)

#77 ::: influxus ::: (view all by) ::: May 12, 2015, 12:09 PM:

P J Evans @ 73: I think you'll find felice backed picking the next on the list @12. (Now that I've fully read and followed Annie Y and felice's conversation - sorry about earlier).

#78 ::: john, who is incognito and definitely not at work ::: (view all by) ::: May 12, 2015, 12:17 PM:

Sorry if this has been posted already--I missed it if so--but would it be possible to see how this sort of nomination system would affect the ballot in various years (both with and without slate voting, e.g. 2015, 2010, 2005, 2000, etc.)?

#79 ::: Tim Illingworth ::: (view all by) ::: May 12, 2015, 12:18 PM:

OK: I've added a possible revise of 3.8.8 to the Google Doc, which makes things more consistent with the rest of the Constitution.

#80 ::: influxus ::: (view all by) ::: May 12, 2015, 12:23 PM:

BUMPE is clearly the best acronym and avoids the slight propagandist overtones of ONEVOTE. However, I find Proportionally Allocated Pairwise Elimination more descriptive of the system.

So:

1 - Single Divisible Vote, Least Popular (SDV-LP)
1- Single Divisible Vote, Least Popular Elimination (SDV-LPE)
1 - Single Divisible Vote (SDV)
1 - Least Popular Elimination (LPE)
1 - Hybrid Divisible Vote
1 - Quantum Divisible Vote
7 - Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
3 - Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
3 - Bottom Up Elimination (BUE, pronounced “byoo”)
0 - One Notional Equally-divided Vote, with Oppositional Test 0 0 - Eliminations (ONEVOTE)
8 - Proportionally Allocated Pairwise Elimination (PAPE)

#81 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 12:24 PM:

Jameson@75:

That's not actually the way it is coded now. What I found was that looking at previous rounds never made a difference in the 1984 ballots, all the way back to the first round. So the tie-breaking scheme that is currently coded only looks at the current points for the current round. Basically if two or more works are tied for number of nominations, then eliminate the one with the fewest points in that round. If they are tied for number of nominations and number of points in that round, eliminate them both.

I actually think that's simpler to manage and seems to have the same effects. We need something like this no matter what since every tie I found went all the back to the beginning.

Kilo

#82 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 12:25 PM:

Tim@79: Would you mind posting your revision here as well?

Thanks,
K

#83 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 12:33 PM:

[NAME]

I actually don't really like any of the new names, I'm afraid. I think SDV-LPE is a good formal description, since it encapsulates what's actually going on: It's a two-part system which has a vote-distributing part and then a popularity-elimination part. I only suggested "Least Popular Elimination" as a short title, because I think we need a non-technical sounding moniker for discussion. I sort of don't think it's the best choice for an official name, though. What are the thoughts on having a formal name and a descriptive name?

Kilo

#84 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 12:57 PM:

@83: So post your ratings of the names. It's hard for us to turn your description of how you feel into numbers.

As for having two names... I don't think that should be in the proposal, but clearly if you say SDV-LPE anybody with google is going to be able to figure out what you're talking about if they really want to.

#85 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 12:59 PM:

@81: In my bootstrap, I found that it was rare that previous rounds mattered, but that it happened often enough to be worth doing. Basically, most ties happen early on, when there aren't a lot of previous rounds to go on; but the ties that matter happen later, and in those cases, previous rounds are decisive if the current round isn't.

#86 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:14 PM:

[NAME]

0 Single Divisible Vote, Least Popular (SDV-LP)
10 Single Divisible Vote, Least Popular Elimination (SDV-LPE)
0 Single Divisible Vote (SDV)
5 Least Popular Elimination (LPE)
0 Hybrid Divisible Vote
0 Quantum Divisible Vote
0 Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
0 Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
0 Bottom Up Elimination (BUE, pronounced “byoo”)
0 One Notional Equally-divided Vote, with 0 Oppositional Test Eliminations (ONEVOTE)
0 Proportionally Allocated Pairwise Elimination (PAPE)

K

#87 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:21 PM:

Jameson@85:

Okay, it pretty dramatically increases the data storage requirements, since right now the code just dumps the results of the current round to a text file and forgets about it (that's one advantage of the system: eliminated works vanish as though they never existed). That might not be such an issue with a web app + database combination, though. That's outside of my expertise.

Can you give some examples of where it made a difference?

Thanks,
Kilo

#88 ::: Annie Y ::: (view all by) ::: May 12, 2015, 01:25 PM:

Jameson Quinn@27

That would be a change in 3.7.1 as well and we are sticking to the 3.8.x changes (for now anyway) :)

More seriously - if something is so much less worth than something else, what is it doing on your ballot? The idea is to nominate the things you like and would not mind being a winner (or at least nominated) - with the margins we have even now, a single vote can tip the balance somewhere. With a weighted nomination, you are saying what you think is a better work...

@Niall McAuley @ 42
But we are still restricted to 5 (because of 3.7.1)? Your first bullet point sounds as if you are saying that you can go over 5 if you find more than 5 worthy.

@Jim Van Verth @43

Not to mention that in case of withdrawls, anyone that does not make the nominations after that will presume that it is because of whoever withdrew. Which as someone mentioned above - I really do not want to be in the same room with anyone not making the list and the guy that withdrew. :)

@64 [DECLINED NOMINATIONS]I think we should add it to the rules - or down the road someone will decide to complain from the administrators because they decided to rerun this one time.

#89 ::: Tim Illingworth ::: (view all by) ::: May 12, 2015, 01:26 PM:

KW@82: Sure. It's not that long...

3.8.8 The final Award ballots shall list in each category the five eligible nominees as determined by successive rounds of elimination.

(1) In each round, each nomination ballot is allocated a single point, divided equally among its surviving nominations.

(2) The current point total for each surviving nominee is then determined and the two nominees (or more, in the case of a tie) with the lowest point totals will be compared.

(3) Of those nominees, the one that appears on the fewest number of ballots will be eliminated.

(4) If two or more nominees are tied for appearing on the fewest number of ballots, the tied nominee with the lowest point total will be eliminated.

(5) If there is a tie for lowest point total as well as for appearing on the fewest number of ballots, then all tied nominees will be eliminated.

(6) If (due to a tie) elimination would reduce the number of nominees to fewer than 5, then none of the nominees will be eliminated and all remaining works will appear on the final Award ballot.


#90 ::: Duncan J Macdonald ::: (view all by) ::: May 12, 2015, 01:29 PM:

[NAME]

My list is shown below.

0 - Single Divisible Vote, Least Popular (SDV-LP)
10- Single Divisible Vote, Least Popular Elimination (SDV-LPE)
0 - Single Divisible Vote (SDV)
0 - Least Popular Elimination (LPE)
0 - Hybrid Divisible Vote
0 - Quantum Divisible Vote
0 - Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
0 - Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
0 - Bottom Up Elimination (BUE, pronounced “byoo”)
0 - One Notional Equally-divided Vote, with Oppositional Test 0 0 - Eliminations (ONEVOTE)
0 - Proportionally Allocated Pairwise Elimination (PAPE)

Please note that 'cutsey' pronunciations can be a douple-edged sword. For example, I can see, if PAPE is chosen, that the SP/RP Alliance will claim that the modifications are out to RAPE them.

Having waded through all the earlier discussions, using the term-of-art SDV-LPE makes web-searches easier.

#91 ::: Annie Y ::: (view all by) ::: May 12, 2015, 01:29 PM:

[NAME]
Do we really need a clever and/or mathematical name? Call it the Hugo Nominations System (or Algorithm)(and add explanation for the mathematicians if you must) but over-complicating with an acronym does not seem necessary

Plus the way thing works, someone will come up with an acronym at some point that will be cleverer than anything we can figure out now and that will stick despite what we put in the constitution.

#92 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:30 PM:

Brian@62:
The goal is to honor and reward excellence, not to feel fair or make as many people happy as possible.

Someone can correct me if I'm wrong, but I don't think that's actually the goal of the Hugos. The Nebulas, maybe. The Hugos are looking for fan favorites, not a measure of excellence. I don't think there's a real way to evaluate excellence with the Hugo process other than "A work is excellent if it was the one most people liked."

Kilo

#93 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:34 PM:

Jameson#40:

Would Google Docs work for you if we increased the font size? Those of us not on tablets could easily deal with a larger font.

I don't know. I suppose it can't hurt to try. But I think there's something to be said for keeping the discussion "out in the open" (I do realize the Google doc is public, but it's not a conducive place for discussion). It's completely true that we could post changes there and discuss them here, but that extra step is time-consuming. What would you think about waiting for the Google doc until we get to the actual proposal language crafting?

Kilo

#94 ::: junego ::: (view all by) ::: May 12, 2015, 01:34 PM:

Keith "Kilo" Watt @ 39
junego@37: I think the concensus was that people want to rank the final voting, but that for nominations, we essentially want a controlled version of "throwing a name into a hat". Basically nominate anything that people think are Hugo worthy. The final voting then ranks which of those are the best.

Ayyup, that's pretty much what I said. 😏

#95 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 01:36 PM:

[NAME]

I vote 10 for Hugo Nominations System (no acronym). No change to my previous votes.

#96 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:41 PM:

junego@94: Yes, sorry, in reading your quote of what I said, it's not clear that I was agreeing with you. :) I was just trying to reinforce your point with the group.

K

#97 ::: junego ::: (view all by) ::: May 12, 2015, 01:44 PM:

Jameson Quinn @ 56

@49: After eliminating ZYXW, say works U and V are tied with 10 points and 20 approvals each. Look at how many points they had before you eliminated W; say, 10.167 each. Then, look at how many they had before eliminating X. Aha! Now U is ahead with 10.667 to 10.167. So eliminate V. The point is this is almost certain to be decisive except in the very early eliminations (ie, Z vs Y), when both works are doomed anyway. So if this is not decisive, it's OK to just eliminate both.

Thanks, that's clear enough even with the slightly confused earlier point totals.

#98 ::: Brian Z ::: (view all by) ::: May 12, 2015, 01:53 PM:

Kilo @92

Then wouldn't they have called the categories Most Popular Novel, Most Popular Short Story, etc.?

Is the consensus here that rewarding excellence has never been a goal of the Hugos?

#99 ::: Annie Y ::: (view all by) ::: May 12, 2015, 01:57 PM:

Brian Z @98

Because most people will vote what they think is best. Which is why it is most popular basically - the best work may not win if enough people find something else to be the best (for some values of best). Finding undisputed best in a subjective field is impossible. Which is why allowing more people's nominations votes is crucial - it will push up the works that most people think are best

At the end of the day, the awards had chosen a lot of good works through the years.

#100 ::: J Thomas ::: (view all by) ::: May 12, 2015, 02:00 PM:

@46 Bruce Baugh

I don't understand how this system doesn't end up rewarding single-choice voters throughout. It seems like that would be the sensible thing to do for anyone wanting Puppies-like domination: forget the "rule the nominations" part and just focus laser-like on a single pick in every category and carry the day.

The way it is now, what slates do is like bullet-voting five different times with the same votes.

With the new rules, slates will get the same result as bullet-voting once, or probably twice, or maybe three times. It's only in categories that don't get a lot of votes or where the votes are especially scattered that the slate can win four times.

So it's an improvement.

It doesn't come close to solving the problem but it can be a big help.

#101 ::: J Thomas ::: (view all by) ::: May 12, 2015, 02:12 PM:

@49 junego

The nightmare (and "unfairness") of throwing a finalist off the list after the public announcement outweighs the "unfairness" of whoever might have replaced them if someone else hadn't withdrawn.

Do they make the announcment before all the authors have confirmed?

That's terrible! In that case it couldn't be OK to change any nominations.

@43 Jim Van Verth

Having seen first-hand the stress that Hugo and Campbell nominees go through during awards season, I think calling people up and asking them "If you were given a nomination, would you accept?" is darn cruel, especially if you're that 1 in 100 person who got dropped off the list when the algorithm is re-run.

Doesn't it depend? How about if it was "Because of our time pressure and the looming deadline, we are calling the first and second runner-ups to see if they would accept the nomination, in case someone else declines. Would you accept the nomination?"

Hugos are stressful, win or lose. But some writers will use the experience to write a great story.

#102 ::: CHip ::: (view all by) ::: May 12, 2015, 02:13 PM:

[ELIMINATION]: I'm unclear why the bottom two nominees by points should be compared for number of ballots instead of doing a straight elimination. Consider: work Bravo is nominated by 6 people who haven't had any choices eliminated yet (=> 1.2 points), while Charlie is nominated by 5 people who have already had a work eliminated (=>1.25 points); if I parse the rules correctly, it's "Sorry, Charlie!" despite Charlie having more points. It is unclear to me that this matches what I see offered as a feature, that nominating less-popular works doesn't hurt the chance that \something/ a nominator likes will end up on the final ballot.
      I gave up on the primal discussion some time ago, so links to the reasons for this tweak to simple elimination would be helpful if people want to be brief. I'd also be interested in the results of simulations, if any; i.e., did this tweak change the final results any? (I know the above numbers are simplified.)

#103 ::: Annie Y ::: (view all by) ::: May 12, 2015, 02:16 PM:

Because you want the work that had been nominated from more people to actually have a better chance.

This way in a case of Guy 1 moninating only A and Guys 2-4 all nominating 5 each, with B being one of them, in a direct A vs B, the work that actually had more nominations wins even if it has lower fractional points.

#104 ::: Pfusand ::: (view all by) ::: May 12, 2015, 02:25 PM:

How do slates do in votes-then-points relative to points-then-votes? About the same?

#105 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 02:36 PM:

Chip@102:

#796 in the previous thread has the results of SDV-LPE compared to the actual '84 Hugo results. Note that this was using an "eliminate all tied works" instead of the current "look at the points again in a tie" system. #843 has the what changed under the new system.

I'd be happy to run any variations on the actual ballots (one slate, two slates, someone drops out, etc.) that you might be curious about as well. The tie system that Jameson has proposed would require a fairly large re-work of the code, which is doable, but only after I finish grading final exams (so likely this weekend at the earliest).

Kilo

#106 ::: qforzy ::: (view all by) ::: May 12, 2015, 02:38 PM:

(Lurking lurker who has heretofore always lurked... rather nervous about decloaking now)

Brian Z @98

There was a phrase somebody* used in the last thread that I liked a lot: "the works most acclaimed by the fans" (with "fans", in context, being shorthand for members of the relevant WorldCon).

That makes excellence part of the equation (so far as the understanding is that fans will acclaim worthy works), but it's excellence in the eyes of the membership in all their multifarious glory. So "people being happy" and "perceiving the process as fair" also matter. If the final ballot is not broadly representative (as much as it can be!) of member opinion, then the award is losing the crucial "by these fans" element that makes it what it is. Hence, the focus on pushing slate-voter influence back to proportionate levels, not eliminating it entirely (which, separately, people have argued may be impossible anyway without terrible side effects).

I don't want to speak for others. This is only what I've taken from following the discussion. I offer it in case it's helpful, and am happy to be countered by those who are closer to being primary sources than me!

* Sorry to whoever it was for not going back to search... Tablet-posting is my excuse

#107 ::: Brian Gibbons ::: (view all by) ::: May 12, 2015, 02:45 PM:

Brian Z @98: Is the consensus here that rewarding excellence has never been a goal of the Hugos?

... and as soon as you come up with an objective measure for excellence, I'd love to take a look at it. Until then, we'll have to make do with proxies.

The current system could produce a ballot that includes works that 20% of the nominators thought were the second through fifth best works, while leaving out entirely works that 10% of the nominators thought were the best.

If we want to find the most excellent work, I think you have a better chance of doing so with a final ballot comparing a work thought by 20% to be the best vs. a work thought by 10% to be the best, rather than comparing a work thought by 20% to be the best with a work thought by that same 20% to be the fifth best.

#108 ::: Lighthill ::: (view all by) ::: May 12, 2015, 02:50 PM:

Recommendation from a programmer who sometimes makes mistakes: run the data through two or more independent implementations of the algorithm. Require that the results match.

#109 ::: junego ::: (view all by) ::: May 12, 2015, 02:59 PM:

J Thomas @ 101
@49 junego
The nightmare (and "unfairness") of throwing a finalist off the list after the public announcement outweighs the "unfairness" of whoever might have replaced them if someone else hadn't withdrawn.

Do they make the announcment before all the authors have confirmed?
That's terrible! In that case it couldn't be OK to change any nominations.

IIRC 1 or 2 were disqualified after this year's announcement, and of course 2 withdrew after (but hopefully these will remain very rare occurrences), but they still happened.

Doesn't it depend? How about if it was "Because of our time pressure and the looming deadline, we are calling the first and second runner-ups to see if they would accept the nomination, in case someone else declines. Would you accept the nomination?"

Personally, I think this is still cruel and stressful. It also might increase the possible leak of information.

#110 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 03:00 PM:

I have a meta-suggestion about naming: choose something that will turn up related work if someone does a Google search on it. It will be reassuring to research-minded people looking at the Hugos to find some prior art and other related implementations, as a buffer against the worry that this is basically a bunch of autodidacts playing Tom Swift And His Amazing Dean Drive Voting Machine.

Whatever name most readily connects to related work is going to be the best one at doing some self-propelled informing and reassuring.

#111 ::: Annie Y ::: (view all by) ::: May 12, 2015, 03:07 PM:

J Thomas @ 101

The announcement is after the authors had been notified but the notification itself means "yes, you made it". As little as these awards may mean to some, for some authors this is what will probably make their year.

We are talking real people here - some of them quite used to awards, some of them new to the field. To get a call saying "would you accept a nomination if we extend one" is just cruel. And not how the awards had worked.

Can it change? Probably. But I would not want to me the administrator that makes the "sorry, you did not get nominated" calls - because once you call for a "just checking", you need to call both winners and losers from that pool. Plus when the results become final, do you really want to see the guy that withdrew and the guy that would have made the ballot if he had not? Gaiman withdrawing knowing that the next on the ballot makes it; Gaiman withdrawing knowing that the ballot may change and he effectively can knock out someone from the ballot is a different story. And people did vote for Gaiman after all - him refusing under the new system (if we rerun) is not exactly a normal withdrawal.

#112 ::: J Thomas ::: (view all by) ::: May 12, 2015, 03:11 PM:

@73 P J Evans

The current method of dealing with refused nominations, or withdrawn works, is to pick the next on the list.
felice needs to explain WTF we should run the entire process over again, when we have an ordered-by-popularity list already available.

What's different, is that now it can make a difference.

The way we did it before, #6 will still have more votes than #7. If #3 drops out, there's obviously no better choice than #6. We had our answer and there was no other answer.

But with this voting system things change. Kind of like IRV for the finals -- if somehow the final winner was disqualified after the vote, there's no particular reason to think that the person who came in second would be the winner if you counted the votes over. The fifth-place guy might have been #2 on every single vote for the winner, but none of those counted for him because those votes counted only for the winner. Without the disqualified guy in there gumming up the works, the result might be very different.

Of course, for the nominations it usually won't matter. In recent years at most 2/3 of the nominators nominated any of the top 5, and more likely it was half the nominators who had any in the top 5 and less than 20% had two. About the same numbers spread among the next ten out of the top fifteen. It usually just won't make much practical difference, except to slates.

As a moral issue I think it's something that reasonable people can disagree about. Precedent doesn't rule because the new voting system changes this around. Whatever we decide now, fans WILL disagree about it later, maybe in a specific case where it makes a difference.

So it seems to me to be a good idea to make a strongly defensible choice now. Or else fuzz it over and let people decide later when it turns important.

#113 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 03:13 PM:

@104: The problem we're trying to solve is when a slate gets the top 5 vote counts. Votes-then-points wouldn't even consider a single member of such a slate for elimination until the very last elimination. So it would get 4 of 5 slots. So that doesn't work.

@102: Comparing them in votes is like asking "if everyone who had voted for either of these works, had voted for only that, which would win?" In other words, it makes it safer not to bullet vote. The fewer people bullet vote, the more non-slate votes we have, the better slate resistance the system has.

For all: imagine a world where each work had an intrinsic quality, and everyone votes based on their slightly-fuzzy perceptions of those qualities. That is, where any differences in taste are purely idiosyncratic, and there are no groups of people with systematic differences in taste from other groups. In that simplified world, the order of points and the order of approvals would be the same at each round; the points would climb as works were eliminated, but they'd do so essentially proportionally, so that if Alpha has twice the approvals of Beta, it would always have about twice the points too.

So in that world, using the two stage system here, with a matchup based on points and a showdown based on approvals, is just a matter of belt-and-suspenders "double robustness".

Now, obviously, the real world is not like that; there are real correlations and anti-correlations between voters in matters of taste. But with honest voters, you'd actually be surprised how weak those correlations are. Yes, we can all point to the one example of two people who enjoy utterly non-overlapping sets of works; but it's really much more common that if you give two random books to any two fans, they'll be able to agree which one is better. I mean, two really random books: I just went on Amazon and got "Swords Against Death
by Fritz Leiber" vs. "Tactics Of Mistake by Gordon R Dickson", neither of which I've read, but you get the idea. So sure, there will be correlations, but the "first principle component" of undeniable book quality will dominate the other, lesser dimensions that are more controversial matters of taste.

So the only time that points and approvals will seriously disagree is in cases of slates. Which is exactly what the system is designed for.

What that means in terms of strategy is that a strategy which focuses on points (ie, bullet voting) or one that focuses on approvals (ie, broad voting) generally won't work, because it will only move one of the two dials, and the other one will continue to give the same results. So this is one reason this system is generally strategy-resistant. (Although no system can ever be perfectly so.)

tld̦r: In general, using both points and approvals is just a way to double-check. It's only when slates are involved that it becomes important.

#114 ::: Brian Gibbons ::: (view all by) ::: May 12, 2015, 03:17 PM:

This might be a bit nit-picky, but is there a reason why this proposal is to add a 3.8.8 rather than just replace 3.8.1? 3.8.2 through 3.8.7 read better to me as additions to the nominating process rather than coming first. (And if you do want this section to appear at the end, the proposal should re-number the preceding paragraphs.)

3.8.6 and 3.8.7 assume that you can only make five nominations, whereas the explanation implies that this system allows for as many nominations as you want. The proposal itself omits any mention of whether there's a limitation on nominations or not, which I'm assuming means that no such limitation exists, but that seems like the sort of thing that should be called out explicitly (given that one exists now, and a non-trivial number of voters are likely to just assume it still will) and the other sections made to conform (3.8.6 and 3.8.7 seem entirely superfluous to me, with unlimited nominations).

On a bit of a broader nit-pick, the explanations seem pretty heavy on selling this proposal to someone who thinks that slate voting is a horrible problem to be fixed, and explaining (multiple times) how this change will solve the slate voting problem.

What about the member who thinks that slates are a temporary aberration, and is against any permanent solution to what they consider a temporary problem? Such members do indeed exist, and I would expect them to be solidly represented at the Business Meeting. Not only do these explanations not sell this proposal to such a voter, they would actually cause them to pigeon-hole it as an overreaction to slates and something to be opposed.

I'm assuming that this FAQ is going to be used as a reference for people to link to as the proposal starts getting wider circulation. I'd suggest that the explanation lead off with the benefits of this proposal over the current system, even in a world without slates: That (as I understand it) it increases the chance that a given nominator will see at least one of their nominations appear on the final ballot, and tries to produce nominees with a broad range of support.

If I were trying to sell this, I'd probably focus less on slates and use the perhaps more neutral example of BDP Short Form, since the situation and impact is essentially the same (replacing a system where one TV show could theoretically crowd everything else out with one in which a popular TV show would still get at least one episode on the final ballot, but is less likely to have the entire ballot to itself).

[NAME]
As to the name, I'm not sure I'm seeing the necessity of the proposal having one at all, beyond something generic like "Hugo Nominations System Improvements". It's useful to be able to throw around acronyms in discussions, as shorthand so everyone knows what systems are being referred to, but I think you're past that point now.

The purpose of a proposal title is (a) to quickly sum up the proposal for someone unfamiliar with it, and (b) to perhaps influence the mindset a member starts out with as they're reading your proposal. "Single Divisible Vote" has little meaning beyond making someone wonder if the proposal is about limiting voters to a single nomination per category; "Least Popular Elimination" does little beyond making such a member think about gym class dodgeball games.

The sort of member to whom titles like that are meaningful are likely already part of the discussion; as an introduction to the proposal for someone coming to this topic fresh, the only thing such a title means is "*technobabble*". I'd keep the more technical name in the explanations, but leave it out of the proposal title.

#115 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 03:20 PM:

@110: I am as versed in voting theory as anyone, and while I'd definitely say that this system is an essentially conventional voting system, it really combines aspects of at least 4 other systems, so it would be hard to make a single name that strongly harked back to all the relevant prior work. So I think that calling it "hugo nomination system", then making that a page on the Electorama wiki with all the relevant background, is the best we can do for google-friendliness.

#116 ::: P J Evans ::: (view all by) ::: May 12, 2015, 03:40 PM:

112
See, this is where you're not selling it.
The current system's problem is not in how it handles withdrawals and disqualifications, it's how it handles slates in nominating, and you're insisting on fixing the non-problem.

Don't break stuff that's not broken.

#117 ::: J Thomas ::: (view all by) ::: May 12, 2015, 03:44 PM:

@102 CHip

[ELIMINATION]: I'm unclear why the bottom two nominees by points should be compared for number of ballots instead of doing a straight elimination. Consider: work Bravo is nominated by 6 people who haven't had any choices eliminated yet (=> 1.2 points), while Charlie is nominated by 5 people who have already had a work eliminated (=>1.25 points); if I parse the rules correctly, it's "Sorry, Charlie!" despite Charlie having more points.

You could get a pretty good voting system doing it the other way. We got a pretty good system doing it this way. By some standards (the standards several of us chose) this way is better.

It is unclear to me that this matches what I see offered as a feature, that nominating less-popular works doesn't hurt the chance that \something/ a nominator likes will end up on the final ballot.

It's a trade-off. If you want to maximize the chance that something you vote for wins, you should pick the five that you think are most popular.

If you want to maximize the chance that one particular work you care about more than anything else will win, then you should just vote for that one.

If you want to nominate the best works you can think of, independent of strategies to win, you can't go wrong doing that. Each thing you nominate is more likely to win if you nominate it than if you don't.

In general, if you vote for something that's popular and something that isn't, your vote will help the popular one that doesn't need it more than it helps the unpopular one. It's hard to design a voting system that goes the other way round.

Jameson has found that more than 90% of the time, (without slates) the winners are the same as we used to get. I haven't studied that issue but my simulations are compatible with that result.

So when you nominate something that isn't in the top 5 in vote-getting, it will probably lose. And yes, it does hurt your other chances some. If other people voted only for it, it could last long enough to get your more-popular works eliminated. That can happen. It just isn't likely. More likely, your vote for something which is not popular will have not matter at all, just as it would not make any difference whatsoever in the old system. Unless you have misunderstood and it's more popular than you thought.

It's like this: If you nominate something that is unpopular, it will probably get eliminated quickly and have no effect on anything. Similarly anything else you vote for will have little effect on that one -- it's almost surely going to lose.

But if you nominate something that's on the edge of fifth place, you might hurt it by voting for any of the other popular works, particularly the ones that are *more* popular.

If you like two works that are competing for fifth place, it's possible that by voting for both of them you help each of them less than half as much as you'd help just one. That looks like it would be hard to test. It would depend on picky details. Chances are you'll never know precisely enough just how close to fifth place they both are, to predict that sort of thing. And the effect will usually be small.

What *is* clear is that every work you vote for, will do better than if you hadn't voted for it.

#118 ::: Richard Gadsden ::: (view all by) ::: May 12, 2015, 03:48 PM:

[DECLINED NOMINATIONS]
One possible approach would be to rerun the counting procedure, but to add a rule that already-announced nominees can't be eliminated (ie they automatically win a showdown, and if a showdown would otherwise involve two already-announced nominees then you back it out, and eliminate the work with the third-lowest number of points).

That way, you still get to use the system properly (e.g. if there are two works with similar voters and one is withdrawn, then the other will replace it, which wouldn't happen if you just take sixth place), but you don't get the annoying effect of telling someone they've been nominated and then snatching it away and giving it to two other people.

#119 ::: J Thomas ::: (view all by) ::: May 12, 2015, 03:54 PM:

@116 P J Evans

112
See, this is where you're not selling it.

You mean, I'm not doing a good job of selling my own preferences? I'm honestly not certain what's best. I would feel dishonest trying hard to push one solution. Maybe I ought to do that to balance the people who are dishonestly trying to push an alternative, but homey don't play that.

The current system's problem is not in how it handles withdrawals and disqualifications, it's how it handles slates in nominating, and you're insisting on fixing the non-problem.

The current system has an obvious best way to do this. The new system does not. We need to have a strong justification for whatever solution we choose, because it's likely to become a great big issue at some random time we can't predict yet, and now is the time we might head that off. (Of course, very likely by the time it goes bad people will have forgotten it was us that handed them that slimebucket. So it's understandable if you want to consider it Somebody Else's Problem.)

#120 ::: Richard Gadsden ::: (view all by) ::: May 12, 2015, 03:55 PM:

With this system, there's no need to limit the number of works a nominator can nominate - you could list 20 and it wouldn't be a problem.

#121 ::: Idumea Arbacoochee, Gardener of Threads ::: (view all by) ::: May 12, 2015, 04:09 PM:

(Note that I've moved most of the OP behind a jump text because it was...rather long.)

#122 ::: J Thomas ::: (view all by) ::: May 12, 2015, 04:14 PM:

@111 Annie Y

But I would not want to me the administrator that makes the "sorry, you did not get nominated" calls - because once you call for a "just checking", you need to call both winners and losers from that pool.

Yes, certainly you do.

Plus when the results become final, do you really want to see the guy that withdrew and the guy that would have made the ballot if he had not?

I am unclear about the emotional parts of this. If it was clearly understood that getting called in case you might win, does not mean you have won, then I just don't know. I imagine I would feel good to hear that I was at least a second runner-up, and disappointed if it turned out I wasn't on the short list, and I'd prefer that to never hearing anything. But I wouldn't really know how I felt unless it happened. (Which seems pretty unlikely!)

In another context we discussed calling people early when they were in the top 15. Everybody who makes his intentions clear at that point is a help -- if they decline then you can announce it while people are still voting. Of course, it's up to 3 times as much work to contact 15 people as to contact 5, even if it is on a more relaxed deadline.

I think it would help a lot to put out a public plea that authors who want to decline, do it early enough that fans can change their votes.

Gaiman withdrawing knowing that the next on the ballot makes it; Gaiman withdrawing knowing that the ballot may change and he effectively can knock out someone from the ballot is a different story.

In any case, the sooner he withdraws the less problems he makes. And he's very unlikely to knock someone off the ballot. We are discussing this because if we do the wrong thing that has rare consequences, those rare consequences can sneak up and bite somebody once.

As a practical approach, I think the easiest thing is to choose the top 15, contact the top 5, contact others as needed, and immediately destroy the ballots. If anybody asks anything about it, announce that all results are final and say no more. If they can't prove there's a problem, then there's no problem.

To me it seems obviously wrong, but it will usually get the right answer. And if it does go bad, and we stonewall well enough and nobody can get any evidence, then it will all blow over.

#123 ::: Joshua Kronengold ::: (view all by) ::: May 12, 2015, 04:16 PM:

Bruce @46: You've gotten a lot of responses to this, but I don't think anyone has managed to articulate two important points on why bullet voting isn't a dominant strategy in SDV-LP

First: Consider a non-bullet ballot, which because it voted for five different things, can expect to see at least four of them eliminated. As such, at some point, it will have all but one nominee (the strongest) eliinated -- and thus will be competing with the bullet ballots on entirely equal grounds. Because any ballot that doesn't have multiple entries make the final list will eventually become its strongest nominee and thus compete with a bullet ballot on equal grounds, listing fewer nominees doesn't strengthen your ballot against anything except the other nominees you would have otherwise put on your ballot.

Additionally, because victory in eliminations is based not on current votes, but overall popularity, bullet voting can be a serious disadvantage. It is possible that bullet voting a nominee will result in it never having to face a standoff. However, if the numbers aren't high enough for that test, a candidate that got mostly bullet votes will, all things being equal, lose to a candidate that got mostly shared ballots -- because they are pulling from a smaller pool of ballots. If there are three candidates, A, B, and C, facing elimination with equal numbers of adherents, and C's voters only like C, but A's voters like B and vice versa, then A and B will each handily beat C, although if they end up in a showdown against one another, one of them will (naturally) lose -- leaving a very strong candiate left at the end.

Re naming:


0 - Single Divisible Vote, Least Popular (SDV-LP)
9- Single Divisible Vote, Least Popular Elimination (SDV-LPE)
2 - Single Divisible Vote (SDV)
0 - Least Popular Elimination (LPE)
0 - Hybrid Divisible Vote
0 - Quantum Divisible Vote
4 - Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
0 - Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
0 - Bottom Up Elimination (BUE, pronounced “byoo”)
8 - One Notional Equally-divided Vote, with Oppositional Test 0 0 - Eliminations (ONEVOTE)
7 - Proportionally Allocated Pairwise Elimination (PAPE)
10 - Don't name the voting system in the proposal.

I like ONEVOTE -- it's funny and hackery, and the spelled out version (however obviously contrived ["It means someone really wanted it to spell out 'one vote']) spells out the actual point of the system.

PAPE is a well done pronouncable acronym that efficently describes the system.

But of the names, SDV-LPE is in fact the name agreed upon by its originators (I should know), and SDV successfully calls back to STV in terms of the point of the system and basic logic.

However, we don't ever spell out that we use STV in the constitution, and I don't see any reason to specify a name for it here either except maybe in the commentary. Better to describe it, and let people use whatever names they like as long as they're all speaking the same language.

#124 ::: Joshua Kronengold ::: (view all by) ::: May 12, 2015, 04:33 PM:

Oh, also: I've moved my own opinion over time, and now think that we should just calculate a single list of 15 and promote from within.

1. It's easy to describe what's happening, and we don't need a lot of text to show what happened when we have a lot of withdrawals.

2. I don't anticipate that finalist hopefuls will be eliminated that early in the process, so it's not unlikely that something eliminated by a competitor will get another chance if there are a lot of declines.

3. There are obviously cases where eliminating one item would seriously change the ballot, Displaying an entirely different list of nominees and only taking the first one would create a lot of confusion in post-ballot noise.

4. Additionally just like promoting #6 can be a bit unfair, there are cases where calculating a new list of 5 can result in unfair results (removing #1 results in Seanan story #3 coming in first instead of Seanan story #1, so both stories make the ballot). So I don't see a good argument to using a system that still has a possibility of concentrating votes and does make the system far harder to understand. At least this way you make sure the votes are pretty well divided up, even if it's possible some will be eliminated when their candidate withdraws.

Brian Gibbons #114: There's not really a good way to replace section 1, which was my early draft didn't do so. Also, none of the current proposals drop the 5 card limit; there are good reasons to keep it even without it breaking nomination per se, and it's best to move one dial at a time.

Rewarding excellence is totally a goal of the Hugos, but that's a product of and goal of empowering voters. The rules are mostly there so the voters get the most say on what they think is excellent.

#125 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 04:42 PM:

Joshua@123: Thank you, that is helpful - I can look back at earlier discussion and see the substance of it, but something about your exposition clicked with me.

#126 ::: J Thomas ::: (view all by) ::: May 12, 2015, 04:53 PM:

@78 john

Sorry if this has been posted already--I missed it if so--but would it be possible to see how this sort of nomination system would affect the ballot in various years (both with and without slate voting, e.g. 2015, 2010, 2005, 2000, etc.)?

Basicly, no. We have enough data to do that for one year, 1984. More than 30 years ago, back when a dedicated fan could almost read everything in contention that year. Things have changed.

We have some summary data for more recent years, but not the whole thing. I have done simulations using random data that is not too different from recent years. It sort of fits the given statistics for recent years. I think it's good enough for some purposes. You can tell the difference between common problems and rare problems, but not estimate just how common things are.

It would be possible to do that better, but I prefer to use distributions that are good enough, and firm them up if some particular purpose requires it. When I put too much effort into that I may get dogmatic about how good it is, and believe it too much, and so the extra time I spend on it is worse than useless.

I put a link to it on my URL line above, done in Python.

An ad hoc model could fit better, and might be worth doing later.

#127 ::: Cadbury Moose ::: (view all by) ::: May 12, 2015, 04:58 PM:

On Naming, this moose would also prefer to have it called by its proper name (SDV-LPE) rather that either a cutesy acronym or (worse still) something it actually isn't.

(The Hugo (or is it TAFF/GUFF) voting method is frequently incorrectly called "The Australian System", as I recall.)

#128 ::: felice ::: (view all by) ::: May 12, 2015, 05:03 PM:

[DECLINED NOMINATIONS]

Richard Gadsden @118: "One possible approach would be to rerun the counting procedure, but to add a rule that already-announced nominees can't be eliminated"

Actually, that's an idea - rerun and extend the ballot if necessary. If the winners on the first run are: Alpha, Beta, Gamma, Delta, Epsilon;
Then Beta is withdrawn, and the rerun gives us Gamma, Alpha, Sigma, Tau, Omicron;
We get a new final ballot with 7 finalists: Alpha, Gamma, Delta, Epsilon, Sigma, Tau, Omicron.

We already allow extending the ballot for a multi-way tie for 5th place, and in practice it's probably pretty rare for rerunning after a withdrawal to eliminate any of the other original finalists (and the new finalist may even often be Zeta/F). It's theoretically possible for this to result in a ridiculously large final ballot, if the original finalists included a 10-way tie for fifth, and the rerun produced a complete different set of finalists including another 10-way for 5th, but the chances of that actually happening are infinitesimally tiny (if the Hugo keeps running for a few million years, we still probably won't encounter such a situation).

P J Evans #73: "The current method of dealing with refused nominations, or withdrawn works, is to pick the next on the list. felice needs to explain WTF we should run the entire process over again, when we have an ordered-by-popularity list already available."

Because the current list isn't ordered by popularity. If B is withdrawn, and F is moved up to replace it, what if everyone who nominated B hates F, and most of them like G? Under the current system, G would already be a finalist, because it has more nominations than F, but under SDV-LPE, the vote is split between B and G, resulting in G getting eliminated earlier. If B had never been in the running, G would always have been a finalist.

#129 ::: Annie Y ::: (view all by) ::: May 12, 2015, 05:32 PM:

J Thomas @ 122

Noone is even hinting of improper treating - we all are discussing how to make it as open and accountable as possible

felice @ 128
Yeah reruns with guaranteed places will work I suspect. Either version is unfair to someone - any withdrawal could have meant someone nominating differently if they knew not to bother. So we are looking for the least unfair solution (that does not cause too much grief).

#130 ::: Joshua Kronengold ::: (view all by) ::: May 12, 2015, 05:55 PM:

Bruce@125: Thanks, glad it helped. Jameson actually hit both of the exact same points in #53, but I think my explanation was a bit simpler and more focused.

#127: It's worth noting that the "actual name" is very new, coming out of the previous thread (as the concept was invented by JQ riffing off a base idea I came up with). The concepts it's built upon are solid and well-explored (SDV is more or less another way of modelling RAV, for starters), but this particular way of putting them together is as far as anyone here knows original to Making Light.

#131 ::: felice ::: (view all by) ::: May 12, 2015, 06:06 PM:

Annie Y @129: "Yeah reruns with guaranteed places will work I suspect. Either version is unfair to someone - any withdrawal could have meant someone nominating differently if they knew not to bother. So we are looking for the least unfair solution (that does not cause too much grief)."

Well, everyone can already nominate up to four other works they think are Hugo-worthy, so they've already got their alternative choices in the running and nominating differently shouldn't be a big issue. I suppose you might get some people choosing not to nominate their second favourite work because it's too similar to their favourite, and nominating four more diverse works instead, but that's a dubious strategy - if fans of the first two books all do that but disagree over which is better, it could mean neither wins. And on the other hand, if the second book is widely agreed to be inferior to the withdrawn book, does it really deserve a Hugo even if it is popular? And there's nothing we can do about it anyway.

In short, there's no point worrying about how people might have nominated differently.

#132 ::: Annie Y ::: (view all by) ::: May 12, 2015, 06:11 PM:

:) Was just thinking aloud and going through variants in my head (and keyboard).

As long as
1. People are not contacted before they are actually nominees
2. A rerun cannot kick someone off the ballot once they have a space on it

I am fine with either flat single rerun or secondary reruns with no ability to remove someone.

There is no perfect system (and I think we all are acknowledging that)

#133 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 06:33 PM:

JT@122:

These are some of the more offensive intimations you've made so far. I do realize you aren't accusing anyone of doing any of these things directly, but it is simply completely false that we are trying to accomplish anything like the things you suggest. As Annie says, the goal is just the opposite: to make the system as open and accountable as possible.

You made a number of comments of this nature in the previous thread, and I advised you off-list that you weren't winning people to your point of view with them. You chose not to take that advice, which is your right, of course. But you need to understand that I've already seen some of your comments here quoted out of context on Puppy-aligned boards. It undermines everything we are trying to do here. We do -not- want to hide things or be gatekeepers or keep the Puppies off of the ballots. I'd actually like the Puppies to -support- this proposal since it meets their stated aims. Comments like the one made in 122 are just not helpful.

Kilo

#134 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 06:47 PM:

[DECLINED NOMINATIONS]

Jameson@24:

3. If somebody declines a nomination, running the system again from scratch will change one of the other 4 existing nominations only about 1 time in 100.

Would you mind describing the situations where it did matter? That might help us decide whether we need to worry about re-running the system at all, and if so, how to do it. The problem, of course, is that we are dealing with low-probability situations to begin with, so it makes it hard to anticipate what the effects would be.

Thanks,
Kilo

#135 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 06:57 PM:

[TIES]

Jameson@56:

I won't likely get to coding this until later this week (or more likely the weekend), but I wanted to make sure I've got the idea you're proposing as I'm thinking about it.

Basically, I need to keep an array for each work (it will be a two-dimensional array, but that's just a note for me) that continually grows with each round and records the total number of points received in that round, correct? We then go backwards through the array when have a tie for both points and nominations. If at any time one of the works has fewer points, then that work gets eliminated (the number of nominations will never change, of course, hence why we look at points). Is this a correct statement of the system?

Once I get it coded, I may be able to come up with a straightforward way to explain it. We can also then run simulations under both tie-breaking systems (with and without past-round lookback -- hey, that's not a bad short-title name right there!) and see the effects. It sounds like you may already have it coded, but as someone up-list said, it's a good idea to have two independent algorithms doing the same thing as a check.

Thanks,
Kilo

#136 ::: P J Evans ::: (view all by) ::: May 12, 2015, 07:07 PM:

128
You seem to be determined to settle it the hard way.
Explain, in simple terms, Why The Fuck it needs to be rerun to pick a replacement for a declined or ineligible nomination, when the current procedure, under the current rules, is to pick the next one from the list of counted names, and when this has not caused difficulties before?

(I refer you to 1984, where there is one withdrawn nomination, and one ineligible. Which you may not have noticed.)

#137 ::: Cadbury Moose ::: (view all by) ::: May 12, 2015, 07:38 PM:

P J Evans @ #136

I think it's a prime example of "the perfect is the enemy of the good" and that by attempting to get a mathematically rigorous nomination system, they are ignoring the potential human cost of a withdrawal/disqualification.

Once the initial nomination phase has been completed and the successful nominees have been notified that should be it. Any declined nomination must simply bump the entries below it up one place: rerunning the vote to select the mathematically perfect replacement is unacceptable if it could create conditions where the committee have to dash someone's hopes by withdrawing their nomination after they've been informed they are on the final ballot.

Run the numbers, produce a list of nominees + enough extras in case all the nominees decline(!) (so you want at least 10 slots) and notify the top 5. (If they all decline, then you might consider rerunning the nomination process with their votes reallocated, but I can't quite see that happening somehow.)

#138 ::: Annie Y ::: (view all by) ::: May 12, 2015, 07:42 PM:

@137

We do have the full list under the 5 anyway - it is the inverted elimination order (unless if we want to over-complicate that by running with them minus the ones already in the game or something along these lines to recalculate one more time for each place). So at the time when we know the top 5, we already know who is #6 and so on - last eliminated is first on the standby list and so on.

#139 ::: felice ::: (view all by) ::: May 12, 2015, 07:51 PM:

Cadbury Moose @137: "I think it's a prime example of "the perfect is the enemy of the good" and that by attempting to get a mathematically rigorous nomination system, they are ignoring the potential human cost of a withdrawal/disqualification."

No, my current proposal is don't remove anyone (other than declined/ineligible works), and extends the ballot to more than five works if needed (just like we do for ties). Call it "rerun, merge, and extend". There's no human cost there.

And it would be pretty rare for the ballot to need to be extended.


P J Evans @136: "Explain, in simple terms, Why The Fuck it needs to be rerun to pick a replacement for a declined or ineligible nomination"

If #128 wasn't simple enough, I can't help you. SDV-LPE is a weighted system, and removing a finalist changes the weights. The current system is not weighted, so removing a finalist doesn't affect anything else.

#140 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 08:18 PM:

@120: That's exactly right, but it's not part of the proposal, because we think that the minimal change that fixes the problem is what has the best chance of passing.

#141 ::: Brian Gibbons ::: (view all by) ::: May 12, 2015, 08:20 PM:

P J Evans @136: Explain, in simple terms, Why The Fuck it needs to be rerun to pick a replacement for a declined or ineligible nomination, when the current procedure, under the current rules, is to pick the next one from the list of counted names, and when this has not caused difficulties before?

Because the current procedure under the current rules isn't using weights and elimination.

Take a scenario in which the five final nominees in Best Dramatic Presentation, Short Form, are each individual episodes from five different TV shows, and it's discovered that one of the nominated episodes is ineligible for some reason.

The 6th place nominee after the initial run was an episode from a 6th TV show that was the sole surviving nominee on 8% of the nominating ballots. However, if you reran the analysis, excluding the ineligible nominee, 15% of the voters who voted for that nominee would have had another episode from that same show as their sole surviving nominee (but that got eliminated earlier in the original run).

By just going to the 6th place finisher rather than rerunning the analysis, you're essentially throwing away any ballots whose sole surviving nominee was the ineligible/withdrawn one. I think there's a reasonable argument that this is an unfair result.

I'm not in favor of removing any finalists from the list, but running the analysis again to see who should get added seems like a better option than just taking #6.

#142 ::: Annie Y ::: (view all by) ::: May 12, 2015, 08:23 PM:

felice @ 139

I think that everyone understands that it is a weighted system. But in order to keep everyone that was already nominated we already break the proper order in case of withdrawals. So why stick to the weighted at this point and do not go for the old rule (the #6 and down were already weighted).

Or think of this this way - usually eliminations can be led to their ultimate end leaving an ordered list of N elements (the inverted elimination order). We declare this the Hugo Nomination System result and we get as many of them as we need - 5 (if over 5%), more in case of ties or in case we need to get an additional one because of removal.

Don't get me wrong - I understand your logic and in a normal case I would agree with you. But we already need to break the standard case because we are dealing with people :)

#143 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 08:27 PM:

@134: I don't actually know, and it's not trivial to actually check. My best guess is that these are the cases where effectively a slate of (alpha, beta) was knocking out gamma before beta got knocked out; and then, when alpha withdrew, gamma never got knocked out by beta, and thus ended up knocking out delta.

If this is what's happening, then yes, gamma should be in there. But what I really want is to stop arguing about what color to paint this 1% of the bikeshed. I still think that the simple answer is better than the right one in this case.

Consensus is good, but when there's one person blocking it, at a certain point you have to move past them.

@135: Yes, you're right. Of course, you can know the final dimensions of the array beforehand, so there's no need to allocate memory on the fly.

#144 ::: P J Evans ::: (view all by) ::: May 12, 2015, 09:05 PM:

141
It hasn't been a problem in the past. Don't create one where it doesn't already exist - and it doesn't, really.

IOW: DON'T BREAK STUFF THAT WORKS.

Dealing with ineligible/withdrawn nominees is a whole 'nother thing from minimizing slate voting, which is the problem we're supposed? to be handling, and it doesn't have to be fixed Right Now.

#145 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:31 PM:

@133: "I've already seen some of your comments here quoted out of context on Puppy-aligned boards"

I understand that some puppies are just in it for the lulz, and that there's no talking to such people. But there are some who are sincere. If you know of boards where they are discussing this system, I'd like to make the case to them that this system is in their interests too.

#146 ::: felice ::: (view all by) ::: May 12, 2015, 09:57 PM:

Annie Y @142: "I think that everyone understands that it is a weighted system. But in order to keep everyone that was already nominated we already break the proper order in case of withdrawals. So why stick to the weighted at this point and do not go for the old rule (the #6 and down were already weighted)."

Almost everyone. And why not stick to the weighted? Rerun-merge-extend is simple and as fair as possible. Sometimes, it will be the same as promoting the last eliminated; sometimes, it will promote a work that was eliminated earlier, which deserves to be on the ballot - a better result; very rarely, it might promote more than one deserving work, in which case technically one or more of the existing works shouldn't really be there, but pretty much everyone agrees removing them after they've been notified is unreasonable, and they're popular enough that keeping them isn't unreasonable.

Actually breaking the proper order would be even more rare; generally speaking, if the rerun produces two new works instead of one (already unlikely), the ex-top-5 work will very probably be in 6th place, and with the final ballot extended to 6 works, it will be the correct top six. If the ex-top-5 work drops to 7th place, you could conceivably argue it's unfair on the work that comes 6th in the rerun, but I think that's pushing it; it wasn't in the top 5 before the rerun, it's not in the top 5 after the rerun, so it doesn't deserve to be a finalist.

#147 ::: nathanbp ::: (view all by) ::: May 12, 2015, 09:58 PM:

[DECLINED NOMINATIONS]

In the 1984 data a declined nomination may not have caused someone besides the last eliminated nominee to be nominated very often when rerunning the data. However, in the presence of a slate, if a slate nominee withdraws, it seems fairly likely that the last eliminated nominee would not be from the slate and thus the opinions of a large number of votes would being disregarded. You can imagine the same situation with correlated voters for other reasons as well, although it's much less likely. For this reason my preference would be to rerun the system and then replace the withdrawn work with the top work that's not on the ballot (2b from my listing of options at #174 in the previous post).

P J Evans@144: The solution that's been proposed here to deal with slate voting breaks the previously working system for dealing with withdrawn nominees, therefore discussing how to deal with withdrawn nominees is a relevant part of this discussion.

#148 ::: Laertes ::: (view all by) ::: May 12, 2015, 09:59 PM:

Here's Brian Z, over at file 770, characterizing @133:

But I just saw on Making Light, in a post ostensibly aimed at reaching consensus on a rules change, a regular participant raise a useful and fairly critical but also fairly dry, technical point, and then be chastised and put back in his place by a community leader on the grounds that one of his early remarks was supposedly quoted approvingly on a “pro-puppy” site. We can’t have that. The puppies should be approving what we are doing without being engaged in discussion because we are inherently right and good, and if they have ideas of their own then something must be wrong with them.
#149 ::: johnofjack ::: (view all by) ::: May 12, 2015, 10:19 PM:

J Thomas @ 122: If anybody asks anything about it, announce that all results are final and say no more. If they can't prove there's a problem, then there's no problem.

This is exactly the opposite of what I believe and what I want. I think the nominating and voting process needs to be open and transparent, not dismissive and authoritarian. There are people who trust people by default but I don't tend to be one of them.

#150 ::: Vicki ::: (view all by) ::: May 12, 2015, 10:21 PM:

I may be missing something, but:

This is a proposed way of selecting five nominees. It's possible that a tie might lead to six, seven, or even ten nominations. (Two-way ties have happened under the current system; it doesn't seem to be a major problem.)

If one of those 6-10 choices declines the nomination, there's no need to recalculate anything. The list without the person who declined would still include at least five finalists.

Can't we save recalculating for categories/years where the algorithm produces a list of five and then someone declines, or the (I suspect rare) cases where it produces six or more, and then enough people decline to bring it below five?

#151 ::: nathanbp ::: (view all by) ::: May 12, 2015, 10:35 PM:

Vicki@150: I don't know that what happens when the category already has more than 5 nominees and one of them declines has been considered. In that case it would seem reasonable not to replace them. But in the majority of cases where a nominee declines the system will have only selected 5 nominees for that category (since in the majority of cases the system will have only selected 5 nominees).

#152 ::: Brian Z ::: (view all by) ::: May 12, 2015, 10:41 PM:

@ 148, If you are trying to throw a bomb by bringing this discussion over here, could you please cite my entire comment in the context of the other discussion and link to the exchange about it. Thanks for your consideration.

@ 149, "the nominating and voting process needs to be open and transparent, not dismissive and authoritarian." Does anyone disagree with this? I have been following the discussion in this thread, and thought it was about when it is the appropriate time to say "this result is final" and not go back to recalculate the results again. That has absolutely nothing to do with whether or not someone trusts a WorldCon committee (who have won a bid!) to administer the Hugos in an ethical manner. Or did I misunderstand the topic under discussion?

#153 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:50 PM:

I would like to include sad puppies in this discussion. I doubt that inviting them over here would end well though; there would be people on both sides looking to take offense. Any suggestions for neutral territory?

#154 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:51 PM:

Laertes@148:

Well, I guess I'm flattered that I'm a "community leader" now. Thanks Brian! :) But this is an example of how comments can taken out of context. As the folks reading here know, it wasn't a "dry, technical comment" it was an implication that we want to destroy the ballots and hide the evidence. Which Brian Z well knows, of course, but hey, that's okay if it fits your agenda.

Here's one quote of JT's that was posted on .Clue and commented on here. The specific quote is:

“My view is that when we specifically try to change the rules to exclude the Sad Puppies, and we judge how well the changes work by how well they would have excluded the Sad Puppies given historical data, we will have some difficulty explaining to journalists that we are not doing it to exclude the Sad Puppies.”

If you go back and look at the thread, JT is -not- saying this is what we shold try to do, even though the article makes it look like it is. But the point is that there are certain people (Brian Z, apparently as well) who are going to take things out of context to "prove" their point. In this case, JT's comment @122 looks -explicitly- like he is suggesting this is the goal. That's not okay, so it was necessary that we make it clear -- and also be cognizant of the fact that words matter. I don't know if he was joking or not, but it sure didn't come across that way.

Kilo

#155 ::: J Thomas ::: (view all by) ::: May 12, 2015, 10:53 PM:

@133

These are some of the more offensive intimations you've made so far.

I am not "intimating". I find it offensive that you would read that in.

I do realize you aren't accusing anyone of doing any of these things directly,

Or at all.

but it is simply completely false that we are trying to accomplish anything like the things you suggest.

Of course not. I do point out, though, that this approach in some ways would be the most practical.

As Annie says, the goal is just the opposite: to make the system as open and accountable as possible.

If we do make it as open and accountable as possible, we might find ourselves providing people with enough data that they can interpret it as the voting administrators doing something wrong.

Consider how it goes with IRV, where the top winner gets votes that don't go to anyone else, and then the 2nd place has votes that don't go to anybody else, etc. If the top winner got disqualified and you didn't recount the votes, that would be likely more than half the votes that didn't get counted. It would be a travesty.

It isn't that bad with our system since the votes do get counted to decide eliminations, and get partly counted for choosing who to compete.

But I can imagine the argument. "You guys say that this is the right way to choose the final nominations. But now here's somebody who isn't running after all, and you aren't using the method you think is the right one." You can say that you don't do it one time and not the next, it isn't arbitrary because you have a rule that you always follow. "So let me get this straight, you jumped the gun and announced the winners before you found out this guy was going to withdraw or get disqualified, and now you say it's too embarrassing to take it back from the guys you falsely announced were the winners so they get to keep their wrongly-acquired awards. While these other guys who really won get nothing."

I don't know that this argument is valid. I can easily imagine a reasonable-sounding argument that it's wrong. But it's better not to have to argue in the first place. We ought to have a really good explanation why it's right. "How would you feel, going to somebody who'd already been told he won the Hugo and having to tell him he didn't win after all?" will work with some people.

I know that nobody here wants to keep secrets. But if we did keep all the details secret we wouldn't have as many angry arguments about what's fair, as we can expect to get if people do find out details of individual elections.

It's predictable that most of the time, ties and withdrawals won't make any difference to the results. If people find out about one time when they do make a difference, then we need to have our superbly fair explanations ready.

#156 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:03 PM:

Brian@152:

"the nominating and voting process needs to be open and transparent, not dismissive and authoritarian." Does anyone disagree with this?

Certainly not me, which is why I felt JT's comment was out of line. And which certainly makes it curious that you would frame your comment on 770 as you did.

For the record, I don't think a "pro-Puppy" site is a bad thing at all, and in fact, I would really like to involve those who count themselves Puppies in the discussion, but I'm not sure there's many willing to have that discussion. I have been clear from the beginning that I am actually the Puppies' target demographic in my reading interests and I do -not- want any genre excluded. This system explicitly ensures that if there are enough people who like the types of works Brad T recommends, then they will appear on the final ballot -- but they won't also force other types of works off. Brad's complaint was that the types of fiction he enjoys was being ignored. I'm not sure if that's true or not, but one way or another, this system works to prevent -any- group -- Puppy, Kitten, or Monkey -- from being able to keep other groups' works from being considered. I'd like to think they'd welcome that, but I'm probably being naive.

But you know what, you're right that this -is- a post trying to reach a consensus on a Hugo nomination system. And this particular topic probably sidetracks that, so I'll stop.

Just my thoughts,
Kilo

#157 ::: Brian Z ::: (view all by) ::: May 12, 2015, 11:07 PM:

@ Kilo,

I would have made a similar comment over here too (not just on File 770) had there had not been that lightning-quick accusation from an alert reader. I am just back at the computer catching up on these threads, and things move quickly. I hope you get a chance to read my full comments on the other side and see that I am feeling disappointed with the totality of the situation, not with you personally.

I do think you seem like a "community leader" and I see you are doing great work. If I missed aspects of the full context of your criticizing the other poster that would have helped me better understand your remark, then I accept your clarification with thanks.

More generally, I believe that engaging more actively with those who people have sometimes referred to as if they were "the other side" (though I would rather think of all of us as first and foremost fans, and members of the WSFS) is a crucial step, and one that would be in what I consider to be the spirit of the award.

#158 ::: Annie Y ::: (view all by) ::: May 12, 2015, 11:10 PM:

J Thomas @ 155

"If we do make it as open and accountable as possible, we might find ourselves providing people with enough data that they can interpret it as the voting administrators doing something wrong."

And that is easily provable by releasing all the data if need be.

My problem with your comment in 122 were the last 2 paragraphs - you essentially say (or so I read it) that the easiest way is to build the system and then ignore any attempt at scrutiny by stonewalling and whatsnot. This is the worst thing we can do. I think you said it in jest and more to point to the worst alternative but... it is easily taken out of context and half of the conversations these few weeks were for things taken out of context. It does not help.

#159 ::: felice ::: (view all by) ::: May 12, 2015, 11:21 PM:

nathanbp @147: "my preference would be to rerun the system and then replace the withdrawn work with the top work that's not on the ballot (2b from my listing of options at #174 in the previous post)."

What do you think of "rerun-merge-extend" (see #128)? I originally supported your 2b, but that gives us trouble when it's time to publish the extended nomination data (see #6).


Vicki @150: "This is a proposed way of selecting five nominees. It's possible that a tie might lead to six, seven, or even ten nominations. (Two-way ties have happened under the current system; it doesn't seem to be a major problem.) If one of those 6-10 choices declines the nomination, there's no need to recalculate anything. The list without the person who declined would still include at least five finalists."

If one of the tied works is withdrawn, then yes, I'd agree there's no need to replace it. But if one of the higher-ranked works is withdrawn, fairness still requires a replacement. Of course it's possible that rerunning will promote one of the tied works above the other(s) to replace the withdrawn work, so the number of finalists will still drop. Eg if the finalists were ABCD[EFG], and B was withdrawn, a rerun might give us ACDE[FG], or even ACDEF; either way, the run-merge-extend result would be ACDEFG, ie no replacement. Though it is also possible that the rerun will promote something other than E, F, or G, so the final ballot could be AZCD[EFG].

#160 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:27 PM:

Brian@157:
More generally, I believe that engaging more actively with those who people have sometimes referred to as if they were "the other side" (though I would rather think of all of us as first and foremost fans, and members of the WSFS) is a crucial step, and one that would be in what I consider to be the spirit of the award.

I agree wholeheartedly. I would dearly like to engage with "the other side" (I also hate that term, since I agree there shouldn't be "sides"), but I'm not sure how many people on that "side" are willing to do so. As I said in the other thread, I'm a retired Naval officer, a combat verteran, a Navy marksman, and Robert Heinlein is my hero. Honor Harrington rocks. I'm about as close to what the Puppies say they support as anyone, genre-wise (by the way, I loved Ancillary Justice, too). So, really, my problem is with slates in general, not with politics, and I actually think that's true of just about everyone here -- whatever their reading preferences might be. What I'm not sure is that there are Puppies who are willing to put aside politics and work together. I do hope there are, though.

At any rate, thanks for coming back and accepting the clairfication of what was happening. The more voices we get on this proposal, the faster we can make the Hugos work for everyone. That really is the goal.

Thanks again,
Kilo

#161 ::: nathanbp ::: (view all by) ::: May 12, 2015, 11:31 PM:

felice@159: Without having put too much thought or calculation into it, the potential issue I see with "rerun-merge-extend" is that it seems like it could potentially end up with a large number of nominees if multiple people drop out or are disqualified in in sequence (like happened this year). I'm not sure how likely that really is to happen but it is a concern. (2b) seems simpler and less likely to have odd side effects.

#162 ::: Brian Z ::: (view all by) ::: May 12, 2015, 11:31 PM:

Brad's complaint was that the types of fiction he enjoys was being ignored. I'm not sure if that's true or not, but one way or another, this system works to prevent -any- group -- Puppy, Kitten, or Monkey -- from being able to keep other groups' works from being considered.

Why wouldn't fans of Larry Correia, Jim Butcher, Kevin J. Anderson, John C. Wright, et al., be interested in that discussion? I bet a lot of them are.

My own perspective, FWIW, is different. I believe the culture surrounding the voting process has got to be a more vital ingredient in the long success of the Hugo Awards than some highly technical rules, and believe that the message sent to voters by the current rules (we will shortlist the works that are acknowledged as best of the year by the greatest number of a community of readers) is preferable to what I fear may be a subtly but crucially different message sent by the proposed rules (we will shortlist works in a way that maximizes voter happiness).

Whether I'm right about that or not, voting is explicitly open even to people who don't attend and were not regular voters in the past, so when thousands of new readers are motivated to participate, I still think the best way of ensuring a successful Hugo voting culture survives is to reach out to them, welcome them, and encourage them to read as much as possible and vote accordingly. Or perhaps, the Hugo voting culture (not necessarily the rules) might even evolve as the membership evolves - I don't think we can know for certain yet.

#163 ::: Annie Y ::: (view all by) ::: May 12, 2015, 11:43 PM:

Mike Glyer linked to this post in today's roundup. So let's see if someone will decide to join everyone here with fresh eyes and ideas. :)

#164 ::: P J Evans ::: (view all by) ::: May 13, 2015, 12:01 AM:

162
Dammit, I thought we'd cured these guys of trying to maximize voter happiness. That's not one of the things that's on the short list (see the previous post):

Whatever we choose should have these properties:

It should be fair.
It should be perceived as fair.
It should be relatively easy to explain, both to the voters and at the WSFS business meeting.
It should be relatively easy to administer.
It should encourage people to nominate.
It shouldn’t result in too many nominees for the electorate to reasonably read and rank by the Worldcon.
It should be resilient to some degree against strategic voting: i.e., minority voting blocs.

#165 ::: Brian Z ::: (view all by) ::: May 13, 2015, 12:21 AM:

@164 P J Evans

I didn't participate in that previous thread primarily because Bruce asked us not to discuss "whether it is a good idea to change."

But if you don't mind the late addition, I would support not abandoning one that is already codified.

Whatever we choose should have these properties:

It should represent the works acknowledged as best of the year by the greatest number of voters.

(If you can figure out a way to avoid slates and still do that, I'm all for it.)


"voter happiness"

You can call it "perceived fairness" too, I don't object, if maximizing the number of voters who get something from their ballot nominated is what both makes people happy and is perceived as fair.

For me, to use the thought experiment of 1970 above, if I had nominated Left Hand of Darkness, Slaughterhouse five and Bug Jack Barron, but Spinrad ended up getting knocked off the ballot by Piers Anthony and Michael Crichton (for example), an explanation that my acknowledgement of Bug Jack Barron as one of the best works of the year had been discounted because I felt the same way about LeGuin and Vonnegut would not strike me as fair at all. Not fair to Spinrad, that is. And being not fair to Spinrad would make me unhappy.

I don't mean to be too argumentative but I do hope you can see that some people may not be coming to this discussion from the perspective that "the greatest good for the greatest number" is particularly applicable to the question of how the Hugo Awards have on a fairly consistent basis managed to pick great SFF most of the time over the years.

#166 ::: Brian Z ::: (view all by) ::: May 13, 2015, 12:32 AM:

Self @165

If you can figure out a way to avoid slates and still do that, I'm all for it

Well... in fact I also do have a concern with limiting the number of slot to less than five on the grounds that encouraging voters to think outside the box and consider some things that are not their "top favorites" is a valuable signal sent to voters by the nominating ballot in its current form. But that is a slightly different discussion and may be off topic for this thread.

#167 ::: felice ::: (view all by) ::: May 13, 2015, 12:39 AM:

nathanbp @161: "Without having put too much thought or calculation into it, the potential issue I see with "rerun-merge-extend" is that it seems like it could potentially end up with a large number of nominees if multiple people drop out or are disqualified in in sequence (like happened this year). I'm not sure how likely that really is to happen but it is a concern. (2b) seems simpler and less likely to have odd side effects."

Theoretically possible, but the multiple withdrawals this time are mostly if not all slate-based, and RME is almost certain to just add the next work off the slate as a sole replacement. And ideally, wait as long as possible before rerunning, so if there is more than one withdrawal in the same category, one rerun can account for all of them. Even if multiple rounds of RME are required, I'm confident that the chances of the ballot growing much are pretty slim. At some point, the ballot is finalised and any late withdrawals just don't get replaced (eg Black Gate this year).

2b is pretty much guaranteed to cause a fuss the first time the rerun puts more than one new work in the top 5, and the other new work(s) don't end up as finalists. That information will come out when the extended nomination data is released.


P J Evans @164: "Dammit, I thought we'd cured these guys of trying to maximize voter happiness."

In this context, "happiness" is defined as "believing at least one and preferably more of the best eligible works have made the final ballot". That strikes me as a very reasonable goal.


Brian Z @165: "For me, to use the thought experiment of 1970 above, if I had nominated Left Hand of Darkness, Slaughterhouse five and Bug Jack Barron, but Spinrad ended up getting knocked off the ballot by Piers Anthony and Michael Crichton (for example), an explanation that my acknowledgement of Bug Jack Barron as one of the best works of the year had been discounted because I felt the same way about LeGuin and Vonnegut would not strike me as fair at all. Not fair to Spinrad, that is. And being not fair to Spinrad would make me unhappy."

The only way Spinrad could be knocked off would be if more people liked LeGuin and Vonnegut. If it's knocked off by other works that more people nominated, how is that unfair?

#168 ::: Chris ::: (view all by) ::: May 13, 2015, 12:43 AM:

I'm going to agree with Seth Gordon @44. The voting system takes a bunch of ballots, and outputs a ranked list. If one of the nominees withdraws, give the spot to the next work on the list. (If there are more than 5 nominees, and any one of them withdraws, do not replace.) It may not be perfect on the "should be fair" criterion for a mathematician's definition of "fair", but it's a hell of a lot better on "perceived as fair", "easy to explain", and "easy to administer".

#169 ::: Seth ::: (view all by) ::: May 13, 2015, 12:46 AM:

We should do re-runs.

The released data must include the calculations involving the declined nominee; that's required to show that the results are correct.

As for slates, a group of 20% of the nominators ought to be able to get one item on the ballot (whether they nominate a full slate or a single item). If there were five such slates they'd fill the entire ballot, which would annoy the remaining nominators except there aren't any.

I don't see any particular reason to limit people to 5 nominations (except perhaps the way it interacts with the 5% rule).

#170 ::: Steven desJardins ::: (view all by) ::: May 13, 2015, 12:48 AM:

PJ@136: The administrators don't rerun the counting in the current system because rerunning the counting wouldn't change any of the totals. Everything gets one nominating vote for each ballot it appears on, whether the other items they list are withdrawn or not.

On the other hand, if a bunch of voters nominate something like THE MARTIAN, which turns out not to be eligible, then I think the administrators should drop THE MARTIAN from their ballots before they even start the SDV elimination process. If the administrators find out after the fact that THE MARTIAN isn't eligible, then I think they should try to get the same results as if they'd realized that in the first place. And ideally we shouldn't require voters to research the eligibility of works or figure out what's likely to be withdrawn: people should be able to vote for what they like, without worrying that a miscalculation will devalue the rest of their choices.

I get that it's weird and counter-intuitive, but any option is weird and counter-intuitive in some way.

#171 ::: P J Evans ::: (view all by) ::: May 13, 2015, 12:50 AM:

167
I don't see how you're going to get there by futzing with unbroken stuff. What has people unhappy NOW is not the voting method, but slate voting. So you want to minimize the effects of slate voting. All else is frosting, and possibly unwanted frosting at that.

(How long have you been in fandom? Because this is stuff you should be aware of already.)

#172 ::: Brian Z ::: (view all by) ::: May 13, 2015, 12:50 AM:

felice @ 167,

Hypothetically, in my example, folks who liked Bug Jack Barron probably also voted for the LeGuin and Vonnegut, which both would have steamrolled onto the final ballot regardless. In contrast, in my example, fans of Crichton and Anthony hypothetically tended not to vote for Bug Jack Barron (perhaps finding it weird and challenging, not liking the authors politics, or for other reasons that seemed relevant to them in the summer of 1970). Why should a brilliant edgy writer who was ahead of his time have votes for him discounted because his voters also liked several more mainstream choices?

To extend the thought experiment: you are so blown away by reading LeGuin that you can think of nothing else and just want her to win at all costs. Why not just leave Spinrad's also-noble effort off the ballot entirely so that you don't detract from LeGuin's greater glory?

Ask whether voting strategies might change under the new rules, and if they might, ask if that is the best for the long term continued success and prestige of the award.

#173 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 01:06 AM:

Regarding final-place ties: the largest Hugo finalist field I recall there ever being was seven. I see no reason to try to break a last-place tie for finalists. I've been encouraging anyone writing rule changes that touch the ties language (this isn't the only proposal out there that might do so) to generalize the language into "ties involving last place" so that if we decide to modify the number of finalist positions, we don't have to change a bunch of hard-coded references elsewhere.

#174 ::: influxus ::: (view all by) ::: May 13, 2015, 01:10 AM:

Brian Z @ 172:

Under the current system there are only 5 slots, it is "top five past the post". If you are all about glory to the LeGuin then you are better off depriving any potential competition of extra votes, and just nominating one work.

Your objection applies equally to the current system. If we are going to use pathological voters as test cases we need to show that they would behave and impact on the current system in a better fashion.

#175 ::: Laertes ::: (view all by) ::: May 13, 2015, 01:12 AM:

I've been staying out of the rule change threads because they look like a lot of work, I have no unique expertise to offer, and it looked as though there'd be enough people involved that one more wouldn't be a make-or-break.

Just wanted to pop in to say I love what y'all have come up with. It's elegant and seems likely to achieve its aims.

It's very nice work. Bravo!

#176 ::: felice ::: (view all by) ::: May 13, 2015, 01:13 AM:

Brian Z @172: "Hypothetically, in my example, folks who liked Bug Jack Barron probably also voted for the LeGuin and Vonnegut, which both would have steamrolled onto the final ballot regardless. In contrast, in my example, fans of Crichton and Anthony hypothetically tended not to vote for Bug Jack Barron (perhaps finding it weird and challenging, not liking the authors politics, or for other reasons that seemed relevant to them in the summer of 1970). Why should a brilliant edgy writer who was ahead of his time have votes for him discounted because his voters also liked several more mainstream choices?"

Because more people nominated the mainstream choices; the Hugos are a popular award, not a judged award, and you opinion of a work's merit doesn't make a difference if other people don't agree with you. And SDV-LPE is just as likely to help Bug Jack Barron as hurt it; what if Macroscope had been released in two volumes, and most of its supporters has nominated both parts? Under the current system, it takes up two slots on the final ballot, leaving no room for Spinrad; SDV-LPE would probably eliminate one of them, and leave the 5th slot free for Bug Jack Barron.

"To extend the thought experiment: you are so blown away by reading LeGuin that you can think of nothing else and just want her to win at all costs. Why not just leave Spinrad's also-noble effort off the ballot entirely so that you don't detract from LeGuin's greater glory?"

That's exactly the same as under the current system. A vote for anything else in addition to LeGuin could be the difference between those two works tying for 5th, and the other one beating the LeGuin.

#177 ::: Laertes ::: (view all by) ::: May 13, 2015, 01:13 AM:

Also, seriously? Watch out for Brian Z. He's concern-trolling you, and at any minute is liable to run off to some other blog and spread vicious lies about what's going on here at ML.

#178 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:22 AM:

@174

"Your objection applies equally to the current system."

Are we sure? My gut reaction to this is that people may be given more incentive to nominate fewer works under the proposed system. In a perfectly well-meaning and innocent way. ("I'm a Jim Butcher maniac and the most important thing is for him to get as much support as he can, better not to split it with something else. I liked Goblin Emperor but Butcher is more important. And somebody told me Three Body Problem is a great book, but if lots of people like Goblin Emperor and 3PB they are probably going to win anyway and don't really need my vote, and if they did it would discount my support for Butcher and oh look at the time, why its already February anyway!")

#179 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:27 AM:

Laertes: I'm a knight, not a troll, and if you don't do exactly as I demand, I shall say "Ni" to you! :D

#180 ::: Jeffrey Smith ::: (view all by) ::: May 13, 2015, 01:31 AM:

Brian Z:

And just think how unhappy you'd be if some slate came in and knocked ALL of your choices off the ballot. That's the system you're advocating for here.

#181 ::: influxus ::: (view all by) ::: May 13, 2015, 01:31 AM:

Laertes @177:

I think Brian Z plays an important role as a hostile vetting process.

Brian Z @ 178:

That whole monologue you just imagined, works with the current voting system. See felice @ 176 for the precise case. By just voting once, currently, you cast 1 vote for Butcher and four for nobody but Butcher.

#182 ::: Annie Y ::: (view all by) ::: May 13, 2015, 01:35 AM:

Brian,

This explanation works for both the current and the proposed system. If anything, it is more likely to be a problem in the current system (I like A and B and C but a lot of people seem to like C so if I add a vote it can knock off B so let me skip it). Of course some people do not think strategically (which is a bad thing during nominations but that's a different story) and just nominate the works they will be perfectly happy to see on a ballot and winning - if I think less of book C than A and B, why would I put C on my ballot?

As for Butcher's fans - we can think, we do not go and just nominate Butcher because we really like him :)

#183 ::: Idumea Arbacoochee, who really is a moderator here ::: (view all by) ::: May 13, 2015, 01:35 AM:

I don't have time to untangle this in detail until tomorrow, but a few points:

1. I think we have come so far with this particular proposal that we should refine it rather than rewriting it in the light of late-coming commenters. I'd suggest focusing on practical objections rather than philosophical ones.

If other people want to make other proposals, I'd welcome links to similar discussion threads pursuing those. This is the one we seem to have settled on; let's run with it here. If a better proposal comes up elsewhere, it'll sweep the Business Meeting.

And that's also a win.

2. Someone please provide a link to the File 770 conversation so I can have a look at the context?

But I ask out of morbid curiosity as much as anything else. To a certain extent, we must accept that no matter what we say, there will be people elseweb who will spin it in the worst possible way, or flat-out lie about it. It's the nature of the conflict, because of the participants.

3. Brian, what I've seen of you elseweb does not fill me with confidence, but if you're participating in this conversation in good faith, that's fine. People who do conversation differently should also be able to use this system.


I will be back in a little while, when real-life obligations are not so pressing. I hate to sound like a mother when I say that I don't want to be ashamed on anyone's behalf when I do.

But there you go.

#184 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:38 AM:

influxus, thanks for the vote of confidence.

I agree some voters could (and do) already feel that way. I'm asking if the proposed system might incentivize more of it. I don't know the answer. Its a question about how voter strategy might change.

#185 ::: Annie Y ::: (view all by) ::: May 13, 2015, 01:40 AM:

abi, http://file770.com/?p=22470&cpage=5#comment-260835 is where it starts, go down from there.

#186 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:46 AM:

@ 182,

If I think less of book C than A and B, why would I put C on my ballot?

For example, I think Echopraxia is a breakthrough to a new level of science fiction, it is criminal that it doesn't even make the ballot. I think The Peripheral is a terrific return to writing stunning novels, but not a breakthrough of the same magnitude. I think Three Body Problem is a flawed book in some respects but also of tremendous importance to the field and absolutely deserving of a Hugo. So if I COULD rank my nominations in order of priority, that would be the order. Don't we all have such private rankings?

As for Butcher's fans - we can think, we do not go and just nominate Butcher because we really like him :)

Yes, up to a point, but if that starts to drown out more "traditional" voting practices of reading and nominating widely, the awards as a whole probably won't function as well (I think).

I could be wrong about this, but I didn't see anyone else asking on this particular how the proposed system will effect voters' strategies, and think that is an valid question in evaluating a new proposal.

#187 ::: Annie Y ::: (view all by) ::: May 13, 2015, 01:49 AM:

Brian @184

Well - we all (you included) may be SF fans but none of us can read minds (unless if someone steps up and say otherwise). So we cannot predict how fans will react and if the nomination patterns will change.

So let's take that one fan at a time. Would your vote been different last year if this was the rule? I've asked myself this question and my answer is "no". But I've already said that I nominate the works I will be happy to see win and not the ones I like and the once I really do not mind but rather not see win...

#188 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:51 AM:

185 thanks for posting the link.

Please note exasperation is aimed at the general tone (the File 770 thread is mostly meta-discussion) and I do apologize for leaving Kilo to be caught in the crossfire as it were by posting there and having people react to it - in retrospect I really should have posted my reaction here first.

#189 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:55 AM:

@187

Interesting question. Last year, I knew full well that none of my picks would make the ballot, and sure enough they didn't. Even altering my voting strategy wouldn't have helped. But if I feel all out determined for a particular thing to win and think it has an outside chance? I'm not sure.

#190 ::: Annie Y ::: (view all by) ::: May 13, 2015, 01:59 AM:

@186 Brian

Would you be happy if any of the three win the Hugo? I suspect the answer is yes - so you are fine nominating the three, right? Different books can make the nomination list for different reasons - as long as you are fine with them winning, it is your ballot.

But if you are putting a work that is not really at the same level and you know it but you just want to give it a nod, you may be responsible for this book knocking one of your other choices. In both systems :)

@188 Considering that I was the one that called you on it, I kinda knew how to find it fast. Citing (and reading) out of context - that's exactly what happened. Again. :) You (the generic you) start talking with the people, things can get resolved - as long as the good faith and the desire for that is there.

#191 ::: Brian Z ::: (view all by) ::: May 13, 2015, 02:06 AM:

For 2015, yes, I'd be fine with any of them making the ballot though I wouldn't have necessarily expected any of them to.

For 2014, I had wanted to vote my conscience. But I can at least imagine myself taking some kind Butcheresque tack ("Nicola Griffiths really deserves a nomination - this whole situation is hopeless and the best I can do is put my eggs in her basket.") I'd like to think I'd instead strive to be as "conscientious" as possible to all potential nominees but when the chips are down, who know?

"good faith and the desire for that"

If I didn't have some, I wouldn't be here concern trolling you. :D

And this highlights something else that I think is important. The Hugos should be a gesture that we the fans are collectively giving our authors - not something we are claiming for ourselves. Which is part of why it is not that important whether we get what we want are not.

But what do the authors want? To be honored as an example of excellence? To be ordained as Mr. and Ms. Popularity? Honestly these days it is getting hard to tell.

#192 ::: Brian Z ::: (view all by) ::: May 13, 2015, 02:16 AM:

180 sorry I missed this. But that's exactly my point. My personal happiness with having my personal opinion validated, or my belief the system if fair in the sense of giving as many people as possible something they personally like on the ballot, is not all that important to me. What's important to me is that the authors get a sense that all this hard work for almost no pay is worth something to us and we are grateful. That's the only reason not to let standards of excellence slip - the honor of being given this gift by fandom one might be lessened.

#193 ::: Annie Y ::: (view all by) ::: May 13, 2015, 02:25 AM:

@191/192 Brian
Ah, the authors. Most of them would love an award won cleanly - but not a lot of them are ok with being handed a rocket for being liked by 1 guy that just can organize a crowd. :) Which is what the current proposal is trying to solve - the Hugos are supposed to be a celebration of fandom and the authors and the works; not a celebration of 1 person's opinion of the genre. If the unwritten rules are not followed, we need to change the written ones.

Or do you think that a slate chosen by 1 person (or 2 or 3) is a fair way to nominate for Hugos? That it honors the authors? That it shows who fandom wants to honor? Fandom is everyone nominating; not just the ones that exploit a known crack in the process to make sure their ballots win the day. With your own words:
"My personal happiness with having my personal opinion validated, or my belief the system if fair in the sense of giving as many people as possible something they personally like on the ballot, is not all that important to me."

Then why are you supporting the slate voting when it is exactly the opposite of this? And if you are not supporting it (somehow), why are you willing to leave the system exposed to it again?

#194 ::: Annie Y ::: (view all by) ::: May 13, 2015, 02:31 AM:

OK - back on topic :)

Sounds like the biggest disagreement at the moment is how we treat withdrawals...

I am not really up to speed with all the rules but it is possible to change things during the ratification in Kansas, right? So why don't we set both possibilities, with Administrators set to be deciding at the start of the nominating period what they want to choose and then discuss again after Spokane when we know where we are with the rest of the proposal? Just... thinking aloud. That should allow us to get something that resembles an agreement together?

#195 ::: Brian Z ::: (view all by) ::: May 13, 2015, 02:42 AM:

@193,

Bear with me for a moment: In my opinion, Sad Puppies were originally trying to rally a fan base who doesn't often nominate to be more active in the process (which included a few individual authors or small groups of authors promoting themselves, like you are expected to do as an author these days.) Not to create slates or take over the award. But people started attacking them, some of them drank their own kool aid, and some seemingly started to believe the narrative that their silly campaign was actually some kind of vital front in the culture wars, which is ridiculous, and they let their rhetoric spiral out of control. Science fiction can never be reduced to any simplistic left-right split. In fact, very old and established elements of SFF debates (like the libertarianism on the right and also the left) are only now reaching the mainstream. We're light years ahead of the US culture wars, for crying out loud, why look backwards?


No, I don't support lockstep voting at all. I don't think anyone supports lockstep voting. In fact, if you proposed a rule with a clear and simple mechanism for preventing a group of lockstep voters from sweeping (Say, "in the event a set of identical ballots is large enough to place five works on the ballot in any category, the bottom two (or heck, maybe three) works shall be discarded and replaced with the works with the next highest numbers of ballots"), I'd have a lot more sympathy with it, personally speaking.

But the best solution is not to legislate - it is to welcome new voters into your culture and help them understand your traditions better - and get ready for a likely tidal wave of nominations in the future.

#196 ::: felice ::: (view all by) ::: May 13, 2015, 03:05 AM:

Brian Z @195: "if you proposed a rule with a clear and simple mechanism for preventing a group of lockstep voters from sweeping (Say, "in the event a set of identical ballots is large enough to place five works on the ballot in any category, the bottom two (or heck, maybe three) works shall be discarded and replaced with the works with the next highest numbers of ballots"), I'd have a lot more sympathy with it, personally speaking."

So the group put up a slate of four works, and sweep all but one slot; that's almost as bad. Or they tell their supporters to pick four out of the five slate candidates, and they might still sweep all the slots (because the ballots won't be identical). A voting bloc like this will just work around any rules that attempt to define a slate.

#197 ::: Brian Z ::: (view all by) ::: May 13, 2015, 03:11 AM:

196 felice, OK, fair point. Maybe you could say "in the event a set of ballots with either four or five identical works is large enough to place either four or five works on the ballot in any category, the bottom one or two (or else two or three) works shall be discarded and replaced with the works with the next highest numbers of ballots". Just an example of the kind of thing I'd personally be more receptive to.

#198 ::: Brian Z ::: (view all by) ::: May 13, 2015, 03:19 AM:

196 "tell their supporters"

Also, who is handing out marching orders to their minions? First, telling fans what to do is herding cats. Second, it seems Vox Day may have said to his little pirate crew that they should vote his straight ticket, but SP3 didn't say that. They made a mistake of perhaps inadvertently encouraging lockstep voting once, and I doubt they use that confused language again (I could be proved wrong I suppose). And if VD does, is he going to overpower the groundswell regular enthusiastic voters who are now motivated to nominate more vigorously? Seems highly unlikely to me. That's my personal take.

#199 ::: junego ::: (view all by) ::: May 13, 2015, 03:23 AM:

[WITHDRAWALS]

Annie Y @ 142
felice @ 139
I think that everyone understands that it is a weighted system. But in order to keep everyone that was already nominated we already break the proper order in case of withdrawals. So why stick to the weighted at this point and do not go for the old rule (the #6 and down were already weighted).
Or think of this this way - usually eliminations can be led to their ultimate end leaving an ordered list of N elements (the inverted elimination order). We declare this the Hugo Nomination System result and we get as many of them as we need - 5 (if over 5%), more in case of ties or in case we need to get an additional one because of removal.
Don't get me wrong - I understand your logic and in a normal case I would agree with you. But we already need to break the standard case because we are dealing with people :)

I've been wobbling back and forth about the "fairest" way to resolve this. I *think* there is consensus that once the finalists are notified, those nominees should not be removed by a later recalculation.

This year Worldcon "froze" the list to replacements when the ballots went to the printers. Was that "fair" to those who could have made the finalists otherwise? It's not like it would have been impossible to fix the problem, just really inconvenient and confusing.

Once the original calculations are completed we have a snapshot of the members' choices for the Hugos.

The present system uses the next-in-line replacement rule, and as mentioned upthread changing less is easier.

Given that we would be freezing the finalists list (and thereby being "unfair" to anyone who might have advanced in a full recalc) AND all of the above resons including Annie Y's points....I think it would be simplest to freeze the entire list and replace disqualifications/withdrawals #6, #7, etc. on the frozen list.

ps: I wouldn't be completely opposed to recalc'ing #6 down in case of withdrawal, but I think that might actually open up the process to even more rules lawyering and disagreement than just making a firm rule.

#200 ::: Soon Lee ::: (view all by) ::: May 13, 2015, 03:31 AM:

A suggestion Brian Z. and others: Can we stay on topic please? That is, the proposal for a change to the Hugo Rules to minimise the effect of slate or bloc-voting?

The wider ranging discussion is not germane here and to be honest, is an unnecessary distraction. There are other threads here & elsewhere where those sorts of discussions are being held.

#201 ::: Brian Z ::: (view all by) ::: May 13, 2015, 03:41 AM:

@ 200 Soon Lee,

Well, yes and no. The previous thread to discuss proposed rules changes carried an advisory at the top not to argue about whether changing the rules is a good idea or not, but this one doesn't (yet). I was sort of yanked back here and since then I've been mostly answering direct questions. But if no one else has questions, thanks for giving me this hearing, and I hope I've at least stated my case clearly.

#202 ::: Annie Y ::: (view all by) ::: May 13, 2015, 03:48 AM:

[SPECIFIC LANGUAGE]
(well... could not think of a better tag)

I was looking at 3.8.5
With the current rule, you need to count anyway so there is no point counting what is under 5% before you start the proper counting.

With the proposed change, doesn't it make sense to do straight counts first to eliminate all under 5% (because otherwise there is a chance that the elimination removes a work over 5%, leaves in the ballot one under 5% and then we are in the ineligible conversation again.

So shouldn't we first remove everything that we know cannot make the ballot (

#203 ::: influxus ::: (view all by) ::: May 13, 2015, 04:14 AM:

[SPECIFIC LANGUAGE] Annie y @ 202,

(Hopefully I'm following you this time.)

Given the exception to 3.8.5:

"except that the first three eligible nominees, including any ties, shall always be listed."

I don't think it is possible to rule the less than 5% out altogether. Also making something explicit around this would surely make the proposal contingent on keeping the 5% rule. I think you have just found another nugget for the withdrawal/ineligible conversation.

#204 ::: Annie Y ::: (view all by) ::: May 13, 2015, 04:34 AM:

Note to self - look before pressing post - sorry - one of the special characters ate the bottom part of my message and I managed to miss it in preview. Somehow. Need either coffee or sleep.

@203 influxus

Yup - this is what I was driving at. The rest of the message was talking about some other options besides full elimination early on - mainly to say to run all and grab the top 5 that are over 5% (maybe abi can fish it out from the backend?)

I still like the idea to just build the full list and then grab the first 5 that are over 5%. If we need replacements, we go down the list for the next one. It won't be mathematically perfect but we need to make allowances for things already on the ballot and for the 5% things which is a lot of non-perfect allowances already.

Otherwise we may need to rerun the thing a lot of times to fish the over 5% story out from the pull of the under 5% (and in some categories, it is a close call indeed on what is over 5 and what under 5 and the fractionals will allow some of the over 5 to slip down while the just under 5 raise up)

#205 ::: influxus ::: (view all by) ::: May 13, 2015, 05:14 AM:

@204 Annie Y

I did wonder about that final bracket.

IANAL, but... :-)
Does grabbing the first 5 over 5% violate 3.8.5 as it stands? Shouldn't it be the first 3 that are grabbed, and then 4 & 5 are selected from the next two works that have over 5%, if such works exist?

Maybe stating something along the lines of a general rule for removal due to late discoveries of ineligibility, withdrawals, and being below the threshold (5%) is easier.

I am also in favour of this rule being based on treating the first complete list as fixed and just moving down that list.

#206 ::: Annie Y ::: (view all by) ::: May 13, 2015, 05:26 AM:

That will depend if under 5% is considered a reason to make the work ineligible.

The way I read it is that it is there to guarantee that we have at least 3 works in any ballot. However - when 3.8.5 was written, the top 3 were always the real top 3 (so there was no way something with more than 5% to get under them if they were under 5% (or at least #3 was)).

However with the new proposal #3 can be 4.9% for example with 2 or 3 over 5% under it. So is #3 in this case protected under 3.8.5 and staying or is it eliminated because of the 5% rule as long as there is at least one over 5% work under it? No idea. We need someone that understand the rules a lot better to weight in (and we may need rewriting of 3.8.5 a bit to fit better). I think it is not protected but I am the first to admit that I am not good with legalese. :)

#207 ::: felice ::: (view all by) ::: May 13, 2015, 05:33 AM:

Annie Y @204: "I still like the idea to just build the full list and then grab the first 5 that are over 5%."

Anything below 5% needs to be eliminated upfront before feeding the nominations into SDV-LPE. Doing it the other way around would be deliberately introducing more of the same sort of unfairness caused by not rerunning after a withdrawal, and has absolutely no benefits. If there are fewer than 3 works above 5%, then don't eliminate anything upfront, but run SDV-LPE until three finalists are left instead of five. I believe that's a reasonable interpretation of how the existing 3.8.5 should be applied under SDV-LPE.

The 5% threshold is a bad thing and should be abolished, but that's a separate issue.

#208 ::: J Thomas ::: (view all by) ::: May 13, 2015, 05:44 AM:

158 ::: Annie Y ::: (view all by) ::: May 12, 2015, 11:10 PM:

@158 Annie YJ Thomas

"If we do make it as open and accountable as possible, we might find ourselves providing people with enough data that they can interpret it as the voting administrators doing something wrong."

And that is easily provable by releasing all the data if need be.

Well, no. The data will tend to show that we did not corruptly falsify the results.

But -- for example -- if we run the voting twice and get two different results, we can expect a controversy about which result is right.

Run it once, get Andrew, Betsy, Charlie, Drusilla, Edward. We publish this with numbers.

Andrew is removed, we run it again and get Betsy, George, Drusilla, Henry, and Inigo. We publish this with numbers too.

We should definitely declare Betsy and Drusilla winners. George almost certainly. Charlie, Henry, Inigo, and Edward are less certain.

If we publish both outcomes we have trouble. P J Evans argues that we shouldn't worry about it because today's problem is dealing with slates. I don't think we should mine our own future path. "Don't worry about it, it's only one little landmine and we're very unlikely to step on it."

One solution is to announce that we will never re-run the data. Take everything from the old data and never let anybody find out what would have happened if it got rerun. If people can't find out who would have won otherwise, then they won't argue about it.

A second solution is to rerun the data and declare that this is the correct result. Don't publish the old, incorrect data because it would cause trouble. Or publish it with the observation that Andrew threw it off.

Publishing two different ways to count the votes and including some winners from one and some from the other is just asking for argument.

We get the same problem though less if we publish two results, and either take the #6 from the obsolete result or the top new result from the new result, particularly if it's #4 or higher. Between those two choices, we do better to take the old #6 and never run the data again or allow anyone else to.

The most defensible approach is to rerun the data following the rules, and accept what it says. If people disagree, point out that this is the correct result without Andrew. But that's bad for authors who were told (or implicitly given reason to think) that they had won and who then lost.

The next-most-defensible approach is to run the data once and use those results, and never let anybody run it again.

This is not related to slates. This is a new problem that might never come up because with luck it will not happen before the rules get changed again.

#209 ::: Annie Y ::: (view all by) ::: May 13, 2015, 05:49 AM:

felice@207

In which case in most categories we will have an elimination round or 3 and we will be done. I really do not like removing all the under 5% before running the eliminations - it will kick a huge percentage of people's ballots out immediately, changing the vote distribution (especially when there is a lot of works around 5% but just under it).

Let's look at London:
Novellete has 7 works over 5% - that's 2 elimination rounds
Short story is at 4 (not too surprising really)
Best Related has 7 - so 2 here as well.
Even novel is with 11 over 5%. Which means 6 rounds if we start from these.

So what 15 rounds of elimination are we talking about (for the publication)? That will never happen (you need 20 nominees for that)...

#210 ::: influxus ::: (view all by) ::: May 13, 2015, 05:50 AM:

felice @ 207:

There is a big reason not to do that. In the short story category you could wipe out almost everything but a slate. IE reduce the field to 6 before SDV-LPE.

#211 ::: felice ::: (view all by) ::: May 13, 2015, 06:12 AM:

J Thomas @208: "Publishing two different ways to count the votes and including some winners from one and some from the other is just asking for argument"

What about including all the winners from both? Betsy, Drusilla, George, Charlie, Henry, Inigo, and Edward, in this example. It's going to be pretty rare for rerunning to change the outcome that much.

Anything that relies on keeping the data secret is a bad idea.


Annie Y @209: "In which case in most categories we will have an elimination round or 3 and we will be done. I really do not like removing all the under 5% before running the eliminations - it will kick a huge percentage of people's ballots out immediately, changing the vote distribution"

Changing the vote distribution is important! If works aren't eligible, people's votes need to be redistributed to works that are (if they nominated any). Intentionally having ineligible works eliminating eligible ones makes no sense at all.

"So what 15 rounds of elimination are we talking about (for the publication)?"

Obviously we can only publish the rounds that happen; if that's less than 15, it's less than 15. This should probably be a supplement to the existing publication of the top 15 nominations, rather than a replacement for it.


influxus @210: "There is a big reason not to do that. In the short story category you could wipe out almost everything but a slate. IE reduce the field to 6 before SDV-LPE."

I agree that's a bad thing, but the problem is the existence of the 5% threshold, not how it applies to SDV-LPE in particular. We've got exactly the same problem with the current system.

(As an alternative to abolishing the threshold, I've got a longlist proposal that would increase nominations for the most popular candidates, but that's on hold till after SDV-LPE is finalised - search for "Option 5b" in the previous thread if you're interested)

#212 ::: influxus ::: (view all by) ::: May 13, 2015, 06:13 AM:

J Thomas @ 208:

"One solution is to announce that we will never re-run the data. Take everything from the old data and never let anybody find out what would have happened if it got rerun. If people can't find out who would have won otherwise, then they won't argue about it."

Have you met humans?

Anyway, if we only run the data once, and

(1) this is part of the rules, and
(2) we release all the data.

Then any controversy about alternate histories can simply be referred to the rules.

My argument for not rerunning boils down to:
The rules are fair because a fair procedure was used to produce the ranked list. Removals were not removed as part of ordinary eliminations, hence they are a special case. Unless there is a clear reason otherwise the simplest rule should be used to fill special cases. Given the length of this debate over removals, it seems pretty clear that there is no definitive reason rerunning is fairer than keeping the list intact. Keeping the list intact is simpler.

#213 ::: Ben O. ::: (view all by) ::: May 13, 2015, 06:23 AM:

On [DECLINED NOMINATIONS], here's another vote for the "run once" option articulated in #43. While there's been much discussion of the nominal theoretical benefits of rerunning the nomination data in the event of a declined nomination, we also need to recognize that it will take a certain amount of time and effort to generate a revised calculation. What would rerunning the nominees entail - can we assume that there will be robust code that asks the user to enter one (or more) nominees to ignore, or will the nominee data need to be massaged? What if a nomination is declined at the last minute and the final list is due at the printers in a few hours - it would be a shame to mistakenly select the wrong nominee just because the exigencies of the moment didn't allow for a proper calculation. It may sound like a picky point, but all this will be on the shoulders of volunteers who probably have jobs and families.

On another point, it seems to me that it may be worth setting an upper limit on the number of nominations that a person could make in a category. Without one, odds are that some joker will nominate a thousand titles. A generous but reasonable limit shouldn't give cause for complaint - ten? twenty? If someone nominates more than the limit, their ballot should be disregarded for that category.

#214 ::: J Thomas ::: (view all by) ::: May 13, 2015, 06:32 AM:

@178 Brian Z

My gut reaction to this is that people may be given more incentive to nominate fewer works under the proposed system.

That depends on their gut reactions, which are hard to predict.

If one person does that, it won't have much effect at all. If a bunch of people do it, it has a bad effect on the system as a whole, and probably doesn't help them since they'll tend to act against each other.

In 2013, people nominated 475 novels. 5 of them won. If you care more about increasing the probably small chance that your most-favorite work wins than about increasing the chance that one of your five most-favorite works win, then you should vote for just the one.

It's always been that way. Quantitatively it isn't much more that way now. It's hard to quantify, and probably people will go by their gut feel, and I can't predict that but I hope they won't go that route.

#215 ::: Annie Y ::: (view all by) ::: May 13, 2015, 06:32 AM:

Ben O. @ 213

There is no proposed change in 3.7.1 which caps the nominations at 5 (exact text: "up to five (5) equally weighted nominations in every category.") so the above text need to be read in the context of this restriction.

#216 ::: influxus ::: (view all by) ::: May 13, 2015, 06:39 AM:

felice @ 211: "I agree that's a bad thing, but the problem is the existence of the 5% threshold, not how it applies to SDV-LPE in particular."

But we are discussing how it applies to SDV-LPE in particular. If SDV-LPE gets up and the threshold is not removed SDV-LPE should behave as robustly and simply as possible. I think folding ineligibilities, withdrawals and threshold eliminations into a single rule is important for simplicity.

#217 ::: J Thomas ::: (view all by) ::: May 13, 2015, 06:42 AM:

@211 felice

"Publishing two different ways to count the votes and including some winners from one and some from the other is just asking for argument"

What about including all the winners from both? Betsy, Drusilla, George, Charlie, Henry, Inigo, and Edward, in this example. It's going to be pretty rare for rerunning to change the outcome that much.

Yes, that sounds excellent to me. Usually it would make no difference. Rarely it would be different and you'd have six. Nobody is hurt much if it happens -- six works competing on IRV instead of five. If we assume that the two candidates that only won once will tend to be the weakest ones, they don't much hurt each other or anybody else when they both get their chance.

So the result would be six candidates rarely, seven candidates extremely rarely, you can publish any data you want and nobody should have a gripe with the result.

Beautiful!

#218 ::: influxus ::: (view all by) ::: May 13, 2015, 06:46 AM:

Ben O. @ 213, Annie Y @ 215:

Maybe this issue of capped nomination numbers should go in the FAQ. I made this mistake, as did another person on File770.

Can I nominate hundreds of works?

There is no proposed change in rule 3.7.1 which caps the nominations at 5 (exact text: "up to five (5) equally weighted nominations in every category."). SDV-LPE is still governed by this rule.

#219 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 07:20 AM:

I agree with @181: insofar as BZ is maintaining the appearance of legitimate debate, it doesn't matter if in his heart he is trolling. We should be careful about three-goalpost-monte, but if he raises something that others will worry about too, we should respond without name-calling.

So, what are the legitimate points he has raised?

He's brought up the thing about bullet voting. I think we've dealt with that one. Joshua @123 did a good job explaining why it's not actually a very powerful strategy. A few further points: in the end, if a few people do use this strategy, the system is robust to that; and they probably would have done it under the current system; and if they'd do it under any system, you can hardly even call it strategy, because it's just how they feel. I think if people come back to this point again, we can just point them to the same arguments, and not get distracted by restating them in different words.

He's brought up the question of what the goal is, and alleged that the goal of SDV-LPE is to maximize voter happiness. If he were a troll, this would be pretty sophisticated work; given a random member of my electology organization, probably the quickest way to troll them is to argue against voter happiness as a metric. But actually, even though voter happiness is largely the point of my organization, it was not the point when I designed this proposed Hugo nomination system.

The point of this system (SDV-LPE or whatever we call it) is to try to ensure that the eventual Hugo winner is excellent. The reason that the nominees should represent a variety of fans is not to make them happy, but because by representing a variety you avoid making the same mistake 5 times.

Are there cases where this system would pass over a more excellent work to nominate a less excellent one? Sure; there are several ways that could happen.

First, if the voters are all wrong about something, then there's no way a voting system can get it right. As far as I can see, there's nothing more than that to the "Macroscope" examples. If Spinrad and Vonnegut got votes only from people who'd already nominated "Left Hand", while "Macroscope" was supported by a separate but slightly smaller group of fans, then yes, there is a possibility that Spinrad and Vonnegut would end up together in a matchup, and that only one of those two would make the final ballot. There is not a possibility that both would be eliminated. So the final list might have LeGuin, Vonnegut, and Anthony, while Spinrad might be knocked out. If that's what the voters said, then that's the result.

Separately, assuming voters do NOT bullet vote, there is a chance that a well-crafted but essentially formulaic work could beat out an actual masterpiece. But imagine what that would look like on the ballots: "Formula 1" has support from 40% of the voters, while "Master and Apprentice" has support from 15%. So while I can imagine designing a voting system that would be more likely than SDV-LPE to elect "Master and Apprentice", I can also say that this hypothetical "pearls before swine" voting system would get things embarrassingly wrong (think "Chthon", by Piers Anthony) at least as often as it got a better answer than SDV-LPE.

I'd be interested to hear if other people think this is an adequate response to BZ's concern.

#220 ::: Louis Patterson ::: (view all by) ::: May 13, 2015, 07:30 AM:

Not a hugo voter, but:

If a work is determined to be ineligible countback/rerun is really the only option, I think. If work X shouldn't have been included in the count in the first place then including it is an error and the only way to fix that error is by rerunning the process properly, with ineligible entries excluded.

If the two runs generate different shortlists then it's pretty obviously the first one -- that counted the ineligible works -- that's in error, and it's the ones from the initial list that are getting concessionary treatment if [as I think they should be] they are included in the final shortlist.

[it's unclear to me how/if this applies to voluntary withdrawals]

#221 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 07:33 AM:

[DECLINED NOMINATIONS]

I think that each worldcon committee should have the freedom to predeclare whether they want to rerun and merge lists, or just take the last out, first in candidate. This predeclaration should be made at any time prior to the close of voting. If no such predeclaration is made, the default should be last out first in (under the assumption that a committee which misses the declaration deadline is one which can't handle complexity).

Remember, our best estimate, grounded in the 1984 data, is that this would only make any difference for under 1 in 100 cases of declined nominations.

#222 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 07:42 AM:

Oops. When I said "Chthon", I meant "Phthor".

#223 ::: J Thomas ::: (view all by) ::: May 13, 2015, 09:05 AM:

@219 Jameson Quinn

If Spinrad and Vonnegut got votes only from people who'd already nominated "Left Hand", while "Macroscope" was supported by a separate but slightly smaller group of fans, then yes, there is a possibility that Spinrad and Vonnegut would end up together in a matchup, and that only one of those two would make the final ballot. There is not a possibility that both would be eliminated.

This is a very minor thing, but they could both be eliminated as follows: If LeGuin also had a lot of shared ballots, LeGuin could also have a low enough score to eliminate both Spinrad and Vonnegut, or whichever of them survived their own matchoff. We can assume something else in 2nd and 3rd place, so this would require that Anthony and another both have scores higher than LeGuin has during the last elimination round. In the extreme case, if all of Spinrad's votes are shared with LeGuin and the 2nd place and 3rd place, and all of LeGuin's are shared with the 2nd and 3rd place, Spinrad, and some of the lesser works that haven't been eliminated yet, then Anthony and the others could have a little more than 1/4 as many votes as LeGuin while Spinrad and Vonnegut both have almost as many as Leguin.

But the extremely rare pathological cases don't really matter much. It could be argued that they aren't really even pathological, the system is doing what it's supposed to do. Just not quite what a naive voter might expect.

But a naive voter should expect slates to win big. If the voting system does everything he expects, that isn't good either.

#224 ::: JAB ::: (view all by) ::: May 13, 2015, 09:24 AM:

I like the idea of SDV. To be honest, though, it didn't really click for me until I started making a physical model in response to Bruce Baugh's request in @52. I don't see that anyone has posted one yet, so I'd be glad to make and share photos of it.

But I need an example to illustrate. Does anyone have a link to a very small example that I can implement? Is there an official small example? I'm just dipping in after not following the previous threads closely.

#225 ::: Seth ::: (view all by) ::: May 13, 2015, 09:39 AM:

If eliminating every candidate that appears on under 5% of the ballots leaves fewer than three, then don't do it (obviously). In that case, run the elimination rounds down to 3 (plus ties).

If eliminating every candidate that appears on under 5% of the ballots leaves 3-5, then done. Those appear on the final ballot (subject to declined nominations).

If we use re-run in the event of declines (which is the right thing to do), then there's a potential ambiguity with multiple declines. I would say that in the case of simultaneous declines, re-run with all of them eliminated. Else, use sequential re-runs. I define "simultaneous" as the new decline becomes known to the administrators prior to notification of the new potential nominee(s) after a re-run.

#226 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 09:42 AM:

@223: The scenario you paint, where both Spinrad and Vonnegut are knocked out despite being 4th and 5th place and not having significant overlap with 2nd and 3rd place, allowing 6th place Anthony and 7th place whatever as nominees, may be possible. But I'd bet that it would take me at least 3 tries to put numbers on it to make it work right. If somebody like me, mathematically sophisticated and steeped in voting theory, can't get it to work on the first try, I think that the probability of it happening by chance are effectively 0 — that is, less than one in ten thousand.

#227 ::: J Thomas ::: (view all by) ::: May 13, 2015, 09:59 AM:

If somebody like me, mathematically sophisticated and steeped in voting theory, can't get it to work on the first try, I think that the probability of it happening by chance are effectively 0 — that is, less than one in ten thousand.

I agree it's very unlikely. It isn't very important that it's possible, I just got caught up in the moment when I saw you say it was impossible.

What's impossible is for Leguin with the most votes to lose. No combination of improbable voting circumstances can get that result. Also, nothing with less than 1/5 of Leguin's starting score can possibly win, provided there are at least 4 others with more than 1/5 of Leguin's starting score.

I thought it wasn't completely irrelevant that the extremely rare event could happen. Noticing how it could happen might help people to better understand the system.

But it isn't worth a lot of talk unless someone doesn't understand it.

#228 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 10:14 AM:

JAB@224: I can run anything you need. Can you describe the type of set you're looking for?

[DECLINED NOMINATIONS]
All:
I've been reading all the posts on the subject, and I think we can all agree that there are two very distinct opinions on "fairness" with regard to this subject. I don't think we can say, therefore, that there is an objective definition of "fairness" in this case. My thoughts are this:

- The current constitution does not specify how to handle declined nominations. We have a method based on tradition, but that's it. Right now, it's ultimately up to the Hugo admins to handle it however they like. Strictly speaking, they don't have to replace -anyone- if the nomination is declined.

- Therefore, let's keep it this way, and let's allow tradition to grow and decide what's "fair". As a first cut, the Hugo admins could decide to stick with the current tradition and move the work eliminated in the next to last round to the ballot.

- If, after a year or so, fandom decides that they don't like this tradition, the Hugo admins can try something else (such as re-running the election) and see what fandom thinks about that. The important thing here is that as it stands, this would -not- require a constitutional amendment.

I think doing it this way gives the Hugo admins the agility to respond to what fandom decides is fair. More importantly, it let's fandom decide what's fair. Organically grown tradiations are kind of what our community is about. I think this is a perfect opportunity. If a new tradition is needed, it will develop naturally, so long as the Hugo admins have the freedom (as they do now) to handle it as they see fit.

Kilo

#229 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 10:30 AM:

[5% RULE]

I'd like to see the 5% rule eliminated, as I don't really think it serves its intended purpose. However, I believe Kevin said there is already a proposal to do that, so I don't think we should include it in ours. However, we do need to be sure we can handle it if the 5% rule is not removed.

If the 5% rule is not removed, I think the simplest way -- which draws back to the previous discussion -- is to treat any work getting less than 5% as an ineligible nomination. However the admins decide to handle those is used here.

That said, however, I will lobby extensively in support of those removing the 5% rule, as I think that's the most logical course of action.

Kilo

#230 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 10:43 AM:

Annie Y @194:

I am not really up to speed with all the rules but it is possible to change things during the ratification in Kansas, right?
Yes and no. Here is the exact wording, right out of the current WSFS Constitution:
Section 6.6: Amendment. The WSFS Constitution may be amended by a motion passed by a simple majority at any Business Meeting but only to the extent that such motion is ratified by a simple majority at the Business Meeting of the subsequent Worldcon.
This means that if you pass a change to the constitution this year, you can modify it next year, but only if doing so does not increase the scope of the amendment.

Example: Suppose we pass an amendment reducing the number of nominations per member per category from 5 to 3. Next year, you could move to change "3" to "4" and ratify it immediately because it would be less of a change than what was originally passed; however, if you changed it to "2", the change would be greater than what originally passed and the motion would have to lay over for one more year. (Possibly longer, if the Popular Ratification proposal is ratified this year.)

Wholesale reworking of a proposal is not allowed, and the more you try to change something at the ratification stage, the more likely it is to be ruled a greater change and thus require an additional round of ratification. I also suggest that anything that completely rewrites the existing nomination system, effectively throwing away the current system and starting over, would be very difficult to change at the ratification stage in any way that would not be considered a greater change.

#231 ::: Lydy Nickerson ::: (view all by) ::: May 13, 2015, 10:50 AM:

A quick data point, here. I am hugely math-challenged, but I don't find divvying up 60 points, or 100 points, easier to understand than divvying up one point into fractions. In some ways, I actually find the multiple points more confusing, not less. It requires an additional amount of explanation as to why you choose that many points, whereas one point divvied up between the total number of nominations seems easier to understand. If, in fact, it's numerically identical, then I'd go with the one vote thing.

Oh, and I _hate_ fractions.

#232 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 10:53 AM:

Keith @229:

I'd like to see the 5% rule eliminated, as I don't really think it serves its intended purpose. However, I believe Kevin said there is already a proposal to do that, so I don't think we should include it in ours.
That is correct; there is such a proposal already submitted. I'm discouraging anyone submitting additional proposals from trying to touch that section. Indeed, I've recommended that anyone doing omnibus changes try to break them into individual pieces so that the meeting can divide them into separate questions. (There's a proposal cooking up on Facebook, for instance, to combine the two "middle length" story categories and to carve out a "Best Saga" category for very long series works; I asked them to draft it as two clauses so they can be considered separately if the meeting so desires. They've put the revised Best Saga proposal on Google Docs for public scrutiny.)

We now have under the WSFS Business Meeting section of the Sasquan web site a list of New Business Submitted to the 2015 WSFS Business Meeting. We'll update this as we receive additional proposals.

#233 ::: Yarrow ::: (view all by) ::: May 13, 2015, 10:53 AM:

[DECLINED NOMINATIONS]

I'm writing mostly to support Kilo's "let the Hugo admins handle it" position.

I also think that, on one view, moving the next work up is not only less work than rerunning the process, but also more fair.

On that view, what a nominator gets when a work they like gets on the Hugo ballot is the opportunity for that work to be sent to the Hugo electorate. (Presumably this part is not controversial!) And what a nominator gets when a work they like is about to go on the ballot - that is, just before the work's author gets the call from the admins - is also the opportunity for the work to be sent to the Hugo electorate. But for that electorate, one voter (the author) gets a veto - if the author declines the nomination, the work doesn't get the Hugo.

So when an author declines, nominations for that work have not been wasted, any more than nominations have been wasted for works that turn out not to win the Hugo. In both cases, the work has passed the first gate, of getting on the short list. If an author declines the nomination, the work has not passed the second gate, and if the Hugo voters don't select the work, it has not passed the third gate; but in neither case have the nominators' choices been ineffective.

#234 ::: Lydy Nickerson ::: (view all by) ::: May 13, 2015, 10:56 AM:

And an additional thought: if it is described as divvying up some number of points, a lot of people are likely to want to know why they can't assign their points as they please. 35 points to choice A, 10 points to choice B, and so on, up to the number of points allotted. Which is so totally not what you want.

#235 ::: Buddha Buck ::: (view all by) ::: May 13, 2015, 11:02 AM:

Kevin Standlee, while you are here,

Would it be possible for the WSFS Business meeting to pass multiple, possibly contradictory, revisions to the nomination process, allowing those options to be discussed and experimented with until the following year, when only one (hopefully) would be ratified?

#236 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 11:04 AM:

Lydy@234

Regarding 60 points vs fractions and wanting to divide however the voter wants..

That's a good point. That argument actually has me convinced that we should just stay with a single point and divide it up.

Kilo

#237 ::: P J Evans ::: (view all by) ::: May 13, 2015, 11:24 AM:

233
THIS.

#238 ::: Cadbury Moose ::: (view all by) ::: May 13, 2015, 11:59 AM:

P J Evans @ #237 wrote:

233

THIS.

Indeed, and any declined nomination is not wasted since it will still appear in the published nomination stats flagged as (declined) after the awards have been made.

(So no rerunning of the selection is necessary (or, for that matter, desirable).)

#239 ::: P J Evans ::: (view all by) ::: May 13, 2015, 12:09 PM:

238
Andy Porter declined in 1984 for semiprozine (for 'Starship') - that's on the 'more details' page. There's some stuff going on there that I never knew or don't remember, since he doesn't seem to have had the votes anyway.

Timothy Zahn had the nominations for the Campbell, but he wasn't eligible, so they moved up the next two nominees (there was a tie).

#240 ::: Nancy Lebovitz ::: (view all by) ::: May 13, 2015, 12:26 PM:

I'm wondering whether an animated explanation-- watching the votes come in and seeing the ranks move up and down-- would help.

This is very tentatively suggested.

#241 ::: Bruce Baugh ::: (view all by) ::: May 13, 2015, 12:38 PM:

I spent some time last night thinking about suggesting a no-fractions approach, then had the same realization that Lydy describes in #234, and thought, nope, sit on that one. :)

#242 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 12:55 PM:

Buddha Buck @235:

Kevin Standlee, while you are here,
I've been watching this discussion, so assume I'm always here.
Would it be possible for the WSFS Business meeting to pass multiple, possibly contradictory, revisions to the nomination process, allowing those options to be discussed and experimented with until the following year,...
Possible, in the sense of not prohibited, yes. I personally think it's a bad idea and discourage it, but I recognize that there seem to be a bunch of people who are convinced that We Must Change Everything In Every Possible Way it Can Be Changed. *sigh* Also, I don't like dumping a parliamentary mess into my successor's lap.
...when only one (hopefully) would be ratified?
In case of contradictory proposals, it's axiomatic that only one can possibly be ratified. How we get to that point is not yet clear. In the absence of any other decision (see below), I'd rule (if it were my call, which it is not) that the last thing ratified is the most recent change, and that it overwrites anything ratified before it. That's not terribly satisfying, I admit, and therefore I really hope that people come up with ways to not have their proposals contradicting each other.

(One possibility would be to put all conflicting proposals up to a ballot vote by IRV, which would guarantee that only one of them could get ratified. But that has its own baby-with-the-bathwater problems.)

As far as "experimenting" with them goes: Nothing passed this year will be in effect next year. I don't know what you mean about "experimenting" with them. You can't compel next year's Administrator to do anything not in the current rules. If you think you're going to convince the 2016 Hugo Awards Administrator to run multiple simultaneous versions of a voting system in order to test theories, I suggest you contact him first to find out how plausible that is. (Not very, IMO, but I'm not Dave McCarty and therefore do not know for certain.)

#243 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 01:10 PM:

@242: "(One possibility would be to put all conflicting proposals up to a ballot vote by IRV, which would guarantee that only one of them could get ratified. But that has its own baby-with-the-bathwater problems.)"

Aaak, no! From a voting theory perspective, the clearly "right" thing to do is to use approval voting; that is, whichever passes with the most "yea" votes is the one that stands. That's what the state of Oregon did when they had that problem one year, and it avoids the complicated ballots and potential pathologies of IRV.

One way to do that would be, in the ratification year, add language to each of the proposals saying "If this proposal does not get more yea votes than either X or Y, it is null and void." Since that's clearly a reduction of scope, it is OK to add it in year 2.

But I agree with you that by far the best way to deal with this problem is not to have it.

...

Seperately: Is there currently a proposal to lift the 5-works-per-category-per-ballot limit? I think that such a proposal would go well with this voting system, but they should not be tied together.

#244 ::: Tim Illingworth ::: (view all by) ::: May 13, 2015, 01:41 PM:

@243 I think I'd treat them all as amendments-by-substitution to a null motion.

#245 ::: JAB ::: (view all by) ::: May 13, 2015, 01:51 PM:

Keith Watt @228:

Thanks for the offer. What kinds of ballots to include comes down to, what are the important parts of the system? What does the model need to show? What sorts of scenarios should be included? Definitely some slates; what else? How much should the ballots overlap on works? As I said, I don't know what would be most illuminating. Coming up with a good small but nontrivial numerical example to demo the system is hard, and right now I am too overwhelmed to spare the skull-sweat.

So, here's what I thought of: (and if someone else wants to do it -- especially if they can animate, go ahead with my blessing)

1. Ballots are equal-sized strips of thick cardboard, 1 per voter. The nominated works are represented as equal-sized stripes. Each work in the pool has its own color (+pattern for the colorblind folks). So a ballot for ABCD has 4 stripes, red, blue, green, and yellow. A ballot for CD has the top half green and the bottom half yellow. A ballot for E only is a solid black. And so on.

2. Each strip is cut apart on the color lines and the votes for each color are lined up (basically, a bar chart).

3. Each of the 2 shortest bars are picked up and their components stacked on top of each other. The shorter stack is out. (This is why chipboard/binder's board, not paper or cardstock).

4. Then the ballots are reassembled minus the missing strips. Some of these ballots are now smaller/shorter than others. The purpose of this picture is to show why it's fair to reweight votes and/or be a nice sound bite for "what's wrong with the existing system".

5. The ballots are shaved down so that they're the same size again (proportionally for each color). There's a sidebar that shows how equal amounts were removed from each ballot and it was split between remaining colors. Going back to the example above, if A is eliminated, the CD ballot loses 2 strips (one green and one yellow), each half the width of A. The E ballot loses a strip the size of A, so that now the BCD ballot is again the same size as the CD ballot and the E ballot. Etc etc.

6. Rinse and repeat until n nominees remain.

Basically, cutting is going to be a pain. So I'd want enough voters and works to show what's going on, but only 3-4 rounds to get winners.
It'd be nice if the fractions were easy to cut -- so fewer works per ballot, but not too few.

Tie-breaking and nominees declining is overloading the cardboard strips. But it wouldn't be hard to show the minimum nominations rule - just stack up by works (colors) and have a "You must be this tall to ride" sign. Is that important to include?

There's probably an easier way to do it; feel free to suggest how.

#246 ::: Brian Gibbons ::: (view all by) ::: May 13, 2015, 01:54 PM:

Jameson Quinn @243: Is there currently a proposal to lift the 5-works-per-category-per-ballot limit?

I would be surprised if there were not a proposal to simply have the final ballot include every nominee that cleared the 5% threshold. That seems to me to be the obvious philosophical opposite of this one: Rather than going to great lengths to determine who the 'best' top five nominees are, just put everyone who qualifies on the ballot.

That obviously comes at the expense of making 'Hugo nominee' a much less meaningful descriptor (10 nominees would become fairly commonplace, and even 15 not unheard of), but if you think that internecine fights are already doing that and will continue to do so, it might be a cost you're willing to pay. It also has obvious appeal to anyone who prefers simple solutions.

It's not clear to me that passing multiple contradictory proposals to Kansas City is the worst possible outcome.

Picking the One True Change at Sasquan means that next year's BM is going to be faced with a "take it or leave it" proposition. If you think that this a serious problem that needs to be solved (and I don't think we should be amending the Constitution if it isn't), then that's essentially giving them no choice but to ratify whatever this year's BM does.

In a sense, picking just one of the options this year is the "Adopt in Haste" path. I have to assume that a meaningful percentage of attendees at the Business Meeting are to be learning about the issues and possible solutions for the first time, and I'm not sure that it's a good idea to force an immediate choice as to which good option is the best one.

I don't think it would be a bad thing if this year's Business Meeting was treated, well, like picking nominees for a final ballot. Winnow through the candidates, of course, but if there are multiple options that the voters think are acceptable, I think that leaving for next the choice as to which is the best option is not a bad thing.

#247 ::: Fragano Ledgister ::: (view all by) ::: May 13, 2015, 02:19 PM:

It is correct that this system would weaken slates, though not completely eliminate them. It would, in effect, tend to benefit bullet voting.

Also, I want the time I wasted reading Macroscope back.

#248 ::: Buddha Buck ::: (view all by) ::: May 13, 2015, 02:24 PM:

When I talked about "experimenting" with them, I meant in much the same way we are doing here: Writing different implementations, testing them with variants of ballot data, seeing how resilient they are to various forms of strategic nominating (slates, bullet nominations, etc).

I would definitely consider it the responsibility of the promoters of the various proposals to do the experimenting, not the Hugo Administrators.

#249 ::: Chris ::: (view all by) ::: May 13, 2015, 02:27 PM:

JAB @245:

5. The ballots are shaved down so that they're the same size again (proportionally for each color). There's a sidebar that shows how equal amounts were removed from each ballot and it was split between remaining colors. Going back to the example above, if A is eliminated, the CD ballot loses 2 strips (one green and one yellow), each half the width of A. The E ballot loses a strip the size of A, so that now the BCD ballot is again the same size as the CD ballot and the E ballot. Etc etc.

Rather than shaving ballots down, I'd suggest stretching the ones that have lost candidates back to full size. Mathematically identical, but the visual metaphor (you haven't lost any of your voting power when you lost a candidate) is better.

#250 ::: Annie Y ::: (view all by) ::: May 13, 2015, 02:34 PM:

Kevin Standlee @230

Thanks. So it will go down to what the Kansas business meeting attendees believe to be narrowing in scope.

[5% RULE]
So if there are other proposals to get rid of that one (yey - it made sense when the number of ballots was very low but not anymore...) shouldn't this proposal not quote the articles that are not changed (3.8.2-3.8.7) but add somewhere in the 3.8.8 section that all works that are ineligible under any previous articles are removed before the elimination rounds?

This way we leave it open enough for a change in 3.8.5 and we do not end up with accepting the old 3.8.5 again on top of already accepted new one?

It may be me just not understanding the process, but if this text as it stands is accepted, the non-changing sections are also getting re-accepted? Or do we add them here only for clarification (put them in brackets then?)

#251 ::: JAB ::: (view all by) ::: May 13, 2015, 02:39 PM:

Chris @249:

Yeah, that does work better. If I could animate on the computer, I'd do that. Shaving is a workaround because I was going to use actual chipboard and magic marker. I wonder if there's a better (household) material? Maybe play dough and food coloring, if I could figure out how to mix enough different colors.

#252 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 03:37 PM:

@247: In a world without slates, there is essentially no benefit to bullet voting. Approval totals and honest point totals would agree on which candidates were better, so if your favorite candidate was losing, boosting its point total would only put off the inevitable (and if everybody did it, you could actually be worse off, because your favorite would be getting no second-choice votes).

In a world with slates, there is a small chance that boosting a work's points will let it avoid becoming collateral damage to a slate. But... if slates can't get more than their share of slots, they're much less likely to exist.

#253 ::: viktor ::: (view all by) ::: May 13, 2015, 04:05 PM:

The answer to FAQ #10 should be changed to Yes.

Question: why can't I list a work more than once on my ballot? If I want Alpha to get more of my 1.0 than another nominee, I should be allowed to nominate as follows:

Alpha
Alpha
Beta
Charlie
Delta

Thus giving Able 0.40 pts.

#254 ::: P J Evans ::: (view all by) ::: May 13, 2015, 04:08 PM:

253
Because that's Not Done: it's a form of ballot-box-stuffing, which is Really Not Done.

(In the past, people have put stuff on their list more than once, and it's been counted - but only as one.)

#255 ::: viktor ::: (view all by) ::: May 13, 2015, 04:15 PM:

Under the proposed rules, I should be allowed to allocate my 1.0 anyway I want.

#256 ::: Laertes ::: (view all by) ::: May 13, 2015, 04:31 PM:

viktor @255: Can you elaborate on why that's necessarily true? It's not obvious. I might, for example, have some idiosyncratic urge to break my vote up into chunks sized 2^-2 and 1 - 2^-2 because I've got a thing for irrational numbers, but it's not clear to me that the rules are flawed if they don't permit this.

#257 ::: Joshua Kronengold ::: (view all by) ::: May 13, 2015, 04:32 PM:

viktor@253: It's an interesting form of ranking. But it's also more complicated, so we decided to go with a simpler, less nuanced proposal.

Annie@250: No, we want to list the unchanged bits because it provides context for the rest. Functionally, an "as follows" amendment is a diff. Strikeouts mark the bits that are being removed; underlines mark the bits that are being added (we used italics here because you can't do underlines on ML).

#258 ::: Laertes ::: (view all by) ::: May 13, 2015, 04:37 PM:

Laertes @256: Notation fail. Those numbers should be 2^(-1/2) and 1-2^(-1/2). Or to make the point more simply, 1/π and 1-1/π.

#259 ::: Joshua Kronengold ::: (view all by) ::: May 13, 2015, 04:38 PM:

It seems to be another retread of the "the rules are flawed if they don't let me rank" argument.

Functionally, the reason the rules don't allow ranking on nomination (either by repeat votes, scoring, or ordering) is because they are following the tried and true coding principle of minimal change. The only thing that -needs- to change to fix the block nomination issue is to restrict people to (effectively) one vote at a time. One can, of course, make arguments about and ascribe value to various ranking forms [as well as dropping the 5% rule, opening or keeping closed nomination visiblity, etc ad infinum]. But none of those are part of the base change, so keeping them out of the argument makes the changes smaller and therefore better.

#260 ::: Annie Y ::: (view all by) ::: May 13, 2015, 04:46 PM:

viktor@255

The proposed change does not change 3.7.1 which states very clearly: "up to five (5) equally weighted nominations in every category."
So you cannot give different weights. Combined with 3.8.4 which is also not changing: "If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category", it means that you cannot give more power to one entry.

So, no, you cannot split your point anyway you want.

#257 ::: Joshua Kronengold

Thanks. So what is getting voted on are just the changes (add these, remove these) and not the whole proposal as it is listed above. (Sorry for asking weird questions - I am just trying to get the hang of what's going on). So if 3.8.2 (for example) changes independently, because we are not changing it here, that change gets incorporated with our changes (if both pass).

#261 ::: junego ::: (view all by) ::: May 13, 2015, 04:51 PM:

Keith "Kilo" Watt @ 236
Lydy@234
Regarding 60 points vs fractions and wanting to divide however the voter wants..
That's a good point. That argument actually has me convinced that we should just stay with a single point and divide it up.

I saw both of Lydy's objections to my proposal and will bow to the expertise of someone who is "fraction phobic". ;-)

I had also wondered about people wanting to redistribute their points if there was more than 1 but thought that there was enough resistence to the idea of ordering nominations that it wouldn't be a big issue. OTOH, if others think it will just make the issue more complicated, I'm ok with dropping it. I'm all for KISS as much as possible.

#262 ::: viktor ::: (view all by) ::: May 13, 2015, 04:51 PM:

@257
Ranking is built-in to the proposal as is. However, by eliminating my duplicate changes my "ranking" in favor of some predetermined "ranking". Under the proposal, I can only allot 1.0, 0.5, 0.33, 0.25, or 0.20 to my nominees (assuming 5 nom slots). By allowing dupes, trips, quads, my options open to include 0.8, 0.75, 0.67, 0.6.

Yes, I think it's a flaw. Never before in the nominating round has a nom been allocated fractional points based upon the number of other noms on the ballot.

Other than that, I have no problem with the proposal.

@256
I think you're being silly.

#263 ::: Joshua Kronengold ::: (view all by) ::: May 13, 2015, 04:52 PM:

Annie@260:

Yes, exactly. Because they're not modifying exactly the same parts, the two don't cause any sort of conflict and can happily co-exist.

Practically speaking, the WSFS chairman will avoid (with extreme prejudice) the meeting passing contradictory amendments -- instead, instructing the proponents to create a merged proposal or a base proposal followed by a series of proposed amendments (if there are some disagreements) rather than pass a nightmare of contradiction.

There is also a committee of last resort in the form of the nitpicking and flyspecking commitee (which near as I can tell, is empowered to make small wording tweaks to the Constitution as part of integrating amendments as long as they don't change meaning, and would likely get involved in coming up with something coherent if WSFS managed to nearly contradict itself. But this is for obvious reasons to be avoided.

#264 ::: Aan ::: (view all by) ::: May 13, 2015, 05:08 PM:

Keith "Kilo" Watt @228:

[DECLINED NOMINATIONS]
Isn't a problem with leaving this free, and letting a new tradition form, that there's only going to be a difference in the results in 1% of the cases?
So if the first couple of times there's a declined nomation, it's decided to do a full rerun, then those reruns would likely lead to the same result as just picking #6. That means no one is going to be unhappy, and the tradition forms of always doing a full rerun. And then 10 years down the road, we get that 1% case cropping up, and suddenly a previously nominated author is off the ballot, and all hell breaks loose.

A nominated author no longer being nominated seems eminently "perceived to be unfair" to me, and something the proposal should guard against.

(Also: hi all! Don't think I'll be posting much, but I peeked in to see the current progress based on the file770 mention, and got suckered into reading the entire discussion, so here I am for now. I mentioned to a friend a while ago how I was confident that all y'all would come up with a great proposal, and I'm gratified to see how right I was.) :)

#265 ::: Seth ::: (view all by) ::: May 13, 2015, 05:14 PM:

#262 There is no ranking in this proposal. All of my nominations are treated equally.

#266 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 05:19 PM:

Jameson Quinn @243:

Is there currently a proposal to lift the 5-works-per-category-per-ballot limit?
No. The list of proposals submitted to the Business Meeting for this year is on the Sasquan web site and we will update it as we receive new proposals.

Brian Gibbons @246:

I would be surprised if there were not a proposal to simply have the final ballot include every nominee that cleared the 5% threshold.
Have you ever attended a WSFS Business Meeting?
That obviously comes at the expense of making 'Hugo nominee' a much less meaningful descriptor
That devaluation is why we stopped calling the finalists "nominees" a couple of years ago. People who got even one nomination started insisting that they were "Hugo Award Nominees" by the "plain meaning" of the term.
Picking the One True Change at Sasquan means that next year's BM is going to be faced with a "take it or leave it" proposition. If you think that this a serious problem that needs to be solved (and I don't think we should be amending the Constitution if it isn't), then that's essentially giving them no choice but to ratify whatever this year's BM does.
That's funny; I haven't heard of any proposals that include the phrase "no further Constitutional Amendments shall ever be allowed."

If next year's Business Meeting doesn't like what's submitted to it for ratification, there's nothing stopping them from rejecting the ratification of what's passed on and passing a new change that would have to be ratified again in 2017 (and by the members of the 2018 Worldcon, should the pending Popular Ratification amendment up this year be ratified).

Joshua Kronengold @263:

Practically speaking, the WSFS chairman will avoid... the meeting passing contradictory amendments -- instead, instructing the proponents to create a merged proposal or a base proposal followed by a series of proposed amendments (if there are some disagreements) rather than pass a nightmare of contradiction.
Yes, that's what I'll try to do; however, if the proponents can't agree, I can't force them to agree. And it will not surprise me a bit if we have multiple factions all convinced that they have the One True Answer. I will strongly suggest that people unable to reach compromises will not be in the majority. That, unfortunately, makes me a "politician," because I think in terms of something that can actually get enough support to pass as opposed to something that's ideologically pure.
There is also a committee of last resort in the form of the nitpicking and flyspecking commitee (which near as I can tell, is empowered to make small wording tweaks to the Constitution as part of integrating amendments as long as they don't change meaning, and would likely get involved in coming up with something coherent if WSFS managed to nearly contradict itself. But this is for obvious reasons to be avoided.
The NPFSC dislikes getting involved in pending legislation if it can be avoided. Our remit is mostly to try and codify existing practice and put it in writing, by maintaining the Rulings & Resolutions of Continuing Effect and to sometimes recommend rule changes that bring our written rules in line with what we actually do in practice.

#267 ::: junego ::: (view all by) ::: May 13, 2015, 05:26 PM:

Under the proposed rules, I should be allowed to allocate my 1.0 anyway I want.

Yoicks! Irony.

Obviously the 1 point rule isn't proof against someone wanting to rank their nominations. I still bow to Lydy's knowledge about those who dislike fractions. :-p

#268 ::: gin-and-whiskey ::: (view all by) ::: May 13, 2015, 06:01 PM:

I think it’s a bit facile to pretend that this is somehow more objective. It merely gives weight to a particular type of voting. The SP group you’re dealing with is a very strong willed minority, and this particular method is designed to basically disempower them. Sure, their votes would count even if they didn't make the ballot…. so long as they voted for stuff they don't like, thus contributing to disenfranchising themselves and keeping their items off the ballot. That’s a Hobson’s choice.

Also, on principle, weasel words like "genuinely popular" shouldn't be used to hide value judgments. What's more "genuinely popular:" something that is the absolute hands-down only favorite of 25% of the population, or something which is "OK, I guess" for 80%?

As a comparison, imagine a rule where you got 10 points, and you can spend your points as you wish, across as many categories as you wish…. but without the initial popularity screen.

In that scheme, if you want to spend all 10 points on “best novel,” you can do so. But you lose your say for all other categories. OTOH, if you care a bit about every category instead of caring strongly about only one, then you can spend one point on each… but not have as much effect. You can’t do both. And since there are multiple categories and multiple nomination slots, you basically have to choose how much you care about things–you can’t game multiple categories.

This would practically limit the effect of slates, but would not wholly eliminate minority viewpoints. It’s a bit like politics: if a group reeeeeeally cares enough about a very small and limited # of nominations, why shouldn’t they be able to get a win on a small field?

Also, it deters worthless voting. IOW it forces people who are nominating to actually consider what is important to them. You can certainly “give 1/100 of a point to everything that isn’t a Sad Puppies recommendation” if you want to, but if you were planning to do that for 40 books which you hadn’t read just because they were published by a house you like, perhaps you’ll reconsider.

You could include popularity if you really wanted to. For example: starting at the bottom and working up, for any two submissions within some range of points (say, 25%) you could make it a popularity “how many people spent at least 1/2 of their ten points on this book?” elimination. (This allows for some gaming of the popularity aspect, but limits it to 20 submissions across all categories.)

#269 ::: Joshua Kronengold ::: (view all by) ::: May 13, 2015, 06:04 PM:

268: You don't seem to understand the voting systems under discussion, and are arguing your conclusion, rather than anything under discussion in the thread. Come again?

#270 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 06:07 PM:

gin@268:
this particular method is designed to basically disempower them

That's actually not true. If you read the topic and the descriptions, this system is specifically -not- designed to keep them off of the final ballot. What it is designed to do is keep any one group (whoever that might be) from keeping everyone -else- off the ballot.

Kilo

#271 ::: felice ::: (view all by) ::: May 13, 2015, 06:08 PM:

[DECLINED NOMINATIONS]

Keith "Kilo" Watt @228: "I've been reading all the posts on the subject, and I think we can all agree that there are two very distinct opinions on "fairness" with regard to this subject. I don't think we can say, therefore, that there is an objective definition of "fairness" in this case."

That's not my impression. As I understand it, we all agree that it's unfair to tell people they aren't finalists after all due to someone else declining; and I don't think it can reasonably be argued that rerunning without the withdrawn finalist(s) isn't technically the right thing to do. My impression is the RME proposal is being objected to by people who want simplicity or are strongly attached to the status quo, not on grounds of fairness (and most of the objections to changing the process predate the RME proposal, or ignore it - eg #264).

"- Therefore, let's keep it this way, and let's allow tradition to grow and decide what's "fair". As a first cut, the Hugo admins could decide to stick with the current tradition and move the work eliminated in the next to last round to the ballot. - If, after a year or so, fandom decides that they don't like this tradition, the Hugo admins can try something else (such as re-running the election) and see what fandom thinks about that."

I think it's safe to assume they'll stick with the current tradition until it produces a blatantly unfair result and upsets a lot of people, and there's the potential for people to react badly (eg "this whole SDV-LPE thing is silly, we should revert to the old system"). And as long as there hasn't yet been a blatantly unfair result, more subtle unfairness will contine. Why not get it right from the start?

Yarrow @233: "I also think that, on one view, moving the next work up is not only less work than rerunning the process, but also more fair. On that view, what a nominator gets when a work they like gets on the Hugo ballot is the opportunity for that work to be sent to the Hugo electorate. (Presumably this part is not controversial!)"

I would say the primary purpose for nominating works is to get the opportunity for those works to be included in the voting process. Being "sent to the electorate" is pretty meaningless if it's withdrawn shortly afterwards. It doesn't go in the voter packet, nobody is going to read it in order to decide how they should rank it, it won't appear in most historical listings. And if a work is withdrawn and not replaced in accordance with the preferences of the people who nominated it, they might be left with nothing on the final ballot they consider worth voting for.


--------------------------

RME = Rerun-Merge-Extend: rerun the SDV-LPE process with any withdrawn finalists removed; merge the new set of finalists with the old set of finalists; if necessary, extend the ballot to more than five finalists (just as we do with ties). Nobody gets removed from the final ballot because of reruns. In most cases, RME won't actually need to extend the ballot, because the new finalists will include all of the remaining old finalists (and one new work to replace the withdrawn work). Occasionally, it will extend the ballot to six finalists. Very rarely, maybe more than six (particularly if we get withdrawals and ties involving last place).

#272 ::: Ken Burnside ::: (view all by) ::: May 13, 2015, 06:12 PM:

Speaking as someone who is the beneficiary of both the Sad and Rabid Puppies slate, I think the SDV-LPE solution is a good one.

This merely dilutes the votes - a bloc of voters can still have the effect of the slates given here, but it will, in a typical voting year, raise that to about 40% of the voters rather than say 23-28% as appeared to happen this year. (Numbers based on looking at historical nominating ballots).

Since Sad Puppies 2, the number of nominating ballots has increased by more than 100%. People who expect this to "eliminate" Puppies will be gravely disappointed (as, to be fair, has been pointed out three times in the proposal and in the FAQ).

Unrelated:

I am in the process of scoping out a project that should also help the process along: An automatically generated and hand-pruned publisher-neutral list of "What's Hugo Eligible In A Given Year." If there are any people who are good at building datascrapers and running databases, please contact me via my website.

I want to avoid situations where things like Three Body Problem get squeezed out (until someone drops out) because people didn't know they were eligible.

#273 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 06:17 PM:

felice@271:
I don't think it can reasonably be argued that rerunning without the withdrawn finalist(s) isn't technically the right thing to do.

Actually, it can, and a number of people have done just that. They haven't changed your mind, and you haven't changed theirs. This is the situation I was referring to earlier.

Both ideas have good and bad things to say for them. I've been watching the discussion, and I don't think we're likely to reach a definitive conclusion on the issue. The point of my post is that there is nothing in the constitution that deals with the issue now, and an argument can be made that, since there is genuine disagreement on what -should- be done, there's no compelling reason for mandating what -must- be done.

The system works either way. In fact, it seems to me the best thing to do is what Kevin suggests: write up your declined nominations idea as a separate proposal. Then they can all be voted on piecemeal. Since SDV-LPE doesn't require it, there's no reason it can't be separate. And since the default method doesn't require a constitutional change there's no real reason to discuss it in the SDV-LPE proposal.

What are the group's thoughts?

Kilo

#274 ::: abi ::: (view all by) ::: May 13, 2015, 06:21 PM:

Ken Burnside @272:
People who expect a rule change to eliminate any voters or nominators, Puppy or not, should look elsewhere. Without, if it needs to be said, the blessing of this community. Because bah.

With regard to the eligibility-listing process, are you in touch with Analee Flower Horne (@leeflower on Twitter)? I think she's started a similar thing.

#275 ::: Annie Y ::: (view all by) ::: May 13, 2015, 06:32 PM:

#273 ::: Keith "Kilo" Watt

My opinion is up in 194 (after the note for Kansas that Kevin Standlee clarified (Thanks Kevin!))

As long as the method of replacements is known before the nominations close (so there are no cries that the method was selected in order to kick a specific work (or add it)*), I am fine with it. Or we can just leave it alone and let the administrators decide how they want to handle this (and it ties again into the 5% mess and how that gets resolved anyway)

This year it is clear that it is either next on the list or no replacement at all. If this change gets approved, we have three options - two of them producing possibly different nominees.

* Before this year, I would not have thought that someone can complain in this situation. This year? My internal cynic is out in full power.

#276 ::: felice ::: (view all by) ::: May 13, 2015, 06:35 PM:

Keith "Kilo" Watt @273: "Actually, it can, and a number of people have done just that."

Can you point me at specific post numbers? My impression is it's a conflict of fair vs easy, not different interpretations of fair - what have I missed?

#277 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 06:45 PM:

Felice@276:

I think yarrow@233 has one of the clearest explanations.

Kilo

#278 ::: P J Evans ::: (view all by) ::: May 13, 2015, 06:45 PM:

273
I think they're very much separate issues, and should be handled separately. (I think I've made my views clear already.)

#279 ::: influxus ::: (view all by) ::: May 13, 2015, 06:48 PM:

#273 ::: Keith "Kilo" Watt

I'm not sure I'm part of the group, more of a recent blow to this elegant thing y'all put together, but I think this sounds like a perfectly reasonable compromise.

#275 ::: Annie Y:

"As long as the method of replacements is known before the nominations close (so there are no cries that the method was selected in order to kick a specific work (or add it)*)"

Unfortunately, I don't think before nominations close would prevent this. As the administrators would have access to the nominations as they roll in, to defeat all perceptions of bias the process would need to be declared before the nominations open. That being said, if we are at that level of distrust, I don't see why people wouldn't just make accusations of electoral fraud.

#280 ::: Darth Paradox ::: (view all by) ::: May 13, 2015, 07:09 PM:

[DECLINED NOMINATIONS]

I've been following this conversation on and off with interest, but haven't had a chance to catch up to present until now, and I feel like I need to defend the slate voters on this one. Specifically, I think that rerunning the system from the beginning is necessary to avoid disenfranchising people that voted for the declining nominee.

I think a slate-voting bloc consisting of one fifth of the total nominating voters should be reflected in one fifth of the final nominations, which is to say one nomination out of five; if I understand the proposed system correctly, that's basically what happens today. (Which of the slate's choices makes it on to the ballot will probably be determined by other voters whose ballots don't match the slate exactly, or minor deviations from the slate within the bloc.)

And I think should be true even if the nominee that turns out to be their top choice withdraws. Suppose roughly one fifth of voters nominate a slate of Victor, William, Xenia, Yorick, and Zola, and thanks to those minor deviations William ends up on the nomination ballot (and Xenia was the last one of the five to be eliminated, but another non-slate nomineed was eliminated later). If William then declines the nomination, Xenia should replace him on the ballot; as I understand it, that will happen when the system is rerun to replace the declined nomination but not when the sixth-place winner is promoted.

I don't have the math to back this up at the moment, but my intuition tells me that promotion of sixth place runs a high risk of disenfranchising the slate bloc entirely.

(That said, I do think it should be rerun-and-merge to avoid "unnominating" anyone; I think an "unnomination" is a sufficiently bad outcome (on an emotional and political level if nothing else) to warrant specifically being avoided.)

#281 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 07:43 PM:

I think that the "rerun and merge" contingent should make a separate proposal. It wouldn't have to involve lots of lawyerly language; just "It is the sense of the membership that if a nominee declines, and the ballots have not yet been finalized, then the award administrators should find all works which would have been nominees if the decliner were removed from all ballots, and offer them places as finalists, without withdrawing finalist status from any works which already have it."

#282 ::: felice ::: (view all by) ::: May 13, 2015, 07:56 PM:

Keith "Kilo" Watt @277: "I think yarrow@233 has one of the clearest explanations."

That seems to boil down to "nominations for declined or ineligible works aren't wasted", which strikes me as a very dubious proposition. I don't recall anyone else bringing it up beforehand (though a couple have endorsed it subsequently), and even Yarrow describes it as "one view" rather than their own view (Devil's advocate?).

I would certainly consider my nomination to have been wasted if the work in question was withdrawn after becoming a finalist, and extremely peeved if any of my other nominations missed out on being finalists because of the wasted nomination. Surely I'm far from alone in that? Having it mentioned in a footnote in detailed nomination statistics isn't much of a consolation.

#283 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 08:01 PM:

Felice, I understand you are sincere. Please accept that others, such as Yarrow, are too. You seem to be in the minority, but you seem to think that anybody who disagrees with you can't really believe what they say they do.

#284 ::: Soon Lee ::: (view all by) ::: May 13, 2015, 08:07 PM:

I think I agree with Darth Paradox #280: "That said, I do think it should be rerun-and-merge to avoid "unnominating" anyone".

It seems to me that it's more fair to the nominees, than "unnominating" someone.

#285 ::: P J Evans ::: (view all by) ::: May 13, 2015, 08:10 PM:

280 et seq
If the work was withdrawn, or the nominee declined, don't take it as aimed at you personally.

It's ridiculous to rerun the the counting just for that: it's not actually unusual (this year, only in the number of withdrawn/declined nominations). Besides which, any reasonable program will have counted all the names, and have an order which should go far past any point where this would be a problem. That's why there are names ready to fill in for declined/withdrawn nominees.

(I refuse to call them votes; that's where you're getting confused, I suspect: this isn't an election, it's a simple tally. Or should be.)

#286 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 08:14 PM:

(On my phone, so apologies for edits, our the lack thereof.)

felice@282:

Well, that's certainly a reasonable question of debate, as we've seen. But one way or the other, there's still no reason it has to be presented as part of the SDV-LPE proposal. It's an easy thing to draft as its own proposal, present it immediately after SDV-LPE, and if people feel it needs to be in there, it will. I really don't see why that's not the best route in this case. I'd even be willing to help you draft it, once we have the current proposal in the can (I don't think you'll have any trouble finding the required two sponsors). Jameson's 281 is real close to all you need, though.

Kilo

#287 ::: Brian Z ::: (view all by) ::: May 13, 2015, 08:19 PM:

@214 J Thomas

@178 Brian Z
That depends on their gut reactions, which are hard to predict... If one person does that, it won't have much effect at all. If a bunch of people do it, it has a bad effect on the system as a whole, and probably doesn't help them since they'll tend to act against each other... It's hard to quantify, and probably people will go by their gut feel, and I can't predict that but I hope they won't go that route.

If you are worried about more concerted efforts to hack the award in future, better be worried about something that "if a bunch of people do it, it has a bad effect on the system as a whole."

If it is hard to quantify, then people will go only by their gut feeling - which could have surprising consequences if they are being given conflicting messages that they should vote exactly as before and that the effect of their vote has probably changed.

#288 ::: Brian Z ::: (view all by) ::: May 13, 2015, 08:55 PM:

@ 219 Jameson Quinn

[RESPONSES TO CRITICISM]

"So what are the legitimate points he has raised?"

"I'd be interested to hear if other people think this is an adequate response to BZ's concerns.?

Classy.

Let's start by reviewing my "illegimate concerns."

1. Many "Puppies" want the same thing and you (this group) could engage better with others to build consensus.

Look I understand why a body would feel defensive when an outsider comes in and asks pointed questions. But take me. I'm sympathetic to your cause, and I've been called a troll and a liar repeatedly, had my words on another forum stripped from the topic sentence of the paragraph to paint my words in the worst possible light and cited out of context without a link, and it has been implied that I shouldn't engage further and that my questions are only useful in case any actual concerned person wants to know the answers in future. And you know what? Don't get mad: I'm pleasantly surprised by this reception. I had expected to be disemvoweled. Still, don't you think this sounds to outside ears like "we have enough attending at WorldCon that we don't particularly care what you think"? Not be the best way to repair a torn community.

2. Maybe it ain't broke. Possibly what happened this year won't happen again with the higher participation we are seeing.

3. Micro-managing the rules is going to be less effective than welcoming new blood and demonstrating through your words and actions that existing WorldCon culture is valuable and should be preserved.

4. The goal should be rewarding and honoring excellence, not fairness.

Now to your responses:

5. Bullet voting. I'm not convinced @123 has addressed the concern fully. Currently, if I think LeGuin and Spinrad are probably going to win already there is not much penalty to my other nominees to adding my voice to the chorus, but in the proposed system the reverse could be true. Might work out better for LeGuin than Spinrad. Note my focus is on honoring worthy nominees as a community not getting the most satisfactory outcome as an individual.

6. Goals and voter happiness. You may not have explicitly said in your mission statement that you are designing this to maximize happiness (which we agree is not the productive way to go) but you effectively have done so by emphasizing "fairness." Fairness doesn't choose excellent science fiction. The ONLY reason fairness might be important is that people feel happy when they get a piece of the pie (or in today's culture, apparently, when they perceive that everybody who wants one gets a little piece). I question whether that is a valid goal. It certainly doesn't achieve MY Hugo voter happiness, which comes when an author receives a heartfelt gift from their fans that they know very well is valuable because it consistently picks greatness.

6. There may be cases where mediocrity triumphs. Of course this is true of the current system as well (many fans were unhappy with the 2014 ballot, whether you think it was perfect or not). But 6a) the current system has a consistent, time honored message: if everybody thinks about what is the best and nominates it, a consensus shortlist will emerge. Your new message is "everybody's opinion is equally valid and we will try to be fair to everybody." I think that's the wrong message and in your own discussion of my 1970 thought experiment you have recognized that the danger of mediocrity is a real concern. 6b) whether this system is as effective at choosing excellence is a real concern and should considered to the best of our limited ability, considering the math but also changing voter strategies. My own feeling is: we know the old one did, that's a real treasure, so let's try to repair it without making drastic changes too quickly.

#289 ::: Seth ::: (view all by) ::: May 13, 2015, 08:56 PM:

#268 This doesn't disempower anybody, it gives any group strength proportional to its size.

20% of the nominators can get one work (that nobody else likes) onto the ballot. Or, depending on how the numbers work out, they might get the works that would (according to the other 80%) have been #6 and #8 onto the ballot. Either way, that seems about like the right amount of power for 20% of the nominators to have.

#290 ::: Walter Daniels ::: (view all by) ::: May 13, 2015, 08:58 PM:

As an ex-computer specialist (multiple injuries and permanent disabilities/pain caused giving that up), you have done a first class job. There is only one problem. The "Traditional (Big) 5 Publishers" all live/work in NYC. That means they all attend pretty much the same parties, know each other, etc. As a result, in an area with 8.5 Million people, you have 1,000 (at maximum), who can "informally agree" that "Author X, of Publisher Y, should win." At worst it's 5 agreed on. Giving the "group/slate") 200 to 500 assured votes.

Now, this is fine, and acceptable, as long as it's understood that employees publishers "block vote." The problem is that in the "universe" of 312 million other potential voters, a "slate" is nearly impossible. Fen are like herding cats, for "bloc anything." :-)

Those screaming about "bloc voting" fail to understand that the "slate" is, and always has been, "here are authors/artists/works to be considered by you (generic) for consideration for Hugo Nominations." If someone asks me. "What are some good Mil/SF authors to read?" I'll mention David Weber, Mike Z. Williamson (Note: I've known MadMike for almost 23 years, but I also like his writing), an ex(?)-Marine Officer female author whose name escapes me, and John Ringo. If they ask for possible Hugo Nominations (assuming all have books out in the relevant period), I would still give several names. Unless I know their individual tastes. If I do, I'll say. "I plan to vote for X, but these others are almost as good, IMO." That was, and is, the intent behind Sad Puppies, as I understood it. Had I become a "supporting member" early enough, I would have voted for the one I thought most deserving. I can't speak for what anyone else thought/did.

As I said earlier that puts me, and those like me, at a disadvantage. By population, I make up 0.00266 of an NYC "Cabal member's vote." That's a heavy disadvantage. That means 36.67 of me, to offset one of them. (Note: I used population to be as generic as possible. I know of no "count" for actual F&SF fans in the general pop.) So, if there is any "Bloc voting," going on, it's the publisher employees doing it. I doubt that they actually decide, "we'll all vote for X." It's more likely they all "like and talk up" X, better than any others. (Peer pressure at it's finest.) Nevertheless, it's the best way possible that you've come up with. Publishing (traditional)employees will always have an advantage. It's called, "Small area, where most live/work/play together." That some don't think that's enough, is a sign of their insecurity.

#291 ::: junego ::: (view all by) ::: May 13, 2015, 09:06 PM:

Keith "Kilo" Watt @ 273

The system works either way. In fact, it seems to me the best thing to do is what Kevin suggests: write up your declined nominations idea as a separate proposal. Then they can all be voted on piecemeal. Since SDV-LPE doesn't require it, there's no reason it can't be separate. And since the default method doesn't require a constitutional change there's no real reason to discuss it in the SDV-LPE proposal.
What are the group's thoughts?

I wouldn't object to leaving it up to the administrators. The less the proposal tries to modify current rules and practices, the better chance it has of passing.

I have a couple of concerns:

1) We might should warn administrators that this could be an issue?

2) When there is finally a need to make such a decision, those future administrators might benefit from this discussion of possible options. Would it be appropriate to include a list of what was discussed here for their consideration? I think there are 3 proposals a) No recalculations and just take the next in line from the final run. b) Freeze the finalists list and rerun the program for #6 down on the final run to replace only those removed/withdrawn. c) Keep the remaining finalists, rerun the entire list and add any qualifying nominees to the finalists if there are changes to the top 5 beyond the replacements.

#292 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 09:09 PM:

@287: So, you're concerned that somebody might give voters conflicting messages? I wonder why you'd think that...

More seriously: "the effect of their vote has probably" not "changed". The Condorcet Jury Theorem still says that if enough voters define quality in similar enough ways, this system will get the right answer, which in that case is the same answer as the current system. The only difference is that any slate voters' ability to manipulate the system, and hurt other voters and the outcome in the process, will be reduced. Not entirely removed, but reduced. They'll become like concern trolls — actually kinda helpful — rather than like hate trolls — toxic.

#293 ::: P J Evans ::: (view all by) ::: May 13, 2015, 09:17 PM:

290
Except that it doesn't actually work that way in practice. That's the Big Lie involved with the juvenile canines and their slate.

How do I know this? I've been on a Hugo administrative subcommittee - that's how (and why) we have access to a set of actual data. And I can assure you that no one outside the subcommittee gets input to the counting, and they sure as hell don't get any input to the final ballot counting. (Nor do they get advance knowledge of who gets rockets. It's need-to-know, all the way.)

#294 ::: Yarrow ::: (view all by) ::: May 13, 2015, 09:20 PM:

felice@ 282: That seems to boil down to "nominations for declined or ineligible works aren't wasted", which strikes me as a very dubious proposition.

The case of declined nominations is different from that of ineligible works. Ineligible works are eliminated from (or moved to the right category on) all nominating ballots before counting begins, so there's no need to rerun the process.

Nominations that boost a work to the preliminary short list are not, I believe, wasted, even if the author declines the nomination - they get the work to the "make a phone call to the author" stage.

even Yarrow describes it as "one view" rather than their own view (Devil's advocate?).

It is my view: I believe it's the fairest choice among those available (some of which are also reasonable).

#295 ::: felice ::: (view all by) ::: May 13, 2015, 09:21 PM:

Jameson Quinn @24: "I am not the only person who has coded this up, but my code (and data) is visible at https://github.com/The-Center-for-Election-Science/HugoVotesim. I did it in R, the statistical computing environment. I realize that R is not at all the most accessible language for non-statisticians, and I'm sorry about that, but it would have taken more work to do bootstrapping in python."

I've got it somewhat running, but it throws the error:

Error in if (hugos[i, cat] < 7) { : missing value where TRUE/FALSE needed
In addition: Warning message:
In min(titdist) : no non-missing arguments to min; returning Inf

Any idea what the problem is?

#296 ::: felice ::: (view all by) ::: May 13, 2015, 09:39 PM:

Yarrow @294: "The case of declined nominations is different from that of ineligible works. Ineligible works are eliminated from (or moved to the right category on) all nominating ballots before counting begins"

Not if they're found to be ineligible after the finalists are announced, as happened in Novelette this year.

#297 ::: Chris ::: (view all by) ::: May 13, 2015, 09:49 PM:

Walter Daniels @290:

Those screaming about "bloc voting" fail to understand that the "slate" is, and always has been, "here are authors/artists/works to be considered by you (generic) for consideration for Hugo Nominations."

I quote:
"I encourage those who value my opinion on matters related to science fiction and fantasy to nominate them precisely as they are." (Vox Day, introducing the Rabid Puppies slate) (emphasis added)

That's not "authors/artists/works to be considered by you". That's "vote for exactly this list", a tactic that lets a coordinated 10% drown out the emergent consensus of the other 90%.

#298 ::: P J Evans ::: (view all by) ::: May 13, 2015, 09:53 PM:

296
It happens. Usually it's discovered before the ballot is sent to the printers.
Move up the next name on the list, and cope.

#299 ::: Yarrow ::: (view all by) ::: May 13, 2015, 09:59 PM:

felice@296: Not if they're found to be ineligible after the finalists are announced, as happened in Novelette this year.

Point. Now that I think of it, it would be massively more work to investigate the eligibility of every work that appears on any ballot, rather than what comes up on the preliminary short list.

#300 ::: Seth Gordon ::: (view all by) ::: May 13, 2015, 10:12 PM:

[DECLINED NOMINATIONS] Just adding my “vote” in favor of the rerun-merge-extend option. When faced with tricky boundary cases, erring on the side of inclusion sounds like the right way to travel.

#301 ::: Vicki ::: (view all by) ::: May 13, 2015, 10:18 PM:

Walter Daniels @290:

Would you care to rephrase that in the subjunctive, as appropriate for a hypothetical statement contrary to fact?

A few people here might even agree with you that collusion would be "fine and acceptable" as long as it was known about. (I say "a few" because if most of us did, this conversation wouldn't be happening, because the Puppies' activities are no secret.) But I hope that nobody here would consider it "fine and acceptable" for people to falsely believe that a few hundred people in the New York area are bloc-nominating in the Hugos.

Do you have any evidence for your charges? "Well, that's what I would do in their shoes" isn't evidence of anything except your psychology, which is not at issue here.

#302 ::: Brian Gibbons ::: (view all by) ::: May 13, 2015, 10:18 PM:

Keith "Kilo" Watt @273: Since SDV-LPE doesn't require it, there's no reason it can't be separate. And since the default method doesn't require a constitutional change there's no real reason to discuss it in the SDV-LPE proposal.

I'm not sure that I see why you think that the default method isn't in the constitution right now, or that the SDV-LPE proposal doesn't already remove it as an option.

3.8.1 tells us what works appear on the final ballot: the five eligible nominees receiving the most nominations.

When a nominee declines/is declared ineligible, my understanding is that the Constitution is agnostic as to what the Hugo administrator is required to do, but we know one thing: the only thing the final ballot can include is "the five eligible nominees receiving the most nominations" (ties aside).

The Hugo administrator can't replace a declined nominee with the nominee's friend or an ineligible work with another one by the same author. He can leave the ballot alone if he wants, but if he chooses to produce a new version of the final ballot, he must do so strictly in accordance with the rules for what can appear on a final ballot.

This is a pretty simple process (because the new member of the set of "five eligible nominees receiving the most nominations" will always be the one that was in sixth-place before), but this isn't just a default custom, it's simply producing a final ballot in accordance with 3.8.1.

3.8.8 in this proposal gives us an entirely new version of how to determine what works appear on the final ballot.

If a nomination is declined or a work declared ineligible, the Hugo administrator can similarly leave the ballot alone, aside from striking out those nominations, but if he does want to replace the ballot with a new version with additional nominee(s), he must determine what works appear on this new version of the final ballot by using 3.8.8.

Neither adding the work that was in the #6 spot in the last run, nor expanding the list to include extra nominees (as in a RME scenario) is an option, because they're simply not options the rules give. The administrator can't just decide to put those works on the ballot, any more than he can just decide to put his favorite there.

The administrator's only option (assuming he wants to replace the old version of the final ballot with a new one, and not just leave it alone) is to follow 3.8.8, wherever that might lead, which might include retroactively removing nominees.

... or, at least, that's how I understand the rules.

As I see it, SDV-LPE causes this problem and SDV-LPE should fix it. A separate proposal is kind of silly, since its necessity would stand or fall with this proposal passing.

#303 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 10:36 PM:

All -

First, hello and welcome to the new voices who have come by, particularly those who identify themselves with the Puppies! As Brian and I discused, I really don't like to think of "sides" in fandom. Ultimately, the Hugo process needs to work for everyone (to the greatest extent possible, of course) if it is truly to be the "fans' award". I'm glad that we can all rise above the politics of the situation and focus on getting the job done.

The proposal is nearing its final form, so I'd welcome any questions about how the system works, and if there are holes that we aren't seeing, it's great to have fresh eyes looking at it. We do have to keep in mind that there really isn't a perfect system -- that realization was one of the results of this discussion -- but in general, I do think this proposal is going to be a good way to go.

To that end, it's worth reiterating that the prime goal is not to keep anyone -- whoever they may be -- off of the final Hugo ballot. The goal is to prevent anyone - whoever they may be -- from preventing -others- from having a chance at appearing on the ballot. This is a win for everyone. For example, Walter@290 mentions a bloc of NYC publishers. Such a thing may or may not exist, but it realy doesn't matter: the idea here is for -no- bloc to be able to prevent works from being considered. So, I do think this proposal serves everyone's aims.

I think you'll find everyone here to be reasonable people and open to productive discussions. My only request is that you please do consider how far along we are in the process and understand that we really can't start from scratch at this point. My hope is that we can answer questions and address concerns as we put together the final proposal.

Thanks much,
Kilo

#304 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 10:47 PM:

Brian@201:

The previous thread to discuss proposed rules changes carried an advisory at the top not to argue about whether changing the rules is a good idea or not, but this one doesn't (yet).

Just to be clear, this is a continuation of the previous thread, so yes, the idea is to work on this proposal and isn't really to debate whether we should do it at all. That's certainly a worthwhile debate to have, but it's probably better put in its own topic. I don't feel we will have wasted our efforts trying to put this together if it turns out fandom decides they don't want to change anything. Unless we have something to show them, they certainly have no reason to do otherwise.

Kilo

#305 ::: JJ ::: (view all by) ::: May 13, 2015, 10:51 PM:

Keith "Kilo" Watt, #304:

Brian is quite fixated on trying to dissuade Worldcon members from making any changes to the current nomination process; he's made numerous posts in various threads not just here, but over at File770, trying to discourage everyone from making any changes.

Draw from that what conclusions you will.

#306 ::: P J Evans ::: (view all by) ::: May 13, 2015, 11:02 PM:

299
Well, some things are obviously ineligible. (The 1984 examples: Dune for Best Novel and Asimov for the Campbell Award. [/snicker])

The administrators check the top 10, I suspect.

#307 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 11:04 PM:

BrianG@302:

First of all, apologies for any confusion -- I didn't realize we had two Brians!

I see what you're saying, but the constitution deosn't actually mandate five nominees on the final ballot. It does mandate that there will be at least three (3.8.5). As you point out, he can leave the ballot alone -- not add anyone if someone declines, but the constitution does not actually give him the authority to do -anything- if someone declines. It doesn't even say that he can take the 6th place person. But he's not limited to the powers explicitly granted to him. Hopefully Kevin or someone will correct me, but my understanding is that anything not explicitly set out in the constitution is left to the administrator's judgment. I may be wrong about that, of course, but that is my understanding.

You say he can't replace someone with his friend. Well, most of us would agree that'd be an ... extremely ... poor decision for an administrator to make. But the constitution itself doesn't prevent him from doing so. I think that would universally be declared to be an unfair result, and who knows, an amendment might get passed to prevent it from ever happening again. It's never been a problem before (and isn't likely to be in the future, of course).

But as we've seen here, it's not quite so cut and dried in this case. An adminstrator could make an argument for either one. You say that SDV-LPE should fix it, and you may have a point, but I'm not sure that there is a clear and unambiguous fix. As I said, personally, I'd like to give the Hugo admins the freedom to figure out what the best course of action is. Changing the constitution may take up to three years in the very near future. For myself, I think that's too long a wait.

But if you really feel SDV-LPE needs to take a stand one way or the other, I still think the best way is with a separate proposal. In fact there's no reason not to propose -both- of the leading methods as separate proposals and let the business meeting decide which is best. Kevin has indicated he'd actually prefer that sort of thing in general. I'd be glad to help draft both of those proposals. If either one really is self-evidently more fair, then that will be made immediately apparent at the business meeting -- why bother theorizing now?

Just my thoughts,
Kilo

#308 ::: P J Evans ::: (view all by) ::: May 13, 2015, 11:05 PM:

305
I came to a conclusion earlier about his viewpoint, based on his comments at File 770.
I do wish those supporting the juvenile canines would say so, instead of leaving the rest of us to come to conclusions that may be incorrect, but are definitely unflattering to them.

#309 ::: viktor ::: (view all by) ::: May 13, 2015, 11:09 PM:

[DECLINED NOMINATION]
If a re-tallying is done after a declined nomination, are the ballots that included that nomination re-weighted? By that I mean, if I nominate Able and Baker -- thus assigning them each 0.5 pts -- and then Able declines, does Baker then get the full 1.0?

If so, then I would be against a re-tallying after a declined nomination. I weighted my ballot they way I wanted it. In this case, I think it's best to take the #6 finisher.

#310 ::: Cat ::: (view all by) ::: May 13, 2015, 11:11 PM:

Brian Z @ 288

O Hai! I remember you from File 770. You're the one who said we all jumped on someone for proposing something that Puppies quoted approvingly.

I'm so glad we got that straightened out, aren't you? It's great that several other people jumped in and pointed out that Making Light was trying to be fair to everyone, instead of trying to force Making Light commenters to disenfranchise the Puppies.

1. Many "Puppies" want the same thing and you (this group) could engage better with others to build consensus.

[snip]. But take me. I'm sympathetic to your cause,

I'm sorry--I haven't quite noticed this. How are the Puppies prepared to build consensus with us?

And what parts of what we are trying to do, exactly, are you sympathetic to?

Still, don't you think this sounds to outside ears like "we have enough attending at WorldCon that we don't particularly care what you think"? Not be the best way to repair a torn community.

The Puppies tore the community, right? Just pointing that out, to make sure nobody's getting the impression that we are responsible for what they did. I know the Puppies claim that, but it doesn't seem like a mistake you would make, being sympathetic to our cause. Just want to make sure nobody else makes it either.

Given that they tore the community, please explain to us how the Puppies are proposing to build consensus with us and help us repair it?

2. Maybe it ain't broke.

What an interesting idea. 17% of the nominators using a slate locked 83% of the nominators completely out of about 3/4 of the ballot including whole categories. That looks broken to me. Will the business meeting at Worldcon agree with you? Or with me? We'll see.

I wrote a couple of pieces about how a slate increases the nominating power of the group that uses it. The most relevant part is here. I think you are being far too optimistic.

I think we'd need ten to twelve times as many honest nominators as slate nominators to be able to get one or two of the non-slate group's favorites on the ballot. There were about 350 slate nominators, so we'd need a total of at least 3,850 nominators next year, and that's provided there isn't anybody left on Breitbart or in Gamergate to be drawn in. I do expect an increase in nominators next year. I don't expect it to be *that* large.

3. Micro-managing the rules is going to be less effective than welcoming new blood and demonstrating through your words and actions that existing WorldCon culture is valuable and should be preserved.

I for one welcome our new Puppy overlords--oh, wait, actually I don't. If they wanted to join as honest nominators, reading what appealed to them and individually choosing their particular favorites without resorting to a slate to guide their reading or nominating, of course I would welcome them, but then they wouldn't be Puppies anymore, would they.

4. The goal should be rewarding and honoring excellence, not fairness.

Currently the system is rewarding and honoring works the three or four curators of the slate (the Puppies have been remarkably opaque about who they actually were) choose to cram down our throats to make a political point. To the point where such works have forced more excellent works off the ballot in the view of the slatemakers themselves.

Rewarding and honoring excellence won't be possible until non-slate nominators can also get some work on the ballot, by individually reading what appeals to us and nominating our own particular favorites.

You may, of course, think that "excellence" consists of successfully catching the eye of one of these three or four slate curators. You are welcome to attempt to persuade the business meeting attendees of that.

You think that we mustn't worry about slate voters shutting us out of the nomination process because "excellence is better than fairness." This would be a consideration if there were any reason to think that the slate curators are successfully choosing excellence. They themselves admit they fell short this year, but perhaps you disagree with them; that is certainly your right.

But until you can make a good case that slates are better at choosing excellence than a fair process would be (a case you might do better to make to the business meeting than to us) I reject the dichotomy. We're not choosing between fairness and excellence but between fairness and a load of Puppy...chow...which so far has been almost uniformly abysmal leavened with the occasional lump of "meh."

Of course I still have four of the John C. Wright works to read. Perhaps you think I'll find excellence there? (shudder)

6. There may be cases where mediocrity triumphs.

This year, for example. Though there is always "No Award" thank goodness.

But 6a) the current system has a consistent, time honored message: if everybody thinks about what is the best and nominates it, a consensus shortlist will emerge.

That's still the message. We're just working to make it true again.

6b) whether this system is as effective at choosing excellence is a real concern [snip] My own feeling is: we know the old one did, that's a real treasure, so let's try to repair it without making drastic changes too quickly.

The old one did because nobody wanted to be the jerk who introduced slate nominating. It was a real treasure--until a big enough jerk came along. But slates certainly aren't going away now, and while a nominating pool of 10,000 Hugo voters that would overwhelm any resentment-driven political attempt to slate is a lovely dream, I don't see it happening anytime soon.

Besides, it takes two years to make changes. We can pass this this year (if the members at the business meeting approve). If the Puppies really are going to try to repair the community they tore and don't slate anymore and even stop being jerks blaming everybody but themselves for their bad behavior (I can dream), next year's business meeting can go, "oh, whew--it's so much easier to just count the usual way" and decline to ratify.

#311 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 11:12 PM:

junego@291:

We might should warn administrators that this could be an issue?

This might actually be a good argument for separate dual proposals for each method of handling declined nominations. Even if -both- fail, the next admins will be aware of the potential issue.

Still, like Jameson said, I don't really think we need to make more of this than what it is. The odds of this ever being an issue are so very, very small that we shouldn't get too much up in arms about it. The chance is not zero, however, so I think it's a good discussion, but we should make sure we keep things in perspective.

Kilo

#312 ::: JJ ::: (view all by) ::: May 13, 2015, 11:14 PM:

P J Evans, #308: "I came to a conclusion earlier about his viewpoint, based on his comments at File 770."

Likewise. At this point, I'm rather amused by the repeated protestations of "I can't believe what a nasty person you are, to accuse me of being a Puppy just because I keep trying to ram a bunch of Puppy Talking Points down everyone's throats while continually proclaiming that I'm really one of you!"

#313 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 11:17 PM:

viktor@309:

By that I mean, if I nominate Able and Baker -- thus assigning them each 0.5 pts -- and then Able declines, does Baker then get the full 1.0?

If I understand felice's proposal correctly, yes, that's exactly what the case would be. Please correct me if I've got a misconception, felice!

K

#314 ::: Cat ::: (view all by) ::: May 13, 2015, 11:46 PM:

FWIW, In the case of declined or ineligible nominations I favor, in order

1) Generate a list of ten, take the top five, if one declines, take the sixth and so on. It has the virtue of simplicity and being unlikely to put too many extra nominees on the final ballot. (You might get the occasional tie for sixth.)

2) Generate a list of five, if one declines, just go with four. This is also simple, and does not risk ejecting anyone already on the ballot (which I think would lead to extremely bad blood.)

3) Generate a list of five, if one declines, rerun, merge, extend. This is arguably "most perfectly fair" but clunky and threatens to extend the ballot. Longer ballots (especially in the Best Novel and Best Related Work categories) make it more difficult to read everything.

In the case of nominations "wasted" on ineligible work, you wouldn't expect that to happen very often. One hopes that most of the people planning to nominate the work would check what year it was published. Being nominated by only a small percentage of those who loved it would see such a work dropped early and any ballots containing it re-weighted in its absence and thus not penalized by it. Large numbers of people would only nominate an ineligible work in the case where accurate information about its year of publication was withheld; this would be the only time such a work would reach the ballot. In that case the person to blame for "wasted nominations" wouldn't be the Hugo administrators, surely. So I don't see that the Hugo administrators would have to re-run the nomination process to make good on it.

#315 ::: felice ::: (view all by) ::: May 14, 2015, 12:00 AM:

[DECLINED NOMINATION]

viktor @309: "If a re-tallying is done after a declined nomination, are the ballots that included that nomination re-weighted? By that I mean, if I nominate Able and Baker -- thus assigning them each 0.5 pts -- and then Able declines, does Baker then get the full 1.0? If so, then I would be against a re-tallying after a declined nomination. I weighted my ballot they way I wanted it."

That's not how SDV-LPE works. If you nominate Able and Baker, they start off with 0.5 points each from you, but if either one gets eliminated during the initial tallying, the other gets the full 1.0. There's no way to indicate you want them both to stay at 0.5 points (and no reason to want it); all you can do is nominate the works you consider worthy, and the weighting and reweighting is an automatic process. If Able was found to be ineligible before the initial tallying, or got eliminated before Baker, then Baker would get the full 1.0 during the initial run. If Able declines or is found ineligible after the initial announcement of finalists, why should it be any different? There's no way to rerun the tallying without changing the weightings (other than not removing the withdrawn work, which would make the exercise pointless).

Though I don't see any problem with your suggestion in #253 to allow nominating the same work more than once; the only issue is explaining to people why it isn't a problem, and I don't think it's useful enough to be worth that hassle.

#316 ::: P J Evans ::: (view all by) ::: May 14, 2015, 12:05 AM:

3114
As I understand it, they'll go with 4 if there are too many tied in the next spot - meaning if it would put more than 7 on the ballot, counting No Award. (There used to be physical limits on ballots, as in 'only so much space available'.)

And, also, people have a harder time when there are more than about 5 or 6 to rank (as well as this multiplying the amount of time the counting takes - the final ballot program (AIUI) has to go through the process once for every nominee on the ballot, plus the check process for No Award.)

#317 ::: Annie Y ::: (view all by) ::: May 14, 2015, 12:07 AM:

felice @ 315
I don't see any problem with your suggestion in #253 to allow nominating the same work more than once

3.8.4 is the problem. So if you want multiples of the same work nominations to be allowed, you need to propose a change in it (that needs to be approved and ratified as well). So it is not just explaining - it also requires a change in the rules.

#318 ::: felice ::: (view all by) ::: May 14, 2015, 12:17 AM:

Annie Y @317: "3.8.4 is the problem. So if you want multiples of the same work nominations to be allowed, you need to propose a change in it (that needs to be approved and ratified as well). So it is not just explaining - it also requires a change in the rules."

True; I was thinking in terms of fairness rather than rules. We'd still need 3.8.4, since listing the same work multiple times would still only count as one nomination, but 3.8.8 could specify that 3.8.4 doesn't apply to division of points. I don't particularly care either way, though.

#319 ::: P J Evans ::: (view all by) ::: May 14, 2015, 12:35 AM:

felice, can you explain why counting the same work multiple times is more fair than the current rule of only counting it once?

How is this not like a slate, because it gives a some people more voting power than others?

(I have a lot of trouble with your suggestions, as they seem geared to making the process less fair than it currently is.)

#320 ::: Julie ::: (view all by) ::: May 14, 2015, 12:49 AM:

(slightly nervous lurker here who has been following the conversation):

[DECLINED NOMINATIONS]

As I understand it, this procedure does not produce a ranked list. All one can say about the final ordering is: #1-4 are ranked in order of points (they may never have been compared to anything on nominations); #5 has more nominations than any of #6, #7, #8... .

#6 and #7 may never have been compared (I'm think of a case where the eventual #5 starts low down on points, but works its way up the list via strong nomination counts).

So one could find a final ordering with roughly
#1 200 points, 800 nominations
#2 199 points, 199 nominations
#3 198 points, 600 nominations
#4 197 points, 200 nominations
#5 50 points, 401 nominations
-------------------------------
#6: 100 points (before elimination), 100 nominations
#7: 98 points (before elimination), 400 nominations

.... does that give another reason why taking the next one on the list might not be the best choice?

#321 ::: Darth Paradox ::: (view all by) ::: May 14, 2015, 12:54 AM:

[DECLINED NOMINATION]

It occurred to me after posting #280 that it was easy to construct an example to demonstrate my problem with the pick-the-last-eliminated method of replacing a declining nominee. Consider a situation where the electorate is comprised solely of five slate-voting blocs of roughly equal size, with just enough deviation in each bloc to avoid ties (and cause an ordering of each bloc's nominees). Specifically, suppose:

20.2% of the electorate nominates a slate of Abner, Adelaide, Ajay, Alice, and Arianna (and the deviations break ties in that order).
20.1% of the electorate nominates Bart, Bess, Blaine, Brianna, and Burns.
20.0% of the electorate nominates Cameron, Celeste, Chris, Colette, and Cyril.
19.9% of the electorate nominates Dawn, Deanna, Dina, Doris, and Dule.
19.8% of the electorate nominates Eaton, Edsel, Emily, Erica, and Eula.

Given that, the final nominated ballot would be Abner, Bart, Cameron, Dawn, and Eaton; the last five eliminated would be Adelaide most recently, then Bess, Celeste, Deanna, and Edsel.

Now suppose that Dawn declines the nomination. Replacing her with the sixth-place nominee, Adelaide, means the "A" bloc now has two nominees on the ballot and the "D" bloc has none, despite the fact that the two blocs are nearly the same size. But rerunning the process from the beginning (or even from the fifteen-nominees-left point) would replace Dawn with Deanna, the collective second-place choice of the bloc that got Dawn onto the ballot in the first place.

I acknowledge this example doesn't demonstrate the case when the rerun causes other changes to the top five besides simply replacing the withdrawn nominee, but I think the merge/extend discussion has the right of it.

Ultimately, there are two principles I'd like the replacement process to follow:
1) Anyone who would have been on the ballot if the withdrawal/decline was known prior to the votes being counted should still be on the ballot.
2) Anyone who was announced as being on the ballot (and has not intentionally withdrawn or been found ineligible) at any point should not be removed from the ballot.

If this ends up causing six nominees instead of five after a withdrawal, then so be it. As others have said, I'd rather be inclusive in this kind of situation. If it's likely to cause seven or more nominees instead of five after a single withdrawal, then maybe that's an indication that the results produced by the system are too fragile; it seems to me that slight changes to the field only causing slight changes to the results is a desirable property of a voting system.

Keith @ 307:

Is there a particular process for introducing proposals that are dependent on previous proposals, or proposals that are mutually exclusive? I suppose the former can just be handled by waiting to introduce the modifying proposals until the previous one has passed, but taking up competing proposals one at a time seems undesirable (compared to introducing both of them, debating, and then holding a vote between approving A, approving B, or rejecting both).

#322 ::: Ken Burnside ::: (view all by) ::: May 14, 2015, 01:01 AM:

I'm in favor of "Make a list of 10 nominees, use the top 5, if one declines, bring in number 6" if RME cannot be made to work acceptably.

#323 ::: Julie ::: (view all by) ::: May 14, 2015, 01:01 AM:

Actually the numbers I used don't make sense--- you can't have 50 points and 401 nominations, especially not at the end of the process.

Make it
#5 100 points 401 nominations
----------------------------
#6 120 points 120 nominations
#7 119 points 400 nominations


#324 ::: Nancy Lebovitz ::: (view all by) ::: May 14, 2015, 01:02 AM:

#319 ::: P J Evans

Each person gets the a total weight of one vote per category. In effect, if they only vote for one nominee, it's as though they voted a fifth of a vote five times.

I don't see the problem if, instead of voting their five fifths for one work or giving one fifth vote to each of five works, they give two of their fifths to one work, and three of their fifths to another.

#325 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 01:15 AM:

Darth@321:

Right, the two declined nomination proposals would only be introduced afte SDV-LPE passes (assuming it does, of course). My only reason for suggesting the two proposals would be to let the business meeting participants vote, since it wasn't looking like this group could come to a consensus.


But that said, a few people who were against RME now seem to be okay with it. I honestly don't have a strong opinion either way, as always, assuming we can explain what we come up with in a simple, straightforward manner.

Is it time to take another poll?

Everyone (whether you actively post or not, don't feel you can't express your instinct), if you wouldn't mind ranking from 0-10 (per Jameson's system) the following:

In the case of a declined nomination:

1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
3. Do not replace the declined nomination
4. Re-run the election with the declined nomination removed; accept the new results as-is
5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

I think these are the possible permutations that have been discussed. If I've missed any, please let me know.

Thanks,
Kilo

#326 ::: viktor ::: (view all by) ::: May 14, 2015, 01:22 AM:

#324
Long story short, what she said.
And this is why this proposal will ultimatley fail.

#327 ::: viktor ::: (view all by) ::: May 14, 2015, 01:29 AM:

How hard is it to say, "If you nominate something more than once on your ballot, it gets more points! So list it as many times as you like!" That explanation is simpler than the one that the current proposal will have to go through.

I think overall, changing rule 3.8.4 is simpler, clearer, more direct, and mitigates slate voting well enough. Nominate a work as many times as you like on your ballot. Each slot will count.

#328 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 01:32 AM:

viktor and Nancy:

What you are proposing has been discussed before. There's nothing inherently stopping us from allowing this in SDV-LPE, except for the fact that it is another rule change. This can be proposed, obviously. Not having this ability doesn't hurt the system, either, though.

Ultimately, we decided that the goal of the nomination stage was to suggest works that the fans think might be Hugo-worthy. That's all. In the final voting stage, voters actually rank their preferences. Not having ranked nominations also removes a large portion of strategy in voting, which was another design goal when we start all of this.

So, I don't think the proposal will fail because of this. You may disagree with the goals that were laid out, but we arrived at those goals after a great deal of very thoughtful discussion.

Kilo

#329 ::: Laertes ::: (view all by) ::: May 14, 2015, 01:43 AM:

I like option (2).

On its face, option (1) is more appealing, but this corner case isn't so cornery anymore. We can't see the future, but now that declining nominations has become more common, it seems proper to spell out how it's to be handled.

(3) would hurt the proposal's chances since replacing nominees who declined clearly improved the ballot this year. (4) invites catastrophe. (5)-(7) are excessively fiddly.

#330 ::: felice ::: (view all by) ::: May 14, 2015, 02:01 AM:

P J Evans @319: "felice, can you explain why counting the same work multiple times is more fair than the current rule of only counting it once? How is this not like a slate, because it gives a some people more voting power than others?"

It doesn't give anyone more voting power; the elimination comparison is based on how many ballots the work appears on, and the number of times it appears on each ballot is irrelevant. This really only matters in situations like "I like Alpha, Beta, and Gamma, but I like Alpha most, and I'm not sure I want to nominate the others because I don't want to help them beat Alpha". Nominating Alpha Alpha Alpha Beta Gamma would mean Alpha gets 0.6 points, and the other get 0.2 points; that nominator still only gets one point total. If they felt really strongly about Alpha, they could just list Alpha once and leave the rest of the ballot blank, so Alpha would have the full 1 point right from the start. This basically makes bullet voting a bit less appealing, which is a good thing, but the effect would be negligible.

#331 ::: P J Evans ::: (view all by) ::: May 14, 2015, 02:15 AM:

felice, dammit, pay attention: the process of choosing nominees is not preferential. You're trying to change that, and you have to convince people that it's better if we allow this kind of overvote.

So far, all you're saying that you think people should be able to put the same name in more than once in the same category, which currently is the same as putting it in ONCE. This is FAIR, it means that you can't stuff the ballot box by getting a group together to put the same name (or names) five times by each person. You see the problem? It's a slate in a slightly different form. We don't want that.

#332 ::: Darth Paradox ::: (view all by) ::: May 14, 2015, 02:28 AM:

P J Evans @ 331: If a nomination ballot of "Alice" (allocating 1.0 point to Alice) is fair, and a nomination ballot of "Alice, Bob" (allocating 0.5 points to Alice and 0.5 points to Bob) is fair, why isn't a nomination ballot of "Alice, Alice, Alice, Bob" (allocating 0.75 points to Alice and 0.25 points to Bob) fair? It doesn't violate the requirement that each person only has one total point to be allocated at any given time. And it doesn't enable slate voting any further; a bunch of people getting together to vote "Alice, Alice, Alice, Alice, Alice" is no different than a bunch of people getting together to just vote "Alice".

I agree that the lack of simplicity in describing such a system in the constitution is a strike against it, but I really don't see the argument on fairness that you're making here.

#333 ::: felice ::: (view all by) ::: May 14, 2015, 02:35 AM:

#331 - I really don't think you're getting how SDV-LPE works at all. With viktor's proposal, the ballots "A" and "AAAAA" would be exactly equivalent, just as they are now. The ballots "ABC" and "AAABC" would have slightly different effects on the relative rankings of A, B, and C, but they'd affect the rankings less than switching from "ABC" to "A" would. In no case would listing A more than once help it win an elimination comparison.

And I'm not interested in convincing anyone that viktor's proposal is a good idea; I don't care if it happens or not.

#334 ::: felice ::: (view all by) ::: May 14, 2015, 02:48 AM:

Keith "Kilo" Watt @325: " if you wouldn't mind ranking from 0-10 (per Jameson's system) the following: In the case of a declined nomination:"

3 - "Let the admins decide" - this would just result in the debate being repeated every year.
4 - "Add the works eliminated in the next-to-last round to the ballot" - easy but potentially unfair on those who nominated the declined work.
2 - "Do not replace the declined nomination" - easiest but even more unfair.
0 - "Re-run the election with the declined nomination removed; accept the new results as-is" - telling people they're finalists, then that they're not, isn't acceptable.
9 - "Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary" - not difficult, the fairest option, and the chances of the ballot getting unreasonably big are very small.
1 - "Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points" - if there are other new finalists, they'll find out when we publish the nomination data, and could quite reasonably be very upset about being excluded from the final ballot.
1 - "Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations" - ditto.

#335 ::: viktor ::: (view all by) ::: May 14, 2015, 02:49 AM:

@331
"the process of choosing nominees is not preferential."

Not now. However, the current proposal would make it so.

Simpler:

"The more you vote for your nominee, the more points it gets!"

#336 ::: Seth ::: (view all by) ::: May 14, 2015, 03:02 AM:

#325 In the case of a declined nomination:

8. Re-run the counting until there is only one finalist who was not on the original list, and add that one.

#337 ::: John M. Burt ::: (view all by) ::: May 14, 2015, 03:10 AM:

Does anybody have advice on how to deal with a parasite called http://www.general-ebooks.xyz/ ?

They are selling access to my writing without paying me. How can I stop them?

Sorry to be interrupting with such an elementary problem, but it is a new one to me.

#338 ::: Idumea Arbacoochee, Bad Cop ::: (view all by) ::: May 14, 2015, 03:12 AM:

Let me confirm what Keith "Kilo" Watt @304* has said: this thread is not about whether to change the process. That's a decision for the business meeting at Worldcon.

This thread is about one proposal to change it, which can then be brought to the conversation at Sasquan.

Given that, we're going to leave "whether" conversations aside from this point forward, along with a rehash of the behaviors that have led us to this point, except to the degree that they constitute valid input data.

Brian Z, if you're looking to get disemvoweled, now you know how. You can also achieve those heights by being personally insulting to other members of the conversation, trolling pointlessly, and ignoring moderators. It's actually not a very easy thing to do, for someone who is operating in good faith. Which, until I see evidence to the contrary, I assume everyone is.

My comparison of what you said on File 770 to what you say here is not yet sufficient evidence, but I would strongly suggest a better unity of message across sites that the same people read. It may be easier to remind yourself to do that if you include links back and forth, as you yourself advised others to do.

Also, Walter Daniels @290, you might have a better run in this community if you increased the spacing between your paragraphs (which I have amended), and maybe dropped the emotive language "screaming" and the accusatory tone. I know conspiracy theories are a delight to the intricately-oriented, but you either need evidence or objective probability, and I don't see either in your comment.

(And, for the record, I don't live in New York or work in publishing.)

------
* whom I esteem very highly indeed, though I have never previously had the honor of his acquaintance, online or off. Keith, do you write poetry?

#339 ::: Louis Patterson ::: (view all by) ::: May 14, 2015, 03:41 AM:

Declined nominations should use the same process as is used for handling late-detected ineligible entries, whatever that process is decided to be.

#340 ::: junego ::: (view all by) ::: May 14, 2015, 04:03 AM:

Keith "Kilo" Watt @ 325
In the case of a declined nomination:

I'd like to change and clarify the above condition for this proposal.

In the case of any change (elimination or withdrawal) in the finalist list after the finalists have been notified (and up to the point where the administrators determine that the ballet must be frozen to any further changes, at which point no new finalists will be added):

7 pts - Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
5 pts - Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
2 pts - Do not replace the declined nomination
0 pts - Re-run the election with the declined nomination removed; accept the new results as-is
10 pts - Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
0 - Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
0 - Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

Rationale: I've nuanced my previous positions. IF we are going to take any position on how to handle changes in the finalist list I now feel that felice's proposal is the cleanest and fairest. (I didn't originally grasp her point way up thread, so 'Oops'). If we are not going to address this question, then it would by default be left up to the administrators but I feel we should make them aware of the potential problem and our discussion/possible solutions.

Any eliminations or withdrawals *before* finalists are notified should entail a full rerun of SDV-LPE.

#341 ::: junego ::: (view all by) ::: May 14, 2015, 04:39 AM:

Nancy Lebovitz @ 324
P J Evans @ 319

Each person gets the a total weight of one vote per category. In effect, if they only vote for one nominee, it's as though they voted a fifth of a vote five times.
I don't see the problem if, instead of voting their five fifths for one work or giving one fifth vote to each of five works, they give two of their fifths to one work, and three of their fifths to another.

To the best of my understanding (and someone correct me if I'm wrong) the problems with this and all the permutations of proposing that members can/should prioritize their nominations are:

1) It would require additional constitutional changes because article 3.7.1 of the constitution says the nominations shall be "up to five (5) equally weighted nominations in every category." My emphasis.

2) After discussions in the last 2000+ comments, the consensus was that minimal changes had the best chance of being accepted. So the fewer ammendments and simpler the proposal, the better.

3) Those discussions also came to a consensus that more members than not disliked the idea of ranking the nominations, so any proposals to include weighting the recommendations was dropped.

And a clarification (I'm not sure from the above if there is confusion about this but better too much than too little explanation), If you nominate 1 thing it gets 1 point, 2 things means each gets 1/2 point, 3=1/3 pt, etc. up to 5 nominations.

As the Nominees 'challenge' each other and get eliminated, the proportions of your nominating point change if and when your nominees are eliminated. So if you had 3 and 1 loses its challenge, the remaining 2 go from having 1/3 point each to 1/2 point each.

#342 ::: junego ::: (view all by) ::: May 14, 2015, 04:40 AM:

Nancy Lebovitz @ 324
P J Evans @ 319

Each person gets the a total weight of one vote per category. In effect, if they only vote for one nominee, it's as though they voted a fifth of a vote five times.
I don't see the problem if, instead of voting their five fifths for one work or giving one fifth vote to each of five works, they give two of their fifths to one work, and three of their fifths to another.

To the best of my understanding (and someone correct me if I'm wrong) the problems with this and all the permutations of proposing that members can/should prioritize their nominations are:

1) It would require additional constitutional changes because article 3.7.1 of the constitution says the nominations shall be "up to five (5) equally weighted nominations in every category." My emphasis.

2) After discussions in the last 2000+ comments, the consensus was that minimal changes had the best chance of being accepted. So the fewer ammendments and simpler the proposal, the better.

3) Those discussions also came to a consensus that more members than not disliked the idea of ranking the nominations, so any proposals to include weighting the recommendations was dropped.

And a clarification (I'm not sure from the above if there is confusion about this but better too much than too little explanation), If you nominate 1 thing it gets 1 point, 2 things means each gets 1/2 point, 3=1/3 pt, etc. up to 5 nominations.

As the Nominees 'challenge' each other and get eliminated, the proportions of your nominating point change if and when your nominees are eliminated. So if you had 3 and 1 loses its challenge, the remaining 2 go from having 1/3 point each to 1/2 point each.

#343 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 04:46 AM:

On excellence and happiness: this system is designed to balance different kinds of evidence to find the 5 most-likely-to-be-excellent works. The fact that this is fair is no coincidence; unfair systems ignore evidence and promote strategic voting (bad evidence). The fact that this promotes happy voters is just a happy coincidence.

On multiple votes: there's nothing wrong with the idea except complexity. A separate proposal to allow this would be fine.

On withdrawn nominees : I vote 10 for "let the administration decide " or "let the BM decide (multiple proposals)". 7 for everything else (considered as proposals in isolation).

#344 ::: felice ::: (view all by) ::: May 14, 2015, 04:51 AM:

junego @340: "I'd like to change and clarify the above condition for this proposal. In the case of any change (elimination or withdrawal) in the finalist list after the finalists have been notified (and up to the point where the administrators determine that the ballet must be frozen to any further changes, at which point no new finalists will be added)

Any eliminations or withdrawals *before* finalists are notified should entail a full rerun of SDV-LPE.

Good points.

#345 ::: Aan ::: (view all by) ::: May 14, 2015, 05:11 AM:

[2] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[9] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[6] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[10] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[1] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#346 ::: Soon Lee ::: (view all by) ::: May 14, 2015, 05:31 AM:

[DECLINED NOMINATION]

[3]1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[2]2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[2]3. Do not replace the declined nomination
[0]4. Re-run the election with the declined nomination removed; accept the new results as-is
[8]5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0]6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[0]7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#347 ::: J Thomas ::: (view all by) ::: May 14, 2015, 06:04 AM:

@281 Jameson Quinn

I think that the "rerun and merge" contingent should make a separate proposal. It wouldn't have to involve lots of lawyerly language; just "It is the sense of the membership that if a nominee declines, and the ballots have not yet been finalized, then the award administrators should find all works which would have been nominees if the decliner were removed from all ballots, and offer them places as finalists, without withdrawing finalist status from any works which already have it."

That makes sense, and the SDE proposal should not mention what to do in the case of declines or removals.

If the BM rules allow it, maybe the second proposal should be a proposal-for-a-proposal. List the alternatives suggested so far and see which way the meeting chooses to amend.

It will rarely cause trouble any way we do it, and if the meeting considers it too unimportant to deal with, then it's settled.

And if the SDV proposal fails then it's moot.

#348 ::: Nancy Lebovitz ::: (view all by) ::: May 14, 2015, 06:41 AM:

#319 ::: P J Evans

I don't see how allowing multiple votes for the same thing makes the process less fair.

#326 ::: viktor

However, I also seriously doubt that whether preferential nominating votes are permitted is anywhere near a make-or-break for SDV-LPE.

#342 ::: junego :

Thanks for the explanation-- I'll quit pushing the idea in this round.

I have no idea how much people in general would prefer being able to vote preferentially.

#349 ::: J Thomas ::: (view all by) ::: May 14, 2015, 07:10 AM:

@280 Darth Paradox

I don't have the math to back this up at the moment, but my intuition tells me that promotion of sixth place runs a high risk of disenfranchising the slate bloc entirely.

The results will depend on circumstance. We can make up examples where a big variety of things happen, and the question is what's more likely, and that depends on lots of things. It's a hard question.

I have run simulations based on votes that are a lot like what happened in 2013. I can't be sure I got it right, but it probably isn't real far off. When I add a slate of 15% of the nonslate numbers, they average about 1.7 finalists. They are out of proportion to their numbers still, though not as much. This is because the other votes are very scattered.

Assuming that my simulated results are representative of later years, chances are if one slate work got removed late in the process there may quite likely already be another slate finalist.

Suppose though there's only one. Then it comes down to the details. While there are two slate nominees with around N votes, they will have scores around N/2. They will be paired up with nonslate nominees that have scores less than N/2 and beat every one of them, until they meet each other and one is eliminated. If there are only three nonslate nominees that have scores above N/2 at that time, then both slate items will be finalists. If there are four, then one of them will have the 6th spot and will be next in line if any of the top five is removed. If there are five or more, then the 2nd slate item is out of the running.

How can we predict that? If a nonslate work has less than N/2 votes total, then it will definitely be eliminated. In 2015, in a whole lot of categories, there were few works that had more than N/2 votes. If that's a precedent then you're home free.

Anyway, we can't expect to make it entirely fair to slate voters no matter what happens. If we go with "throw in whatever's in 6th place" and there's a 1/5 chance that a slate work will be in 6th place, then if an existing slate item gets removed that's a 1/5 chance we get another one. But if any of the other 4 items are removed we also get a 1/5 chance of a slate item. Is that close to fair?

Agreed that if we run the numbers again we definitely get at least one slate item the new way. It's more representative of the actual ballots.

All this may be academic. In 2013 the Puppy slate essentially did not exist. In 2014 it was a handful of people. In 2015 it was hundreds. By 2016 it could be well over a thousand. And by 2017 you guys may have moved on to something more important.

This whole exercise might turn into a case of locking the barn door after the horses have escaped, formed a commune, taken ownership of the farm, and burned the place down. But it's still worth preparing for some other slate.

#350 ::: J Thomas ::: (view all by) ::: May 14, 2015, 07:24 AM:

@250 Annie Y

[5% RULE]
So if there are other proposals to get rid of that one (yey - it made sense when the number of ballots was very low but not anymore...)

This is a minor picky point, but the issue is not how many ballots there are, but how many works that people might nominate. If they split their votes over 20 possible nominations, then somebody *has* to get 5%. If they split among 1000 works, then quite possibly none of them get 5%. That is true for 100 voters or a million voters. If they're limited to 5 votes each, and they split them among 1000 choices, then most choices will get very few.

#351 ::: J Thomas ::: (view all by) ::: May 14, 2015, 07:37 AM:

@252 Jameson Quinn

In a world without slates, there is essentially no benefit to bullet voting. Approval totals and honest point totals would agree on which candidates were better, so if your favorite candidate was losing, boosting its point total would only put off the inevitable (and if everybody did it, you could actually be worse off, because your favorite would be getting no second-choice votes).

I don't think that's quite fair, though it's pretty accurate.

If your preferred finalist is unpopular, it has little chance in any reasonable voting system.

If your preferred candidate is very popular, it has a very good chance in any reasonable voting system.

If it is popular enough to have a chance, you increase its chance by bullet-voting more than by voting for others also. If it is not one of the five with the most votes, its only chance comes if some among the top five have enough shared votes that they are selected to eliminate each other. The more of the ones with more votes than yours do that, the better chance yours has. The more shared votes your favorite has, the better the chance it will be eliminated.

However, usually it doesn't make much difference. And when you bullet-vote, you lose any chance to help out the other nominations you like. In general, when you vote for a second nomination you help the second one much more than you hurt the first one. So if you care about the second, vote for it.

I don't have the math for that and haven't thought about how to do it. Can you think of how? When you vote for a second work, how much on average does that hurt the first one's chances? How much on average does it help the second one's chances? How much does a third nomination hurt the first two compared to helping the third?

I can vaguely imagine a brute-force approach on that, but it seems ugly and it might take a long time. ;-)

#352 ::: Yarrow ::: (view all by) ::: May 14, 2015, 08:08 AM:

[10] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[6] 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[3] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[4] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[2] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[2] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#353 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 08:11 AM:

@295: it appears that something is going wrong with the loading of data. Try "tail (hugos)" to see what the data looks like.

#354 ::: nathanbp ::: (view all by) ::: May 14, 2015, 08:20 AM:

[2] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[5] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[2] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[8] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

I think it makes most sense to decide this clearly in the proposal rather than leave it to the admins later, that seems like a recipe for people being unhappy later.

J Thomas@351: That seems like the kind of thing that would require a lot of simulation with simulated voters with different voting styles. Your individual marginal vote is unlikely to change the results.

#355 ::: Cat ::: (view all by) ::: May 14, 2015, 08:43 AM:

For the purpose of weighting what I said upthread and also because two options are very close to even in my mind and right now I slightly prefer the other one.

[DECLINED NOMINATION]

[4]1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[9]2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[6]3. Do not replace the declined nomination
[0]4. Re-run the election with the declined nomination removed; accept the new results as-is
[7]5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0]6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[0]7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#356 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 08:45 AM:

No election theorist here, just a fan with gut feelings. But since you asked....

[9] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[5] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[7] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[6] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#357 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 08:52 AM:

Nancy Lebovitz @348, I think part of the problem here is terminology. Everyone gets to vote preferentially; that's what voting IS. What seems to be a sticking point is nominating preferentially. Which is a different thing.

And on a gut level, to me, nominations are "this is the thing that I loved; I hope you love it too!". Weighting nominations is "I'm going to throw three slips of paper with the same thing into the hat!" which, on a visceral level, strikes me as more unfair than "I'm going to throw three slips of paper with three things that I loved into the hat. Not necessarily rational, but I much much much prefer the second option. Please, tell me all the things you loved. But if you only loved one thing, you only have to tell me once.

Anyway, if you only loved one thing, then bullet-voting makes sure it gets 100% of your points in that category, so what's the point of pro-rating? If you loved it three times more than the other two things on your ballot, why are you nominating the other two things on your ballot?

#358 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 09:07 AM:

@351:

If your preferred finalist is unpopular, it has little chance in any reasonable voting system. Bullet voting doesn't help.

If your preferred candidate is very popular, it has a very good chance in any reasonable voting system. Bullet voting doesn't help.

If it is right on the edge - that is, if it comes in 6th place with your honest vote - then most probably it comes in 6th place in both points and approvals. Bullet voting can improve its points, possibly bringing it up to 5th place. But it still gets chosen for the final elimination round, and its approvals are still in 6th, so bullet voting doesn't help.

If the nonstrategic result is that it is in 6th place or below on points, but 5th place on approvals, then it would already be winning based on approvals, so bullet voting doesn't help.

If the nonstrategic result is that it is in 6th place or below on approvals, and 5th place on points, let's call it a "squeaker". Whether bullet voting helps still depends on some other factors.

If it's a squeaker, and the other things you might vote for are not in the top 5, then your ballot is already giving it full points when it matters. Bullet voting doesn't help.

If it's a squeaker, and without your ballot it is more than half a point below the fourth place work, then bullet voting almost helps, but isn't enough.

If it's a squeaker, and you would have voted for one of the top 5, and it is less than half a point from 4th place... then, bullet voting helps.

But even then, it's worth analyzing further. Why would a work end up being a squeaker? That is, why would it do better on points than on approvals?

There are basically two reasons that could happen. Either it could be getting more points than you'd expect from its approvals, or its closest rivals could be getting fewer points than you'd expect based on their approvals.

If it's the former — the work in question has more points per approval than most others — then that could be because of something about the work itself, or something about its fans. Perhaps the work is something that naturally makes some people say "it would be a crime if this one thing didn't get the Hugo"; that is, perhaps it's really outstanding, but it's a difficult or controversial work so it is still a squeaker. In that case, bullet voting could help, but it's entirely appropriate that it does; the work probably truly deserves a place among the finalists. Or perhaps its fans are just more inclined to bullet voting strategy than other fans. In that case, it probably doesn't deserve a slot.

So finally, after analyzing 8 different possible scenarios, each less likely than the last, we've come up with a situation where bullet voting strategy is helpful to the voters but harmful to the ideals of the Hugo.

But we're not quite done. What if your favorite is a squeaker not because it got more points per approval than average, but because two of its closest rivals got fewer? In that case, its rivals are probably on a slate. So if slates are common, bullet voting may help. But note that in this case, too, the bullet voting result is arguably "fairer" than the non-bullet-voting result.

So final tally: 9 scenarios, in order of increasing unlikeliness and specificity. Bullet voting only helps in scenarios 7, 8, and 9; the three least likely ones. Bullet voting leads to a worse overall result (less "excellence" among the finalists) only in scenario 8.

....

As to running simulations to see how likely all this is in practice: this kind of thing is the focus of much of my research. It's tricky to do; you have to make assumptions about what kind of voting is likely, and results can depend a lot on those assumptions. Based on the intuition I've gotten from running these sims, I think that you could make a sim where the average effect of bullet voting was non-negligible, but only by making assumptions I'd characterize as implausible.

#359 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 09:08 AM:

viktor @309, I don't understand your objection. Your nominating ballot *isn't* weighted in any way, either the "way you wanted it" or some other way. Each nomination in a category has exactly the same weight as every other nomination you make in the same category. As works you nominated get eliminated, the remaining works you nominate in that category get stronger. Equally. Until we're down to five candidates. If you nominated Alex, Barbara, Charlie, Dora, and Edward, and after all the tests Alex, Dora, Edward, Frances, and George end up on the ballot.... your 1/3 nomination each helped them get there. (But note, it started as a 1/5 nomination each, until Barbara and Charlie were eliminated.)

And if Alex declines (or is found invalid), and they rerun the results, you're starting with 1/4 nomination for Barbara, Charlie, Dora, and Edward. Which might be just enough of a difference to kick Barbara or Charlie into the finals, on a re-run. Is that a bad thing? I mean, you liked them enough to put them on your ballot... <smile>

(Note: I favor keeping Dora, Edward, Frances, and George on the ballot regardless of the results of the rerun. I like the "merge the two runs" option best, but I'd take "additional top point-getter added")

I hope this clarifies; it looks to me like you're operating from a fundamental misunderstanding of the process. I have every sympathy; I'm not a math or voting geek, and if I've misrepresented the system I hope someone corrects me right away.

I personally had real trouble following this discussion for a while until I realized that the word "vote" used about the nomination process was distorting my understanding of what was going on, because a "vote" implies ranking on a ballot, and nominators aren't ranking on a ballot. They're throwing names into a hat.

#360 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 09:12 AM:

This is a repost of my previous comment, with numbering added, so it's clear how I'm counting the scenarios.

@351:

1. If your preferred finalist is unpopular, it has little chance in any reasonable voting system. Bullet voting doesn't help.

2. If your preferred candidate is very popular, it has a very good chance in any reasonable voting system. Bullet voting doesn't help.

3. If it is right on the edge - that is, if it comes in 6th place with your honest vote - then most probably it comes in 6th place in both points and approvals. Bullet voting can improve its points, possibly bringing it up to 5th place. But it still gets chosen for the final elimination round, and its approvals are still in 6th, so bullet voting doesn't help.

4. If the nonstrategic result is that it is in 6th place or below on points, but 5th place on approvals, then it would already be winning based on approvals, so bullet voting doesn't help.

If the nonstrategic result is that it is in 6th place or below on approvals, and 5th place on points, let's call it a "squeaker". Whether bullet voting helps still depends on some other factors.

5. If it's a squeaker, and the other things you might vote for are not in the top 5, then your ballot is already giving it full points when it matters. Bullet voting doesn't help.

6. If it's a squeaker, and without your ballot it is more than half a point below the fourth place work, then bullet voting almost helps, but isn't enough.

If it's a squeaker, and you would have voted for one of the top 5, and it is less than half a point from 4th place... then, bullet voting helps.

But even then, it's worth analyzing further. Why would a work end up being a squeaker? That is, why would it do better on points than on approvals?

There are basically two reasons that could happen. Either it could be getting more points than you'd expect from its approvals, or its closest rivals could be getting fewer points than you'd expect based on their approvals.

If it's the former — the work in question has more points per approval than most others — then that could be because of something about the work itself, or something about its fans.

7. Perhaps the work is something that naturally makes some people say "it would be a crime if this one thing didn't get the Hugo"; that is, perhaps it's really outstanding, but it's a difficult or controversial work so it is still a squeaker. In that case, bullet voting could help, but it's entirely appropriate that it does; the work probably truly deserves a place among the finalists.

8. Or perhaps its fans are just more inclined to bullet voting strategy than other fans. In that case, it probably doesn't deserve a slot.

So finally, after analyzing 8 different possible scenarios, each less likely than the last, we've come up with a situation where bullet voting strategy is helpful to the voters but harmful to the ideals of the Hugo.

9. But we're not quite done. What if your favorite is a squeaker not because it got more points per approval than average, but because two of its closest rivals got fewer? In that case, its rivals are probably on a slate. So if slates are common, bullet voting may help. But note that in this case, too, the bullet voting result is arguably "fairer" than the non-bullet-voting result.

So final tally: 9 scenarios, in order of increasing unlikeliness and specificity. Bullet voting only helps in scenarios 7, 8, and 9; the three least likely ones. Bullet voting leads to a worse overall result (less "excellence" among the finalists) only in scenario 8.

#361 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 09:23 AM:

Can I request that the people voting in the [DECLINED NOMINATIONS] poll (including those who have already voted) rate a couple of additional options? To wit:

We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better

We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.

I'd vote 10 for either of the above options.

#362 ::: J Thomas ::: (view all by) ::: May 14, 2015, 09:26 AM:

@354 nathanbp

J Thomas@351: That seems like the kind of thing that would require a lot of simulation with simulated voters with different voting styles. Your individual marginal vote is unlikely to change the results.

Yes, exactly!

Your individual vote makes so little difference, and your careful strategy based on your estimate of what's popular makes so much less difference, that it's best to just pick the things you think most deserve the Hugo. Putting much thought about voting strategies is surely not worth doing unless you are part of a cabal that intends to work together to get a disproportionate effect on the nominations.

So it becomes a question how much can organized groups of voters affect the results by various strategies, and how large a group does it take to make much difference.

And yet, some people will go by their gut feeling of what works better, and then use their gut feeling about what's popular, to do strategic voting. And if there are a lot of them it will affect the results in unknown ways.

Jameson is right that if a lot of people bullet-vote at random, the result is that they individually have less effect (because their 2nd and 3rd choices are less likely to win, while their 1st choices don't get the benefit of other people's 2nd and 3rd choices). And by reducing the number of total votes, they make other voting strategies work better.

My guess is that bullet-voting by a cabal is likely to have the largest effect if the item they bullet-vote for is in 6th place, somewhat less in 5th place or 7th place. By the time it gets up to 3rd place or down to 9th place it will probably have little effect even for a large group.

How much effect it has depends on how spread the ballots are for works that are more popular, and on who their ballots are split with. You will not know that, and depending on how it comes out, a group's effort at bullet-voting could have no effect even in the best case. Your group could bullet-vote for something that would win anyway, or that would inevitably lose anyway.

The strategic voting rule that I think works, is:

If you like Annie and Bob, and you would hate for Bob to win while Annie loses, then vote only for Annie. Particularly if you think Bob is more popular.

There are voting systems that let you arrange to vote for Bob only after Annie has lost, or only after it's decided that Annie has won or lost. They tend to have other problems that got us to put them aside.

#363 ::: J Thomas ::: (view all by) ::: May 14, 2015, 09:27 AM:

@354 nathanbp

J Thomas@351: That seems like the kind of thing that would require a lot of simulation with simulated voters with different voting styles. Your individual marginal vote is unlikely to change the results.

Yes, exactly!

Your individual vote makes so little difference, and your careful strategy based on your estimate of what's popular makes so much less difference, that it's best to just pick the things you think most deserve the Hugo. Putting much thought about voting strategies is surely not worth doing unless you are part of a cabal that intends to work together to get a disproportionate effect on the nominations.

So it becomes a question how much can organized groups of voters affect the results by various strategies, and how large a group does it take to make much difference.

And yet, some people will go by their gut feeling of what works better, and then use their gut feeling about what's popular, to do strategic voting. And if there are a lot of them it will affect the results in unknown ways.

Jameson is right that if a lot of people bullet-vote at random, the result is that they individually have less effect (because their 2nd and 3rd choices are less likely to win, while their 1st choices don't get the benefit of other people's 2nd and 3rd choices). And by reducing the number of total votes, they make other voting strategies work better.

My guess is that bullet-voting by a cabal is likely to have the largest effect if the item they bullet-vote for is in 6th place, somewhat less in 5th place or 7th place. By the time it gets up to 3rd place or down to 9th place it will probably have little effect even for a large group.

How much effect it has depends on how spread the ballots are for works that are more popular, and on who their ballots are split with. You will not know that, and depending on how it comes out, a group's effort at bullet-voting could have no effect even in the best case. Your group could bullet-vote for something that would win anyway, or that would inevitably lose anyway.

The strategic voting rule that I think works, is:

If you like Annie and Bob, and you would hate for Bob to win while Annie loses, then vote only for Annie. Particularly if you think Bob is more popular.

There are voting systems that let you arrange to vote for Bob only after Annie has lost, or only after it's decided that Annie has won or lost. They tend to have other problems that got us to put them aside.

#364 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 09:34 AM:

@361, We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better

We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.

I'm not entirely sure what your second proposal means; is it (a) "let the administrators decide, and these are their two options to decide from"? Or is it (b) "Let the administrators decide" and then two separate proposals after the initial proposal which modifies that one way or the other? (On re-reading, I think this is what you meant, but I'm not positive).

If (b) is what you mean, I vote ten on both proposals. If (a) is what you meant, it just adds confusion, in my opinion.

And I forgot to put [DECLINED NOMINATION] in my 356. My bad.

#365 ::: Duncan J Macdonald ::: (view all by) ::: May 14, 2015, 09:36 AM:

In the case of a declined nomination:

10 -- 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
0 -- 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
5 -- 3. Do not replace the declined nomination
0 -- 4. Re-run the election with the declined nomination removed; accept the new results as-is
0 -- 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
0 -- 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
0 -- 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#366 ::: Duncan J Macdonald ::: (view all by) ::: May 14, 2015, 09:46 AM:

[DECLINED NOMINATIONS] Addendum

0 -- We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better

10 -- We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.

#367 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 09:50 AM:

@364: Yes, it's (b). Here's a better wording:

We create a base proposal which lets the administrators decide, and two supplementary proposals which would change that to "promote #6" or "rerun and merge". Let the business meeting decide which to pass.

#368 ::: J Thomas ::: (view all by) ::: May 14, 2015, 10:02 AM:

[10] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)

If they create a common practice the BM can confirm it later.

[0] 2. Add the works elininated in the next-to-last round to the ballot

If it's controversial here it may be controversial at the BM.

[4] 3. Do not replace the declined nomination

Not great but do-able.

[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is

If it's controversial here it may be controversial at the BM.

[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary

It doesn't hurt anybody much. If somebody becomes a finalist you think shouldn't, they don't hurt the other finalists much unless they win, and if they do win the Hugo then probably they deserved to be on the ballot.

[0] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points

bad compromise between two controversial proposals.

[0] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

bad compromise.

[1] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better

OK if the chair likes it.

[1] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.

OK if the chair likes it.

[10] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

I think that RME is the obvious best choice, while "add the old 6th" is one of the controversial also-rans which does not deserve equal billing. Others may disagree. ;-) Better to give only "let the administrators decide" top billing, since that's what they will want to do if it's controversial at the meeting and they don't want to spend much time on it.

#369 ::: albatross ::: (view all by) ::: May 14, 2015, 10:05 AM:

[0] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[10] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[5] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[5] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[5] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

This new voting scheme makes it hard to know how to deal with declined nominations or disqualifications after the nominations were announced, so I think it's important to nail that down up front. I don't see why the admins of the election will be in a better position to decide what to do later on.

I also think it's a disaster to announce someone's nomination and then withdraw it because of someone else's disqualification. That's pretty much optimal for creating bad feelings all around.

As I understand it, any way we rerun the election with removed candidates has the possibility of giving us weird results like A,B, C,D,E wins the first election, and after disqualifying A getting some very different outcome like B, F, G, H, I. Radically different outcomes are unlikely but not impossible, right? That leaves us with some ugly decision where we either end up with 8 nominees, or have to decide which winners of the second election get nominated.

It seems simpler and better all around, to me, to just get an ordered list from the original election, and then fill in removed nominees from that list. It's not a disaster to do this in a more complicated way, but I don't think it's a very good idea.

#370 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 10:12 AM:

abi@338:

Keith, do you write poetry?

Yes, but sadly nothing worth reading. One of the only downsides to learning to write is that you suddenly realize just how atrocious some of your stuff is. :) My prose is somwhat better. Stanley Schmidt said my writing was "quite good at times" -- but he still didn't buy the piece.

Kilo

#371 ::: J Thomas ::: (view all by) ::: May 14, 2015, 10:22 AM:

@282 felice

I would certainly consider my nomination to have been wasted if the work in question was withdrawn after becoming a finalist, and extremely peeved if any of my other nominations missed out on being finalists because of the wasted nomination.

We're arguing about what's fair, and there's fundamental disagreement.

It's something which will come up rarely. (Though with 17 or so different votes/year, and possibly declines will go up, it might come sooner than we'd expect.)

I don't think you will convince your heart-felt opponents, whatever you say.

So we need a way to put this off. When we argue about this topic we are delaying a proposal. (That's good for people who don't want a proposal.)

It will be hard for us to agree, and then if the business meeting is also disagreed then anything we decide on will be a reason for them not to pass it.

When you and I disagree about what's fair to you versus me, we can probably work out some compromise. When we disagree about what's fair to some third party in the future, that's a lot harder. We need to finesse this and get on with the other issues.

#372 ::: Laertes ::: (view all by) ::: May 14, 2015, 10:34 AM:

Okay, I think I see now how this is done. Here's another attempt at my @329, maybe formatted properly:

[7]1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[10]2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0]3. Do not replace the declined nomination
[0]4. Re-run the election with the declined nomination removed; accept the new results as-is
[4]5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[1]6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[1]7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#373 ::: Laertes ::: (view all by) ::: May 14, 2015, 10:42 AM:

Brian Z @288:

had my words on another forum stripped from the topic sentence of the paragraph to paint my words in the worst possible light and cited out of context without a link

You know, I let this go the first time (@152) you demanded a link that was already present, but since you're still banging on about it: In my @148, I omitted no meaningful context, and I provided a link.

You seem to be aware that "I was quoted out of context" is phrase with some power, but you're not getting how it works. Used correctly, it means that there was surrounding material that would make the quote less damning. In a case like this one, where you regret the quote because it turns out to have been a dumb thing to say, what you're looking for is "I apologize for lying about you all," not "I was quoted out of context.

#374 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 10:42 AM:

I've not been following this thread too closely, because, honestly, my eyes start glazing over after a while. My problem, not yours. I have, however, been ducking in every so often to check it out, and noticed the voting on how to deal with withdrawals/disqualifications. After scrolling back to get a sense of what each option means, here're my gut choices:

[8] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[5] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[4] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[3] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[5] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[10] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[8] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#375 ::: albatross ::: (view all by) ::: May 14, 2015, 10:44 AM:

I don't think the new voting scheme proposal is complete until it deals with withdrawn candidates, because that's an important feature of Hugo nominations, and it potentially interacts badly with SDV-PE. Worse, it's a little unintuitive and complicated to think about, which means there's a lot of opportunity for people to make dumb decisions and also a lot of opportunity for people to convince themselves some dumb decision was made from evil motives. Most peoples' experiences with voting would lead them to expect that dealing with a withdrawn candidate would be trivial, not something that could conceivably change the whole set of nominees.

I also don't see why kicking the can down the road makes it more likely to be solved well later--the worst case is to have the admins making the decision after the votes have been cast and the situaiton has come up, and that's an awful situation to leave them in.

#376 ::: Shane ::: (view all by) ::: May 14, 2015, 10:51 AM:

[FAQ / ARGUMENT]
"1.How does this eliminate slates?"
I think the FAQ should acknowledge the Puppies main concern (valid or not) by saying SDV-LPE limits the effect of hidden, informal or accidental slates the same as it does public ones.

And then say "With SDV-LPE, slates or blocs should get results in proportion to their numbers."


"15. Wasn’t this system just designed by Social Justice Warriors to block the Good Stuff?"

This answer should say, probably at the start "Those of us who worked on the system feel it is the best way to address the Puppies' concerns about hidden or informal slates. We are keen to discuss it and to run model data with any interested parties."

I think 15 spends too much time on TNH and PNH, (and Kilo's laudable bio ;). Better to say that all the discussions were held in the open and can be read at (link), as well as the bit about the goals and the developer's broad tastes in fiction.

#377 ::: Buddha Buck ::: (view all by) ::: May 14, 2015, 10:56 AM:

[DECLINED NOMINATIONS]

I've seen folks earlier talk about the chosen finalists as if there was some inherent order. I'd like to stress that there isn't, in my opinion, an order to the finalists (or to the nominated works in general), there's only a "not eliminated yet". So proposals which basically state "generate the top N (>5), take the top 5, and work down the list if a nomination is declined/ineligible" are, in my opinion, fundamentally flawed because there isn't a "top 5" and the a list to work down. Similarly with statements about #1-4 of the finalists have property A and #5 has property B. As such, I disagree with the parenthetical comment in option 2


[7] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[0] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[3] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[3] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[3] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[7] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[8] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[8] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#378 ::: J Thomas ::: (view all by) ::: May 14, 2015, 11:00 AM:

@290 Walter Daniels

The "Traditional (Big) 5 Publishers" all live/work in NYC. That means they all attend pretty much the same parties, know each other, etc. As a result, in an area with 8.5 Million people, you have 1,000 (at maximum), who can "informally agree" that "Author X, of Publisher Y, should win." At worst it's 5 agreed on. Giving the "group/slate") 200 to 500 assured votes.

That's a specific proposal, that could be somewhat testable.

We have the voting data for 1984. I assume that the big deal is novels (where the publishers would win if their last-year's wins keep selling, and for reprints, and having authors in their stables with Hugos could help some).

We could look at the novels from 1984, and measure correlations among ballots. Votes for winners by employees of one publisher would tend to favor just that publisher. We could get a sense how important that was, in that one year.

It would be pretty easy to see that for winners. With some effort it could be expanded to losers also -- see how much people voted for just one publisher. That requires finding the publishers for all the losers, which is a bigger effort.

We could do the same for later years if the anonymized data becomes available.

I will do this someday in my copious spare time. Right now my spare time goes into some other things that are higher priority for me and more time-critical, but I will put it on the list.


On rereading, it looks like you're saying that publisher employees don't just vote for their boss's products, but instead that they talk to each other and form a consensus about what's good, and then they tend to vote for the same things regardless who wrote them. That is much harder to test for. If you think there are 200-500 people like that, an average of 40 to 100 people working for each publisher and paying for Worldcon supporting memberships or better, I tend to have my doubts. Anyway there are more fans than that who go to a lot of conventions and discuss SF with each other. They don't slate-vote because they disagree a lot.

#379 ::: P J Evans ::: (view all by) ::: May 14, 2015, 11:09 AM:

348
They're all the same thing: putting it on your list more than once may be enthusiasm, but it shouldn't get you more say. And it's easier to game than the current system. (Think of this year, and imagine if they'd done that instead, with a system where each appearance of a name on a list counts as a separate name. Let's not do it that way.)

#380 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 11:09 AM:

Supplementary vote on [DECLINED NOMINATIONS]

[10] Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

....

As far as I can tell, the name issue is now resolved, with a consensus of "just call it the Hugo Nominations System, and if you want to refer to the discussions where it was created, SDV-LPE is OK too."

The tiebreaker issue is resolved to my satisfaction, though possibly others may disagree. Using points from all previous rounds will break over 99.9% of significant ties; in the less than 0.1% of cases remaining, eliminating both is an acceptable answer.

So as soon as we've got the declined nominations question resolved, we're ready to finish up the language and make the proposal formally. I think we should also have an FAQ, but somehow keep it dynamic, not freeze it and attach it to the formal proposal.

There are several other associated proposals that have been mentioned (aside from the stuff with declined nominations):

1. Remove the 5% cutoff.
2. Remove the 5-per-category-per-ballot limit.
3. Allow multiple votes for the same work, so people can tune how their points are distributed.

I weakly favor 1 and 2, and weakly oppose 3. I think that 1 and 2 should be proposals if they aren't already. With 2, I think there should still be some high cutoff, just to prevent annoying spam — 20 should be plenty.

#381 ::: Brian Gibbons ::: (view all by) ::: May 14, 2015, 11:14 AM:

Keith "Kilo" Watt @307:

That does make the point of disagreement a bit more clear.

You're saying that the Constitution doesn't say anything about what to do with the final ballot after a nominee declines or is ineligible, so if there's a gap left, the administrator can do anything he wants to fill it.

I'm saying that 3.8.1 defines what works can be listed on the final ballot, so if an administrator decides to issue a new version of the final ballot, they're simply using 3.8.1 to determine "the five eligible nominees receiving the most nominations" (with an updated list of 'eligible nominees') and coming up with a new final ballot according to those rules. The Hugo administrator can choose to leave the ballot alone, but the only way he can issue a new ballot is by using the rules that say what nominees go on the final ballot.

So, under your interpretation, changing 3.8.1 to 3.8.8 doesn't matter. If there's a decline or ineligibility, the administrator can still do whatever he wants to fill in the gap.

Under mine, changing the system to 3.8.8 changes everything, because it now means that if the administrator wants to issue a new final ballot after a decline/ineligibility, the only way he can do so is by following the directions that 3.8.8 gives as to how the nominees on the final ballot are determined (which means running the elimination procedure again, and potentially kicking someone off the ballot who was on it before), and that if we don't consider that an acceptable solution, then we need to include a clause saying so.

This means that, as I see it, "Let the admins decide" is a null option, because it assumes they have powers which they actually do not.

I have zero experience of WSFS rules interpretation or Hugo administration, so perhaps someone with more experience in that area might give their opinion as to which is correct.

It should be noted that while the number of withdrawals this year is unusual, that the ballot needed to be adjusted is not. This isn't some bizarre edge case that hardly ever happens. Six out of the last seven Hugos have had nominees withdraw or declared ineligible. Each of the last three years before this one have had three separate categories that needed to be adjusted, including several cases in which the nominators could not have predicted this and were entirely blameless (e.g., in 2012, Game of Thrones Season 1 was nominated in BDP Long Form and they chose to accept the nomination, which meant the three episodes nominated in BDP Short Form were declared ineligible; in 2013, the nomination for Game of Thrones Season 2 in BDP Long Form was declined, in favor of having a single episode nominated in BDP Short Form).

There is a very good chance that the first year that this is put into effect, some nominee will withdraw or be declared ineligible. Unless it is clear how the SDV-LPE system would handle this situation, I think it's likely that it will be seen as flawed and insufficiently detailed to deal with the nomination process.

#382 ::: J Thomas ::: (view all by) ::: May 14, 2015, 11:17 AM:

@375 albatross

I also don't see why kicking the can down the road makes it more likely to be solved well later--the worst case is to have the admins making the decision after the votes have been cast and the situaiton has come up, and that's an awful situation to leave them in.

Welcome to the world of politics.

One vital reason to kick the can down the road to the business meeting is that we are fundamentally disagreed about the issue and (at least I) see no reason to expect a consensus from further argument.

It looks to me like RME is gaining ground, but if it gets a solid majority with a minority who hate it, that isn't good. If we do get a consensus without a whole lot more argument, then great!

It might likely make sense for the business meeting to kick it down the road to later business meetings or to the administrators, if they can't agree quickly. They have limited time to argue. If we disagree they might easily disagree too. I'd rather see the voting system pass with this issue left to the administrators, than see it fail either pre-nomination or at the business meeting.

The administrators will probably be as good as anybody else at figuring out what to do. They have two years to think about it, and then probably more years before it turns into a crisis. If they can't reach a consensus, every year they have a leader who can decide it for that year so they can get on with their work.

The danger is mostly that they might decide it isn't a problem, or forget about it, and then at some unpredictable time it comes back and bites them. I don't know how likely that is. Right now the argument is getting in the way of getting anything else done.

I agree that it would definitely be better to agree on a great approach and go with it. But if in fact we can't do that, we do better to kick the can down the road rather than pick something a lot of people think is unfair and try to run with that.

#383 ::: Shane ::: (view all by) ::: May 14, 2015, 11:26 AM:

[DECLINED NOMINATIONS]
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary.

[0] for all other options.


Albatross @ 375 I agree, especially about not kicking the can down the road. Except I think re-running will rarely be different and when it is the difference is important.

Say 200 Making Light readers nominate two works, "Bitten Plums, Collected Poetry" and "Idumea's Favourite Tribute and Cookbook." Except I don't nominate the first because Ick, plums. You'll end up with the cookbook at 200 points and Bitten Plums eliminated at 99.5.

Say further Idumea's cookbook is nominated, along with four other works that have around 150 nominations each. If somehow cookbooks are found ineligible, then if we just take number 6, Making Light readers won't have any nominations, and the ML poetry book would be skipped. Whereas if we recounted it would be far the most popular. (which seems better to me).

#384 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 11:26 AM:

Brian Gibbons @381

If I understand you correctly, you're saying that "let the administrator(s) decide" means they'd they'd not be able to do anything but rerun the whole ballot, and potentially un-nominate something already nominated? If that is the case, then my vote above would change from "8" for let the admins decide, to "0". Because whether or not that is technically fair, emotionally it's horribly unfair.

If I'm misunderstanding you (or if you're misunderstanding the situation), then my vote stands. But given the choice between a) "rerun the whole vote" and either of b)"rerun and merge" or c)"add the last eliminated", I'm emphatically against a and for either b or c (though I prefer b).

#385 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 11:29 AM:

The dreaded Internal Server Error has struck again.

#386 ::: Yarrow ::: (view all by) ::: May 14, 2015, 11:33 AM:

Brian Gibbons @ 381: I'm saying that 3.8.1 defines what works can be listed on the final ballot, so if an administrator decides to issue a new version of the final ballot, they're simply using 3.8.1 to determine "the five eligible nominees receiving the most nominations" (with an updated list of 'eligible nominees')

This convinces me. I'd now say that yes (alas!) we need to specify what to do in the case of ineligible or declined nominations.

And the example of the Game of Thrones, where either the season or the episode(s) were going to be declared ineligible, but the nominators could not know which -- that has convinced me that rerun-and-merge is the best option.

The following is the libretto of my changed tune:

[0] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[5] 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[5] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[1] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[1] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[5] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[1] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[1] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#387 ::: J Thomas ::: (view all by) ::: May 14, 2015, 11:38 AM:

@380

As far as I can tell, the name issue is now resolved, with a consensus of "just call it the Hugo Nominations System, and if you want to refer to the discussions where it was created, SDV-LPE is OK too."

Agreed.

The tiebreaker issue is resolved to my satisfaction, though possibly others may disagree. Using points from all previous rounds will break over 99.9% of significant ties; in the less than 0.1% of cases remaining, eliminating both is an acceptable answer.

Say you go back five steps and one of them is ahead on points. But from six steps back to the beginning, the other is ahead on points.

I don't see that this is much better than nomination date to break a tie on. If they have the same number of votes and the same number of points when you're choosing between them, eliminate them both or keep them both.

But it isn't a giant issue that overrides everything else.

I think we should also have an FAQ, but somehow keep it dynamic, not freeze it and attach it to the formal proposal.

Yes. A note with the proposal might include a link to one or more FAQs?

1. Remove the 5% cutoff.

Separate proposal. This one works either way.

2. Remove the 5-per-category-per-ballot limit.

Separate proposal. This one works either way.

3. Allow multiple votes for the same work, so people can tune how their points are distributed.

It would change the results of this kind of voting, I don't know just how. I think it would let a smaller voting block win one nomination. But if you count it as five votes but 1/5 of a point, or vice versa, I don't know what to expect. If it's set up so a smaller voting block wins, then an existing slate could split into 5 groups that each nominates one item 5 times, and be just as good as a slate. I definitely don't want that. Since it has a big effect, and the effect varies by details of how the voting system handles it, maybe it can't be done as a separate proposal.

#388 ::: Brian Gibbons ::: (view all by) ::: May 14, 2015, 11:45 AM:

Cally Soukup @384: If I understand you correctly, you're saying that "let the administrator(s) decide" means they'd they'd not be able to do anything but rerun the whole ballot, and potentially un-nominate something already nominated?

Yes. It's entirely possible I'm wrong about the rules (and hey, if someone wants to light up the Kevin Standlee signal, I'm happy to be slapped down if I am), but the way I'm reading them, the Hugo administrator has no obligation to issue a new version of the final ballot after someone withdraws. However, if they do decide to do so, they can't just decide on their own who goes on the new version of the final ballot, they're essentially just following 3.8.1 which tells them who goes on the final ballot.

If you replace 3.8.1 with a new set of rules for determining who goes on the final ballot, then that also changes the rules for what a Hugo administrator needs to do if they want to issue a new version of the final ballot. They don't have any discretion beyond choosing whether or not to issue a new version; if they do decide to issue a new version of the final ballot, the only way to determine what goes on it is the procedure in 3.8.8 (which could potentially lead to nominees who were on the first version not appearing on the new version).

#389 ::: colin roald ::: (view all by) ::: May 14, 2015, 11:50 AM:

I find this proposal really interesting. It seems compelling to me as a way to reduce the power of slates while having minimal effect on the way the Hugos have always run.

I've taken a stab at rewriting the Simple Explanation to be clearer for nontechnical people or just ones who aren't really paying attention. But since I haven't participated in the discussion til now, it didn't feel right to make an edit as big as this to the Google Doc. If you guys think it's improvement, feel free to use it.

The most important change, I think, is clearing up some of the vocabulary. "Nominating" works to be "nominated" for a Hugo is confusing. I've fixed things to always refer to "proposed works" until the final ranking is established, at which point some of them become "nominated" works.

SIMPLE EXPLANATION

Least Popular Elimination (formally called "single divisible vote with least popular eliminated" or SDV-LPE for short) is straightforward and fair.

The goal is to allow people to simply nominate the list of works in each category that they feel would be deserving of a Hugo, very much as they always have. The procedure for a nominator remains exactly as it has been: in each category you can submit a list of up to five unranked “proposed works" for nomination. From the collection of all nomination ballots, the five works representing the widest selection of nominators will be selected to appear as the nominated candidates on the final Hugo ballot.

The procedure for ranking proposed works by the Hugo Administrators is the only thing being altered.

- Each nominator is given 60 “nominating points” for each category. (These are assigned transparently during the counting process. The nominators themselves never have to think about them.)
- Those points are divided equally among all proposed works. If you propose one work, it gets all 60 of your points. If you propose two, they initially get 30 each; if you propose five, they initially get 12 each.
- All the points for each work from all the ballots submitted are added together, and the two works that got the least number of points are compared with each other. The least popular is eliminated. (To account for possible ties, the rules for comparing are somewhat technical. See [reference] for details.)
- The eliminated work is “erased” from all submitted nominating ballots and the point totals for all remaining works are recalculated as if the eliminated work had never been present. For instance, if you had proposed “Dune”, “Tau Zero”, and “Ubik” on your ballot, and it was determined that “Ubik” was least popular and eliminated, then your 60 nominating points are reassigned with 30 to “Dune” and 30 to “Tau Zero”.
- We start over for the next round, and repeat the process until five works remain, which become the nominated works.

This means you have no penalty for “wasting” a vote by proposing an obscure work. If it turns out not to be popular enough to be nominated, your nominating ballot is recalculated exactly as if you hadn’t mentioned it.

Since points are reassigned as works are eliminated, they become concentrated in the final nominated list. If only one proposed work from a nominator survives, then it receives all 60 of your points in the tally, while if two survive, they are each receiving 30 of your points. It is therefore difficult for a slate pushed by a minority of nominators to take more than a minority of the nominations.

#390 ::: colin roald ::: (view all by) ::: May 14, 2015, 11:56 AM:

Brian@62:
The goal is to honor and reward excellence, not to feel fair or make as many people happy as possible.

Actually, I think "make as many people [sort of] happy as possible" *is* the goal, since it's the opposite of "let a minority push through a slate that everyone else has to live with."

#391 ::: P J Evans ::: (view all by) ::: May 14, 2015, 11:57 AM:

348
Or put it this way: the nominees are determined not by how many times they're named, but by how many people name them. So: one person, one time.

#392 ::: Yarrow ::: (view all by) ::: May 14, 2015, 12:00 PM:

Jameson Quinn @ 380:, J Thomas @ 387: 3. Allow multiple votes for the same work, so people can tune how their points are distributed.

Like Jameson, I'm mildly against this. (I see it as requiring a more complicated explanation.)

Requiring unique works on the same ballot allows (only) the following distributions of points:

1) Alpha, Beta, Gamma, Delta, Epsilon - 0.20 point each
2) Alpha, Beta, Gamma, Delta - 0.25 points each
3) Alpha, Beta, Gamma - 0.3333... points each
4) Alpha, Beta - 0.5 points each
5) Alpha - 1.0 point

Allowing multiple mentions (with a maximum of five total mentions) allows also point distributions like

a) Alpha 0.6 points, Beta 0.4 points (Alpha Alpha Alpha Beta Beta)
b) Alpha 0.6 points, Beta 0.2 points, Gamma 0.2 points (Alpha Alpha Alpha Beta Gamma)
c) Alpha 0.8 points, Beta 0.2 points (Alpha Alpha Alpha Alpha Beta)

(a) is hardly different from (4), while (b) and (c) diverge more definitely from the single-mention distributions. I'm not seeing the harm -- but neither am I seeing any great advantage for the nominators.

#393 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 12:04 PM:

All: I'm on the road, so will read the latest posts in detail this afternoon. I did want to say, though, that based on the previous arguments, I'm fairly strongly opposed to giving 60 points to be divided up just to avoid fractions. Also, as a result of our discussions in the earlier thread, I'm opposed to having voters rank their choices, which is what the AAABC idea does. The latter is against the principles we worked out for what the nomination process should be -- and I still strongly feel those principles are valid.

Kilo

#394 ::: Kevin Standlee ::: (view all by) ::: May 14, 2015, 12:11 PM:

Business Meeting Administration Request:

If your discussion results in you not being able to settle upon a single proposal, but instead on two variations of the same general process, I suggest you submit them both, in this way:

1. Main Proposal (whatever you perceive as having the most support)

2. Variation submitted as an Amendment by Substitution for the Main Proposal.

It will then be the task of the Preliminary Business Meeting to decide (in effect) whether to consider version 1 or version 2, where version 2 is a complete replacement for version 1.

I recommend this for any cases where you have heavily overlapping proposals that can't easily be reconciled by piecemeal amendments. Also, this gets around a mechanical issue with the WSFS Standing Rules that normally prohibits what are called "second-order" amendments. (I can explain the parliamentary neepery if anyone wants me to do so.)

#395 ::: Cadbury Moose ::: (view all by) ::: May 14, 2015, 12:14 PM:

This moose agrees with Shane at #383 now I've thought about it some more.

Reasons:

1) The nomination process must be nailed down tight, be easy to understand, and seen to be done fairly.

2) There should be no scope for anyone to get trampled on due to voting reruns triggered by a withdrawal.

3) There should be no "human judgement" involved that could possibly bias the result.

So:

1) Open nomination period. (Fandom gets busy nominating favourite stuff)

2) Nominations close.

3) Ballots are collected up, validated and entered into the system.

4) Works nominated are checked for elegibility and correct category, invalid ones are dropped or moved as appropriate.

5) Votes are then processed by SDV-LPE to produce the initial "ranked list", the top five (or whatever) are double-checked for elegibility (just in case), and then frozen as the first cut of nominees.

6) Nominees are notified and asked if they are willing to appear on the final ballot.

7) Once they have all answered, any declined nominations are removed from the pool and the SDV-LPE process rerun to identify potential replacements. The result of that run is matched with the accepted nominations and new nominees that make the 'cut' are merged in and notified (as at 6, above). Repeat as necessary until the nomination pool has the desired number of members.

8) Final check for errors, if required, then announce the nominees, prepare the ballot papers and send off for printing.

9) Do Hugo ballot, as per usual, award rockets.

10) Publish full nomination details (suitably anonymised) but with all the working shown. (i.e. elegibility eliminations, the initial SDV-LPE round, any withdrawn nominations (declines), and subsequent SDV-LPE rounds to select replacement nominees.)

11) Make the (anonymised) ballot data available for independent scrutiny if required. (That way, you've got the data for testing alternative processes in the future if anything needs to be adjusted to handle edge-cases or unforseen problems.)

Does that seem reasonable/make sense?

#396 ::: Aan ::: (view all by) ::: May 14, 2015, 12:18 PM:

J Thomas @ 382: It looks to me like RME is gaining ground, but if it gets a solid majority with a minority who hate it, that isn't good. If we do get a consensus without a whole lot more argument, then great!

Based on the voting so far, I don't think there's actually all that much hate for it. There's a bunch of people, myself included, who prefer options 2 and/or 6 (in my case mostly because I dislike the quite-rare-but-will-happen-at-some-point 7/8-work ballot which can result from RME), but everyone's giving at least some points for RME, with the sole exception of (if I tabulated correctly), Duncan, who was all for letting the admins decide (but that's presumably presuming that the admins _can_ decide, where I also follow Brian's interpretation that they can't).

I don't have any hope that consensus will emerge for 2 or 6 (too many 0 point votes for that), so RME seems like a good-enough middle ground to get behind.

#397 ::: Seth Gordon ::: (view all by) ::: May 14, 2015, 12:19 PM:

Allowing someone to mention a work multiple times on the same ballot, or letting them weight their nomination choices, would enable some interesting strategic-voting possibilities. Imagine a bloc of Enthusiastic Marmots submitting a hundred ballots that say “Alpha, Alpha, Alpha, Beta, Gamma”. Or imagine a voter scratching their head and agonizing over the question “I like Delta better than Epsilon, but do I like it twice as much?”

I would rather keep it simple, conform to existing practice, and make all the choices equally weighted.

#398 ::: colin roald ::: (view all by) ::: May 14, 2015, 12:22 PM:

CHip @102:
[ELIMINATION]: I'm unclear why the bottom two nominees by points should be compared for number of ballots instead of doing a straight elimination.

Comparing by number of ballots actually increases the slate resistance. Suppose we have a year with a slate with ten nominators all proposing the same five works, and fifty other nominators proposing a wide scatter of things. Consider the point when all the no-hopers have been eliminated. The scatter nominators have all probably had several of their proposed works eliminated, so they now only have a few works left on their ballots. Using the system where there are 60 nominating points per nominator, that means each of their proposed works is probably getting 20 or 30 points, while the slate nominators are still giving only 12 points each to all five of their proposed works. So when we reach the point where we're comparing the bottom slate-nominated work against a non-slate work, the slate work has 120 points from 10 nominators, but the non-slate work has 120 points from maybe four nominators, and therefore is preferred.

#399 ::: viktor ::: (view all by) ::: May 14, 2015, 12:34 PM:

391 et al

Yes. Under the current system. Believe me, I understand the current process.

With this proposal, however, all that changes. My ballot will get chopped up into predetermined fractions. This has never happened before. I'm suggesting that if my ballot is to be chopped up, I should be able to decide how fine the chopping. Rule 3.8.4 makes sense under the current process, but not under this proposal.

That's what I'm getting at.

The explanations of Not Done and Really Not Done are unsatisfactory to me.

Irony alert:
Please note that folks right here on this board are being asked to weight preferences on an issue.

#400 ::: colin roald ::: (view all by) ::: May 14, 2015, 12:39 PM:

J Thomas @117
If you want to maximize the chance that one particular work you care about more than anything else will win, then you should just vote for that one.

Actually, no, you can still vote for whatever you want. Suppose you love "The Left Hand of Darkness" and want to be sure it's nominated. But you really love the novelization of the movie version of "John Carter" even more, so you put it on your ballot just in case.

It turns out you were the only person who even considered "John Carter", so it's eliminated first. It's erased from you ballot, and now all of your nominating points go just to "Left Hand of Darkness" as if you'd never had that silly idea about "John Carter" in the first place. The same goes for any work that doesn't end up on the shortlist: go ahead and throw them on your ballot. If other people do too, then great. But if they're not popular enough to make it to the top, no harm done. Your least popular works will be eliminated and all your nominating points roll forward to the proposals you share with the most other people.

#401 ::: albatross ::: (view all by) ::: May 14, 2015, 12:41 PM:

Budda Buck:

The simplest way to get a ranking from SDV-PE is to rank the candidates by how many rounds they survived. If we ran to completion, Alice would have won. If we selected two, we'd have Alice and Bob. If we selected three, we'd have Alice, Bob, Carol, and Dave (because of a tie). And so on.

This falls naturally out of the vote counting process--at the end of each round of counting, you have a list of remaining candidates. It's easy enough to write the program to return that list for all the rounds where you had fewer than (say) 10 candidates.

There are other ways to get rankings (like counting points or number of ballots for the last 10 surviving candidates), but I think they're needlessly complicated.

#402 ::: Brian Gibbons ::: (view all by) ::: May 14, 2015, 12:43 PM:

I think the "Each nominator is given 60 nominating points" structure is the wrong way to describe this, because it almost begs for a discussion of "Well, what if I want to allocate my points non-uniformly?", which I think should be a completely separate discussion. (That's 3.7.1; this discussion is about 3.8.1, and combining the two discussions doesn't add anything.)

Indeed, I think the entire description of a voter having points that they decide to distribute among works is a poor fit. SDV-LPE doesn't change anything about the way a member nominates; it changes the way the final ballot is produced. The description should be coming from that perspective, rather than using "you distribute your vote" and "you divide your vote" language. You aren't doing anything different under SDV-LPE than you were before.

Something like:

* This system does not change the way a member submits their nominations. Assuming no other proposal changes that system, a member continues to simply submit a list of five unranked nominations in each category.

* In order to determine which nominated works appears on the final ballot, the Hugo administrator goes through a multi-round process in which he assigns points to nominees based on how many ballots they appear on and how many nominees on those ballots have not yet been eliminated.

* If, for example, a ballot has five nominees that have not yet been eliminated, the administrator gives each of those nominees a fifth of a point; if a ballot has four nominees that have not yet been eliminated, each of those nominees is given a quarter of a point, and so on, down to ballots in which only one nominee has not yet been eliminated, in which case that nominee gets one point.

* In each round of this process, the administrator adds up the points that each nominee has received that round, compares the two nominees with the least number of points to each other, and eliminates the nominee which appears on the fewer number of ballots.

* The administrator repeats this process again each round until he has eliminated all except the final five (or more, in the case of a tie) nominees. Each round, point totals start afresh and the points a nominee gains from a ballot are based on how many nominees have not yet been eliminated from that ballot. For example, if a ballot has five nominees that have not yet been eliminated, then each of those nominees are given a fifth of a point; when one of those nominees is eliminated, the four surviving nominees will each receive a quarter of a point instead.

#403 ::: Laertes ::: (view all by) ::: May 14, 2015, 01:02 PM:

viktok @399: "I'm suggesting that if my ballot is to be chopped up, I should be able to decide how fine the chopping."

Right. And at 256, I was suggesting that I should be able to decide how fine the chopping is too. I like irrational numbers, so why can't I split my vote into pieces sized 1/pi for Bob and 1-(1/pi) for Alice?

But I'm not finished yet. Because in the event that Alice is eliminated, I'd like for 2/3 of my vote for her to go to Carol instead, while the remaining 1/3 of it goes to Bob. There's this whole complicated thing where I like Carol's work a lot, but I think that if she were on the ballot along with Alice's work that I prefer, she'd draw primarily from Alice's voter base, so I want to nominate only Alice. But should Alice be eliminated, then my objections to Carol being on the ballot vanish with her and I'd like to swing my support to her.

Well, most of it. But some to Bob because I really like Bob's work.

If my votes are to be redistrbuted when one of my picks is eliminated, I should be able to decide how it's redistributed.

But at about that point I realize that you were correct @262, and I'm being silly. These additional levels of detail are fun to describe, but don't really deliver any value besides their inherent beauty and the gratification I get out of promoting them. Having my support be evenly distributed among the works I've chosen to nominate, and reshuffled to remain even when some are struck, isn't a perfect expression of my preferences, but it's simple and does the job.

#404 ::: Laertes ::: (view all by) ::: May 14, 2015, 01:03 PM:

viktor: Sorry about botching your name. I noticed it literally as I hit the POST button, and sat there hoping for a server error. No such luck.

#405 ::: P J Evans ::: (view all by) ::: May 14, 2015, 01:12 PM:

377 et seq

As it's been done for the last mumble years, the admins generate a list of 10 or more (actually, they count all the names in a category). So it should not be necessary to re-run the count.

#406 ::: P J Evans ::: (view all by) ::: May 14, 2015, 01:20 PM:

viktor, based on your comment, I don't think you quite understand the system as it is now. Nominees are not decided based on how enthusiastic the people naming them are, but on how many people name them: that's why there's a rule that says if you put the same thing in a category more than once, it counts as once only. Otherwise, it invites gaming the system, which we don't want.
Think of it as the difference between number of likes, and number of people liking. What we're after is liked-by-the-largest-number-of-people, not has-the-most-likes.

(This isn't the preferential-ballot part of deciding the Hugos: this is the find-out-what's-popular part. No ranking, no strategic voting, only 'what did you think was best from last year'.)

#407 ::: Yarrow ::: (view all by) ::: May 14, 2015, 01:23 PM:

Laertes @ 404: Having my support be evenly distributed among the works I've chosen to nominate, and reshuffled to remain even when some are struck, isn't a perfect expression of my preferences, but it's simple and does the job.

Hear hear!

#408 ::: junego ::: (view all by) ::: May 14, 2015, 01:24 PM:

Jameson Quinn @ 380

1. Remove the 5% cutoff.

This is already on the meeting agenda. We don't need to address it.

2. Remove the 5-per-category-per-ballot limit.

This requires a separate constitutional amendments and should be a separate discussion/proposal. Our only business might be to include language in our proposal that allows for this change.

3. Allow multiple votes for the same work, so people can tune how their points are distributed.

Again, this requires another constitutional amendment. My sense of previous discussion and my own opinion is that the nominations shouldn't be weighted.

#409 ::: J Thomas ::: (view all by) ::: May 14, 2015, 01:28 PM:

@400 colin roald

J Thomas @117
"If you want to maximize the chance that one particular work you care about more than anything else will win, then you should just vote for that one."

Actually, no, you can still vote for whatever you want.

....

Your least popular works will be eliminated and all your nominating points roll forward to the proposals you share with the most other people.

That's how it ought to be, and that's mostly how it is.

But there's a possibility it will result in the one you like best being eliminated.

It goes like this: Unknown to you, a bunch of people have bullet-voted for the one you like less. And a bunch of people have voted for the one you like better along with popular works. You voted for both of them, so your vote counts for them both equally. And unknown to you, something that is more popular than either of them also has a bunch of shared votes.

The one you like better gets chosen early to consider for elimination. So does the very popular one. It accidentally turns out that the extremely popular one gets matched up against the one you like, and eliminates it. Your other choice becomes a finalist.

Again, this is possible but unlikely. More likely when the one you want more is the less popular one than when it's the popular one. Also, there's a chance that when you give half a point to the more popular choice you are in fact *raising* its score and making it less likely to be matched against your favorite.

But in almost any unranked system, adding a vote for something else reduces the chance for your favorite by some small amount. If you hate for your second choice to win and not your first choice, don't vote for the second choice. It isn't good enough for you.

It's completely safe for you to vote for it, though, if you are completely sure that nobody else will vote for it.

#410 ::: Duncan J Macdonald ::: (view all by) ::: May 14, 2015, 01:56 PM:

Aan @396:

Duncan also voted 10 on "Create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide."

I'm more partial to "promote 6" than I am to "run it again".

The first run-through produces an ordered list of 1 - n. If there are withdrawals or late noticed ineligibilities, push those out of the list, apply pressure to the bottom, and squeeze the spaces out. Contact any new members of the top five club, and iterate as necessary until top five remain unchanged.

#411 ::: Annie Y ::: (view all by) ::: May 14, 2015, 01:56 PM:

@350 J Thomas
This is a minor picky point, but the issue is not how many ballots there are, but how many works that people might nominate.

Yes and no. My point is that with 100 ballots, the lack of the under 5% restriction will put on the ballot works with 4 nominations (or even 1 in slow years or categories). You increase the number of ballots, your 5% are a lot higher (50 votes on 1000). And you probably get more works nominated - so chances of someone getting 1 nomination and making it on the ballot is very small. So you do not really need the artificial protection. (this had always been my reading on why we have the rule)

#412 ::: junego ::: (view all by) ::: May 14, 2015, 02:01 PM:

Brian Gibbons @ 402
I think the "Each nominator is given 60 nominating points" structure is the wrong way to describe this, because it almost begs for a discussion of "Well, what if I want to allocate my points non-uniformly?", which I think should be a completely separate discussion. (That's 3.7.1; this discussion is about 3.8.1, and combining the two discussions doesn't add anything.)

I'm the one who originally proposed this idea (except with 100 pts). I later unproposed it after this and another objection were raised (although I will note that only having one point hasn't squashed the 'discussion' you describe...see viktor). I think colin is the only one still backing the proposal. Unless there's a sudden groundswell of support, I consider it doa.

Indeed, I think the entire description of a voter having points that they decide to distribute among works is a poor fit. SDV-LPE doesn't change anything about the way a member nominates; it changes the way the final ballot is produced. The description should be coming from that perspective, rather than using "you distribute your vote" and "you divide your vote" language. You aren't doing anything different under SDV-LPE than you were before.

YES! Excellent point. I totally agree and have stated similar upthread. I also think we need to stop slipping into "vote" language. We are polling/suggesting/nominating. (The constitution doesn't use "vote" wrt nominations and does use "vote" for the final, well, vote.)

#413 ::: Pfusand ::: (view all by) ::: May 14, 2015, 02:37 PM:

Why have the Hugo administrator wait until a withdrawal or disqualification before rerunning the election?

Run the election, and record the top six (or seven) finishers. Eliminate one of the top five finishers and rerun. Note the new top five (or six (for ties)) finishers. Repeat for each of the other top five. Are there any surprises? No? Then we're done.

Yes? Then there is the work to immediately check for validity, and the individual to call first to see if tey accept the nomination. Should this person be told that this is a linchpin nomination? Oh, what a good question.

#414 ::: colin roald ::: (view all by) ::: May 14, 2015, 02:59 PM:

J Thomas @409:
But in almost any unranked system, adding a vote for something else reduces the chance for your favorite by some small amount. If you hate for your second choice to win and not your first choice, don't vote for the second choice. It isn't good enough for you.

Okay, yes. That's true of the existing system, and it's also true of the LPE system. I don't see any scenario along these lines where LPE makes anything worse than it would have been under the current nomination system.

I have trouble thinking of scenarios where I'd be truly tempted to write down a second work -- call it Clifford -- in addition to my first choice -- call it Algis -- if I thought I'd actually be unhappy to see Clifford on the Hugo ballot. We're talking about the nomination process, right? Most nominations are a gamble, and the amount you can actually influence things by nominating only one work instead of five is pretty modest.

On the other hand, if I wanted to nominate five works that all seemed like longshots, and I didn't know which of the five had the best chance ahead of time, I'd definitely prefer a LPE ballot to a conventional one. Under LPE I could put down all five longshots, and my preferences would automatically roll forward to the one where it does the most good.

#415 ::: Darth Paradox ::: (view all by) ::: May 14, 2015, 03:08 PM:

[DECLINED NOMINATIONS]

[2] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[3] 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[2] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[5] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[5] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[9] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[5] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[2] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#416 ::: junego ::: (view all by) ::: May 14, 2015, 03:46 PM:

Pfusand @ 413
Why have the Hugo administrator wait until a withdrawal or disqualification before rerunning the election?
Run the election, and record the top six (or seven) finishers. Eliminate one of the top five finishers and rerun. Note the new top five (or six (for ties)) finishers. Repeat for each of the other top five. Are there any surprises? No? Then we're done.

This is a good procedural idea that the administrators could use and may very well prevent some potential grief.

The problem comes in when changes are made *after* the finalists have been notified. What if someone changes their mind after accepting and after the finalists are made public? What if a work is disqualified because of an issue not known or caught in time? What about the Game of Thrones example above?

Those are the cases we're trying to set rules for. The new system, unlike the old, has the property that such withdrawals/eliminations could change who the remaining finalists are and unnominating a finalist after notification because of a tally rerun would be all kinds of not good.


#417 ::: junego ::: (view all by) ::: May 14, 2015, 03:57 PM:

And I forgot an important point, what if more than 1 finalst in a category is removed?

#418 ::: Doire ::: (view all by) ::: May 14, 2015, 04:18 PM:

junego @ 417
Instead of extra 5 runs it's 31 (I think). Would that be a combinatorial function? Too, too many years ago

#419 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 04:46 PM:

viktor@399:
Irony alert: Please note that folks right here on this board are being asked to weight preferences on an issue.

Actually, no irony at all. In fact, it's a good illustration of what we have been talking about. Everyone threw out "nominations" for how they want the issue to be handled. Once all the nominations are on the final ballot -- unranked, I might add -- then we weight our preferences in the final vote. This is precisely how the Hugos are supposed to work and why we want to maintain that under SDV-LPE.

Kilo

#420 ::: colin roald ::: (view all by) ::: May 14, 2015, 04:54 PM:

junego@412:
I think colin is the only one still backing the proposal.

"Backing it" might be a little strong. I still think there's an argument in favour of being able to show people math done with whole numbers, but on the flip side there's the "60? why 60?" reaction that will sometimes need explaining.

In practice, all the arithmetic is going to be done by computers so it hardly matters.

#421 ::: Buddha Buck ::: (view all by) ::: May 14, 2015, 04:57 PM:

Doire @418: Technically it would be a combinatorial function/explosion, but with a small enough N that it's feasible. It's a different matter when you consider all the ways the 15 works on the short-list can decline nominations (32,767 possibilities).

#422 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 05:01 PM:

I've made a tally spreadsheet for the poll. I'm sure I've missed a supplementary vote or two but you can begin to see patterns.

https://docs.google.com/spreadsheets/d/1tZ9eSJ74TT_2A2bMVbuCoNwKJVS3QeV5ShCAap4ipCw/edit?usp=sharing

The current highest average is for RME. But: several people have not voted on the last three options (the ones regarding multiple proposals to be decided on at the BM). One thing that interests me is that the minimax of the last three is 5, so everybody seems to find at least one of those proposals at least minimally acceptable; but the maximin is 1, so each of those three options has at least one person who detests it. This is strange to me, as those three options seem to me to be broadly similar. It also suggests to me that if we could combine the advantages of those three and avoid their disadvantages, we could get consensus on an option in that direction.

#423 ::: Annie Y ::: (view all by) ::: May 14, 2015, 05:04 PM:

#422 ::: Jameson Quinn

OK - I got lost in what the current options are (you all talking too much when I am trying to sleep and then I need to work instead of reading what happened...) :)

In which post are the current options?

#424 ::: Shane ::: (view all by) ::: May 14, 2015, 05:04 PM:

#399:::viktor:::

No, I'm pretty sure you are incorrect. I believe some of the wording has misled you (in which case good, lets clean it up).

The proposal in this thread does not involve weighting of the works within a ballot.

There is a discussion here of weighting but it's one you started. I mildly favour it, btw.

#425 ::: Soon Lee ::: (view all by) ::: May 14, 2015, 05:15 PM:

[DECLINED NOMINATIONS SUPPLEMENTARY]

[7]8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[2]9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[2]10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default

#426 ::: Shane ::: (view all by) ::: May 14, 2015, 05:16 PM:

#420:::Colin Roald:::

1 point vs 60 points

Seems to me the 60 might be useful for explaining the system to some folks, and not needed anywhere else "Ok, if that's not making sense, let's say we counted each ballot as sixty points instead of one. ... [explanation with 60]... It works out the same either way. The designers decided to go with 1 point." To be clear, this would not be in the documentation anywhere, just use it if it helps you in discussion.

#427 ::: Shane ::: (view all by) ::: May 14, 2015, 05:17 PM:

[FAQ / EXPLANATION]
Per colin roald @414 - One of the things that delights me about this system is it gives your point wherever it can do the most good, i.e. to the least popular of your proposed works which your support can actually help. Colin's phrasing might be used verbatim:

"If I wanted to nominate five works that all seemed like longshots, and I didn't know which of the five had the best chance ahead of time, I'd definitely prefer a LPE ballot to a conventional one. Under LPE I could put down all five longshots, and my preferences would automatically roll forward to the one where it does the most good."

#428 ::: P J Evans ::: (view all by) ::: May 14, 2015, 05:21 PM:

I am tempted, when Frisbie can get to his LSI-11 and pull the stuff off the RX01s (and possibly the RK05s), to offer the 1984 version of the program that counted the final ballots.
(RX01: 8-inch floppy, holding slightly less than 256K; RK05: 5 or 10 MB hard disk - the 10MB version is the RK05F, where it's fixed, not removable.)

#429 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 05:46 PM:

Jameson@422:

I had just finished tallying results when your message came through. I'm getting the same totals you are, so I think we've covered everyone. However, you're right that few people expressed an opinion on the last three options (since they were in the original poll, most people likely missed them). For reference, they are:

8. Re-run the counting until there is only one finalist who was not on the original list, and add that one
9. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
10. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.


I believe 5 (RME) and 8 are functionally equivalent, if I'm interpreting 8 correctly.

9 and 10 are -almost- functionally equivalent, I think. Am I correct that the only difference is that 10 is the same as 1, just with a plan to have two more proposals afterwards to deal with the issue? And then 9 is the same thing, but we leave nothing to the adminstrators and just make two nearly-identical versions of the SDV-LPE proposal that includes the two major declined nomination schemes?

If we can agree on a scheme, then I don't think we need worry about 9 or 10. From Kevin's messages, I believe (correct me if I'm wrong, Kevin) that he would strongly prefer 10 to 9 -- separate out as much of the contentious issues as possible.

Kilo

#430 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 05:50 PM:

[DECLINED NOMINATIONS]

All:

Warning: Long philosophical post ahead...

It looks to me like 5 (RME) has the most broad support, with the fewest number of people hating the idea. 1 (let the admins decide) has the next highest support, and also has only a few people who hate the idea. 2 (add the works from the previous round) is nearly tied with 1, but even more people hate the idea.

1 requires us to do nothing, but if we were to go that route, I feel strongly that we should actually go for 10 -- do nothing in the SDV-LPE proposal, but have the two leading schemes (which would actually be 2 and 5) ready for voting.

I think this boils us down to three real possiblities:

2 - Add the work(s) eliminated in the previous round back to the final ballot.

5 - Re-run SDV-LPE without the eliminated work and add any new finalists to the finalists remaining from the first round.

10 - We can't decide. Do nothing, craft a proposal for both 2 and 5, and let the business meeting decide.


Pros and Cons:
In reality, the odds of eliminating someone who used to be on the final ballot as a result of a re-run are very, very small. In fact the odds are extremely high that 2 and 5 will produce identical results. So the stakes aren't very high here. We therefore need to consider two factors: general philosophy and fringe cases.

2 has the advantage of being very simply explained. It's what we do now. Pretty much everyone would accept 2 on face value. 2 says that just because the fans want something on the ballot doesn't mean that the author doesn't have a say -- he can veto the will of the fans, and they have to accept that veto without creating an "alternate history".

However, there is a good question as to whether this is truly fair. 5 maintains that the most fair way is to run SDV-LPE as though we knew that the declined nomination was essentially ineligible. If re-running SDV-LPE somehow would have swapped an original finalist for a new one, that seems odd and counter-intuitive, but only if you try to hold with both "alternate histories" at the same time. The power of 5 is that it neatly blends those two "alternate histories" in such a way that no one from the "original history" is denied, but the results from the "re-run history" are also recognized.

The downside to 5 is that you get results that seem counter-intuitive at a surface level, while with 2 you do not. The downside to 2 is that it is possible that works that might have made the Hugo might have been eliminated before the next-to-last round, and so won't be considered.

The disagreement, it seems to me, therefore boils down to whether you recognize the validity of "alternate histories" or not. You can definitely make a strong case (as 5 does) that the true history is the re-run ballot, since it was run with only eligible works. You can also make a strong case that because 2 does not cause any "cognitive dissonance", then it is the true history that needs to be respected.

So, to boil it down even further, it seems to me that:

2 has the advantage of simplicity, at the expense of compromise between/blending of the two histories.
5 has the advantage of compromise between/belnding of the two histories at the expense of simplicity.

Given the above, which do we value more? Blending the two histories or simplicity?

Thanks,
Kilo

#431 ::: P J Evans ::: (view all by) ::: May 14, 2015, 06:02 PM:

430
I prefer simplicity - it's easier to explain than rerunning the counting.
(and, especially if you're using a computer to count, there's no reason to stop when you get to five or six. Count until you hit the 5% limit, or run out of stuff to count, whichever comes first.)

#432 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 06:10 PM:

So the options are 6th place, RME, or punt. My meta-feeling is that if there are 2 or more strong votes against 6th place or RME, we should punt. I like RME better aesthetically, but am worried that it might make people wrinkle their noses, and stop them from passing anything. But if that's the only reason to prefer 6th place over RME, then punting is the worst of both worlds.

So I guess if I could get everyone here to agree, I'd prefer 6th place, under the theory that 1% probabilities that the BM doesn't understand won't hurt them. But that's not happening. Given that fact, I hope we can get consensus for RME; if not, we have to punt.

It looks as if Duncan J Macdonald, Laertes, and Cally Soukop are the ones who oppose RME. Given that it seems unlikely that we'll get consensus for anything but RME or punt, and if you oppose RME for reasons of complexity you may oppose punting even more, are any of you willing to rethink your stance on RME?

#433 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 06:13 PM:

Note to Duncan J Macdonald, Laertes, and Cally Soukop: even if you don't want to change your mind, a response would be helpful.

Note to Brian Z: if your goal here is to throw sand in the gears, you can make that pretty obvious right now by voting against RME.

#434 ::: Evil Rob ::: (view all by) ::: May 14, 2015, 06:28 PM:

While it may certainly be that I've missed it - I would think you would want to use separate methods for a declined nomination, and a disqualification.

@233 I would agree that bumping the last eliminated up in the case of a declined nomination makes sense.

In the case of a disqualification, I would think that RME is a better plan, since to at least avoid the appearance of impropriety, the work should never have been counted in the first place.

#435 ::: Buddha Buck ::: (view all by) ::: May 14, 2015, 06:38 PM:

[DECLINED NOMINATION]

Keith @430: I'm not sure it's correct to say that extremely unlikely that 2 and 5 will give different results. It's been tested that if one work drops out, the far most likely result is that the re-run will retain the other four works, with one new one. I don't know if it's been checked to see if, in that case, the new one was the last-eliminated in the original run. It could be that 'The Naked Now' was the last eliminated in the first run (and would thus get nominated under 2), but without the declined work, it would be eliminated earlier, leaving 'The Naked Time' as the replacement work. From the point of view of the problem of declined nominations, I don't see this as a problem; one declined/ineligible work was replaced by an eligible work not previously nominated. But it would be a difference between the two options.

If we are concerned about fidelity to the "true history", based solely on eligible works, we'd have picked the "rerun and accept new results" option (4), but it seems like we rejected that because of the logistical nightmare of recalling an accepted finalist slot.

Given the idea of "two histories" (one with "Spock's Brain" accepting the finalist slot, one with "Spock's Brain" declining the finalist slot), we (or at least I) don't want to (a) tell folks that they were a finalist, but now they are not, or (b) have to explain why "The Corbomite Maneuver" was not nominated when, without "Spock's Brain", it "should" have been. It seems more fair (to me, at least) to give a spot to those who would have earned it under either of the "two histories".

The only draw-back I can see is a potential unboundedness coming from multiple declined ballots. "Spock's Brain" turns it down; "The Corbomite Maneuver" and "The Tholian Web" get added, for a 6 work ballot. The Tholians decline the slot, and re-running again yields two more new finalists, for a total of three merged histories and eight finalists -- unless one of those two decline, etc. The chances are exponentially[1] low that it'll happen too many times.

[1] There's a 1/100 chance that eliminating a nominee will add multiple new nominees, and a low chance that the new nominees will decline, yielding a factor of 1/500 (or less) that another rerun will be required. So the chance of N reruns is on the order of (1/500)^N, so, yeah, exponentially low.

#436 ::: CHip ::: (view all by) ::: May 14, 2015, 06:44 PM:

[DECLINED NOMINATIONS]

my votes on handling, based on @325 and @361

option 2 (take last-eliminated): 10
option 5: 10
options 6-7: 5
(various patches to prevent RME from causing somebody to be tossed after we thought they were on)
option 9 (base proposal with supplements for handling declined nominations): 10
all others 0.

#437 ::: Joshua Kronengold ::: (view all by) ::: May 14, 2015, 06:45 PM:

[10] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[8] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[4] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary

6, and 7 are simply wrong, they should be:
[3] 42. Rerun the election without all eliminated nominations to completion (until no candidates remain). Add the last eliminated candidate(s) that did not originally make the ballot.

Regarding the procedural questions, Kevin is [obviously] right that if we have fundamentally two different amendments, we should do the amendment by substitution thing.

For additional supplementary amendments (that add but do not remove anything from the main proposal), we can simply have additional amendments to the main proposal, I believe.

For instance, we could have the main proposal without a way of handling withdrawals, and then an amendment that proposes that if any candidate becomes ineligible after nominee has been notified in that category resulting in four or fewer candidates, that the last eliminated candidate may be added.

And another competing amendment stating that instead, in that case, we should act as in 2 or 42 [42 has less chance than 2 of resulting in too many things on the ballot, but also more chance of one constituency being double-represented by a quirk of fate].

#438 ::: JJ ::: (view all by) ::: May 14, 2015, 06:45 PM:

I've not posted here yet, because I've been reading and thinking through what everyone else has to say.

I would be inclined to agree with Evil Rob at #434. I think RME is really needed in the case of a DQ (especially in the case of 2 items DQ'ed from the same category). I don't think the 6th Place method would be satisfactory in such a case. (I suspect, if we can persuade Sasquan to provide anonymized ballot data, we'll see a serious case of this example this year with Andy Weir's The Martian. I would be very curious to see how RME would affect those ballot results.)

In the case of a Declination, I could be persuaded to go with either 6th Place or RME -- but I think I prefer RME.

#439 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 06:56 PM:

[DECLINED NOMINATIONS]

Jameson's post reminded me that I didn't give any time to option 10 (leave the issue out of the main proposal and craft two separate proposals -- or amendments, referencing Joshua@437 -- for 2 and 5). Apologies for that; I'm trying to give an unbiased (as much as a human can be) summary of our options.

10 has a single, but powerful advantage: If there is disagreement here among those who are working on the system, then there is very likely to be a roughly equal amount of disagreement at the business meeting. If either 2 or 5 is encoded explicitly in the proposal, then there is a small chance that opposition to whichever was included would be enough to sink the whole the proposal.

Alternatively, 10 allows us to encode both options separately. When presenting the amendments, we can even state that a majority of the developers feel that [2 or 5] is the best way to go, but here are the options, and we will let you choose the one you're most comfortable with.

This option isn't quite a punt (as Jameson puts it), since it does allow that developer preference to be expressed, but it maintains the ability to get the main system passed safely.

Kilo

#440 ::: Joshua Kronengold ::: (view all by) ::: May 14, 2015, 07:03 PM:

FWIW, the meeting itself isn't shy about amending proposals, so some disagreement could be handled that way in any case regardless of what we propose.

By the same token, it's worth noting that -new- business is particularly hard (2/3 majority) to add, (but amendments are easy).

#441 ::: Brian Gibbons ::: (view all by) ::: May 14, 2015, 07:07 PM:

Keith "Kilo" Watt@430: 2 has the advantage of being very simply explained. It's what we do now.

Sure, it's what we do now in the simple first-past-the-post nomination process in which "nominee that would have been on the ballot if nominations for the ineligible nominee weren't considered" and "last nominee eliminated before the final five" are one and the same.

It's not what we do on the final ballot. After listing first place, the administrator then goes and (for reasons of completeness and, I suppose, to satisfy people's curiosity) runs the analysis again for each position. Second place isn't "last nominee eliminated before first place", it's "nominee that would have been in first place if we ignore all votes for the actual winner". Third place is the nominee that would have won if all votes for the first two are ignored, and so on.

If there's any group that can handle the concept of alternate histories, I think it's SF fandom.

I am surprised that Options 2 and 5 yield identical results. I would have assumed that, with highly-correlated data, you would end up with the exact opposite. For example, if the most popular Dr. Who episode nominated is ineligible for whatever reason, my assumption is that rerunning the analysis would cause the second-most popular Dr. Who episode to pop up as a nominee, even though that probably would have been eliminated earlier than the final elimination in the first run (due to it mostly appearing on ballots that also had the most popular one).

In any event, I don't believe the admins have the discretion that you believe they have, so if you are going to go with a "just leave it out" concept, I'd suggest making sure that the default is as you believe it to be, if Kevin Standlee or someone else is willing to opine about hypotheticals.

#442 ::: junego ::: (view all by) ::: May 14, 2015, 07:09 PM:

Keith "Kilo" Watt @ 430
2 - Add the work(s) eliminated in the previous round back to the final ballot.
.
5 - Re-run SDV-LPE without the eliminated work and add any new finalists to the finalists remaining from the first round.

I vote for 5.

#443 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 07:17 PM:

Brian G@441

In any event, I don't believe the admins have the discretion that you believe they have, so if you are going to go with a "just leave it out" concept, I'd suggest making sure that the default is as you believe it to be, if Kevin Standlee or someone else is willing to opine about hypotheticals.

Actually, I think that option was replaced by option 10 (not in the main proposal, but with amendments describing each scheme ready to go), so I think the question of how much discretion the admins have is now pretty much moot.

K

#444 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 07:22 PM:

Joshua@440:

A quick procedural question for you...

Which is easier to do, procedure-wise:
- Include one of the two schemes in the main proposal (RME, I'd think, based on the poll) and have an amendment for 6th place ready to discuss if necessary, or,
- Leave the schemes out of the proposal and have an amendment for both ready to dicuss.

THanks,
Kilo

#445 ::: Vicki ::: (view all by) ::: May 14, 2015, 07:27 PM:

Pfusand @413:

Realistically, the administrators will need to give the potential nominees some time to answer the question “do you accept the nomination?” Never mind genuine indecision, someone might not answer the phone immediately, and in fact might take several days to even check their messages before calling back.

Yes, you can set the computers to calculating what-ifs in those days, but any proposal should allow for reasonable turnaround time for accepting/declining a nomination.

#446 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 07:48 PM:

If RME is the one where we rerrun and merge the new results without kicking anybody who was previously nominated off, I'm actually for it. If it's not the rerun/merge one, umm, which is it exactly?
The one I'm against is "rerun and un-nominate the previously nominated", because that's a very cruel thing to do to people who had been notified they were nominated and had accepted.

It's quite possible I messed up and marked the wrong line on my "ballot"; if that's the case, I apologize for the confusion I've caused.

#447 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 08:04 PM:

Hi Cally -

Yep, sounds like you're in favor of RME (option 5 in the original list).

Thanks!
K

#448 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 08:05 PM:

Hi Cally -

Yep, sounds like you're in favor of RME (option 5 in the original list).

Thanks!
K

#449 ::: felice ::: (view all by) ::: May 14, 2015, 08:06 PM:

Cally Soukup @446: "It's quite possible I messed up and marked the wrong line on my "ballot"; if that's the case, I apologize for the confusion I've caused."

No, it's not your mistake - the spreadsheet at https://docs.google.com/spreadsheets/d/1tZ9eSJ74TT_2A2bMVbuCoNwKJVS3QeV5ShCAap4ipCw/edit?usp=sharing doesn't have the correct values from your post #374, and needs to be updated.

#450 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 08:17 PM:

felice: Ahh, thank you. I probably should have tried to find my previous post, but had to run off to deal with dinner. In any case, no harm, no foul!

#451 ::: felice ::: (view all by) ::: May 14, 2015, 08:30 PM:

Keith "Kilo" Watt @439: "10 has a single, but powerful advantage: If there is disagreement here among those who are working on the system, then there is very likely to be a roughly equal amount of disagreement at the business meeting. If either 2 or 5 is encoded explicitly in the proposal, then there is a small chance that opposition to whichever was included would be enough to sink the whole the proposal."

There is the other question of whether the people who oppose RME actually support SDV-LPE at all - participating in this discussion doesn't necessarily mean they'd vote in favour of it, irrespective of what we decide on handling withdrawals.

I'm reluctant to give the Business Meeting any more decisions to make than we have to; it's likely already going to be a nightmare with all the different puppy-inspired proposals, and passing off something too controversial for us to reach agreement on is unlikely to go down well. "This is SDV-LPE: it's good, vote for it" is going to have more chance of success than "Here are a couple of variants on SDV-LPE: which do you think is better?"


Brian Gibbons @441: "I am surprised that Options 2 and 5 yield identical results. I would have assumed that, with highly-correlated data, you would end up with the exact opposite. For example, if the most popular Dr. Who episode nominated is ineligible for whatever reason, my assumption is that rerunning the analysis would cause the second-most popular Dr. Who episode to pop up as a nominee, even though that probably would have been eliminated earlier than the final elimination in the first run (due to it mostly appearing on ballots that also had the most popular one)."

I think you're right; but the testing has been done with 1984 data, when the field was rather smaller and there weren't any awards given for TV episodes (technically they were eligible, but in the same category as movies). I suspect this would be a bigger issue with modern nominations, with fandom more divided into subgroups because it's just not possible to keep up with everything. RME and 6th place will still sometimes (maybe even often) get the same result, but I'd expect them to be different more often.

#452 ::: Annie Y ::: (view all by) ::: May 14, 2015, 09:03 PM:

[DECLINED NOMINATIONS]

After finally catching up with the reading

[8] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[8] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[5] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[0] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[10] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[10] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[10] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#453 ::: Laertes ::: (view all by) ::: May 14, 2015, 09:47 PM:

Jameson Quinn @433: I dislike RME on the grounds that it's complex--it's much simpler to lock down the finishing order and then just move down the list.

When, for instance, a runner in a foot race is later disqualified for some reason, they simply shuffle the other finishers up, this despite the fact that runners interact with one another during the race in complicated ways, and the results with the disqualified runner struck from the final standings isn't necessarily the same as the result would have been had the race been run without her.

But I (think I?) scored RME a 4, by which I meant to say that it's far from my favorite, but it's not a disaster like the 0s.

Short version: If a consensus is forming around RME, please consider me to be a fully paid-up member of that consensus.

#454 ::: Laertes ::: (view all by) ::: May 14, 2015, 09:55 PM:

Also? I tuned in to this effort very late in the game. The heavy lifting of hammering out this really terrific proposal was done entirely without me. If I manage to contribute something that's helpful, great. If not, please ignore me, and under no circumstances allow me to be any kind of impediment.

#455 ::: Laertes ::: (view all by) ::: May 14, 2015, 10:24 PM:

If we could all pretend that we didn't see that dreadful mixed metaphor in @454, I'd be eternally grateful. I'll be over here at the blackboard, writing "the Fascist octopus has sung its swan song, the jackboot is thrown into the melting pot" 500 times.

#456 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 10:57 PM:

felice@451:
There is the other question of whether the people who oppose RME actually support SDV-LPE at all

Of course. But I haven't really seen much objection from anyone, so I have faith that no one is intentionally being a bad actor here.


I'm reluctant to give the Business Meeting any more decisions to make than we have to

That's what the business meeting is for. I don't think they need to be protected from making decisions, so I'm not terribly worried about that.


I suspect this would be a bigger issue with modern nominations

I'm actually going to stand by my estimate. I ran well over 50 simulations using both the 1984 data and the 2013 data that Cheradenine modelled from the data that was published. I was only able to generate one situation in which the 6th place work would not have been on the ballot if the sim was re-run without a declined nomination. That case seemed to be exclusively when there was a slate present and it was one of the slate works that was declined. In that case, another slate work took its place instead of the 6th place work from the original run. You may have a point about TV shows, however -- if, for example, Dr. Who fans are going to nominate all Dr. Who episodes, then that will be a similar situation.


However, out of all of these sims that I ran, in no case did any of the other four original finalists change, so that's certainly good news.


My analysis is still that it doesn't matter much -- but much is not zero, and that's why I have no problem with either method, so long as we can come up with a clear and non-technical explanation for whichever one (and I think that we can).


===========
You may have noticed that I intentionally haven't voted to this point. That's because I don't think it matters much, and I'm much more interested in seeing what kinds of debates might come up at the business meeting. Subject to what Joshua/Kevin say about the procedural requirements for passing amendments, I think I have enough data to form an opinion and a suggested course:

What is the group's view on including RME in the SDV-LPE proposal and having an amendment ready to replace it with 6th place if there is significant resistance to RME? In my view, again, subject to procedural constraints, this is the smart play that will satisfy almost everyone (albeit to lesser and greater degrees).

Opinions?

Kilo

#457 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 11:10 PM:

A quick P.S. to my #456:

In the one case where the two schemes make a difference (when a slate has its work's nomination declined), if 6th place is used, then slate loses its finalist slot to a non-slate work. If RME is used, it maintains its slot, just with a different work from the slate.

This fact alone is enough to make me lean towards RME over 6th place, even though I'm still not sure how I'd "sell" RME at the business meeting yet. Possibly with that argument right there, I don't know.

K

#458 ::: P J Evans ::: (view all by) ::: May 14, 2015, 11:16 PM:

456
That will satisfy me - I still prefer taking the next one on the list, but if it isn't usually going to matter, then it isn't that important a difference.

#459 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 11:17 PM:

@456, What is the group's view on including RME in the SDV-LPE proposal and having an amendment ready to replace it with 6th place if there is significant resistance to RME?

I like either version being included, with an amendment ready to replace it with the other. I honestly don't know which would fly better at the Business Meeting, so I don't have an opinion on which should be in the original proposal and which should be the amendment... but, speaking personally, I prefer RME slightly over 6th place. For what it's worth.

#460 ::: Annie Y ::: (view all by) ::: May 14, 2015, 11:32 PM:

@456

Sounds like a good plan. I prefer #6 to RME but can support either and as it sounds like there is no much difference anyway, it does not make such a big difference anyway.

#461 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 11:38 PM:

I think we should proceed with Keith's idea. An RME proposal, with an amendment ready to use 6th place. If my parliamentary understanding is correct, that would mean there would first be a vote to see who prefers 6th place over RME, then in the vote on the full proposal, it would include whichever one has more support. I think this would work well. Any actual admins who spoke in favor of either one would probably sway it, which is how it should be.

If that is the answer, then all that remains is to finalize language. I agree with the suggestion to remove the "you"s from the counting procedure. I also think that the two stages should have names (such as "matchup" and "showdown") and the numbers being counted should also have names ("points" and "approvals"; not "votes" or "nominations" in either case, because those are confusing). In fact, it's probably better to avoid "nominate" and "nominee" and all related words altogether, and stick to "finalist". I have various ideas for the FAQ too, but that can come later.

Is it time to go back to the google doc? I made the text size large; Keith, see if it works for you like that. I agree that we should be posting versions here, but it is probably good to work on it there...

#462 ::: junego ::: (view all by) ::: May 15, 2015, 12:03 AM:

I'm good with Kilo's proposal to present RME with the amendment and have #6 as the alternate.

Jameson - I had some trouble with my iPad and the Google doc, so I'll definitely observe but if it keeps freezing on me I may come back here to comment. Will try not to make it harder on you guys.

You've all done a tremendous job. Thank you, again.

#463 ::: felice ::: (view all by) ::: May 15, 2015, 12:05 AM:

Keith "Kilo" Watt @456: "Of course. But I haven't really seen much objection from anyone, so I have faith that no one is intentionally being a bad actor here."

I'm not accusing anyone of being a bad actor; just that taking an interest in the SDV-LPE proposal isn't necessarily the same as liking it more than the current system (it could pass whether they like it or not, after all). Is there anyone participating in this discussion who doesn't like SDV-LPE?

"I'm actually going to stand by my estimate. I ran well over 50 simulations using both the 1984 data and the 2013 data that Cheradenine modelled from the data that was published"

We don't have any information on how nominations for different works actually overlap between ballots for 2013, though, do we? Does Cheradenine's model account for cases like "what happens if "2312" is withdrawn, and everyone who nominated "Existence" also nominated "2312" and nothing else in the top 10, while everyone who nominated any of the other top 10 works nominated more than one? Or for short story, if we ignore the 5% threshold, I'd bet if "No Place Like Home" was withdrawn, RME would replace it with "One Hell of a Ride" while 6th place would replace it with "Robot".

[TIES]
Unless simultaneous elimination takes "One Hell of a Ride" out at the same time as "We Will Not Be Undersold!" (two Seanan McGuire shorts with the same number of nominations).

#464 ::: Vicki ::: (view all by) ::: May 15, 2015, 01:18 AM:

The business meeting doesn't have to be protected from making decisions. It does have finite time (realistically speaking, and given that everyone needs sleep, and nobody wants to spend their entire Worldcon on the business meeting). So it helps to have a coherent decision to make: "should we do this thing?" rather than "what would this thing do anyway?" My impression is that people here are doing a fair job of that.

#465 ::: Seth ::: (view all by) ::: May 15, 2015, 02:04 AM:

#400 Not quite. Putting my second choice on my ballot can cause my first choice to be eliminated: when it's down to 6, my first choice is #5, and loses to #6. But if I hadn't put that second choice on, my first choice would have had an extra half point, making it #4, and something else would have been eliminated.

#466 ::: Joshua Kronengold ::: (view all by) ::: May 15, 2015, 02:27 AM:

The reason I still support #6 over RME (or highest_new(RME), for that matter) is, in fact, my intuition that it will frequently not make a difference [that even the pairs to a winning entry will generally be high enough ranked that they have a good chance of either being in the final ballot or making it to #6], combined with the complexity. But it's very much not a deal-breaker, and while there are reasons to support hn(RME), RME is clearly fairer [if also more likely to result in an extended ballot].

I've implied this before, but to be clear, I've attended a bunch of BMs, made successful floor amendments, and drafted a couple of proposals including one that made it (with substantial amendments) to passage. Kevin is an acknowledged expert at this, with several times my experience just in BM time alone, regularly chairs WSFS committees, and is also a several time chairman of the WSFS business meeting itself (including this year). The gulf between our experience could hardly be larger.

That said, yes, I think if we think there's an optimal and dominant way to handle ties, we should include it in the motion -- with, if we see substantial room for disagreement (and I do), submitted amendments (or prepared ones -- though anyone who is eligible to add new business can submit the amendment by substituion (or by striking out words/paragraphs); it need not be the same parties that submit the main motion and any on the floor amendments).

There's also a slight procedural advantage in having one of the replacement proposals in the original proposal -- in that it provides a convenient handle to ammend by substituion to add the other replacement proposal -- and that in terms of floor time it's easier to move to amend to strike a paragraph out than to draft and insert one. So it's probably a more efficient use of debate time to have RME (or 6th) in the original proposal. [motions to amend consume debate time asigned to the main motion, so having a lot of motions to amend considered can eat a lot of the main motion debate time--although if we do end up moving the debate for this to Sunday, that will reset debate time, since it's only per-day).

#467 ::: Brian Z ::: (view all by) ::: May 15, 2015, 06:57 AM:

@433 JQ

Note to Brian Z: if your goal here is to throw sand in the gears, you can make that pretty obvious right now by voting against RME.

I'm not sure I would want to either "punt" or "throw sand." But I support a conversation about whether or not this change is a good idea. And I have not been convinced to support any rules change, especially if aimed at increasing fairness, which distracts us from selecting excellence.

I would support a very minor change which did nothing but allow replacement of the bottom two nominees in the event a set of lockstep voters swept a category. But even that is distracting from the real problem of encouraging new voters to value the traditional voting culture, which should be the priority especially until emotions subside and all heads are clearer.

Thanks.

#468 ::: Shane ::: (view all by) ::: May 15, 2015, 07:28 AM:

#467 ::: Brian Z :::

I think there have been a lot of good responses to your statement that we just need more voters, but you don't seem moved by them. Have you any response?

and I don't understand your distinction between "fair" and "excellent". We want a fair way of collating everyone's opinion on what's excellent. Making sure everyone gets equal consideration doesn't distract us from selecting excellence *at all*.

What we can't do is specify how people choose what is excellent, nor make them prove that excellence really is what they selected for. We take each person's opinion at face value, and then count them as evenly, as fairly as we can.


This last sounds so ridiculous to say that I kinda hope I've been successfully trolled. I can't see how you can oppose fairness in voting without being against democracy. What am I missing?

#469 ::: Duncan J Macdonald ::: (view all by) ::: May 15, 2015, 07:32 AM:

Jameson Quinn @ 432 & 433:

I didn't log back in until this morning. having reviewed posts up to #466, I have no problem with being a lone hold-out if needs be. More discussion follows.

It looks as if Duncan J Macdonald, Laertes, and Cally Soukop are the ones who oppose RME. Given that it seems unlikely that we'll get consensus for anything but RME or punt, and if you oppose RME for reasons of complexity you may oppose punting even more, are any of you willing to rethink your stance on RME?

I am a fan of simplicity. 6th place is the current solution, and it takes no special vote by the Board to make it the default in this case either. The best way to get this proposal through the Board (twice) is to make as few changes as possible.

The strengths that I see at this point are:
1) Nominating process does not change (from the membership point of view).
2) Nominations are counted/processed in a way that minimizes the impact of slates.
3) If a nomination is declined or found ineligible, the method by which the replacement is made does not change.

When I was a project manager in the Defense Contractor sphere, the watchword was always "Give the customer what they ask for. If they ask for a Volkswagen, don't design a Cadillac." The expressed objective here was to limit the effect of slate nominations. SDV-LPE in the counting the nominations stage does that. No need to change anything else at this time.

#470 ::: Seth Gordon ::: (view all by) ::: May 15, 2015, 08:54 AM:

May I propose a slight variation on #10 (base proposal plus amendment)?

Put a clause in the proposed constitutional amendment saying:

If a finalist withdraws or is disqualified after the votes have been tallied, the administrators shall exercise their discretion regarding what work or works to replace it with, subject to the following restrictions:

(a) No finalist who has already been notified shall be struck from the ballot without his or her permission;

(b) The WSFS may adopt a Standing Rule that specifies a procedure for replacement.

The Business Meeting can then vote the main SDV-LPE proposal up or down. If it passes, then there can be another debate on the standing rule for handling withdrawals (sixth-place vs. rerun-merge-extend vs. some other idea). If, by the time the amendment passes, the whole meeting is sick and tired of arguing about Hugo voting rules, then debate over the standing rule can be postponed to 2016.

#471 ::: Brian Z ::: (view all by) ::: May 15, 2015, 10:01 AM:

@467 I haven't replied except to direct questions because several folks have said some of my points are out of bounds for this thread and I can't really tell where the line is. But if you point me to the comments you mean I can respond.

It seems whether we need more voters or not, we have them.

Excellence vs democracy: I see no conflict. The WSFS, through democratic process, chose the rules which seek excellence. The Hugo nomination is not a democratic election: the business meeting is.

#472 ::: Steve Halter ::: (view all by) ::: May 15, 2015, 10:20 AM:

[DECLINED NOMINATIONS]

After finally catching up with the reading

[0] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[10] 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[5] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[0] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[5] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[0] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[0] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#473 ::: Steve Halter ::: (view all by) ::: May 15, 2015, 10:42 AM:

Is there a list of [TIE] proposals to vote on? It wasn't clear from the (lengthy) discussion to this point. Thanks.

#474 ::: Jon Lennox ::: (view all by) ::: May 15, 2015, 11:00 AM:

So Keith "Kilo" Watt in @456 demonstrates that in the 1984 data, there are no cases where a withdrawal would cause a previously-selected finalist to become unselected if the administrator just did a straight re-run of SDV-LPE (i.e., option 4).

Can someone construct a scenario in which this *would* happen? I'm having a hard time envisioning what the ballots would have to look like for it to occur.

Also, is there a standard term from voting theory for this sort of paradoxical outcome?

#475 ::: nathanbp ::: (view all by) ::: May 15, 2015, 11:23 AM:

Jon Lennox@474: It only makes a difference with heavily correlated ballots (such as might be found in the presence of a slate).

Suppose the following ballots in one of the last few rounds:
100x Alpha
95x Beta
90x Gamma
85x Delta
1x Psi
82x Psi, Omega
80x Epsilon

The second to last round is Omega vs Psi with Omega eliminated. The last round is Psi vs Epsilon with Epsilon eliminated. Now if Psi withdraws, under RME Omega would be the next nominee even though Epsilon placed 6th.

#476 ::: Joshua Kronengold ::: (view all by) ::: May 15, 2015, 11:31 AM:

Jon@474: This feels like something that requires real math.

My instinct is that this is impossible -- that something being removed will frequently cause it to be replaced by something else on the same ballot, but it won't usually cause anything else to be shifted out.

It's the LPE part that makes it hard here. If you look at the following ballots:

18: A
20: C, A
19: D, A
18: B

If we run these against SDV(2) [sdv with two winners], then in the initial ballot, C and D get eliminated and the final ballot is A, B, whereas if you remove A, then B gets eliminated first, leaving C and D as winners.

It's harder to construct stuff like this with LPE, because it will favor things with more votes -- you'd have to force two things that were eventual winners into an early showdown, so one eliminates the other (which then doesn't get to make the ballot.

But that requires a ballot where multiple things make the ballot due to the early showdown, so SDV-LPE(2) isn't going to work for this. SDV-LPE(3) might work--with the ballots:

18: A
20: C, A
19: D, A
18: E, A
17: B
16: F

With an early run of SDV-LPE, E and then D are eliminated, then F; the resulting ballot is A, C, B. But if you remove A from contention, then it's a straight popularity elimination, with C, D, and E making the balot, and B bumped.

#477 ::: colin roald ::: (view all by) ::: May 15, 2015, 11:32 AM:

Seth @465:
But if I hadn't put that second choice on, my first choice would have had an extra half point, making it #4, and something else would have been eliminated.

To be clear, that happens only if your second choice has not been eliminated yet, and since we're talking about the 5/6th place bubble, that means your second choice made the Hugo ballot.

Basically, LPE expects you (like now) to list works that you'll be happy to see on the Hugo ballot, and not to list works that you wouldn't. So seeing one of your choices make the ballot should be a good outcome for you, even if it was your second choice.

But yes, I acknowledge that I overstated slightly saying there's could never be any downside to listing a second, or a third, or a fifth choice that you like less well than your first. I *do* think the circumstances required for it to make a difference are exceedingly particular, and in the real world so rare as to not make a difference. And also no worse than the likelihood of the same thing happening under the current nomination system.

That is, LPE does not seem particularly vulnerable to "bullet voting", to the extent that anyone even thinks bullet voting is a bad thing.

(The initial problem at hand is that a minority voting a slate can shut their political opponents entirely out of a category. Bullet voting will never allow any minority group to do that, and I can't see it doing any other very bad thing either. My thoughts ramble on from here, but I think I'm going to break them out into a new comment.)

#478 ::: colin roald ::: (view all by) ::: May 15, 2015, 11:49 AM:

I don't know to what extent this stuff has been hashed through in previous thousand-comment threads, and will confess to not having kept up. If it's already been worked through, please feel free to reply just pointing me to the thread I should search through.

I'm willing to bet publically that next year there will be an "SJW" slate of some kind to oppose, presumably, SP4/RP2, and that the "SJW" slate will have a bigger group backing it. It's the obvious immediate response. So in my view, the goal of the LPE proposal is to make sure that nominators in 2017 who don't care to back either slate will have the option to nominate their own preferences without getting run over between mobs.

But *that* raises the question for me: have simulations been run with two or three competing slates amongst a majority still nominating whatever they like? I'm relatively confident that *one* group won't be able to lock everyone else out of a category, but how about two or three? Will anything that wasn't on *somebody's* slate have a chance? This is probably a question for Jameson's simulations.

#479 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 11:57 AM:

Colin@478:

I ran a number of multiple-slate scenarios, and in all cases there was at least one (and usually more, depending on the specific scenario) non-slate work on the final ballot. Obviously having a slate makes it more likely to get at least one of your works in the finals, but it didn't lead to sweeps.

Kilo

#480 ::: Jon Lennox ::: (view all by) ::: May 15, 2015, 12:31 PM:

Joshua Kronengold @476: Ah, that makes sense. So it sounds like finalist instability can happen when a broadly-popular work (nominated on a lot of semi-uncorrelated ballots) withdraws, leading to rejiggering the point counts in the early elimination rounds.

I'm tempted to call this a "Pratchett Withdrawal", though of course I don't know how the actual 2005 nominations were correlated.

I definitely like Pfusand @413's suggestion of having the software pre-calculate if any finalists' withdrawals would cause instability in the list of the other finalists, and letting the administrators use their judgment about whether to contact those finalists first. (Hopefully an administrator has some intuition about whether a withdrawal is likely.)

Obviously this doesn't help post-announcement withdrawals, but those were an unprecedented occurrence this year and only happened as a consequence of nomination slates, so hopefully they won't happen very often.

#481 ::: nathanbp ::: (view all by) ::: May 15, 2015, 12:45 PM:

Me@475: Oops, I misread Jon Lennox's request, I thought he was asking for a case showing a difference between RME and taking the 6th place.

#482 ::: Jameson Quinn ::: (view all by) ::: May 15, 2015, 01:41 PM:

@478: I agree with Keith. Multiple slates are problematic, but they have a hard time getting a sweep.

#483 ::: colin roald ::: (view all by) ::: May 15, 2015, 02:01 PM:

Buddha Buck @421:
(32,767 possibilities)

It's 2015. Not to be too flip, but I think running eliminations 32k times is what the supercomputer in my pocket does when I put down Crossy Road to go microwave a burrito.

#484 ::: Buddha Buck ::: (view all by) ::: May 15, 2015, 02:44 PM:

colin @483:

Yeah, 2^15 isn't too bad today, but it's still exponentially worse than 2^5.

Of course, the quickly-rejected proposal to find the 5 nominees which maximized some measure of total nominator happiness really showed a combinatorial explosion, as it scaled with N^5, where N was the total number of nominated works. With 100 nominated works, that's 72 million. Not impossible, but still a lot.

#485 ::: J Thomas ::: (view all by) ::: May 15, 2015, 03:01 PM:

@484 Buddha Buck

Of course, the quickly-rejected proposal to find the 5 nominees which maximized some measure of total nominator happiness really showed a combinatorial explosion, as it scaled with N^5, where N was the total number of nominated works. With 100 nominated works, that's 72 million. Not impossible, but still a lot.

You're probably talking about mine. It attempted to maximize diversity more than voter happiness. It didn't scale with N^5 because when one work had X bullet-votes then no work that had more than X total votes could compete with it, which cut down the number of combinations.

Combinations, not exponential.

Also, if we keep the 5% rule and the rule that you get only 5 nominations each, no more than 100 works can ever get 5%.

That proposal was shouted down before anyone understood it at all. I was deeply disappointed.

But what we have looks reasonably adequate.

#486 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 05:38 PM:

Jameson: I am off for the summer (finals graded, woo hoo!), so I can work on my laptop more. I reset the font size to a more reasonable size, but if someone else needs it larger, feel free. I appreciate your doing that for me.

That said, since some people still have trouble with the Google Doc, I think we should still paste a working copy here. In fact, it might be useful to have separate documents for the formal proposal language and the FAQ/commentary.

Kevin:
I'm assuming that all that can be submitted to the business meeting is the actual amendment and the FAQ's are just for discussion at the meeting itself. Can the FAQ's be submitted as well? Is there value in doing so?

Thanks,
Kilo

#487 ::: felice ::: (view all by) ::: May 15, 2015, 06:08 PM:

Jon Lennox @480: "Joshua Kronengold @476: Ah, that makes sense. So it sounds like finalist instability can happen when a broadly-popular work (nominated on a lot of semi-uncorrelated ballots) withdraws, leading to rejiggering the point counts in the early elimination rounds."

More or less, but it still requires a fairly large amount of unusual correlation. In #476, if A is so broadly popular, why didn't anyone who nominated B or F like it? One possibility is B and F are the last survivors of two different slates. So 6th place (or 4th place in this example) gives us a final ballot of CBF - ie both slates now get a spot, and nearly 3/4 of A's supporters miss out; while RME gives us CBDE - the B slate retains its spot, but the more popular D and E get on instead of F, and more than 3/4 of the people who nominated A get one of their other choices on the ballot instead (the other 18 either bullet voted A, or their other nominations were for unpopular/obscure works that got eliminated early).

In practice, there'd probably also be some overlap between C, D, and E, and a final ballot of CBD (no extension required) is more likely.

#488 ::: Tim Illingworth ::: (view all by) ::: May 15, 2015, 06:29 PM:

KW@486: it's normal to submit motions in the form:

Title: Change Something
Moved, to amend the WSFS Constitution by making this change.

Discussion: This change is really trivial and needs no discussion. We should just do it ;-)

#489 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 07:18 PM:

Tim@488:

Okay thanks. So if the discussion is also going with the proposal, we should probably keep that as tight as possible while still making the case.

K

#490 ::: felice ::: (view all by) ::: May 15, 2015, 07:47 PM:

Steve Halter @473: "Is there a list of [TIE] proposals to vote on? It wasn't clear from the (lengthy) discussion to this point."

The key question is: is it better to eliminate two works at once, or use an essentially random (but repeatable) method to pick one over the other?

Personally I'm strongly in favour of random when necessary; for the rare ties that make a difference (ie either one of the pair could become a finalist, but the tie eliminates them both at once and doesn't reallocate their points), it doesn't matter which wins because there have to be people who nominated both, so either way more people nominated the winner than didn't.

The system described in #135 will probably suffice to break important ties if we abolish the 5% threshold; but if we eliminate everything under 5% before running SDV-LPE, there may well be insufficient previous rounds to help. And if we keep the threshold but don't remove the works that fail to meet it before running SDV-LPE, a work below the threshold may cause a work above the threshold to be eliminated (eg C with 6 points and 3% causes A with 5 points and 10% to eliminate B with 4 points and 8%, when otherwise A and B would both be finalists).

#491 ::: Seth ::: (view all by) ::: May 15, 2015, 08:03 PM:

#477 colin, I could construct a case where putting my second choice on my ballot eliminated my first without itself making the ballot. Let's see:

At some point, my first choice (A) has 20.2. X has 20, Y has 20.4, my second choice (B) has 21. A gets eliminated. There are 8 left, and B also doesn't make the last 5. If B weren't on my ballot, A would have had 20.7, Y would have eliminated X (or vice versa) and that would have given more points to A, enabling it to survive to the last 5.

#492 ::: Seth ::: (view all by) ::: May 15, 2015, 08:06 PM:

#467 Brian Z., this entire proposal is simpler than any useful definition of "a set of lockstep voters swept a category."

#493 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 08:31 PM:

Seth@491:

I only see one problem with your analysis, though: You didn't look at the number of nominations each work received. That's a critical part of SDV-LPE and what makes it different from simple SDV. in your example, A and X would be -eligible- for elimination, but it might actually be that X gets eliminated if it had fewer nominations than A. It's true that if you hadn't nominated B then your A would have half a point more, but there's more to it than just points. Your nomination of B might actually have gotten it on the ballot since it's number of nominations that determines which of the eligible works gets eliminated, not points.

That said, I imagine there's still a way to construct a scenario where it happens, but it would be highly contrived and seems to me pretty unlikely. I dunno, you'd have to try it.

Kilo

#494 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 08:53 PM:

[TIES]

Steve@473: In the previous discussion, it's true that felice spoke up for random elimination because of disapproval of ever eliminating more than one work per round (correct me if I'm mis-stating your case, felice). Random elimination seems to be a non-starter for a largish group of people (including me -- I won't support a non-deterministic nomination system), however.

According to Jameson's 1300 simulations, the method described in #135 only eliminates one work per round in all but three cases (and all of these were fairly rare):

- Works appearing on just a few ballots, eliminated early. (which couldn't possibly win anyway)

- Works appearing (almost) exclusively as bullet votes. (Two groups of exactly the same size each nominated only a single work and no one else nominated those works)

- Works appearing exclusively on slate ballots. (The slate had perfect discipline for its works, giving exactly the same number of nominations to two of its works.)

None of these are really a problem, when it comes down to it.


My only remaining quesiton is how necessary it is to look back to previous rounds (comparing points from previous rounds until there isn't a tie) before eliminating more than one work. I believe felice came up with a situation where it could matter, but it didn't show up in any of the simulations I ran without lookback.

In short, the tie system seems fairly settled:

(Note that if there is a tie for lowest or second-lowest POINTS then all tied works are eligible for elimination along with the second-lowest or lowest point total work(s).)

What we are really looking at is ties for total nominations. If there is a tie for total number of nominations, then we look at points again and eliminate the work with the lowest point total. Ties aren't common, but this is by far the most common tie that occurs. I've forgotten the exact number Jameson found, but it was 99% of the time or something like that. Correct me if I've got it wrong, Jameson.


So, the tougher case:

- If there are works that are tied in both points and nominations, then we look at point totals for those works in previous rounds until we find a difference -- we eliminate the one with fewer points whenever we find it.

- If we get all the way back to the first round and the works are still tied for points and nominations, we just eliminate them both (this is the part that felice doesn't like).


My original code doesn't do lookback. If they are tied for both points and nominations, then both were immediately eliminated. For all of the cases I ran with realistic data (no exceptions), the ties existed all the way back to the first round, so both methods would have eliminated both works no matter what. As I mentioned, I believe felice came up with a hypothetical situation where you broke the tie before round 1, so I'm not admantly opposed to lookback. It just makes tie breaking slightly more complicated, is all.

So, in summary, the only real voting left is for lookback or no lookback. I think the consensus is for lookback, but I don't know that we had a formal poll on it.

Kilo

#495 ::: Jon Lennox ::: (view all by) ::: May 15, 2015, 08:57 PM:

felice @ 487: So it sounds like a plausible scenario where this could happen is if you have two slates (each making up about 20% of the nominators) with one overlapping work -- this work ends up on 40% of the nominating ballots and so ends up easily as a finalist, eliminating the rest of those slates' nominations. If that work withdraws, each slate's second-place work ends up on the ballot, knocking some other finalist out of the top five.

#496 ::: Brian Z ::: (view all by) ::: May 15, 2015, 09:00 PM:

@492, Seth, if it were hard to define, why would we know what it means? I define a sweep by lockstep voters as when: without a set of identical ballots, a set of five works could not have been selected.

If we worry about an almost-sweep: it is when a set of ballots identical in four places causes four works to be selected.

If anyone wants to classify sets of ballots with less than four identical works as lockstep voting, we should push back against that. This is a crucial point. Using the WSFS constitution to regulate what somebody who may not even be a member says on the internet (about slates or anything else) is not going to work. We are discussing how to evaluate the actual ballots cast by members.

#497 ::: Steve Halter ::: (view all by) ::: May 15, 2015, 09:32 PM:

Keith@494:Thanks for the synopsis. The lookback method sounds good. The tied all the way back case would seem to be ultra rare enough that it will probably never occur.
felice@490:Thanks.

#498 ::: JJ ::: (view all by) ::: May 15, 2015, 11:54 PM:

Brian Z, #467: "But even that is distracting from the real problem of encouraging new voters to value the traditional voting culture, which should be the priority especially until emotions subside and all heads are clearer."

Firstly, you're saying that encouraging the Puppies to "value the traditional voting culture" is going to be an effective strategy? I think the Puppies have made it amply clear that that is not a viable strategy. But hey, you're sure welcome to give it a try, and let us know how much success you have.

Secondly, your comment "until emotions subside and all heads are clearer" (apart from being a direct quote from Brad Torgersen and being extremely condescending and patronizing to the people here) shows your unfamiliarity with the people with whom you are dealing. It's been 6 weeks. Emotions have subsided, and the heads of a great many people are very clear -- especially about the fact that there are changes which can be made to reduce the effect of bloc-voted nominations.

Seth's point at #492, which you apparently missed, is that there is no need to define "lockstep voting". That would be a subjective judgment -- and there is no need to resort to subjective judgments when what has been developed here is an objective method of counting nominations which does not rely on the subjective judgment of anyone.

#499 ::: felice ::: (view all by) ::: May 15, 2015, 11:55 PM:

Jon Lennox @495: "So it sounds like a plausible scenario where this could happen is if you have two slates (each making up about 20% of the nominators) with one overlapping work -- this work ends up on 40% of the nominating ballots and so ends up easily as a finalist, eliminating the rest of those slates' nominations. If that work withdraws, each slate's second-place work ends up on the ballot, knocking some other finalist out of the top five."

Yep, except RME extends the ballot in this case, so the other (non-slate) finalist doesn't get knocked out. And two equally powerful slates with one overlapping work isn't that likely a scenario (this year, the rabid puppies were significantly more powerful than the sad, and they overlapped by a lot more than one work per category).


[TIES]

Keith "Kilo" Watt @494: "I won't support a non-deterministic nomination system"

The proposal is for deterministic pseudo-random system (first appearance in nominations or pre-generated set of coin-tosses), so the results are repeatable.

"According to Jameson's 1300 simulations, the method described in #135 only eliminates one work per round in all but three cases"

All based on 1984 data. The 2013 ballots had a potential tie that could have been a problem - two Seanan McGuire short stories tied on 28 nominations (part of a four way tie for 5th), which probably very strongly overlapped with each other (maybe even 100%), so would have had relatively low points and could be subject to early elimination, and it's quite possible that they would have tied for points, too (but we don't know if they were or not). A slate-like effect, but presumably just the result of Seanan fans being very enthusiastic about her work, rather than any kind of organised voting bloc. If the threshold has been 4% instead of 5%, one of them could have been a finalist, and there wouldn't be any previous rounds for the #135 method to check. And Dan Dos Santos and Julie Dillon tied in Pro Artist, so depending on overlaps, it's theoretically possible John Picacio could have eliminated them both. There was also a tie in BDP Short Form, with only 6 nominations separating 1st from 5th

And in 2014, there were two pairs of works tied on nominations amongst the finalists for Best Graphic Story, and a three way tie for 5th in Best Fancast, plus ties in Best Fanzine, Pro Artist, and BDP Short Form; the degree of overlap is less clear there, but there's clearly at least the possibility of simultaneous elimination.

Multiple Doctor Who episodes regularly get close to tying, with probable significant overlap. A simultaneous elimination there one day is quite possible.

The only finalist ties in 1984 were Campbell Award (not tied on points), and non-fiction (tied for most nominations, so no risk of being eliminated).

The probability of two potential finalists being simultaneously eliminated is very low, but it's not zero, and it's bad if it happens. The lookback method is close to useless unless the 5% threshold is eliminated.

#500 ::: Keith "Kilo" Watt ::: (view all by) ::: May 16, 2015, 12:19 AM:

[TIES]
felice@499:
The proposal is for deterministic pseudo-random system

Fair enough. I won't support a pseudo-random system, either.

As for the 5% rule, I agree we should support those working to remove that rule. A proposal is already out there -- and Kevin doesn't want it in ours -- so I don't think there's anything to do there.

K

#501 ::: Keith "Kilo" Watt ::: (view all by) ::: May 16, 2015, 12:37 AM:

[FORMAL PROPOSAL]
All:

Here is the latest draft of the formal proposal. This is based on what I perceive to be the majority opinion on the system, and it includes the edits suggested for the Google Doc. I think in the main, we all agree, but some have some differences of opinion in some of the details. At this stage, I'd like for your thoughts and what, if any, dealbreakers you see in this version.

Note that we also need to write the "6th place" amendment as well...

Jameson: I didn't want to upload this on top of the Google Doc you have up there, as this just includes the formal proposal itself. Do you want to upload it as a separate document? I can email you the Word doc if that is easier for you, just let me know.

Kilo
========================================
Short Title: Change to Hugo Nomination System

Moved, to amend section 3.8 (Tallying of Nominations), section 3.9 (Notification and Acceptance), and section 3.11 (Tallying of Votes) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed.
3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.
3.8.3: Any nominations for “No Award” shall be disregarded.
3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.
3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.
3.8.6: The Committee shall move a nomination from another category to the work’s default category only if the member has made fewer than five (5) nominations in the default category.
3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.9 or 3.2.10, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category.
3.8.8: The final Award ballots shall list in each category the five (5) eligible finalists as determined by successive rounds of elimination, where each member gets a single vote that will be divided equally among their nominations.
3.8.8.1: In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots.
3.8.8.2: In the event that two (2) or more works are tied for both total points and for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, the point totals for each work in previous rounds will be compared. If the two works have had equal point totals and equal number of nominations since the first round, then all members of that tie will be eliminated.
3.8.8.3: If elimination would reduce the number of ballots to fewer than five (5), then instead none of the works should be eliminated and all remaining works shall appear on the final ballot.
3.8.8.4: After a work is eliminated, it is removed from the ballots of the members who nominated it, and those members’ votes are reallocated equally among their remaining nominations, if any.


Section 3.9: Notification and Acceptance.
3.9.1 Worldcon Committees shall use reasonable efforts to notify the nominees potential finalists, or in the case of deceased or incapacitated persons, their heirs, assigns, or legal guardians, in each category prior to the release of such information. Each nominee potential finalist shall be asked at that time to either accept or decline the nomination. If the nominee potential finalist declines the nomination, that potential finalist shall not appear on the final ballot. In this event, the nomination system shall be re-run as described in Section 3.8 without the declined nomination. The five (5) eligible finalists from this re-run shall be merged with the remaining potential finalists from the original run. If this merge would result in more than five (5) finalists, then the ballot shall be extended to include the finalists from both the original and the re-run of the nomination system. This procedure shall also be used in the event that a potential finalist is deemed ineligible.

Section 3.11: Tallying of Votes.
3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, … places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes. During the same period a record of at least the last fifteen rounds of the selection process for each category shall also be published.

Submitted by: (insert members here, ideally including at least one attending member who will be at all the relevant WSFS meetings. A proposing member gets the privilege of speaking first to the pro side of the proposal)

Commentary: In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn't penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate -anything- you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don't, that's okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

#502 ::: Annie Y ::: (view all by) ::: May 16, 2015, 01:03 AM:
at least the last fifteen rounds of the selection process for each category

What if there are less than 15 rounds? Shouldn't the rule account for that case?

#503 ::: Shane ::: (view all by) ::: May 16, 2015, 01:04 AM:

#501 ::: Keith "Kilo" Watt :::

[FORMAL PROPOSAL]
[DECLINED NOMINATIONS]

No dealbreakers. I'd support that as is.

One thing sticks out at me that we discussed but didn't focus on upthread. The way it reads now I picture rerunning for each declined nomination as we receive them. But actually we want to do the re-run only after all potential finalists have had a chance to decline, right? With any and all decliners removed.

Is it just me with that reading? Can we rewrite the new bit of 3.9 to make that clear?

"If any potential finalist(s) decline the nomination, those potential finalists shall not appear on the final ballot."

"In this event, the nomination system shall be re-run as described in Section 3.8 without any declined nomination or nominations."

#504 ::: Shane ::: (view all by) ::: May 16, 2015, 01:14 AM:

#501 ::: Keith "Kilo" Watt :::

[FORMAL PROPOSAL]
[DECLINED NOMINATIONS]

Is the sentence accurate where it says "The five (5) eligible finalists..."? Ties can mean either the first run, or the re-run, can end up with not 5.

Should we remove the number, and just say "The eligible finalists...".

And hold on, I lost this in the discussion before. What if we have 6,7 or 8 potential finalists, and some withdraw but we still have 5 or more? Do we only want to re-run when there's under 5?

No, y'know what, thinking about it, I want to re-run when anyone's declined because of the chance that some interest group's second choice is way down the rankings, but will make it on a re-run. Disregard my previous para.

#505 ::: colin roald ::: (view all by) ::: May 16, 2015, 01:16 AM:

BrianZ @496:
a set of ballots identical in four places causes four works to be selected

So, what happens if there are two slates? Next year, I virtually guarantee there will be some kind of Justice Kitten slate as a reply to the Puppies. So, suppose they get crazy and lock up all five short story ballots with turgid post-Marxist workshop exercises about hero/heroines with so many sex changes they don't know what gender they are anymore. Your rule kicks in, and bumps off the bottom two to be replaced by the erstwhile #6 and #7, which are neo-Nazi guns-and-boobs shoot-em-ups from Rabid Puppy Press.

Please go ahead and tell me that this seems like a good result to you, well representative of Excellence in Science Fiction.

#506 ::: JJ ::: (view all by) ::: May 16, 2015, 01:26 AM:

colin roald, #505: "Next year, I virtually guarantee there will be some kind of Justice Kitten slate as a reply to the Puppies."

I think you are vastly underestimating the number of non-Puppies who find slates, of any persuasion, to be utter anathema -- and that any non-Puppies attempting to spearhead such a movement would very quickly experience a tsunami of disapproval and disincentives from those people, just as the Puppies will.

I think most non-Puppies have come to the realization that the only acceptable response to slates this year, and next, until the WSFS nominating process is changed, is to suck it up and rely on Mr. Noah Ward.

#507 ::: nathanbp ::: (view all by) ::: May 16, 2015, 01:46 AM:

[FORMAL PROPOSAL]

Looks good overall. A couple comments:
3.8.8: The final Award ballots shall list in each category the five (5) eligible finalists as determined by successive rounds of elimination, where each member gets a single vote that will be divided equally among their nominations.
3.8.8.4: After a work is eliminated, it is removed from the ballots of the members who nominated it, and those members’ votes are reallocated equally among their remaining nominations, if any.
I think vote/votes should be changed to point/points here, to match the language in the rest of the proposed 3.8.8. Or change points there to votes, but I think points everywhere is clearer.

In the spirit of making the least changes possible, I think the proposal should leave the phrasing in 3.9.1 as "nominee" instead of changing it to "potential finalist".

In this event, the nomination system shall be re-run as described in Section 3.8 without the declined nomination
I think "with the declined nomination(s) removed from the ballots on which they appeared" might be clearer here?

We may also want to clarify 3.9.1 slightly to indicate that the results of the rerun should be combined with the original run and any previous reruns. Right now it could be interpreted to say that only the original run and the most recent rerun are combined.

#508 ::: colin roald ::: (view all by) ::: May 16, 2015, 01:51 AM:

Kilo @501:

3.8.8 says "each member gets a single vote", but then 3.8.8.1 goes right to talking about "works ... with the least and second-least total points". Points need to be defined, possibly simply by replacing "vote" in 3.8.8 and 3.8.8.4 with "nominating point".

3.8.8.2 says "In the event that two (2) or more works are tied for both total points and for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated." This is mis-stated, since obviously if two works are tied for total points, then neither one has a lower point total. I think it should be instead: "In the event that two (2) or more works being compared are tied for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated."

In the Commentary, it seems like it would be worthwhile to add two points:

- It is to be emphasized that this proposal does not change the nominating ballot from the perspective of Worldcon members: they still list one to five unranked works in each category they choose, just as they have previously.

- It is also an explicit goal of this proposal not to disenfranchise anyone, including members nominating from a recommended slate of works. Rather, this proposal seeks to ensure that a minority of members cannot disproportionately dominate an entire category, no matter how coordinated they are. It will still be quite possible for a motivated group to get one or perhaps two works onto the final ballot, but not to lock all other perspectives out of the running.

#509 ::: colin roald ::: (view all by) ::: May 16, 2015, 02:02 AM:

JJ@506:

You are an optimist. I mean, I think you are right that the majority of traditional fandom despises slates and won't join them, but I bet there is at least another similar-sized minority for whom political ends are more important than Hugo culture, and they won't be able to resist promoting something. Tradition or social disapproval didn't stop the Sad Puppies, did it? I doubt it will stop their enemies, either.

Also, the Puppies (especially Rabid, I think) stacked their vote by drawing in a lot of outside activists who never really cared much about the Hugos other than as a way to "show" the SJWs. I bet there are at least as many feminist activists out there willing to come in and stuff ballot boxes on the Justice Kitten side.

#510 ::: felice ::: (view all by) ::: May 16, 2015, 02:30 AM:

[FORMAL PROPOSAL]

Keith "Kilo" Watt @501: "If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, the point totals for each work in previous rounds will be compared."

Perhaps "If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, the point totals for each work in each previous round will be compared until a round with different point totals is reached."

"3.9.1 Worldcon Committees shall use reasonable efforts to notify the nominees potential finalists"

Is "potential" necessary? They are definitely finalists unless they actively decline, right? Ie if there's no response, they go on the final ballot anyway?

"During the same period the nomination voting totals shall also be published"

I'd suggest leaving this in, since unless the 5% threshold is removed, there could be a lot fewer than 15 SDV-LPE rounds (none, in some cases). And add instead "Additionally, a record of at least the last ten rounds of the selection process (or all rounds, if fewer than ten) for each category shall also be published." Ten rounds shows ten works eliminated plus the 5 finalists, ie is equivalent to the current top 15.

"Commentary"

I'd suggest including something along the lines of "This system increases the chance that one of the works you nominate will make the final ballot, but decreases the chance that more than one will" to simply explain how this system differs from the current system in practical terms.


Shane @504: "And hold on, I lost this in the discussion before. What if we have 6,7 or 8 potential finalists, and some withdraw but we still have 5 or more? Do we only want to re-run when there's under 5?"

I would suggest something like "In the event that there is a tie involving last place, and one of the tied works declines or is found ineligible, it should not be replaced." Though actually, maybe that's not necessary; I would guess the most likely result of RME in this situation is to not add a replacement anyway, eg finalists are ABCD[EFG], E withdraws, and the rerun produces ABCD[FG]. But saying tied works don't need to be replaced does no harm, and would be helpful to have already there if the "take the 6th" amendment beats RME.

#511 ::: Shane ::: (view all by) ::: May 16, 2015, 02:38 AM:

#510 ::: felice :::

Does it work out differently if one or more of the non-tied works (ABCD) declines?

#512 ::: felice ::: (view all by) ::: May 16, 2015, 02:51 AM:

Shane @511: "Does it work out differently if one or more of the non-tied works (ABCD) declines?"

It's more likely to; any second choice for E nominators is going to be less popular than F and G (because it's less popular than E by definition, and F and G are equal to E)*, but second choices for A, B, C, or D could still be more popular than E, F, and G, but eliminated earlier due to point splitting.

#513 ::: junego ::: (view all by) ::: May 16, 2015, 04:21 AM:

3.8.8: The final Award ballots shall list in each category the five (5) eligible finalists as determined by successive rounds of elimination, where each member gets a single VOTE [Please remove the word "vote" and "voter" and "voting" from any section of the proposal dealing with the nomination. The current Constitution does, in fact, differentiate the language between the nomination process and the final voting process. I don't know if this was due to some earlier discussion/disagreement or not, but it makes a lot of sense not to confuse the two things we are doing. We are not voting (in a strict sense) at this stage, we are recommending/suggesting/polling. We should comply with the current Constitution's useage unless there's some compelling reason not to.]. that will be divided equally among their nominations.

#514 ::: junego ::: (view all by) ::: May 16, 2015, 04:37 AM:

@ #513 ::: junego ::: (view all by) ::: May 16, 2015, 04:21 AM:
Forgot to put header on.

[FORMAL PROPOSAL]

3.8.8: The final Award ballots shall list in each category the five (5) eligible finalists as determined by successive rounds of elimination, where each member gets a single VOTE [Please remove the word "vote" and "voter" and "voting" from any section of the proposal dealing with the nomination. The current Constitution does, in fact, differentiate the language between the nomination process and the final voting process. I don't know if this was due to some earlier discussion/disagreement or not, but it makes a lot of sense not to confuse the two things we are doing. We are not voting (in a strict sense) at this stage, we are recommending/suggesting/polling. We should comply with the current Constitution's useage unless there's some compelling reason not to.]. that will be divided equally among their nominations.

#515 ::: junego ::: (view all by) ::: May 16, 2015, 05:59 AM:

colin roald @ 508 and
Kilo @501:

[COMMENTARY]

colin, I agree with your recommendations for changes to the formal proposal.

- It is to be emphasized that this proposal does not change the nominating ballot from the perspective of Worldcon members: they still list one to five unranked works in each category they choose, just as they have previously.

I think this needs to be the first point in the Commentary and emphasized. Our proposal doesn't affect the way members nominate, it just changes the way the nominations are tallied.

- It is also an explicit goal of this proposal not to disenfranchise anyone, including members nominating from a recommended slate of works. Rather, this proposal seeks to ensure that a minority of members cannot disproportionately dominate an entire category, no matter how coordinated they are. It will still be quite possible for a motivated group to get one or perhaps two works onto the final ballot, but not to lock all other perspectives out of the running.

1)I really don't want to put any language anywhere in the proposal the explicitly or implicitly condones, supports or even accepts slates nor do I think we want to enter into arguments about what a slate is, what recommendation lists are, etc.. There may be people at the meeting motivated to oppose the proposal, we should reduce points of contention. So suggest changing the first sentence to "It is an explicit goal of this proposal not to disenfranchise anyone."

2) Suggest rewording second sentence to "Rather, this proposal seeks to ensure that a motivated, coordinated minority of members cannot disproportionately dominate an entire category." Just for brevity (which I don't notice when editing my own verbosity for some reason :-p ).

4) Now for just nitpicky edit-foo(lishness). Could change sentence to "A coordinated group can still get one, or maybe two, works on the final ballot, but they will not be able to lock all other perspectives out of the running.

#516 ::: J Thomas ::: (view all by) ::: May 16, 2015, 06:01 AM:

@493 Keith "Kilo" Watt

That said, I imagine there's still a way to construct a scenario where it happens, but it would be highly contrived and seems to me pretty unlikely. I dunno, you'd have to try it.

Given a "normal" collection of votes, the only likely result is that the five with the most votes will win. That's about 90% right there. *Any* other result is unlikely, and the more specific you make it, the less likely it is.

If we were doing approval voting, you would vote for the works you thought should be finalists. You are saying that the ones you vote for are worth being finalists and you say that for all of them equally. If it's more important to you to vote for Able over Baker than to vote for both of them, then you should vote only for Able. If you think they're both good enough, then vote for both.

This other way, the total votes count more than 90%. If you particularly care about the choice between voting for Able versus Able+Baker, the chances are probably around 96% that any variation from "most votes win" will not affect those two.

If you want to get a different result, you need to be part of a large cabal that tries to have more influence than anybody else. This voting system reduces the effect of slates to be no more than that of bullet-voting.

Bullet-voting is the equivalent of single-issue voting. You get more influence on your single issue, and you lose all influence over anything else. The effect is likely to be small, because if your cabal is large enough to get your work into the finals with bullet-voting, it's probably large enough to do that even if all of you also vote for your own individual other choices. You give up all influence on the rest of the nominations, in exchange for an improved chance (total improvement well under 10%) to get this one in.

The reason I say under 10% is that it's 90% the top 5 will win, and so if your cabal is 20% of the voters and you all bullet-vote, that increases your work's chance from better than 90% to 100%.

That reasoning isn't exactly correct, but it suggests the order of things. If your work is not in the top 5 for votes, by bullet-voting you can increase the chance that if the top 5 don't all win, it will be yours that replaces one of them. And that chance is probably not over about 10% whether your whole group bullet-votes or not.

The odds are worth thinking about while considering whether to adopt the system. It's worth thinking about while we argue whether the system is fair enough. When peculiar results are *possible*, does that make it bad?

It's probably not worth building a cabal to have that sort of effect. You'd do better to just write reviews telling whoever will listen how great the work is.

#517 ::: J Thomas ::: (view all by) ::: May 16, 2015, 06:57 AM:

@494 Keith "Kilo" Watt

So, in summary, the only real voting left is for lookback or no lookback. I think the consensus is for lookback, but I don't know that we had a formal poll on it.

I have a moderately strong preference against lookback because it is harder to explain. Particularly, if you want to make an animation describing the process, it's harder to draw it.

Still possible, just harder. So I don't like it.

And the farther you have to look back to find a difference, it seems to me the less difference it makes.

Imagine you're explaining it to the guy whose work lost. "It was a tie, so we looked back, and at the beginning you were almost tied, but one of your votes was shared with four others, while he had a vote that was shared with only three others. The first round one of those four dropped out and you were both tied from that point on. We decided that that extra vote for somebody else meant that his deserved to be a finalist and yours didn't."

So I prefer the simpler system. If they tie on votes, eliminate the lowest score. Eliminate all that tie on lowest votes and lowest scores both.

For the last round, better to have 6 finalists than 4, better to have 4 than 7, better 7 than 3, better 3 than 8 or more.

If we have a 5-way tie on the last round, something has gone seriously wrong.


#518 ::: Jameson Quinn ::: (view all by) ::: May 16, 2015, 08:24 AM:

@494 Re: ties: I did 1300 bootstrap runs, and there were no unbreakable ties with fewer than 15 candidates remaining. (It's not that there was one with 16 remaining; it's just that I arbitrarily only started checking when there were 15. There were unbreakable 1- and 2-vote ties, I don't know exactly when the latest one was.

"Unbreakable" means same points all the way back to the first round.

This was without a 5% threshold.

In theory, you could make a scenario where eliminating or not eliminating the 5% works could make a difference as to who was a finalist. But really the only way it could matter is in deciding which was the second slate candidate finalist. And that would have to mean that a 20% slate was getting two slots, which wouldn't happen unless non-slate voters were really all over the map.

So anyway... if you need the point counts from the pre-5% rounds to break a tie, you can just run the system without doing the 5% elimination to get those numbers, and make sure that it doesn't eliminate an over-5% candidate before an under-5% one, which except in case of failed slates won't happen anyway.

As a voting theorist, I consider the question what to do in case of ties resolved, unless somebody (who otherwise supports the proposal) has an objection to the "points and historical points, then mass elimination" option.

#519 ::: Jameson Quinn ::: (view all by) ::: May 16, 2015, 08:56 AM:

So it seems that people are doing the edits here rather than on Google Docs. That's a big slower but more inclusive. If people want to use Google Docs instead, say so, and I'll curate the document. If the consensus is that slow and steady wins the race, then I won't, and I'll just leave it here.

I think that @516 puts it very well; well enough to go into the commentary. Something like:

"Under this system, looking at computer-generated elections based on historical data¹, it seems that if a work has one of the top 5 vote tallies, it will win about 90% of the time. That means that in the absence of slate voters, the results of this system will be 90% the same as historical results. It also means that unless you have near-perfect knowledge of exactly how everyone will vote, the additional advantage to be gained from coordinated strategic voting is no more than about 10% better chances of getting the desired result, and even that is only true if the result is reasonably popular already. Given the chances of a "no award" backlash in the following round, a less than 10% chance of an advantage in the nominations is probably not worth the work of coordinating a strategy.

¹ We made 1300 simulated elections, generated by resampling ("bootstrapping", in statistical parlance) from the 1984 elections. (100 simulations from each category). The simulated elections had only 100 ballots each, so these results are likely to be conservative; that is, if anything, they will exaggerate the possibility of unusual results."

....

I'm just going to put this out there; I don't expect it to be picked up, but it can't hurt to try.

I'd enjoy going to Sasquan, but I can't justify it financially on a graduate student's earnings. (Yes, I'm a graduate student who's old enough to have read "tea with a black dragon" when it first came out in paperback.) I've also done an amount of work on this proposal which would have easily earned me enough for the necessary tickets if I had been consulting. (Though if I did earn money on this, I'd probably donate all of it to electology.org) Would anybody be willing to sponsor me? Obviously, I'd go to the BM, and be available to answer questions there; it has been suggested that would have value.

And similarly: if I deserve being sponsored to attend, then Keith surely deserves it twice over.

If you're interested in sponsoring either one of us, you can email me, using my first and last names separated by a dot, on google's email service (or using my names with no dot in between, at Faculty of Arts and Sciences [FAS] dot harvard dot edu).

#520 ::: Jameson Quinn ::: (view all by) ::: May 16, 2015, 09:00 AM:

@517: But the idea is that if there is a tie at the last step, you just keep both. So the explanation you're giving is why a work lost when it lost, but unless it's part of a slate, the fact that it was up for elimination at all probably means it would have lost a round or two later anyway. The tiebreaker is just a way to allow the eliminations to proceed; it mostly won't make a difference. And finally, the alternative, as I see it, is to eliminate both; so the person who is eliminated by the tiebreaker hardly has grounds for complaint.

#521 ::: Jameson Quinn ::: (view all by) ::: May 16, 2015, 09:02 AM:

Note that my post @518 was in limbo (database error) for some time, so it may no longer have been really germane when I finally posted it.

#522 ::: J Thomas ::: (view all by) ::: May 16, 2015, 09:53 AM:

@520 Jameson Quinn

@517: But the idea is that if there is a tie at the last step, you just keep both. So the explanation you're giving is why a work lost when it lost, but unless it's part of a slate, the fact that it was up for elimination at all probably means it would have lost a round or two later anyway.

We're talking about a rule that applies from the 2nd to next-to-last step. At the next-to-last step, it could make the difference between being a finalist and not.

The tiebreaker is just a way to allow the eliminations to proceed; it mostly won't make a difference.

Eliminations proceed slightly faster if both are eliminated. Agreed, it doesn't make a big difference. It's simpler and also easier to explain without the lookback. I think it's fairer without that, but not more merciful. It won't usually make much practical difference.

And finally, the alternative, as I see it, is to eliminate both; so the person who is eliminated by the tiebreaker hardly has grounds for complaint.

He does, but he reveals himself as small-minded if he talks about it. "My work is just as good as his, but he's a finalist and I'm not." I could imagine Correia and Scalzi complaining about that, whichever one lost, but it wouldn't be a big surprise if they just congratulated the other instead. It's a bad loser who complains that somebody else didn't lose too. We probably have some of those.

#523 ::: junego ::: (view all by) ::: May 16, 2015, 10:09 AM:

[SPONSORSHIP]

Jameson

I would be willing to contribute to a couple of available, interested and knowledgeable people going to Sasquan. Maybe you can discuss this with whoever is interested in representing the proposal and start something like a kickstarter campaign?

#524 ::: junego ::: (view all by) ::: May 16, 2015, 10:32 AM:

[TIE-BREAKING]

I am almost positive what I'm about to ask/propose was covered in the previous thread. (I can't find it and don't remember details.) If so, and it is just too something-or-other to work, would someone explain quickly and simply why it won't work and/or point me to the discussion? tia

Wait, was the reasoning that the method below would favor slates?

PROPOSED
If 2 works are tied on points and nominations both they challenge the work with the next lowest points. At the challenge, if their number of nominations don't exceed that work, both are eliminated. If their nominations exceed the challenged work, that work is eliminated. If the tied works win, after the recalculation the tied works will either no longer be tied or will still be tied and will challenge the next lowest work.

#525 ::: colin roald ::: (view all by) ::: May 16, 2015, 11:10 AM:

junego@515:
I think I like felice@510's wording way better than my too-verbose attempt. How about:

"It is an important goal of this proposal not to disenfranchise anyone. This system increases the chance that one of the works you nominate will make the final ballot, but decreases the chance that more than one will."

#526 ::: colin roald ::: (view all by) ::: May 16, 2015, 11:44 AM:

junego @524:
If their nominations exceed the challenged work, that work is eliminated

I believe that will favor slates, since they will tend to be allocating fewer points per approval -- or conversely, will have more approvals per point. So tied slate works would be likely to win this challenge.

#527 ::: junego ::: (view all by) ::: May 16, 2015, 12:01 PM:

colin roald @ 525
junego@515:
I think I like felice@510's wording way better than my too-verbose attempt. How about:
"It is an important goal of this proposal not to disenfranchise anyone. This system increases the chance that one of the works you nominate will make the final ballot, but decreases the chance that more than one will."

So, no statements about reducing the power of coordinated minority groups at all? Why should anyone want to go to the trouble of changing the constitution if it's not going to help with the problem we've got? I can see the point of putting as positive a light on this as possible, but shouldn't we mention the elephant in the room? Or is that going to be somewhere else?

First sentence is fine. I'd rephrase the second sentence.
"This system increases the chance that each member can get one nominee on the ballot, but decreases the chance that anyone can get 2 or more on the ballot."

And add something like the following.
"This also generally reduces the possibility of coordinated minority groups getting more than 1 ballot slot per category."

#528 ::: J Thomas ::: (view all by) ::: May 16, 2015, 01:05 PM:

@524 junego

PROPOSED
If 2 works are tied on points and nominations both they challenge the work with the next lowest points. At the challenge, if their number of nominations don't exceed that work, both are eliminated. If their nominations exceed the challenged work, that work is eliminated. If the tied works win, after the recalculation the tied works will either no longer be tied or will still be tied and will challenge the next lowest work.

We discussed this a little bit, before. I'll say more now.

Without slates, this works maybe a bit better than discarding both. After awhile either they will both lose, or they will stop being tied and things go back to normal, or they stay tied while they work their way to the top -- if they win against the #5 then presumably they become tied for 5th place. Maybe you'd want to run them against #4 and if they win then #4 is gone and they become #4 and #5. It's better for them and worse for their competitors, no big deal. And it would be very very rare.

With slates, it is not so rare. There's a good chance that people who vote slates will make some mistakes and not all the slate nominees will get the same number of votes. But it only takes two, and so there are 10 chances for a tie, with the ten different combinations of 5 nominees.

If it's the top two slate nominations that are tied, then there is a small chance they will lose to the top nonslate candidate if it has enough shared votes to compete against them, and if it has more votes than they do.

Otherwise, if there are four or more nonslate works that have more than half their points, they will be tied for #5. If there are only three they will be tied for fourth place.

If both tied works were discarded, then in this particular case the slate would lose out when they collided. This rare event would make a tiny step toward balancing the extra slate items that win because the nonslate votes are so scattered. Remember that they deserve to win one when they have 20% of the vote, but with 15% of the vote they will often win two final places, and this gives them a slight chance to not get any.

The case of tied slate works that have fewer votes than some other slate work doesn't make as much difference. The other work will eliminate them if there are enough nonslate works that have scores above 1/3 its number of votes. Otherwise they get 3 slate wins.

It doesn't really make a lot of difference. It lets us be nice to the very rare tied nonslate nominations, and it lets us be even nicer to the somewhat-rare tied slate nominations.

I don't see that it's worth doing, and the net result looks negative to me, but if people really wanted it I'd go along. It's a small effect, not very important except possibly on principle, about fairness. I think the argument would be that given the choice, we try to be nice to a few specific people while hurting everybody else a little, rather than be mean to a few people to help everybody else a little.

#529 ::: colin roald ::: (view all by) ::: May 16, 2015, 02:06 PM:

junego @527:

I take your point, but on the flip side, being too specific about a current issue can make the proposal sound too specifically tailored to today's circumstance. People may be rightly suspicious of an overreaction. This is not an "anti-Puppy" proposal, and I don't think we want to make it sound like it is.

And I might as put the Commentary back together and suggest a couple tweaks to the wording again while I'm here:


Commentary: In the past we have simply counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

We regard it as destructive to the tradition of the Hugos if a minority of members using coordinated strategic voting is able to overwhelm the collective input of all others who simply nominate their own diverse preferences.

Using this proposed system, fandom isn't penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate anything you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don't, that's okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

It is an important goal of this proposal specifically not to disenfranchise anyone. This system actually increases the chance that one of the works you nominate will make the final ballot, while decreasing the chance that more than one will.

It is to be emphasized that this proposal does not change the nominating ballot from the perspective of Worldcon members: they still list one to five unranked works in each category they choose, just as they have previously.


Under this system, looking at computer-generated elections based on historical data¹, it seems that if a work has one of the top 5 vote tallies, it will win about 90% of the time. That means that in the absence of slate voters, the results of this system will be 90% the same as historical results. It also means that unless you have near-perfect knowledge of exactly how everyone will vote, the additional advantage to be gained from coordinated strategic voting is no more than about 10% better chances of getting the desired result, and even that is only true if the result is reasonably popular already. Given the chances of a "no award" backlash in the following round, a less than 10% chance of an advantage in the nominations is probably not worth the work of coordinating a strategy.

¹ We made 1300 simulated elections, generated by resampling ("bootstrapping", in statistical parlance) from the 1984 elections. (100 simulations from each category). The simulated elections had only 100 ballots each, so these results are likely to be conservative; that is, if anything, they will exaggerate the possibility of unusual results.

#530 ::: Keith "Kilo" Watt ::: (view all by) ::: May 16, 2015, 02:48 PM:

[FORMAL PROPOSAL]

Sitting at my daughter's ice skating practice...

One important stylistic factor we need to keep in mind is that I think the commentary section needs to be brief and therefore very tightly written. Take a look at the proposals that have already been submitted on the business meeting site, and you'll see what I mean. I don't think we need to be minimalistic, but I think we need to use the commentary to make our case convincingly and save the really detailed explanations for the meeting itself, if required.

It's like when my astro students are presenting their research projects. You don't put the kitchen sink in your presentation, but that doesn't mean that -you- don't have to know the topic in minute detail. So while we need to have all these persuasive arguments worked out, they don't necessarily need to be in the commentary.

Just my thoughts,
Kilo

#531 ::: Brian Gibbons ::: (view all by) ::: May 16, 2015, 02:48 PM:

As was discussed upthread, I think the way this proposal is worded is slightly flawed. It's coming from the wrong perspective.

"Each member gets a single vote." No, they don't. Members don't get votes. Members don't allocate their votes. Members aren't doing anything under this section.

Members submit their nominations, and that's under a completely separate section (3.7.1). This section is all about telling the administrator how to tally those nominations.

Also, the "two (2)" language isn't required (see, e.g., Sections 2.4, 2.8, and 3.8.1), and really comes across to me as trying too hard to look all official and lawyerly. It's not even recommended for lawyers in modern practice.

I'm also not sure why we would want to leave a blank 3.8.1 and make a new 3.8.8. Striking out and replacing 3.8.1 seems a much better option.

So, here would be my suggestion for wording:

Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed. determined by the following multi-round process:

3.8.1.1: In each round, each non-eliminated nominee shall be assigned points based on the number of ballots on which they appear and how many nominees on those ballots have not yet been eliminated: For each ballot which contains a single non-eliminated nominee, that nominee shall receive one point; for each ballot containing multiple non-eliminated nominees, each of those nominees shall receive a fraction of a point equal to one divided by the number of non-eliminated nominees on that ballot.

3.8.1.2: After points have been assigned to all nominees, the two nominees with the lowest point totals for that round (or more than two, in the event of a tie) shall be compared to each other, and the compared nominee which appears on the least number of ballots shall be eliminated. If there is a tie as to the number of ballots on which the compared nominees appear, then both compared nominees (or all, if more than two are being compared) shall be eliminated. Should an elimination of tied nominees result in fewer remaining nominees than the number to be listed, then no nominees will be eliminated and all tied nominees will be listed on the final ballot.

3.8.1.3: After a nominee is eliminated, it is treated as though removed from all ballots on which it was listed, for purposes of determining how many points the non-eliminated nominees on that ballot receive.

3.8.1.4: Should a nominee decline nomination, be declared ineligible or otherwise be removed from the final ballot, and should a Committee, in its discretion, choose to add nominees to the final ballot to replace the removed nominee, it may do so by using the process described in this Section to determine who the nominees listed on the final ballot would have been with the removed nominee(s) eliminated at the start of the process, and then adding to the final ballot all nominees who would have appeared on the final ballot under that condition, even if this requires listing more nominees than described in 3.8.1. This procedure may only result in the addition of nominees, not the removal of otherwise-eligible nominees.

--------------------

I'm sure there's room for improvement. There are a few redundancies in here (e.g., specifying one surviving nominee gets one point; specifying that a rerun of the final ballot can't delete nominees) that make it come across as less elegant to me, but are probably necessary to increase the likelihood of immediate comprehension.

I haven't included 3.11.4 here, because I think that's worth a separate discussion.

#532 ::: Kevin Standlee ::: (view all by) ::: May 16, 2015, 03:57 PM:

I agree that you want extensive commentary. Commentary included with a proposal becomes part of the "legislative history," giving people who come later an opportunity to understand what the people who proposed something meant.

It is unfortunately rather common for people to only read the bare technical rules without comprehending that there is an entire body of practice and commentary built up around them. This is true of "mundane" law as well as what we do in WSFS; witness the rather wooden, context-less readings of portions of the US Constitution for examples.

#533 ::: Keith "Kilo" Watt ::: (view all by) ::: May 16, 2015, 04:00 PM:

Kevin@532:
OK excellent. That actuality makes the job much easier. Would you also include the FAQs we've written?

K

#534 ::: Steve ::: (view all by) ::: May 16, 2015, 05:05 PM:

I coded up a webpage to let people visualize how SDV-LPE works. It's at: http://www-users.cs.umn.edu/~damer/hugos/SDVLPE.html
I'd appreciate any feedback people have - especially bugs. I think it accurately implements what is outlined in the proposal, but then I never think that code I write has bugs until someone points them out to me. There isn't much in the way of error checking, but if something goes wrong you can reload the page to start over.

#535 ::: David Goldfarb ::: (view all by) ::: May 16, 2015, 05:37 PM:

Steve: I didn't notice any specific errors, but I will say that your interface is a bit obscure. In particular, when I clicked "Load Votes", I couldn't at first make head nor tail of the chart that appeared. Possibly you could put line numbers next to the nominating ballot data? Or at least put in some sort of explanatory note. Also, when you put a new entry in the activity log, you really really need it to have a trailing newline. (I.e.: separate log entries with a line of whitespace.)

I don't think I could swing a full sponsorship for someone, but I'd happily kick in $10 or so to some sort of crowdfunding for it.

#536 ::: felice ::: (view all by) ::: May 16, 2015, 05:46 PM:

Steve @534: "I coded up a webpage to let people visualize how SDV-LPE works. It's at: http://www-users.cs.umn.edu/~damer/hugos/SDVLPE.html"

Nice work! It would help if you could order the Current Votes by points, and add text explaining that the numbers in the grey boxes are voter IDs, and the width of the box is determined by how many of the voter's nominations are still in the running. And add "box-sizing: border-box" to the css.

#537 ::: felice ::: (view all by) ::: May 16, 2015, 06:18 PM:

[FORMAL PROPOSAL]

Keith "Kilo" Watt @501: "Short Title: Change to Hugo Nomination System"

That's really not very informative, particularly since there are already two other change to the Hugo Nomination System on the agenda, and are likely to be more. Can we stick to "Short Title: SDV-LPE (Single Divisible Vote with Least Popular Eliminated)"?

"3.8.8.1: In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared."

We should probably be more specific about how many works should be compared. How about "all works with the least or second-least total points will be compared (at least two works, but more if there are ties for least points and/or second-least points)"? The current wording isn't clear on what happens when there's a tie for least - it could mean only more than two if there's a three-or-more-way tie for least.


Keith "Kilo" Watt @533: "OK excellent. That actuality makes the job much easier. Would you also include the FAQs we've written?"

It might help to look at the New Business section at http://loncon3.org/wsfs-business.php for examples of more detailed commentary sections.


junego @527: "So, no statements about reducing the power of coordinated minority groups at all? Why should anyone want to go to the trouble of changing the constitution if it's not going to help with the problem we've got?"

I think that's covered by "This system increases the chance that one of the works you nominate will make the final ballot, but decreases the chance that more than one will." The problem of coordinated minority groups is that they currently get too many of the works they nominate on the final ballot. But that sentence was intended as an introductory summary of what SDV-LPE does; there's certainly room to elaborate afterwards.

#538 ::: colin roald ::: (view all by) ::: May 16, 2015, 06:35 PM:

Steve @534:

That's really cool. For educational purposes it would help a lot to highlight what's *going* to happen before you hit the Eliminate button -- otherwise you end up with a "well, *something* just happened, but I'm not sure what" reaction.

Like felice@536 suggests, it would help a lot to sort the current candidates by points, and also to explicitly list the current points. Then you could highlight the works that are going to be compared (usually the bottom two, but sometimes more), and an annotation in red next to the one that's going to be eliminated that says something like "Next to be eliminated: has same approvals (3) but lower point total".

It would also look sharper if you changed the code for your chart elements to <span class="vote-element" style="width: 60px;">10</span>

And added a stylesheet

.vote-element {
display: inline-block;
box-sizing: border-box;
background-color: rgb(177, 214, 234);
border: 1px solid gray;
line-height: 1.2;
}

But don't take this as complaining -- this is a cool thing.

#539 ::: Seth ::: (view all by) ::: May 16, 2015, 06:56 PM:

#493 I left out the number of ballots each appeared on, under the assumption that those numbers were such that the eliminations I suggested were the ones that would happen.

#540 ::: Seth ::: (view all by) ::: May 16, 2015, 07:11 PM:

junego@515: I think it would be better (and more accurate) to specify that under this proposal, a bloc can get a number of finalists proportional to the percentage of nominators in the block. 20% can get one item, 40% can get 2 items, 60% can get 3 items.

#541 ::: J Thomas ::: (view all by) ::: May 16, 2015, 07:29 PM:

@540 Seth

junego@515: I think it would be better (and more accurate) to specify that under this proposal, a bloc can get a number of finalists proportional to the percentage of nominators in the block. 20% can get one item, 40% can get 2 items, 60% can get 3 items.

Except that isn't true. It's possible for 10% to get 3 nominations, or 4, just less likely than with approval voting.

#542 ::: felice ::: (view all by) ::: May 16, 2015, 07:29 PM:

Seth @540: "I think it would be better (and more accurate) to specify that under this proposal, a bloc can get a number of finalists proportional to the percentage of nominators in the block. 20% can get one item, 40% can get 2 items, 60% can get 3 items."

It's not quite that simple, though. Some non-slate nominators will have all their nominations eliminated early on (because they only nominated works not many other people nominated), for example, which will increase the representation of the slate. The number of slate finalists will tend to be significantly closer to proportional than under the current system, but we can't give exact figures.

#543 ::: J Thomas ::: (view all by) ::: May 16, 2015, 10:04 PM:

I have an animation to help show how the voting system works. You can click on my name.

http://www.glowscript.org/#/user/J_Thomas/folder/hugo/program/Nominations

It's currently less flexible than I'd like, it cycles through the same steps and your only choice is to decide when to click forward to the next step. I'd do better to lay it out like a webpage so you can click on any part of it when you like, etc. But I should get some feedback before I put too much work into it.

The language is supposed to work on all modern computers with modern graphics cards, but it doesn't. I hope it works on enough to be useful.

It doesn't quite match the current leading version of the voting system; it would take some work to get it to.

#544 ::: Brian Z ::: (view all by) ::: May 16, 2015, 10:35 PM:

@505 colin roald

"Next year, I virtually guarantee there will be some kind of Justice Kitten slate as a reply to the Puppies"

First, more old and new members must and will nominate more vigorously, especially in short fiction categories. That's a given at this point. I think.

If something like half the voting members then proceed to organize themselves to attack and subvert the system a la Game of Slates, of course that is not going to lead to an optimal outcome. The award depends on most voters participating in the spirit of the award. If they don't, the award will not be worth much. This new proposal could lead to Game of Slates even more quickly: it ensures each faction gets something they want. Lowest common denominator. I would almost rather see rival slates battling each other on their merit.

But why don't we see if we can save the true spirit of the award first, before capitulating.

#545 ::: JJ ::: (view all by) ::: May 16, 2015, 10:48 PM:

Brian Z., : "But why don't we see if we can save the true spirit of the award first, before capitulating."

Capitulating to what? To the Puppies' repeated insistence that we should just accept their forced nominees as valid despite the obvious lack of quality, and not use "No Award"? Or to their repeated insistence that we should not try to change the nominating process?

I don't think there are very many people who are interested in or willing to capitulate to either of those things.

#546 ::: Seth ::: (view all by) ::: May 16, 2015, 11:03 PM:

#541 and #542: True, I didn't mean to say that a slate supported by 20% will always get exactly one finalist. It deserves to get exactly one finalist, and will typically get around one finalist (if no non-slate nominators like any of the slate's candidates). It will do a much better job of accomplishing that than the current method.

#547 ::: P J Evans ::: (view all by) ::: May 16, 2015, 11:10 PM:

544
But why don't we see if we can save the true spirit of the award first, before capitulating.

That's what we're doing here. The true spirit is things readers-who-are-members-of-Worldcon think are 'Oooh shiny!'

#548 ::: Brian Z ::: (view all by) ::: May 16, 2015, 11:15 PM:

@546

Do you expect many voters will follow slates under this new system? How many slates?

#549 ::: Steve Halter ::: (view all by) ::: May 16, 2015, 11:30 PM:

Brian Z@548:The general idea is to render slates moot. In the past slates were not a problem as they didn't happen. The new voting system will act to remove the leverage of minority slates.

#550 ::: Brian Z ::: (view all by) ::: May 16, 2015, 11:34 PM:

@549 render them moot, or help ensure they'll succeed in getting something on the ballot?

#551 ::: J Thomas ::: (view all by) ::: May 16, 2015, 11:56 PM:

@550 Brian Z

render them moot, or help ensure they'll succeed in getting something on the ballot?

If you have enough people to run a successful slate, you don't need a slate to get something on the ballot. Get everybody to vote for one thing, and then they have four more votes to vote for anything they like.

It cleans up the evidence, too. If you have a slate, a cluster analysis will reveal it to anybody who has the data and the expertise. But if a bunch of people happen to vote for one candidate, statistics from the ballots won't say much about that.

#552 ::: Brian Z ::: (view all by) ::: May 17, 2015, 12:11 AM:

@551 Yes. But in addition to disliking slates, traditionalists object to all rallying of Hugo voters through all campaigns. I'm wondering whether this proposal further incentivizes the crass campaigning that has been increasing ever since the internet hive mind was first awoken. (Future fan historians may end up pointing to that behavior as a major cause of dilution of the award.)

#553 ::: Keith "Kilo" Watt ::: (view all by) ::: May 17, 2015, 12:32 AM:

Brian:

The ultimate goal is to prevent any one group from keeping everyone else off of the ballot. That's really our -only- goal. Yes, it will also make slates basically pointless, because you don't need a slate to get a single popular work on the ballot.

You don't think the nomination process should change. We get that. But you are talking to a group of people in a thread expressly created to change the nomination process. We aren't likely to change your mind, I understand that. But you aren't likely to change ours either, given the whole purpose of this thread. There are a great many people who -do- think the process needs to be changed. We think this is the best way to meet those needs.

Kilo

#554 ::: colin roald ::: (view all by) ::: May 17, 2015, 12:49 AM:

Brian Z, I think you're probably actually a much sharper guy than the relentlessly thick-headed one you're presenting yourself as here. Do you seriously think we're trying to "help ensure" slates can get something on the ballot, or is that just the latest convenient way to pretend to misunderstand us?

#555 ::: Brian Gibbons ::: (view all by) ::: May 17, 2015, 12:58 AM:

Colin Roald @505: Next year, I virtually guarantee there will be some kind of Justice Kitten slate as a reply to the Puppies

I take a statement of absolute belief that there will be some sort of SJW slate next year as akin to stating a certainty that this winter's snow will taste like chocolate.

It belies a view of the world entirely at odds with my own, and indicates premises so far removed from what I consider reality that I can only shrug and suppose that we'll find out in due course. It's not really worth arguing over, and if it does turn out that you're right then, hey, free chocolate.

I would have an easier time believing that a group of actual cats will gather together, form a paramilitary organization and/or musical band, and head out into the streets to right wrongs under the name Justice Kittens, than I would in believing that "SJWs" could manage to get a meaningful number of people organized and all pointed in the same direction long enough to put together a counter-slate.

What I expect from next year is that there will be an awful lot more people nominating. This will mean that the categories that people care about will largely be Puppy-free, and ones that usually receive less attention will have at least a nominee or two that's sufficiently promoted (self- or otherwise) to stand out enough from the crowd to make it on the ballot even with Puppies still around. Presuming they are, of course. I would give at least 50/50 odds on VD simply declaring victory and moving on, now that he won't have the advantage of surprise, and the Sad variety are likely to retreat back into irrelevance as public scrutiny means they have to actually live up to their talk of democracy and public participation.

Oh, there will no doubt be people promoting works (their own or otherwise) more heavily, and I'd give odds on at least a dozen people (none of whom will be particularly high profile or with a meaningful following) trying to push different recommendation lists and/or slates. Some unified "SJW slate" on the level of the Puppies, however, is simply a laughable concept.

... none of which is necessarily that relevant to the SDV-LPE discussion. I think that this is a system that's more robust and will produce better results that the simple first-past-the-post nomination system being used now, and so hope that it ends up being adopted despite the fact that I expect the Puppies to become nothing more than a historical curiosity to explain to future Hugo historians what the asterisk next to this year's awards means.

#556 ::: Brian Z ::: (view all by) ::: May 17, 2015, 02:34 AM:

@553, 554

I fear some may be hearing things I'm not saying. Please indulge me in one last summation:

We stand at crossroads. In one direction, Hugos remain a charmingly retro fannish tradition in which a community of avid readers come together to identify the works most widely acclaimed as best of the year.

The other road leads to more internet tribalism and us vs. them mentality, twitter feuds, bestelling and wannabe bestselling authors setting themselves up as glorified thought leaders and opinion makers, cliques that may include some of the cleverest people on the planet organizing to hack a process that is by its very nature wide open, uncountable public and secret single work campaigns and slate campaigns, entrenchment of a culture in which fans are perpetually offended when they don't get what they want, and enough eligibility posts to sink the Titanic.

I'd like to see us go with charmingly retro.

Failing that, the Brave New World is at least interesting. We can do our best to help shape the voting culture that emerges. But it is going to be beyond anyone's control.

If nearly or more than half of the voters were to subscribe to multiple slates, then the game has probably already changed so much that rearguard action is pointless.

Ditto if multiple factions start going to battle with each other by rallying around favorite authors/works and driving other deserving potential nominees off the ballot. We don't "win" by getting a nominee we like most on the ballot. We "win" by giving a meaningful gift to an honored author representing the collective wishes of the members.

A new rule that does nothing but prevent a set of identical ballots from sweeping, period, is fine. If it were passed, I'd vote to ratify it.

Changing the goals to prioritize the "fairness" of everybody getting a thing they want over the spirit of fans reading widely and striving to reach consensus about what is excellent is not fine, in my view. It sends the wrong message, it will be that much harder to stay retro, and if the second road is taken will not make much difference and might make things worse.

#557 ::: P J Evans ::: (view all by) ::: May 17, 2015, 02:54 AM:

552
the crass campaigning that has been increasing

I think you need to be a little more explicit here, because the only really successful campaigns most of us have seen have been run by the juvenile canines.

(Authors saying that 'these are eligible' is not a campaign, nor is a recommended list.)

#558 ::: Laertes ::: (view all by) ::: May 17, 2015, 03:00 AM:

I'm honestly surprised that so few concern trolls have showed up.

#559 ::: Jeremy Leader ::: (view all by) ::: May 17, 2015, 03:07 AM:

Brian Z. @557: A new rule that does nothing but prevent a set of identical ballots from sweeping, period, is fine. If it were passed, I'd vote to ratify it.

Were you paying attention during the 2000+ comments on this thread and its 2 predecessor threads, where some very smart knowledgeable participants attempted to do exactly that, and came to the conclusion that this proposal was the best way to achieve that goal? Are you asking that the discussion be restarted from the beginning, because the consensus it reached wasn't what you wanted?

It seems that you're the only person who thinks the current proposal doesn't support "the spirit of fans reading widely and striving to reach consensus".

I've seen no evidence that doing nothing as you seem to want will lead to the "charmingly retro" outcome you claim to want. Given that the SP/RP supporters have said they don't want that, and their actions this year have shown that under the current system they have in fact prevented the Hugos from operating that way...

On the other hand, I've also seen no evidence that the SDV-LPE proposal will lead to the "Brave New World" you fear. Since SDV-LDE provably makes slates less effective, why do you believe SDV-LDE will lead to more slates, rather than fewer?

If you want to discuss other ways to improve the Hugos, there have been 4 threads so far about the Hugos that haven't been dedicated to coming up with and refining a proposal to improve the nomination mechanism; perhaps you should propose how you think the "crass campaigning" can be reduced on one of those threads?

(on preview, Laertes @558 was far more succinct than I!)

#560 ::: felice ::: (view all by) ::: May 17, 2015, 03:20 AM:

Brian Z @556: "We stand at crossroads. In one direction, Hugos remain a charmingly retro fannish tradition in which a community of avid readers come together to identify the works most widely acclaimed as best of the year."

There is no "coming together" with the current system. People nominate what they like, and the biggest tiny minorities win the finalist spots. Last year a whopping 23.1% of nominators backed the most popular novel; that means the other 77.9% didn't express an opinion on it. The least popular novel finalist had no support from 93.9% of nominators. 90.9% of nominators gave no support to the most popular short story.

"Changing the goals to prioritize the "fairness" of everybody getting a thing they want over the spirit of fans reading widely and striving to reach consensus about what is excellent is not fine, in my view."

There's never been any consensus. SDV-LPE aims to represent a range of opinions on what's excellent rather than just one minority's opinion, which is a good thing irrespective of whether or not there are any slates involved. And it does nothing to encourage "the other road" any more than the existing system; on the contrary, it removes the need for alternative slates to counter the puppies.

#561 ::: Brian Z ::: (view all by) ::: May 17, 2015, 03:27 AM:

No, I didn't say that this proposal would be responsible for leading us down one path or the other.

I intended that summation to be my final word on this thread - so thank you all for your additional questions (and accusations) but I think I've said my piece. Best of luck with your proposal, whatever form it takes.

#562 ::: JJ ::: (view all by) ::: May 17, 2015, 06:00 AM:

I would be able to contribute $20 to sponsor sending Keith or Jameson to Sasquan. Is someone in a position to set up a Kickstarter / indiegogo / GoFundMe campaign? (I am not, unfortunately.)

#563 ::: Jameson Quinn ::: (view all by) ::: May 17, 2015, 09:12 AM:

Keith has told me that he's interested in going, but not interested in fundraising for himself. I'm going to set up a crowdfund campaign for myself this afternoon, when I have time.

#564 ::: Jameson Quinn ::: (view all by) ::: May 17, 2015, 09:13 AM:

Cheradenine, if you are interested in being in on the crowdfund, please contact me today.

#565 ::: Shane ::: (view all by) ::: May 17, 2015, 09:48 AM:

I was going to say not Kickstarter, because it only pays if fully funded. But it might be that's what we want - decide the minimum you need to make it affordable, and if we can't reach it we can't.

I expect to subscribe, but I'd like to read your costings and reasons.

#566 ::: J Thomas ::: (view all by) ::: May 17, 2015, 10:18 AM:

In case this got lost in the argument with Brian....

I have an animation to help show how the voting system works. You can click on my name.

http://www.glowscript.org/#/user/J_Thomas/folder/hugo/program/Nominations

I'd like to get feedback whether something along these lines is worth doing.

#567 ::: Shane ::: (view all by) ::: May 17, 2015, 10:38 AM:

For Brian Z, from Jameson Quinn @ 219

The point of this system (SDV-LPE or whatever we call it) is to try to ensure that the eventual Hugo winner is excellent. The reason that the nominees should represent a variety of fans is not to make them happy, but because by representing a variety you avoid making the same mistake 5 times.

#568 ::: P J Evans ::: (view all by) ::: May 17, 2015, 11:14 AM:

Shane
Having a variety of nominees is what you get when there are lots of excellent works available; slates kill that. Look, for example, at the three non-slate novels: that's the kind of variety that we want.

#569 ::: junego ::: (view all by) ::: May 17, 2015, 11:58 AM:

J Thomas @ 566

http://www.glowscript.org/#/user/J_Thomas/folder/hugo/program/Nominations

That looks pretty cool, JT. The basic visual concept really gives a good graphic of how the system works. Obviously, a few more refinements would be great, like text to explain each event and maybe tie it to the sections of the proposal, slow or stop as the votes travel from the ballot to the nominees to show how many points are being dropped in, etc.

As a proof of concept I'd say "Do more." Good job.

#570 ::: junego ::: (view all by) ::: May 17, 2015, 12:53 PM:

Brian Gibbons @ 531
As was discussed upthread, I think the way this proposal is worded is slightly flawed. It's coming from the wrong perspective.
"Each member gets a single vote." No, they don't. Members don't get votes. Members don't allocate their votes. Members aren't doing anything under this section.
Members submit their nominations, and that's under a completely separate section (3.7.1). This section is all about telling the administrator how to tally those nominations.

Good point. I agree and support this approach. We should emphasize that there is NO change in how members nominate, only in how the nominations are tallied.

I suggest the following modification: I think that it makes more sense to leave the current 3.8.2 thru 3.8.7 as is. There are important instructions there that logically come before describing the new tally process. I've modified 3.8.1 (shown in bold) and will work on combining your, my and the original verbiage in a separate post. (Doing this on a tablet is a pain!)


Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations determined by the multi-round elimination process described in 3.8.8. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

#571 ::: junego ::: (view all by) ::: May 17, 2015, 12:55 PM:

Brian Gibbons @ 531
As was discussed upthread, I think the way this proposal is worded is slightly flawed. It's coming from the wrong perspective.
"Each member gets a single vote." No, they don't. Members don't get votes. Members don't allocate their votes. Members aren't doing anything under this section.
Members submit their nominations, and that's under a completely separate section (3.7.1). This section is all about telling the administrator how to tally those nominations.

Good point. I agree and support this approach. We should emphasize that there is NO change in how members nominate, only in how the nominations are tallied.

I suggest the following modification: I think that it makes more sense to leave the current 3.8.2 thru 3.8.7 as is. There are important instructions there that logically come before describing the new tally process. I've modified 3.8.1 (shown in bold) and will work on combining your, my and the original verbiage in a separate post. (Doing this on a tablet is a pain!)


Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations determined by the multi-round elimination process described in 3.8.8. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

#572 ::: colin roald ::: (view all by) ::: May 17, 2015, 01:28 PM:

J Thomas @566:

It's a promising start, but I fear still pretty opaque to anyone who hasn't carefully read the proposal. I do think something along these lines is worth doing, if it's taken to the point where it's clear to someone brand new to it (and who is not an election theorist).

Suggestions:

(1) I think it would help to have the ballots always shown, maybe in a split-screen kind of view. I think you want to animate the changes to the ballots when a work is eliminated, also.
(2) I think listing the works vertically and the scores horizontally (like the way Steve's simulator does it) would be a big improvement. You could write longer names for works (at the moment I'm not sure it's clear to a newcomer that "A", "B", "C" etc are meant to be the names of proposed stories), and you could write the scores as a simple decimal instead of the awkward improper fractions.

I'm not familiar with Glow, but it doesn't look too bad. If I have some time this evening I might be interested to poke a bit and see if I can help, if you like. Do you have this in version control anywhere? Poking around glowscript, I don't see anything, but maybe it's hiding.

Steve@534: I'll also volunteer to help work on your simulator, if you would like to share.

#573 ::: junego ::: (view all by) ::: May 17, 2015, 02:46 PM:

[PROPOSAL LANGUAGE]

OK, here's my shot at combining the original proposal, Brian Gibbons suggestions and my own. It is way too tedious trying to do strike through and italics on a tablet. Cutting and pasting loses all formatting. So I've just rewritten Section 3.8.8 and added 3.8.9 (Brian's 3.8.1.4, which covers withdrawals and I thought needed its own para.)

My rewrite of 3.8.1 is @
#571 ::: junego ::: (view all by) ::: May 17, 2015, 12:55 PM:

Here 'tis
*******************************************************
3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees on the ballot. If there is one nomination it gets 1 point, two nominations get 1/2 point each, three get 1/3 point each, etc. The total number of nominations for each separate work in the category on all ballots are totaled, each nomination counting as one tally. Points for the nominees change as each elimination round is started, the nomination tally for each nominee does not change.

3.8.8.1 Elimination rounds are as follows: At the beginning of the first round, all points for each work nominated are totaled. The two works with the least total number of (fractional) points will be compared. Of those works, the one that appears on the fewest ballots (the lowest nomination tally number) will be eliminated from contention.

3.8.8.2 Subsequent rounds begin by reallocating points as follows: The work eliminated in the previous round is treated as though it had never been nominated by the ballots in which it appeared. For example, if a ballot had 4 nominations at 1/4 point each in that category and one is eliminated, the remaining nominations are now allocated 1/3 point each. All points for all remaining nominees are recalculated and the next round proceeds as in 3.8.8.1. These rounds continue until there are 5 remaining nominees (except as described in 3.8.8.4 and 3.8.9).

3.8.8.3 In the event that two or more works are tied for lowest points and nomination tallies, the tie will be broken by looking back to previous rounds...[SOMEONE NEEDS TO DESCRIBE THIS METHOD, I DON'T FEEL I CAN DO IT JUSTICE]

3.8.8.4 If (due to a tie) elimination would reduce the number of remaining nominees to fewer than 5, then none of the works should be eliminated and all remaining works should appear on the final ballot.

3.8.9 Should a nominee decline nomination, be declared ineligible or otherwise be removed from the final ballot, and should the administrators choose to replace the removed nominee(s), they may do so by using the process described in 3.8.8 to determine who the final nominees would have been without contending with those removed. The administrators may then add to the final ballot all nominees who would have otherwise qualified, even if this requires listing more nominees than described in 3.8.1. This procedure may only result in the addition of nominees, not the removal of the previously qualified nominees.

#574 ::: J Thomas ::: (view all by) ::: May 17, 2015, 03:42 PM:

3.8.8.3 In the event that two or more works are tied for lowest points and nomination tallies, the tie will be broken by looking back to previous rounds...[SOMEONE NEEDS TO DESCRIBE THIS METHOD, I DON'T FEEL I CAN DO IT JUSTICE]

This is an argument for not doing it that way.

It's hard to explain.

Should we go back until the latest time they're different, and throw out whichever one had the lower score then?

What about going back to the beginning and throwing out the one that had the lower score more often?

What about going back to the beginning and throwing out the one which had the lower average score?

It isn't simple, it's hard to explain, there are multiple ways we could do it with no indication which is fairer. It doesn't get us anything useful.

The two which are tied for last place right now, are in fact tied for last place. They are the two which most deserve to be eliminated. It's possible if we gave one of them a reprieve it would do better and go on to win later, but that's true of a lot of the ones that get eliminated. That's true of any of the five with the most votes, which will all win unless one of them gets eliminated and doesn't get a reprieve.

I don't see the advantage of doing it.

Except for one minor advantage. If the last two slate members go head to head and tie, it would be fairer to the slate if they don't eliminate each other and have no slate members left among the finalists. No, wait, if the slate members are tied at the end then they will have been tied all the way through. Either they'll both be eliminated anyway, or they'll both be finalists. So not even that dubious advantage is real.

I must have missed the advantage somewhere.

#575 ::: Steve Halter ::: (view all by) ::: May 17, 2015, 04:20 PM:

Brian Z@550:

render them moot, or help ensure they'll succeed in getting something on the ballot?

Since you left @561, this is probably also moot, but helping to ensure slates get something on the ballot is clearly not the intent of the proposal.

Wind attacks the sand
Balance of voting impinged
Sand abides in dunes

#576 ::: J Thomas ::: (view all by) ::: May 17, 2015, 04:31 PM:

@572 colin roald

(1) I think it would help to have the ballots always shown, maybe in a split-screen kind of view.

That ought to be possible. It's a complication but probably not that difficult.

I think you want to animate the changes to the ballots when a work is eliminated, also.

That's probably harder for me, but I think possible.

(2) I think listing the works vertically and the scores horizontally (like the way Steve's simulator does it) would be a big improvement. You could write longer names for works (at the moment I'm not sure it's clear to a newcomer that "A", "B", "C" etc are meant to be the names of proposed stories),

I can write a description at the start for that. I let the layout and the code get co-adapted and did what looked simplest for both, and so changing it would require me to change my thinking around. That might be better done by someone else.

... and you could write the scores as a simple decimal instead of the awkward improper fractions.

That's valuable. I thought of them as more obvious to people who weren't math-inclined, and I didn't bother to reduce the fractions because there was always something higher priority. But if decimal approximations are clearer, that's an easy fix once it's laid out sideways to make room for them.

I'm not familiar with Glow, but it doesn't look too bad. If I have some time this evening I might be interested to poke a bit and see if I can help, if you like.

You'd be very welcome! It's very easy to use as long as you don't do anything complicated. You get a choice between javascript, Coffeescript, and Pyjeon. I find javascript excruciating. Pyjeon is just like Python that runs in javascript, except when it isn't. Lots of Python is missing including the min function, the sort function, sets, inheritance is subtly different, etc. It's all easy and obvious as long as you don't try to do anything that isn't easy and obvious.

Do you have this in version control anywhere?

My coding style makes me ashamed to show anybody my version control. I try out big variations, trying to do that mostly early in the process but also later when I find I've gotten to a dead end. I code faster than I can write workable specs without coding.

I could give you something that starts with the current version, but you could pick that up from the site about as easily.

Poking around glowscript, I don't see anything, but maybe it's hiding.

The site has no version control at all, but they keep a copy of your latest version for you. It's designed for visual demos. You can roll your own by installing the sources. If you want to putter around with code on their site, using Chrome gives by far better error messages. They host your code for anybody to look at, running their software, for free.


@569 junego

a few more refinements would be great, like text to explain each event

I can do more of that, but it's probably better to stretch it out into a website with a series of canvases you can click on to see animations. I did it with one canvas and changing text because that was what I got working quicker.

and maybe tie it to the sections of the proposal,

Interesting idea! It's still a moving target now, but....

slow or stop as the votes travel from the ballot to the nominees to show how many points are being dropped in, etc.

I set the slider so it can go slower whenever you want, down to almost stopped. I'll look at putting in a stop button, but I don't know ahead of time how to do that which leaves me wanting to do easier things first.

Thank you for the feedback!

#577 ::: felice ::: (view all by) ::: May 17, 2015, 05:02 PM:

J Thomas @574: "Should we go back until the latest time they're different, and throw out whichever one had the lower score then?"

Yes.

"It's possible if we gave one of them a reprieve it would do better and go on to win later, but that's true of a lot of the ones that get eliminated"

No, it's not; getting a reprieve isn't a random act of benevolence by the administrator, it's a product of sharing a ballot with another work that gets eliminated, and obtaining extra points from the eliminated work. If you randomly decide not to eliminate a work in one round, then the next round will be identical and it will be eliminated anyway. If you randomly eliminate the more popular work instead, that's obviously unfair. There is no way to give a reprieve to a non-tied work that should get eliminated.

"Except for one minor advantage. If the last two slate members go head to head and tie, it would be fairer to the slate if they don't eliminate each other and have no slate members left among the finalists. No, wait, if the slate members are tied at the end then they will have been tied all the way through. Either they'll both be eliminated anyway, or they'll both be finalists. So not even that dubious advantage is real."

Not necessarily. What if they're tied for 150 nominations each, but 148 people nominated both, and four other people each nominated just one of them (either because they were following the slate but chose not to stick to it exactly, or they weren't following the slate at all and coincidentally overlapped with it)? Lookback at previous rounds will likely find a difference for those four nominators. Lookback won't help with 100% overlap ties, but such ties between potential finalists should be pretty rare (they're still a problem on those rare occasions, though). And it's not just slates that could be affected; I listed a bunch of finalist ties from the last couple of years in #499, any of which could theoretically have tied for points and been compared for elimination under SDV-LPE. We don't have the data, so we don't know for sure.

#578 ::: Cally Soukup ::: (view all by) ::: May 17, 2015, 05:15 PM:

Junego @573

[PROPOSAL LANGUAGE]

May I suggest that in 3.8.8 instead of:

3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees on the ballot. [etc.]

it say:

3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees in that category on the ballot. [etc.]

thus making it extra-clear that the point is per category, not per ballot? For a second when I read it, I read that a point would be divided equally among all nominees on the WHOLE ballot, and it took me aback. On a second reading, I understood it, but personally, I'd rather err on the side of stating the obvious in a document like this when there's even a small chance of ambiguity.

#579 ::: J Thomas ::: (view all by) ::: May 17, 2015, 05:52 PM:

@577 Felice

"It's possible if we gave one of them a reprieve it would do better and go on to win later, but that's true of a lot of the ones that get eliminated"

No, it's not; getting a reprieve isn't a random act of benevolence by the administrator, it's a product of sharing a ballot with another work that gets eliminated, and obtaining extra points from the eliminated work.

It might get points from the other work that happens to have the same number of nominations and points. It might not. If it does, the other would get the same points if we chose that one instead, because we found that the other had been ahead on points more of the time.

But then, it might get points if it beats its next competitor, that would never have faced it except you decided it should be spared. What's good for one is bad for the other. If it had been eliminated then the competitor who shares ballots with it would have gotten those points and might not have had to compete on the next round after all.

If you randomly decide not to eliminate a work in one round, then the next round will be identical and it will be eliminated anyway.

Yes, if you put it up against exactly the same competitor again.

If you randomly eliminate the more popular work instead, that's obviously unfair. There is no way to give a reprieve to a non-tied work that should get eliminated.

Yes, it's obviously unfair to eliminate the more popular work. I'm not sure how it's more fair to look back in history to find some tiny difference to use to eliminate one and not the other. I see that it looks that way to you. I just don't see it myself, and I haven't seen an explanation that made sense to me.

Is there some technical reason why it works out better to keep one and not the other, apart from fairness?

"Except for one minor advantage. If the last two slate members go head to head and tie, it would be fairer to the slate if they don't eliminate each other and have no slate members left among the finalists. No, wait, if the slate members are tied at the end then they will have been tied all the way through. Either they'll both be eliminated anyway, or they'll both be finalists. So not even that dubious advantage is real."

Not necessarily. What if they're tied for 150 nominations each, but 148 people nominated both, and four other people each nominated just one of them (either because they were following the slate but chose not to stick to it exactly, or they weren't following the slate at all and coincidentally overlapped with it)? Lookback at previous rounds will likely find a difference for those four nominators.

Well yes, that's possible. You might look back and find that at the very first round, one nominator picked slate work Alpha and also Evil Clowns of Gor, while another nominator chose slate work Beta, Evil Clowns of Gor, and also Evil Dentists of Gor, and both of the extra nominations were eliminated the first round, having gotten no other votes. So that difference means that slate work A is better than slate work B?

I can sort of see it if there's some important technical reason we must minimize ties, and other people won't let us flip a coin or go by first nomination date. In that case sure, find some random criterion you can base on the votes to say that one is a tiny tiny sliver of a defunct point better than the other. But what's wrong with ties that we have to go to such extremes?

And it's not just slates that could be affected; I listed a bunch of finalist ties from the last couple of years in #499, any of which could theoretically have tied for points and been compared for elimination under SDV-LPE. We don't have the data, so we don't know for sure.

There's on particular reason to expect two nominees that have the same number of nominations to also have the same number of points. It seems pretty unlikely, when the points start to add up. Unless they have the two lowest numbers of points they won't be compared, and won't be tied.

Without the data we can't be sure it wouldn't have happened, but it looks unlikely.

Anyway, it's very likely that finalists who tie wouldn't have been paired up for competition.

And what's wrong with the reasoning that if the rules are that we use SDE and they do get paired up for competition and eliminated, then that's how it's supposed to come out?

We have a collection of SDE voting systems that are pretty similar and get pretty similar results. What method should we use to choose among them? I like the simplicity of eliminating them both. (But then, I didn't like the simplicity of just using #6 if #1 got disqualified. I thought it was better to actually find out who won.) If it's basicly arbitrary, let's keep it simple. But if there's an important reason, I want to understand it.

#580 ::: junego ::: (view all by) ::: May 17, 2015, 06:14 PM:

Cally Soukup @ 578
Junego @573

[PROPOSAL LANGUAGE]

May I suggest that in 3.8.8 instead of:
3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees on the ballot. [etc.]
it say:

3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees in that category on the ballot. [etc.]

thus making it extra-clear that the point is per category, not per ballot? For a second when I read it, I read that a point would be divided equally among all nominees on the WHOLE ballot, and it took me aback. On a second reading, I understood it, but personally, I'd rather err on the side of stating the obvious in a document like this when there's even a small chance of ambiguity.

Excellent! Thank you, this is a great help. It can be difficult to proof your own writing because your brain knows what you *meant* to say. :^}

#581 ::: Jameson Quinn ::: (view all by) ::: May 17, 2015, 06:17 PM:

Personally, I think that debating back and forth about the tiebreakers is pretty much the epitome of bikeshedding. In more than half of the cases, a tiebreaker rule will never even be invoked; in the overwhelming majority of the remaining cases, it will not affect the final outcome; and in the overwhelming majority of the tiny remaining fraction cases where it does matter, it only matters in terms of deciding which of two essentially-equivalent works from the same voting slate ends up as a finalist. So really, who cares. Let's just decide on something and move on.

It would be inappropriate for me to argue the above, and then also say, "...so just do what I say." So you all know which option I prefer, but I'm not going to argue for that again here. I say: let Keith decide. Or, if he'd rather, he can decide how to decide, as long as there's a time limit. If he wants us to have a poll, so be it. If he thinks he knows best and he's just going to call it for tiebreaker X, I support him 100%, no matter what X is.

If spending extra time now on the decision helps us build support for the eventual proposal, then it's well worth it; but if we're all just arguing over the aesthetics of an aspect that will almost certainly never make a difference that matters, it's a waste of time. For me, I suspect it's more of the latter, and I say we should leave it up to Keith. If he thinks I'm being too impatient, I'll accept that answer too.

#582 ::: felice ::: (view all by) ::: May 17, 2015, 07:09 PM:

J Thomas @579: "It might get points from the other work that happens to have the same number of nominations and points. It might not. If it does, the other would get the same points if we chose that one instead, because we found that the other had been ahead on points more of the time."

True, but that's no reason to not give either a chance. If it was a work you nominated, would you rather a 50% chance of your nomination surviving, or a 0% chance? If it was two works you nominated (and it will be for some if not most nominators, if the tie matters) would you rather a 100% chance of one of your nominations surviving, or a 0% chance? If you were the author, would you prefer a 50% chance of your work becoming a Hugo finalist, or a 0% chance? For everyone directly affected, using a tiny difference to distinguish between then is better than eliminating both.

"Is there some technical reason why it works out better to keep one and not the other, apart from fairness?"

Not aside from fairness, no. But fairness seems like quite sufficient justification to me.

"You might look back and find that at the very first round, one nominator picked slate work Alpha and also Evil Clowns of Gor, while another nominator chose slate work Beta, Evil Clowns of Gor, and also Evil Dentists of Gor, and both of the extra nominations were eliminated the first round, having gotten no other votes. So that difference means that slate work A is better than slate work B?"

Not really, no, but keeping one is better than eliminating both, and it's an acceptable deterministic method of choosing one.

"There's on particular reason to expect two nominees that have the same number of nominations to also have the same number of points. It seems pretty unlikely, when the points start to add up. Unless they have the two lowest numbers of points they won't be compared, and won't be tied. Without the data we can't be sure it wouldn't have happened, but it looks unlikely."

Most of the time, yes, they'll have different points, but it's not that unlikely they'll be the same. By the time potential finalists are up for elimination, it's likely that everything else on the ballots that nominated the tied works will have been eliminated, in which case they'll definitely have the same number of points. And if some of them also nominated another finalist, the numbers are likely to be low enough that matching numbers aren't unlikely.

If two potential finalists in short story (A and B) are tied on 40 nominations each, and overlap by 20 nominations, it doesn't matter what else the overlapping ballots nominated, and of the 20 who nominated just A and the 20 who nominated just B, maybe 5 from each nominated another surviving work? Those don't seem like unrealistic numbers to me. A and B would be tied on 27.5 points or less, and could well be up for elimination against another potential finalist with relatively low points but higher nominations - one of the last two works on a slate would be an obvious possibility. I'd say it's better to have one slate work and either A or B as finalists than it is to have two slate works and neither A nor B.

In 1984, 69% of people who nominated one of the best novel finalists didn't nominate any of the others. 54% didn't nominate anything else in the top ten. Only 5% nominated more than two of the finalists. The chances of potential finalists which are tied on nominations being tied on points as well when they're up for elimination is not that low.

#583 ::: Seth ::: (view all by) ::: May 17, 2015, 07:20 PM:

Slight disambiguation:

3.8.8.2 Subsequent rounds begin by reallocating points as follows: The work eliminated in the previous round is treated as though it had never been nominated by the ballots in which it appeared.

Change to

3.8.8.2 Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are treated as though they had never been nominated on the ballots in which they appeared.

That covers the case in which the previous round eliminated more than one (due to ties), and makes it clear that eliminated works don't reappear.

Also, if the 5% rule remains in force, I still think there should be a Round 0 that eliminates everything that isn't eligible, whether due to actual ineligibility (e.g. published in the wrong year), the 5% rule, or early withdrawal.

#584 ::: felice ::: (view all by) ::: May 17, 2015, 07:27 PM:

Jameson Quinn @581: "Personally, I think that debating back and forth about the tiebreakers is pretty much the epitome of bikeshedding"

Given that we're pretty much agreed on everything except the bikesheds, I don't see that as a huge problem. But I'd say it's more like discussing a safety feature to deal with a highly unlikely reactor meltdown.

"it only matters in terms of deciding which of two essentially-equivalent works from the same voting slate ends up as a finalist"

It matters in terms of deciding whether one of two essentially-equivalent works from the same voting slate ends up as a finalist. I don't care how ties are broken, as long as they are broken; J Thomas is arguing against tie-breaking.


Seth @583: "3.8.8.2 Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are treated as though they had never been nominated on the ballots in which they appeared."

Yep, that's good.

"Also, if the 5% rule remains in force, I still think there should be a Round 0 that eliminates everything that isn't eligible, whether due to actual ineligibility (e.g. published in the wrong year), the 5% rule, or early withdrawal."

Saying "points are divided between all eligible works on each ballot" at the relevant point in the rules should be sufficient to cover that, rather than an explicit round one. Works below 5% currently aren't eligible unless there are fewer than three works above 5%.

#585 ::: felice ::: (view all by) ::: May 17, 2015, 07:27 PM:

Jameson Quinn @581: "Personally, I think that debating back and forth about the tiebreakers is pretty much the epitome of bikeshedding"

Given that we're pretty much agreed on everything except the bikesheds, I don't see that as a huge problem. But I'd say it's more like discussing a safety feature to deal with a highly unlikely reactor meltdown.

"it only matters in terms of deciding which of two essentially-equivalent works from the same voting slate ends up as a finalist"

It matters in terms of deciding whether one of two essentially-equivalent works from the same voting slate ends up as a finalist. I don't care how ties are broken, as long as they are broken; J Thomas is arguing against tie-breaking.


Seth @583: "3.8.8.2 Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are treated as though they had never been nominated on the ballots in which they appeared."

Yep, that's good.

"Also, if the 5% rule remains in force, I still think there should be a Round 0 that eliminates everything that isn't eligible, whether due to actual ineligibility (e.g. published in the wrong year), the 5% rule, or early withdrawal."

Saying "points are divided between all eligible works on each ballot" at the relevant point in the rules should be sufficient to cover that, rather than an explicit round one. Works below 5% currently aren't eligible unless there are fewer than three works above 5%.

#586 ::: J Thomas ::: (view all by) ::: May 17, 2015, 07:46 PM:

@581 Jameson Quinn

If spending extra time now on the decision helps us build support for the eventual proposal, then it's well worth it; but if we're all just arguing over the aesthetics of an aspect that will almost certainly never make a difference that matters, it's a waste of time.

That sounds good to me.

When I try to make an animation of it, the lookback is a great big extra hassle that I'm not sure how to present.

I don't know which woykd pass easier, I suspect that the lookback complication is a liability but for all I know for people new to it, the double elimination would look bad.

Maybe there's a way to ask them what they'd prefer, or whether it matters to them.

#587 ::: J Thomas ::: (view all by) ::: May 17, 2015, 08:20 PM:

@582 Felice

If it was a work you nominated, would you rather a 50% chance of your nomination surviving, or a 0% chance?

I would prefer that mine had a 50% chance, of course. But if it was somebody else's and not mine, then of course I'd prefer it had a 0% chance so my own nominations' chances were improved. And it's not mine far more often than it's mine.

"Is there some technical reason why it works out better to keep one and not the other, apart from fairness?"

Not aside from fairness, no. But fairness seems like quite sufficient justification to me.

I don't see that it's fair at all. But usually when people argue fairness they don't reach an agreement unless they're willing to spend an effort to look at deep issues. And it is not very central to the proposal unless people at the board meeting take sides on the fairness issue.

So my first thought is to just do something and stop talking about it and hope everybody forgets it, because if the people at the meeting notice and argue about what's fair and can't come to an agreement then they're likely to vote it down. And if it comes to an argument they're far less likely to agree than we are.

But my second thought is that somebody like Brian Z is likely to do his level best to start a controversy about it, taking whichever side we don't take, hoping for that exact outcome.

So we do better to have simple clear arguments why the way we do it is fair.

Here's my argument why double elimination is fair:

At each step we look at the two with the lowest scores and we eliminate the one with the lowest number of nominations. That's how we do it. If they're tied on nominations and tied on points there isn't a ha'penny worth of difference between them. They are the two lowest. Of course we eliminate both.

What else would we do? Flip a coin? Rummage around in old records to see how they used to do? We don't do that for anything else. There are lots of ways to choose between them -- we could for example choose the one that's first in alphabetical order if they have an even number of nominations, and reverse alphabetical order if the votes are odd. But there's no obvious reason to do anything except eliminate the two lowest when they tie for last place.

#588 ::: Brian Z ::: (view all by) ::: May 17, 2015, 08:39 PM:

557-60,75

I didn't mean to stomp off, I just worried taking up too much of the thread was inconsiderate. I'll try to be brief:

We seek consensus to honor a widely acclaimed author: I loved Moorcock. But I know everyone wants to honor LeGuin and it is just and right to nominate her too. You think Herbert should be on there, but you agree LeGuin should be too. That guy over there loves Michael Crichton, but even he grudgingly admits The Left Hand of Darkness was also a breakthrough. We all nominate LeGuin without being afraid supporting her will work against getting our own personal favorite one of the other slots. We have failed to agree on Moorcock, Herbert and Crichton, but we achieved consensus on LeGuin.

I don't know why people have lost sight of this - maybe in the last decade or two we have spent too much time voting people off the island and being team this or team that, and we've lost sight of the spirit in which the award is intended.

This still have a chance of the system working for all fiction categories except short story, which is broken due to the fragmented readership. We need to address that through informal mechanisms: passing around shortlists and reviewing and discussing them. We should probably be spending more money to buy magazines. But fixing this mess with a technical rules change? No chance.

Don't worry: With all this enthusiasm, would-be slate voters won't have a prayer.

#589 ::: JJ ::: (view all by) ::: May 17, 2015, 09:31 PM:

Brian Z., #588: "I don't know why people have lost sight of this - maybe in the last decade or two we have spent too much time voting people off the island and being team this or team that, and we've lost sight of the spirit in which the award is intended."

You're showing your Puppy side again. This has not been going on in the last decade or two in the Hugos.

We have not lost sight of anything here.

You have bought into the Puppy lies about the Hugo Awards hook, line, and sinker. You can accuse the Puppies of "voting people off the island and being team this or team that" -- because that's exactly what they've been doing.

But your making that sort of accusation of the rest of the Worldcon members, who have been participating in good faith, goes beyond the pale -- and you owe everyone here an apology for the fact that you've just accused them of this.

#590 ::: Duncan J Macdonald ::: (view all by) ::: May 17, 2015, 09:36 PM:

Brian Z @588

We seek consensus to honor a widely acclaimed author...

No, we don't seek consensus. Nowhere in any of these discussions or in the history of Hugo nominating or voting has there been consensus on either the slate or the losers.

I really don't care who you loved -- I'm nominating my loves, and when it comes time for the final voting, I'll vote for the one I like the best, or dislike the least, or No Award if I believe that all the offerings are dreck.

And none of that is consensus.

... maybe in the last decade or two we have ... sight of the spirit in which the award is intended.

No, it is a popularity contest plain and simple. The rules change as proposed restores the pop in popular.

Don't worry: With all this enthusiasm, would-be slate voters won't have a prayer.

A devoutly to be desired result.

#591 ::: Keith "Kilo" Watt ::: (view all by) ::: May 17, 2015, 09:40 PM:

Hi all -

I'm having surgery on my knee tomorrow (I took a rapier shot under the kneecap -- I don't recommend the experience...), so I'll be out of pocket pretty much all day. On Tuesday I'll combine all of your suggestions into the next draft of the proposal and see if we can't start to lock it down.

Regarding lookback or no lookback, for me, so long as we can explain them simply, either one works about as well as the other. As Jameson says, it's really pretty much a non-issue from a practical standpoint. Since simplicity (or the lack thereof) seems to be the major issue with lookback, let me come up with a description on Tuesday for each method, and then I'd like to get a sense of what the general group thinks -- just a "this one, that one, or I don't care". I don't think a formal poll is really necessary. I have some ideas as to how to handle the issue either way during the actual business meeting, so I think we'll be fine no matter what.

Regarding sponsorships, as Jameson mentioned, while it's a little bit of a financial issue for my family, I'm not comfortable asking for money, crowdsourced or otherwise. We will make it happen so that I can be there at least for the three days of the business meeting. So, I think we need to discuss who is able to go to Worldcon and who can be at the meeting -- even if you are there just to support the proposal, the more bodies we have, the better. Even if you can't go in person, if you are a supporting member, you are eligible to sign on as a submitter. If the proposal meets your satisfaction, I'd be honored if all of the people here -- even if you've been just following along -- would consider being a signer. We only need at least one person to present at the meeting, and if the group is okay with it, I'm willing to do that. But how many others of you are going to Worldcon and would be willing to attend the business meeting as well? If you'd post under the [SIGNING] or [ATTENDING] tag (or both, obviously), I'll add your name to the next draft of the proposal. Please include your actual name, as some of us have different handles. if you aren't comfortable posting your real name here, just email me -- apparently Googling my handle gives my email address most readily. :)

Thanks all, and I'll check in with you again on Tuesday.

Kilo

#592 ::: Tammy Coxen ::: (view all by) ::: May 17, 2015, 09:47 PM:

[SIGNING] [ATTENDING]

I'll be at the meeting, and would like to be a cosigner for the proposal. (And that's my real name)

#593 ::: Tim Illingworth ::: (view all by) ::: May 17, 2015, 10:04 PM:

KW@591: you should probably plan for a Sunday Business Meeting as well, as the nomination numbers aren't usually released until after the Hugo ceremony, which is Saturday this year.

I expect all the Hugo business to be postponed until after that, which will give us the Preliminary BM on Thursday, Main on Friday, Site Selection on Saturday and Hugo Proposals on Sunday.

#594 ::: junego ::: (view all by) ::: May 17, 2015, 10:08 PM:

Seth @ 583

[PROPOSAL LANGUAGE]

3.8.8.2 Subsequent rounds begin by reallocating points as follows: The work eliminated in the previous round is treated as though it had never been nominated by the ballots in which it appeared.
Change to

3.8.8.2 Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are treated as though they had never been nominated on the ballots in which they appeared.

That covers the case in which the previous round eliminated more than one (due to ties), and makes it clear that eliminated works don't reappear.

The language change is fine with me, it covers all possible cases that way. I may be misunderstanding, but I think the new tie breaking rule (LookBack) would do away with tied works challenging another work? Either way, this change covers it.

Also, if the 5% rule remains in force, I still think there should be a Round 0 that eliminates everything that isn't eligible, whether due to actual ineligibility (e.g. published in the wrong year), the 5% rule, or early withdrawal.

Section 3.8.5 already says this, so we don't have to. If the proposal to get rid of the rule is passed I,d assume that's a section that would disappear.

#595 ::: Soon Lee ::: (view all by) ::: May 17, 2015, 10:20 PM:

Keith "Kilo" Watt #591:

Best wishes with the surgery.

(At least it wasn't an arrow in the knee?)

#596 ::: junego ::: (view all by) ::: May 17, 2015, 10:26 PM:

Keith "Kilo" Watt @ 591

[SIGNING]

I can't attend but would be honored to sign even tho I've barely done anything. Name is CJ Cabourne

I'm having surgery on my knee tomorrow (I took a rapier shot under the kneecap -- I don't recommend the experience...), so I'll be out of pocket pretty much all day. On Tuesday I'll combine all of your suggestions into the next draft of the proposal and see if we can't start to lock it down.

Ouch on the kneecap!

Yes, we need to consolidate.

Regarding lookback or no lookback, for me, so long as we can explain them simply, either one works about as well as the other. As Jameson says, it's really pretty much a non-issue from a practical standpoint. Since simplicity (or the lack thereof) seems to be the major issue with lookback, let me come up with a description on Tuesday for each method, and then I'd like to get a sense of what the general group thinks -- just a "this one, that one, or I don't care". I don't think a formal poll is really necessary. I have some ideas as to how to handle the issue either way during the actual business meeting, so I think we'll be fine no matter what.

I think simplicity and appearance of fairness are key.

Regarding sponsorships, as Jameson mentioned, while it's a little bit of a financial issue for my family, I'm not comfortable asking for money, crowdsourced or otherwise. We will make it happen so that I can be there at least for the three days of the business meeting. So, I think we need to discuss who is able to go to Worldcon and who can be at the meeting -- even if you are there just to support the proposal, the more bodies we have, the better. Even if you can't go in person, if you are a supporting member, you are eligible to sign on as a submitter. If the proposal meets your satisfaction, I'd be honored if all of the people here -- even if you've been just following along -- would consider being a signer. We only need at least one person to present at the meeting, and if the group is okay with it, I'm willing to do that. But how many others of you are going to Worldcon and would be willing to attend the business meeting as well? If you'd post under the [SIGNING] or [ATTENDING] tag (or both, obviously), I'll add your name to the next draft of the proposal. Please include your actual name, as some of us have different handlle.

If you're going to represent me at the meeting, I would feel better if I could help defray your costs. Hope you will reconsider.

#597 ::: Steve Halter ::: (view all by) ::: May 17, 2015, 10:31 PM:

[SIGNING]

Steven Halter

I haven't decided on attending yet or not.

#598 ::: P J Evans ::: (view all by) ::: May 17, 2015, 10:57 PM:

[SIGNING]
Supporting, and already signed on the 4/6 proposal.
But can do this one also, I think.

The only way I'm likely to have convention budget again is with a lottery win. (I don't play the lottery: the times I've bought tickets, I've been lucky to get one number. This is The Universe Speaking.)

#599 ::: colin roald ::: (view all by) ::: May 17, 2015, 11:23 PM:

J Thomas @586:
When I try to make an animation of it, the lookback is a great big extra hassle that I'm not sure how to present.

It's not obvious how to visually display what's going on, but it's not that difficult to track internally for the purposes of tallying votes. I think it's the kind of detail that could be left for those who care to deal with in text, but glossed over in tutorial demos.

#600 ::: David Goldfarb ::: (view all by) ::: May 17, 2015, 11:27 PM:

Keith "Kilo" Watt @591: [SIGNING]
I'm a supporting member and will not attend, but I'd be happy to be listed as a signer of the proposal.

#601 ::: felice ::: (view all by) ::: May 18, 2015, 02:32 AM:

J Thomas @587: "I would prefer that mine had a 50% chance, of course. But if it was somebody else's and not mine, then of course I'd prefer it had a 0% chance so my own nominations' chances were improved. And it's not mine far more often than it's mine."

If it's not one of yours, why should your opinion be taken into account? How about we make all works by authors born in January ineligible? Should we assume the majority of authors will approve, because it improves their chances, or should we agree it's a bad idea because it's unfair to authors born in January, irrespective of what authors born in other months might think?

And it's just as likely that it will be one of yours as that it will make a difference to you if it isn't; in the overwhelming majority of cases, simultaneous elimination of someone else's nominations will not change whether or not any of your nominations make the final five. Actually, no, it must be twice as likely (approximately) - there are two works each with a 50% chance of making the final five, and just one work that one of them could replace. Ie, it's in everyone's interest to prevent simultaneous elimination; if it affects you at all, it will hurt you twice as often as it will help you.

"At each step we look at the two with the lowest scores and we eliminate the one with the lowest number of nominations. That's how we do it. If they're tied on nominations and tied on points there isn't a ha'penny worth of difference between them. They are the two lowest. Of course we eliminate both."

And if the two lowest aren't tied on nominations or points, they're still the two lowest; why not eliminate both of them anyway? SDV-LPE doesn't work that way. Simultaneous elimination is akin to a divide-by-zero error.

The most common tie that matters would be a tie for 6th place between works with overlapping support. Eliminate one, and the other gets enough points to place it above the current 4th; then the former 4th and 5th are compared for elimination. There are very few other works left at this stage, so the chances that works tied for nominations will also tie for points is pretty high, and ties for nominations aren't that rare. It's an issue that won't happen every year, but it will happen. And it's not just slates that will be affected; you could expect significant overlap for different episodes of the same TV show, stories by the same author, stories published in the same magazine, standout books in the same subgenre, etc.

#602 ::: J Thomas ::: (view all by) ::: May 18, 2015, 09:39 AM:

@601 felice

"I would prefer that mine had a 50% chance, of course. But if it was somebody else's and not mine, then of course I'd prefer it had a 0% chance so my own nominations' chances were improved. And it's not mine far more often than it's mine."

If it's not one of yours, why should your opinion be taken into account?

For that matter, if it is one of mine why should my opinion be taken into account? We're trying to do what's fair, we don't need to listen to special pleading from the people who individually stand to benefit.

The whole argument is bogus.

Suppose that we reprieve one of two tied, and then it loses later. That was no real benefit to it. But during the time it survived maybe it knocked out some others, so the final winners were different. That's good for the ones who won, and bad for the ones who lost. It comes out exactly even.

OK, suppose we reprieve one and later it does win. Because the order of challenges has changed, it and maybe some others are finalists that otherwise would not be, and others are knocked off. It's good for the ones it helped and bad for the ones it hurt. Again it comes out exactly even.

Why are the ones that win due to lookback more deserving than the others that lose because of lookback? I see that you want to be nice to one of the two that tied for last place, but if it makes a difference it will be at the expense of some who were not tied for last place.

"At each step we look at the two with the lowest scores and we eliminate the one with the lowest number of nominations. That's how we do it. If they're tied on nominations and tied on points there isn't a ha'penny worth of difference between them. They are the two lowest. Of course we eliminate both."

And if the two lowest aren't tied on nominations or points, they're still the two lowest; why not eliminate both of them anyway?

Because the rule is to eliminate the lowest.

SDV-LPE doesn't work that way.

We're in the process of deciding what rules SDV-LPE will actually follow. If you want to propose eliminating the two lowest when they aren't tied then go ahead, but I won't support it.

The most common tie that matters would be a tie for 6th place between works with overlapping support. Eliminate one, and the other gets enough points to place it above the current 4th; then the former 4th and 5th are compared for elimination. There are very few other works left at this stage, so the chances that works tied for nominations will also tie for points is pretty high, and ties for nominations aren't that rare.

If the two lowest both get eliminated at 6th place, then we're done.

I understand that you want to give one of the last-place works a break so it will have a chance to beat out one of the winners. I don't see why you favor one of the tied last-place nominees over one of the winners.

#603 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 10:09 AM:

@602: The problem with that argument is that it proves too much. Any decision between two works is zero-sum in some sense; one wins and one loses. And if you go back and recapitulate the reasons why the voting system decided which one wins, then it is "fair", because the voting system was designed using impartial rules. But that's circular reasoning.

There is a valid reason to prefer not to eliminate both: the system without dual elimination is Droop-proportional, and under certain assumptions of voter utility, any Droop-proportional system gives higher utility than any non-proportional system.

Of course, you can argue the other way, too. If dual elimination happens, it usually harms slates; and if you thing slate winners are inherently illegitimate, then that's a good thing. But if you want to make that argument, you have to make it explicitly, and not rely on circular reasoning.

I want to reiterate that I don't think the tiebreaker matters much and I support whichever solution can resolve the issue for the community. The argument I'm making here is not that J Thomas is wrong, but rather that you're both right, and it doesn't really matter.

#604 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 10:29 AM:

So, I'm figuring out which crowdfunder to use. There are a few questions.

Number one, what if Keith changes his mind, and decides to accept some money? I would not hesitate to split the money with him 50/50 if that were to happen. No offense, Keith; I respect your stance, but I also respect your right to keep an open mind.

Number two, what happens if the funding goes over or under the budget I've established? I'd be inclined to give the money to Electology.org in those cases (or, if Keith asked for it, 100% to him). The work I've done on this proposal, and the time I've spent here, was motivated by my desire to further understanding of and comfort with election reform. I think that some money going to support a charitable organization to further that cause is reasonable. Do others strongly object to that stance? (I want to respect all opinions here, but in the end, I can't end up like the boy and the old man and the donkey, so I'm going to have to make the call myself.)

#605 ::: Seth Gordon ::: (view all by) ::: May 18, 2015, 11:00 AM:

[SIGNING] (but not, alas, attending)

Good job, everyone!

#606 ::: Tammy Coxen ::: (view all by) ::: May 18, 2015, 11:08 AM:

I think as long as you are clear on where any over-donations go, then people can decide if they want to give money after you've funded.

#607 ::: Idumea Arbacoochee, Bad Cop ::: (view all by) ::: May 18, 2015, 11:19 AM:

Brian Z @ 550, 552, 556, 561, and 588:

So remember when I said back in 338 that This thread is about one proposal to change it, which can then be brought to the conversation at Sasquan, and that we're going to leave "whether" conversations aside from this point forward?

Also, remember that I said Brian Z, if you're looking to get disemvoweled, now you know how?

I let you ignore these things once, because you said you would have your say and then leave. It was disruptive to a conversation that was about something else, but calling you on it would have been more disruptive. So I was being nice, at the expense of the other people here who are trying to have a particular, focused conversation. I apologize to them for permitting it; it was an error in judgment.

Because then you came back to do it again. So now hear this. The very next time you try to derail this conversation, you're losing vowels.

I know you will then go back to File 770 and wave your martyr's palm in triumph. I double-dog-dare you to link to this comment when you do, because if you don't, I'm going to. I will also include a detailed discussion of why it's not appropriate behavior, neither in a moderated environment in general nor in this conversation.

You will not enjoy the experience.

#608 ::: Ginger ::: (view all by) ::: May 18, 2015, 11:36 AM:

[SIGNING]

Been following along since the first post, and admiring all the hard work put into the proposal. Can't attend, but signing with my real name, Ginger Tansey.

#609 ::: J Thomas ::: (view all by) ::: May 18, 2015, 11:39 AM:

@603 Jameson Quinn

@602: The problem with that argument is that it proves too much. Any decision between two works is zero-sum in some sense; one wins and one loses. And if you go back and recapitulate the reasons why the voting system decided which one wins, then it is "fair", because the voting system was designed using impartial rules. But that's circular reasoning.

Yes, exactly. If we want to say that one way is fairer, we need some reason why that nominee deserves to win more than the other does. My reasoning is simple, at every other step we care only about the number of nominations and the current score. Why should we add an exception and a complication to change the results?

There is a valid reason to prefer not to eliminate both: the system without dual elimination is Droop-proportional, and under certain assumptions of voter utility, any Droop-proportional system gives higher utility than any non-proportional system.

You've proved that this system with tie-breaking is Droop-proportional? That's interesting! I'm sure nobody has ever proven that before. I'd be interested to see your proof. It would probably change my mind.

I wonder what assumptions of voter utility it's true for.

The argument I'm making here is not that J Thomas is wrong, but rather that you're both right, and it doesn't really matter.

It matters most if it affects the proposal passing. We disagree about which way is fairer, and if other people disagree about that, it could affect the vote. That is its main practical significance.

If you want to argue that the Hugo nominations are won by small splinter groups with little consensus, none of whom have read that much of the literature and so the results are kind of random, and it really doesn't matter whether the results are fair as long as we stop the slates that people are upset about, I guess I could go along with that. But I don't think we should tell the business meeting that we don't care whether it's fair.

#610 ::: Steve Wright ::: (view all by) ::: May 18, 2015, 12:07 PM:

OK. I've not been chiming in on this one, but I have been following with interest.

It is clear that Something Must Be Done; this is a fair and well-reasoned Something, and I would be happy to support it.

[SIGNING] - Steve Wright is my real name, but I'm "Steve Wrigh" to Sasquan (and they have bigger things to worry about than fixing that typo).

#611 ::: Tom Whitmore ::: (view all by) ::: May 18, 2015, 01:33 PM:

I'm attending, may or may not be at the business meeting at various times. Have not followed closely enough here to sign. I'm interested in the discussion, but it's a lower priority than some other issues for me right now.

#612 ::: Annie Y ::: (view all by) ::: May 18, 2015, 01:45 PM:

[SIGNING] [ATTENDING]

At least the plan for now is to be in Sasquan and to be in the business meetings. :)

Full name is Yoana Yotova

Keith "Kilo" Watt @ 591:

Ugh. Good luck with the knee and hopefully you will recover fast. :)

#613 ::: Chris Battey ::: (view all by) ::: May 18, 2015, 02:00 PM:

[SIGNING] [MAYBE ATTENDING]

Which is to say, I'll be at Sasquan, and I intend to attend the business meetings, but my wife and I will have a three-year-old in tow, and I'm sure most of you know how a toddler tends to randomize plans.

#614 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 03:34 PM:

OK, one final funding question, before I set it up:

Would people rather have the money be tax-deductible, by going through electology.org; or would you rather have the option of giving it to Keith if he changes his mind? I realize I can't do both, and I have to decide up front. I think tax-deductible is better, because I don't actually expect him to change; but I've said I would keep that option open, so I want to check before I have to close it.

(Note that aside from Keith, there's also Cheradenine, who also gave well-informed help in the early stages of developing this proposal, and whose presence in the BM would I think be helpful, but who doesn't seem to still be keeping tabs on the discussion. I'd be willing to split any monies with them too if it came to that.)

In the interest of actually telling people what the fundraising will involve: I'll be looking for money for a Sasquan ticket, a round-trip Amtrak ticket there from Boston (yes, that's 1 week of just travel time, but since I can work on the train I'd rather do that than a plane trip with attendant CO2.), and a bit of money for spartan expenses while I'm there. So that's about $1000, which, based on the number of people who have responded positively to even an indefinite idea, seems probably doable.

#615 ::: felice ::: (view all by) ::: May 18, 2015, 03:53 PM:

J Thomas @602: "Because the rule is to eliminate the lowest."

Not the two lowest. Eliminating more than one at a time means potentially throwing out all of a person's nominations at once without ever reallocating any of their points.

"I understand that you want to give one of the last-place works a break so it will have a chance to beat out one of the winners. I don't see why you favor one of the tied last-place nominees over one of the winners."

Because it would have more points than the winner it replaces. If F and G are tied, and have fewer nominations than E, but would have more points than D if one was eliminated, that means E must share a significant number of nominations with the other finalists. Replacing a work nominated by the same people as other finalists with one nominated by a different set of people is the whole point of SDV-LPE. (Or if E beats D in the elimination, D either has fewer nominations than F or G, or also overlaps with other finalists, but to a lesser degree than E - either way, getting rid of it is still the correct result.)

Eg if ABCDE is a slate that's swept the category, replacing E with F or G is obviously a good thing.

If either F or G was declined or found ineligible before the initial count, one of them would be a finalist. If either F or G had 1/5 of a point more or less, one of them would be a finalist: if one of the F or G nominators added D and E to their ballot, that would make D or E lose - a clearly pathological result.

#616 ::: Cat ::: (view all by) ::: May 18, 2015, 04:34 PM:

I'm a supporting member and would be happy to sign. My name is also on the 4/6 proposal as a co-sponsor but I believe I am allowed to sign this one also.
[SIGNING]
I can be reached at ["handle" under which I am posting]faber @ toast.net if you need to contact me in e-mail to arrange it.

#617 ::: Joshua Kronengold ::: (view all by) ::: May 18, 2015, 04:42 PM:

[SIGNING] [ATTENDING]

Good luck on the surgery, Keith. I ended up giving up fencing [classical and rapier] after my miniscus surgery, although frankly I'm not sure exactly what happened to cause it to tear (fencing is hardly a bad bet, though).

#618 ::: Annie Y ::: (view all by) ::: May 18, 2015, 04:50 PM:

@614 and the whole funding question

I can help with a plane ticket or 2(provided that it is not too late and awards are not available anymore) - have enough miles so I can spare some so keep this as an option as well together with the funding campaign.

#619 ::: Andy H. ::: (view all by) ::: May 18, 2015, 04:56 PM:

[SIGNING]

As a continual lurker on these threads, I admire and endorse this proposal. I am a supporting member as 'Andy Holloway'.

#620 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 05:06 PM:

@618: I suggest you make that offer to Keith. He's said he doesn't want to take people's money, but miles may be a different story. Meanwhile, I'm looking at doing it by train, since (a) I've always wanted to take that train and (b) I can afford the time at that moment, as long as I do work while on the train and (c) I want to minimize my carbon footprint.

#621 ::: Duncan J Macdonald ::: (view all by) ::: May 18, 2015, 05:14 PM:

Keith @ 591

[SIGNING]

That's my real name up there.

#622 ::: Annie Y ::: (view all by) ::: May 18, 2015, 05:28 PM:

@620 Have fun with the train ride. :)

Keith @ 591

Keith, let me know if that would help?

#623 ::: J Thomas ::: (view all by) ::: May 18, 2015, 06:01 PM:

@615 felice

Eliminating more than one at a time means potentially throwing out all of a person's nominations at once without ever reallocating any of their points.

That sounds reasonable. It might be somebody's last two surviving nominations. If he has another that would get the points that's fine, but maybe he doesn't.

Of course, after they are spared they might knock out various other people's only surviving nomination. It's hard to predict. Any time a work gets eliminated it might be taking away some nominator's last surviving nomination.

It does seem like a bad thing when a ballot loses its last nomination, but that does happen. In 2013, for novels there were 1113 ballots cast and 717 votes for finalists. At least 396 ballots had no finalist on them at all, more than a third, and probably rather more than that since some ballots would have two finalists.

If we want to make that happen less, maybe instead of looking back to find some slight edge, we should instead look at how many ballots are down to their last chance, and eliminate the nominee that has fewer of those ballots.

"I understand that you want to give one of the last-place works a break so it will have a chance to beat out one of the winners. I don't see why you favor one of the tied last-place nominees over one of the winners."

Because it would have more points than the winner it replaces. If F and G are tied, and have fewer nominations than E, but would have more points than D if one was eliminated, that means E must share a significant number of nominations with the other finalists. Replacing a work nominated by the same people as other finalists with one nominated by a different set of people is the whole point of SDV-LPE. (Or if E beats D in the elimination, D either has fewer nominations than F or G, or also overlaps with other finalists, but to a lesser degree than E - either way, getting rid of it is still the correct result.)

That makes sense. We'd like to have the best combination of winners. So if F is spared and then it beats E, F must be better than E.

Either F has more votes than E and therefore F deserves to win.

Or maybe when G is eliminated that gives F more points so D and E must compete and one of them loses, and even if the loser has a lot more votes than F, still F is better because F has more points.

Wait a minute, though. One way, we say that F is better because it has more votes, and the other way we say it's better because it has more points. What combination of votes and points does it really take to make it better?

If the important thing is to have the best ones win, according to some combination of votes and points, shouldn't we look at ABCDEFG and find the combination of them that is best?

See, if we paired them off at random we could get lots of different finalists, and some of them would be better than others. When we have a way to tell which ones are best, shouldn't we just use that instead of using the haphazard ordering we get from sequential redistribution of points?

Well, but we have a voting system that works OK. I think that to claim that one way of handling ties gets a better result than the other, we'd need some way to tell what's better, and then we could work out which is better on average. But we don't have anything like that.

It's easy to tell which approach is simpler, but simplicity isn't the only thing we care about. It's just the only one that's easy to measure.

#624 ::: P J Evans ::: (view all by) ::: May 18, 2015, 06:41 PM:

623
There's no reason for you to worry about x being the last item left in someone's list for a category. That's Not Hugo's Problem.

It happens very, very frequently - or hadn't you noticed, with the 1984 data, how very many lists people submitted had one or two items in a category?

Choices: YMMV, and you shouldn't expect to get what you want.

#625 ::: Cassy B. ::: (view all by) ::: May 18, 2015, 07:07 PM:

[SIGNING] [SUPPORTING]

My name is Pynhqvn Ornpu, rot13ed to prevent casual googling.

#626 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 07:34 PM:

I've made a GoFundMe campaign. Since all funds will be going through the Center for Election Science, donations are tax deductible.

Thanks a lot for any support you can afford! It's really great to feel that the work I've done here is appreciated.

#627 ::: felice ::: (view all by) ::: May 18, 2015, 08:26 PM:

J Thomas @623: "It does seem like a bad thing when a ballot loses its last nomination, but that does happen"

Losing the last nomination isn't a problem, it's losing last and second last in one go without ever increasing the last's points.

"Wait a minute, though. One way, we say that F is better because it has more votes, and the other way we say it's better because it has more points. What combination of votes and points does it really take to make it better?"

The combination specified by SDV-LPE. And F will never have more nominations than E, otherwise E would lose the elimination comparison between E, F, and G (if I'm right about how we pick the works to compare - it's ambiguous in the various drafts).

"If the important thing is to have the best ones win, according to some combination of votes and points, shouldn't we look at ABCDEFG and find the combination of them that is best?"

Now you're arguing against the whole SDV-LPE system. And what happens if ABCFD and ABCGD are equally good, which seems very likely?

"When we have a way to tell which ones are best, shouldn't we just use that instead of using the haphazard ordering we get from sequential redistribution of points?"

The SDV-LPE order isn't haphazard; popular works are only considered for elimination when they share a lot of nominations with other popular works. If everyone bullet voted or nominated one popular work and a bunch of unpopular works, it would produce the same order as the current system.

#628 ::: J Thomas ::: (view all by) ::: May 18, 2015, 08:54 PM:

@624 P J Evans

There's no reason for you to worry about x being the last item left in someone's list for a category. That's Not Hugo's Problem.

I agree. Felice suggested that one reason her preferred method to handle ties is better, is that it is less likely to remove a ballots' last choice along with the second-last choice.

I say that if preserving ballots' last choices a longer time is what we want, there are better ways to get it. But I don't think that's a particularly important goal. I could be wrong, and I might see it her way when I understand better.

#629 ::: Brian Z ::: (view all by) ::: May 18, 2015, 09:02 PM:

@590 No, we don't seek consensus. Nowhere in any of these discussions or in the history of Hugo nominating or voting has there been consensus on either the slate or the losers.

Exhibit A: http://file770.com/?p=22247

@607 wave your martyr's palm in triumph.

This screen name is Abi Sutherthand? Hi Abi. No, it's fine. I was replying to others' specific questions and concerns, but of course you don't have to host a discussion of criticisms of the proposal if you don't want to. Hope to have a chance to chat with you again another time.

#630 ::: P J Evans ::: (view all by) ::: May 18, 2015, 09:21 PM:

629
The screen name on 607 is Idumea Arbacoochee, not abi. You need to pay attention to things like that.

(We also care about capitalization.)

#631 ::: P J Evans ::: (view all by) ::: May 18, 2015, 09:23 PM:

also 629
So your 'evidence' is from 1953? That's really solid and convincing. Not.

#632 ::: JJ ::: (view all by) ::: May 18, 2015, 09:39 PM:

Obviously my definition of the word "consensus" differs vastly from Brian Z's.

Maybe he misunderstood, and thought you were talking about antonyms of "consensus".

#633 ::: Laertes ::: (view all by) ::: May 18, 2015, 10:08 PM:

The concern troll thing was getting pretty old, so the transition to super snotty is a refreshing change.

#634 ::: Brian Z ::: (view all by) ::: May 18, 2015, 10:25 PM:

@630, 631

I think the moderator has appeared to warn that this discussion is off topic. Isn't that Abi Sutherland? If I have misidentified the moderator, my apologies.

That's "evidence" that early Hugo awards saw people trying to convince each other about what was worthy. But we've been told that the history of the Hugos is off topic, so would you like to join me over on File 770 to discuss it? Another place is fine too.

#635 ::: Carrie S. ::: (view all by) ::: May 18, 2015, 10:41 PM:

Brian Z: Since PJ Evans appears to have been too subtle for you, perhaps it should be made explicit that abi's screen name around here does not begin with a capital letter. It's considered polite to spell a person's name in the way that they spell it.

One also generally attempts to use the name a person gives, rather than any other name one may be aware they use. Insisting on linking two identities as if one were bravely making a revelation ranges from faintly amusing (as here, where we are all perfectly aware of the nature of Idumea's relationship to abi) to actively harmful.

#636 ::: Carrie S. ::: (view all by) ::: May 18, 2015, 10:42 PM:

Brian Z: Since PJ Evans appears to have been too subtle for you, perhaps it should be made explicit that abi's screen name around here does not begin with a capital letter. It's considered polite to spell a person's name in the way that they spell it.

One also generally attempts to use the name a person gives, rather than any other name one may be aware they use. Insisting on linking two identities as if one were bravely making a revelation ranges from faintly amusing (as here, where we are all perfectly aware of the nature of Idumea's relationship to abi) to actively harmful.

#637 ::: P J Evans ::: (view all by) ::: May 18, 2015, 11:01 PM:

If the gnomes will allow, from 'Worldcon' on Wikipedia:
11th ... 1953 ... Philcon II ... Philadelphia, Pennsylvania ... United States ... GOH: Willy Ley ... Attendance: 750

That's totally comparable to Sasquan. /sarcasm

#638 ::: Brian Z ::: (view all by) ::: May 18, 2015, 11:12 PM:

@635, I'm not a regular here. I only read the blog occasionally when a topic of special interest comes up. In trying to understand who is the moderator of this thread (and follow moderators' instructions) I noted the name "Abi Sutherland" is listed (capitalized) just above this text box I'm typing in now as "the moderator most frequently onsite" in addition to four others, but if I have inadvertently mis-addressed or misidentified anyone in any way, I do apologize.

#639 ::: JJ ::: (view all by) ::: May 18, 2015, 11:27 PM:

Brian Z.: "I'm not a regular here. I only read the blog occasionally when a topic of special interest comes up."

Since you've been showing a similar sort of disrespect over at File770, it seems clear that you've never bothered to educate yourself at Netiquette. You would be well-served to do so, instead of just assuming that you know everything already.

Rule 3: Know where you are in cyberspace


#640 ::: Laertes ::: (view all by) ::: May 18, 2015, 11:29 PM:

You suck at flouncing.

#641 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 11:57 PM:

Hey, I have an awesome idea! Since we have a troll in the thread, it would be a nice change of pace if I turned into a spammer, and started posting regular updates (in which I just happen to mention my organization's URL) every time somebody kicks in another few dollars.

Or, I could not do that, even though I'd kinda like to, because even in my short time here, I've learned that it would be cosidered insufferably rude, even if I tried to be clever or humble or whatever.

Brian Z, you are obviously capable of googling the people you're talking to, to help you figure out what you can say. If we googled you, and we had your social engineering skills, what could we figure out to say to you that would make you carry out your terrible threat to leave us alone?

#642 ::: abi, who is also Idumea ::: (view all by) ::: May 19, 2015, 01:07 AM:

Actually, I don't care about capitalization for myself (though I try to be meticulous about it for others). I do care about good conversation, and let's all let Brian Z actually leave without poking him on the way out, shall we?

#643 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 03:02 AM:

Re tiebreakers: I'm not sure what's unclear here. The issue is that of removing entire demographics--possibly ones that would make the ballot otherwise with a strong showing, just because of an edge case.

That's not desirable. If you consider the following ballot set:

10 x A
11 x B
12 x C
13 x D
14 x E
15 x F
40 x GHIJK

Without a tiebreaker, G, H, I, J, and K will all be eliminated at the first elimination -- removing 34% of the ballots (and all their nominations) at the first pass. This is obviously not proportional -- those ballots should get 1-2 nominations--they have a third of the votes!

Add a bulletproof tiebreaker -- (although frankly, I know of no bulletproof one beyond coinflips other than "first encountered on any ballot") and the non-bullet slate (even if it's random happenstance--which while unlikely, is still totally possible) will take three of the entries (given the terrible voting all around [just because I needed a simple example], its fairest calculatable share).

#644 ::: Shane ::: (view all by) ::: May 19, 2015, 04:19 AM:

I couldn't see how you donate with a nickname, so an onymouse. There ya go.

Moderator - Would you find Cheradenine's email address and give her/him a heads up?

#645 ::: J Thomas ::: (view all by) ::: May 19, 2015, 06:12 AM:

@643 Joshua Kronengold

10 x A
11 x B
12 x C
13 x D
14 x E
15 x F
40 x GHIJK

Without a tiebreaker, G, H, I, J, and K will all be eliminated at the first elimination -- removing 34% of the ballots (and all their nominations) at the first pass. This is obviously not proportional -- those ballots should get 1-2 nominations--they have a third of the votes!

I find that argument kind of convincing. That is a bad outcome. Of course, by far the easiest way to get this result is when GHIJK is a slate.

What happens the other way?

GHIJK get a score of 8, and H is removed.

GIJKA then have a score of 10. K is removed.

GIJ have a score of 13.3. A, B, C, D are removed.

Winners GIJFE.

If A is removed instead of one of the others the outcome is the same. They deserve a third of the slots and they get nearly two thirds! The slate doesn't deserve 3 slots, but I guess denying them one is worse. It's more important that the slate gets at least one finalist than that it doesn't get three.

You give a rare case where our voting system gives a bad result either way. That will happen sometimes. Of course, if the slate isn't perfect and G and I get a few more votes than the rest, then there's a strong chance that balance is restored and we will get the normal outcome of GIJFE, with three slate winners.

tl,dr----------------

Felice presented an interesting result earlier. The numbers 1 4 11 29 have a special relationship which has the result that whenever the votes are tied and the scores are tied, both nominees have the same number of votes with only themselves surviving, the same number of votes with one other, the same number with two others, and so on.

So double ties might be common among the rare ones --there might be two that each have one vote shared with four others, or two that each have one vote shared with four others, etc. They get less common for nominees with more votes. But then they start to get more common late in the process as there are fewer and fewer ballots that have more than one surviving vote. So while it will be rare apart from slates, when it does happen it's likely to be important.

So in Felice's example with F and G tying for 5th place, in the reasonable case that F and G each have about 8% of the vote, when G is eliminated that will increase F's score by something less than 8% of the ballots F shares with others. Unlikely this will bring the score above D and E both, so in most cases F will lose to D or E and the outcome is the same.

Sorry for the digression. I was interested and didn't want to just delete it.
--------------

You have a plausible argument that is likely to affect people's feelings about the issue independent of the facts. I can't think of any argument that would affect them more. So you are right.

How should we break ties?

Add a bulletproof tiebreaker -- (although frankly, I know of no bulletproof one beyond coinflips other than "first encountered on any ballot") and the non-bullet slate (even if it's random happenstance--which while unlikely, is still totally possible) will take three of the entries

The current proposal is to look back for a minor past difference. Here is another idea -- look at the results of each outcome. When one is eliminated (or both are) that will increase other scores. Of the three choices, pick the one that has the least effect on the other nominees.

That is, pick the one that changes the relative order of scores the least.

Say it's a tie between F and G, and the scores are now ranked ABCDE. If you eliminate F and the scores are then ranked ABCED, but when you eliminate G the scores are ranked ABDEC, then eliminate G because it has affected three others instead of two.

We want to be fair to the tied nominees F and G by treating them different, but we should also try to be fair to all the others. Eliminating F and G will change the same *number* of ballots, but they're likely to have different effects. Why should C be hurt by arbitrary resolution of a tie that had nothing to do with C? Better to reduce that effect.

And maybe we don't need a bulletproof tie-breaker. If we apply a reasonable tie-breaker and it doesn't work because they're from a slate and identical except ballot order, why bend over backward to pick one? They still have a great big advantage over everybody else because the rest waste most of their votes on things that can't win, while the slate can take things that couldn't win and win with them -- because they're focused. The outcome where 30% of ballots gets 60% of finalists is not implausible. It can easily happen that 15% of ballots gets 60% of finalists. If slates have a very rare disadvantage, what's wrong with that?

#646 ::: Jameson Quinn ::: (view all by) ::: May 19, 2015, 08:39 AM:

So I lied about SDV-LPE without a tiebreaker being Droop proportional. It's just Dummet (Hare) proportional, not Droop. I also think, but have not yet thought of a proof, that if it fails Droop proportionality for some group, it must also succeed for some other group. But anyway: with simultaneous elimination it's clearly not any of those things, because slates can be eliminated wholesale.

#647 ::: J Thomas ::: (view all by) ::: May 19, 2015, 10:03 AM:

@646 Jameson Quinn

So I lied about SDV-LPE without a tiebreaker being Droop proportional. It's just Dummet (Hare) proportional, not Droop.

I'd be interested in seeing an argument for that, too. My email address is easy to find.

But anyway: with simultaneous elimination it's clearly not any of those things, because slates can be eliminated wholesale.

Sure, which says that maybe those are criteria we don't want. Being nice to slates in rare corner cases is not my highest priority, anyway. Slates still have a giant advantage with SDE-LPE and will get finalists out of proportion to their numbers, because of the way our nonslate nominators vote.

Anyway, thinking over the arguments by Felice and Joshua Kronengold, I think I see a reasonable argument against double elimination.

SDE-LPE is already quasi-chaotic. A small difference in votes can lead to a great big difference in outcomes. I'm reasonably sure that a pathological case could be constructed where a change in one vote would result in all five finalists being different. (It would be a lot of work to construct it, though. You agree it's possible, right?)

Similarly, with double elimination, we could get a case where the 2nd and 3rd nominees by votes go up against each other fairly early, and both are eliminated leaving #1, #4, #5, #6, and #7 to be the finalists. But with just one vote different #2 survives and then eliminates #4, #5, #6, and #7 leaving #1, #2, #8, #9, and #10 for finalists.

This sort of thing is slightly less likely if we have some way to avoid ties. Any reasonable thing we can do to make the system a tiny little bit less quasi-chaotic, is good.

So we should look for a good criterion to break ties. Currently I think rather than finding some insignificant difference between the tied candidates, we should choose one based on the principle of least chaos. Make the winner be the one that has the least significant effect on the other surviving nominees.

Instead of saying "This is fair because one of the tied nominees was a tiny tiny bit worse" we can say "This is the one that's most fair to all the others."

#648 ::: Derry Earnshaw ::: (view all by) ::: May 19, 2015, 10:07 AM:

[SIGNING]
I'm a supporting member because I'm too far away to attend practically - and too time-zone displaced to comment before someone else has said what I think. I have been following these discussions and think this is a good balance of fairness, robustness and understandability.

Jameson Quinn @620 I believe the Business Meeting Chair is also very fond of trains.

#649 ::: Pfusand ::: (view all by) ::: May 19, 2015, 10:40 AM:

J Thomas @ 643

I think you made a mistake in your computing. You started with Joshua's example:

10 x A
11 x B
12 x C
13 x D
14 x E
15 x F
40 x GHIJK

[Joshua's example deleted.]

What happens the other way?

GHIJK get a score of 8, and H is removed.

GIJKA then have a score of 10. K is removed.

This is where I think the error is. With H removed, its points are redistributed, and now G, I, J, and K have 10 points, sure. (As does unchanged A.) But they still have 40 nominations, and should win over A with only 10 nominations.

#650 ::: Cally Soukup ::: (view all by) ::: May 19, 2015, 10:43 AM:

Jameson Quinn:

It probably won't mesh with your schedule, but just in case it does (and for anyone else who likes trains and Worldcon) I thought you might like to know that a group of fans has put together "Traincon", for people who are planning on taking the Empire Builder to Worldcon. http://sasquan.org/wp-content/uploads/2014/10/Traincon2flyer.pdf

#651 ::: Jameson Quinn ::: (view all by) ::: May 19, 2015, 10:50 AM:

@647: "(It would be a lot of work to construct it, though. You agree it's possible, right?)"

Yes, I agree it's possible. But if it were Brian Z making that claim, I'd challenge him to give me an example, because I also agree that it would not be easy to construct one (I expect it would take more than one gimmick to build it, and getting more than one gimmick to work in the same scenario is not easy.)

#652 ::: J Thomas ::: (view all by) ::: May 19, 2015, 11:01 AM:

@649 Pfusand

I think you made a mistake in your computing. You started with Joshua's example:

....

This is where I think the error is. With H removed, its points are redistributed, and now G, I, J, and K have 10 points, sure. (As does unchanged A.) But they still have 40 nominations, and should win over A with only 10 nominations.

You're right. A loses now, instead of having a 1/5 chance of being the one that loses. So at the next step GIJK still have 10 points and one of them loses (unless we do it the other way and throw in B in which case B loses.) Then B wins against A, C wins against B, D wins against C, GIJ win against D, leading to three slate wins plus EF.

If we do it the other way and take everything with the lowest two scores, GIJK wins all the way leading to finalists FGIJK. Four slate wins.

I included an extra case that can't happen, but which leads to the same result.

#653 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 11:43 AM:

Hi all! Made it through surgery with no problems, though I won't be walking much for the next couple of days. Thanks for all the good wishes!

[TIES]

@643 (and following)
The reason these types of scenarios are so implausible is that you're only likely to get the distributions you cite when there are very few total ballots. For example, in 2013, the category with the fewest ballots counted was Best Fanzine, with 820 ballots (the lowest literary category was 1067 ballots). To get Joshua's situation, we would need to multiply all those ballots by a factor of 10. How likely is it that you're going to get 100-150 independent people bullet voting a single work with no overlap whatsoever? And how likely are you to get 400 slate ballots with absolutely perfect discipline? This is why none of these cases show up in realistic data. Without explicit coordination among the non-slate nominators, it's very close to statistically impossible for it to occur. And if even 5 of those 400 ballots differ from the slate, then there are no ties at all.

If someone really wanted to be fair about it, you could just advise slate coordinators to make sure at least 5 people change their nominations. But I think odds of perfect slate discipline among those 400 ballots is pretty small.

Kilo

#654 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 11:54 AM:

It's not untrue that 60% slate wins in my constructed example isn't ideal -- but it's one of the less non-ideal of the possible exaples--and since the other demographics don't have more than 15%, probably no better than any other plausible solution (RAV/N (linear weights) would get us exactly the same answer -- two slate wins off the bat, followed by two non-slate wins, and the slate would take another win at 40/3). If you consider the slate broken up into three rough demographics of 13.3 people each, it seems entirely fair that they'd get 3 wins alongside E and F -- as if you eliminate the no-hope demographics, they're three of the biggest pluralities. One of the features (and penalties) of SDV-LP is that it tries to give neither an advantage nor a disadvantage to people voting in a larger clump, after all.

I think "past scores" is a reasonable tiebreak for most cases. The place where it fails, though, is slates that are too successful -- which is going to happen on occasion, if not very often. Obviously, in straight voting ties in an early round, there may be no past scores to use.

The reason I referred to "first seen on first ballot" is that it's actually bulletproof -- the ballots are entered in a particular order, and while the order on them doesn't normally matter, there are no ties.

Of course, while I forgot to mention it, random is also a bulletproof tiebreaker by that measure.

I'd be pretty happy, fwiw, with what amounts to random without spelling it out:

Ties not including last place should always be broken (rather than removing all tied candidates), since if the tie is not moot (with the winner being eliminated regardless of the result), it is the same voters being affected if both are removed. The administrators should therefore use a fair and arbitrary method of breaking such ties in cases where no other method is specified.

#655 ::: J Thomas ::: (view all by) ::: May 19, 2015, 12:14 PM:

@651 Jameson Quinn

"(It would be a lot of work to construct it, though. You agree it's possible, right?)"

Yes, I agree it's possible. But if it were Brian Z making that claim, I'd challenge him to give me an example, because I also agree that it would not be easy to construct one

It turns out to be surprisingly easy with double-elimination, given one different ballot.

ABCDE 50 ballots
F 10
G 10
H 10
I 10
J 10
FGHIJ 1 ballot

ABCDE each have score 10.
FGHIJ all have scores 10.2.

So ABCDE are all double-eliminated, leaving FGHIJ.

Without the ballot FGHIJ, every nominee has score 10 and FGHIJ are eliminated, leaving ABCDE.

It can happen by changing one vote.

ABCDE 50 ballots, 10 points.
FG 9 ballots, 4.5 points for each
GH "
HI "
IJ "
JF "

FK 2 ballots, worth 1 points for each
GK
HK
IK
JK

K 5 ballots
So everything has 10 points. With simultaneous elimination, everything but ABCDE is eliminated the first round.

Suppose that there were only 4 K ballots.

Then K has 9 votes and 9 points. K is eliminated first. Then FGHIJ each have 11 points, and in the second round ABCDE are all eliminated.

One single changed vote gives an entirely different result.

I don't see in detail how to do that with invent-a-way-to-break-ties. The one with the most votes would win either way. We can turn that into a tie with the 2nd most votes, but one of the two of them would still win. So we'd need to make #2 lose the first way, and win the tie the second way, and the details would depend on precisely which arbitrary way we chose to break the tie. It looks tedious.

SDE-LPE is quasi-chaotic generally. A small difference in votes can lead to an extreme difference in outcomes. But usually it doesn't. And yet it could. One mailed ballot with the right postmark that arrives late, could give you everything we've worried about declined nominations.

#656 ::: P J Evans ::: (view all by) ::: May 19, 2015, 12:32 PM:

One mailed ballot with the right postmark that arrives late

Don't worry so much about it. You can't plan for every situation that might come up. Deal with the likely ones first, then the more probable of the unlikely ones.
(The post office is more of a constant. One of the final ballots showed up in December 1984. With black marks that seemed to say it had been stuck in a machine.)

#657 ::: J Thomas ::: (view all by) ::: May 19, 2015, 12:40 PM:

@653 Keith "Kilo" Watt

The reason these types of scenarios are so implausible is that you're only likely to get the distributions you cite when there are very few total ballots.

Sure. We look at examples with a few ballots because we can understand them, and they demonstrate some of the strange things that can happen.

When there are a whole lot of ballots we can't expect to understand it at all. Lots of weird things will happen that we don't even notice. But usually they won't call themselves to our attention in any way that forces us to notice them. So that's good.

#658 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 12:44 PM:

@655

With small-number statistics, you will almost always have problems in any field. This is the big problem in education research, for example, and makes it very difficult to draw any conclusions from a small data set. It's why education research is so hard. Again, let me reiterate: Getting these distributions when scaled up to statically-valid sample sizes is -highly- improbable.

It is not chaotic (which means "sensitive dependence on initial conditions"); "quasi-chaotic" doesn't have any rigorous meaning that I know of.

I think at this point, we should stick with one of the two remaining tie break choices, with lookback or without. The number of cases where this is going to make a difference in real data is vanishingly small as it is. The cases you are talking about are even less likely. If it really bothers someone in the meeting that there is even a chance to eliminate more than one work at a time, then they can propose an amendment based on some additional criteria at that time. I really see no value in complicating the system for cases that may literally never occur.

Kilo

#659 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 01:18 PM:

Kilo: I don't see any value in overcomplication, but I do see some value in specifying priority (ie, "break ties some other way" rather than "throw out tied potential nominees if you can't break a tie").

#660 ::: J Thomas ::: (view all by) ::: May 19, 2015, 01:29 PM:

@658 Keith "Kilo" Watt

With small-number statistics, you will almost always have problems in any field.

This is not statistics. We haven't particularly come up with names for pathological cases, much less defined them well enough to look for statistics about how often they will arise in plausible data. Our single real data set is more than 30 years old and doesn't represent current conditions. Our simulated datasets have potential flaws. So our efforts at statistics have been entirely amateur-3-minutes. What we've done with small understandable examples is to look at what's possible, not what's likely.

Getting these distributions when scaled up to statically-valid sample sizes is -highly- improbable.

We'd have to see, wouldn't we? It does look rare. Ties are rare. In 2013 there weren't any among finalists, and in 2012 there were only 2. Felice has made an argument that vote-ties in late stages are likely to be score-ties too. There just aren't many others left to share ballots with, and most of the ballot-sharing was done with the ones that are already gone. If the #5 and #6 works both have N nominations, not that unlikely they will both have N points, or possibly both have N-0.5 points or N-1 points etc.

I haven't tried to quantify that yet, but there could be something to it. I personally believe that you're probably right that any problems of any sort are unlikely, but I'd rather we make a serious effort to test that before we claim it's true.

It is not chaotic (which means "sensitive dependence on initial conditions"); "quasi-chaotic" doesn't have any rigorous meaning that I know of.

[sigh] Are you feeling belligerent? Five minutes of websearch would have given you some meanings. The one I'm using is given here.

http://www.hawaii.edu/uhmfs/documents/2010_11/20110216_reso_votingmethodsalternatives.pdf

quasi-Chaos: Small changes in voting can lead to large changes in outcome.2

I think at this point, we should stick with one of the two remaining tie break choices, with lookback or without. The number of cases where this is going to make a difference in real data is vanishingly small as it is.

I believe and hope that you're probably right.

If it really bothers someone in the meeting that there is even a chance to eliminate more than one work at a time, then they can propose an amendment based on some additional criteria at that time.

I want to have a simple clear argument ready for why the way we propose it is fair. If they get into an argument over two versions that neither one sound good, they may throw the whole thing out.

So far the best argument I've seen in favor of double elimination is that lookback is basicly arbitrary and complicated and may have unpredictable results.

And the best argument I've seen in favor of lookback is that double elimination is unfair to slates and may eliminate things disproportionately, plus it may have unpredictable results.

Maybe we should go with a variation on Joshua Kronengold's suggestion and punt. "The administrators should therefore use a fair and arbitrary method of breaking such ties."

#661 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 01:33 PM:

[TIES]

All:

As promised, here are the descriptions of the system with and without lookback. As noted, with lookback, the only time we eliminate more than one work per round is if the works have been tied for both points and nominations from the very beginning. As we have discussed, about the only way this can matter is if there is a slate with perfect disciplne. It would be trivially easy for a slate organizer to avoid this situation by just having five people not vote a perfect slate. I'm still not convinced we need anything more than that, but propose your "last-ditch" tie-break rule, and we'll discuss it. I am so far of the opinion that this should be handled via amendment if necessary. My thoughts are that we should explain the possiblity to the business meeting and if they see it as a problem, then fine.

I think I've described both systems (with and without lookback) in a fairly simple manner. Without lookback is simpler, obviously, but with lookback isn't too bad, in my opinion. What are your thoughts on these two, and then we'll hash out the last-ditch tie-breaker issue.

[I will post the text in the next message.]

Kilo

#662 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 01:43 PM:

[TIES]

Hugo Nomination System FAQs

1.Can you explain the system in plain language?
The important thing to remember is that nothing changes in how you nominate. If you think a work is Hugo-worthy, then nominate it. That’s all. All we are doing at this stage is throwing names into a hat. The final voting system, which actually chooses the winner, is unchanged. We could, in theory, simply put everyone’s nominations on the final ballot, but that would make for a very long ballot indeed. We therefore need to narrow the nomination list down. This system narrows down the list by eliminating the least popular works until only five (under current rules) finalists remain. Here are the basic steps to the elimination process:

a.You have one nomination “point” for each category that will be divided equally among the works you choose to nominate. So, if you nominate two works, each will get half a point; if you nominate three works, each will get one-third of a point, and so on.

b. All the points given to each work from all ballots are added together. The two works that got the least number of points are eligible for elimination. One of these works is the least popular and will be eliminated.

c. To determine which of these two works is least popular, we compare the total number of nominations they each received (that is, the number of ballots on which each work appears). The work that received the fewest total number of nominations is the least popular and now completely vanishes from the nomination process as though it never existed.

d. We start over for the next round and repeat the process, however, if one of your works was eliminated, then you now have fewer works on your nomination ballot. This means that each work gets more total points, since you aren’t dividing your point among as many works.


2. How are ties handled? [LOOKBACK]
First, there are two types of ties. If there is a tie for least or second-least number of points, then all the tied works are eligible for elimination. That’s straight-forward. It is possible, however, for two works to also have the same total number of nominations. In this case, we look at points again: the tied work with the fewest points is eliminated.

What if there is a tie for both total number of nominations and for total points? In this case, we just look back through the previous rounds to the first round in which they were not tied for points. The work which had the fewest points in that round is now eliminated. If the two works are tied for both points and total number of nominations all the way back to the first round, we just eliminate them both.

2. How are ties handled? [NO LOOKBACK]
First, there are two types of ties. If there is a tie for least or second-least number of points, then all the tied works are eligible for elimination. That’s straight-forward. It is possible, however, for two works to also have the same total number of nominations. In this case, we look at points again: the tied work with the fewest points is eliminated. If there is a tie for both total number of nominations and for total points, then we just eliminate both works.


#663 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 01:52 PM:

JT@660:
[sigh] Are you feeling belligerent? Five minutes of websearch would have given you some meanings. The one I'm using is given here.

No, I'm not feeling belligerent. My master's research is chaos theory, studying with Jim Yorke, who invented the term. I'm think I'm fairly knowledgable about the mathematical definitions of chaos. The definition in the article you cite is not mathematically rigorous, as I said.

If you want to cite reference articles, then you need to give complete references. This one isn't even signed. The footnote doesn't give the reference to support the claim -- situations matter, particularly with statistics (and yes, we are dealing with statistics).

That said, I'm not really interested in debating the minutia of terminology with you. We have a proposal to write. But you don't get to accuse me of being belligerent in order to discount something I've said, sorry. If I come across as being short with you, then I apologize.

Can we get back to work?

Kilo

#664 ::: Laertes ::: (view all by) ::: May 19, 2015, 02:04 PM:

Been following this, but not contributing, since big brains are clearly on the job and I've nothing unique to add. I'll open my mouth now that it's voting time.

It's not entirely clear to me which of the options (LOOKBACK or NO LOOKBACK) generates better results. It seems likely that the difference in results quality, whatever it is, is very small.

LOOKBACK is substantially more complex than NO LOOKBACK, and doesn't obviously offer substantially better results. I think it's more trouble than it's worth. I prefer NO LOOKBACK.

#665 ::: P J Evans ::: (view all by) ::: May 19, 2015, 02:25 PM:

[NO LOOKBACK]

KISS. I think some people keep forgetting this principle.

#666 ::: junego ::: (view all by) ::: May 19, 2015, 03:01 PM:

Kilo @ 662

2. How are ties handled? [LOOKBACK]
First, there are two types of ties. If there is a tie for least or second-least number of points, then all the tied works are eligible for elimination.That’s straight-forward. It is possible, however, for two works to also have the same total number of nominations. In this case, we look at points again: the tied work with the fewest points is eliminated.

I'm a bit confused about the bolded terminology. Not sure what you mean. Let me run down what I think would happen and see if I understand the process.

a) If there is a point tie at the least position, these two works challenge, nomination tallies are compared. If still tied, whatever tie-breaker is decided on is employed.

b) If there is a point tie at second-least position, nomination tallies for all works in least and second least position are compared. One should be eliminated (on points or nominations) and a new run calculated.

The tie-breaker would only be needed for works tied for last place in points that are also tied in nominations. Right?


#667 ::: J Thomas ::: (view all by) ::: May 19, 2015, 03:19 PM:

@663 Keith "Kilo" Watt

The definition in the article you cite is not mathematically rigorous, as I said.

Yes, that's true. It does give the sense I was using the term. The mathematically rigorous uses aren't appropriate here.

I guess it could be said that we have dealt with statistics a little bit, kind of. I've given hardly any description of what I've done, you've given a little more but hardly anything about what you've done, Jameson likewise. I haven't noticed any definitive results, but we've all gotten some that are suggestive, that imply it ought to sort of work.

If I come across as being short with you, then I apologize.

Accepted!

Can we get back to work?

Yes!

#668 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 03:23 PM:

junego@666:

Suppose works A and B tie for the least number of points and work C has the second-least number of points. We'll compare the number of nominations for all three works. Whichever has the fewest (even if it's C) gets eliminated.

Similarly, if A has the least number of points and C and D are tied for the second-least number of points, then all three will be compared for total number of nominations.

If A and B are tied for least number of points, and C and D are tied for second least-number of points, then we compare all four.

The tie breaker is needed when there is a tie for number of nominations, so that is what the other cases are referring to. Being tied for points can't cause a double elimination (unless they are also tied for number of nominations), so isn't really a problem for people as far as I can tell.

Does that help?

Kilo

#669 ::: abi, who is also Idumea ::: (view all by) ::: May 19, 2015, 03:24 PM:

Shane @644:

Forgive me, but I'm a little confused what, precisely I should point Cheradenine at. (Assuming the email address they used is a working one.)

#670 ::: J Thomas ::: (view all by) ::: May 19, 2015, 03:25 PM:

@662 Keith "Kilo" Watt

It is possible, however, for two works to also have the same total number of nominations.

I want to suggest something more like this:

It is possible, however, for two works to also be tied for the lowest total number of nominations.

I think the way you said it was good enough, and this leaves even less to think out.

#671 ::: Annie Y ::: (view all by) ::: May 19, 2015, 03:31 PM:

Keith "Kilo" Watt @ 668

I have a slow day so I will just ask - why would we touch C. We have two bottom work, we work only with them. Why grab C as well? Unless if we have a 3-way (or more) tie on points at the bottom, why wouldn't we just get the last 2 works at the bottom?

Now - if we have A with least points and B&C tying for second-least, then I can see us grabbing them all...

#672 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 03:39 PM:

Annie@671:

In truth, we can do it either way. My thoughts are to just handle everything the same way -- it reduces the number of "cases" that we have to deal with.

Kilo

#673 ::: J Thomas ::: (view all by) ::: May 19, 2015, 03:51 PM:

@668 Keith "Kilo" Watt:

Suppose works A and B tie for the least number of points and work C has the second-least number of points. We'll compare the number of nominations for all three works. Whichever has the fewest (even if it's C) gets eliminated.

I had thought we had a consensus on this. If we already have two works A and B with the lowest number of points, why do we need to add a third?

Here's my practical issue:

Say there is a slate that has more votes than most nonslate works.

If two or more slate works are tied with a score of N/5 or so, and all works with score

#674 ::: P J Evans ::: (view all by) ::: May 19, 2015, 03:55 PM:

The idea is to have the same procedure for all the names, regardless of whether they're on a slate.

#675 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 04:11 PM:

[FORMAL PROPOSAL]

All:

Okay, as promised, here's the next draft of the proposal. junego, it was recommended to us that we not include exmaples in the actual amendment, and instead include those in the explanatory text. I just wanted to make sure you knew why those examples weren't included.

Next post will have the text...

Kilo

#676 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 04:17 PM:

Short Title: Change to Hugo Nomination System
Moved, to amend section 3.8 (Tallying of Nominations), section 3.9 (Notification and Acceptance), and section 3.11 (Tallying of Votes) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed. determined by the following multi-round process described in 3.8.8.
3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.
3.8.3: Any nominations for “No Award” shall be disregarded.
3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.
3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.
3.8.6: The Committee shall move a nomination from another category to the work’s default category only if the member has made fewer than five (5) nominations in the default category.
3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.9 or 3.2.10, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category.
3.8.8: The final Award ballots shall list in each category the eligible finalists as determined by successive rounds of elimination, where each member gets a single nomination “point” for each category that will be divided equally among their nominated works in that category.
3.8.8.1: In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots for subsequent rounds as if they had never appeared on the ballots.
3.8.8.2: In the event that two (2) or more works are tied for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, the point totals for each work in the previous rounds (in reverse order) will be compared until the first round in which they differ in points is found. The work with least number of points in this round is then eliminated. If the two works have had equal point totals and equal number of nominations since the first round, then all members of that tie will be eliminated.
3.8.8.3: If elimination would reduce the number of ballots to fewer than the minimum number of finalists, then instead none of the works from that round should be eliminated, and all remaining works shall appear on the final ballot.
3.8.8.4: Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are removed from all ballots and treated as if they had never appeared on a nomination ballot. Members’ points are reallocated equally among their remaining nominated works, if any.

Section 3.9: Notification and Acceptance.
3.9.1 Worldcon Committees shall use reasonable efforts to notify the nominees finalists, or in the case of deceased or incapacitated persons, their heirs, assigns, or legal guardians, in each category prior to the release of such information. Each nominee finalist shall be asked at that time to either accept or decline the nomination. If the nominee any finalist(s) declines the nomination, that nominee finalist(s) shall not appear on the final ballot. In this event, once all finalists have had the opportunity to decline, the nomination system shall be re-run as described in Section 3.8 with the declined nomination(s) removed from the ballots on which they appeared. The eligible finalists from this re-run shall be merged with the remaining potential finalists from the original run. If this merge would result in more than the maximum number of finalists, then the ballot shall be extended to include the finalists from both the original and the re-run of the nomination system. This procedure shall also be used in the event that a finalist is deemed ineligible.

Section 3.11: Tallying of Votes.
3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, … places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes. During the same period a record of all rounds of the selection process for each category shall also be published.

Submitted by: Keith “Kilo” Watt, Tammy Coxen, Yoana Yotova, Joshua Kronengold, Chris Battey, CJ Cabourne, Steven Halter, P.J. Evans, David Goldfarb, Seth Gordon, Ginger Tansey, Steve Wright, Catherine Faber, Andy Holloway, Duncan J. Macdonald, Claudia Beach, Derry Earnshaw

Commentary: It is to be emphasized that this proposal does not change the nominating process from the perspective of Worldcon members: They still list works, unranked, they feel are Hugo-worthy (up to the maximum permitted) in any category they choose, just as they have previously. In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. However, because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn't penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate anything you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don't, that's okay – when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

It is also an explicit goal of this proposal not to disenfranchise anyone. Rather, this proposal seeks to ensure that no minority of members can disproportionately dominate an entire category. Instead, this system allows the broadest range of works that are popular with fandom-at-large to be considered for the Hugo Award.

#677 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 04:20 PM:

Okay, the boss says it's time for pain meds, so I'm off for a bit. Had to scramble to get that uploaded, but hopefully it's formatted correctly.

Let me know of changes and if I forgot any signed (It just occurred to me that I did forget one person who emailed me -- I'll get you on the next draft!)

Kilo

#678 ::: Annie Y ::: (view all by) ::: May 19, 2015, 04:30 PM:

Kilo@672

I'd vote for the simplest case. Always lowest two. If there is a tie with the higher (or with both), grab all that tie.

Otherwise we may end up with a 3-way tie at 2.0 and 5-way split at 1.0 (for example). Why force it when we can just go 5-ways instead of 8-ways? :)

#679 ::: Tim Illingworth ::: (view all by) ::: May 19, 2015, 04:46 PM:

KW@676: I 3.8.8.3 should "ballots" not be "nominees"?

It could also be worth breaking out 3.8.8 as a new Section 3.X: Nominee Tallying Procedure. S 3.8 is getting a bit unwieldy...

#680 ::: felice ::: (view all by) ::: May 19, 2015, 04:52 PM:

[TIES]

Keith "Kilo" Watt @662: "If there is a tie for least or second-least number of points, then all the tied works are eligible for elimination."

That should be "then all works with least or second-least number of points are eligible for elimination" - the current text could be interpreted as "if there is a tie for second-least points, only the works with second-least points are eligible for elimination, and the single work with least points is automatically safe", or less blatantly unreasonable, "if there is a tie for least points, the single work with second least points is automatically safe".


Annie Y @671: "I have a slow day so I will just ask - why would we touch C. We have two bottom work, we work only with them. Why grab C as well? Unless if we have a 3-way (or more) tie on points at the bottom, why wouldn't we just get the last 2 works at the bottom?"

It allows tied works with more nominations than any of the other low point works to bubble up together to tie for 5th place. The alternative is more frequent simultaneous elimination; we agreed that having tied works eliminating each other was worse than having them both eliminated by a third work with more nominations. Note that in this case, the tied works might be entirely unrelated, just happening to have the same number of nominations rather than being nominated by the same people.

#681 ::: Annie Y ::: (view all by) ::: May 19, 2015, 04:54 PM:

@676
OK - someone with English as a native language should weight in.

Does "the least and second-least" in the following case 1,1,2 means pick 1 and 1 or does it mean pick all three. If the latter, we need other wording here. Actually does it not matter what it means - if it is confusing readers, we probably need to restate it. I am confused and I had been here all along :)

So why don't we replace


In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared

with

In each round, the two (2) works with the lowest number of points (more than two if there are works with the same number of points as at least one of the two) will be compared.


More - when I read the whole thing again :)

#682 ::: Annie Y ::: (view all by) ::: May 19, 2015, 05:05 PM:

felice @ 680

OK - how is a 40.1, 40.2, 40.3 split and 40.1,40.1,40.3 so materially different that it is worse to leave the 40.3 alone in one case but not in the other?

Or in the case of 40,40,40,40,40,40.1. Let the 40s cannibalize one of themselves, the redistribution will change it again. The only case when this may be bad if the 40s are all slate works that noone else (at all) selected and it is a full slate of 5 (so they won't ever change - they will always stay at 40, cannot be resolved and so on). How moving the whole slate to start eliminating the next up work will help in that case? They will have the most nominations at each step.
This will lead to the slate eating ALL other works. Which frankly is the last thing we want.

Yes - I know we are not trying to exclude the slates. But we are also not trying to give them a clear strategy. And I would argue that this is a viable problem considering the quality of works that they nominate.

We are over-complicating it here I think. Get the two lowest and all that tie with them; don't try to pull more than 2 if the lowest two are a tie but do not have a third to tie with them... It also is a lot easier to explain to people :)

#683 ::: P J Evans ::: (view all by) ::: May 19, 2015, 05:25 PM:

681
1 would be 'least' and 2 would be 'second least' (or next-to-least').
(English. Drives everyone a bit crazy.)

#684 ::: J Thomas ::: (view all by) ::: May 19, 2015, 05:30 PM:

@682, from Annie Y,

So why don't we replace


In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared

with

In each round, the two (2) works with the lowest number of points (more than two if there are works with the same number of points as at least one of the two) will be compared.

Get the two lowest and all that tie with them; don't...

I think you did better the next post.

"In each round, two (2) works with the lowest point scores (and all that are tied with them) will be compared.

Assuming this is what we want to say, and not something else.

... may be bad if the 40s are all slate works that noone else (at all) selected and it is a full slate of 5 (so they won't ever change - they will always stay at 40, cannot be resolved and so on). How moving the whole slate to start eliminating the next up work will help in that case? They will have the most nominations at each step.
This will lead to the slate eating ALL other works. Which frankly is the last thing we want.

My comment saying that got cut off, but I think you said it better. Thank you.

#685 ::: Doire ::: (view all by) ::: May 19, 2015, 05:37 PM:

Kilo @672 Firstly, take care of yourself and recover soon.
From the commentary -

However, because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Might "diverse" be a dog-whistle word? I can see it being used to suggest that this proposal assumes diversity in SFF is good, and that this is enough reason to oppose it. How about "wide"? That should be neutral.

Is there any advantage to submitting this proposal early? New, innocent eyes might see gaps in the wording that are almost impossible to spot when (generic) you've been working on the details for so long.

#686 ::: felice ::: (view all by) ::: May 19, 2015, 05:46 PM:

Keith "Kilo" Watt @653: "Hi all! Made it through surgery with no problems"

Glad to hear it!


[TIES]

#680: Ah, I see that's fixed in #676.


J Thomas @660: "We'd have to see, wouldn't we? It does look rare. Ties are rare. In 2013 there weren't any among finalists"

Aside from BDP Short Form and Best Professional Artist, and single-nomination differences in five more categories. There would have been a four-way tie for fifth in Short Story too, if not for the 5% threshold. And last year, there were finalist ties in five categories. Ties are not rare!

"Felice has made an argument that vote-ties in late stages are likely to be score-ties too. There just aren't many others left to share ballots with, and most of the ballot-sharing was done with the ones that are already gone. If the #5 and #6 works both have N nominations, not that unlikely they will both have N points, or possibly both have N-0.5 points or N-1 points etc."

And lookback probably won't help if works below 5% are excluded from the process; there won't be enough rounds to look back at. The 5% threshold might (hopefully) be abolished, but that's a separate proposal and we can't rely on it. And frankly I don't think lookback is really any less arbitrary than looking at first appearance on any ballot, which has the advantage of always working and being easier to explain. I mean, it's choosing the winner based on whose nominators supported the fewest semi-popular-but-already-eliminated works; entirely arbitrary, and it encourages bullet voting, because bullet-voting nominators are more likely to win a lookback tiebreaker. Lookback was designed for STV elections (ranked, many voters rank all candidates); it just doesn't make any sense for SDV-LPE (unranked, nobody nominates more than a tiny fraction of candidates).

"quasi-Chaos: Small changes in voting can lead to large changes in outcome."

As I understand it, those possible large changes are only due to simultaneous elimination. Can you come up with any examples of single votes drastically changing the outcome that don't involve ties?

#687 ::: Annie Y ::: (view all by) ::: May 19, 2015, 05:55 PM:

J Thomas @ 684

I had been trying to figure out how to say it. :) I know what I mean, I am not always able to say it in a concise sentence.

So what is the consensus then. In the case of 1,1,2, do we get 1,1 or 1,1,2 into eliminations? I was under the impression that is 1,1, sounds like not everyone thinks so.

#688 ::: felice ::: (view all by) ::: May 19, 2015, 06:02 PM:

Annie Y @682: "OK - how is a 40.1, 40.2, 40.3 split and 40.1,40.1,40.3 so materially different that it is worse to leave the 40.3 alone in one case but not in the other?"

It's to minimise the chance of a tie for points and nominations. If the two 40.1s are tied for nominations as well, there's no non-arbitrary way to pick which one to eliminate. Comparing to the 40.3 as well means we might not need to break the tie.

"Or in the case of 40,40,40,40,40,40.1. Let the 40s cannibalize one of themselves, the redistribution will change it again. The only case when this may be bad if the 40s are all slate works that noone else (at all) selected and it is a full slate of 5 (so they won't ever change - they will always stay at 40, cannot be resolved and so on). How moving the whole slate to start eliminating the next up work will help in that case? They will have the most nominations at each step.
This will lead to the slate eating ALL other works. Which frankly is the last thing we want."

They will only eat all other works up till they tie for 5th place, in which case the four other finalists will be non-slate works. A slate tied at 40 means they have 200 nominations each; but we only need four works with more 41 or more unique nominations each (more to the extent that they overlap) to stay above them right to the end. And if we don't have any works with 41 or more unique nominations each, then all the non-slate works will eliminate each other early on and the slate will never even be compared for elimination, and the fact that they're tied is irrelevant.

Having the entire slate tied for 5th is a nuisance - it means 9 finalists instead of 5 - but it's not really that awful an outcome, and it's incredibly unlikely to happen (it requires no slate supporters deviating from the slate in any way, and nobody outside the slate supporters coincidentally nominating any of the same works).

#689 ::: Annie Y ::: (view all by) ::: May 19, 2015, 06:11 PM:

felice @ 688

Unless if someone else also ties at the 40 with them somewhere along the way (especially high up), allowing these 5 and that 6th one to eat everyone else. Which is as likely as having the full slate not matching anyone else's single nominations (which is the only case we are getting here). If we are in that case, the slate will end up eating more than one of the top 5 works more likely than not.

And picking up the 2 in the 1,1,2 is saving the slate only in this case as well (all 1s will go away otherwise) - in all other cases, there will be some of them a bit off step because of other ballots. So I still do not see why we should not go for the 1,1 in this case.

PS: Not a deal breaker. I will support it either. I just prefer the simpler solution when there is no real reason to go for the more complicated one. Makes it cleaner to explain. And easier to convince people that it is the proper way.

#690 ::: felice ::: (view all by) ::: May 19, 2015, 06:56 PM:

Annie Y @: "Unless if someone else also ties at the 40 with them somewhere along the way (especially high up), allowing these 5 and that 6th one to eat everyone else. Which is as likely as having the full slate not matching anyone else's single nominations (which is the only case we are getting here). If we are in that case, the slate will end up eating more than one of the top 5 works more likely than not.

A perfect slate and other potential finalists tied for second least points is even more incredibly unlikely. This must be a tie involving at least two of the last five non-slate works, otherwise there'd still be four non-slate works above them to stay on the final ballot, or the elimination would already be over. Simultaneous elimination of the tied works with fewest nominations could theoretically leave fewer than four non-slate finalists, but in practice this is never going to happen; and also, I'm strongly opposed to ever doing simultaneous elimination. Only one of the tied works should be eliminated, which leaves us with the same 9 finalists as without a tie between non-slate works.

"And picking up the 2 in the 1,1,2 is saving the slate only in this case as well (all 1s will go away otherwise) - in all other cases, there will be some of them a bit off step because of other ballots. So I still do not see why we should not go for the 1,1 in this case."

The works tied for last aren't necessarily slate works, though! Eg the ballots below have A, B, C, and D tied for 1 point and 3 nominations each, even though no two ballots are the same, while E is second-last with 2 points and 2 nominations. This isn't a particularly realistic scenario, but more complicated sets of ballots with two partially overlapping works tied for last place aren't implausible.

ABC
ABD
ACD
BCD
E
E

#691 ::: J Thomas ::: (view all by) ::: May 19, 2015, 06:56 PM:

@688 Felice

"This will lead to the slate eating ALL other works. Which frankly is the last thing we want."

They will only eat all other works up till they tie for 5th place, in which case the four other finalists will be non-slate works.

I think I followed that. You have 5 slate works that each have 40 points, and you don't eliminate any of them when they are tied, instead you compare them to the work with the next highest score. They eliminate everything until they are in 5th place and at that point you stop. Even though there are 9 survivors and you could compare them to the next lowest as you've been doing all along, instead they are tied for 5th place so that's where you stop.

Neat trick!

The other 4 works are the ones with the highest point scores at that time, which have had high point scores all along. None of them have competed since the slate started competing every turn; if they had they would probably have lost. (Though maybe the one with the most or second-most number of votes might have won.)

That outcome doesn't look as bad as I was thinking. There might not be that many of them tied, and if there's a tie and there's one slate member that has more votes, presumably it would eliminate all the others. When there are 6 left and it has the 5th highest score and another slate member is at #6, it will beat that one.

I dislike including items with the second-lowest score when there's a tie at the lowest score, and I dislike keeping all members of a tie when there's a tie, but the combination of these can work better than I thought.

#692 ::: Jameson Quinn ::: (view all by) ::: May 19, 2015, 07:18 PM:

I agree that it should be only 1,1 not 1,1,2. @688 is, I think, wrong; they could in fact eat everyone else, because the system doesn't realize they're tying for 5th, it thinks they're tying for 8th/9th. The question remains, how to word it simply and unambiguously; I think "In each round, two works with the lowest point scores (and all that are tied with one of them) will be compared." (This is not my suggestion of course, but I'm not sure whom to credit.)

(I also think we should avoid the "two (2)" tic.)

...

In terms of "alerting cheradenine", it had to do with the fundraiser. But now that I've promised that donations will be tax deductible, I will not be able to split them with cheradenine. If Cheradenine is interested in getting some of the fundrisen money, they should contact me, and I'll set up a separate gofundme and encourage people to donate to both equally.

...

Moderators: I want to promote my gofundme campaign so I can go to sasquan (already over $200 raised, over 1/7 of the way there), but I don't want to spam. Can I get some guidance as to what would be in bounds?

...

Keith: Good that the surgery went well. Did you see @618/@622?

#693 ::: Annie Y ::: (view all by) ::: May 19, 2015, 07:35 PM:

"In each round, two works with the lowest point scores (and all that are tied with one of them) will be compared." is still ambiguous though. Or at least sounds ambiguous to me.

What about:

In each round, all works with the lowest point scores are compared. If only one work is selected this way, all works with the next lowest points score are added to the comparison.

Or something along this lines... Makes it clear enough for anyone reading.

#694 ::: Jeremy Leader flags a moderator ::: (view all by) ::: May 19, 2015, 08:16 PM:

Jameson Quinn @692 above has a question for the moderators which has some degree of time-sensitivity (he'd like to promote his GoFundMe while people are still engaged with this thread).

#695 ::: J Thomas ::: (view all by) ::: May 19, 2015, 08:25 PM:

@686 Felice

"quasi-Chaos: Small changes in voting can lead to large changes in outcome."

As I understand it, those possible large changes are only due to simultaneous elimination. Can you come up with any examples of single votes drastically changing the outcome that don't involve ties?

Sure, but it's more complicated. Here's a simple example:

A 1 vote 1 point
AB 100 votes 50 points
ACDE 1 vote 1/4 point

C
D I don't care how these go but they should be more than 50.34 points and less than 51.25 points
E

CDF 1 vote 1/3 point
F 50 votes 50 points

There are 6 works left so one should go. The low scores are B with 50 and F with 50.33.
B has 100 votes while F has 51 votes, so F is eliminated and the winners are ABCDE.

But if the single vote for A is removed, then for the last round,

A has 50.25 points while B has 50 points. A has 101 votes and B has 100 votes. B is eliminated. The winners are ACDEF.

It's easy but tedious to extend this. Have some BG pairs in place of some of the AB pairs. Normally B eliminates G and gains points, leaving F in 6th place losing to E. But when B is instead eliminated, G vaults into 4th place, and E (which had some EG pairs) into 2nd, giving F the chance to beat C or D.

Etc.

Does it seem plausible to you that one vote different could possibly result in four of the finalists changing, without involving any ties?

#696 ::: J Thomas ::: (view all by) ::: May 19, 2015, 08:33 PM:

@692 Jameson Quinn

I agree that it should be only 1,1 not 1,1,2.

I like that better but I'm no longer sure the other way is very bad.

@688 is, I think, wrong; they could in fact eat everyone else, because the system doesn't realize they're tying for 5th, it thinks they're tying for 8th/9th.

Are you sure? If it was #6 and #5 tied, they would be tying for 5th place. If it was #7 and #6 and #5 tied, wouldn't that be a 3-way tie for 5th place? Etc. How else would you get a tie for 5th place?

I can kind of see this one both ways.

#697 ::: J Thomas ::: (view all by) ::: May 19, 2015, 08:37 PM:

@693 Annie Y

"In each round, two works with the lowest point scores (and all that are tied with one of them) will be compared." is still ambiguous though. Or at least sounds ambiguous to me.

With human beings, if it *sounds* ambiguous then it *is* ambiguous.

What about:

In each round, all works with the lowest point scores are compared. If only one work is selected this way, all works with the next lowest points score are added to the comparison.

Much better!

How about:

In each round, all works with the lowest point score are compared. If only one work is selected this way, all works with the next lowest point score are added to the comparison.

#698 ::: felice ::: (view all by) ::: May 19, 2015, 08:54 PM:

J Thomas @691: "The other 4 works are the ones with the highest point scores at that time, which have had high point scores all along."

Not necessarily all along - some of them might have had fewer points than the slate early on, but won an elimination comparison against another non-slate work and thereby gained enough points to put them above the slate; and each work eliminated by the slate could change the relative positioning of works ranked above the slate; but that's a minor technicality.


Jameson Quinn @692: "@688 is, I think, wrong; they could in fact eat everyone else, because the system doesn't realize they're tying for 5th, it thinks they're tying for 8th/9th."

You're right, that's a slight bug in the system; I think the way I described it is the way it should work, though. Perhaps 3.8.8.3 should be something along the lines of: "If fewer than four remaining works are not being compared, then instead none of the works from that round should be eliminated..." or "If elimination of all but one of the compared works would reduce the number of ballots to fewer than the minimum number of finalists, then..."

Though that would mean a tie between 4th and 5th would always make the 6th a finalist too (or more if there's a tie for 6th too), which we don't really want. Though with simultaneous elimination, that will sometimes happen anyway: we always have to compare all three of 1,2,2, and if eg 4th and 5th are tied on 30 points, 40 nominations each, while 6th has 29 points and 41 nominations, we have to keep all 6 to prevent the ballot dropping to four finalists. And I don't think we want to break ties involving 5th place.

How about this?
1, 1, 2 - compare all
1, 2, 2 - compare all
1, 1, 2, 2, - compare 1s only

That way, a tie involving 4th and 5th won't let more than one extra finalist in. There could theoretically be a 5-way tie for 4th, but that's effectively only possible with a perfect slate, and just means we get four non-slate finalists again; not a bad outcome.

Or if there's a tie for second-least points, only compare the one with fewest nominations against the work(s) with least points, eg in the example below, only compare E, F, and G, so if D and E are tied on points for 4th, E gets eliminated rather than all four being compared and nothing eliminated:

D: 10 points, 15 nominations
E: 10 points, 12 nominations
F: 9 points, 18 nominations
G: 9 points, 18 nominations

#699 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 09:14 PM:

[NO LOOKBACK] I don't think lookback gains us much and it's worth having a last ditch tiebreak that can't result in a tie, but lookback isn't it.

Jameson, I'm not a lawyer or accountant, but I do have some familiarity with non profit law due to my experience working with and running sf cons. I don't know of anything that would prevent Cherardine being covered by the same gofundme you use.

The measure looks good, but the title is awful -- dry as dust, and completely indistinguishing this measure from, well, any other measure that is trying to change the Hugos this year.

A WSFS short title should ideally be funny, memorable, and remind one of what the measure is about. It need not be totally descriptive, nor need it be dry.

Keep in mind that the short titles for new business this year are:

4 and 6
The Five Percent Solution
I remember the Future

The business passed on for ratification from last year is:

Popular Ratification (2 plus 1)
A Story by any other Name
Hugo Finalists
and WSFS Membership Types and Rates

Obviously, not all of these are great, and the last one isn't even good -- but particularly in a year with multiple hugo change proposals, this simply won't do.

I propose:

Put Proportion into Nomination

But frankly anything memorable and evocative would do:

Nomination by Pie
Sharing the Hugo
E Pluribus Hugo
Single Divisible Vote
Single Divisible Nomination

The last two aren't funny, but has the advantage of being -immensely- memorable, so we should probably do one of them.

That said, I think I favor "E Pluribus Hugo" as long as it won't cause a derail based on my non-existent Latin.

#700 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 09:17 PM:

Oh, also, I agree that we should bias towards two entries enter, one entry leaves. So if there's a tie for lowest, only the things tied for lowest should be compared.

In terms of stopping at a tiebreaker, if there's a tie that -includes- fifth place, the tie should not be resolved and all of the tied works shall be considered finalists.

#701 ::: Annie Y ::: (view all by) ::: May 19, 2015, 09:26 PM:

felice,

There would have been a four-way tie for fifth in Short Story too, if not for the 5% threshold. And last year, there were finalist ties in five categories. Ties are not rare!

OK - missed that somewhere in the shuffle - sorry.

Agree. In the number of nominations. But with a double system (points and nominations), ties will happen a lot less often. We are in a real tie only if both the nominations and the points match exactly and then going back all of them keep matching. This will happen a lot less often than the ties under the current system. Comparing them directly is not exactly called for -- you introduce second variable here.

Even if these ties come against each other, chances of them tying on points as well (especially with the looking back as a third variable) are small unless if we are talking very similar ballots...

I still have the feeling that we are over-complicating for the sake of small sets of data designed to show tying. These sets are good for looking at corner cases but selecting a strategy based on corner cases is not what we are trying to do I think. Still thinking on all this though...

#702 ::: Annie Y ::: (view all by) ::: May 19, 2015, 09:37 PM:

felice @ 698,

Now describe that example with the 1s and looking forward to see if we also want the 2s in a way that does not require 3 paragraphs (and does not make people wonder why we need to do that and if we are not actually manipulate data)? :)

And what are you doing for:

1,1,1,2,2
1,1,2,2,2
1,1,2,2,2,2
1,1,1,2,2,2
(and so on)
Just the 1s? If so - that first 1,1,2 is like a sore thumb. Grab the 2s somewhere - what is the common element that makes us grab them?

If you want to look at nominations to figure out what you are grabbing (your last paragraph)

Points: 1,1,1,2,2,2
Nominations: 2,2,2,2,2,2

Why do we need to bring the 2s with us?

Same example:
Points: 1,1,1,2,2,2
Nominations: 2,2,2,4,4,4
Same question?

Points: 1,1,1,2,2,2
Nominations: 2,2,2,6,2,4
The 1s are still the ones loosing a member.

Points: 1,1,1,2,2,2
Nominations: 3,4,5,6,2,3

This is a case where the 2s are losing one of their members if we bring it in the elimination but that outlier will get kicked on a next round anyway because it is lower than all other 2s. Plus this is the case with a single nomination anyway - these will always track high at the start.

It is not just about building a model here - it is also about being able to explain it clearly enough so people can buy into it and support it.

#703 ::: P J Evans ::: (view all by) ::: May 19, 2015, 09:52 PM:

702
It's also important that we not get hung up on less-likely conditions and edge cases. You cannot foresee everything, and this isn't a production system, but a test system, being tested with, AFAICT from what people are saying, minimal sets of test data. (Has anyone actually run this model with a full category of data?)

#704 ::: felice ::: (view all by) ::: May 19, 2015, 10:14 PM:

[TIES]
Ok, how about this:

"3.8.8.1: In each round, the two works with the least and second-least total points will be compared. In the event of a tie for least and/or second-least distinct total points, all out of those works having the least or second-least distinct total nominations will be compared. Of the compared works, the one that appears on the fewest number of ballots will be removed from all nomination ballots for subsequent rounds as if they had never appeared on the ballots.

...

3.8.8.3: If fewer than four remaining works are not being compared, then instead none of the works from that round should be eliminated, and all remaining works shall appear on the final ballot."

Eg A: 10/20, B: 10/15, C: 9/12 - only compare B and C.
A: 10/20, B: 10/15, C: 9/13, D: 9/12 - only compare C and D.
A: 10/20, B: 10/15, C: 9/18, D: 9/18 - compare B, C, and D.
A: 10/20, B: 10/15, C: 9/27, D: 9/27, E: 9/27 - compare A and B.
A: 10/15, B: 10/15, C: 9/18, D: 9/18 - compare all.

Comparing only the works with least or second-least distinct total nominations makes absolutely no difference except when there's a tie involving 5th, but stating it as a general principle rather than additional special handling for ties involving 5th seems tidier to me.

That way, if there's a points-only tie between 4th and 5th, only one of them (at most) will be compared against the work(s) with 6th most points. Though if there's a points-and-nominations tie between 4th and 5th, and all 6th placed works have the same number of nominations, they're all compared and nothing is eliminated. This is a pretty unlikely situation, but will happen occasionally. It's the best way I can see to let tied works bubble up to 5th place together but prevent them swallowing up the 4th or higher places.

If we break all ties and don't ever allow simultaneous elimination, then this process isn't necessary. But I think we agreed that allowing tied works to eliminate each other was worse than letting them be eliminated simultaneously by a third work with more nominations, which is the reason we introduced the "compare all of 1,1,2" rule.

#705 ::: Annie Y ::: (view all by) ::: May 19, 2015, 10:25 PM:

@703 - agree, see my last paragraph in 701 :)

A set can be designed for any eventuality. The work with the most votes always* makes the ballot - it beats anyone in a straight elimination showdown no matter how low the points are. Actually in a normal year, the nominees will be close enough...

*OK - there is a special case here.
The top 2 (or more works - can be from a slate, can be just unlucky pair of works) gets the same number of votes and tie on points lower than at least 5 other works. This can kick them both out as an unresolved tie. The looking back should clear them from this case (I think?) but we are still loosing one of them. We can add an explicit rule that the work(s) with the most nominations are always added to the ballot at the end if they are not there already. This will put a clean slate full on the ballot (so too many nominees, ugh) but only if it is a real slate - 300 people voted it straight; noone else voted even one of the works (1 person picking up a work will kick the points and votes and deal with the whole situation)

It is such an outlier case but if the proposal makes it through, Murphy's laws will make sure we get that in at least one category. And kicking out the most popular nomination is just ugly.

OK - just thinking aloud. Did I miss where we discussed that earlier?

#706 ::: felice ::: (view all by) ::: May 19, 2015, 10:42 PM:

Joshua Kronengold @699: "[NO LOOKBACK] I don't think lookback gains us much and it's worth having a last ditch tiebreak that can't result in a tie, but lookback isn't it."

Agreed.

"The measure looks good, but the title is awful -- dry as dust, and completely indistinguishing this measure from, well, any other measure that is trying to change the Hugos this year. A WSFS short title should ideally be funny, memorable, and remind one of what the measure is about. It need not be totally descriptive, nor need it be dry."

Agreed.

"That said, I think I favor "E Pluribus Hugo" as long as it won't cause a derail based on my non-existent Latin."

That works for me. Or "Hybrid Divisible Vote", or just "Single Divisble Vote (LPE)".

Annie Y @701: "Agree. In the number of nominations. But with a double system (points and nominations), ties will happen a lot less often. We are in a real tie only if both the nominations and the points match exactly and then going back all of them keep matching. This will happen a lot less often than the ties under the current system."

Real ties will happen more often than you think, because by the time we're down to the last 6 or 7 candidates, there are very few other nominees left for points to be split with. Only a small fraction of the people who nominated one or more of the tied works will also have nominated one or more of the other 4 or 5 remaining works, and the chances of it being the same fraction aren't insignificant (it could quite possibly be zero, ie none of the people who nominated D and/or E also nominated A, B, C, F, or G). Maybe a tie that matters will only happen every few years; it's still a bad thing that we should prevent happening.

Lookback will usually break the ties, unless the 5% threshold stays in which case it may well be useless, and as I discussed in #686 it seems to be an utterly arbitrary tiebreaker, and if we're going to break ties arbitrarily (and I think we should), we may as well use something simpler and 100% effective ("first appearance on any ballot" works nicely).

J Thomas @695: "B has 100 votes while F has 51 votes, so F is eliminated and the winners are ABCDE. But if the single vote for A is removed, then for the last round, A has 50.25 points while B has 50 points. A has 101 votes and B has 100 votes. B is eliminated. The winners are ACDEF."

That's an extremely unlike unstable situation, requiring the highest and lowest scoring of the last six works to be separated by only 1.25 points. Rather less probable than two works tying for points and nominations, I think.

#707 ::: Jameson Quinn ::: (view all by) ::: May 19, 2015, 10:45 PM:

@699: It's not that it would be illegal for electology.org to (hypothetically) buy tickets for cheradenine. It's that it's no longer my decision. I have clearance to say that buying my own tickets is a valid use of money to further the organization's goals; and while I personally think that would also be true for cheradenine, I don't have the authority to decide that by myself, and thus I can't collect that money under the same gofundme.

It's probably a moot point, though, because cheradenine has yet to express any interest in this possibility.

...

I still think that if it's 11,11,12, only 11,11 should be considered. If 12 is eliminated in this case, then 13, then 14, while 14 could have beaten 15 and been a finalist... I think the point of the SDV-LPE process is defeated, and the voters for 14 have too much of a bullet-voting incentive.

...

I like "E pluribus Hugo". Or is it too US-centric? If we want something dry and descriptive, then "Proportional Hugo Nominations System".

#708 ::: colin roald ::: (view all by) ::: May 19, 2015, 10:56 PM:

To clarify 3.8.8.1: "In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots for subsequent rounds as if they had never appeared on the ballots."

Is it the intention of this proposal that if two works are tied for lowest point total (say Alpha and Beta both have a score of 0.45) you nevertheless *also* select Gamma (which has a score of 0.5) for showdown, and then if Gamma has only one approval while Alpha and Beta each have two, then Gamma is eliminated and both Alpha and Beta survive the round?

That appears to be the meaning of the wording here, but it does not match my expectation from the casual phrasing "select the two works with the lowest point totals for showdown". I'd expect you'd only expand to three in the showdown if you "have to" – ie, if Alpha is lowest and Beta and Gamma are tied for second-lowest.

#709 ::: J Thomas ::: (view all by) ::: May 19, 2015, 11:09 PM:

@704 felice

If we break all ties and don't ever allow simultaneous elimination, then this process isn't necessary. But I think we agreed that allowing tied works to eliminate each other was worse than letting them be eliminated simultaneously by a third work with more nominations, which is the reason we introduced the "compare all of 1,1,2" rule.

I'm more unsettled about all this than I was day before yesterday.

I thought double ties would be very rare, so we needed something that would sound fair to people at the meeting. It didn't really matter how well it worked if it didn't actually happen.

But we shouldn't decide how to do things that *do* happen based on how they affect things that *don't* happen. Ties for lowest score will happen every year, and we should handle them based on what works for them, not on what might hypothetically happen someday.

But now I don't know how common double-ties will be. It was rare in the 1984 data. That was a third the number of ballots today, and there weren't as many works to dilute the vote. Also we lack data about slates.

If double ties really are important, important enough to change common routines to affect them, we need to be careful. It isn't just about saying something that sounds good, we really need to make it work.

At a minimum, we need it to work for 10 to 20 years, until people forget who's responsible for this or they change the voting system again. When we get 17+ runs a year, thinking it will only happen one time in a hundred is not that reassuring.

#710 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 11:11 PM:

I'm awake again, so checking in here. :)

Joshua@699:

E Pluribus Hugo -- "Out of the many, a Hugo"

Perfect, absolutely perfect. This radiates awesomeness as far as I'm concerned.

PJ@703: We've run the model with actual 1984 data for all categories, and with a randomized distribution based on the 2013 data. We then did variations on those data (slates of varying size and number, bullet votes,etc.)

Annie@622: Thank you very much for your kind offer! My wife is coming as well, so we decided it'd be best if we just went ahead and got the tickets. We do appreciate the thought, though!

And speaking of travel plans, we will be flying into Spokane Wednesday night about 11pm and will be staying at the Davenport (the party hotel, natch!).

We will have to be at the airport by 2:00 on Sunday, so I'm really hoping at least our part of the business meeting is over before then.

I'm reading the 1,1,2,2 posts, so I'll comment on those later on.

Thanks,
Kilo


#711 ::: Annie Y ::: (view all by) ::: May 19, 2015, 11:12 PM:

felice@706
Considering that neither you, nor me ever said how many ties we expect, I am not sure who has more in mind actually. :) I expect less ties than under the current system. Only time will show.

How do you define first appearance? How do you order the ballots? Based on submission date? And if both are on the same ballot? You take whatever is higher? Now we are talking ranking in nominations :) Which I am really not happy with.

Or am I misunderstanding your proposal? Lookback is not arbitrary... It's looking at what would have happened if these two had met a step (or more) earlier. If that cannot resolve it, they are full equals so they either stay or go. We cannot let them stay so they are off together.

#712 ::: P J Evans ::: (view all by) ::: May 19, 2015, 11:15 PM:

710
The way some people were talking, it sounded like they were using samples. (I'd like to get hold of a much larger set, but Oh Well.)

#713 ::: Annie Y ::: (view all by) ::: May 19, 2015, 11:23 PM:

@708 colin roald

Yep, you are reading it as I do - and turned out that we still have a small disagreement on what it would mean :)

Keith "Kilo" Watt @ 710

How is the knee?
No worries - would have been happy to help.

Any weirdness in any of the runs (most nominated gets kicked out for example)? And was there any case with a tie at the top of nominations - this case does worry me a little bit?


Joshua Kronengold @ 699:

E Pluribus Hugo - we are all geeks, why not? :) Unless if it sounds better in Klingon (just kidding, I like it as it is)

#714 ::: J Thomas ::: (view all by) ::: May 19, 2015, 11:23 PM:

@706 Felice

"B has 100 votes while F has 51 votes, so F is eliminated and the winners are ABCDE. But if the single vote for A is removed, then for the last round, A has 50.25 points while B has 50 points. A has 101 votes and B has 100 votes. B is eliminated. The winners are ACDEF."

That's an extremely unlike unstable situation, requiring the highest and lowest scoring of the last six works to be separated by only 1.25 points. Rather less probable than two works tying for points and nominations, I think.

You asked for an example. I provided one. Every individual example is unlikely. If you want to get an idea how unlikely it is to have any pathological situation you need to sum over all the pathological situations, and all the likely voting patterns, etc.

That's an extremely unlike unstable situation, requiring the highest and lowest scoring of the last six works to be separated by only 1.25 points.

I chose it because it was easy to think about. You can separate them a lot more if you want to.

My point is that this is quasi-chaotic. Small numbers of changed votes *can* give a very different result. Acceptance Voting is not nearly so chaotic. Change one vote and the most that happens is a 5th place win turns into a tie, or a tie turns into a loss. Much more stable to small changes in votes. But it has its own problems, one being that slates can walk all over it.

#715 ::: Annie Y ::: (view all by) ::: May 19, 2015, 11:27 PM:

P J Evans @ 712

The way some people were talking, it sounded like they were using samples. (I'd like to get hold of a much larger set, but Oh Well.)

I mentioned samples more than once but I was talking about the outlier cases and corner cases that me and felice had been slicing and dicing. Just in case you mean me :)

As for the larger set - do we know who is in the Hugo committee for Kansas and is there any chance we start convincing them to release anonymized ballots (both users and works if need be) after they declare the nominees next year? Even if this proposal fails, it will mean a year of current data to work with for any proposal.

#716 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 11:34 PM:

Annie@713: How is the knee?

Codine is my very best friend. :)

It's not too bad, really. I've got a couple of weeks of PT ahead of me, but then I should be right as rain.

There was no weirdness in any of the data sets I ran. But then again, given that we're talking about very low probability cases, you really wouldn't expect there to be. All of the double ties (points and nominations) extended all the way back to the first round, but most of those (maybe all -- addled memory here) had just a couple of nominations anyway. So they had no chance of being finalists no matter what we did.

Kilo

#717 ::: Soon Lee ::: (view all by) ::: May 19, 2015, 11:40 PM:

+1 to Joshua Kronengold's title proposal, "E Pluribus Hugo"

(Been reading but not commenting & wish to thank everyone doing the heavy lifting.)

#718 ::: Annie Y ::: (view all by) ::: May 19, 2015, 11:48 PM:

Kilo @ 716

A few weeks of PT, a few years of not needing the weather channel :) Speedy recovery!

#719 ::: felice ::: (view all by) ::: May 19, 2015, 11:53 PM:

Annie Y @711: "Considering that neither you, nor me ever said how many ties we expect, I am not sure who has more in mind actually. :) I expect less ties than under the current system. Only time will show."

Good point :) Fewer than under the current system, yes, but not that much fewer; double ties will definitely happen.

"How do you define first appearance? How do you order the ballots? Based on submission date? And if both are on the same ballot? You take whatever is higher? Now we are talking ranking in nominations :) Which I am really not happy with."

Yes. It's not ideal, but it's better than eliminating both. People don't want the order on their ballots to matter, but if it's a choice between one of their nominations losing or two of their nominations losing, I think most people would prefer that one survived. But if even this minimal reliance on ranking is unacceptable, a pregenerated set of coin tosses is equally fine with me.

"Lookback is not arbitrary... It's looking at what would have happened if these two had met a step (or more) earlier."

They can't actually meet a step earlier, though. That's functionally equivalent to saying "what if the most popular eliminated work that shares a ballot with A or B had been more popular than it was in reality?" That's totally arbitrary.


Keith "Kilo" Watt @710: "We've run the model with actual 1984 data for all categories, and with a randomized distribution based on the 2013 data. We then did variations on those data (slates of varying size and number, bullet votes,etc.)"

The actual distribution for 2013 nominations isn't random, though, and we don't know what it really is. The ways in which the nominations overlap between ballots is crucial, and I'm not confident randomized distributions would produce results similar to reality.

#720 ::: Annie Y ::: (view all by) ::: May 20, 2015, 12:07 AM:

felice,

You and me have very different definitions of arbitrary :) Look back is a standard mechanism to resolve ties if you do not want to kick both out in eliminations. It beats picking a loser from a hat. Or kicking the guy/work with less letters in the name :)

It is not perfect but it is explainable and the data from the previous eliminations is fully available. As long as all ties are resolved the same way, it is workable. NO - they did not meet but that does not mean that they are not comparable.

Which brings me to another thing
[3 WAY COMPLETE TIE].
We step back and we have 1,1,2 in the points on the previous run(if it is 1,1,1 or 1,2,2 or 1,2,3, it is obvious). Do we keep looking back next steps only for the ones that are now a tie 1,1 or for all 3?

#721 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 01:18 AM:

[TIES]

The way I see it, we have too many cases. That's what's making the tie breaking rules seem so complicated. Let's look for a single rule that can be applied consistently. We may come up with more than one. Let's consider those the set of possible tie breakers, then we can choose which one of those seems to work best. It won't be perfect, that's obvious. But I'm reminded of the quote my boss at NASA was fond of: Perfect is the enemy of good enough. I think in the quest for the perfect tie breaker, we've lost our way.

Since we have two characteristics that can tie, we need no more than two rules. No special cases.

So, just as an example, one possible rule (and not one I'm necessarily suggesting we use) is: Always compare the set of all works having the two lowest total point values. Simple, unambiguous.

What can you come up with?

Kilo

#722 ::: Annie Y ::: (view all by) ::: May 20, 2015, 01:40 AM:

@721 [TIES]
Let's run with this for a while then.

If one of them is lowest on votes, we are done.
If we tie on votes, we move back on points:
- if we have only one from the lower set in the tie, we are done
- if more than one from the lower set, then what? Eliminate all? Or try to find a single to eliminate at this point?

I am thinking of kicking them all out - they are beaten by the ones from the higher set and maybe by some of their own) on two separate criteria? The border case would be when we reach 5... (#4,#5,#6 and #7 go for elimination, we need at least 2 back, 4,5 and 6 who were the lower points tie at this point). We go for a ballot of 7 is probably the simplest - so if we get to a tie resolving that would end with less than 5, we put all on the ballot. If that last tie is huge, chances of it tying so badly are not so great. So I kinda like it.

Hm - this is workable. And easily explained...

#723 ::: felice ::: (view all by) ::: May 20, 2015, 02:03 AM:

[TIES]

Annie Y @720: "You and me have very different definitions of arbitrary :) Look back is a standard mechanism to resolve ties if you do not want to kick both out in eliminations. It beats picking a loser from a hat. Or kicking the guy/work with less letters in the name :)"

I'm not sure it does. It makes sense in a top-down system like STV: Alpha wins one of the spots, but then Beta and Gamma are tied. The people who voted for Alpha have technically already used up their votes getting Alpha in, but their opinion on whether Beta or Gamma is better is still valid, so looking back to see what they preferred before their votes were taken out is a reasonable way to break the tie. SDV-LPE is a bottom-up system that works completely differently, though; the order in which the less popular works were eliminated before the current round is irrelevant.


Keith "Kilo" Watt @721: "The way I see it, we have too many cases. That's what's making the tie breaking rules seem so complicated."

It seems to me that all the complexity stems from the possibility of ties not being broken. If we include a rule that always breaks every tie, then all our problems go away. Any such tiebreaker will be arbitrary, but I don't see that as a problem. For any tie that matters, picking an arbitrary winner will benefit more people than it hurts.

"So, just as an example, one possible rule (and not one I'm necessarily suggesting we use) is: Always compare the set of all works having the two lowest total point values. Simple, unambiguous."

And allows a perfect slate to sweep the ballot in most categories. Not the most likely prospect, but a highly undesirable outcome if it does happen.

If we break all ties, "Always compare the two works having the lowest total point values (or more if tied)" would work for me.

#724 ::: Soon Lee ::: (view all by) ::: May 20, 2015, 02:07 AM:

[TIES]
Keith "Kilo" Watt #721 & Annie Y #722:

That works for me. I prefer going for a ballot of more than 5 in case of ties especially at this nominating stage. My only concern is if we get a scenario where the tie results in an unwieldy large number of finalists, especially in the Novel category (because I'm a slow reader) but I don't have a good grasp of how likely it is to occur.

I also like the simplicity & clarity which is a consideration for improving its chances of success at the Business Meeting: easier to explain & straightforward to administer.

#725 ::: Annie Y ::: (view all by) ::: May 20, 2015, 02:14 AM:

@723 felice

And allows a perfect slate to sweep the ballot in most categories.

And a perfect slide is very very unlikely to happen (it will work only if all the works stay together and noone had nominated anything outside of the slate nominations and the slate is actually winning the top position - otherwise there is a work that in an elimination will kick the whole slate out). In every other case it behaves well. It's a corner case. We cannot make a rule that can kill all corner cases. Guaranteeing the demise of a perfect slate under this rule is as easy as someone adding one of the slate works on their ballot or someone forgetting to copy a work or something like that. :) If someone can orchestrate a perfect slate at the nominations, well...

OK - so what do you propose then? :)

#726 ::: Annie Y ::: (view all by) ::: May 20, 2015, 02:21 AM:

[TIES]

@724

A big amount of works will happen if a lot of works tie both at points and at nominations around the 5th place. Looking just at nominations at London for novels, we don't have a single tie in the first 16 so the number of nominations will be kicking them out cleanly and orderly. Quite honestly, a big ballot will happen only in the same conditions where we could have seen it today - with a lot of works with the same number of votes around #5. So this does not make it worse than the current system...

#727 ::: Annie Y ::: (view all by) ::: May 20, 2015, 02:27 AM:

(and addition to 726 because apparently I cannot finish a thought today)

Which does not mean that the new algorithm won't bring forth some new works instead of the 16 at the top now. But I will be really surprised if that leads to a big tie.*

* Yes. I know. Possible in theory. Let's talk realities though.

#728 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 02:46 AM:

Here is my attempt at wording, with both the "1,1 not 1,1,2" rule and the "no lookback" rule

"3.8.8.1: Matchup phase: In each round, the two works with the lowest point total(s) will be considered for elimination. If there are any other works which tie on points with one or both of these, those will also be considered for elimination.

3.8.8.2: Showdown phase: Of the works being considered for elimination, the one that appears on the fewest number of ballots will be eliminated. In the event that two (2) or more works are tied for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, then all members of that tie will be eliminated.
Eliminated works are removed from all nomination ballots for subsequent rounds as if they had never appeared on the ballots."

I think it's worth giving names to these two sub-phases, so that we can refer to them unambiguously in the FAQ. I like "matchup" and "showdown", but if somebody can find better names for these phases, then great. (Note that as I've set it up, we could also just remove the verbal names and call them "3.8.8.1" and "3.8.8.2"; but I still slightly prefer having descriptive words for them, if possible.)

#729 ::: Annie Y ::: (view all by) ::: May 20, 2015, 02:56 AM:

Selection and Elimination phases maybe?

Now I need to figure out which of the two I like more:

1,1,2,2 leads to 1,1 (both lowest elements and their ties) or leads to 1,1,2,2 (both lowest different numbers and their ties). My instinct still kicks towards 1,1 - despite running with the other up thread. I can live with the second though for simplicity sake.

Any chance we ca run the real data that we have through the two tie breaker rules?

#730 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 02:57 AM:

Oh, and if you're up this late at night, Keith, turn your volume down to one bar and watch the obligatory video.

#731 ::: Soon Lee ::: (view all by) ::: May 20, 2015, 03:32 AM:

"Pair-up & Elimination"?

I like "elimination" as a better descriptor of what the phase actually does.

#732 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 03:51 AM:

Bracketing and elimination?

#733 ::: Annie Y ::: (view all by) ::: May 20, 2015, 04:00 AM:

Comparison and Elimination
Identification and Elimination

#734 ::: junego ::: (view all by) ::: May 20, 2015, 04:11 AM:

"3.8.8.1: In each round, the two works with the lowest individual point totals are compared. (For ties see 3.8.8.....) Of the compared works, the one that appears on the fewest number of ballots will be eliminated and removed from all nomination ballots for subsequent rounds as if ......."

Above is my suggestion for wording of this section. It's usually clearer to separate the complications from the main explanation. Explain the main rule, then explain the complicated or edge stuff separately.

I think that if we have a
a) 1,1, 2 case on points, we treat the 1,1 as the bottom works that challenge each other. If they are tied on points and nomination tallies, we use the tie breaker method.

b) 1, 2, 2 (or 1, 2, 2, 2) case we compare all three (or more) works and eliminate the one with the lowest nomination tallies. If all are tied in nomination tallies, eliminate the lowest points.

In all cases, after 1 work is eliminated the point totals are recalculated. In all cases, if there is a tie in the lowest total point position, that tie should be resolved and points recalculated before determining the next challenges.

I'm pretty sure these relatively simple rules take care of almost all possible challenge point ties. This is simple, understandable, "fair", and consistent.

#735 ::: J Thomas ::: (view all by) ::: May 20, 2015, 06:36 AM:

@723 felice

[TIES]

"Look back"

It makes sense in a top-down system like STV: Alpha wins one of the spots, but then Beta and Gamma are tied. The people who voted for Alpha have technically already used up their votes getting Alpha in, but their opinion on whether Beta or Gamma is better is still valid, so looking back to see what they preferred before their votes were taken out is a reasonable way to break the tie. SDV-LPE is a bottom-up system that works completely differently, though; the order in which the less popular works were eliminated before the current round is irrelevant.

So with STV we've thrown away the information about what people who already picked winners want, and we can get it back. With SDV you didn't throw away what people who picked losers wanted, it's there now in the current scores. Previous scores depend on the order that others were removed and are quasi-chaotic, a few votes difference could change them around. So if you want to use previous scores for a tie-breaker, possibly the original scores would be better than later ones, except that it gives only one chance to break the tie while gradual lookback could give many chances.

"The way I see it, we have too many cases. That's what's making the tie breaking rules seem so complicated."

It seems to me that all the complexity stems from the possibility of ties not being broken. If we include a rule that always breaks every tie, then all our problems go away.

The complexity comes from ties. ;) Double elimination also avoids complexity, you just didn't like the side effects.

Putting aside the rights of individual works and individual voters, the following can happen:

1. Both eliminated works would soon be eliminated anyway, it makes no difference. This is likely. Or

2. If either eliminated work were kept, it would eliminate some others before it was eliminated, and a ripple effect would change the result. This one would still be eliminated, though. This may be likely, but less likely near the end when there's less room for ripples.

3. If either eliminated work were kept, it would eliminate some others before it was eliminated, but different others so that the three choices give us three different results. If we had criteria to decide which is best, we could apply it to the three results and decide whether eliminating both was worse than the average of eliminating one. But we don't have a way to do that. Anyway, three different results is probably pretty unlikely.

4. Maybe the eliminated works are high-vote/low-score works. If one of them survives this challenge it will eventually be a finalist, replacing another finalist that has fewer votes. Finalists with more votes are better because more people voted for them. So this is a bad outcome.

Apart from slates, this bad outcome should be fairly rare. It happens less than 10% of the time that two of the top five on votes challenge each other. (A lot depends on that 10% idea, we should look at it carefully and see how confident we are.) It happens when they both have so many more shared votes that they are the two lowest on points -- more shared votes than any that have fewer votes. To tie they must have the same number of votes that are shared the same amount. If that does happen, eliminating both leaves room for two with fewer votes to replace them. Instead of just one.

That should be fairly rare. It usually isn't that bad, the #6 and #7 tend to have almost as many votes as the #5 and #4. And the ones that lose did tie for last place on points.

Any such tiebreaker will be arbitrary, but I don't see that as a problem.

It's a problem, but it might still be the best way to go depending on the problems we run into otherwise.

For any tie that matters, picking an arbitrary winner will benefit more people than it hurts.

I don't see that at all. There will be 5 or more finalists. Do something to change which ones win, and just as many are hurt as are helped. Except when we get extra winners, which is a bad thing we must sometimes accept. It can't help more people than it hurts. What it might do is help the people who ought to be helped and hurt the people who deserve to be hurt.

There's nothing wrong with having an occasional rare event that hurts tied slate works. They get more than their share from the normal course of events, because they are organized as slates. If they rarely lose from a tie, that won't be enough to balance their inherent advantage.

If we break all ties, "Always compare the two works having the lowest total point values (or more if tied)" would work for me.

Agreed. The main problem I have with breaking all ties is that it's basicly arbitrary. It doesn't cause any horrible practical consequences that I've noticed.

Double elimination is better, but I could live with this.

#736 ::: P J Evans ::: (view all by) ::: May 20, 2015, 08:52 AM:

If you're left with six names, that's fine under the current rules. So is having four left. The problems are ending up with fewer than four or more than six.

#737 ::: J Thomas ::: (view all by) ::: May 20, 2015, 09:17 AM:

@736 P J Evans

If you're left with six names, that's fine under the current rules. So is having four left. The problems are ending up with fewer than four or more than six.

That can happen with anything but "break all ties", though it should be very rare.

Say you have 4 works that all have the same number of votes, and suddenly when those four and three others (with higher scores) are left, for the first time they all have the same points. If you keep them all you have seven, if you eliminate them all you have three.

Similarly for 5 works that all have the same number of votes, when there are 8 total left. Or Six works with the same number of votes when there are 9 total left.

If we always break a tie by leaving one, there can't be a problem like this. If you get six with the same votes and the same score then you throw one away. If you have 5 next time, you throw one away. If the other 3 are tied for lowest place after that, you throw one away. If there's a tie when you're down to 6, you have six finalists. If you can throw one away then, keep 5.

I don't see any problem with the "throw one away" approach except that you're arbitrarily throwing one away. It perfectly handles all the problems of ties except for that.

#738 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 10:03 AM:

Morning all!

Getting caught up on the posts from last night, but here what I like for the name of the two phases, since I feel they explain what's actually happening:

Eligibility phase
Elimination phase

The first I'm flexible on, so long as it gives the sense of "we're determining who is eligible for elimination". I think most of us agree on the second name.

Kilo

#739 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 10:07 AM:

Double jeopardy phase? da-DA-da-da, da-DA-da...

#740 ::: Yarrow ::: (view all by) ::: May 20, 2015, 10:18 AM:

Kilo @ 738:
Eligibility phase
Elimination phase

How about

Vulnerability phase
Elimination phase

("Eligibility" to me sounds confusing, particularly if we're talking about "eligible works" elsewhere.)

#741 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 10:30 AM:

Yarrow@740: You're right that we use eligible elsewhere in the proposal, so I can see avoiding it here.

Annie mentioned selection phase, that also gives the same sense, I think. Opinions?

K

#742 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 10:58 AM:

Jameson@728:

Your 3.8.8.1 rule sounds like you actually support 1,1,2,2. Am I misreading that?

K

#743 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 11:02 AM:

I like vulnerability, elimination.

@742: I think you are misreading. Two works with the lowest point total(s) is 1,1; since 2 is tied with neither, that leaves just 1,1. This is not the same as "works with the lowest two point totals", which is 1,1,2,2.

#744 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 11:08 AM:

One way to be even clearer is:

"In each round, the two works with the lowest point total(s) will be considered for elimination. If there are not exactly two such works due to a tie, such as a three-way for last place or a tie for second-to-last place, all tied works which could be considered to be among the last two will be considered for elimination."

#745 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 11:15 AM:

"If, because of a tie, there are not only two such works, then all tied works which could be considered to be among the last two will be considered for elimination. For instance, if three works are tied for last place, or if two works are tied for second-to-last place, then all three of the lowest works will be considered for elimination."

#746 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 11:18 AM:

Junego@734: I agree tie breaking rules should be separated out. The system itself is simple, but I think ties are making things more complicated than they need to be.

Kilo

#747 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 11:23 AM:

Jameson@728:

I'm actually referring to this part:

If there are any other works which tie on points with one or both of these, those will also be considered for elimination.

This says to me that 1,1,2,2 is okay.

Kilo

#748 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 11:38 AM:

If the two lowest works are 1,1, then 2,2 are not tied with either. That said,I like my later suggestions better, because I see how it could be confusing.

#749 ::: Duncan J Macdonald ::: (view all by) ::: May 20, 2015, 11:44 AM:

Jameson Quinn @744:

"In each round, the two works with the lowest point total(s) will be considered for elimination. If there are not exactly two such works due to a tie, such as a three-way for last place or a tie for second-to-last place, all tied works which could be considered to be among the last two will be considered for elimination."


Which is equivalent to Keith "Kilo" Watt @501:
"3.8.8.1: In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots."

We can leave well enough alone at this point.

#750 ::: Cassy B. ::: (view all by) ::: May 20, 2015, 11:50 AM:

On naming: I really like "E Plurbus Hugo" -- but is that too USA-centric for Worldcon?

#751 ::: Duncan J Macdonald ::: (view all by) ::: May 20, 2015, 12:02 PM:


Kilo


Jameson Quinn @748

If the two lowest works are 1,1, then 2,2 are not tied with either. That said,I like my later suggestions better, because I see how it could be confusing.

Given the set of 1,1,2,2; then the least value is 1. Include its tied works. (1,1). The second-least value is 2. Include its tied works. (2,2). The result of least and second-least is 1,1,2,2.


#752 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 12:03 PM:

[TIES]

Okay, summary post one: ties with points

The question is essentially, "Do we -have- to include the second-lowest point total (however many works that might be) as being eligible for elimination?"

The intent of comparing lowest and second lowest point totals is just so that we have two works to compare. This is because we want -elimination- to be based on number of nominations; we want -eligibility- for elimination to be based on points. That is the fundamental philosophy of SDV-LPE, so we need to keep that in mind.

Based on this, I think most people here are saying, no, there's no need to consider the second-lowest if we don't have to -- we are still determining eligibility based on points.

So, the only reason to even consider 1,1,2,2 (or 1,1,2) is if this will tend to help with the tie in number of nominations rule. In other words, will including the second-lowest point totals reduce the number of cases where we have a tie for both points and number of nominations?

What are your thoughts?

I'd love to run some sims and see what we get, but I can't get to my computer right now. Let's start with general and most likely cases first, then think about edge cases.

If you don't mind, I'd like to focus the discussions on specific questions for a bit. I think we've got some good ideas for the possibilities at this point, so let's see if we can't focus some light on them using some specific lenses.

Thanks,
Kilo

#753 ::: junego ::: (view all by) ::: May 20, 2015, 12:07 PM:

I like

Selection phase
Elimination phase

Don't like "vulnerability", not sure why.

#754 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 12:10 PM:

@749: no, they're not equivalent. @501 includes 1,1,2,2 where @744 includes just 1,1.

@730: while I have that song on my head, new lyric:

I used to want voting fair and stable too,
then I took Ken Arrow to the knee.
Just ask the marquis de Condorcet, he'll tell you that it's true,
that I took Ken Arrow to the knee.

#755 ::: junego ::: (view all by) ::: May 20, 2015, 12:32 PM:

Kilo @ 752

[TIES]

Okay, summary post one: ties with points
The question is essentially, "Do we -have- to include the second-lowest point total (however many works that might be) as being eligible for elimination?"

My gut reaction is "no".

The intent of comparing lowest and second lowest point totals is just so that we have two works to compare. This is because we want -elimination- to be based on number of nominations; we want -eligibility- for elimination to be based on points. That is the fundamental philosophy of SDV-LPE, so we need to keep that in mind.
Based on this, I think most people here are saying, no, there's no need to consider the second-lowest if we don't have to -- we are still determining eligibility based on points.

This is my position. We have to resolve double ties (points and nomination tallies) occasionally no matter what. Just resolve when they are at the lowest point position for the sake of simplicity.

So, the only reason to even consider 1,1,2,2 (or 1,1,2) is if this will tend to help with the tie in number of nominations rule. In other words, will including the second-lowest point totals reduce the number of cases where we have a tie for both points and number of nominations?

Instinctively my head says it shouldn't matter, but not a math boffin. Someone else will have to crunch numbers. Unless there is some really large advantage, I think we should just keep to the simple rule of "select the bottom two works, compare challengers, eliminate least supported/popular, plus Look-Back (or whatever) for ties (which are *only* ties if both points and tallies are the same).

#756 ::: junego ::: (view all by) ::: May 20, 2015, 12:42 PM:

[TIES]

Oops, forgot only when there is a point tie at second lowest point total and only 1 challenger at the lowest point total should the rule be slightly more complicated. I posted it upthread but:

"compare lowest 3 (or more) challengers, eliminate least supported, if all are tied in nominations, eliminate lowest points."

#757 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 12:52 PM:

@755: I agree, it shouldn't matter. I am a bit of a boffin, but I haven't actually done the sims, so you shouldn't trust me 100% on that.

So my vote is for simplicity: bottom two works (three or more only if a tie makes it unavoidable); eliminate lowest on approvals, if tied eliminate lowest points, if still tied eliminate both.

I'll endorse "selection" and "elimination", if that's what other people like. Some more options for replacing "selection":

filtration
winnowing
screening

Hmm... that last one looks good to me. What do you all think?

#758 ::: Annie Y ::: (view all by) ::: May 20, 2015, 01:16 PM:

I still prefer selection - it is kinda... self-explanatory.

PS: I am catching up on what else happened while I was not looking.

#759 ::: Jameson Quinn flags a moderator ::: (view all by) ::: May 20, 2015, 01:16 PM:

Since no moderator has chimed in with a ruling, I guess it's OK to say that my gofundme is almost half-funded already. Thanks a lot to everyone who's donated!

But I'm really not trying to break the rules here, and any clear guidelines I could get would be appreciated.

#760 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 01:23 PM:

@759: Make that, more than half-funded, since I have been offered a free (that is, prepaid) attending membership.

#761 ::: Annie Y ::: (view all by) ::: May 20, 2015, 01:34 PM:

[TIES, LANGUAGE, THE 1,1,2,2 CASE]

The language in @501 - it sounds to me as if we are trying to say that in 1,1,2,2, we get all 4. I am not sure if it is the mathematician in me that is kicking my language consideration askew but this is how it sounds in my head. And I am pretty sure this is not Kilo is trying to say.

I'd say, let's keep it simple - grab the ones with least number of points. If less than 2, grab the next group as well. So how about:

In each round, all the works with the least total points will be compared. If there is only one work with the least number of points, all the works with the second-least number of points will be considered as well. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots.

#762 ::: abi ::: (view all by) ::: May 20, 2015, 01:43 PM:

Moderators are occasionally busy. This is an unpaid gig and I have a lot on my plate right now.

In this case, go ahead and mention it, Jameson. I'd want to check with the people whose name is on the URL before promoting it on the front page post, and they're otherwise occupied right now. But in-thread, just go ahead.

#763 ::: Aan ::: (view all by) ::: May 20, 2015, 03:09 PM:

Jameson @ #707, Cassy @ #750:

Non US person here. I think it's US-centric to think that "E Pluribus Hugo" is US-centric. ;P
I'm only vaguely aware that the original phrase is commonly known in the USA because of reasons, but know and understand the phrase itself just fine, and also favour it as a great and catchy title.

#764 ::: Erik V. Olson ::: (view all by) ::: May 20, 2015, 03:17 PM:

For declined nominations, the answer seems simple enough. You've stopped the process too soon.

When you hit 14 left, you need to start recording. The one you eliminated to leave 14 is the 15th placed work. Run another round. For simplicity, I'll assume only one is eliminated each time. Handle ties normally. The one that's eliminated this time was the 14th placed work. Repeat until you eliminate the work that leaves you five. That work was the 6th placed work, and you have your preliminary final ballot.

You now do a check on validity. If any are not valid nominees that you didn't catch earlier, you eliminate and promote up. You then call your current five and ask if they accept. If any of them decline, you promote #6, then #7, until you have 5 accepted and valid nominations.

We show the top 15 nominations in the Big List after the award ceremony, so this is data you need to acquire, and if this system can't give it to us, this system will have trouble getting traction.

And, yes, if you were at 8 left and ended up dropping 3 on the next round, you dropped to 5 and had a three way tie for 6, and if someone on the first cut ballot refuses, you have no choice but to expand the final ballot to 8, and make three more phone calls. That's how it works now, so no worries there -- expect behavior is expected.

This gives the same listing that we have now and that the general fannish population would expect -- the top fifteen nominees, as selected, with notations if top-five nominees didn't appear on the final ballot as to why (refused or not valid in category.)

As to ties -- The Hugo has *NO PROBLEM WHATSOEVER* with ties. We're good with ties. We are happy to expand the ballot with ties. If they match on points, expand the ballot. Ties are OK. Don't spend an iota of space complicating things with ties.

More importantly, if you do, this will be the thing that will become the nightmare in the business meeting, and that could very well lead to failure.

Accept the ties. Cherish the ties. When in doubt, let the voters resolve the tie problem in the best place -- on the final ballot!

#765 ::: Annie Y ::: (view all by) ::: May 20, 2015, 03:21 PM:

Cassy B. @ 750

As I said somewhere upstream - we are all geeks, a little Latin never harms. :) I did not even realize it is a US term (Aan @763 - thanks for mentioning that) -- plus it fits what we are doing.

#766 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 03:32 PM:

Annie@761: Actually, you're reading it correctly. My intent in 501 was for 1,1,2,2 the way that was written.

But given the lens suggested in 752, I can see that 1,1,2,2 is not -necessary-, and if it doesn't help with reduced number of nomination ties, it probably is better to stay with (1,2), (1,1), or (1,2,2) being the only cases for the elimination step. (I think I can write a clear way to describe it once we decide.)

I have a thought that 1,1,2,2 might help with nomination ties, but it's still ill-formed, so I'm more interested in what other people have as insights.

K

#767 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 03:56 PM:

Erik@764:

Yes, I agree with you regarding ties when we are considering the 5th finalist -- expanding the ballot is definitely the way to go. The issue we're dealing with is ties before that point. The goal is to eliminate the least popular works so that we converge to the five finalists. Early in the game, the tied works are obviously the least popular, so shouldn't be kept, but the question before us is whether we get rid of both (the original system) or find a simple way to break the tie at this stage and just keep one. The problem with the latter occurs when you start to get layer upon layer of tie breaking rules, which, as you point out, is not something Hugo voters really want to deal with.

But I'd like to hold off on that question until we've resolved the first question I posed in 752. I think we can come to agreement on this easier one first, but I feel that we need to focus our efforts just a bit.

Kilo

#768 ::: Cassy B. ::: (view all by) ::: May 20, 2015, 04:09 PM:

For non-USAns, "E Plurbus Unum" is on the Great Seal of the United States; it's a sort of quasi-offical motto. (The official legislatively sanctioned motto is "In God We Trust", but until just now when I googled it to be sure, I honestly thought "E Plurbus Unum" was officially the US motto. Memory is a funny thing...)

That's why I was afraid it might be too US-centric. Glad to hear that it doesn't bother the nonUSAns. Worldcon, and the Hugos, is for the whole world, after all, whatever certain despondent or diseased canines may think....

#769 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 04:11 PM:

Me@766:

Okay, I've worked out the case I was thinking of, and while in that one case, it prevents having to look at ties for nominations, it won't -eliminate- the need to look at ties for nominations, so I think it's ultimately complexity to no good end.

So, for ties in points, can anyone think of a reason to compare 1,1,2,2 in addition to (1,2), (1,1), and (1,2,2)? At this stage, I really don't, and we are still staying true to the simple SDV-LPE principles if we don't include it -- we are simply selecting the two lowest-scoring works to be eligible for elimination (so 1,1 or 1,2). The only possible exception is if there is a tie for second-lowest, so in that case we must include all second-lowest scoring works (1,2,2) or we won't be left with two works to compare.

I'm pretty happy with this, what do you guys think?

Kilo

#770 ::: Cassy B. ::: (view all by) ::: May 20, 2015, 04:21 PM:

@769, I'm all for reducing complexity, so it sounds good to me.

No opinion as to how to break ties, however, but I do agree that until we get to the 5th place we have to do something....

#771 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 04:28 PM:

@769: I agree; you probably knew that.

#772 ::: Joshua Kronengold ::: (view all by) ::: May 20, 2015, 04:47 PM:

I'm glad people other than me like E Pluribus Hugo. It's nicely descriptive without being preachy, so I'm quite fond of it. (and while yes, the base motto is on the seal of the US, it's hardly jingoistic enough to worry about). If we wanted to be slightly less US centric, we could hearken back to Augustine, but I don't think we need to. http://en.wikipedia.org/wiki/E_pluribus_unum

Q: Why not just state that the two (not "at least two", not "in case of ties", just two) lowest scoring works are Selected for potential removal. And then the one work of those which has the fewest number of votes is removed.

That way, we can cover how to break ties (and when -not- to break ties) in a separate section, but the base rules are very simple -- two works enter, one work leaves!

Something like:

3.8.8.1: In each round, the two works with the least points (breaking ties, if necessary) will be compared. Of those works, the one(s) that appear on the fewest number of ballots (also breaking ties as needed) will be removed from all nomination ballots for subsequent rounds as if they had never appeared on the ballots.
3.8.8.2: In the event of a tie, first look at the main consideration (score or number of votes) that is not the one that produced the tie. If both match, then instead use some other fair method to break the tie; ties should always be broken, lest voters be unfairly have all their nominations removed at once.
3.8.8.2.1: If a tie is encountered in either step (Selection or elimination) and the number of remaining candidates is lower than the number of expected nominees, elimination is finished, and all remaining works are added to the ballot.
3.8.8.3: Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are removed from all ballots and treated as if they had never appeared on a nomination ballot. Members’ points are reallocated equally among their remaining nominated works, if any.


Also, we want to look at the 4 and 6 measure, and if possible make sure our measure is compatable with it without merge conflicts. There's no particular reason our proposal has to directly conflict with "4 and 6", and it's better for everyone if they can be considered independently. This may necessitate rewording.

#773 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 04:55 PM:

Joshua@772:

I agree about not conflicting with 4/6, so the most recent draft has all of our sections referring to "up to the maximum number of finalists" and the like.

I also agree with handling ties in a separate section -- and I think that section should be as simple and straightforward as possible.


Once we've given people a chance to weigh in on the tie for points lens question in 752, I'll propose the next one for ties in nominations.

Kilo

#774 ::: Annie Y ::: (view all by) ::: May 20, 2015, 04:58 PM:

Kilo@766 Ah, good. I was starting to wonder where the disconnect in my mind is.

@769 After spending most of yesterday defending exactly this (I swear, I will be dreaming of my 1,1,2 example for a while but I think it helped everyone to see where we were having a disconnect as a group) I guess it is clear that I agree :) Keep it simple - all with the lowest points and if only one is there, all with the second-lowest as well. Have fun writing it down properly.

#775 ::: Buddha Buck ::: (view all by) ::: May 20, 2015, 05:40 PM:

[TIES]

In response to #752, I would say no, it is not necessary to include the second-lowest point total in the selection phase. If the goal of the selection phase is to select two candidates for elimination, then it makes sense for it to select the smallest set of at least two candidates for elimination.

If I am understanding the notation being used, I would support 1,1; 1,1,1; etc or 1,2; 1,2,2; 1,2,2,2; etc, but not 1,1,2 or similar.

I may have missed the discussion which came to this conclusion, but is there any particular reason why selecting 1,2 is better than selecting 1?

#776 ::: Annie Y ::: (view all by) ::: May 20, 2015, 05:48 PM:

@775 Buddha Buck

but is there any particular reason why selecting 1,2 is better than selecting 1?

You mean why not always eliminate the one with the lowest points without looking at the votes if there is no tie on points at the bottom? The idea is always to do 2 levels of comparison. Find the 2 (or more with ties) which are weakest on points; then eliminate the one that is weakest on nominations amongst them.

This way a work that is low on points because it is on everyone's ballot with something else but is on more ballots will beat a work that is higher on points because it is an only work but is on less ballots.

#777 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 05:49 PM:

Buddha@775:

We need to compare nominations as well as points in order to get the slate-limiting effects we are looking for. That's the main reason.

Kilo

#778 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 06:03 PM:

@775: Also, comparing nominations as well as points almost eliminates the incentive for bullet voting.

#779 ::: junego ::: (view all by) ::: May 20, 2015, 06:33 PM:

Keith "Kilo" Watt @ 769


So, for ties in points, can anyone think of a reason to compare 1,1,2,2 in addition to (1,2), (1,1), and (1,2,2)? At this stage, I really don't, and we are still staying true to the simple SDV-LPE principles if we don't include it -- we are simply selecting the two lowest-scoring works to be eligible for elimination (so 1,1 or 1,2). The only possible exception is if there is a tie for second-lowest, so in that case we must include all second-lowest scoring works (1,2,2) or we won't be left with two works to compare.
I'm pretty happy with this, what do you guys think?

Since this is pretty much my understanding of what the process was -supposed- to be back at #666 (oooooooh, scary #) yesterday, I'm more than happy to agree!

#780 ::: junego ::: (view all by) ::: May 20, 2015, 06:49 PM:

On the 4/6 proposal, didn't Jameson or JT or someone say that this would slightly improve defense against coordinated voting when combined with SDV-LPE? Or am I misremembering?

#781 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 07:04 PM:

Junego@780:

It's actually slightly better if they nominate more works than fewer, but not really enough to make a huge difference. I think we're in good shape either way.

K

#782 ::: J Thomas ::: (view all by) ::: May 20, 2015, 07:07 PM:

@769 Keith "Kilo" Watt

>... we are simply selecting the two lowest-scoring works to be eligible for elimination (so 1,1 or 1,2). The only possible exception is if there is a tie for second-lowest, so in that case we must include all second-lowest scoring works (1,2,2) or we won't be left with two works to compare.

That looks good to me too.

#783 ::: Soon Lee ::: (view all by) ::: May 20, 2015, 07:13 PM:

[SIGNING]

I am now a Supporting Member. (Membership name same as my commenting name here.)

I nominated this year because I was a member last year, but then the finalists were announced & being disheartened, I wasn't planning on renewing my membership.

But this proposal is a positive & constructive step forward for the Hugos, which is something we have seen precious little of since the finalists were announced. So I've paid to back this proposal & to add my voice to this year's vote. (I expect my name will get added to the Sasquan member list page at the next page update)

#784 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 07:29 PM:

Soon Lee@783:

Thanks for that. I just wanted to let you know how much your comment meant to me personally, and I'm sure to most of those here. I was pretty disheartened, too, but I think we're actually going to be able save the Hugos with this proposal if it passes. I think (hope) it's going to be something everyone can get behind.

Thanks again,
Kilo

#785 ::: David Harmon ::: (view all by) ::: May 20, 2015, 07:41 PM:

[SIGNING]

I'm also deeply impressed by what's been going on lately: Both the constructive efforts here, and the deep reflection I've been seeing within the fannish community.

#786 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 08:17 PM:

@780: re, 4/6:

The "6" part would combine well with this system (EPH voting?). The "4" part is actually a step in the wrong direction, but frankly, it's a small one. I think that the net impact of 4/6 when combined with this system would be essentially neutral to average slate percentages. The main difference 4/6 would make would be more things to read and bigger packets.

#787 ::: junego ::: (view all by) ::: May 20, 2015, 09:13 PM:

Thanks Jameson & Kilo on the 4/6. (Note to self, "Don't derail at this late stage.")

#788 ::: David Goldfarb ::: (view all by) ::: May 20, 2015, 09:59 PM:

Out of "matchup, selection, vulnerability" I like "selection" best. Put me down for "selection / elimination".

#789 ::: Jameson Quinn ::: (view all by) ::: May 20, 2015, 10:03 PM:

@788: and "screening"?

#790 ::: Keith "Kilo" Watt ::: (view all by) ::: May 20, 2015, 10:47 PM:

All:

Okay, I think we've reached consensus on the question of how to handle ties in points. We will compare the two lowest point totals, which will be either 1,1 or 1,2. In the event of a tie for second-least points, we will select all the tied second-lowest works (1,2,2), otherwise we wouldn't have any works to compare for the elimination phase. (Addendum: The names "selection phase" and "elimination phase" seem to be the most descriptive and most popular, so we'll go with that unless anyone strenuously objects.)

Well done, all! Let's consider the question closed unless there is a -strongly- compelling reason to revisit it.

I'll post the next lens question on the issue of ties in number of nominations soon. I think I've got a good way to focus our discussions, but since I'm currently in the middle of a codine high, I think I should wait until that wears off to type it up. :)

I'll report back soon!

Thanks all,
Kilo

#791 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 02:02 AM:

Okay, I'm still in a bit of a fog, so this isn't going to come out as clearly and concisely as I'd hoped. I apologize in advance! Also, long post warning, so please bear with me.


Summary post two: ties for number of nominations

This one has been major sticking point for us, because there hasn't been a clear way to distinguish between the various options. Ultimately, though, we need to go back to the goals for the elimination phase: to eliminate the least popular work from consideration.

However, we don't want there to be undesirable effects as a result of that elimination, and here is where we run into trouble: What does "undesirable" consist of?

When we were discussing RME vs. 6th place for declined nominations, it became clear that there was one major difference in the results of the two systems: With 6th place, if the declined nomination was a slate work, it was most likely to be replaced with a non-slate work. With RME, that slate work was most likely to be replaced by another slate work. There was a clear, unambiguous difference between the two, and that enabled us to choose our preferred method.

So, I propose that a preference for one tie breaker for total nominations must be accompanied by a similarly clear enunciation of its effects. Saying you get a different result isn't enough, as there isn't a "fated" result that is supposed to occur. Fandom, I believe, will accept a Hugo awarded under this system so long as we follow the system's own rules. If there is a clear difference of effect between systems, yes, that's something we can compare, but if not, then one set of rules -- and one set of results -- is as good as any other. Remember, the goal is to eliminate the least popular work. Effects that propagate out from that elimination are of much less concern, so long as those effects are consistent.


So, here are the major tie breaking systems for number of nominations that we have discussed:

1. If tied for total nominations, eliminate both works. The rationale is that both are by definition equally the least popular and should be eliminated.

2. If tied for total nominations, look again at number of points. Eliminate the one with fewer points. The rationale is that the one with more points was weighted more strongly by its nominators, or its nominators' other works were eliminated, so this is a reflection of the convergence we are trying to achieve.

2a. If the works are tied for both nominations and points, eliminate both works. The rationale is similar to number 1: These two works are even more equally the least popular and since there is no way to distinguish between them, we should just eliminate them both.

2b. If the works are tied for both nominations and points, look back to previous rounds. If there was ever a round in which the works were -not- tied, eliminate the one with fewer points. The rationale is similar to number 2: The work with more points is either weighted more strongly or is the result of the convergence process. It should be noted that the rationale here is somewhat weaker than in number 2, since the convergence has obviously changed since the round under scrutiny.


So, here we have rationales for each system, and these rationales should be our primary guides. However, we should also understand the effects of each of these systems (in the sense discussed with RME and 6th place). So, for this lens, I'd like for us to look at each system and describe the case(s) where the system makes a difference. If we can do that, together with the rationale, we should be able to choose between the cases.

A couple of ground rules, however. First, let's assume that a slate does -not- have perfect discipline. It's unlikely that they will in any event, and it is easy for slate organizers to ensure five of their ballots are different if they need to.

Secondly, we are by definition considering fringe cases here. Try to estimate the odds of any scenario you propose occurring when scaled up to realistic numbers.

Finally, as I've mentioned, I personally can't support an arbitrary tie breaker system. I think this would be a deal-breaker for a number of people at the business meeting as well. Frankly, I don't think I can sell it. I agree that having a single always-working tie breaker would be an advantage, but I can't think of a non-arbitrary one that we can use. I don't think it's worth jeopardizing the whole proposal for this edge case.


I know some of you have already come up with cases to distinguish between systems. Present those cases here, and let's look at them.

Thanks,
Kilo

#792 ::: felice ::: (view all by) ::: May 21, 2015, 02:42 AM:

Keith "Kilo" Watt @791: "Finally, as I've mentioned, I personally can't support an arbitrary tie breaker system."

Are you able to explain why it's a problem for you?

#793 ::: J Thomas ::: (view all by) ::: May 21, 2015, 05:25 AM:

@792 felice

"Finally, as I've mentioned, I personally can't support an arbitrary tie breaker system."

Are you able to explain why it's a problem for you?

"I think this would be a deal-breaker for a number of people at the business meeting as well."

If he's right about this, and it's a large enough number of people, then it isn't important to consider the reasons. Even understanding the reasons it's unlikely we could persuade enough of them to quickly change their minds.

So the question is how many people like that are there, and how important are they? If no one objects, we could ask the smofs what they think. Particularly if influential people like recent or future voting administrators oppose it, or if it looks particularly controversial, we should give it up. If they don't care more than the usual freewheeling argument, that would say it's likely not to be an issue.

#794 ::: felice ::: (view all by) ::: May 21, 2015, 06:30 AM:

[TIES]

Hmmm... how's this for a radical idea? If works are tied for elimination, both keep them all and discard them all. Ie, in the next round, distribute points to other works as if they'd been eliminated, but also distribute points to them as if they hadn't - a temporary point doubling. Call them ghost works at this point. If at any point a ghost work ties with a regular work on both points and nominations, the ghost work loses automatically; and if all other ghost works from a particular tie are eliminated, the survivor stops being a ghost and reverts to normal status. If two ghost works are tied, eliminate them both; they haven't benefited from point reallocation, so neither has a chance of winning.

I'm not sure if it's a good idea or not, but it's something to consider :)

An example for two finalists:

4 x AB
1 x A
3 x ACD
4 x BD
4 x C
2 x D

A and B are tied for elimination on 4 points and 8 nominations each, while C has 5/7, and D has 5/9. A and B become ghosts.
Next round, gA has 6/8 (as if B was gone), gB has 6/8 (as if A was gone), and C has 5.5/7 and D has 7.5/9 (as if both A and B were gone). C goes up against A and B, and is eliminated.
Next round, gA has 6.5/8 (as if B was gone), gB has 6/8 (as if A was gone), and D has 9/9 (as if both A and B were gone). Ghost A goes up against Ghost B, and wins on points. So the finalists are A and D.
If there was another round, A would have 6.5 points, and D would be back down to 7.5 points (since A is no longer a ghost).

#795 ::: J Thomas ::: (view all by) ::: May 21, 2015, 06:50 AM:

@791 Keith "Kilo" Watt

What does "undesirable" consist of?

tldr; I am going to state the obvious for a bit, new stuff at the bottom.
-------------
The problem we are trying to palliate is that slates get a lot of votes by a method we consider wrong. We can't create consensus so that other works can also get a lot of votes.

Our solution is a voting system that encourages votes from different sources. If one pair of nominees and a second pair of nominees gets roughly the same number of votes, but the second pair overlaps more so the votes come from a fewer number of voters, then it's more likely that one member of the second pair will be eliminated.

This helps, but in 2015 there were so many slate voters compared to others that in most arenas it would not have lowered slate wins to anything like proportional to their numbers.

We prefer that nominations with more votes should win, but we also prefer that the winners should have votes from a variety of sources and not be too concentrated from a few sources.

Sometimes that second goal takes precedence over the first goal.
-------------

1. If tied for total nominations, eliminate both works. The rationale is that both are by definition equally the least popular and should be eliminated.

This is pretty good. It's clear and consistent and simple. Sometimes it will eliminate works that have a lot of votes, and when those aren't slate works some people will think they should have been kept. But they were works with a lot of votes that had the most votes shared with other works -- it's tha second criterion that we also care about.

This way, pairs of slates will occasionally have the lowest scores and if they have the same number of votes, both will be eliminated.

Without the 4/6 rule that will almost always leave one slate work on the ballot. The last slate work would have to meet up with a nonslate work with more votes, and there would have to be 5 of them for it to go up against one of them. The last slate work cannot lose unless it ought to.

If slates try not to have tied votes, that's OK, they'll still match up against each other and lose one at a time some, in the usual process. If they make a mistake and have tied votes by accident that's OK too.

There's nothing wrong with any of this.

It can be argued that nonslate works can be selected that have a low score because they share a lot of votes with *each other* and so their voters lose out. If only one lost, their votes would be transferred to the other, but when both lose their votes are just lost unless they voted for something else that survives. Yes, that will occasionally happen. When it happens to slates it's a good thing. Maybe. In 2015 the slate would still have gotten more than its share in most categories, I wouldn't shed a tear if it lost out completely once. When it happens to nonslates it's a moderately bad thing. More than a third of the time, another work that inherits the points will lose anyway, but it's sad when a ballot loses its last chance and when it loses its two last chances at the same time, that's even sadder.

As I understand it, this is the main reason not to eliminate both when they have the lowest votes. It isn't important to me, but it may be important to enough fans to be an issue.

System 1.0 isn't perfect but it's pretty good.

#796 ::: David Harmon ::: (view all by) ::: May 21, 2015, 06:57 AM:

J Thomas #793:

Your last sentence attempts to insert a massive non-deterministic digression into our present deliberations. ("Ask the smofs", find out/decide just how important the objectors are, spend some time arguing about that....) Keith himself is both a representative of the SMOFs, and a topic expert in what the "other SMOFs" (that is, the Business Meetings) might be persuaded into. If he says something's a non-starter, I'm inclined to believe him.

Accordingly, you, and we, need to stick with it isn't important to consider the reasons. Even understanding the reasons it's unlikely we could persuade enough of them to quickly change their minds.

That said, I think I mostly understand where Keith is coming from, and it's what you might call a "philosophy-of-governance" issue: As he's previously put it, the algorithm needs to be deterministic, such that rerunning it from the same ballots will reliably give identical results each time. I gather that first-nomination timestamp, while otherwise appealing, is "no good" here because it depends on what order the officials pick up the ballots.

#797 ::: J Thomas ::: (view all by) ::: May 21, 2015, 07:16 AM:

@791 Keith "Kilo" Watt

2. If tied for total nominations, look again at number of points. Eliminate the one with fewer points. The rationale is that the one with more points was weighted more strongly by its nominators, or its nominators' other works were eliminated, so this is a reflection of the convergence we are trying to achieve.

This is a small complication. We have a new rationale for comparing, which only matters when the votes are tied.

The net effect is to reduce the number of ties, which is important if you think the outcome of ties is bad.

I figure this can't do very much harm. It's a complication, but a simple complication. It saves some works from immediate elimination, but there's a plausible reason to save them. If an author came to me in later years and said, "This voting system sucks, I was tied and the other guy got an unfair break" I would first be pleased, it would mean that somebody believed in the voting system and didn't just consider it a crutch.

I could respond, "Maybe it isn't exactly fair, but it didn't hurt you any -- yours would have been eliminated either way. Maybe next year it will help you."

And if he said, "This voting system sucks, I was about to be a finalist and some other guy got an unfair break when he should have been eliminated and he beat me" I could respond, "It isn't that bad a rule, people had objections to the other way too. Maybe next year it will help you instead of hurt you."

This version can't do much good either, unfortunately. The same people who objected to 1.0 are still just as dissatisfied because there are still ties that have results they don't like.

It can't do much good, it can't do much harm. It isn't worth opposing if somebody wants it.

#798 ::: J Thomas ::: (view all by) ::: May 21, 2015, 07:32 AM:

@796 David Harmon

Keith himself is both a representative of the SMOFs, and a topic expert in what the "other SMOFs" (that is, the Business Meetings) might be persuaded into.

I didn't realize that! I knew that was true of Kevin Standlee (who I believe will be chairing the meeting) and Joshua Kronengold. P J Evans is a highly respected member. I believe all of them expressed personal distaste for random selection. I don't remember them predicting what could pass at the meeting, leaving a possibility that their own objections were not predictive.

If we have experts here who predict it wouldn't work, that's good enough for me.

#799 ::: Duncan J Macdonald ::: (view all by) ::: May 21, 2015, 07:38 AM:

[TIES]

Keith "Kilo" Watt @791:

I'd like to record my preference for Tie Breaker 2b. "If the works are tied for both nominations and points, look back to previous rounds. If there was ever a round in which the works were -not- tied, eliminate the one with fewer points."

That is about as non-arbitrary as we can get -- especially if the Look Back goes all the way to the initial round and they remain tied.

#800 ::: Jameson Quinn ::: (view all by) ::: May 21, 2015, 07:42 AM:

Status update on the GoFundMe campaign:

Currently at $720 plus a pledge of an attending membership ($210 value) = $930. The goal is $1400, but I've said I'll definitely go if we reach $1000 in cash. Thanks again to anyone who's donated.

However, the momentum seems to be fading a bit; there hasn't been a donation since yesterday afternoon, which is clearly the biggest gap so far. In order to recover momentum, I'd love the opportunity to write some guest posts about election theory on other people's blogs. I'd write to your specifications — topic, tone, length — and I'd include a quick link to the gofundme at the end.

To sweeten the pot, I'll put up $20 of my own money for every guest post someone lets me write. You can send it to the charity of your choice, or even keep it for yourself if that's what you want. Of course, I'd suggest that one relevant charity would be electology.org, and so if you want I'll count it towards the $1400 goal.

#801 ::: J Thomas ::: (view all by) ::: May 21, 2015, 08:11 AM:

2a. If the works are tied for both nominations and points, eliminate both works. The rationale is similar to number 1: These two works are even more equally the least popular and since there is no way to distinguish between them, we should just eliminate them both.

If we don't do 1.0, 2.a is the obvious next choice. All the same reasoning applies. The objection is mainly that it isn't fair to the works that have tied for lowest score and same vote -- that it would be fairer to them if only one of them was eliminated. Or fairer to any voters who had those two as their last surviving nominations.

This is a complication over 1.0 that can't help much and can't hurt much, but if it stops the dissention that's worth a lot.

Except it doesn't.

#802 ::: Jameson Quinn ::: (view all by) ::: May 21, 2015, 09:19 AM:

@800: Note in terms of my $20/guest-post pledge: my family income is, technically speaking, below the poverty line, so even though as a Harvard student with no housing expenses I'm not in any sense needy, a pledge that could reasonably go past $100 is not a trivial amount for me.

#803 ::: Pfusand ::: (view all by) ::: May 21, 2015, 10:07 AM:

J Thomas @801 wrote:

The objection is mainly that it isn't fair to the works that have tied for lowest score and same vote -- that it would be fairer to them if only one of them was eliminated.

I do not accept that definition of "fairer." They are indistinguishable, and so should be treated equally. If there are at least 5 works with more points (since that is the order they are in), then they should both be discarded. If there are fewer than 5, then we have a tie, and there will be six works on the final ballot. We're done.

(All of this is ignoring withdrawals, invalid works, and 4/6 passing.)

#804 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 10:17 AM:

Morning all!

I'm reading through the morning's posts, but I did want to mention one thing. In fairness, I don't know that I'd be considered a SMOF, as that's generally the group of con organizers. I am a writer of SF, though, and like most writers, I've got a fairly extensive network of contacts among other writers. So, my informal survey probably shouldn't be considered statistically valid, necessarily, but given the overwhelming opposition to arbitrariness in Hugo awarding that I did find, I think it's enough to seriously consider arbitrary methods to be a problem.

To answer Felice's question in 792, as a writer myself, I would strongly prefer to find out I didn't get a Hugo because not enough people liked my work than because somebody just didn't submit their nomination ballot early enough. The former seems completely fair, the latter doesn't.

Just my thoughts,
Kilo

#805 ::: J Thomas ::: (view all by) ::: May 21, 2015, 10:24 AM:

2b. If the works are tied for both nominations and points, look back to previous rounds. If there was ever a round in which the works were -not- tied, eliminate the one with fewer points.

This also is mostly harmless.

I don't like it for the following reason:

Say that F and G are tied. If F had one single point more there would be no question, eliminate G. If G had one point more then without any question we would eliminate F. Since they are tied we look for some excuse to save one, and pick an arbitrary reason. There are a great many criteria we could choose to decide which one to save. We chose to look at the score because we already had it, and it was an obvious choice. We chose to look at past scores because we already looked at score once. That's why we chose this way, when many of the other ways might be more worthy.

It doesn't solve the original problem because there could still be cases where the scores are tied all the way back. Luckily its supporters chose not to look for third, fourth, and fifth backup tiebreakers. Two is enough. Anyway, if it's tied for everything all the way back, it's probably a slate.

I don't think it makes sense to assume that a slate can never arrange a tie if they try to (though they can often prevent ties if they try not to). If a slate *can* arrange a tie, can it do better this way? No. Slates that tie get double-eliminated. If that's a good thing, we still have it.

Nonslates that (rarely) tie for votes and points both? They will sometimes (maybe often) have a difference somewhere and one will be spared. I don't see that as important but some people do.

It's a big complication and not very important. I don't think it's worth much but we can live with it if enough people want it.

#806 ::: Kevin Standlee ::: (view all by) ::: May 21, 2015, 10:28 AM:

J Thomas and anyone else who didn't know:

Yes, I am the designated presiding officer of the 2015 WSFS Business Meeting. The rest of the staff can be found in the Sasquan Committee/Staff List under the WSFS division. Among my contributions to the convention web site is the Guide to the WSFS Business Meeting.

I've been trying (not always successfully because I am an opinionated loudmouth) to not get involved in the specific debates on various issues out there on account of my having to preside over them (and potentially use my casting vote if it comes down to it). But I'll answer technical questions about procedure. I want everyone to know what the correct procedure is so that debate can focus on substance, not confusion over procedure.

#807 ::: Jameson Quinn ::: (view all by) ::: May 21, 2015, 10:41 AM:

I have a weak preference for 2a or 2b over 1; but any would be fine, and I don't think it's a big deal.

Also, just for the record, I'd like to remind people that my sims show that there's no necessity for a 2c; that is, that 2b will almost certainly break all significant ties, even though it's not actually logically guaranteed to.

#808 ::: Joshua Kronengold ::: (view all by) ::: May 21, 2015, 11:37 AM:

The reason I'd far rather, at this point, punt tiebreakers to the administrators is that the administrators can put in place an overly complicated series of tiebreakers -- or can choose to just flip a coin.

The "best" tiebreak (or at least second order tiebreak) is clearly to look at rank, even though rank usually doesn't matter. This allows us to get a potentially meaningful priority where otherwise no preference is expressed. However, that tiebreak -is- best coded in the meeting if we choose to use it, because it can affect voting.

Order of processing is obviously no better than coin flip (except for being predictable given a given dataset). But it's no worse, either, and coin flip, while it's not fair to -works-, is -eminently- fair to nominators (if not as fair as letting their relative rank matter).

2b is obviously "good enough" (usually) -- but I want to give the admins enough license to do better than that if they have to, because SDV-LP functions much better if ties are broken. I also don't really -like- 2b -- because it feels to me like it contains hidden perverse incentives, but it's not that strong a feeling.

#809 ::: David Harmon ::: (view all by) ::: May 21, 2015, 11:50 AM:

J Thomas 798: I may be a little confused between Keith and Kevin, as I was dashing off a note before heading out (and I'm now "out" and on an iPad, so harder to review/respond in detail. My comment about it being a major digression still stands.

On another thread, I agree that "E Pluribus Hugo" is winful.

#810 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 12:01 PM:

Joshua@808: The only problem with letting the admins decide is that presumably the nomination-counting system will be automated, so any tie breaking that happens would probably have to be decided by the admins before the code is written.

All: One other data point to keep in mind, in all the simulations I ran with the actual 1984 data, all ties for both points and nominations went all the way back to the first round. In this data set, therefore, there was no functional difference between any of the tie breaker systems discussed above. So, in this one case, there was no reason not to stay with option 1, as that is by far the simplest. I believe, however, that felice or JT (forgot which) came up with a situation where it would matter, so that would be worth looking at again.

Kilo

#811 ::: Jameson Quinn ::: (view all by) ::: May 21, 2015, 12:11 PM:

@810: When I was doing my sims, I did see that there were a non-zero number of cases where 2b came into effect. I have no reason to doubt that all of those cases led to the same ultimate result as 2a would have, but if 2b can be invoked in "realistic" situations, then I'd guess that if slates are in play, there may be a slight (0.1%-1%) chance that 2b could make an actual difference. This could be an improvement (2b means that a slate gets the slots it deserves, 2a would mean it got fewer) or a clearly worse result (2b means that a slate is in play in more comparisons, leads to "collateral damage", eliminating candidates that otherwise would have progressed.... requires several unlikely circumstances but no outright implausible ones.)

#812 ::: Annie Y ::: (view all by) ::: May 21, 2015, 12:40 PM:

@793 J Thomas

So the question is how many people like that are there, and how important are they?

How are you planning to figure out who will be in this year's BM (and in Kansas where it needs to get ratified if it passes this year? Have a crystal ball or something? :)
If it gets down to a vote, it does not matter if all administrators and regulars are convinced if there are enough people to vote against it. It needs to be something that is easily understood and defensible - not what we think will be the easiest to wiggle in. Trying to figure out what people will think instead of what everyone here thinks is the correct/least wrong way is just the wrong move.

PS: reading through the ties breaker discussion now.

#813 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 01:47 PM:

Felice@794:

It's an interesting idea, though I do think it's a bit hard to intuitively understand. Thanks to your example, though, I think I follow how it works.

Basically, this method gives A and D as finalists, whereas all of the other methods would presumably lead straight to C and D. In this case, I'm not sure that there's any real reason why A should win over C. As I see it, the fundamental difference between A and C is that C has survived as the result of the convergence process in the previous rounds (since it appears uniquely on 4 ballots -- that's why it has more points). A, on the other hand, still shares most of its ballots with other works. Seen from this standpoint, I think A is probably less popular than C because there hasn't yet been as much convergence towards A as there has been towards C. Given that, it seems that the simpler case of eliminating A and B seems to be justified.

Kilo

P.S. This is an example of the kind of analysis I think we should be looking for: What does it -mean- if a work is eliminated in one scheme but not in another?

#814 ::: J Thomas ::: (view all by) ::: May 21, 2015, 01:52 PM:

@803 Pfusand

"The objection is mainly that it isn't fair to the works that have tied for lowest score and same vote -- that it would be fairer to them if only one of them was eliminated."

I do not accept that definition of "fairer."

I don't either. I only understand an argument when I can say it back in my own words and the other guy says that's what he meant. I didn't do that; this isn't how they'd say it.

I'll try again: It's an argument about fairness.

There are multiple possible issues. We should be fair to the works and their authors. We should be fair to the voters. And we should be fair to the voting system.

I don't understand how it's unfair to the works and their authors well enough to repeat it. I get that it's *nicer* to one author to give his work a second chance. And in a way it's sheer bad luck if his work met up with one that tied, when with one vote different it would have won.

At the voter level it's sad when a voter's ballot has no winners, and when he loses two chances at once that's extra sad. If only one of them is eliminated the other will get whatever shared votes they had and so may increase its score and not be eliminated right away, it could even be a finalist. So it's nicer to the voters for that nominee to give it a second chance. I think I understand that argument. But it could be argued that we aren't here to be fair to voters, we're here to find a good set of finalists.

The important argument is about being fair to the process. If you believe that SDV-LPE is the right way, then we should give it a chance to pick the right winners. Any time we throw out an extra nominee, we reduce the number of SDV-LPE steps. If more steps gives SDV-LPE a better chance to zero in on the right finalists, then less steps is bad. When we break a tie instead of dropping both, that allows an extra step without a tie which is a good thing.

But since we can't break ties at random, we need some way to decide which tied nominee is better. It doesn't have to be a lot better, just anything to break the tie. The important thing is not so much to have a tie-breaker that's clearly better, it's to allow more SDV-LPE which will get better results.

I think I got that one right.

#815 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 02:05 PM:

@814

I'm still not convinced that eliminating one work at a time is preferable to eliminating more than one at a time. I'm not quite ready to take that as an axiom.

Can someone propose some arguments why that might be true?

K

#816 ::: Joshua Kronengold ::: (view all by) ::: May 21, 2015, 02:42 PM:

@815: I've stated it before, so I'm not sure what more is needed to convince you.

In short:

As long as only one thing is eliminated at a time, SDV-LPE has no odd edge cases that produce nonsensical results.

However, if multiple things can be eliminated at once, it -does- have such edge cases.

For instance, as long as you only eliminate one thing at a time, it's impossible to construct scenarios where the largest plurality gets no nominees onto the ballot.

If you eliminate multiples, however, it's totally possible for this to happen (if cross-elimination on ties is allowed, at least).

The other thing is that it's more complex to talk about eliminating multiple things at once. If we're not cross-eliminating on "unbreakable" ties, there's no case where it's needed, and it complicates the description of the process to allow for elimination of multiple things vs assuming you're always eliminating a single item.

Re 794: It's an intriguing idea, but I don't like it. It doesn't solve the problem of big lumps of slates (or slate-like) that manage ties well unless -some- of the slate is out of balance (which is likely, but not guarunteed), and it produces a more complicated algorithm overall.

Re 810: Sure, but like every other part of Worldcon, not everything -has- to be spelled out in the constitution. And the administrators can make calls that the BM wouldn't necessarilly approve, but are simpler and saner than what the BM would legislate. Just like we wouldn't ask the BM to run a worldcon, we don't have to micromanage the tabulation as long as we set overall goals and principles, and enough expectation for people to be able to nominate sanely.


#817 ::: J Thomas ::: (view all by) ::: May 21, 2015, 02:52 PM:

@815 Keith "Kilo" Watt

I can't give reasoned arguments, but yesterday when we were choosing 11 over 112 it just seemed intuitively obvious to me then, that it's better to compare only two when you *can* compare only two. That seems like a related question to me. Why make extra comparisons in the same step, when you don't have to? It just seemed intuitively obvious to me that it's better to compare the smallest number you can.

It might seem intuitively obvious to other people that it's better to eliminate the smallest number you can. But I'm pushing my luck on understanding other people's points of view. It may not be one of my strong suits.

#818 ::: Annie Y ::: (view all by) ::: May 21, 2015, 03:32 PM:

[TIES]

I really do not like 1 - there is such an easy way to break ties (the back to points) for some of the ties, that it makes no sense not to use it. It will lead to a lot of works being kicked out early (all those will get kicked out eventually but still).

I am still not sure if I prefer 2a or 2b though. Instinctively I want to go for 2b - if we can break the tie without going to arbitrary ways, I think we should. If they are equal all the way to the end - well - this is a tie we cannot break. Leaving just one of them based on whatever criteria (time of ballots, number of letters in the title, pull from a hat) amounts to telling an author that they did not make it because it was not their lucky day. Which is not a good way to decide things in my book. Leaving one because it was higher up earlier makes some sense - at some point, if they had gone against each other, the second one would have lost anyway.

So 2a vs 2b. I still think I prefer 2b over 2a. If someone is counting, count me for 2b, then 2a and all the way at the bottom 1. Still reserving the right to change my mind around the 2a/2b though.

#819 ::: J Thomas ::: (view all by) ::: May 21, 2015, 03:37 PM:

@816 Joshua Kronengold

For instance, as long as you only eliminate one thing at a time, it's impossible to construct scenarios where the largest plurality gets no nominees onto the ballot.

This is an important criterion that I keep forgetting.

Worst case example:

ABCDE 1000 votes

F 202 votes
G 201 votes
H 201 votes
I 201 votes
J 201 votes

ABCDE have 200 points, they are all compared on the first round.

If they are all eliminated on the first round that eliminates almost half of the ballots and the winners are FGHIJ

If the first round eliminated only E then the winners would be ABCDF.

Results almost this counterintuitive are possible without double elimination, but without that, if A has the most votes then A cannot be defeated. Only BCDE can be defeated and replaced by other works that have significantly less votes.

The reason it works that way is that we have two goals, not just one. We encourage winners to have lots of votes, but we also encourage them to have ballots that don't include other winners.

We still get weird edge cases without double elimination, but double elimination can give us more of them. That's a valid concern. I don't know how to estimate how often it will happen, but it's looks very very likely that it's more than we'd get by eliminating one thing at a time.

I prefer 1.0 or 2.a because they're simpler. But to be sure they're OK I have to believe that they won't too often cause results fans would hate, and I don't know enough to be sure of that.

Like, take something that happens only 1% of runs. The chance it won't happen on any of 17 runs in one year is about 84%, and the chance it won't happen for 10 years is about 18%. We can't afford controversial things to happen that often.

Something that happens in only 0.1% of runs has only about a 15% chance that it *will* happen in 10 years. (And a 29% chance for 20 years.) That's closer to acceptable.

I see no reason to expect that this sort of thing would come as often as one in a thousand, but I'm not ready to swear it won't.

There's the question too just how weird it has to look to be bad. If something that has 400 votes loses to finalists that have 380, 360, and 350, that isn't horrible when the one with 400 votes had a lot more shared votes. When it has 1000 votes it looks worse. That can in theory happen though, ties or not.

ABCD 1 vote
ABC 1 vote
AB 1 vote
A 1 vote
ABCDE 1000 votes

BCDE can all lose when their scores are around 200, leaving four lesser works to be finalists.

There's nothing wrong with that, but it can look bad.

#820 ::: Steve Halter ::: (view all by) ::: May 21, 2015, 03:41 PM:

J Thomas@817: :-)

#821 ::: felice ::: (view all by) ::: May 21, 2015, 03:48 PM:

Keith "Kilo" Watt @804: "To answer Felice's question in 792, as a writer myself, I would strongly prefer to find out I didn't get a Hugo because not enough people liked my work than because somebody just didn't submit their nomination ballot early enough. The former seems completely fair, the latter doesn't."

What if the tallying was paused at the point the tie occurred, and the admins contacted you and said "you have enough nominations to be a Hugo finalist, but so does Writer X, but you can't both be finalists because you have too many of the same nominators in common; would you prefer we eliminate both, or toss a coin to pick the winner?" Would you as a writer be happy to give up a 50% chance of winning? Would you be happy to deny Writer X a 50% chance of winning? Would you be happy to deny some of your fans a certain Hugo nomination for one of the works they like? (Whether you or Writer X wins, more than half of the nominators will be happy, because some of them nominated both of you).


Keith "Kilo" Watt @815: "I'm still not convinced that eliminating one work at a time is preferable to eliminating more than one at a time. I'm not quite ready to take that as an axiom."

If a tie matters, then eliminating both works is eliminating two works from the same ballot(s) in one round. That's not how SDV-LPE is supposed to work; nominators should feel confident they can nominate as many works as they like, because whenever one gets eliminated their other nominations get more points, and in the end their last nomination will have just as many points as if they had bullet voted. If that can't be relied on, it's either an encouragement to bullet vote (bad), or people will think it's unfair when their last two nominations are simultaneously eliminated (bad).

#822 ::: Lydy Nickerson ::: (view all by) ::: May 21, 2015, 04:01 PM:

[SIGNING] I am a supporting member at Sasquan, but completely unable to attend due to time and money. Legal name is Lydia Nickerson, which is how Sasquan knows me.

Am home sick, and trying to catch up. I don't have a strong preference about withdrawals. Either the RME with the provision that no one who has been told they have a nomination is then un-nominated, or finalizing the order once, and then moving people up per that order works for me. I do think that because of the way the distributed vote works, it should not be left to the discretion of the administrators. Because there are several reasonable methods, but those methods yield different results, it would be best if that piece of it were nailed down before the ballots are counted. Choices afterwards are vulnerable to an accusation that the particular method was chosen to favor a particular work. Not fun. And the temptation to not make a decision until you have to will interact badly in this case. Hugo administrators are probably as liable to procrastination as anybody else.

I am genuinely confused by the tie-breaking conversation. I do believe that you will find something fair. Good luck.

I want to call out Keith for his incredibly clear prose. It's been so very useful in my attempting to follow the conversation. I began this conversation in despair and confusion. Some of the confusion persists, but oh, am I hopeful. I hope your knee gets better immediately.

Joshua, E Pluribus Hugo is lovely. I hope that we use it.

Jameson, I'm going to throw a little money at your GoFundMe site. Because damn, boy, you've done a ton of wonderful work and it would be really helpful if there's a wonk at the business meeting to, you know, wonk.

As always, Kevin Standlee, I admire your patience and perseverance. I really hope the business meetings are not hellish.

Gods, I love my community. When attacked we 1) do math, and 2) write snarky poetry. Yay us.

#823 ::: felice ::: (view all by) ::: May 21, 2015, 04:13 PM:

[TIES]

David Harmon @796: "That said, I think I mostly understand where Keith is coming from, and it's what you might call a "philosophy-of-governance" issue: As he's previously put it, the algorithm needs to be deterministic, such that rerunning it from the same ballots will reliably give identical results each time. I gather that first-nomination timestamp, while otherwise appealing, is "no good" here because it depends on what order the officials pick up the ballots.

Virtually all (all but three, last time) nominations are made online, and the database always has a set order in which nominations were added; that won't change on a recount. And I see no reason for a recount to need to re-input the handful of paper ballots, and even if it did it's incredibly unlikely to change anything (any work popular enough to win will be nominated by somebody online before a paper ballot can make its way through the postal system). First-nomination timestamp is deterministic/repeatable, and nobody is advocating a non-deterministic tiebreaker.


Duncan J Macdonald @799: I'd like to record my preference for Tie Breaker 2b. "If the works are tied for both nominations and points, look back to previous rounds. If there was ever a round in which the works were -not- tied, eliminate the one with fewer points." That is about as non-arbitrary as we can get -- especially if the Look Back goes all the way to the initial round and they remain tied."

Any comments on my thoughts on how arbitrary lookback is? (#686 and #723)


J Thomas @817: "I can't give reasoned arguments, but yesterday when we were choosing 11 over 112 it just seemed intuitively obvious to me then, that it's better to compare only two when you *can* compare only two."

If you can pick which to eliminate out of the two, yes; but if the 1s are perfectly tied, adding the 2(s) might allow a single elimination without needing a tiebreaker.


Annie Y @818: "I really do not like 1 - there is such an easy way to break ties (the back to points) for some of the ties, that it makes no sense not to use it."

Agreed. Back to points is simple and fair; if two works are tied for nominations, keep the one that shares fewer ballots with more popular works.

#824 ::: J Thomas ::: (view all by) ::: May 21, 2015, 04:51 PM:

BCDE can all lose when their scores are around 200, leaving four lesser works to be finalists.

Or E can easily lose at 200, D at 250+, C at 334+, and B at 500+. If it works that way all the replacements must have votes over 500 which doesn't look so bad, right?

I can build an example where BCDE lose to items with about 1/5 their votes without ties, but it takes more than 80 works and I'd have to make up new names for them.

#825 ::: felice ::: (view all by) ::: May 21, 2015, 04:51 PM:

[GHOSTS]

Keith "Kilo" Watt @813: "Felice@794: It's an interesting idea, though I do think it's a bit hard to intuitively understand. Thanks to your example, though, I think I follow how it works.

A bit tricky to explain, yes, though I think it has a certain conceptual elegance.

Basically, this method gives A and D as finalists, whereas all of the other methods would presumably lead straight to C and D. In this case, I'm not sure that there's any real reason why A should win over C. As I see it, the fundamental difference between A and C is that C has survived as the result of the convergence process in the previous rounds (since it appears uniquely on 4 ballots -- that's why it has more points). A, on the other hand, still shares most of its ballots with other works. Seen from this standpoint, I think A is probably less popular than C because there hasn't yet been as much convergence towards A as there has been towards C. Given that, it seems that the simpler case of eliminating A and B seems to be justified."

A has more nominations than C, and only has fewer points because of overlap with B, which also has more nominations than C. If you remove B, then A appears on more unique ballots than C as well as having more nominations; it's obviously the more popular work. And if you remove A, then B appears on the same number of unique ballots as C, but has more nominations; it's also obviously more popular than C. Why should C win when either of the alternatives are better, just because they're equally better? Once C is gone, between A and B, A appears on more unique ballots, and overlaps less with the other winner (D), so is the better choice. Keeping both A and B as ghosts gives them a chance to break their tie and one to become a clear winner over the other; you could call this lookforward; a bottom-up system's logical counterpart to the lookback that makes sense with top-down systems like STV.

Joshua Kronengold @816: "Re 794: It's an intriguing idea, but I don't like it. It doesn't solve the problem of big lumps of slates (or slate-like) that manage ties well unless -some- of the slate is out of balance (which is likely, but not guarunteed)"

Yes, it will push a perfect slate straight to the top and let it sweep the ballot, which is bad.
But the alternative is to eliminate a perfect slate entirely, even though more people nominated each work on the slate than nominated any non-slate work, which is also bad. Particularly if there's a unplanned perfect tie between two non-slate works, which can happen.
Or pick an arbitrary subset of the slate in proportion to its share of the total set of ballots, which is also bad.

Basically, there are three bad options, and we have to pick one. Personally I'd go with arbitrary; it does the least harm.

For the Ghosts option, a possible workaround would be that ghost works only count as filling half a slot if a set of them makes it to the top five and none of them can be eliminated. So a perfect slate would take 2.5 slots, and three non-slate works would also be finalists - eliminating the third would take the ballot below 5 (to 4.5), so we keep it and have eight works on the ballot. A pair of ghost works would mean six works on the ballot, and it really is comparable to the 5th place ties we get now - the ghost pair have more nominations than the last-placed finalist but fewer points, so in a sense could be considered equal to it. Actually, maybe it's the same thing as the current ties for 5th! If two works are perfect tied for 5th place, this system makes them ghosts, so they both win (taking up half a slot each) - this eliminates the need for special handling of ties involving 5th, which compensates a bit for the added complexity elsewhere.

And remember, if one of a ghost pair gets eliminated, the other is no longer a ghost, so getting ghost finalists would be pretty rare.

#826 ::: Annie Y ::: (view all by) ::: May 21, 2015, 04:52 PM:

@819 ::: J Thomas

Corner cases. The first of these can happen because of a very special case (a tie because of a perfect slate). In the real world, at least one person will mess up their slate (because people make mistake and in a 1000? or because they replaced one work or whatever) or because someone else will nominate at least one of the slate works outside of the slate. I somehow do not seem to be to worried if in this case things look wonky.

Now the second case is a bit different - it is a perfect slate + people nominating partials but not touching anything outside the slate. I still think that someone will have ABCDE somewhere with something else to also break the example.

Will there be a case where things look awfully wrong at the end? Probably. In the correct circumstances. They can happen. Or not. The way usual voting goes, I highly doubt that a slate can get almost 5 times the next in the list votes (which is what you need for these to work - and adding 20 more will tilt it the other way)) or that the slates will stay cleanly split. And with a big slate that is supposed to be able to win under the old system, it will high enough in the comparisons as well - that it may as well end up being the tie for #5. And if they do? If noone else nominated something that is on the slate, there are probably good enough reasons. Just saying. :)

Now don't get me wrong - you need corner cases for perfect system. But we are not trying to do that - we are trying to have a decent system that is not too complicated, at least mostly logical and that helps with slates without effectively killing them.

felice @ 821

It is at a point where you really want to start calling people, you can as well leave both on the ballot. But when it happens low down, maybe none of them will make it all the way up. So you call 10 people, they all say yes, you act based on that and none of them make the final ballot? This year's outrage at the slate will be like whispers compared to the noise that this will cause.

If a tie matters, then eliminating both works is eliminating two works from the same ballot(s) in one round.

Not necessarily. They do not need to be on the same ballots to perfectly tie. Plus - even if that is the case, there is a huge chance that these two would have gone down under the current system as well. Are you really trying to make this system perfect for everyone? That won't happen. The old system is the most fair one until an idiot decided to go for slates. Now we are trying to find something that is fair enough - so yes, people will loose all their nominations. Happens.

Let me show you another example:

First round:
I am a weird reader so all my 5 nominees are only on my ballot. So they tie at 0.2
The other weird readers also have some 0.2s and then some 0.25s.

Nominations are 1 for each, back to points kicks all my 5 nominees (and all other 0.2). Ooops. Well - they would have gone under any system. But kicking all is not a reason to say "let's keep them"

One option may be to remove the lowest few on nominations (the 5% rule or a 3% rule) - this will eliminate my poor ballot as well. Any system will. Even if 2 of them survive, if they are not popular, I will loose them. Trying to keep as many ballots in place as possible even to the point of not resolving ties just makes no sense to me.

#827 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 05:01 PM:

JT@819:
ABCDE have 200 points, they are all compared on the first round.

Except that won't happen because of our ground rules. You're expecting perfect slate discipline, which isn't likely to happen (and slate coordinators can guarantee won't happen, if they are aware of the risk). Supposing that A is the most popular slate work, your example will end up with AFGHIJ (given the four-way tie for third). This is precisely the behavior we want -- slates should not dominate the ballot. But in this case the tie breaker rule was never applied, so it doesn't help clarify ties much, I realize

I do caution that I see people define a "lesser" work to be one that gets fewer total nominations. If that's how we want to define "lesser" or "greater", then that is no different from the first past the post system that we have now. This was my problem earlier up-thread: I disagree with that value judgment because slates have made it impossible to use as a marker of "merit" for a work. Many of the results that have been called "odd" depend upon accepting this definition of merit. We can't do that anymore, thanks to the slates.

Kilo

#828 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 05:08 PM:

Felice@821:
Would you as a writer be happy to give up a 50% chance of winning?

Absolutely I would give up the chance to have a 50% chance of winning. A Hugo won by luck is meaningless to me. This shouldn't be too surprising, given the authors in the current situation that have indicated that if they only won a Hugo by being on a slate, then that Hugo wasn't worth anything. It doesn't indicate that my work was the best of the year. I think many writers who value what a Hugo means would feel the same way. What feels good is to have your work be recognized as great fiction, not to get a plastic rocket.

Kilo

#829 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 05:28 PM:

Felice @821,Annie@826:

Annie has brought up the same argument I was about to make: There will be -lots- of nominators who have all the works on their ballot eliminated. That's true under the current system and nobody has a problem with it. So this isn't a terribly compelling reason not to double eliminate. In most cases, SDV-LPE will eliminate first one, then the other on the ballot in question, but the end result is that both are still gone.

I do understand that you might be able to create a case where a work goes on to be a finalist (which is the only case that matters) if one is eliminated, but doesn't (obviously) if both are eliminated. I haven't been able to think of situation for this, though, because it seems to me that the only way for this to happen is if you are very near the five-work cutoff. In this case, I think you're much more likely to just tie for fifth and have both works kept. Of course, we can always come up with low-probability cases, so that's why one of our ground rules is to always scale up these cases to reasonable sizes when we evaluate their likelihood.

So, in short, I guess the fundamental difference of opinion is that I don't see preserving one work on each ballot submitted to be a goal of the system. There's just no real way to guarantee that if we want to honor fandom's diverse tastes.

Kilo

#830 ::: J Thomas ::: (view all by) ::: May 21, 2015, 05:43 PM:

@818 Annie Y

[TIES]

I really do not like 1 - there is such an easy way to break ties (the back to points) for some of the ties, that it makes no sense not to use it. It will lead to a lot of works being kicked out early (all those will get kicked out eventually but still).

If they would all be lost eventually, what's the harm? But unfortunately I'm not sure they would, so you have more of a point there than I'd prefer.

Leaving one because it was higher up earlier makes some sense - at some point, if they had gone against each other, the second one would have lost anyway.

Might-have-beens. If one of them had a 0.2 higher score on round 12, it would have won if they'd had the two lowest scores on round 12. And if the other had a 0.25 higher score on round 11, that one would have won if they had the two lowest scores on round 11. And if the first one had a 0.33 higher score on round 10.... Looking at the last time the scores were different is an arbitrary method that doesn't particularly mean anything. But it is a potential way to say they're different, so it is a workable tie-breaker.

@821 Felice

What if the tallying was paused at the point the tie occurred, and the admins contacted you and said "you have enough nominations to be a Hugo finalist, but so does Writer X, but you can't both be finalists because you have too many of the same nominators in common; would you prefer we eliminate both, or toss a coin to pick the winner?"

If they are in 5th place, they're both finalists.

But in general this is why we don't ask authors how to run the Hugo voting. When you get two people together and you tell them "You two can have a special advantage over everybody else if you can agree among yourselves" that isn't an honest election. That's politics.

If a tie matters, then eliminating both works is eliminating two works from the same ballot(s) in one round. That's not how SDV-LPE is supposed to work; nominators should feel confident they can nominate as many works as they like, because whenever one gets eliminated their other nominations get more points, and in the end their last nomination will have just as many points as if they had bullet voted.

We can't make it completely the way you want. There's always a possibility that nominating Beta will make Alpha lose, and there's also a possibility that Beta will then lose too. It can happen. Voting for Beta too can make Alpha lose in any voting system that allows multiple votes, unless everything that gets enough votes wins independent of any other nominee.

If we allow double elimination there's a possibility that your ballot will lose two votes in one round. There's a possibility it will lose its last two votes, though if you have 5 votes and it happens once then it's the last two only 1/4 of the time. If we don't allow double elimination there's a chance that your ballot will lose two votes one after the other, and they could be your last two votes. The chance that all your votes will lose is the same either way.


#831 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 05:45 PM:

All:

I'm going to try to go back through the threads to find the edge cases that were posted as being a concern for each of the four methods under consideration. However, I seem to recall that a great number of them depended upon slate discipline to show their effects. It would be really helpful if the original authors of those cases would re-propose them, making whatever modification is needed to eliminate the need for slate discipline.

It also seems to me that some of them were considered "odd" because a "stronger" work was eliminated. But as I mentioned a little while ago, this definition of "stronger" relies on number of nominations as a measure of strength. Thanks to the slates, we don't have that definition available to us any more. Any system that relies on this definition by necessity gives all available slots to the slate -- they've shown that they will always win in pure number of nominations. Obviously moving away from this definition is the whole reason we are having thus conversation.

Thanks a bunch,
Kilo

#832 ::: felice ::: (view all by) ::: May 21, 2015, 05:49 PM:

Keith "Kilo" Watt @828: "Absolutely I would give up the chance to have a 50% chance of winning."

And would you be happy to take that 50% chance of winning away from your competitor, irrespective of their feelings on the matter? I'm not talking about slates here, just another author whose work appeals to some of the same fans as yours, and you coincidentally got the same number of nominations. Or what if the other writer decided to decline nomination and concede the victory to you? Without them in the running, you're a clear winner, with more nominations than at least one of the other finalists; there are no grounds for saying your work wasn't recognised as great fiction (under the current system, you'd both be finalists). And this is just winning a finalist spot, not the rocket itself; you won't get that without the support of a majority of voters.

Annie Y @826: "It is at a point where you really want to start calling people, you can as well leave both on the ballot."

I'm talking hypothetically about what the nominees would want, not suggesting people actually get called. If it's a tie for 6th, we don't leave them both on the ballot, but both might have more nominations than the current 5th (but lower points because they share points), so eliminating them both is unreasonable.

"Not necessarily. They do not need to be on the same ballots to perfectly tie."

No, but they have to be on the same ballots for the tie to matter; otherwise, neither would benefit from the other being eliminated. If they were tied for 6th and don't share points, then they'll have fewer nominations than 5th, and whichever isn't eliminated in the tie will be eliminated immediately afterwards. Oh, I suppose that's not necessarily the case under 1,1 selection; if say they both share points with the 1st placed work, they could have more nominations than the 5th, and either tie survivor would beat the 5th in the next round. That wouldn't be an issue with 1,1,2 selection; then they'd eliminate the old 5th and tie for 5th themselves.

"Plus - even if that is the case, there is a huge chance that these two would have gone down under the current system as well."

In which case an arbitrary tiebreak doesn't do any harm, because whichever survives will probably be eliminated in the next round.


Keith "Kilo" Watt @829: "I do understand that you might be able to create a case where a work goes on to be a finalist (which is the only case that matters) if one is eliminated, but doesn't (obviously) if both are eliminated. I haven't been able to think of situation for this, though, because it seems to me that the only way for this to happen is if you are very near the five-work cutoff. In this case, I think you're much more likely to just tie for fifth and have both works kept."

A tie for 6th isn't significantly less likely to be a problem than a tie for 5th. A tie for 7th, or 8th, or 9th could also be a problem (the further down you go, the less likely a problem is, but it's theoretically possible for a tie that matters to have less than half as many points as the 5th placed work). And ties aren't that unlikely; 26 nominations can be enough to make you a finalist in some categories.

#833 ::: Annie Y ::: (view all by) ::: May 21, 2015, 05:53 PM:

felice,

Why does a tie matter only if it is on the same ballot but does not matter if it is with works on different ballots?

#834 ::: David Harmon ::: (view all by) ::: May 21, 2015, 05:55 PM:

felice #821, #823: Given online voting... I assume the the nomination timestamp would need to be calculated at counting time, from ballot timestamps. Even so, it gives slight incentive for a slate (or other players) to get in a few early ballots with selections from their state.... Uh, oh. What if the the original nominations for the tied works are all on the same early ballot? That doesn't have to be a slate ballot, either, sometimes there are obvious favorites (especially, in say, Dramatic Presentations).

I'm starting to think that ballot order really should be brought back in, even if it has to be specifically noted as relevant only in cases of voting ties.

Let me try crossing with another track, that of withdrawals. Suppose we consider Lydy Nickerson #822's finalizing the order once, and then moving people up per that order . If we compute the nominees out to, say, 10 places down, do we get any further information that could help with the tiebreaking?

#835 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 05:57 PM:

JT@830 and others:

My problem with lookback/2b (such as it is -- I don't actually strongly object to it) is that, as mentioned in the description, its rationale is weak. The only reason to use lookback is to claim that in earlier rounds fandom was converging towards one work faster than another. But since they are tied at this round, that earlier convergence effect wasn't strong. Something happened as works were eliminated that gave these two works equal convergence at the time they were selected for elimination. It's not clear to me that their earlier difference has any real meaning. It might, though, so I'm undecided, but it looks to me like it might be an example of complexity for its own sake.

That's why I'd like to see a comparison where a work that was kept due to lookback became a finalist when it (obviously) wouldn't have if it was eliminated without lookback. I think knowing when or if this can occur would allow us to choose based on something other than instinct.

K

#836 ::: J Thomas ::: (view all by) ::: May 21, 2015, 06:02 PM:

@827 Keith "Kilo" Watt

I do caution that I see people define a "lesser" work to be one that gets fewer total nominations. If that's how we want to define "lesser" or "greater", then that is no different from the first past the post system that we have now. .... Many of the results that have been called "odd" depend upon accepting this definition of merit.

Agreed. This is a potential problem, that a whole lot of people may not understand and agree that this is a good voting system when it gets results they don't see are good.

We have to change our ideas about what results we're looking for. And so I don't consider it a giant virtue of the system that the one nomination with the most votes will always be a finalist (without discarding both ties). It isn't true for #2 through #5. There's no particular moral reason it ought to be true, it turned out that way as an accidental result when the rules were designed with other goals in mind.

So I don't consider it a bad thing that with double elimination the one with the top votes can possibly be eliminated in some corner cases. Except that there are people who do think it's a bad thing, and if there are too many of them we must cater to their opinion.

"ABCDE have 200 points, they are all compared on the first round."

Except that won't happen because of our ground rules.

You can say we aren't allowed to consider the possibility. I can come up with an example that creates a 5-way tie without them being a slate, I was just being lazy.

But all that shows is that it's possible, that says nothing about how likely it is. I disagree with Joshua's idea that weird-looking corner cases are not possible without eliminating both sides of a tie. They can still happen, but I think they will be somewhat rarer because in the ones that *would* happen with double elimination, some of them *won't* happen without. And I have no evidence that there would be just as many new ones from single-elimination. That would require that some things that get a plausible result with double-elimination would turn implausible with single elimination. There might be less of those.

#837 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 06:04 PM:

Felice@832:

And would you be happy to take that 50% chance of winning away from your competitor, irrespective of their feelings on the matter?

Yes, absolutely. Because I have to be true to and consistent with my feelings on what the Hugo is. That's not quite as cold as it sounds, since I'd be willing to bet that most writers agree with me. And haven't most fans this year expressed a similar view as well? The slate works are not being universally acclaimed as the best fiction. Many people are willing to say that they don't deserve a Hugo. We can debate (in another thread!) whether that attitude is justifiable, but there's no denying that a great many fans share it. This is why I don't think an arbitrary method has a chance of passing.

Kilo

#838 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 06:18 PM:

Felice@832:

but both might have more nominations than the current 5th (but lower points because they share points), so eliminating them both is unreasonable.

And here is the core of our disagreement. I strongly disagree that just because a work has more nominations it is unreasonable to eliminate it. The whole goal of SDV-LPE is to add an additional consideration, the number of points received. This is in some sense a measure of convergence towards the most popular works (by eliminating the least popular). We -cannot- use total number of nominations as our only measure of deserving a Hugo, because doing so is the precise system we have today -- and if that worked, we wouldn't be needing to come up with a new system.

So, if a work has more points, then fandom is converging as whole towards that work. We may eliminate a work with more nominations. This is okay, since a great many of those nominations may be from a slate (remember, our stated goal is to prevent -any- slate from being able to dominate the final ballot).

Of course, number of nominations is important, because that is how we decide which of the works showing least convergence is the least popular. From the other side, absent a slate, a work with more nominations will also have more points, so that works too. That, in a nutshell, is why the system works.

Kilo

#839 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 06:31 PM:

A hasty P.S. to my previous message, and apologies for the multiple posts -- I try to reply between pain meds, since it seems to knock me out.

Something else to keep in mind:

In SDV-LPE, the total number of nominations never changes. The point total for each surviving work continually increases, however. For this reason, while number of nominations is important and should not be ignored, number of points is probably a better indicator of the works fandom is converging to. If we valued number of nominations only, there's no reason to use SDV-LPE at all -- because number of nominations never changes. The system we use today does this explicitly. But unfortunately, we now know that system doesn't work.

Kilo

#840 ::: J Thomas ::: (view all by) ::: May 21, 2015, 06:56 PM:

@838 Keith "Kilo" Watt

I strongly disagree that just because a work has more nominations it is unreasonable to eliminate it. The whole goal of SDV-LPE is to add an additional consideration, the number of points received.

Agreed! But people still tend to think that the ones with the most votes ought to win. That's something that can get in our way.

This is in some sense a measure of convergence towards the most popular works (by eliminating the least popular).

We decide which are most popular by which don't get eliminated. This is a quasi-chaotic process, a difference of a few votes for something that doesn't look like it has any connection to the works you're interested in, can have a big effect on them. That doesn't mean it's a bad system -- you can't have a voting system that does everything we want, and if we didn't have this to complain about it would be something else. It's something we have to to work with.

So, if a work has more points, then fandom is converging as whole towards that work.

No. The ballots are converging as a whole toward works that currently have more points. A work that right now has the highest score could perhaps in a few rounds have the lowest and be eliminated. There is no convergence for individual works, but the system is converging toward some collection of works that has a good combination of number of nominations and diversity of sources of nominations. Reasonably high number of votes, and also low number of shared votes. We can't say ahead of time which works will be finalists except that they are among the ones that have not yet been eliminated. (And the single one with the most votes will be one of them if we never eliminate both tied works.)

We may eliminate a work with more nominations. This is okay,

Yes, we *want* to do that some when it's slates. But what if it isn't slates, what if it's only works that accidentally got votes on the same ballots? Surely we never want to eliminate a work with more nominations if it isn't from a slate! But this voting system can do that sometimes. It just follows the rules, without deciding whether something is a slate or not. It will not perfectly do what we want, but maybe it will come closer if we make it more complicated.

The more complicated we make it, the harder it is to predict what will happen, and so the easier it is to suppose that it will do what we want.

#841 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 07:33 PM:

JT@840:

Surely we never want to eliminate a work with more nominations if it isn't from a slate!

Sure we do. As I said, points are a better measure than nominations. Even for non-slate works, if people are converging towards a particular set of works -- as measured by the fact that their points are growing the most -- then these are obviously (to me, anyway) the works that should be maintained.

As I said to felice, we fundamentally disagree on whether nominations are a better measure than points. I contend that the current system shows the flaws with valuing nominations alone. The whole problem with slates is that they block the rest of fandom from participating in this convergence to the final ballot. Any system based solely on nominations is going to have that problem. Because SDV-LPE cleverly introduces a new, more fair method of determing that convergence, slates are not able to do that.

This is why I have yet to see a "problem" case in which the problem wasn't that a work with more nominations got eliminated. I'm not saying those cases don't exist, mind you, but I can't find them -- I'm hoping the others here can.

Kilo

#842 ::: felice ::: (view all by) ::: May 21, 2015, 07:36 PM:

Annie Y @833: "Why does a tie matter only if it is on the same ballot but does not matter if it is with works on different ballots?"

Because if it's on the same ballot, then eliminating one the tied works will give the other more points, and push it up the rankings; it might not be one of the last two any more. But if they don't have any ballots in common, whichever survives elimination won't gain any points and will still be in last place, and will probably be eliminated in the next round (definitely eliminated next round with 1,1,2 selection).


David Harmon @834: "felice #821, #823: Given online voting... I assume the the nomination timestamp would need to be calculated at counting time, from ballot timestamps. Even so, it gives slight incentive for a slate (or other players) to get in a few early ballots with selections from their state.... Uh, oh. What if the the original nominations for the tied works are all on the same early ballot? That doesn't have to be a slate ballot, either, sometimes there are obvious favorites (especially, in say, Dramatic Presentations)."

A meaningful tie between a slate work and a non-slate work isn't that likely, especially at the early stages when the slate still has all its works and therefore low points (any non-slate work down that low either doesn't have a chance, or shares a lot of nominations with other finalists; either way it deserves to be eliminated). So I don't think early slate nominations are that big a problem. But as an alternative, what about using the median timestamp rather than the first timestamp? That requires more than half the slate's supporters get in early in order to guarantee a tiebreak win.

And yes, if they're on the same first ballot, ballot order becomes the tiebreaker.

Or just pregenerated cointosses works for me. Or compare the md5 hash of the work titles (I don't think even puppy authors would go to the extend of renaming their works to get better md5 hash values).

Keith "Kilo" Watt @835: "The only reason to use lookback is to claim that in earlier rounds fandom was converging towards one work faster than another. But since they are tied at this round, that earlier convergence effect wasn't strong. Something happened as works were eliminated that gave these two works equal convergence at the time they were selected for elimination. It's not clear to me that their earlier difference has any real meaning. It might, though, so I'm undecided, but it looks to me like it might be an example of complexity for its own sake.

I'm confident it doesn't have real meaning. They weren't converging faster, it's just that supporters of one of the tied works was more likely to also support a particular less popular work which has already been eliminated. Penalising a work for sharing support with another more popular work makes sense, but penalising it for sharing support with a less popular already-eliminated work does not.


Keith "Kilo" Watt @837: "Yes, absolutely. Because I have to be true to and consistent with my feelings on what the Hugo is. That's not quite as cold as it sounds, since I'd be willing to bet that most writers agree with me. And haven't most fans this year expressed a similar view as well? The slate works are not being universally acclaimed as the best fiction."

Because we believe the slate works were nominated by lots of people who didn't actually read them and don't genuinely consider them to be the best work of the year. A coincidental tie is not the same thing at all.

How about "Two of your short stories are potential finalists; under the old system, they'd both be finalists, but because a lot of the same people nominated both of them, they can't both be finalists under SDV-LPE. Would you rather both be eliminated, or we picked one randomly?" (in a real phone call, asking which the author would prefer to win would make sense, but I'm trying to establish a general principle here.)


Keith "Kilo" Watt @838: "And here is the core of our disagreement. I strongly disagree that just because a work has more nominations it is unreasonable to eliminate it. The whole goal of SDV-LPE is to add an additional consideration, the number of points received. This is in some sense a measure of convergence towards the most popular works (by eliminating the least popular)."

But simultaneous elimination is cutting off the process before it finishes converging. There's nothing wrong with eliminating a work with more nominations than some finalists, but eliminating two at once is a problem. Even under SDV-LPE, we do still use number of nominations as a measure of strength; the difference is that we don't let one person give all their strength to more than one work at a time. If the last two works on someone's ballot are eliminated at the same time, they never get to give more than half their strength to any of the works they nominated - that is a problem. It means their ballot is only worth half as much (less in the case of multi-way ties) as other people's ballots. That's unfair. And while slates are the most likely to be affected, it's not just slates; anyone could randomly have the power of their ballot halved relative to other people's ballots.

#843 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 08:00 PM:

Felice@842:

Referencing your last paragraph (so I won't quote the whole thing)...

If you could show me an example where eliminating both members of tie kept one of the eliminated works from becoming a finalist (with or without slates, so long as the example doesn't require the unlikely case of perfect discipline), then I would wholeheartedly agree that it matters. That should be the only time it matters. In fact, I hope you can find that case, as it would shed some definitive light on the choices. It doesnt matter if both works get removed at once if both works would have been eliminated anyway. I just haven't been able to come up with a situation where that can matter, but I do recognize that you guys are better at seeing those situations than me.

I won't accept that just because you get a different set of finalists (when neither set involves the tied works under consideration), then that's a problem. At some point we have to draw the line against complexity, and I don't think that's a bad pace to draw it.

Kilo

#844 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 08:10 PM:

"Two of your short stories are potential finalists; under the old system, they'd both be finalists, but because a lot of the same people nominated both of them, they can't both be finalists under SDV-LPE. Would you rather both be eliminated, or we picked one randomly?"

If it were due to a slate, then both should be eliminated. That actually happened this year. But if there is a broader appeal than the bloc voters ("We love Kilo!!!"), then I won't have a problem -- one will be slightly more popular than the other, or maybe they will be so popular that both make it. Ideally, the Hugo should (in my opinion) be awarded for broad appeal, not just because of a small loyal cadre of fans. The odds of it just happening to make a difference without bloc voting seem to me to be so small as to not be something to worry about.

Kilo

#845 ::: J Thomas ::: (view all by) ::: May 21, 2015, 08:24 PM:

@842 felice

If the last two works on someone's ballot are eliminated at the same time, they never get to give more than half their strength to any of the works they nominated - that is a problem. It means their ballot is only worth half as much (less in the case of multi-way ties) as other people's ballots. That's unfair.

I see what you're saying! In every single other case, when your ballot's next-to-last work is eliminated the ballot turns into one whole point for the last work. That work might get eliminated right away anyway, but at least you got the chance to give it one whole point for at least one round.

But this way, your tied last works share that full point up to the end, because they're tied. You never get to give a whole point to either of them.

I like something PJ Evans said about this. The nominations aren't about being fair to nominators. Nominators get to suggest good works to be finalists, and the selection system is supposed to give us five works that represent what the nominators thought were most worthy. The system has no responsibility to any individual nominator, or his ballot, or his votes. The responsibility is to represent the whole of the nominations well.

This is a boundary case. If G and H are tied, just one less vote for G would mean G loses, and one more would mean G wins.

With 2.0, if there is one ballot that has G on it and 4 others, then one fewer other vote on that ballot would make G's score go up by 0.05 point and G would win. If there's one ballot that has G and 3 others, then one more vote for any survivor would make G's score go down by .05 point and G would lose.

If we can put aside our duty to be fair to individual votes, or to individual nominations, then I say the remaining concern is being fair to the process. If we eliminate both sides of a tie with 2.a, that gets the process to converge to the answer it ought to have. Once we make those the rules, the answer they produce is the right answer.

You might argue that there is some righter answer. OK, show me how to measure how good an answer is, and I can work with that. Until I have a way to measure how good an answer is, I can't tell which variation on SDV-LPE will give better results. It might be hard after I have a rule to judge voting systems by, but I can't even get started without one.

#846 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 08:37 PM:

JT@845:

The system has no responsibility to any individual nominator, or his ballot, or his votes. The responsibility is to represent the whole of the nominations well.

This I would accept as axiom.

This process is supposed to be about the whole of fandom converging to a list of finalists. It's not supposed to be about the wishes of one group (this year, a slate). Carrying that to its logical conclusion, it's not about the wishes of any one individual either, except in as much as that individual is a part of the whole.


"Out of the many, a Hugo."


Kilo

#847 ::: felice ::: (view all by) ::: May 21, 2015, 08:39 PM:

Keith "Kilo" Watt @843: "If you could show me an example where eliminating both members of tie kept one of the eliminated works from becoming a finalist (with or without slates, so long as the example doesn't require the unlikely case of perfect discipline), then I would wholeheartedly agree that it matters."

2014 Novel: let's say 22 people who liked Neptune's Brood also decided to add Parasite to their ballot, and they already overlapped by 38 ballots. There are no other overlaps amongst the potential finalists. So Neptune's Brood and Parasite are tied on 90 points, 120 nominations. This puts them below Shining Girls and A Stranger in Olondria (96 and 92 points respectively), so Neptune's Brood and Parasite come up for elimination and wipe each other out. If they hadn't been tied, one would make the final ballot. And they're not a slate; only one third of the people who nominated at least one nominated both.

Or Pro Artist: Dan dos Santos and Fiona Staples really are tied for 5th on 49 nominations, no need to adjust the data. And the next in line is on 48 nominations. Assuming no other overlap, it would take just three shared nominations between Dan and Fiona to put them below the next work on points and get them simultaneously eliminated.


Keith "Kilo" Watt @844: "But if there is a broader appeal than the bloc voters ("We love Kilo!!!"), then I won't have a problem -- one will be slightly more popular than the other, or maybe they will be so popular that both make it."

One will probably be slightly more popular than the other, but not necessarily - see Seanan McGuire's short stories tied for 5th on 28 nominations in 2013. And for them to be popular enough to both make it is unlikely; you'd need up to twice as many nominations to overcome the split vote, depending on how much overlap there is.

#848 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 08:56 PM:

Felice@847:

I agree they are tied for nominations, but points are what matters (and obviously we don't have that information), but this looks promising as a way to construct a full hypothetical situation where it can matter.

I'll see what I can come up with; I hope you'll do the same. As I said, up to this point, we haven't been able to identify any cases where it matters, so we haven't been able to take the next step and identify the -class- of cases where it matters. That's what we need to make a decision with. We need to know if we care about the class where it makes a difference. So, let's see what we can find.

I think any of the four systems can work (though maybe lookback adds an unneeded layer of complexity), so it would be great to have something other than instinct to base a decision on.

K

#849 ::: J Thomas ::: (view all by) ::: May 21, 2015, 09:12 PM:

If you could show me an example where eliminating both members of tie kept one of the eliminated works from becoming a finalist (with or without slates, so long as the example doesn't require the unlikely case of perfect discipline), then I would wholeheartedly agree that it matters.

A 350 votes
B 320 votes
C 310 votes
DG 200 votes
DE 10 votes
EH 200 votes
F 106 votes
G 6 votes
H 6 votes

Scores first round
A 350
B 320
C 310
D 105
E 105
F 112
G 106
H 106

First round, D and E are tied. If they are both eliminated it's over for them.

Breaking ties, we find some way to eliminate only E.
Scores 2nd round

A 350
B 320
C 310
D 110
F 112
G 106
H 206

D and G are selected.
D wins.

Scores 3rd round

A 350
B 320
C 310
D 210
F 112
H 206

D has more votes than anything it could possibly be matched against, so D will win.

From my point of view any combination of D E G H would be pretty good. DE is a little worse because they share close to 5% of votes. GH is a little better because they have two more votes and no overlap. CE or DH would be better still because they have no overlap and a few more votes.

So in this particular example, we got different results, both were good, the one from single-elimination was a tiny bit better.

I don't have any idea how many ways it can happen, or how likely they are. But it is possible.

#850 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 09:18 PM:

Folks:

The boss says it's time for happy drugs, so I'm off for a while. It turns out I seem to have a bit of pneumonia as a result of the surgery, but fortunately her dad is a pulmonary specialist and is coming over to check me out, so I'm in good hands. It maybe a bit before I can get back on, though.

TTYL,
Kilo

#851 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 09:45 PM:

(Sneaking one last post in...)

JT, I really like your example. So basically without any tie breakers, the results are ABCFGH (G and H because they are tied for 5th). With a tie breaker, we end up with ABCDH.

So, the real difference seems to be between F and D. How do those works differ in the larger sense?

K

#852 ::: felice ::: (view all by) ::: May 21, 2015, 09:48 PM:

Keith "Kilo" Watt @848: "Felice@847: I agree they are tied for nominations, but points are what matters (and obviously we don't have that information), but this looks promising as a way to construct a full hypothetical situation where it can matter."

This is a full hypothetical situation, where all overlapping works have already been eliminated aside from the two tied works, so for everything except the tied works, points and nominations are the same. Adding some more overlap between the various works would make for a more realistic hypothetical situation, but if everything is affected approximately proportionately by the extra overlap (ie everyone's points drop by the same percentage), the result is the same.

Eg if everyone looses another 1/3 of their points due to overlap with other finalists, Dan and Fiona drop from 47.5 points to 31.67 points, and the next work drops from 48 to 32 points, so Dan and Fiona still eliminate each other because they overlap just a teeny bit more than average and are still coincidentally tied.

Keith "Kilo" Watt @850: "The boss says it's time for happy drugs, so I'm off for a while. It turns out I seem to have a bit of pneumonia as a result of the surgery, but fortunately her dad is a pulmonary specialist and is coming over to check me out, so I'm in good hands. It maybe a bit before I can get back on, though."

I hope you're feeling better soon! Good luck.

#853 ::: felice ::: (view all by) ::: May 21, 2015, 09:54 PM:

Keith "Kilo" Watt @851: "So, the real difference seems to be between F and D. How do those works differ in the larger sense?"

D has 210 nominations, F has 106 nominations. F doesn't overlap with any other potential finalists. D overlaps a lot with G and a little bit with E, but since E and G are eliminated, that's irrelevant; D doesn't overlap with any of the other finalists either. D seems far and away the better result.

#854 ::: Jameson Quinn ::: (view all by) ::: May 21, 2015, 10:08 PM:

If we want "simple and >99% right", then 2a works.

If we want "As close to 100% right as possible, no matter how complex", then it's true that previous rounds are no help; it really shouldn't matter what already-eliminated works are on a ballot. In fact, what you should care about is look-ahead, not look-behind. That is, temporarily make the two tied works "protected", but let them keep their point allocation; continue (temporarily) eliminating other works according to the normal process, and reallocating votes, until the tie is broken; then eliminate the loser of the tie, and go back and un-eliminate all the temporarily-eliminated works, and continue as normal. Except, if you're doing the temporary eliminations, and before you get down to 4 works remaining, you get to a work that has more points than the tied works have nominations, you give up and eliminate both of them.

Which is totally a valid process, but not something I would look forward to trying to explain at the BM. So again, I think 2a is the best choice here on planet earth. Or, explicitly say "2a, and then any other non-random pre-announced tiebreakers as decided by the admins, and if the tie remains unbroken, then eliminate both". And the admins can talk to somebody like me, who can explain the above paragraph to them more patiently, without having to bother the BM.

#855 ::: J Thomas ::: (view all by) ::: May 21, 2015, 10:11 PM:

@851 Keith "Kilo" Watt

So basically without any tie breakers, the results are ABCFGH (G and H because they are tied for 5th). With a tie breaker, we end up with ABCDH.

The way I see it, with D and E both gone, G and H easily defeat F so the winners are ABCGH. But with a tiebreaker it's ABCDH as you said.

In both cases all of the shared votes are gone. D has 4 more votes than G so that's a better solution, 1390 versus 1386. A bit less than 0.3% better. Or if we compare only D to G, close to 2% better.

(When there are no shared ballots left on either side, I figure comparing the total votes is a good measure of results. I don't know how to compare when there are differences in vote and differences in shared vote both.)

#856 ::: Keith "Kilo" Watt ::: (view all by) ::: May 21, 2015, 10:36 PM:

Felice@853:

D and G are virtual equals in both points and nomination, as are E and H. Prior to eliminations, G is slightly ahead in points and D is slightly ahead in nominations. I don't think you can discount G so readily. Certainly you can only make the case D is better after you've eliminated G -- but without tiebreakers we haven't eliminated G.

Could this be the condition in which it matters? When you have two almost equal works, but one is winning in one measure and one in the other?


JT@855: Check your totals, with no tiebreakers, G and H don't defeat F.

Kilo

#857 ::: Jameson Quinn ::: (view all by) ::: May 21, 2015, 10:49 PM:

So, an update on the gofundme:

It's up to $930 in cash, or $1140 counting the pledged (but still unrealized) in-kind donation of a membership. I am really in awe of the generosity of this community. I thought I would have to resort to gimmicks; but no, the community is really doing it all by itself.

I'd still be happy to write guest posts about voting theory for anybody's blog, and I'll still keep my promise about giving 20 bucks to your favorite charity if you let me do that, but it's not looking as if that will be necessary. Just $260 to hit the goal. Wow.

#858 ::: felice ::: (view all by) ::: May 21, 2015, 11:09 PM:

Keith "Kilo" Watt @856: "D and G are virtual equals in both points and nomination, as are E and H. Prior to eliminations, G is slightly ahead in points and D is slightly ahead in nominations. I don't think you can discount G so readily."

I was going by your suggestion that "the real difference seems to be between F and D", which turns out not to be the case.

"Could this be the condition in which it matters? When you have two almost equal works, but one is winning in one measure and one in the other?"

Three almost equal works, one winning in one measure and two in the other. And the more overlap between the two tied works, the less close to equal with the third work they need to be. In this example, D and E only slightly overlap, so D can only be slightly better than G.

"JT@855: Check your totals, with no tiebreakers, G and H don't defeat F."

That would be true if G and H still had fewer points than F, but after D and E are eliminated, G and H are above F in both points and nominations, so F is the one that loses.

But aren't you supposed to be in bed? *worried frown*

#859 ::: Joshua Kronengold ::: (view all by) ::: May 21, 2015, 11:47 PM:

[TIES]

I've got a new idea re ties, which might satisfy the ultimate edge case, and still not usually produce bad results. It's not -entirely- dissimilar to felice's "ghosts" proposal, but I think it's a lot easier to understand, and the result is -entirely- non-arbitary. The worst possiblity is having a few extra things on the ballot (which frankly, for a weird edge case that usually won't happen--we can deal with):

When we get an unbreakable tie [tie on points -and- nominations], we don't bother resolving it. Instead, we combine the works -- treat any nominations where the works appear alone as a fraction of a nomination for the set, but ballots where both appeared now show as a single nomination for both works (with the scores reapportioned). The tied works are now joined, and survive and fall together--but until the nomination system is done, they only count as a single works.

This will allow a slate with -perfect- discipline -- and no out-of-slate votes -- to push extra things onto the ballot on occasion (in theory) but generally won't result in a change to nominations; it will in general mean that in a corner case where we'd normally pick a finalist randomly among two or three items, we just put them all on the ballot instead (which I'd argue is vastly preferable). And in cases where a tie would (as usual) result in all the works getting eliminated later, the same thing would occur -- the works would be combined in the tie, and then in a later round the tie would itself get eliminated.

So for a SDV-LPE(3):

5x A
2x B
3x BE
3x C
3x CD
3x DE

Scores:
A: 5, B 3.5, C: 4.5, D: 3, E: 3

Result: First, DE are combined, and this becomes:

5x A
2x B
3x B (DE)/2
3x C
3x C (DE/2
3x (DE)

Scores: A: 5, B: 4, C: 5, DE: 5

Result: B is eliminated.

There are now only three candidates left, so we stop. Nominees: A, C, D, E.

#860 ::: Joshua Kronengold ::: (view all by) ::: May 21, 2015, 11:52 PM:

At least, I think B is eliminated. In the last election, we effectively have a 4 way tie, so we eliminate the thing with the fewest votes. The number of votes for each thing are:

A: 5

B: 5

C: 6

(DE): 6 [(DE) is still prorated when it was an expansion from fewer votes, so it's 3+1.5+1.5=6; the in the half ballots people voted for B or C + half of DE.]

A and B tie, so we look at scores, and yes, B is eliminated on score.

#861 ::: Cally Soukup ::: (view all by) ::: May 22, 2015, 12:48 AM:

Keith "Kilo" Watt: Pneumonia sucks, especially when it's not the only thing you're trying to recover from; please take care of yourself. Best wishes for fast and full healing!

#862 ::: JJ ::: (view all by) ::: May 22, 2015, 02:50 AM:

Jameson Quinn, #857: "I'd still be happy to write guest posts about voting theory for anybody's blog, and I'll still keep my promise about giving 20 bucks to your favorite charity if you let me do that"

I'd personally rather see you do magic tricks, but hey, YMMV. Can I buy you a drink at Sasquan?

#863 ::: JJ ::: (view all by) ::: May 22, 2015, 02:55 AM:

Keith "Kilo" Watt:

Yes, please do take care of yourself. Sasquan is 3 months away. It will keep. Take care of yourself, and get everything better.

#864 ::: felice ::: (view all by) ::: May 22, 2015, 06:19 AM:

Joshua Kronengold @859: "I've got a new idea re ties"

Interesting. Took me a little while to get my head around the scoring, but I think it makes sense. The disadvantage over Ghosts/Lookforward is that it doesn't allow for ties to be broken in later rounds, but it looks like an improvement over simultaneous elimination. Is there a risk of ties accumulating? Eg (DE) ends up last equal with (ZX), creating a new tie (DEZX)?


An alternative explanation of Ghosts:

If the least popular works are tied on both nominations and points so it's not possible to pick a single work to eliminate, the tied works are put on probation. While works are on probation, they are allocated points as usual, but other works on the same ballots are allocated points as if the probationary works had been eliminated (the total number of points temporarily increases).

If two works already on probation tie for least popular, both are eliminated. If a work on probation ties with a regular work, the work on probation is eliminated. A work is taken off probation if the other work(s) it originally tied with are all eliminated. If a work becomes a finalist while still on probation, it only counts as half a work for determining the number of finalists. (Alternatively, a set of works on probation could count as a single work for determining the number of finalists, as per Joshua's idea.)

In most cases, works put on probation will remain least popular and be simultaneously eliminated in the following round, but there is potential for one or more of them to move up the rankings and avoid the next elimination round. A perfect slate will go straight to the top, but will leave three (or alternatively four) additional finalist slots for non-slate works.

#865 ::: J Thomas ::: (view all by) ::: May 22, 2015, 06:38 AM:

@858 Felice

"Could this be the condition in which it matters? When you have two almost equal works, but one is winning in one measure and one in the other?"

Three almost equal works, one winning in one measure and two in the other. And the more overlap between the two tied works, the less close to equal with the third work they need to be. In this example, D and E only slightly overlap, so D can only be slightly better than G.

Yes. Here's another:

A 350 votes
B 320 votes
C 310 votes
DG 10 votes
DE 200 votes
EF 10 votes
F 106 votes
G 102 votes

Votes:
A 350 votes
B 320 votes
C 310 votes
D 210 votes
E 210 votes
F 116 votes
G 112 votes

1st round score:

A 350
B 320
C 310
D 105
E 105
F 111
G 107

By double elimination, DE are lost. By single elimination, just E.

With double elimination we are down to 5 so the finalists are ABCFG.

Single elimination 2nd round scores:

A 350
B 320
C 310
D 205
F 116
G 107

F beats G, final result is ABCDF.

The difference is D versus G. D has nearly twice as many votes that aren't shared with anybody. Back when we were counting, D's votes were shared with E which made D look a little worse than G. But E got eliminated too.

When we let D and E eliminate each other, we can get a worse result. It can be almost as much as the difference between their votes and their scores. If the third one has votes better than theirs, then it makes no difference unless we decide to accept both for 5th place tie rather than eliminate one then. (If we have a way to break ties, there's no need to accept a 5th place tie except for tradition.)

I was wrong and Felice was right. It really can make a difference, and right now I see no reason to expect double elinination to get a better result often enough to make up for it, except for reducing the number of slate finalists. And if we penalize ties, slates will tie only by accident.

This might be a very small effect, but it's an effect in favor of breaking ties.

#866 ::: felice ::: (view all by) ::: May 22, 2015, 06:41 AM:

Another variant on the same basic concept: Lookforward

In the event of a tie for least popular, carry out a hypothetical run of the following rounds for each tied work as sole survivor. Whichever work is eliminated first in the lookforward runs should be eliminated for the actual next round. If works are tied for first lookforward elimination, they are both eliminated for the actual next round. If none of the tied works are eliminated in any of the lookforwards, then they are all finalists, and shall not be considered for elimination irrespective of point totals in any given round. They shall count as a single work for determining the total number of finalists.

This could potentially get a bit exponential with lookforwards inside lookforwards, but probably not to the extent that it would risk exceeding the capacities of modern PCs.

#867 ::: David Harmon ::: (view all by) ::: May 22, 2015, 07:01 AM:

Keith "Kilo" Watt #844: If it were due to a slate, then both should be eliminated....

Well, that's the problem. The voting algorithm does not know whether candidates are part of a slate. It fundamentally can't know that: EPH is a numeric algorithm operating on nominations and ballots, while "slatedom" is a matter of context -- pool-wide context at minimum, and properly, the social context surrounding the nominations.

That means that any tiebreaker can hit non-slate worlks as well as slates.

felice BTW, #842: I'm not really concerned about the pressure for early noms, it does seems pretty weak. My real point in #834 was just that the two double-tied works might well have the same datestamp, and not necessarily because they're on a slate. Figuring a separate median datestamp for each nominee might help with that, especially if it's calculated in floating-point.

Come to think of it, I think that's a general condition: A calculated real-number score breaks ties better than an integer score. That's because (unless it's quite poorly chosen) it can't perpetuate the tie. (Which is the basic problem with falling back from points to ballot counts.) And a real value is more appealing than a lottery draw such as hashing the names.

#868 ::: J Thomas ::: (view all by) ::: May 22, 2015, 07:22 AM:

@867 David Harmon

A calculated real-number score breaks ties better than an integer score.

Using floating point error to break ties is basicly flipping a coin.

#869 ::: David Harmon ::: (view all by) ::: May 22, 2015, 08:01 AM:

J Thomas #868: If it's coming down to floating-point error, then your scoring method qualifies as "poorly chosen" -- that is, it's not actually disambiguating all ballots.

Note that the original point system is technically real-valued, but according to the simulations reported here, it's not actually breaking all the ties -- my casual guess would be, that's because it's working with just a few small denominators, and even those drop out as points coalesce onto survivors.

The problem here seems to be finding a score that seems meaningful, but doesn't just reduce back to the point and nomination counts, which are already known to have ties even in combination. Some elements that would seem helpful:

1) Some non-linear function in the calculation, to get away from those small denominators?

2) Some transfer of information from outside the current round. Lookahead and lookback transfer information between rounds, but as described so far, they might not have enough information available. Hmm. Suppose nominees get a "coup score" based off a fraction of the total points their prior wins have eliminated from the ballot? That might at least be a suitable tiebreaker after the first round.

#870 ::: Jameson Quinn ::: (view all by) ::: May 22, 2015, 08:32 AM:

I endorse the "sticky ties" or "combined ties" idea. But writing out full, unambiguous language for how that all works is going to gunk up the proposal and delay us. How about if we say, "If the two selected works are tied on both approvals¹ and points, then from that point forward they will be considered as one work, with the same number of approvals. Full, unambiguous procedures for how to deal with such combined works can be adjusted by the election administrators as needed, as long as the adjustments are made public before the votes are counted."

¹I'm still dissatisfied with the word "nominations", and would prefer "approvals" or even "ballots". But that's not my main point here.

#871 ::: J Thomas ::: (view all by) ::: May 22, 2015, 08:39 AM:

Here's yet another approach to handle ties:

When there is a tie, there are three possible results. We can eliminate one, eliminate the other, or eliminate both.

I say, try every possibility and then choose the one that gets the best outcome.

There are some things I don't like about this idea. But I have an example with double-elimination where you can say "This definitely is not the best result. We would have had a better result doing it some other way." That cannot happen with this approach to tie-breaking.


----------- details of how it could work
We decide what result we want and we break ties in the way that comes closest to getting that result this time, with these ballots. If somebody says it isn't fair, we can point to what we said we wanted and tell them that we got it.


For early-stage ties (say, fewer than 10 votes per work involved) eliminate both.

For late-stage ties, try it all three ways and see what results we get. I'm going to claim (without evidence so far) that we'll hardly ever have more than five late-stage ties, so we'll hardly ever have more than a few hundred different runs to do. Usually it will be only a few.

Rate the outcomes by how many ballots include a finalist. If one outcome has the most ballots with finalists, use that. The rationale is that we don't have an obligation to works or to voters, but we have an obligation to the process. We want the result that represents the most fans.

If two or more outcomes tie on number of winning ballots, then count the number of ballots for each finalist that contain no other finalist. Choose the one whose fifth-place finalist has the most votes. Other things equal we prefer a better distribution of finalists. Better each of them has solid backing, than a couple of winners and a few little also-rans that fill in the cracks.

If there's also a tie on distribution, check whether the winners are only permutations on tied slate items. If so, choose them based on the order presented on the first nomination.

This might not resolve all ties, but when it does resolve them, it does it based on the outcomes we want. The way I wrote it, we might sometimes eliminate one and sometimes both, whatever gets the best result. If we always eliminate both or else never eliminate both, there will be fewer alternatives when there's more than one late-stage tie.
----------------

#872 ::: Jameson Quinn ::: (view all by) ::: May 22, 2015, 08:51 AM:

The GoFundMe is now fully funded, assuming that the promise of an in-kind donation of a paid membership comes through. Thank you all! Again, I'm really humbled by how generous this community is.

Now, I'm looking for a place to stay. I'm a nice guy, fully housetrained, not the least bit creepy. I can do my guest's share of cooking and cleaning (where the host gets full discretion deciding what "guest's share" is, down to a minimum of proportional washing duty). I can also chip in a for shared lodging, if that's what's available. Anybody with an offer can contact me at my google email, which again is my first and last name separated by a dot (or not... google doesn't actually care about dots.)

I'm leaving the gofundme up, in case people want to make extra donations to electology.org to prevent puppylike pathologies percolating past prizes, and promote proportional politics. Silly alliteration aside, if you've ever felt about electoral politics the same way you feel about this year's initial Hugo finalist list, similar solutions apply; and that's what Electology.org is about. We're a small organization, but definitely growing, and we already have gotten relevant legislation sponsored in over 4 states, as well as helping private organizations like Worldcon improve their voting systems.

#873 ::: J Thomas ::: (view all by) ::: May 22, 2015, 09:02 AM:

Note that the original point system is technically real-valued, but according to the simulations reported here, it's not actually breaking all the ties -- my casual guess would be, that's because it's working with just a few small denominators, and even those drop out as points coalesce onto survivors.

That's right! It comes down to 1/60'ths and you never get a smaller fraction than that.

1) Some non-linear function in the calculation, to get away from those small denominators?

That could help break ties.

There's a problem that what we have at this point is highly co-adapted. Things mostly fit together in a package that's easy to explain, there are just a few sticking points. Many of the things we could change cause problems elsewhere.

We settled on using scores that are split 1/2 1/3 1/4 1/5 partly because it's easy to explain and it sounds fair. If we switched to something else now, we would have to rethink how to explain it. We would have to redo the simulations etc that imply we might not run into unexpected problems.

It might be a good approach but it would take a lot of work when we want to think we're close to finishing.

#874 ::: David Harmon ::: (view all by) ::: May 22, 2015, 09:50 AM:

J Thomas #873: There's a problem that what we have at this point is highly co-adapted.

Yeah, but the tie-breaking bit is still open, and disproportionately contentious. Indeed, our difficulty in finding a resolution here, may well be a hint that once past points and ballots, tie-breaking should be left to the Worldcon/election administrators.

#875 ::: junego ::: (view all by) ::: May 22, 2015, 10:39 AM:

[TIES]

I'm waffling a bit between 2a and 2b. I could accept either at this point.

On the new ideas being thrown in the ring - I need more examples/information to evaluate. So, please, those of you working these angles, keep the discussion going.

#876 ::: Keith "Kilo" Watt ::: (view all by) ::: May 22, 2015, 11:43 AM:

JT@865
When we let D and E eliminate each other, we can get a worse result. It can be almost as much as the difference between their votes and their scores.

Sorry, I'm not following this. Why is it worse? The reason is probably in the second sentence here, but I don't understand that part. Don't get me wrong, I'm not opposed to breaking ties if we can, but I want us to be clear as to why.

JT@871:
I say, try every possibility and then choose the one that gets the best outcome.

That's be great, but there's no definition of "best outcome". Your definition seems to me to be "the most nominations should win", but for reasons discussed above, that won't work.


Regarding combined ties:

I'm intrigued by the idea. It's a "look forward" class of tie breaker, which certainly looks promising from a theoretical standpoint. Felice's "ghost rider" proposal is as well, but I think this one is easier to code and explain.


My understanding is that it only gets used when we have a tie for both nominations and points, correct? We then divide both points and nominations by the number of tie members and proceed as normal, but the tied works live or die together. It's a big enough issue that I think we need to run some sims to see how it plays out. It looks to me like you will start to get "accumulated ties", but they will start doing mass elimination at some point. That should be okay, as they are still playing the game (which JT and felice like) until they are unambiguously eliminated. Let me see what I can code up. If I have the description right, I think I know a fairly straightforward way to do it (famous last words, I know...)

Kilo


#877 ::: Jameson Quinn ::: (view all by) ::: May 22, 2015, 11:55 AM:

@876: I think you're misreading @871. The "best" is not "most nominations per individual", it's "fewest discarded ballots", which I think is pretty darn defensible.

As for all the further tertiary tiebreakers in @871, that's just silliness. If "fewest discarded ballots" doesn't break the tie, just keep both. We're already open to the possibility of ties extending the number of finalists, and frankly, this increases the chances of that by an immeasurably small amount. (If bootstrap were the measure, I am willing to wager that it's literally immeasurably small; that it would never happen in a hundreds of thousands of bootstrap samples.) I like the basic idea of @871, but the extra "and then if that doesn't work..." stuff is a bad idea.

#878 ::: Cally Soukup ::: (view all by) ::: May 22, 2015, 11:59 AM:

[TIES]

For the purpose of this post I'm calling works which reach the elimination stage and are at that point identical on both points and number of ballots "twins". Speaking as someone who has not been following the math, as a purely gut feeling I really rather like the conjoined twin idea where both twins stay or are eliminated together. It simply feels fair: if they're identical at the elimination stage, let them BE identical thenceforth. I've gathered that in a lookback scenario twins are unlikely to be much different from each other; how unlikely is it that in a lookforward scenario one or the other of a pair of twins will leap ahead of the other? Can someone explain in words for me? Sorry, I'm usually good at math, but for some reason Hugo math just isn't sticking in my brain.

#879 ::: Cally Soukup ::: (view all by) ::: May 22, 2015, 12:04 PM:

Attempting to knock loose a Server Error.

#880 ::: Jameson Quinn ::: (view all by) ::: May 22, 2015, 12:16 PM:

@878: "I've gathered that in a lookback scenario twins are unlikely to be much different from each other"

That's actually not really what the evidence says, as I see it. Keith did some simulations, and never saw twins where lookback would resolve it. But I think that's just because he never saw twins at all except early in the process, or artificially created by building a pure slate scenario. In more organic data, I think that most twins that exist, and that aren't so early they would never matter anyway, will be resolvable by lookback.

But I am also sympathetic to the argument that lookback isn't really what we want, it's just an arbitrary way to break the tie.

The situation is somewhat the other way around in a lookahead scenario: the closer to the end of the process it is, the less likely lookahead will resolve a tie. So lookahead is more philosophically grounded, but less likely to help in the cases where it matters.

So yes, I like the "cojoined twins" idea too. But I think it would be better to keep the proposal a bit sketchy on exactly how that works, and let the admins fill in the details. I trust they'd do so sensibly; and trying to write every single unlikely corner case into the statute seems silly to me.

#881 ::: Duncan J Macdonald ::: (view all by) ::: May 22, 2015, 01:03 PM:

Felice @823: Any comments on my thoughts on how arbitrary lookback is? (#686 and #723)

By definition (Mirriam-Webster) arbitrary --"depending on individual discretion and not fixed by law" -- is not what we're speaking of, since we are looking at defining it in the rules.
Additionally, lookback is repeatable and verifiable, so I don't consider it to be arbitrary.

#882 ::: David Harmon ::: (view all by) ::: May 22, 2015, 01:13 PM:

[TIEBREAKING]

Just throwing out a random idea: Suppose we break the original point scores away from their fractions just a little bit... say, adding 0.001 to their score each time they survive an elimination. (That is, from 0.2 to 0.251 to 0.335+ .)

Or alternatively, subtract the same -- one way is "consolation prize" for not being chosen, the other is a penalty for same. Either way, it effectively appends the ballots' "round number" onto the scores for their remaining nominations.

It might not work for perfect slates or some constructed cases, but might help if practical ties are more likely to be assembly-of-fractions rather than identical ballots in lockstep.

#883 ::: J Thomas ::: (view all by) ::: May 22, 2015, 02:05 PM:

@876 Keith "Kilo" Watt

"When we let D and E eliminate each other, we can get a worse result. It can be almost as much as the difference between their votes and their scores."

Sorry, I'm not following this. Why is it worse?

Compare the two results in #865.. ABCFD versus ABCFG. D has 210 votes, G has 112, neither of them shared with any other finalist. With everything else equal, isn't it better to choose a finalist with more votes over a finalist with fewer votes? All else equal? I think it is, in that case.

The *reason* it comes out with the smaller number was that D and E shared a lot of votes which brought down their scores, and they competed with each other and both lost. If either of them survived then G would be knocked out.

It's only because it's a boundary case. One extra vote either way and one of them would survive. Because they're exactly balanced, removing both can give a worse outcome.

"I say, try every possibility and then choose the one that gets the best outcome."

That's be great, but there's no definition of "best outcome". Your definition seems to me to be "the most nominations should win", but for reasons discussed above, that won't work.

"Most nominations should win" is a bad outcome for individual nominations, because that lets ties sweep. But if it's "most nominations should win" for whole groups of finalists, it's much better. The second slate work gives you nothing, or almost nothing.

This might be a good way to choose among outcomes that have have all mostly been created by SDV-LPE. It can give some bad results if it's the only thing you choose by. For example, two of your finalists could be D and E which have 50 and 51 votes not shared by any other finalist, when A which is not included has 101 votes that are shared by D or E. A and E would be a little better than D and E, since other things equal A is better than D and there's nothing about this particular collection of finalists that makes D better.

Here is a related mixup -- throw out both C and D and replace them both by A. Find some F that has, say, 4 ballots that aren't shared with any finalist. AF has 4 more votes than CD. But if F has only 4 votes then F is not a very good finalist in itself.

So to really say what's good we'd need other criteria and not just maximize ballots used. It's one thing we want, but not everything.

#884 ::: Jameson Quinn ::: (view all by) ::: May 22, 2015, 02:10 PM:

@882: Not a bad idea. Essentially equivalent, but less mathy, is to say "if two works are tied on approvals and points, then count up all the works on all the ballots that approve either, and the winner is the one with more." It's a tiny anti-bullet-voting incentive, to begin to balance out the small pro-bullet-voting incentives in the base system.

Again, I think we've now brainstormed at least 4 good ideas (cojoined, 2 or 3 flavors of lookahead, 2a, total works on ballots, add small fraction) and several not-so-ideal-but-perfectly-acceptable ones (1, 2b, etc.). Getting perfect consensus here is probably going to take time, because while I feel comfortable going by my intuitive evaluations of those systems, I don't expect others to trust my intuition or to have exercised theirs, so we'd ideally want to simulate and find scenarios and carefully explain and legislate all of them... which seems like a lot of work. So the more good ideas I read, the more strongly I feel that we should leave this question up to the admins, with certain guidelines (nothing random, must be preannounced or use same system as previous year, etc.).

#885 ::: Keith "Kilo" Watt ::: (view all by) ::: May 22, 2015, 03:24 PM:

JT@883
With everything else equal, isn't it better to choose a finalist with more votes over a finalist with fewer votes? All else equal?

But everything isn't equal. G has more points than D. You have a bias towards valuing nominations, but slates will -always- win in nominations. While I realize you don't have a slate in this example, if we choose that as our standard of measure, then you aren't going to like anything that SDV-LPE does -- you'd prefer the current system. I strongly suspect that a lot of the results you don't like are specifically because most nominations didn't come out on top. But again, that leads us right back to the current system and with slates dominating. You base a lot of your arguments on this one thing, and it's just not something that I can agree with.

I'm not sure we will ever agree that "most nominations should win." It's been a sticking point in the discussion between you and I from the beginning. If you have to have only one measure of "goodness" then choose points, not nominations.

Kilo

#886 ::: J Thomas ::: (view all by) ::: May 22, 2015, 03:52 PM:

@885 Keith "Kilo" Watt

JT@883
"With everything else equal, isn't it better to choose a finalist with more votes over a finalist with fewer votes? All else equal?"

But everything isn't equal. G has more points than D.

No, it doesn't. After the finalists are selected, in either case there are no ballots that have two winners. At that time every finalist has the same number of points.

So other things are equal between those two solutions to the problem of finding 5 finalists.

Imagine the following: voting systems that purport to fit our values have produced the following two outcomes:

A 500 votes, no shared ballots
B 420 votes, no shared ballots
C 380 votes, no shared ballots
D 350 votes, no shared ballots
E 322 votes, no shared ballots

A 500 votes, no shared ballots
B 420 votes, no shared ballots
C 380 votes, no shared ballots
D 350 votes, no shared ballots
F 260 votes, no shared ballots

Which do you think is the better solution?

I think in this case the first solution is better. We might say that the second solution is better because a better process was used to find it. But no, this time the better process got a worse result.

We might say that a voting process is better than another because its methods are fairer to something or other. We might say that a voting process is better because its results are better on average.

But I don't think we can plausibly say that result 2 is better than result 1.

#887 ::: Annie Y ::: (view all by) ::: May 22, 2015, 03:55 PM:

J Thomas @ 883

So how do you differentiate between a partial or full slate and "everyone and their mother put the same two novels on their ballots"? Once the weaker works are out, the first and the second look very similar (especially if the slate is at least semi-competent (one can hope) and other people had nominated slate works as well).

Trying to build a border case around a slate is just counterproductive - if there was a way to determine that something is a slate, we could have dealt with it...

It can give some bad results if it's the only thing you choose by. For example, two of your finalists could be D and E which have 50 and 51 votes not shared by any other finalist, when A which is not included has 101 votes that are shared by D or E. A and E would be a little better than D and E, since other things equal A is better than D and there's nothing about this particular collection of finalists that makes D better.

Why is A better in this case? Because it has more votes? But if that is more important, let's use the current system and just count votes. Plus - this relies on very close math and A being shared with D and E almost equally. Isn't that too much of a border case? Can it happen? Yes. Will it be bad if we loose A? I am not sure that it will be.

#888 ::: Annie Y ::: (view all by) ::: May 22, 2015, 04:06 PM:

felice @ 842

Because if it's on the same ballot, then eliminating one the tied works will give the other more points, and push it up the rankings; it might not be one of the last two any more. But if they don't have any ballots in common, whichever survives elimination won't gain any points and will still be in last place, and will probably be eliminated in the next round (definitely eliminated next round with 1,1,2 selection).

Why would the second 1 be the victim of the 2 coming on the next elimination - it still can beat on votes.

My point is that looking only at how it ties now is not enough to tell us what will happen next time. If they do not share any ballots, points won't change for the survivor so it will stay at the bottom 2 but it does not mean that it cannot beat next time and start picking points going forward.

#889 ::: J Thomas ::: (view all by) ::: May 22, 2015, 04:18 PM:

887 Annie Y

Why is A better in this case? Because it has more votes? But if that is more important, let's use the current system and just count votes. Plus - this relies on very close math and A being shared with D and E almost equally. Isn't that too much of a border case? Can it happen? Yes. Will it be bad if we loose A? I am not sure that it will be.

A long time ago, maybe back in the first of our three threads, I proposed a voting system which was rejected because of this exact example.

Not exactly this, but exactly the same idea.

As I remember the example they gave, there were five winners BCDEF which each had the same number of votes and no overlap, and by my reasoning this would be a very good outcome.

But another work A precisely overlapped B C and D, and the argument was that A was a better work than D but my criteria did not say it was better.

Therefore my proposal was unacceptable.

I revised the proposal to correct this flaw, and it was still utterly rejected.

Since then I have accepted the assumption that other things equal, a finalist that has more votes is better than a finalist that has fewer, even if some or all of the extra votes are shared with other finalists.

If all we care about is the number of votes and nothing about the distribution of votes, we will get slates.

If all we care about is distribution of votes and nothing about number of votes, I don't know what we'll get but it won't be pretty.

I am convinced that other things equal,

A 420 ballots
B 419 ballots

is better than:

A 400 ballots
AB 20 ballots
B 399 ballots

and this is better than:

A 400 ballots
B 399 ballots.

But there are some combinations that I can't tell which is better.

#890 ::: Keith "Kilo" Watt ::: (view all by) ::: May 22, 2015, 04:28 PM:

By the way, I've updated the data structure in my code to allow it to handle some of the new tie breakers that have been proposed. I've confirmed everything using method 2a, so I don't think I've created any new bugs. I'll work on linking ties next. I think we may be at the point where we need some sims.

I'm not adamantly opposed to leaving the final tied-for-points-and-nominations breaker to the admins to decide, so long as we provide some workable suggestions.

One thing to keep in mind is that the admins are volunteers and likely won't have time to code anything complex. SDV-LPE itself is very easy to code, but we don't want to ruin that with an edge-case tie breaker.

K

#891 ::: Annie Y ::: (view all by) ::: May 22, 2015, 04:30 PM:

J Thomas @ 889
I am trying to make any sense of your last example and I think I need some help.

So in your last example A and B are up for elimination, they are somehow on their own with at least 4 more works still in play that do not share any ballots with either of them but have more points than either of them (and because these here are singles, we are talking about a lot of ballots still in play (a full slate of DSEFG hiding at 2500 ballots for example).

But in this case, this makes no sense. And if those have neighbours, then the points should be resolving this, correct?

The problem is when points AND votes match, correct? In neither of those they match so... the lower in votes is off as per the rules.

So what am I missing?

Or are you trying to say something else?

#892 ::: J Thomas ::: (view all by) ::: May 22, 2015, 04:59 PM:

@891 Annie Y

Or are you trying to say something else?

Yes!

At that point I'm not talking about how to run a voting system. I'm saying, after we've run a vote and we have our finalists, how do we decide how good a job the voting system did?

If we get two different results, like we might get from different ways to handle ties, how do we decide which is better?

I can't always tell which result is better, but sometimes I can tell.

If we run two different voting systems and we get results that are exactly the same except for one finalist, and one way the different one has more votes, that is a better outcome.

That will not get you a "better" outcome with more slate finalists. Because if you replace a nonslate finalist with a slate finalist, the result won't be exactly the same except more votes. The nonslate finalist almost certainly had some ballots that went to it alone, and the second slate item will have none of those or only a very few.

But if the only difference between the solutions is that one finalist has more votes that are not shared with any others, I say that's a better solution.

And if the only difference is that one finalist has more votes that are shared with others, and we can pair up the finalists from the diferent solutions so that for every finalist in the worse solution there's a finalist in the better solution that has at least as many votes and at least as high a final score, and at least one of the finalists in the better one is improved (more votes and/or higher score) then the better solution is in fact better.

What got me looking at this, was that I have an example where 2b gets a better result than 2a. And the only example I have where 2a gets a better result than 2b, comes when it is better at eliminating extra slate nominees. But slates can try to avoid ties, while there's nothing nonslates can to to avoid random ties that nobody predicted.

#893 ::: J Thomas ::: (view all by) ::: May 22, 2015, 05:00 PM:

@891 Annie Y

Or are you trying to say something else?

Yes!

At that point I'm not talking about how to run a voting system. I'm saying, after we've run a vote and we have our finalists, how do we decide how good a job the voting system did?

If we get two different results, like we might get from different ways to handle ties, how do we decide which is better?

I can't always tell which result is better, but sometimes I can tell.

If we run two different voting systems and we get results that are exactly the same except for one finalist, and one way the different one has more votes, that is a better outcome.

That will not get you a "better" outcome with more slate finalists. Because if you replace a nonslate finalist with a slate finalist, the result won't be exactly the same except more votes. The nonslate finalist almost certainly had some ballots that went to it alone, and the second slate item will have none of those or only a very few.

But if the only difference between the solutions is that one finalist has more votes that are not shared with any others, I say that's a better solution.

And if the only difference is that one finalist has more votes that are shared with others, and we can pair up the finalists from the diferent solutions so that for every finalist in the worse solution there's a finalist in the better solution that has at least as many votes and at least as high a final score, and at least one of the finalists in the better one is improved (more votes and/or higher score) then the better solution is in fact better.

What got me looking at this, was that I have an example where 2b gets a better result than 2a. And the only example I have where 2a gets a better result than 2b, comes when it is better at eliminating extra slate nominees. But slates can try to avoid ties, while there's nothing nonslates can to to avoid random ties that nobody predicted.

#894 ::: felice ::: (view all by) ::: May 22, 2015, 06:14 PM:

Annie Y @888: "My point is that looking only at how it ties now is not enough to tell us what will happen next time. If they do not share any ballots, points won't change for the survivor so it will stay at the bottom 2 but it does not mean that it cannot beat next time and start picking points going forward."

True - that was the point of 1,1,2 selection; if the tied works with fewest points can beat the work with second-least points, we should eliminate the second-least points work instead of the tied works. With that rule in place, simultaneous elimination is only a problem for overlapping works.


Keith "Kilo" Watt @885: "I'm not sure we will ever agree that "most nominations should win." It's been a sticking point in the discussion between you and I from the beginning. If you have to have only one measure of "goodness" then choose points, not nominations."

Points are a means to an end, and change over time. The best outcome maximises the number of points for the finalists after everything else has been eliminated; what points had been earlier in the process is no longer relevant. A sweep by a perfect slate of 100 people is only worth 100 points (1/5 x 100 x 5), while five finalists each nominated by 50 people with no overlap is worth 250 points (1 x 50 x 5) - a much better outcome. One slate work and four non-slate works is better yet - 300 points.

For ties, consider a set of finalists with 95, 90, 85, 80, and 75 nominations respectively, with no overlap - that's 425 points. Suppose there were two other works with 100 nominations, overlapping by 60, so they were tied on 70 points. With tiebreaking, one of them goes up to 100 points, pushes the 75 nomination work out, and the new set of finalists is worth 450 points. That's a better result on points. With 1,1,2 selection, the pair eliminate 75 and then 80, for a new set of finalists worth 410 points. A worse result on points (a better result on nominations, but we're not using nominations as the measure of goodness); so 1,1,2 selection can produce inferior results. With Probation/Ghosts, both 100s become finalists but only take up one slot, so we get six works on the final ballot, and final points of 490, but at the price of extending the ballot.

#895 ::: Keith "Kilo" Watt ::: (view all by) ::: May 22, 2015, 06:40 PM:

felice@894:

The best outcome maximises the number of points for the finalists after everything else has been eliminated; what points had been earlier in the process is no longer relevant.

I like your analysis (here and following), and I agree this is a better way of looking at it. I was evaluating based on the initial number of points received, but you're right that the final total makes more sense.

For the second part, keep in mind that we ruled out 1,1,2, but I want to make sure I follow your example anyway. Under this, I see that we would compare the pair and the 75, since they are the two lowest point values. The pair has more nominations, so the 75 is eliminated. So, using strictly points, this would be considered a worse result.

With your ghost rider example it's true you will get more points on the ballot total, but I caution you that will always be true any time you expand the number of finalists. (To get -maximum- points, don't eliminate anyone -- which of course is actually likely the best suggestion overall but is impractical for all the reasons discussed in the first thread. We really do have to limit the number of finalists, hence SDV-LPE.)

Still, overall, I like it. You've mathematically shown that 1,1,2 gives a worse result than 1,1 (which is what your 425-point situation is). That's really useful. To be clear, I don't think that ignoring number of nominations completely is the answer either, but if we want to just pick one measure, what you've proposed looks like a pretty good idea to me. Just consider your first example: You've shown mathematically that a final ballot with one slate and 4 non-slate works (which is one of the goals of the whole system) gives the best point total of all -- even better than eliminating the slate entirely, which most here would contend is no more fair than the slate eliminating everyone else.

It might be worth applying this analysis to some other situations, but just based on what you've shown, I think you might have a winner here. It's certainly much more convincing.

Kilo

#896 ::: Buddha Buck ::: (view all by) ::: May 22, 2015, 06:41 PM:

[TIES]

There are other groups which feel that ties at some stage in their process must be resolved, and thus have elaborate tie-breaking strategies. Almost all of them have an ultimate fall-back of a random tie-breaker. As an example, the order of finishers in each group in Round 1 of the FIFA World Cup determines who will play whom in Round 2, and so every team must be ranked in order (no ties for 2nd place allowed). They have 7 criteria they used for ranking, starting at points earned, and ending with lots drawn by FIFA officials. I don't think that a final fall-back to lots is unheard of, nor should be shunned, if we have enough other potential tie-breakers to try first. I think that nominations, then look-back points, then lots if looking back to the beginning doesn't eliminate anyone, is reasonable.

As an alternative, to avoid both randomness and double-elimination in the face of ties, how about split-and-merge: If A and B are "twins", split the future rounds, such that A is eliminated in one set of future rounds and B is eliminated in the other. If it ever happens that a set of future rounds have the same remaining works, then only one set of rounds needs to continue (both branches would be identical from then on). If the two branches yield different finalists, merge the finalist lists.

I suspect that with branch-and-merge, early ties of unpopular works will become moot fairly quickly, and ties of popular (or slated) works will result in both making it in a larger finalist pool.

It is analogous to the rerun-merge-extend solution for dealing with declines/late-ineligibles.

(As a computer person, I'm inclined to call it "Non-deterministic" LPE or SDV-NLPE, in analogy to non-deterministic FSMs, which follow the general rule of "when you come to a fork in the road, take both paths". But it has no randomness, and will always yield the same result when rerun, which is what most people want when they say they don't want a non-deterministic algorithm for this.)

#897 ::: Keith "Kilo" Watt ::: (view all by) ::: May 22, 2015, 07:00 PM:

Buddha@896:

I think branch-and-merge is interesting, but I think the problem may be that there will be too many branches (and branches within branches) to track. Early on, there will likely be lots of ties with works that only have a few points. You'd have to follow a branch at each of the bifurcation points, which will get complex quickly. That said, those early ties have almost no chance of surviving for long, so they don't really matter, but your algorithm would have to track them just to be sure.

K

#898 ::: felice ::: (view all by) ::: May 22, 2015, 08:03 PM:

Keith "Kilo" Watt @895: "For the second part, keep in mind that we ruled out 1,1,2, but I want to make sure I follow your example anyway. Under this, I see that we would compare the pair and the 75, since they are the two lowest point values. The pair has more nominations, so the 75 is eliminated. So, using strictly points, this would be considered a worse result."

It's not a result at all at this stage; it's only once 80 is eliminated as well and the finalists are determined that we have final point totals. There is no worse or better result for a particular elimination round; that's just following the process. The only meaningful comparison is between the outcomes of different variants of the process.

"With your ghost rider example it's true you will get more points on the ballot total, but I caution you that will always be true any time you expand the number of finalists."

True (well, almost - a perfect slate tied for fifth is worth the same number of points as a single slate work, extending the ballot to 9 without any extra points). A better measure might be to give each of the probationary finalists 100 points each and divide by two, for a total of 450 points (the same as using a tiebreaker).

Still, overall, I like it. You've mathematically shown that 1,1,2 gives a worse result than 1,1 (which is what your 425-point situation is).

It gives a worse result in one specific example; I have no idea if it's generally true, and I suspect it's not. For example, if the 4th placed work had 110 nominations, 1,1,2 would only eliminate the 75, then the tied works would be tied for 5th place and stay, extending the ballot (a better result because either individually would have more points than the 75). But I suspect it gives less consistent results than tiebreaking or probation.

That's really useful. To be clear, I don't think that ignoring number of nominations completely is the answer either, but if we want to just pick one measure, what you've proposed looks like a pretty good idea to me.

Points are derived from nominations (and in cases of no overlap, are exactly the same), so nominations are already accounted for; we don't need to worry about ignoring them. NB this is essentially the same measure that JT has been talking about.

Keith "Kilo" Watt @897: "I think branch-and-merge is interesting, but I think the problem may be that there will be too many branches (and branches within branches) to track. Early on, there will likely be lots of ties with works that only have a few points. You'd have to follow a branch at each of the bifurcation points, which will get complex quickly. That said, those early ties have almost no chance of surviving for long, so they don't really matter, but your algorithm would have to track them just to be sure."

I doubt it would be a problem computationally, but it would be a nightmare to check the output. Eliminating works with only a few nominations before we start would reduce it to manageable size, though, I think. Works with fewer than 1/5 the nominations of the 5th most popular work are safe to eliminate at the start, because they can't win (even if the points for the five works with most nominations are split five ways, eg a perfect slate, these works will still rank below them and be eliminated). Eg in my example from #894, the top five are 100, 100, 95, 90, 85; so anything with less than 85/5 = 17 nominations is not a potential finalist under any circumstances (someone else pointed this out in a previous thread).

#899 ::: Darth Paradox ::: (view all by) ::: May 22, 2015, 08:31 PM:

I have been convinced by some of the pathological examples that have been given of odd behavior when eliminating more than one work at a time is a bad idea. I also think eliminating something other than one of the two (or N if tied) lowest-scoring works is a bad idea.

Anything relying on adding more factors into the calculations (e.g. bonus points for surviving eliminations, timestamps of ballots, etc) does not guarantee that the tie will be broken unless you can prove that those factors will always result in different scores for two otherwise tied works. Absent that proof, you're always going to need an "and if there is still a tie after that" clause - which is by its nature going to be arbitrary

The existence of a tie on both points and nominations is a delicate enough situation (e.g. it could be changed by a single nominating ballot that got lost in the mail) that I don't think it's a problem to break the tie with a "coin-flip" approach. After all, we're talking about nominations here, not the final winner; a work that barely makes it onto the ballot due to winning a coin flip probably has a relatively low chance of winning the award anyway[1]. Perhaps to mitigate this, we can still accept that if the tie is for Nth place or higher (where N is the target number for the final ballot, i.e. currently 5), then all works tied appear on the final ballot.

So, to my mind, the only other downside to the coin-flip approach is that it is inconsistent (multiple runs of the voting system ending with different results). I think this can easily be handled with a PRNG seeded to a consistent value, defined in advance (e.g. the year of the award). There are more complex ways we could describe it, depending on how technical we want to get, but a PRNG with a known seed gets us both enough randomness to simulate a coin-flip and enough consistency to prevent different results on different runs of the calculation program.


[1] I understand this is a huge assumption on my part, and maybe someone with better access to the data than I have can say - how often does a work that barely made it onto the ballot end up winning?

#900 ::: Keith "Kilo" Watt ::: (view all by) ::: May 22, 2015, 08:40 PM:

felice@898:
It's not a result at all at this stage

Yes, sorry, that wasn't clear. I meant your whole example, including the last step. That wasn't clear from what I wrote.

Points are derived from nominations (and in cases of no overlap, are exactly the same), so nominations are already accounted for; we don't need to worry about ignoring them.

In this case I agree -- points includes some aspect of nominations, but nominations do not include any aspects of points. I just wasn't ready to throw out other considerations in cases where we get more points but fewer nominations in a winner. I -think- that's still the best result, but I'm not ready to state that conclusively without more situations to look at.

NB this is essentially the same measure that JT has been talking about.

Actually, it's not, at least not the way it's been coming across. On multiple occasions he's advocated specifically for more nominations as being the better measure. As mentioned above, points includes nominations, but nominations don't include points. That's a fundamental difference.

Kilo

#901 ::: felice ::: (view all by) ::: May 22, 2015, 09:11 PM:

Duncan J Macdonald @881: "By definition (Mirriam-Webster) arbitrary --"depending on individual discretion and not fixed by law" -- is not what we're speaking of, since we are looking at defining it in the rules. Additionally, lookback is repeatable and verifiable, so I don't consider it to be arbitrary."

Arbitrary in the sense of making a decision based on an irrelevant factor; it's no better than picking a winner based on alphabetical order of work titles. Nobody is advocating anything that isn't repeatable and verifiable.


Keith "Kilo" Watt @900: "On multiple occasions he's advocated specifically for more nominations as being the better measure"

Only in cases of no overlap amongst the finalists, as far as I recall, where points and nominations are the same thing. Or counting ballots, not nominations, when evaluating a complete set of finalists.

#902 ::: junego ::: (view all by) ::: May 22, 2015, 10:36 PM:

Jameson Quinn @ 884

Again, I think we've now brainstormed at least 4 good ideas (cojoined, 2 or 3 flavors of lookahead, 2a, total works on ballots, add small fraction) and several not-so-ideal-but-perfectly-acceptable ones (1, 2b, etc.). Getting perfect consensus here is probably going to take time, because while I feel comfortable going by my intuitive evaluations of those systems, I don't expect others to trust my intuition or to have exercised theirs, so we'd ideally want to simulate and find scenarios and carefully explain and legislate all of them... which seems like a lot of work. So the more good ideas I read, the more strongly I feel that we should leave this question up to the admins, with certain guidelines (nothing random, must be preannounced or use same system as previous year, etc.).

It may come to giving these more open ended rules over to administrators. I've really liked a couple of the latest suggestions, especially the survivor point additions and the most ballots. I think it would be good to continue discussion for at least another day or two. This is almost the last real procedural hurdle (I think?)

After that we just need to nail down verbiage and maybe some strategy suggestions.

My point is, let's discuss this a little more before punting. If nothing else we will have some excellent tools to recommend to the administrators.

#903 ::: Jameson Quinn ::: (view all by) ::: May 22, 2015, 10:47 PM:

@902: I'm still leaning ever more strongly towards leaving it open for the admins to decide. But yeah, sure, more discussion is good, in terms of giving them more options and basis for that decision.

#904 ::: Keith "Kilo" Watt ::: (view all by) ::: May 22, 2015, 11:13 PM:

Junego@902:
This is almost the last real procedural hurdle (I think?)

Yep, this is all that's left. We're actuality in pretty good shape.

Whatever we decide about letting the admins rule, I think we should have an idea about the effects of the various options. I like Felice's measure of "goodness", though I think some more application and testing of it is order. That might help better evaluate some of these cases, though.

I'm working on coding some sims for the different cases, so that will help as well. For me, I think 2a or linking ties look the most promising. 2a is simple and transparent. I'd almost consider it the default case to be used for comparison. Linking ties is a somewhat less simple, but it is very consistent with SDV-LPE from a theoretical standpoint since it is a look-forward method. Still, it's not so complicated that it can't be explained and coded easily: If the works are really equal, then we'll combine them and let them live or die as one. Simple and intuitive. Makes a lot of sense...

I'll keep you posted on the code; I'm having to work in spurts, which is less than efficient.

Kilo


#905 ::: felice ::: (view all by) ::: May 22, 2015, 11:49 PM:

[TIES]

I think it might help to vote on how we prefer to handle perfect slates - the worst case scenario for ties. That would help decide on a general principle for dealing with all ties. I think these are the only real options (score from 0 to 10):

[ 7 ] Keep them all and expand the ballot to incorporate non-slate works as well (more finalists than ideal, but not unfair to anyone; and voters aren't forced to read the slate works)
[ 6 ] Use some arbitrary but repeatable method to select a subset of slate finalists (proportional but bad luck for the authors who miss out by chance - though at least they were given a chance)
[ 1 ] Eliminate all of them (discarding all of the five works with the most nominations, and bad for all the authors, none of whom are given a chance)

NB A perfect slate is very unlikely to happen in practice, though it is theoretically possible. Whichever principle we settle on isn't just going to affect slates; it will mostly affect random ties of two works that happen to have the same number of points and nominations when they come up for elimination. Sometimes this will be two slate works, sometimes it will be two works that share a lot of fans in common but aren't part of an organised voting bloc, sometimes it will be two entirely unrelated works (to be potential finalists up for elimination, they must be sharing points with something else, but it might not be each other).

#906 ::: Keith "Kilo" Watt ::: (view all by) ::: May 23, 2015, 12:01 AM:

Felice@905:

That's a tough one. For me it's not so simple, because I don't think I can separate my desire to keep slates from running the ballot from the desire to have the most logical way to handle ties. And, as you mention, perfect slates are unlikely in the extreme and absolutely preventable (if it helps the slate to be perfect, you just need an anti-slate voter to nominate a slate work; if it hurts a slate to be perfect, then the slate organizers will make sure they aren't).

Kilo

#907 ::: felice ::: (view all by) ::: May 23, 2015, 01:25 AM:

Keith "Kilo" Watt @906: "That's a tough one. For me it's not so simple, because I don't think I can separate my desire to keep slates from running the ballot from the desire to have the most logical way to handle ties. And, as you mention, perfect slates are unlikely in the extreme and absolutely preventable (if it helps the slate to be perfect, you just need an anti-slate voter to nominate a slate work; if it hurts a slate to be perfect, then the slate organizers will make sure they aren't)."

Yes, it's tough; there are no ideal solutions. We can't rule out perfect slates; if it's organised in secret, the anti-slate voter doesn't know what to add. And in any case we can't know for sure that a voting bloc will have perfect discipline, so we can't know if we need to take measures to counter it. Convincing someone to nominate something they hate on the off chance it will prevent a perfect slate might be a challenge; realistically, several someones and somethings, because nominating just one slate work in a category only reduces the perfect slate to four (not much of an improvement), and we'd need to add a different number of nominations to each work (four people nominating 1-4 slate works each would be the minimum to ensure a tieless slate). And any such plan would have to be public to avoid needless and possibly counterproductive duplication, and this could be a source of rather bad publicity. And perfection can occur by accident (eg a lot of bloc voters leave one or two slate works off their individual ballot, but each work ends up on the same number of total ballots anyway). But none of the options I've listed lets a perfect slate ruin the ballot by excluding non-slate works entirely.

#908 ::: Soon Lee ::: (view all by) ::: May 23, 2015, 05:15 AM:

Darth Paradox #899: "...how often does a work that barely made it onto the ballot end up winning?"

It does happen though I can't say how regularly. Just last year, Best Fanzine went to "A Dribble of Ink" which had the least number of nominations of the finalists. I guess that's why on balance, I prefer to retaining ties somehow rather than eliminating both; to allow a work that was inadvertently overlooked a to rise to the top.

Felice #905 & Keith #906:
With ties, I would rather go with a logical approach where possible, and let the voters decide at the final ballot. We don't necessarily have to solve every problem perfectly (and it's not always possible anyway).

[ 8 ] Keep them all and expand the ballot to incorporate non-slate works as well (more finalists than ideal, but not unfair to anyone; and voters aren't forced to read the slate works)
[ 5 ] Use some arbitrary but repeatable method to select a subset of slate finalists (proportional but bad luck for the authors who miss out by chance - though at least they were given a chance)
[ 1 ] Eliminate all of them (discarding all of the five works with the most nominations, and bad for all the authors, none of whom are given a chance)

#909 ::: David Harmon ::: (view all by) ::: May 23, 2015, 07:14 AM:

The problem is that "slatedom" isn't an intrinsic, it's an emergent behavior. As felice points out, they can be made or unmade by accident as well as by collusion.

I think all of us can agree that keeping both of two tied works is really appealing... but keeping all of 4 or 5 tied works, not so much. And it's not clear that any algorithm can "naturally" handle multi-way ties without breaking one of our ground-rules or punishing natural ties.

I propose (or second) the composite approach:
1) In case of a two-way tie, both entries get kept, expanding the ballot if they are tied for last place.
2) In case of a three-or more-way tie, the election officials are explicitly given discretion to resolve the situation.
3) In case 2, the officials are explicitly given discretion to examine the voting patterns for slate-like voting, and to exclude that subset of the tied works which share the majority of their nominating ballots.
4) If case 3 is inapplicable, the officials may use their discretion to apply a tie-breaking method of their choice. This may include either or both of "internal" methods such as nomination timestamp or ballot order, "external" methods such as random choice, or the removal of all tied works.

#910 ::: David Harmon ::: (view all by) ::: May 23, 2015, 07:17 AM:

The problem is that "slatedom" isn't an intrinsic, it's an emergent behavior. As felice points out, they can be made or unmade by accident as well as by collusion.

I think all of us can agree that keeping both of two tied works is really appealing... but keeping all of 4 or 5 tied works, not so much. And it's not clear that any algorithm can "naturally" handle multi-way ties without breaking one of our ground-rules or punishing natural ties.

I propose (or second) the composite approach:
1) In case of a two-way tie, both entries get kept, expanding the ballot if they are tied for last place.
2) In case of a three-or more-way tie, the election officials are explicitly given discretion to resolve the situation.
3) In case 2, the officials are explicitly given discretion to examine the voting patterns for slate-like voting, and to exclude that subset of the tied works which share the majority of their nominating ballots.
4) If case 3 is inapplicable, the officials may use their discretion to apply a tie-breaking method of their choice. This may include either or both of "internal" methods such as nomination timestamp or ballot order, "external" methods such as random choice, or the removal of all tied works.

#911 ::: David Harmon ::: (view all by) ::: May 23, 2015, 07:22 AM:

Sorry for the double post.

I'll add that my "small fractions" idea would seem to strengthen the composite approach, by making natural ties that much less likely, without affecting (hiding) disciplined slates.

#912 ::: J Thomas ::: (view all by) ::: May 23, 2015, 08:12 AM:

@900 Keith "Kilo" Watt

"NB this is essentially the same measure that JT has been talking about."

Actually, it's not, at least not the way it's been coming across. On multiple occasions he's advocated specifically for more nominations as being the better measure. As mentioned above, points includes nominations, but nominations don't include points.

I was confused. I think at some point I accepted a wrong idea and then it confused me every time I thought about it.

I didn't and don't think nominations are the better measure. I said they were part of the measure, and other things equal, they should count.

Other things equal. Once we have a set of finalists, nominations=total score. For each ballot, divide it into as many equal pieces as there are finalists on it. Add the fractions to each finalist's score. Then add the scores back together. There is no difference between number of nominations and total score. Total score / nominations is one of the criteria for how good the solution is.

I don't know where it came from, but somehow I was thinking as if there was a difference between them.

Then there's votes. Given two different sets of finalists with the same total score, is it better to have more votes? Try an example.

A B C D E each have 40 votes with no overlap.

A B C F G , A B C each have 40 votes with no overlap, F and G have 80 FG nominations. The number of ballots is the same so the score must be the same (and is). F and G each have 80 votes, so the total number of votes is 280 instead of 200. Is that a better outcome? My first thought is to say yes. Other things equal, more support is better. The only difference is F and G have more nominators, which is a good thing. But consider this third choice:

H I J K L 200 ballots for HIJKL

This is a slate. The score is the same. Is it better to have a slate with the same score, rather than 5 nonslate items with the same score? Each finalist has a lot more support than the isolated ones do. But it's a slate. I'm not at all sure that it's better. I'm still confused.

I tend to think now that back when my old proposal got rejected on these grounds I should have pointed out that the other people were confused and wrong, that what they said was better was not in fact better at all, that it makes no more sense to decide the finalists by votes than by a moistened bint with a scimitar.... But they were so certain, somehow I believed them.

Back to other things to consider for what makes a good outcome:

M 180 votes
N 10 votes
O 5 votes
P 4 votes
Q 1 vote

No overlaps. This combination also has 200 nominations, but it isn't as good as ABCDE. Total score/nominations is not the only thing we care about. Sometimes we might want to accept a somewhat lower nomination count to get a more balanced nomination count.

SDV/LPE tends to solve this by eliminating low votes. Proposal 2a gives a loophole; high-vote works can rarely eliminate each other allowing pairs of 3rd-best works to slip in.

This is a good thing when it's surplus slate works that eliminate each other. Slates can attempt to avoid ties, but with thousands of nonslate voters a few of them may vote for slate items and throw off any slate planning. I think slates are probably as likely to get ties as anything else, which balances off some of the disadvantage of 2a.

Late double ties will be somewhat rare. Rarer still that they leave a worse finalist. (Often, late double ties will be bullet votes that cannot win, etc.) A worse finalist that's gotten to late-stage elimination will be pretty good.

So the argument for 2a is that we can accept the possibility that bad things can happen, in exchange for the possibility of good things -- both rare -- and we get a simple solution.

The argument for anything more complicated is that we reduce the already small chance of those particular bad things happening, while accepting a very small increase in the large chance that excess slate works win. And this is worth the complication.

#913 ::: J Thomas ::: (view all by) ::: May 23, 2015, 09:25 AM:

@905 felice

I think it might help to vote on how we prefer to handle perfect slates - the worst case scenario for ties. That would help decide on a general principle for dealing with all ties.

I tend to think that deciding how to handle the worst, very unlikely case is probably not a good way to decide on a general principle. But I'll go along.

[ 1 ] Keep them all and expand the ballot to incorporate non-slate works as well (more finalists than ideal, but not unfair to anyone; and voters aren't forced to read the slate works)

Five extra finalists, leaving 10? Not good. But politically it might be very good in the short run, you can give the puppies everything they say they want while still running the nominations as if they weren't there, and if one of their finalists wins the Hugo then so be it. What about later slates, whose politics we cannot predict? It may set a terrible precedent. The smofs can always change the rules later, though. But 10 finalists?

[ 5 ] Use some arbitrary but repeatable method to select a subset of slate finalists (proportional but bad luck for the authors who miss out by chance - though at least they were given a chance)

Reasonable and blah. If the detailed method is public, then slate voters can probably game it to get the ones they want, if they're willing to go to that trouble. Which is fine. If we're going to have a slate finalist it might as well be the one they think is best.

[ 10 ] Eliminate all of them (discarding all of the five works with the most nominations, and bad for all the authors, none of whom are given a chance)

They might not have all the most nominations, there might be 2 or 3 nonslate works that have more. Our mandate is to reduce the undue influence of slates. Even with SDV-LPE they will tend to get more than their share, and this would balance it a little bit if they actually tied.

It's worth considering partial ties. What if only two of them tie? If it's the two with lowest votes, it won't matter at all. Those two get eliminated and then the others have score around N/3. If the two with lowest votes were eliminated one at a time it would be the same.

If first one is eliminated for low votes and then two tie? Both eliminated, 2 left, probably both of them will be finalists. The only difference is that maybe there would be 3 slate finalists otherwise.

After two others are gone? Both eliminated, one left, the right outcome.

Only when the top two are tied can it unfairly hurt the slate. If the top two can eliminate all but three of the nonslate works first, they win anyway.

So 3/4 of the time it's a good or neutral outcome. In the 4th case the slate has no finalist, which does not balance the many times that a slate with less than 20% of the nominations has two or three.

#914 ::: Yarrow ::: (view all by) ::: May 23, 2015, 11:37 AM:

Back at 834, David Harmon said: I'm starting to think that ballot order really should be brought back in, even if it has to be specifically noted as relevant only in cases of voting ties.

I'm inclined to agree, after trying to wrap my head around the various tie-breaking proposals, and the arguments for and against. I'd like to propose a 2c:

2c. If the works are tied for both nominations and points, eliminate the work which is in the first position on the fewest ballots. If there is a tie for first position, eliminate the work which is in the second position on the fewest ballots, and so forth. If the works are completely tied, by nominations, points, and all positions, eliminate them all (or keep them all, if this is a tie for last place).

Advantages: it's simple to explain, and less arbitrary than date of nomination.

Disadvantages: nominators who don't realize that the order will rarely matter may agonize needlessly over the order of their nominations. Nominators who do realize that the order will rarely matter may not put their nominations in any order at all, making the tie-breaking more arbitrary than advertised.

The best-case situation would be when nominators realize that order might matter, and so put (say) their favorite work first, their two next favorites (which they can't choose between) next, in any order, and the rest after those, also in any order. Then, on the whole, we'll get enough information to make tie-breaking non-arbitrary, but nominators won't have to agonize.

#915 ::: David Harmon ::: (view all by) ::: May 23, 2015, 01:13 PM:

Yarrow #914: 2c. If the works are tied for both nominations and points, eliminate the work which is in the first position on the fewest ballots. If there is a tie for first position, eliminate the work which is in the second position...

Hmm. This makes "first position" disproportionately important in the tiebreak. Pragmatically, the instruction to the voter is "if there's one work you care most about, put it first", full stop. This is not necessarily a bad thing: I would have just summed the position numbers, but your way looks to me like a stronger symmetry breaker.

#916 ::: Keith "Kilo" Watt ::: (view all by) ::: May 23, 2015, 01:32 PM:

[GOODNESS]

(Sorry, that tag was too much fun to turn away...)

All:

I've been looking at some of the previous situations that seemed "pathological" to us, and I think felice's measure of "goodness" really shines a clear light on them. As a reminder, felice's contention is that the -total- number of points in the final round is something we should seek to maximize (yes, I recognize that in the absence of shared ballots, this will equal nominations, but 1) it's points that get you to the finals, and 2) it's entirely possible -- even likely -- for finalists to be on shared ballots). Under this system, the lowet total occurs with a slate running the finals, a better score occurs when the slate is completely eliminated, but an even better score occurs when the slate gets one nomination slot and the rest are non-slate works. This is explicitly what we are trying to achieve with SDV-LPE, so I think it works very well for us.

So, looking back at JT's 849 situation, we looked at two possible scenarios. If there is no tie breaker, then the winners in that situation were:

A = 350
B = 320
C = 310
G = 206
H = 206
Total: 1392

With a tie breaker of some sort that eliminates E, there winners were:

A = 350
B = 320
C = 310
D = 210
H = 206
Total: 1396

[Someone please check my math; I've been having trouble focusing the past couple of days.]

So, in this case, JT and felice are absolutely right: Breaking ties leads to a higher score, so is the better result. But look at how -much- higher the score is: 4 points out of almost 1400 points. That's a difference of around 0.3%. So while one situation is certianly better than the other, it's not -much- better than the other. For all intents and purposes, the two situations are very close to being the same.

This matches what we have beeen thinking: Tie breaking should make a difference, but it shouldn't make -much- difference. We now have a mathematical measure of how much difference it makes. In our previous analysis, we (including me) were comparing two works and trying to decide who "deserved" to be on the ballot. But JT's axiom (845) has pointed out to us that our responsibility is -not- to individual works, but to the process as a whole: E pluribus Hugo. So, felice is right, we have to look at the total and not concern ourselves with individual works.

This also matches Jameson's calulcations that ties matter in a tiny percentage of cases. There is no question that in the case given above, breaking ties is better than not breaking ties. To paraphrase what Jameson pointed out in 854, if we are willing to accept infinite complexity, then we can implement a scheme that will give the absolute maximum total score in the finalists. But at some point, you have to ask yourself: Is it worth it? There has to be a point of diminishing returns. Where that point lies, I think, is going to be different for everyone, and that's really where the tie-breaking debate lies.

Just thoughts for the morning,
Kilo

#917 ::: Keith "Kilo" Watt ::: (view all by) ::: May 23, 2015, 02:24 PM:

[TIES]

And my personal opinion on ties currently...

Linking ties has an intuitive appeal for me, so I would like to investigate how that works through sims. At the moment I don't know enough about its effects to have a good opinion yet.

But apart from that, the above analysis is convincing me that we should break ties when we can (linking resolves ties, but doesn't technically break them -- but that still may be okay). For this reason, I'm not really in favor of option 1 any more. I'm still not sure where the point of diminishing returns is regarding complexity, but I'm thinking that lookback (2b) is beyond that line for me, if not by much.

Using ballot order doesn't sit well with me, just because our philosophy was to throw names in a hat to be considered for nomination. Ranking nominations is antithetical to that philosophy. But is it really awful to abandon that philosophy? I'm not sure. Do the ends justify the means? At the very least it is non-arbitrary, as it would be completely under the control of the nominators, so I certainly am not as violently opposed to it as I'm am with arbitrary means. I'm not sure it will resolve all ties, though. At some point, I think we need to accept that the works really are just equivalent and stop trying to break the tie. Linking ties might be the answer to that (in which case the extra complexity may not be necessary).

Kilo

#918 ::: Yarrow ::: (view all by) ::: May 23, 2015, 03:13 PM:

Kilo@917: I'm not sure it [using ballot order] will resolve all ties, though.

It won't - given any numbers n1, n2, n3, n4, and n5, there could be situations where two (or more!) works are tied in points and nominations and where they each appear first on n1 ballots, second on n2 ballots, third on n3 ballots, fourth on n4 ballots, and fifth on n5 ballots. But that seems vanishingly unlikely.

#919 ::: Keith "Kilo" Watt ::: (view all by) ::: May 23, 2015, 03:20 PM:

Yarrow@918:

Yeah I agree, I think odds are pretty unlikely. In fact, maybe even just keeping the first place choice would be enough. People might not object to just ranking one choice.

K


#920 ::: Soon Lee ::: (view all by) ::: May 23, 2015, 05:57 PM:

I don't like the idea of ranking my nominations because I see the nomination stage of throwing names into a hat.

Ranking nominations would also be a bigger change to the current rule which may hurt its chances of success at the Business Meeting. (Until now, the proposal has been that Hugo voters nominate as they have done for years, that it is only the tallying of the nominations that we are seeking to change to mitigate the influence of bloc-voting.)

#921 ::: J Thomas ::: (view all by) ::: May 23, 2015, 06:26 PM:

@917 Keith "Kilo" Watt

Linking ties has an intuitive appeal for me, so I would like to investigate how that works through sims.

Here's a question about that.

When you link them, if they become finalists they're both finalists, right? And at that time, their value to the finalist list is their unshared votes.

So what about the votes they share with each other? How should those be counted for their score when you're still deciding whether to select them for elimination? Should it have any effect on their votes?

Try an example. E and F each have 130 votes, and thirty of them are EF votes. Of course their scores are tied and their votes are tied. You link them together and one way you count their score, it can be up to 130 because when they don't share ballots with anybody else, treating them as one does that. Another way to count their score, it's only 115 because they will share those 30 EF ballots as long as they're tied.

Say the votes are

E 99
EA 1
EF 30
F 99
FB 1

They are not a slate so they are only partly shared with each other, and they aren't shared with the same things otherwise. If A or B gets eliminated then for at least a brief time E and F won't be tied any more. There will be an opportunity for one of them to lose.

If they miss that chance they will be tied until they both lose or they both win.

And if they are instead a tied slate?

EF 130 votes

They start with score 65 and they beat everything they meet until they reach 5th place and tie there, unless something with more than 130 votes has a low enough score to match up with them and beat them. Ditto for a 3 way tie or a 4 way tie.

It drasticly interferes with elimination of slate works. (To the extent they tie.) The only way to stop a slate work is to match it against a nonslate work that has more votes. But nonslate works that have more votes might turn out to have scores too high to match against slates.

Under these rules it's clear what happens with multiple slates. The biggest slate wins everything, all smaller slates win nothing.

So unless a way is found to fix that problem I strongly prefer a voting system that either usually breaks ties or usually eliminates both.

I don't want to tell the puppies that I'm betting a whole lot that they can't make ties no matter how hard they try. I don't want an earful of cider.

http://quotationsbook.com/quote/39577/

#922 ::: J Thomas ::: (view all by) ::: May 23, 2015, 06:28 PM:

@917 Keith "Kilo" Watt

Linking ties has an intuitive appeal for me, so I would like to investigate how that works through sims.

Here's a question about that.

When you link them, if they become finalists they're both finalists, right? And at that time, their value to the finalist list is their unshared votes.

So what about the votes they share with each other? How should those be counted for their score when you're still deciding whether to select them for elimination? Should it have any effect on their votes?

Try an example. E and F each have 130 votes, and thirty of them are EF votes. Of course their scores are tied and their votes are tied. You link them together and one way you count their score, it can be up to 130 because when they don't share ballots with anybody else, treating them as one does that. Another way to count their score, it's only 115 because they will share those 30 EF ballots as long as they're tied.

Say the votes are

E 99
EA 1
EF 30
F 99
FB 1

They are not a slate so they are only partly shared with each other, and they aren't shared with the same things otherwise. If A or B gets eliminated then for at least a brief time E and F won't be tied any more. There will be an opportunity for one of them to lose.

If they miss that chance they will be tied until they both lose or they both win.

And if they are instead a tied slate?

EF 130 votes

They start with score 65 and they beat everything they meet until they reach 5th place and tie there, unless something with more than 130 votes has a low enough score to match up with them and beat them. Ditto for a 3 way tie or a 4 way tie.

It drasticly interferes with elimination of slate works. (To the extent they tie.) The only way to stop a slate work is to match it against a nonslate work that has more votes. But nonslate works that have more votes might turn out to have scores too high to match against slates.

Under these rules it's clear what happens with multiple slates. The biggest slate wins everything, all smaller slates win nothing.

So unless a way is found to fix that problem I strongly prefer a voting system that either usually breaks ties or usually eliminates both.

I don't want to tell the puppies that I'm betting a whole lot that they can't make ties no matter how hard they try. I don't want an earful of cider.

http://quotationsbook.com/quote/39577/

#923 ::: felice ::: (view all by) ::: May 23, 2015, 07:21 PM:

[GOODNESS]

Keith "Kilo" Watt @916: "So, looking back at JT's 849 situation, we looked at two possible scenarios... Breaking ties leads to a higher score, so is the better result. But look at how -much- higher the score is: 4 points out of almost 1400 points. That's a difference of around 0.3%."

But if we look at the #865 situation, breaking ties increases the score from 1208 to 1306 - an 8.1% improvement. And that's not the theoretical upper limit. The more the tied works overlap each other, the more difference it could make to the final score.

For the example below, tiebreaking AB gives an 18.5% increase in total score; and if we look at individual works, A would contribute 96% more points than G to the total score - A is a much better finalist. (Looks like a 20% increase in total score, and the tied work being twice as good as the one it replaces, would be the upper limits for the benefits of tiebreaking a single pair or works; breaking multiway ties, or breaking ties more than once, could produce an even greater nominal benefit.)

AB 100
C 55
D 54
E 53
F 52
G 51

CDEFG: 265 points; ACDEF: 314 points


[TIES]

Keith "Kilo" Watt @919: "Yeah I agree, I think odds are pretty unlikely. In fact, maybe even just keeping the first place choice would be enough. People might not object to just ranking one choice.

I'm pretty sure they will. I like ranking personally, but those who have an ideological objection to it are unlikely to be swayed by "it's only the first one". And I'm not keen on a last resort tiebreaker that isn't guaranteed to work.


J Thomas @913: "Five extra finalists, leaving 10? Not good."

Three or four extra finalists, depending on how many slots the slate takes up (1 or 2.5 are the obvious possibilities).

"But politically it might be very good in the short run, you can give the puppies everything they say they want while still running the nominations as if they weren't there, and if one of their finalists wins the Hugo then so be it. What about later slates, whose politics we cannot predict? It may set a terrible precedent."

Puppy finalists are extremely unlikely to win a Hugo with 3 or 4 non-puppy alternatives on the final ballot. A voting bloc with less offensive politics should find it literally impossible to deliberately create a perfect slate, because doing so relies on nobody outside the bloc nominating any of the slate works. Basically, the only way to intentionally create a perfect slate is to nominate works that can't possibly win (and even then, there's no way to be 100% certain nobody outside the bloc will nominate anything on the slate); a pretty futile exercise, I think.

#924 ::: J Thomas ::: (view all by) ::: May 23, 2015, 07:45 PM:

@923 felice

"Five extra finalists, leaving 10? Not good."

Three or four extra finalists, depending on how many slots the slate takes up (1 or 2.5 are the obvious possibilities).

I think I see. Four is the maximum, because it would already have one. Or two or three.

"... and if one of their finalists wins the Hugo then so be it. What about later slates, whose politics we cannot predict?

Puppy finalists are extremely unlikely to win a Hugo with 3 or 4 non-puppy alternatives on the final ballot.

Agreed, and if they do somehow manage that then so be it.

A voting bloc with less offensive politics should find it literally impossible to deliberately create a perfect slate, because doing so relies on nobody outside the bloc nominating any of the slate works.

That is beside the point. We're talking about a complicated proposal to handle a rare circumstance. But in the rare circumstance that two or more slate works accidentally wind up tied, the slate problem is not handled.

Important accidental nonslate ties should be very rare. Accidental slate ties will be much more common (but still rare) if people keep making slates.

So if we're going take a lot of trouble designing a system to handle ties, it has to give good results for slate ties. I don't want you to just say that slate ties will be rare so don't worry about them. If we aren't worried about events that rare, then we don't need to worry about combined vote/score ties at all.

#925 ::: Keith "Kilo" Watt ::: (view all by) ::: May 23, 2015, 07:55 PM:

Felice@923:

Good analysis, and 20% looks reasonable as a theoretical maximum. It may not be a coincidence that's roughly equal to one slot's worth of points in a balanced set of finalists. I do want to remind us both that we are assuming no shared ballots among finalists in the end, but in the real world that's a virtual certainty. Still, while we're probing generalities, I think it's fine. At any rate, I think it showcases the utility of using total points as a measure. The question remains as to how much complexity we'd accept to reach that theoretical maximum in the unusual cases where the opportunity presents itself, especially since we aren't likely to need to approach it with real world data. Still, I think you're right that it's good enough to abandon option 1 (no tie break) at least.


I agree that perfect slate discipline is a virtual impossibility. It's why I set that as ground rule earlier.


You may be right about how people feel about ranking. It still kind of leaves a bad taste even for me. I really don't want to change how we do the nomination stage unless we just absolutely have to. And I'm not sure that we do... Still, unlike arbitrary methods, I'm not so uncomfortable with it that I just draw the line against it.

Kilo

#926 ::: felice ::: (view all by) ::: May 23, 2015, 08:46 PM:

Keith "Kilo" Watt @925: "Good analysis, and 20% looks reasonable as a theoretical maximum. It may not be a coincidence that's roughly equal to one slot's worth of points in a balanced set of finalists. I do want to remind us both that we are assuming no shared ballots among finalists in the end, but in the real world that's a virtual certainty."

If there are overlaps between the non-tied finalists, their contribution to the total score will be reduced. Though that also means their points in the round where the tied works are eliminated or the tie is broken will be lower; overlaps while keeping their earlier points the same can only be achieved by increasing total nominations. If there are overlaps amongst C, D, E, and F such that their collective contribution to the total score remains at 214 points, then the advantage of A over G is unchanged.


J Thomas @924: "That is beside the point. We're talking about a complicated proposal to handle a rare circumstance. But in the rare circumstance that two or more slate works accidentally wind up tied, the slate problem is not handled."

In which case I'm not sure what the point is. What is the terrible precedent you're talking about? Why do you say the slate problem is not handled?

"So if we're going take a lot of trouble designing a system to handle ties, it has to give good results for slate ties."

And what is a good result for slate ties? Figuring out that is the point of the poll I started (with three votes in, the current scores are Keep and expand: 5.33; Arbitrary subset: 5.33; Eliminate all: 4).

#927 ::: J Thomas ::: (view all by) ::: May 23, 2015, 10:12 PM:

@925 Keith "Kilo" Watt

Good analysis, and 20% looks reasonable as a theoretical maximum. .... Still, I think you're right that it's good enough to abandon option 1 (no tie break) at least.

Try this one:

I'll need more letters, so I'll use lower case sometimes.

ACDEF 10 ballots
Aabcd 10 ballots
Aefgh 10 ballots
Aijkl 10 ballots
Amnop 10 ballots

Bqrst 10 ballots
Buvwx 10 ballots
BGHIJ 10 ballots
BKLMN 10 ballots
BOPQR 9 ballots

a-x, G- R 9 bullet votes each.
C-F 10 bullet votes each.

First round: A has score 10. B has score 9.8 Everything else has score >= 10.8. B is eliminated.

Finalists: ACDEF, total ballots: 90

If we could find a way to save B, then the total ballots would be 130. We lose 30% of the total score because we must follow SDV-LPE which gets an imperfect result this particular time.

We don't worry about that because we don't have a simple way to always get a good final score. SDV-LPE gets a good result consistently, and every now and then it might miss out on a big score so it only gets a good result instead of a great one.

When we don't worry about always getting within 30% of the best score with SDV-LPE, why worry about always getting within 20% of the best score when we handle ties?

#928 ::: J Thomas ::: (view all by) ::: May 23, 2015, 10:53 PM:

@925 Keith "Kilo" Watt

I agree that perfect slate discipline is a virtual impossibility. It's why I set that as ground rule earlier.

People who run slates can't reliably create a 5-way tie.

But even if they try to avoid ties, they can't reliably avoid them either.

Here is why I think slate ties are more probable than nonslate ties:

A slate attempts to maximize votes for each work. Even if it includes conscious attempts to avoid ties, it will still involve five works with very similar numbers of votes. There is nothing more likely to get at least a two-way tie in votes.

Nonslate ballots tend to be variable. The average number of votes per ballot in one recent case was 3.5. Remove votes for hopeless cases and that drops to 1.5 votes per ballot for works in the top 15. But most slate ballots will have 5 votes. There is less variability in number of votes, and so more chance for a tie.

We can argue that we don't need to think about slate ties because they can't make ties on purpose. But there is absolutely no way to make nonslate ties on purpose, and we're putting a lot of effort into those.

So slate ties are important relative to other ties. If you're going to do something special about ties, make sure it gets the result you want for slates.

I consider it a bad thing for a slate tie to result in two or more slate finalists. Given the way nonslate voters nominate, a slate that has 15% as many voters as there are nonslate voters, will have maybe 2, possibly 3 nonslate candidates that get more votes in the best cases, and may have more votes than any nonslate work in some cases. A slate that has 20% as many voters as there are nonslate voters, is likely to have more votes than any nonslate candidate in all categories.

We are forbidden to do anything to get more votes for leading nonslate candidates.

So it's already likely that a slate the size of this year's, will get 2 finalists and possibly 3. A method to handle ties that results in tied slates both on the ballot, makes it worse. Not much worse because there won't be many tied slates. But when there are likely to be more tied slates than anything else (and slatemasters see that ties are good for them so they do nothing to try to prevent them), getting a good result for nonslate ties looks to me like a lower priority than getting a good result for tied slates. It will probably happen less often, and the results of a failure are milder. If we accidentally remove a worthy candidate, that's bad but likely the next-best will replace it. If we fail to eliminate a slate work that shouldn't be a finalist, that's an extra slate finalist -- which encourages them. And replacing a worthy nonslate work is even worse.

#929 ::: J Thomas ::: (view all by) ::: May 23, 2015, 10:55 PM:

@925 Keith "Kilo" Watt

I agree that perfect slate discipline is a virtual impossibility. It's why I set that as ground rule earlier.

People who run slates can't reliably create a 5-way tie.

But even if they try to avoid ties, they can't reliably avoid them either.

Here is why I think slate ties are more probable than nonslate ties:

A slate attempts to maximize votes for each work. Even if it includes conscious attempts to avoid ties, it will still involve five works with very similar numbers of votes. There is nothing more likely to get at least a two-way tie in votes.

Nonslate ballots tend to be variable. The average number of votes per ballot in one recent case was 3.5. Remove votes for hopeless cases and that drops to 1.5 votes per ballot for works in the top 15. But most slate ballots will have 5 votes. There is less variability in number of votes, and so more chance for a tie.

We can argue that we don't need to think about slate ties because they can't make ties on purpose. But there is absolutely no way to make nonslate ties on purpose, and we're putting a lot of effort into those.

So slate ties are important relative to other ties. If you're going to do something special about ties, make sure it gets the result you want for slates.

I consider it a bad thing for a slate tie to result in two or more slate finalists. Given the way nonslate voters nominate, a slate that has 15% as many voters as there are nonslate voters, will have maybe 2, possibly 3 nonslate candidates that get more votes in the best cases, and may have more votes than any nonslate work in some cases. A slate that has 20% as many voters as there are nonslate voters, is likely to have more votes than any nonslate candidate in all categories.

We are forbidden to do anything to get more votes for leading nonslate candidates.

So it's already likely that a slate the size of this year's, will get 2 finalists and possibly 3. A method to handle ties that results in tied slates both on the ballot, makes it worse. Not much worse because there won't be many tied slates. But when there are likely to be more tied slates than anything else (and slatemasters see that ties are good for them so they do nothing to try to prevent them), getting a good result for nonslate ties looks to me like a lower priority than getting a good result for tied slates. It will probably happen less often, and the results of a failure are milder. If we accidentally remove a worthy candidate, that's bad but likely the next-best will replace it. If we fail to eliminate a slate work that shouldn't be a finalist, that's an extra slate finalist -- which encourages them. And replacing a worthy nonslate work is even worse.

#930 ::: felice ::: (view all by) ::: May 23, 2015, 11:57 PM:

J Thomas @927: "First round: A has score 10. B has score 9.8 Everything else has score >= 10.8. B is eliminated."

That requires a 32-way tie for 7th most nominations, and a 4-way tie for 39th, and a 4-way tie for 3rd. And simultaneous eliminations of two 16 way ties for points and nominations, and a 4-way tie for points and nominations. And these are all works a single bullet-vote away from being finalists, not no-hopers. That's... rather less probable than a perfect slate.

J Thomas @928: "A slate attempts to maximize votes for each work. Even if it includes conscious attempts to avoid ties, it will still involve five works with very similar numbers of votes. There is nothing more likely to get at least a two-way tie in votes."

Though on the other hand, a large pool of nominators is less likely to come up with identical results than a small pool. Ask two people to pick a number between 30 and 40, and you'll get the same answer a lot more often than "pick a number between 300 and 400".

"I consider it a bad thing for a slate tie to result in two or more slate finalists."

Even when they're not pushing other works off the final ballot? We're not getting any fewer non-slate finalists than we would without a slate tie, so to me that's a pretty insignificant bad. Nobody is suggesting keeping slate ties without extending the ballot.

#931 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 01:52 AM:

[CODE]

Just a quick update on the code...

I have finished coding the linked ties case and am testing it. So far, it looks good, though I had to make one call that may or may not be as intended:

If a ballot only has one work on it and that ballot is part of a two-member tie, then the number of points is divided by 1/2 (i.e., multiplied by 2), so points actually increase due to being part of a linked tie in this case. This was sort of an oddball case, but kind of makes sense if you want to give a last-ditch chance to that particular ballot. I can easily change it so that they never have fewer than one work on any given ballot if that's how you want it done. I don't think it matters a great deal. Just let me know.


At any rate, I ran the 1984 Best Novel data and came out with exactly the same results as method 2a (no lookback). In all cases (no exceptions), the linked ties were eliminated on the very next round. All ties were eliminated by round 25 out of 84 rounds.

As mentioned, though, we are worrying about edge cases that won't normally come up in real data, so we don't really expect to see a difference here. So, if someone wants to send me a ballot set (here or via email), let me know and I'll run it. I need some test cases to explore since I doubt the 1984 data will show anything new -- it's just too "normal". :)

Still, very pleased to have this version coded. I now have options 1, 2a, and linked ties available for comparison. 2b would require a fairly major change, though I did add a "history array" that tracks the points and nominations of each work at each round. That was with an eye towards implementing lookback if there is enough support to make it worth doing. It's a lot of work, so I don't plan to code it unless people are really wanting it. I can, though.

Let me know what you want to test so that we can make some decisions!

Thanks,
Kilo

#932 ::: junego ::: (view all by) ::: May 24, 2015, 02:32 AM:

Soon Lee @ 920
I don't like the idea of ranking my nominations because I see the nomination stage of throwing names into a hat.
Ranking nominations would also be a bigger change to the current rule which may hurt its chances of success at the Business Meeting. (Until now, the proposal has been that Hugo voters nominate as they have done for years, that it is only the tallying of the nominations that we are seeking to change to mitigate the influence of bloc-voting.)

I'm with Soon Lee on this. NOT changing what nominators do is a feature of the proposal and a positive selling point. I wouldn't want to give it up without a very compelling reason...this doesn't rise to that level, imho.

#933 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 03:28 AM:

[CODE]

It just occurred to me I may have a misconception. With linked ties, is the intent to compare point totals that each of the linked members have (they will be the same) or to add up all their points together?

I bet it's the latter, but I coded the former. I'll try it the other way tomorrow, but if someone could let me know for sure, I want to be sure I have the definition of the system right.

Thanks,
K

#934 ::: junego ::: (view all by) ::: May 24, 2015, 03:55 AM:

Keith "Kilo" @ 931
[CODE]

So, if someone wants to send me a ballot set (here or via email), let me know and I'll run it. I need some test cases to explore since I doubt the 1984 data will show anything new -- it's just too "normal". :)
...
Let me know what you want to test so that we can make some decisions!

Take the 1984 data:
1) add a 10% increase in votes...a slate that may somewhat mirror what happened this year in terms of increased participation due to campaign...that chooses the 5 lowest scoring novels from the original data. This will simulate non-slate voters nominating slate works coincidentally.
2) Make the slate 75% perfect discipline...because perfect isn't probable.
3) the other 25% of slate votes only choose 3 or 4 of the slate and fill in the rest of their ballot with one of the actual finalists. (Your call on easiest or best way to do this.)
4) Run with the different tie breaker scenarios.

What happens wrt ties and finalists?

#935 ::: J Thomas ::: (view all by) ::: May 24, 2015, 06:52 AM:

@930 felice

"First round: A has score 10. B has score 9.8 Everything else has score >= 10.8. B is eliminated."

That requires a 32-way tie for 7th most nominations, and a 4-way tie for 39th, and a 4-way tie for 3rd.

That's just how I did it. It helped me write it more compactly. I don't know how common it is, I just wanted to demonstrate that SDV-LPE can get results that are at least 30% away from optimal. I think with more work I could get close to 60%.

And I can make most of the scaffolding works be no-hopers, but it takes bigger numbers. Of course in reality we have thousands of ballots, but we aren't trying to make these real.

"A slate attempts to maximize votes for each work. Even if it includes conscious attempts to avoid ties, it will still involve five works with very similar numbers of votes. There is nothing more likely to get at least a two-way tie in votes."

Though on the other hand, a large pool of nominators is less likely to come up with identical results than a small pool. Ask two people to pick a number between 30 and 40, and you'll get the same answer a lot more often than "pick a number between 300 and 400".

Sure, but we aren't interested in ties between no-hopers. It's true that non-slate finalists will not have as many votes as slate finalists which makes it easier for them to tie. But on the other hand there will never be more than 4 nonslate finalists that can tie, while there are always 5 slate nominalists, so they still get more chances.

"I consider it a bad thing for a slate tie to result in two or more slate finalists."

Even when they're not pushing other works off the final ballot? We're not getting any fewer non-slate finalists than we would without a slate tie, so to me that's a pretty insignificant bad.

To me it's still significant, though less bad than pushing the others off entirely. But it's a philosophical point and not a deal-breaker.

Theoretically, it's bad to be extra nice to slates because we don't want to encourage that behavior. Slates work because the nonslate votes are so scattered, and there's nothing we can do about that scatter. Slates will get finalists out of proportion to their number. When a slate tie -- more likely than other ties -- gets special extra finalists because it's a tie, that's an extra lagniappe. It isn't really a significant increase because ties are rare. But this method to handle ties does more to reward slates than it does anything else. I don't like to have a complication in the voting whose main effect is to reward slates.

But practically it doesn't matter, and if people think it's the fairest way then I'll go along.

#936 ::: David Harmon ::: (view all by) ::: May 24, 2015, 08:42 AM:

J Thomas #935: The important question isn't whether a configuration is possible -- we already know that any non-trivial voting system can have its goals foiled by worst-case ballot sets. Actually constructing those sets doesn't add anything unless they're actually plausible in our voting conditions.

Something like a 1% chance is plausible (says the former RPG DM). So is anything can be set up on purpose, by by a group of hostile voters. But a four-way tie, then a 32-way tie, then another 4-way tie, that doesn't look to me like something we could see in the real world.

#937 ::: J Thomas ::: (view all by) ::: May 24, 2015, 10:32 AM:

@936 David Harmon

Something like a 1% chance is plausible (says the former RPG DM). So is anything can be set up on purpose, by by a group of hostile voters. But a four-way tie, then a 32-way tie, then another 4-way tie, that doesn't look to me like something we could see in the real world.

Sure, but there's nothing about the example that depends on those being ties. I did it that way because it took less typing. It doesn't mean anything. Will you believe it doesn't depend on ties, or do I need to build an example that doesn't have ties?

The important question isn't whether a configuration is possible -- we already know that any non-trivial voting system can have its goals foiled by worst-case ballot sets.

I gave an example where the second-strongest candidate was defeated on the first ballot, replaced by something much weaker.

You can't get that result with approval voting. The problem with approval voting is that nominations with the most votes can win when we don't want them to. Some people consider it a problem that they can't rank their votes, another possible problem. It depends on what the goals are.

We agree on at least some of what we want. SDV-LPE is pretty good at getting that, and as far as I know it never gets a terrible result. It can get some results that are only OK.

For ties, if we eliminate both we usually get what we want, and it's possible sometimes to get results that are only OK. For slate ties it gets a good result. If slate voters realize this and go to extra trouble to try to avoid slates, that is not a bad thing.

I don't see that the possibility of rare only-OK results is reason to give it up, when those rare only-OK results are no worse than the rare only-OK results you can get from SDV-LPE without ties.

Something like a 1% chance is plausible (says the former RPG DM). So is anything can be set up on purpose, by by a group of hostile voters.

We know ties on votes can happen more than 1%. Ties on votes and scores both will be rarer. Ties on votes and scores where the tied ones could either be finalists are rarer still. Ties on votes and scores where the tied ones can be finalists and the ones that would replace them are considerably worse, are rarer than that.

We think that nobody can intentionally engineer a tie, slates or otherwise.

#938 ::: David Harmon ::: (view all by) ::: May 24, 2015, 11:49 AM:

Will you believe it doesn't depend on ties, or do I need to build an example that doesn't have ties?

That might possibly be helpful, but really, you need to build an example that could plausibly show up in the voting. It would help if you could specify the more particular conditions that produce the "bad" result, so that someone who's good at combinatorics could figure out if they actually have a probability above epsilon.

More fundamentally, I think focusing on contrived cases is "heading off into the weeds". We do not need to specify rules for completely determining ties without human intervention. If we've established that (point+ballot) ties are scarce, important ties are scarcer, and a slate can't force them onto the ballot, then leaving such occasional cases to the administrators isn't an omission in the system, it's a safety valve!

Worse, attempting to specify "formally complete" rules is adding complexity, delaying completion and distracting from the original goals, of the overall system. It's almost a textbook case of making the perfect the enemy of the good.

#939 ::: J Thomas ::: (view all by) ::: May 24, 2015, 01:27 PM:

@938 David Harmon

"Will you believe it doesn't depend on ties, or do I need to build an example that doesn't have ties?"

That might possibly be helpful, but really, you need to build an example that could plausibly show up in the voting.

So it should involve thousands of votes? If we're mostly interested in potential finalists, with 3000 or so total ballots and winners that get 5% to 20% of them, that's winners with 150 to 600 votes, and ties are pretty unlikely. But we want a solution that's morally right, that is not unfair to anybody we don't want to be unfair to.

If we're only interested in situations that are actually likely to happen, then solution 1 -- on a voting tie, eliminate both -- is just fine. It's dead simple -- eliminate the two lowest when you can't tell them apart. It's just fine.

Or if you are a little uneasy about that and want to make ties even rarer, 2a is just fine. They aren't tied unless they're tied on votes and scores both. In the very rare case that they're tied on both, eliminate both. Still very simple and the possible results which aren't optimal but still OK are even rarer.

Or we could find some method to choose one, and that will be OK in practice too. You might feel like it's fairer than eliminating both, or less fair, but practically it's just fine.

Or we could leave these very unusual cases for the administrators. To me that doesn't look as good because it admits we don't know what we're doing, and it looks like it's leaving a potential hot potato for somebody else to hold, but it should have no practical consequences once the business meeting passes it.

I don't have a grasp on the combinatorics and I haven't seen that anybody else here does. There's the question how long we'll use this system before it gets replaced by something else. If it goes long enough to become a tradition that's hard to break, then any flaw will eventually show up. Maybe after enough years that we won't care any more.

So if we get a solution we say is morally good then we can use that excuse if by chance it actually happens and gets a result people claim is bad. It's morally right and the way it's supposed to happen. That's what we're planning to do if we actually get a result where the #2, #3, #4, and #5 works with the most votes don't win. (Which should be extremely rare.) The administrators will publish the statistics:

Top 15 nominees:

A 560
B 220
C 210
D 205
E 202
------
F 403
G 397
H 359
I 312
J 115
K 108
L 101
M 96
N 87

We will say that this was the right result because of how the votes were distributed. That "Biggest votes is best votes" is not true.

If we can say that it's the morally right result, the we don't have to work out how often it happens, because however often that is, is OK. It's only when we're trying to justify a result we think is bad that we need to argue that it doesn't happen very often.

#940 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 01:30 PM:

David@938:

It would help if you could specify the more particular conditions that produce the "bad" result, so that someone who's good at combinatorics could figure out if they actually have a probability above epsilon.

Agree 100%. Generating situations is great, but only to the extent that they let us figure out the conditions in which they occur. Without those conditions we've got nothing to base decisions on, and the situation ends up not telling us anything useful.

Kilo

#941 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 02:03 PM:

[TIES]

Something else to keep in mind:

In some cases, there -isn't- a morally right or wrong answer, it's simply the rules of the game. Consider baseball as an example. Let's say the batter's first two pitches are foul balls. That's two strikes. Three strikes, and he is out. On his third pitch, he hits another foul ball. Is he out? No, actually he's not. Is that morally wrong? "Should" he have been out? In this case, there's not really a "right" answer, but as long as the rules seem reasonable to everyone, we accept it and go on.

Now, I'm not saying we should just say "These are the rules of the Hugo, accept them or not." But like in baseball, it's possible there are some cases where we get two results that are different (he's out or he's not), but aren't clearly right or wrong. These edge-case ties might be like that. Felice's analysis shows that the efect of ties -- even in the absolute worst case -- are not negligible, but they also can't be overwhelming. At most, we are talking about whether one work or another makes the 5th slot. Given that this worst case scenario isn't going to be common, how much complexity are we willing to tolerate? Could it be that in this case, there's not a "right" answer? One guy is out and one guy isn't. If they weren't razor-close it would never have been an issue. To a certain extent, it's a third foul ball.

So, the bottom line is that we need a system that makes intuitive sense. In baseball, for whatever reason, people don't want a batter to go out on a fly ball. With the Hugo, maybe it needs to be something like "Eliminate that last-ditch tie since they are about equally popular and neither is very popular" or "Link them together an see if they are stronger as a unit". I'm just not seeing in any test that I've done that it makes an overwhelming difference whatever system we choose, even in the rare cases that some difference was present.


And speaking of, I re-coded the linked ties sim, and while some works held on a bit longer, the end result was still the same. Looking at the description I just wrote ("Link them together an see if they are stronger as a unit"), I'm now inclined to say, "Why should we?" There's nothing about these works (unless they are part of a slate) that intrinsically binds them together. It doesn't change anything in realistic data. Maybe it's just complexity for its own sake.

So, based on the above, I'm ready to put out my support for a tie-breaking system: I say let's choose option 2a: If two works are tied for nominations, take another look at points and eliminate the lower-point work. If they are tied for both points and nominations, then they are essentially equally the least popular -- so eliminate them both.

I fully understand the arguments against double elimination, honestly I do. But I just don't think it makes -enough- of a difference to complicate the system any more. I think this is my personal point of dimenishing returns.

Who has strong feelings against 2a? I know there are some, based on our earlier discussions. How much of a deal-breaker is it for you? Is there a chance we can agree on this system and be done?

Please let us know,
Kilo

#942 ::: David Harmon ::: (view all by) ::: May 24, 2015, 02:44 PM:

J Thomas #939: But we want a solution that's morally right, that is not unfair to anybody we don't want to be unfair to.

The experts in this and previous threads have assured us that no system can be perfectly "fair" -- we have some choice about where the flaws end up, but any system we make will have some flaws. Trying to figure a way to get around that dictum is exactly where "making the perfect the enemy of the good" comes in.

Or we could leave these very unusual cases for the administrators. To me that doesn't look as good because it admits we don't know what we're doing, and it looks like it's leaving a potential hot potato for somebody else to hold,

On the contrary, it says that we collectively do know what we're doing, and especially that we know our limitations. It also says that we are willing to trust those officials (and the groups that provide them) to judge the tough calls, instead of trying to create beforehand "one thing to bind them all".

#943 ::: junego ::: (view all by) ::: May 24, 2015, 03:14 PM:

Kilo @ 941

I fully understand the arguments against double elimination, honestly I do. But I just don't think it makes -enough- of a difference to complicate the system any more. I think this is my personal point of dimenishing returns.
Who has strong feelings against 2a? I know there are some, based on our earlier discussions. How much of a deal-breaker is it for you? Is there a chance we can agree on this system and be done?

Based on your report of the sims you've run showing that a) although breaking a tie may keep the winner alive a bit longer it didn't change your final results, b) such ties in 5th place will retain all candidates, and c) such double ties may favor slates slightly, I'm amenable to go with 2a as the KISS solution (unless someone comes up with a killer objection, which we haven't seen yet).

#944 ::: J Thomas ::: (view all by) ::: May 24, 2015, 03:17 PM:

@942 David Harmon

"Or we could leave these very unusual cases for the administrators. To me that doesn't look as good because it admits we don't know what we're doing, and it looks like it's leaving a potential hot potato for somebody else to hold,"

On the contrary, it says that we collectively do know what we're doing, and especially that we know our limitations. It also says that we are willing to trust those officials (and the groups that provide them) to judge the tough calls, instead of trying to create beforehand "one thing to bind them all".

That's a good point. I don't have much opinion about it myself, I'm considering what I expect the business meeting to think -- which I don't know. So if you think that's a good approach I won't argue much, because your opinion about this is at least as good as mine.

#945 ::: junego ::: (view all by) ::: May 24, 2015, 03:22 PM:

Kilo

[SIMS]

Hey, did you run the sim I suggested, or was it similar to others you had already tried? I remember you said you previously ran slates with perfect discipline and don't remember you saying you had run any with undisciplined slates. If you already ran a similar scenario with the same results, that's OK, then.

#946 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 03:52 PM:

Junego@945:

Which sim was it you wanted me to run? I don't recall. I'm glad to run any sim in any system I've coded, just let me know. In 2a, an undisciplined slate tended to converge to the most popular slate work, which then made the final ballot. Was that the case you were looking for?

K

#947 ::: felice ::: (view all by) ::: May 24, 2015, 06:14 PM:

David Harmon @938: It would help if you could specify the more particular conditions that produce the "bad" result, so that someone who's good at combinatorics could figure out if they actually have a probability above epsilon."

I'm pretty confident the conditions in question are the top two works being almost equal to each other but sharing virtually all their ballots with a wide range of other popular works, and a lot of those popular works having a lot of bullet votes (or sharing ballots only with less popular works that are eliminated early), and virtually no overlap between the top two works, and virtually no overlap between any of the various popular works, and the top two works being much more popular than the next tier of popular works. The least plausible bit is I think that while about half of fans like one (and only one) of the top two, and another half like exactly one very popular work but not either of the top two (or any of the other popular works), virtually nobody likes one of the top two but fewer than four of the other popular works. It looks pretty epsilonic to me.


David Harmon @942: "On the contrary, it says that we collectively do know what we're doing, and especially that we know our limitations. It also says that we are willing to trust those officials (and the groups that provide them) to judge the tough calls, instead of trying to create beforehand "one thing to bind them all"."

The administrators won't have a chance to judge the tough calls. If we go with simultaneous elimination, then all the tied works will just get eliminated, and the admins will have no reason to suspect there's a problem even if this does cause a bad result (except possibly after the extended nomination data is released and fans start complaining). Or if the admins do notice in advance and agree it needs correction, what can they do and how can they justify it? If they do make changes to the finalists, surely fans will either object to the changes or conclude SDV-LPE is broken? Or more likely both.

J Thomas @935: "And I can make most of the scaffolding works be no-hopers, but it takes bigger numbers."

I'm not sure you can. A and B must have points no less than 1/5 of their nominations, and any scaffolding with fewer points than that will get eliminated first. And the scaffolding must overlap with A or B, which reduces their points relative to their nominations; so their nominations must all be more than 1/5 of A's.


[TIES]

The ties that matter are essentially overlapping works that under the current system would both be finalists. Under SDV-LPE, because they overlap, one should be eliminated to create a more representative set of finalists. But if they tie, and we eliminate both, we get a bigger difference from the results of the current system, and if they're not slate works, it's a worse result. Ties on nominations between non-slate finalists happen multiple times every year (on average); significant overlap and tying on points as well would be less common, but it will happen (there aren't many other works left to split points with).


Anyone else want to vote in this? Kilo, you seem to have indicated a preference for eliminate all, and a particular aversion to arbitrary; want to put numbers on that?

[ ] Keep them all and expand the ballot to incorporate non-slate works as well (more finalists than ideal, but not unfair to anyone; and voters aren't forced to read the slate works)
[ ] Use some arbitrary but repeatable method to select a subset of slate finalists (proportional but bad luck for the authors who miss out by chance - though at least they were given a chance)
[ ] Eliminate all of them (discarding all of the five works with the most nominations, and bad for all the authors, none of whom are given a chance)


Keith "Kilo" Watt @941: "It doesn't change anything in realistic data."

We don't have realistic data. 1984 and 21st century nomination distributions aren't the same.

[CODE]

Would you be willing to try implementing the Probation (formerly Ghost) option?

#948 ::: J Thomas ::: (view all by) ::: May 24, 2015, 07:13 PM:

@947 felice

"It would help if you could specify the more particular conditions that produce the "bad" result, so that someone who's good at combinatorics could figure out if they actually have a probability above epsilon."

I'm pretty confident the conditions in question are the top two works being almost equal to each other

That maximizes the points lost when A beats B. You can get a significant loss without it.

but sharing virtually all their ballots with a wide range of other popular works,

The more others they share with, the earlier they can compete and eliminate B. It doesn't have to be so many.

and a lot of those popular works having a lot of bullet votes (or sharing ballots only with less popular works that are eliminated early),

That part is necessary, the big boys need shared votes to bring down their scores, but some of the ones they share with need a way to keep their scores high or they'll be eliminated first.

and virtually no overlap between the top two works,

That is not necessary at all, it just made it easier for me to track the details. It's fine for them to share a lot.

and virtually no overlap between any of the various popular works,

The works that A and B share with need a way to keep their scores high. Too much overlap would reduce their scores, requiring more nominations to bring their scores up. It's a gradient, the effects are somewhat less when they overlap a lot.

and the top two works being much more popular than the next tier of popular works.

That's to make the result bigger. If #6 was almost as good as #2, then losing #2 wouldn't look so bad. When #3, #4, #5 are low then the points lost with B look like a bigger deal.

We can have a sizeable difference in total scores while relaxing most of your criteria.

"And I can make most of the scaffolding works be no-hopers, but it takes bigger numbers."

I'm not sure you can. A and B must have points no less than 1/5 of their nominations, and any scaffolding with fewer points than that will get eliminated first. And the scaffolding must overlap with A or B, which reduces their points relative to their nominations; so their nominations must all be more than 1/5 of A's.

Yes, but it's easy to get things with scores more than 1/5 of A's that have no hope to win. In real data, pretty often the winner is at say 20% of votes and each of the top 15 by votes have 10% or more, certainly more than 5%.

Anyway, the point of SDV-LPE is not just to maximize total score. If it was, we could just maximize total score and not worry about the details of SDV-LPE. We have some other criteria too, and we can't expect SDV-LPE to absolutely maximize one of them all the time.

And by the same token, no point rejecting a method to handle ties just because it doesn't always maximize the same criterion that SDV-LPE doesn't always maximize.

#949 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 07:16 PM:

Felice@947:

A clarification: I don't actually have a preference for eliminate all, since I still don't think perfect slate handling is a valid way for me to choose between options. There are other factors that are much, much more important, particularly since I consider perfect slates to be an effective impossibility. I ultimately don't care about any of the options in the polls since they don't inform my decision. That makes it basically impossible for me to rank them.

Regarding ghosts, I think for me this goes past the diminishing returns complexity line. It's hard to explain conceptually (and even harder to code as things begin to bifurcate), and I think that will hurt its chance at passage.

Kilo

#950 ::: P J Evans ::: (view all by) ::: May 24, 2015, 07:21 PM:

941
Works for me.

#951 ::: felice ::: (view all by) ::: May 24, 2015, 08:18 PM:

Keith "Kilo" Watt @949: "A clarification: I don't actually have a preference for eliminate all, since I still don't think perfect slate handling is a valid way for me to choose between options."

Two-way perfect ties between works with enough nominations to both be finalists under the current system, then? The choices are the same.

"Regarding ghosts, I think for me this goes past the diminishing returns complexity line. It's hard to explain conceptually (and even harder to code as things begin to bifurcate)"

There's no bifurcation with Ghosts, just setting a probation flag and checking that flag when allocating points and dealing with ties. (Technically, it probably needs to be a Tie_ID rather than a flag, so when one tied work is eliminated you can remove the probation state from the survivor).

#952 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 08:22 PM:

Felice@951:

Oh, so a points-and-nominations tie for fifth place, then? I thought we already resolved that we would expand the ballot and keep them all. Prior to fifth place, I say eliminate them all.

Kilo

#953 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 08:44 PM:

Felice@951:

Or are you talking something that could be a finalist, but the works are in a tie prior to last place?

In that case, I feel it's not worth prognosticating. Yes, these might lose, but if they win then something else will lose. At this point I think it's a third foul ball, to use the baseball analogy: There's not a "right" answer. It seems obvious to me that in the quest for the "right" answer we're just moving into the "perfect is the enemy of the good" territory. At some point, I think we just need to make a set of rules that are reasonable and logically consistent and just accept the results, because ultimately, I think while they may not be equal to each other, they are very close to being equal to each other or else they wouldn't be in this position.

I think the simple fact is that when things are that closely balanced, we shouldn't be surprised when they can tip one way or the other. In most cases, even your measure of goodness has a small difference, and it's never overwhelming. We can make more complex rules to narrow things down even more, sure. But I've become convinced that the return just isn't worth the complexity. 2a, I think, makes a nice compromise: if they tie here, then they are even more equal than under base SDV-LPE, so we are even more justified in eliminating them. I think anyone at the business meeting -- and more importantly the general Worldcon members -- would be satisfied that this is pretty fair, even if it arguably might not be perfect. To return to the baseball analogy, you could make a good case that the third foul ball should mean an out -- and no less than the World Series could ride on that one call. But it's a -reasonable- rule, so everyone is pretty much okay with it.

Kilo

#954 ::: Cally Soukup ::: (view all by) ::: May 24, 2015, 08:56 PM:

[ties]

While I like the idea of conjoined twins carrying forward, I don't actively dislike the idea of 2a. Both treat the twins as being worth exactly the same, which makes sense to me, given that on both points and ballots, at that moment, they are.

#955 ::: Jameson Quinn ::: (view all by) ::: May 24, 2015, 09:02 PM:

My preferences are:

10: The statutes state that ties are initially broken by points (2a); and that the procedure for any remaining ties must be pre-announced by the admins, or if not, it will be group elimination.

9: As above, but 2a is not stated.

7: 2a alone

6: Some more complicated procedure of the ones we've discussed is spelled out in the statute.

5: 1 alone

4: Anything involving ballot order or timestamp.

3: Anything involving pseudo-random numbers

2: Anything involving random numbers

0: We keep discussing this forever and never get a proposal together.

#956 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 09:10 PM:

Cally@954:

Yes, I was intrigued by the idea as well. But it turns out that it just keeps them alive for a little bit longer. If they tie late in the game, it makes it more possible for them to still be tied at 5th place and so we would extend the ballot to include them. That happens with 2a as well, but they have to already -be- at the 5th place spot in order to survive. So the real difference between the systems seemed to be just a matter of 3-4 rounds. If the work tied before then, it didn't have a chance anyway (and it still might not even within the window). And if we're only extending the window by 3 or 4 rounds, it didn't seem to be worth the extra complexity.


All:

We are rapidly running out message numbers in this thread, and I have this dream of using the next thread to hammer out the proposal language and presentation. If we aren't ready for that, that's fine, but if we could get an up or down vote regarding 2a from as many people as possible, I think it'd be helpful. If enough people strongly dislike it, then yes, we need to keep looking, but I think it's a workable solution.

Kilo

#957 ::: Keith "Kilo" Watt ::: (view all by) ::: May 24, 2015, 09:15 PM:

Jameson@955:

I don't have a problem with phrasing the proposal language to be something like, "If, in the elimination phase, the works in question are tied for both total points and total number of nominations, then all tied works shall be eliminated. The Hugo administrators are empowered to take further measures to break this tie if they deem it necessary, provided those procedures are announced in advance."

Would that work?

K

#958 ::: ULTRAGOTHA ::: (view all by) ::: May 24, 2015, 09:21 PM:

Sasquan is having a sale through tomorrow on Attending memberships. $190 USD


#959 ::: Cally Soukup ::: (view all by) ::: May 24, 2015, 09:39 PM:

Keith "Kilo" Watt @ 957

"If, in the elimination phase, the works in question are tied for both total points and total number of nominations, then all tied works shall be eliminated. The Hugo administrators are empowered to take further measures to break this tie if they deem it necessary, provided those procedures are announced in advance."

I would add "such" between "all" and "tied works": "all such tied works shall be eliminated", thus preventing rules-lawyers from saying that ALL tied works should be eliminated.

Not that there are any rules-lawyers in fandom, of course.

#960 ::: Soon Lee ::: (view all by) ::: May 24, 2015, 10:09 PM:

Keith "Kilo" Watt #956:

I can live with 2a, so that's a yes for me.

#961 ::: Tammy Coxen ::: (view all by) ::: May 24, 2015, 11:43 PM:

I am highly in favor of KISS, so 2a works just fine for me.

#962 ::: J Thomas ::: (view all by) ::: May 24, 2015, 11:51 PM:

2a works for me. Plus the wording in #957 and #959.

#963 ::: felice ::: (view all by) ::: May 24, 2015, 11:59 PM:

Keith "Kilo" Watt @953: "Or are you talking something that could be a finalist, but the works are in a tie prior to last place?"

Yep.

"At some point, I think we just need to make a set of rules that are reasonable and logically consistent and just accept the results, because ultimately, I think while they may not be equal to each other, they are very close to being equal to each other or else they wouldn't be in this position."

We established earlier that the upper limit for difference is either of the tied works being twice as good as the work they'd replace. That doesn't seem trivial to me. The degree of difference possible is proportional to the degree of overlap.

In 2011, Connie Willis' Blackout and All Clear won Best Novel with 125 nominations (I'm not sure how the decision to count them as a single work was reached). If they'd been counted separately under SDV-LPE, it's not that implausible that they could have tied for 6th (or 7th or 8th, or maybe 9th) with less than 74 points - and Who Fears Death with 74 nominations would have replaced them on the final ballot.

#964 ::: Steven desJardins ::: (view all by) ::: May 25, 2015, 02:26 AM:

I like Keith's language in @957, with Cally's amendation in @959. I'm philosophically in favor of breaking ties, even through essentially arbitrary means, but I think trying to embed any further tiebreakers in the Constitution adds one more thing for people to argue about without enough benefit to justify the arguing. Leaving further details to the administrators strikes me as an excellent compromise.

#965 ::: Aan ::: (view all by) ::: May 25, 2015, 03:51 AM:

I can live with 2a as well.

#966 ::: David Harmon ::: (view all by) ::: May 25, 2015, 05:40 AM:

Likewise, I can be happy with the clarified 2a as of Cally Soukup #959.

#967 ::: junego ::: (view all by) ::: May 25, 2015, 06:52 AM:

Keith "Kilo" Watt @ 957

[TIES]

I don't have a problem with phrasing the proposal language to be something like, "If, in the elimination phase, the works in question are tied for both total points and total number of nominations, then all tied works shall be eliminated. The Hugo administrators are empowered to take further measures to break this tie if they deem it necessary, provided those procedures are announced in advance."

1) I already agreed that if there are no significant differences when sims are run (between breaking and not breaking ties) that I can live with 2a.

2) Not sure I agree with the wording above, "announced in advance" of what? That a tie breaking method should be announced before tallying is run, before voting begins, after a problematic tie is found but before breaking it? I see issues with every possible interpretation of this.

If a tie breaking method is announced before nominations/voting, we might as well list the 'best' of our recommendations for them to choose from, otherwise the admins are flying blind about ramifications to the tallying system and opening themselves to all sorts of accusations. If they announce -after- they have a 'problem' , talk about Monday-morning quarterbacking!

3) After thinking about it, I don't think we should spell out the tie issue any more than defining it as requiring both nominations and points being tied and eliminating both as the solution, unless the tie is in fifth place, in the constitution itself. There may be unintended consequences in either restricting or expanding administrative authority outside what already exists. As it stands now, a large finalist list is already a possible, but rare, event and no specific authority is given to the admins to deal with the problem. Unless someone can show that the new system will increase the likelihood of 10 or more finalists by orders of magnitude, I think we should leave it.

We may, however, need to decide how much of the current discussion to put into our official explanation, which I think would be guidance if there is a future rare case that needs a decision beyond specifics in the constitution. At least that's what I'm assuming could happen based on what Mr. Standlee mentioned upthread. In such a case section 1.2 (5) may give the requisite authority ( http://wsfs.org/bm/const-2014.html )

"(5) To perform such other activities as may be necessary or incidental to the above purposes.". (which includes "To choose the recipients of the annual Hugo Awards (Science Fiction Achievement Awards).")

But I would never -dream- of rules-lawyering. :^]

#968 ::: junego ::: (view all by) ::: May 25, 2015, 07:07 AM:

Keith "Kilo" Watt @946
Junego@945:

Which sim was it you wanted me to run? I don't recall. I'm glad to run any sim in any system I've coded, just let me know. In 2a, an undisciplined slate tended to converge to the most popular slate work, which then made the final ballot. Was that the case you were looking for?

See @934. I was wondering about how many and what happened to ties using diff tiebreak schemes and changes in finalists overall with an undisciplined slate that had some cross-over with non-slate voters and some crossover with the most popular works. If what I proposed upthread is similar to other sims you've run, don't bother.

#969 ::: Jameson Quinn ::: (view all by) ::: May 25, 2015, 08:24 AM:

@967: You seem to be saying: "If they don't have discretion to change systems after the votes are in, then I'd rather not give them any discretion at all", which makes no sense to me.

I support 2a+@959, which seems to me to be moving towards consensus. I'd support something else if it were the consensus. I think everyone here is discussing in good faith, but if we can't get a reasonable consensus before @1000, I'll begin to suspect that people are nitpicking for the sake of nitpicking.

#970 ::: Jameson Quinn ::: (view all by) ::: May 25, 2015, 08:58 AM:

Sorry, that came off as more of an ultimatum than I intended. What I suspect or not isn't important; what's important is that we all try to avoid making the best into the enemy of the good.

#971 ::: J Thomas ::: (view all by) ::: May 25, 2015, 09:27 AM:

I support the growing consensus, but I do want to point out that the "best is the enemy of the good" is a meme designed to rush decisions.

Felice's concern is whether our consensus is good *enough*. I think it probably is, she isn't so sure. If I saw a way to quickly make sure one way or the other, I'd charge at it.

I think we're OK, but there's some (I hope small) possibility that what we're facing is "the better is the enemy of the not-good-enough".

#972 ::: Jameson Quinn ::: (view all by) ::: May 25, 2015, 09:39 AM:

Point taken. Less meta, more discussion. Specifically: predeclaring a tiebreaker is not "flying blind" any more than a fixed tiebreaker, so I support allowing admins to predeclare tie procedures for when 2a is not enough.

#973 ::: J Thomas ::: (view all by) ::: May 25, 2015, 09:50 AM:

Agreed. We present something we have reason to think is good enough. If the administrators think otherwise, either after further study or from a bad experience, they can choose something else and announce it ahead of time.

#974 ::: P J Evans ::: (view all by) ::: May 25, 2015, 10:35 AM:

971
It's usually an ultimatum intended to get people to stop nitpicking about the fraction-of-one-percent possibilities. No rules cover 100 percent of everything. (I learned that decades ago, in a data conversion project: we had rules, and we had people who could decide what to do with the weird cases that didn't fit. Is 18 inches one foot or two? If two different pieces of pipe are put in at the same time, and a section is then cut off one and welded to the other, does the welded piece count as old or new?)

#975 ::: Steven desJardins ::: (view all by) ::: May 25, 2015, 12:24 PM:

@967, 2: That's a reasonable point. How about adjusting the language to "provided those procedures are publicly announced before the end of the nominating period"? Given how many ballots come in at the last minute, it would be impossible for the administrators to game the system.

#976 ::: junego ::: (view all by) ::: May 25, 2015, 12:28 PM:

Jameson Quinn @ 969
@967: You seem to be saying: "If they don't have discretion to change systems after the votes are in, then I'd rather not give them any discretion at all", which makes no sense to me.

I'm saying that the current constitution does not give them the specific discretion to break large ties in 5th place. It specifically says to add finalists in that case. It doesn't give them explicit discretion to decide there are too many finalists (or any other problem) and figure out their own method of reducing finalists or resolving ties.

1) I think adding this kind of discretionary power will cause resistance at the business meeting. The less we change current practice, the better the chances of acceptance, IMHO.

2) If I was an administrator faced with some troublesome results I'd want clear rules to resolve the problem. I wouldn't want some nebulous 'discretion' to fix problems that some could use to beat me over the head if they didn't like my solution or thought I should fix something I didn't think should be fixed.

2a is sufficient instructions, afaics.

I won't hold up the consensus if everyone else thinks this is necessary. I just think it adds complexity we don't need, trying for some perfect solution to, reportedly, very rare problems.

#977 ::: Keith "Kilo" Watt ::: (view all by) ::: May 25, 2015, 03:11 PM:

junego@968:

I ran the specific sim you described under option 1, 2a, and linked ties. All three results came out identically: The most popular slate work replaced the least popular of the original finalists. That's in line with what I've seen with similar tests I've done as well.

Let me know if there's anything else!

Kilo

#978 ::: J Thomas ::: (view all by) ::: May 25, 2015, 04:01 PM:

@976 Junego

If I was an administrator faced with some troublesome results I'd want clear rules to resolve the problem. I wouldn't want some nebulous 'discretion' to fix problems that some could use to beat me over the head if they didn't like my solution or thought I should fix something I didn't think should be fixed.

That sounds reasonable. I can see that one either way. Would it make sense to find some current or upcoming Hugo voting administrators and ask them? Or would it be better to decide ourselves?

It needn't cause a delay, we could prepare to do it however we liked and then change it if we heard from them what they preferred.

#979 ::: Pfusand ::: (view all by) ::: May 25, 2015, 07:07 PM:

I've recently come up with another possible tie-breaker, based mostly on a distaste for slates:

Once we have nominees tied for point and votes, we compute the average point scores that each tied nominee began with, and the higher average wins.

My theory is that slate works would average .2, and non-slate works would be higher. Would this work? Would it be acceptable?

#980 ::: junego ::: (view all by) ::: May 25, 2015, 08:03 PM:

Keith "Kilo" Watt @ 977
junego@968:

I ran the specific sim you described under option 1, 2a, and linked ties. All three results came out identically: The most popular slate work replaced the least popular of the original finalists. That's in line with what I've seen with similar tests I've done as well.

Thank you. This just confirms, to me, that going with option 2a is good enough for all but the edgiest of edge cases.

#981 ::: Cally Soukup ::: (view all by) ::: May 25, 2015, 08:23 PM:

Pfusand @979

I'd be worried that it would make nominators more prone to trying to game their nominations. Whether it would work or not I don't know, but literally the first thing I saw when I read that paragraph was "what if I made sure not to nominate any of the outlying stuff I like?" And we WANT people to feel free to nominate weird stuff, because weird stuff sometimes makes the ballot, and even wins.

If that's the thought that comes to a (slightly) informed person, how much more likely would it come to the naive rules-reader?

#982 ::: junego ::: (view all by) ::: May 25, 2015, 08:57 PM:

J Thomas @978

@976 Junego said:
"If I was an administrator faced with some troublesome results I'd want clear rules to resolve the problem. I wouldn't want some nebulous 'discretion' to fix problems that some could use to beat me over the head if they didn't like my solution or thought I should fix something I didn't think should be fixed."

That sounds reasonable. I can see that one either way. Would it make sense to find some current or upcoming Hugo voting administrators and ask them? Or would it be better to decide ourselves?
It needn't cause a delay, we could prepare to do it however we liked and then change it if we heard from them what they preferred.

It certainly can't hurt to put out the call. So ADMINISTRATOR OPINION NEEDED, please!

I've said I won't object to moving forward and it seems the 2a option has the consensus.

The more I've thought about it, though, the less I see where this new discretion makes sense. Unless we're talking about tieing for fifth place, at what point would the administrators even know they had an issue, especially before the end of nominations and before any tallies were counted?

It's all going to be compiled by a computer program, right? Is the program going to have decision points for administrators to check on or will every tie need an operator confirmation before proceeding or will there be special cases that will be defined where the program asks for a decision?

If what we're primarily concerned about is the fifth place tie and having too many finalists a) that has always been a possibility but current rules don't address it, b) I don't think it is going to be orders of magnitude more likely now (based on all the sims), and c) adding a new 'power/responsibility' for administrators may increase resistance at the business meeting without any real gain for the proposal.

#983 ::: P J Evans ::: (view all by) ::: May 25, 2015, 09:21 PM:

982
Administrators have discretion - and use it - because people put names in the wrong category, they put in names that are really seriously ineligible ('Dune' for best novel and Asimov for the Campbell Award in 1984), and sometimes they don't pay a attention to instructions (tell them #2 pencil and get blue ballpoint, red pen, India ink, 2H pencil). Administrators get to fix that stuff, as much as they can.

#984 ::: David Harmon ::: (view all by) ::: May 25, 2015, 09:59 PM:

P J Evans #983: people put names in the wrong category, they put in names that are really seriously ineligible (...), and sometimes they don't pay attention to instructions

I find myself wondering if in this respect, fans would be worse or better than the general public.

#985 ::: P J Evans ::: (view all by) ::: May 25, 2015, 10:04 PM:

984
I don't know. Probably about the same, at a guess - but you expect smarter from fans, so it seems worse.

#986 ::: Keith "Kilo" Watt ::: (view all by) ::: May 26, 2015, 01:59 AM:

[NEXT THREAD?]

All:

Okay, it looks to me like we have a consensus around option 2a, though there is some question as to whether we should give the Hugo admins any explicit future leeway or not. I have written up the next version of the proposal and FAQs leaving in the admin empowerment, but I think we can discuss that more fully in the next topic since it's reall ymore proposal language than anything about the system itself. Since there are fewer than 15 messages left in this thread, I'd like to propose to everyone (and to abi) that we go ahead and close this topic and open a new one to be used for ironing out the formal proposal and FAQ language. Once we have that down, I think the system will be ready to go out into the wild. We do still need to write a "6th place" amendment, but I don't think that will be terribly complicated since we all agree what that needs to do.

So, abi, unless there is any objection from the group here, it'd be great if we could make the next message the start of the new topic.

We're almost there folks!

Thanks for everything,
Kilo

#987 ::: Keith "Kilo" Watt ::: (view all by) ::: May 26, 2015, 02:27 AM:

E Pluribus Hugo: Out of the Many, a Hugo

In this thread we will hammer out the formal language of the proposal, any FAQs we wish to include, and strategize for the presentation at the business meeting itself. At this point, we'll consider the system itself locked in, so we are really only looking at the language. For reference, consensus seemed to coalesce around:

1. RME instead of 6th place
2. (1,1), (1,2), or (1,2,2) for ties in points
3. Option 2a (if there is a tie for nominations, eliminate the one with fewer points; if there is a tie for both nominations and points, eliminate them both)

There is one more issue that is still up for debate: Should we explicitly empower the Hugo admins to use further tie breakers in the future if they decide it's necessary? I've written the proposal and FAQ explanations assuming that we do, however, a case can be made for not worrying about giving them the power explicitly. We should settle that question here. I think that the way I've written the "empowerment" makes it okay to include it, but for myself, I don't feel a strong need to. I'm definitely not opposed to it, however.

If I've forgotten any signers, please let me know!!

Kilo

=========================================

Short Title: E Pluribus Hugo (Out of the Many, a Hugo)
Moved, to amend section 3.8 (Tallying of Nominations), section 3.9 (Notification and Acceptance), and section 3.11 (Tallying of Votes) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed. determined by the following multi-round process described in 3.8.8.
3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.
3.8.3: Any nominations for “No Award” shall be disregarded.
3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.
3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.
3.8.6: The Committee shall move a nomination from another category to the work’s default category only if the member has made fewer than five (5) nominations in the default category.
3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.9 or 3.2.10, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category.
3.8.8: The final Award ballots shall list in each category the eligible finalists as determined by successive rounds of a two-phase elimination process. In this process, each member gets a single nomination “point” for each category, and that point will be divided equally among their nominated works in that category.
3.8.8.1: Selection Phase: In each round, all the works having the least number of points will be selected for the Elimination Phase (3.8.8.2). If there is only one work with the least number of points, then all of the works with the second-least number of points will be also be selected for the Elimination Phase in addition to the lowest-point work.
3.8.8.2: Elimination Phase: Of the works identified in the Selection Phase, the one(s) that appear on the fewest number of nomination ballots will be removed from all nomination ballots for subsequent rounds as if they had never appeared on any ballots.
3.8.8.4: Ties: If two or more works are tied for appearing on the fewest number of nomination ballots, the tied work with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of nomination ballots as well as for lowest point total, then all members of that tie will be eliminated. Should they deem it necessary to do so in the future, the Hugo administrators are empowered to take further measures to break this type of tie, provided those measures are announced at the beginning of the nomination period for the Hugo Award.
3.8.8.5: If elimination would reduce the number of finalists to fewer than the minimum number, then instead none of the works from that round shall be eliminated, and all remaining works shall appear on the final ballot, extending it if necessary.
3.8.8.6: Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are removed from all nomination ballots and treated as if they had never appeared on any nomination ballot. Members’ points are then reallocated equally among their remaining nominated works, if any.

Section 3.9: Notification and Acceptance.
3.9.1 Worldcon Committees shall use reasonable efforts to notify the nominees finalists, or in the case of deceased or incapacitated persons, their heirs, assigns, or legal guardians, in each category prior to the release of such information. Each nominee finalist shall be asked at that time to either accept or decline the nomination. If the nominee any finalist(s) declines the nomination, that nominee finalist(s) shall not appear on the final ballot. In this event, once all finalists have had the opportunity to decline, the nomination system shall be re-run as described in Section 3.8 with the declined nomination(s) removed from the nomination ballots on which they appeared. The eligible finalists from this re-run shall be merged with the remaining potential finalists from the original run. If this merge would result in more than the maximum number of finalists, then the ballot shall be extended to include the finalists from both the original and the re-run of the nomination system. This procedure shall also be used in the event that a finalist is deemed ineligible.

Section 3.11: Tallying of Votes.
3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, … places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes. During the same period a record of all rounds of the selection process for each category shall also be published.

Submitted by: Keith “Kilo” Watt, Tammy Coxen, Yoana Yotova, Joshua Kronengold, Chris Battey, CJ Cabourne, Steven Halter, P.J. Evans, David Goldfarb, Seth Gordon, Ginger Tansey, Steve Wright, Catherine Faber, Andy Holloway, Duncan J. Macdonald, Claudia Beach, Derry Earnshaw, Jason Skiles, Soon Lee, David Harmon, Lydy Nickerson

Commentary: “E Pluribus Hugo” – the name of this proposal is also its goal: recognizing the many opinions within fandom as to what works might be worthy of the Hugo. It is to be emphasized that this proposal does not change the nominating process from the perspective of Worldcon members: They still list the works, unranked, they feel are Hugo-worthy (up to the maximum permitted) in any categories they choose, just as they have in the past. In the past, we have counted the number of nominations each work received, and the top five works were put on the final ballot. However, because SF fandom typically nominates a large number of different works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn't penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate anything you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don't, that's okay – when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates each round, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

It is also an explicit goal of this proposal not to disenfranchise anyone. Rather, this proposal seeks to ensure that no minority of members – of any sort – can disproportionately dominate an entire category. This system allows the broadest range of works that are popular with fandom-at-large to be considered for the Hugo Award.


FAQ’s:
1. Can you explain the system in plain language?
The important thing to remember is that nothing changes in how you nominate. If you think a work is Hugo-worthy, then nominate it. That’s all. There’s no need to rank your choices at the nomination stage, and there’s no reason not to nominate something you think even might be Hugo-worthy. All we are doing at this stage is throwing names into a hat. The final Hugo voting system, which actually chooses the winner, is unchanged. We could, in theory, simply put everyone’s nominations on the final ballot, but that would make for a very long ballot indeed. We therefore need to narrow the nomination list down. This system narrows down the list by eliminating the least popular works until only five (under current rules) finalists remain. Here are the basic steps to the elimination process:
a. You have one nomination “point” for each category that will be divided equally among the works you choose to nominate in that category. So, if you nominate two works in a category, each will get half a point; if you nominate three works, each will get one-third of a point, and so on.
b. All the points given to each work from all nomination ballots are added together. The two works that got the least number of points are eligible for elimination. One of these works is the least popular and will be eliminated. (We call this the Selection Phase.)
c. To determine which of these two works is least popular, we compare the total number of nominations they each received (that is, the number of nomination ballots on which each work appears). The work that received the fewest total number of nominations is the least popular and now completely vanishes from the nomination process as though it never existed. (We call this the Elimination Phase.)
d. We start over for the next round and repeat the process, however, if one of your works was eliminated, then you now have fewer works on your nomination ballot. This means that each work gets more total points, since you aren’t dividing your point among as many works. For example, if one of your five nominated works was eliminated, your remaining works now get one-fourth of a point each instead of one-fifth of a point. If four of your nominated works are eliminated, your remaining work now gets your full point.

2. E Pluribus Hugo?
The designers felt this name accurately reflects what the Hugo nomination system should be trying to achieve. Fandom is the Many. Members of fandom have a wide range of interests, tastes, and desires in what they appreciate in SF. This is a defining characteristic of fandom, and this system embraces it. Out of all of these many tastes in works, this system allows a single set of nominations to slowly emerge as the most popular candidates for the Hugo Award: “Out of the Many, a Hugo.”

3. But surely that’s not what the system is formally called?
Okay, sure. Formally, in the context of election theory, this system would be called “single divisible vote with least popular elimination” or SDV-LPE for short. E Pluribus Hugo is both cooler and geekier, as everything is better with a little Latin.

4. How does this system eliminate slate or bloc voting?
It doesn't, exactly, nor should a work be automatically eliminated just because it appears on a slate. On the other hand, any slate which nominates a full set of five works will find that each of its nominations only count 1/5 as much. With “non-slate” nominating, some of your works will be slowly eliminated, so your remaining works get more and more of your support. Since slate works tend to live or die together, they tend to eliminate each other until, in general, only one slate work remains. With a large enough support behind the slate (five times as much), the slate may still sweep a category; however, if that many voters support the slate, they arguably deserve to win, and no fair and unbiased system of nomination will prevent that. The answer in that case is, simply, to increase the general pool of voters. Regardless, with SDV-LPE, slates will never receive a disproportionate share of the final ballot, as occurred in the 2015 Hugos.

5. What if there are multiple slates (slate wars, “parties”, etc.)?
As with a single slate, the more works a slate nominates, the less their nominations count for each work. The end result is that even multiple slates are unable to sweep the nominations.

6. What happens if a broadly popular work is nominated by a group of unrelated people?
If it is broadly popular, the system will still select that work for the final ballot.

7. What happens if a broadly popular work also appears on a slate?
Even if it is on a slate, if the work garners support from individuals then the system will select that work for the final ballot. Slates neither help nor hurt any given work.

8. Why re-run the system if someone declines a nomination? Why not just take the work that was eliminated in the next to last round?
We can do that, and in fact have an amendment for that ready if that’s what the people decide they want to do. The “re-run, merge, and extend” (RME) system described in the proposed language has only one major difference between it and taking the “6th place” work. Under RME, if a slate work makes the final ballot and that nomination is declined, the declined nomination will tend to be replaced by another work from the slate. Under “6th place”, if a slate work makes the final ballot and that nomination is declined, the declined nomination will tend to be replaced by a non-slate work. Either way works, but since our goal is for all sections of fandom to be fairly represented on the final ballot, RME seemed more in line with that philosophy.

9. Isn't it true that any voting system can be gamed (or strategized, etc.)?
Yes, there is a theorem which proves that all voting systems must have inherent flaws. The objective is to choose a system whose flaws are not in an area of concern to the electorate.

10. What are SDV-LPE’s flaws?
In rare cases, it is possible for eliminating both members of a tie to change the final ballot slightly from what it would be if the tie were broken so that only one member was eliminated. The change is usually in the least popular of the finalists, and requires the two works in question be very close in popularity. This situation was extremely rare using realistic data (in fact in never occurred in any of our situations with realistic data), so the statistical probability of how ties are handled affecting the results is extremely small. There are a number of additional tie breakers that could be used if the Hugo administrators deemed it necessary; however, simulations that have been run by the designers of this system show that it really shouldn’t be required. The proposal language leaves open that option if later Hugo administrators want to add those layers of complexity to the process. SDV-LPE itself works the same regardless.

11. What are SDV-LPE’s benefits?
Simply put, it reduces the power of bloc voting without eliminating the chance that works that do appear on slates will make it to the final ballot. Conversely, it makes it very difficult for slates to prevent non-slate works from appearing on the ballot.

12. Couldn't slates just recommend a single work for a candidate, and it will automatically appear on the final ballot?
Yes, if a slate is large enough that is certainly a viable possibility – it’s also completely fair. It does not force any other works off of the final ballot, and the final Hugo winner is determined by the same voting process we have always had. Just appearing on the final ballot isn't a guarantee of winning a Hugo. However, if any large section of fandom strongly believes that a work deserves a Hugo nomination, then it should, in fact, be represented on the final ballot.

13. What happens with a large field with no stand-out favorites when a slate votes?
Even in this case, we were unable to find a simulation in which no non-slate works appeared on the final ballot. Slate works did receive a larger proportion of nomination slots than they did otherwise, however, again, this could be considered a fair and valid result. If there was no general favorite, then nominators really had no collective preference.

14. How do the results of this system compare to the results under the current nomination system?
Statistical tests showed that this system and the current system overlap in about 4.5 out 5 candidates. In the absence of slates, it generally gives identical results to the current nomination system.

15. I think we should just increase the number of nomination slots on the final ballot to (for example 6), and decrease the number of slots a voter can vote for to a smaller number (for example, 4). Wouldn't that be simpler and easier?
Unfortunately, this simply means that the largest slate will receive four of the nominations and the next largest will receive the remaining two. It doesn't solve the problem of forcing works off the ballot that had a chance to win the final election. In general, we want fandom to nominate as many works as they feel are Hugo-worthy, since under SDV-LPE there is no strategic reason not to do so. For this reason, most of the designers of this system would prefer that members not be limited to nominating only four works. Keep in mind, however, that SDV-LPE will work with this (or most any other) change as well, so one does not preclude the other.

16. I think we should set up a committee to handle these situations as they occur. The committee would be empowered to add nomination slots or throw out slate-influenced ballots as required.
This could work. The problem is that now you have a small group of people who serve as literal gatekeepers to the Hugo Awards. In spite of the word on the Internet, this has never been the case in the past. Establishing it now means that those groups who believed it existed in the past will now be correct. Ultimately, human judgement is fallible. The fairness of a committee’s decisions will forever be subject to opinion. The end result is that the prestige of the Hugo Awards will forever be tarnished.

17. I think we should use [insert other mathematical voting system].
We considered essentially every applicable type of voting system currently in the literature, guided by two experts in the field. It should be kept in mind, however, that the goals and requirements for choosing a set of representatives in a political situation are different from those for choosing a set of Hugo finalists. Some of these systems do, in fact, have positive properties that speak for them. None of them were as simple or as intuitive as SDV-LPE, yet SDV-LPE meets all of the stated goals for a Hugo nomination system.

18. Won’t SDV-LPE be complicated to code and implement?
Actually, no. One of our non-experts coded a full simulator for the system in a matter of days. A full web-based app would not be much more difficult to handle.

19. Wasn't this system just designed by Social Justice Warriors to block the Good Stuff?
It is true that much of the discussion for this system occurred on Teresa and Patrick Nielsen Hayden’s “Making Light” discussion board, and it is also true that groups such as the Sad Puppies and the Rabid Puppies consider TNH and PNH to be The Enemy, and therefore completely biased and not to be trusted. Other than serving as occasional moderators, TNH and PNH had no real input in the discussions of the system, however. Those of us who worked on the system were very clear that our goal was not to keep the Sad/Rabid Puppies off of the Hugo ballot, and that any system which specifically targets any type of work is inherently wrong and unfair. One of the members of the group is a retired US Naval officer, a combat veteran, a certified Navy marksman, a Christian, and considers Robert Heinlein to be the greatest science fiction author who has ever lived. In short, he is exactly the Puppies’ demographic. But any slate, of any sort, be it a Sad Puppy or a Happy Kitten of Social Justice, breaks the Hugo Award because a small percentage of voters can effectively prevent any other work from appearing on the final ballot. This is a major flaw in the Hugo nomination system, and it is a flaw that must be fixed if the integrity of the award is to be maintained. Politics should play no role whatsoever in whether a work is Hugo-worthy or not.

#988 ::: abi ::: (view all by) ::: May 26, 2015, 02:44 AM:

Keith:

If none of the other mods do it, I'll post it this evening my time. Gotta run to work right now.

#989 ::: felice ::: (view all by) ::: May 26, 2015, 04:27 AM:

Keith "Kilo" Watt @987: "2. (1,1), (1,2), or (1,2,2) for ties in points"

If we're not doing any kind of tiebreaking, I have a strong preference in favour of (1,1,2); that way tied works at least have the possibility of surviving to 5th. If either of the tied works was withdrawn, the other would be compared against 2; why not compare both of them against 2?

#990 ::: junego ::: (view all by) ::: May 26, 2015, 07:58 AM:

Keith "Kilo" Watt @987:
2. (1,1), (1,2), or (1,2,2) for ties in points

I feel a bit dimwitted, but could you nudge my memory on this? I thought we had consensus on taking the lowest 2 point totals for elimination phase whether that was (1,1) or (1,2) and only have to have more than 2 if there were ties such as (1,2,2) or (1,1,1) or similar. Is this a different issue or did I misunderstand about having consensus?

#991 ::: Jameson Quinn ::: (view all by) ::: May 26, 2015, 09:42 AM:

@976: It seems that your objection to discretion is that you don't want admins to break ties for 5th place? If so, I agree entirely. Ties for 5th place should both go on the ballot. I thought we were talking about ties before then, which have to be dealt with somehow, but for which, according to Keith's sims, whatever we do usually doesn't end up making a difference to the final results. I think that admins should be empowered to make a plan for dealing with those, even if that plan is more complicated than something we'd want to write into the statutes.

@989: I clearly prefer not to do (1,1,2). (I'm not saying "strongly", because I'm not going to hold up the discussion over this.) If you want to reopen that debate, I can explain again why (not allowing slates to run the table if we can help it). But I'd prefer if we can move on.

@990: I think what you're saying and what Keith was saying are actually the same thing.

#992 ::: Cally Soukup ::: (view all by) ::: May 26, 2015, 10:19 AM:

Section 3.9.1: are you missing a strikeout through "nominee" in "that nominee finalist(s) shall not appear on the final ballot."?

#993 ::: Keith "Kilo" Watt ::: (view all by) ::: May 26, 2015, 11:21 AM:

Junego@990:

Yes, that's right. We have been using (1,1) as shorthand for all ties in lowest points (so also 1,1,1 etc), (1,2) for the "normal" situation, and (1,2,2) when we have only one lowest point work but multiple second-lowest point works.

The case we eliminated was (1,1,2) -- if there is a tie in the lowest point works, we won't look at the second-lowest.

K

#994 ::: Keith "Kilo" Watt ::: (view all by) ::: May 26, 2015, 11:32 AM:

Cally@992:

Thanks, you're right. In 3.9.1 that should be:

If the nominee any finalist(s) declines the nomination, that nominee finalist(s) shall not appear on the final ballot.

K

#995 ::: Cally Soukup ::: (view all by) ::: May 26, 2015, 11:47 AM:

I've skipped chunks of some of the threads (sorry), so I missed the discussion of why we are entirely getting rid of the term "nominee". Can we at least keep it in the phrase "appear on the final ballot as a nominee"? I can see "finalist" until we know they're on the ballot, but surely once we know they are, they're a nominee?

#996 ::: Keith "Kilo" Watt ::: (view all by) ::: May 26, 2015, 12:15 PM:

Cally@995:

I believe that change was at Joshua or Kevin's request (please chime in!). They should give the full tale, but as I understand it, there was some issue about "if I was nominated for a Hugo, then I'm a Hugo nominee even if I didn't make the final ballot". I think that change is going everywhere in the constitution. Hopefully those in the know will shed more light on the issue, as I can't seem to find the original reference, so I'm trusting my often-faulty memory.

Kilo

#997 ::: Tim Illingworth ::: (view all by) ::: May 26, 2015, 12:35 PM:

Cally@995: As KW says in 996, we're in the middle of making the change so that nominees are what's written in on the nominating ballots and finalists are what's on the final ballot.

I think there's other problems in the wording (like the use of "works" in 3.8.8, which should be "nominee"), but we can tackle them in the next thread...

#998 ::: Aan ::: (view all by) ::: May 26, 2015, 12:36 PM:

Typo fix for FAQ #10:
in fact >it< never occurred in any of our situations with realistic data

#999 ::: Darth Paradox ::: (view all by) ::: May 26, 2015, 12:58 PM:

A little late to the consensus stage, but since I had (re)raised the pseudo-random tiebreaker thing, I just wanted to say I'm fine with our having settled on 2a. And I'm really happy with the rest of the proposal as well. Thank you, everyone, for a fantastic discussion.

#1000 ::: Kevin Standlee ::: (view all by) ::: May 26, 2015, 01:03 PM:

There is a pending constitutional amendment (A.3) that changes "nominee" to "finalist." We are deprecating the use of "nominee" as a reference to "person or work appearing on the final ballot" because of misuse people labeling themselves "Hugo Award Nominee" because they got one nominating vote. When called out on it, they complained that the "plain meaning" of the word "nominee" means that they were nominees. Therefore, WSFS is deflating their balloon.

Unlike many other amendments on the agenda, I rather hope this one will be non-controversial.

Whatever you propose here, I hope it will use "nominee" to mean "work/person nominated in the initial voting" and "finalist" to mean "work/person appearing on the final ballot."

#1001 ::: Jameson Quinn ::: (view all by) ::: May 26, 2015, 01:12 PM:

@1000: I'd prefer it not use "nominee" at all, because even if it were used to mean what you want it to mean, some people would initially read it to mean what you don't want it to mean. So I'd like to find words that are unambiguous.

"Vote" is similarly a tricky word.

That's part of why I've been promoting using "approve"/"approval" in the text of our proposal.

#1002 ::: junego ::: (view all by) ::: May 26, 2015, 01:22 PM:

Jameson Quinn @991

@976: It seems that your objection to discretion is that you don't want admins to break ties for 5th place? If so, I agree entirely. Ties for 5th place should both go on the ballot. I thought we were talking about ties before then, which have to be dealt with somehow, but for which, according to Keith's sims, whatever we do usually doesn't end up making a difference to the final results. I think that admins should be empowered to make a plan for dealing with those, even if that plan is more complicated than something we'd want to write into the statutes.

1) Fifth place ties are -one- of my objections. So we agree, admins shouldn't mess with the fifth place tie rule.

2) Next objection: How will the admins know they have a problem with non-fifth-place ties if the tallying is done by computer programming unless the class of problems (which we can't seem to define) that would cause this crisis are coded into the program to alert them? And who should make these programming rules if not us?

3) Next objection: And since there is support to require the admins to define what method they would use to break this very, very rare edge-case tie before tallying even begins, how would they know to do this? And what method would they choose since they would have even less idea than we do about the best/fairest possible solutions that still comply with the goals of SDV-LPE, especially within a tight time schedule?

4) Next objection: If I was at the business meeting and someone made a proposal (about something so integral to the Hugo awards) that included the proviso (in a completely new tally method) that the authors thought there would be problems requiring this level of responsibility put on future administrators, I'd object...loudly. Our 20 minutes of discussion could be wasted trying to explain why we need to put this atypical discretion phrase into the proposal.

If I'm misunderstanding how the administrator clause would work in practice, please unconfuse me.

I think the 2a option covers (as you said upthread) 99%+ of all situations. Adding the admin clause is trying to make 100% coverage, but I think it creates more problems than it solves.

@990: I think what you're saying and what Keith was saying are actually the same thing.

So we have a consensus but we need to rediscuss it?

#1003 ::: Keith "Kilo" Watt ::: (view all by) ::: May 26, 2015, 01:36 PM:

junego@1002:
So we have a consensus but we need to rediscuss it?

I may have not been clear in the post. The three items listed there are the ones we have agreed on so are -not- open for discussion. In other words, the way you are interpreting #2 is correct. I was just reminding everyone what the formal proposal needs to contain.


Admin empowerment is an open issue, as far as I'm concerned. The only way admin empowerment could ever come into play is if in the past someone had serious problems with a tie break and wanted something different for the next year's Hugo. This would enable them to do that without an amendment. I don't think there's any way to give the admins discretion after the results have been determined (which is why I stipulated any changes had to be announced at the start of the Hugo nomination process).

Personally, I don't think it's needed. The cases where it even makes a difference are going to be so uncommon that once every 20 years just isn't going to be an issue. And when some case does comes up, the odds of it being so different that it's contentious are so small that I still don't think it's worth worrying about.

I'm open to what the group wants, but if left to me, I'd leave that part out and just do another constitutional amendment in 20 years if someone really wants to. I just don't see it as that big an issue.

Kilo

#1004 ::: abi ::: (view all by) ::: May 26, 2015, 01:43 PM:

OK, should I use the text in 987 for a new front page post, or has it changed due to the recent discussion?

#1005 ::: J Thomas ::: (view all by) ::: May 26, 2015, 01:43 PM:

1002 junego

How will the admins know they have a problem with non-fifth-place ties if the tallying is done by computer programming unless the class of problems (which we can't seem to define) that would cause this crisis are coded into the program to alert them? And who should make these programming rules if not us?

The circumstance in question, comes up when there is a double tie (tied on votes and on scores) and it is not a fifth-place tie that requires both to be accepted.

We give a clear, consistent rule: eliminate both. If the administrators decided they should make some different rule, or they should make some different rule which applies part of the time, or they should look at the situation personally and decide what to do, they would do the following:

Ahead of time, redo the programming to follow the new rule -- whatever it is. Announce what they are doing, preferably before the voting starts. Certainly announce it before they have seen the votes and before they know what the result of the new rule will be this time.

If they want to change the program so that in some particular circumstance (probably involving a double tie) it stops and displays the results so far and invites them to make a personal decision based on what the votes are doing .... I'd guess they probably don't want to do that. If they do, they should announce it ahead of time. I'm guessing that would sound bad. "If the voting doesn't go the way we want it to, we're going to stop it and decide what we want, and change the rules to make it do what we want." No, probably not what they want to announce.

So basicly, we're telling them what we think the rule should be and letting them change the rule if they want to.

It isn't much opportunity for them to be unfair to somebody, because first it should rarely have an effect and second they haven't seen the votes yet and don't know what will happen.

Next objection: If I was at the business meeting and someone made a proposal (about something so integral to the Hugo awards) that included the proviso (in a completely new tally method) that the authors thought there would be problems requiring this level of responsibility put on future administrators, I'd object...loudly.

That may be an extremely important objection. People may object loudly either way. I'm inclined to think you're right and this is the worse approach, but I don't know. At this point I don't know which is worse.

#1006 ::: junego ::: (view all by) ::: May 26, 2015, 01:51 PM:

@Kilo
[PROPOSAL]
3.8.8: The final Award ballots shall list in each category the eligible finalists as determined by successive rounds of a two-phase elimination process. In this process, each member gets a single nomination “point” for each category, and that point will be divided equally among their nominated works in that category.

I think we need more description of the process here. I'd suggest something like below to modify the final sentence above:

"In this process,...category; all the points from all ballots will be summed for each nominee; and the total number of ballots each nominee appears on will be tallied. These two numbers will be used in the two-phase process as described below."

#1007 ::: Keith "Kilo" Watt ::: (view all by) ::: May 26, 2015, 01:56 PM:

abi@1004:

Let's just go with 987. If you can easily add the strikeout listed in 994, fine, otherwise we'll catch it in the next draft.

Thanks much!
kilo

#1008 ::: abi ::: (view all by) ::: May 26, 2015, 02:15 PM:

Done! Thank you so much for running this, Keith, and thank you to the other participants for the interesting and detailed discussion.

#1009 ::: Jameson Quinn ::: (view all by) ::: May 26, 2015, 02:34 PM:

Here's the link to the new thread. You should also close this one.

#1010 ::: abi ::: (view all by) ::: May 26, 2015, 02:36 PM:

Good point, Jameson.

Choose:
Smaller type (our default)
Larger type
Even larger type, with serifs

Dire legal notice
Making Light copyright 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020 by Patrick & Teresa Nielsen Hayden. All rights reserved.