Back to previous post: Hugo IV: A New Hope

Go to Making Light's front page.

Forward to next post: Dysfunctional Families: Boundaries

Subscribe (via RSS) to this post's comment thread. (What does this mean? Here's a quick introduction.)

May 12, 2015

Discussing Specific Changes to the Hugo Nomination Election: A Post Not By Bruce Schneier
Posted by Abi Sutherland at 02:36 AM *

After a couple thousand posts, here’s the current proposal, summarized in this comment by Keith “Kilo” Watt.


- o0o -


[Remaining Issues “To-Do” List:]

  • Name of the system
  • How to handle ties
  • How to handle withdrawn nominations
  • Best way to present at the business meeting
  • The final formal proposal language itself

[Plain-Language Explanation of SDV-LPE]

Least Popular Elimination (formally called “single divisible vote with least popular eliminated” or SDV-LPE for short) is very simple and straightforward.

- You have one nomination “vote”, which we’ll call one “point” to avoid confusion.

- You can distribute that nomination “vote” among as many works as you feel are Hugo-worthy, and it will get divided among them equally. So, if you nominate two works, each gets half a point, if you nominate three works, each gets one third of a point, etc.

- All the points for each work from all the ballots submitted are added together, and the two works that got the least number of points are compared with each other. One of these works is the least popular and will be eliminated.

- For those works that are eligible to be eliminated, we compare the total number of nominations they each received (that is, the total number of times that work appeared on anyone’s nomination ballot). The work that received the fewest number of nominations is the least popular and now completely vanishes from the nomination process as though it never existed.

- We start over for the next round, and repeat the process, however, if one of your nominations was eliminated, you now have fewer works on your nomination ballot — so each one gets more points since you aren’t dividing your vote among as many works.

Ties (in Points):

- This isn’t really anything different. Basically, if there is a tie for least or second-least number of points, all of those works are eligible to be eliminated.

Ties (in Number of Nominations):

- If two or more works are tied for appearing on the fewest number of ballots, the tied works with the lowest point total will be eliminated. (In other words, if we’re tied for number of nominations, we’ll go back to comparing points for this case.)

- If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, then all members of the tie will be eliminated, unless that leaves the final ballot with fewer than five works; in that case, keep all the members of the tie. (In other words, if we’re tied in both comparisons, in general we’ll just eliminate all the tied works.)

Why does this help? In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn’t penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate -anything- you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don’t, that’s okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

How does this eliminate slates? It doesn’t, not completely — nor should it, as slate proponents have as much right to representation as any other voter. However, by nominating a full slate of works, they have weakened their votes by spreading their points out among the five works on the slate. Since the rest of fandom is slowly increasing the points given to their choices, it’s not possible for slates to control the -entire- ballot. It is very likely, however, that they will get some of the final ballots slots — and again, this is entirely fair and appropriate, since a large number of people are supporting those works. In the end, it is the final ballot that will determine the winner — and the voting system for the final winner is completely unchanged by this system. With this system, a significant fraction of the final ballot will have been determined by independent members of fandom.


[FAQ’s]

1. How does this system eliminate slate or bloc voting?

It doesn’t, exactly, nor should a work be automatically eliminated just because it appears on a slate. On the other hand, any slate which nominates a full set of five works will find that each of its nominations only count 1/5 as much. With a large enough support behind the slate (five times as much), the slate may still sweep a category; however, if that many voters support the slate, they arguably deserve to win, and no fair and unbiased system of nomination will prevent that. The answer is, simply, to increase the general pool of voters. Regardless, with SDV-LPE, slates will never receive a disproportionate share of the final ballot, as occurred in the 2015 Hugos.

2. What if there are multiple slates (slate wars, “parties”, etc.)?

As with a single slate, the more works that anyone nominates, the less their votes count for each work. The end result is that even multiple slates are unable to sweep the nominations.

3. What happens if a genuinely popular work is nominated by a group of unrelated people?

If it is genuinely popular, the system will still select that work for the final ballot.

4. What happens if a genuinely popular work also appears on a slate?

Even if it is on a slate, if the work garners support from individuals - particularly if they list it as their only nomination, or with just a few nominations - then the system will select that work for the final ballot.

5. Isn’t it true that any voting system can be gamed (or strategized, etc.)?

Yes, there is a theorem which proves that all voting systems must have inherent flaws. The objective is to choose a system whose flaws are not in an area of concern to the electorate.

6. What are SDV-LPE’s flaws?

In very rare cases, it is possible for eliminating both members of a tie to change the final ballot slightly from what it would be if the tie were broken so that only one member was eliminated. This situation was extremely rare using realistic data, so the statistical probability of how ties are handled affecting the results is extremely small.

7. What are SDV-LPE’s benefits?

Simply put, it reduces the power of bloc voting without eliminating the chance that works appearing on slates will make it to the final ballot. Conversely, it makes it very difficult for slates to prevent non-slate works from appearing on the ballot.

8. Couldn’t slates just recommend a single work for a candidate, and it will automatically appear on the final ballot?

Yes, that is certainly a viable possibility - it’s also completely fair. It does not force any other works off of the ballot, and the final Hugo winner is determined by the same voting process we have always had. Just appearing on the ballot isn’t a guarantee of winning a ballot. However, if a large section of fandom strongly believes that a work deserves a Hugo nomination, then it should, in fact, be represented on the ballot.

9. What happens with a large field with no stand-out favorites when a slate votes?

Even in this case, we were unable to find a simulation in which no non-slate works appeared on the final ballot. Slate works did receive a larger proportion of nomination slots than they did otherwise, however, again, this could be considered a fair and valid result. If there was no general favorite, then voters really had no collective preference.

10. Isn’t this system too complicated for the average voter to understand?

No, it’s actually quite simple and straightforward, both in terms of voter instructions and in how the system operates. Essentially, the total number of points for each work are totaled (and this will usually be the sum of fractional points). These points are used to determine our two candidates for elimination, since the voters felt the least strongly about them. We then look at the number times the two works appeared on any ballot. The work that appeared the least number of times must be the less popular of the two, so is eliminated. This process continues until the five finalists remain. Note that this is not that different from the STV process used in determining the final Hugo winner once the finalists have been selected.

11. I think we should just increase the number of nomination slots on the final ballot to (for example 6), and decrease the number of slots a voter can vote for to a smaller number (for example, 4). Wouldn’t that be simpler and easier?

Unfortunately, this simply means that the largest slate will receive four of the nominations and the next largest will receive the remaining two. It doesn’t solve the problem of forcing works off the ballot that had a chance to win the final election. Keep in mind that SDV-LPE will work with this (or most any other) change as well, so one does not preclude the other.

12. I think we should set up a committee to handle these situations as they occur. The committee would be empowered to add nomination slots or throw out slate-influenced ballots as required.

This could work. The problem is that now you have a small group of people who serve as literal gatekeepers to the Hugo Awards. In spite of the word on the Internet, this has never been the case in the past. Establishing it now means that those groups who believed it existed in the past will now be correct. Ultimately, human judgement is fallible. The fairness of a committee’s decisions will forever be subject to opinion. The end result is that the prestige of the Hugo Awards will forever be tarnished.

13. I think we should use [insert other mathematical voting system].

We considered essentially every applicable type of voting system currently in the literature, guided by two experts in the field. It should be kept in mind, however, that the goals and requirements for choosing a set of representatives in a political situation are different from those for choosing a set of Hugo finalists. Some of these systems do, in fact, have positive properties that speak for them. None of them were as simple or as intuitive as SDV-LPE, yet SDV-LPE meets all of the stated goals for a Hugo nomination system.

14. Won’t SDV-LPE be complicated to code and implement?

Actually, no. One of our non-experts coded a full simulator for the system in a matter of days. A full web-based app would not be much more difficult to handle.

15. Wasn’t this system just designed by Social Justice Warriors to block the Good Stuff?

It is true that much of the discussion for this system occurred on Teresa and Patrick Nielsen Hayden’s “Making Light” discussion board, and it is also true that groups such as the Sad Puppies and the Rabid Puppies consider TNH and PNH to be The Enemy, and therefore completely biased and not to be trusted. Other than serving as moderators, TNH and PNH had no real input in the discussions of the system, however. Those of us who worked on the system were very clear that our goal was not to keep the Sad/Rabid Puppies off of the Hugo ballot and that any system which specifically targets any type of work is inherently wrong and unfair. One of the members of the group is a retired US Naval officer, a combat veteran, a certified Navy marksman, a Christian, and considers Robert Heinlein to be the greatest science fiction author who has ever lived. In short, he is exactly the Puppies’ demographic. But any slate, of any sort, be it a Sad Puppy or a Happy Kitten of Social Justice, breaks the Hugo Award because a small percentage of voters can effectively prevent any other work from appearing on the final ballot. This is a major flaw in the Hugo nomination system, and it is a flaw that must be fixed if the integrity of the award is to be maintained. Politics should play no role whatsoever in whether a work is Hugo-worthy or not.


[Current Draft of Proposal Language]

Short Title: Change Nominations to Single Divisible Vote, Least Popular Elimination

Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.

3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.

3.8.3: Any nominations for “No Award” shall be disregarded.

3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.

3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.

3.8.6: The Committee shall move a nomination from another category to the work’s default category only if the member has made fewer than five (5) nominations in the default category.

3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.9 or 3.2.10, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category.

3.8.8 The final Award ballots shall list in each category the five eligible nominees as determined by successive rounds of elimination, where each nominator gets a single vote, divided equally among their nominations. For example, if only one work is nominated in the category, that nomination gets a full vote. If two works are nominated in a category, each nomination gets 1/2 of a vote. If three works are nominated in a category, each nomination gets 1/3 of a vote, etc.

3.8.8.1 In each round, the two works (or more, in the case of a tie) with the least total number of (fractional) votes will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots.

3.8.8.2 In the event that two or more works are tied for appearing on the fewest number of ballots, the tied works with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, then all members of that tie will be eliminated.

3.8.8.3 If (due to a tie) elimination would reduce the number of ballots to fewer than 5, then instead none of the works should be eliminated and all remaining works appear on the final ballot.

3.8.8.4 For the next round, votes are reassigned to each work as in 3.8.8. (Example: If a ballot contained five nominations originally and one of those nominations is eliminated, each of the four remaining nomination now gets a 1/4 of a vote)

Section 3.11 Tallying of Votes

3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, … places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes. During the same period a record of at least the last fifteen rounds of the selection process for each category shall also be published.

Submitted by: (insert members here, ideally including at least one attending member who will be at all the relevant WSFS meetings. A proposing member gets the privilege of speaking first to the pro side of the proposal)

Commentary: In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn’t penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate -anything- you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don’t, that’s okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.


- o0o -


Jameson Quinn offers:

I’ve created a public Google doc to hammer out a proposal text and FAQ for SDV-LPE.

There have been numerous proposals in this thread which do not fall under the SDV-LPE umbrella. In my opinion, they have turned out to be either off-topic or dead ends. This is not a criticism; several of the proposals were my own, and I thought and still think that they would have been good ideas. But it seems the community is very reluctant to experiment with anything that goes much beyond fixing the immediate problem, and I can respect that.

So I think the remaining order of business is to wrap up the SDV-LPE proposal, either using the doc above, or through other means.


- o0o -


Joshua Kronengold adds the following:

Re name, if we want an acronym that means something, how about:

One Notional Equally-divided Vote, with Oppositional Test Eliminations?

(ie, ONEVOTE)

I’ve always favored “rerun the vote with declined candidates eliminated, but only fill as needed” when people declined as a rule, since it ends up with the fairest overall result (and withdrawing Hugo nominations is a can of worms we never need open). I’d be fine (5 vs 10) with just taking the next item on the list.

Re tiebreakers: I’m reasonably fine with no tiebreaker (as the system was originally conceived), but also entirely fine with the current “just score” version, or using rank as a not-really-random (but close enough while being verifiable) tiebreaker. It’s probably best to just have “just score” as tiebreak despite potential issues, because it makes things much easier to independently verify and keeps things simple.

Comments on Discussing Specific Changes to the Hugo Nomination Election: A Post Not By Bruce Schneier :
#1 ::: Annie Y ::: (view all by) ::: May 12, 2015, 03:23 AM:

About withdrawn and not- accepted nominations an(because they are different things even if treated similarly) and works found ineligible after the ballot is officially declared - so essentially needing a replacement:

- Do not select 5 but instead select 15 or just run the whole elimination to the bitter end (if automated, this won't take that long). The top 5 get nominated (or more in case of ties); if someone withdraws/does not accept/is found ineligible - use the next on the list.

This way we do not need to rerun eliminations later on and a replacement is clear as soon as one is needed... and the top 15 are easily published from the Worldcon after the vote (whatever the number that is published is)...

Which brings the question - when top 15 are published, will it be based on points or based on nominations? Or maybe we need both?

That will be a bit unfair in case a work is pulled out for ineligibility (as it still can harm another) but makes it easier...

Just thinking aloud (and now copying in the other thread as the old one closed while I was typing)

#2 ::: Martin Wisse ::: (view all by) ::: May 12, 2015, 04:42 AM:

So if I nominate 1 candidate for best novel, I get one point and a slate voter who has nominated five candidates gets 1/5 point per nomination.

Okay, that makes sense. But it doesn't distinguish between people slate voting and those who naturally nominate five choices in a given category (and I could've nominated more this year). Not a problem if you assume slate voting isn't the issue, slate dominating is.

Which I'm not sure I agree with but is perhaps the best practical approach to take.

What I'm struggling with is the assumption that in lesser voted categories slate voters would be at a disadvantage because their candidates have less points. Is the assumption here that in such a situation the odds are high two slate noms would be in the least popular situation and either one of them or both are eliminated, freeing up space for non-slate votes?

#3 ::: influxus ::: (view all by) ::: May 12, 2015, 04:58 AM:

Thanks for doing this. It sounds really promising. Regarding faq 13 & 14, is it possible to release the simulations and code?

#4 ::: Brian Z ::: (view all by) ::: May 12, 2015, 05:01 AM:

Its 1970. TV won’t shut up about My Lai and Cambodia. You get up to go turn down the volume, bumping into the record player and making the first Black Sabbath album skip. Postman’s here. You forgot to lick the stamp on your ballot of Ubik by Philip K. Dick, The Black Corridor by Michael Moocock, The Andromeda Strain by Michael Crichton, Frank Herbert’s Dune Messiah and hey, it seemed kinda nifty at the time, Macroscope by Piers Anthony. Result: your five votes all go to Piers Anthony.

Not sure I can support this one.

#5 ::: influxus ::: (view all by) ::: May 12, 2015, 05:07 AM:

@2: The assumption is that slates dominate because other people's votes are too diluted across a wide field. As the works are successively eliminated everybody who still has books in the field finds their nominations converging towards a vote each on a work each. So, eventually a consensus should emerge that is capable of counter-balancing any slates. And if a consensus doesn't emerge, then it seems somewhat fair that the slate should win.

#6 ::: felice ::: (view all by) ::: May 12, 2015, 05:25 AM:

Annie Y @1: "- Do not select 5 but instead select 15 or just run the whole elimination to the bitter end (if automated, this won't take that long). The top 5 get nominated (or more in case of ties); if someone withdraws/does not accept/is found ineligible - use the next on the list. This way we do not need to rerun eliminations later on and a replacement is clear as soon as one is needed..."

The problem with that is if a popular work is withdrawn for whatever reason after the finalists are chosen, the votes of all the people who supported it are still weighted less for their other nominations, which means their second choices are less likely to be in 6th place waiting to replace it. Rerunning the process without the withdrawn work and taking the highest-ranked new work to fill the vacancy is the fairest practical option.

"Which brings the question - when top 15 are published, will it be based on points or based on nominations? Or maybe we need both?"

At a minimum, I think we'd need to publish the points and nominations for the round which contains 15 nominations; perhaps all the subsequent rounds that whittle it down to the final five, as well.

That does raise a small issue with “rerun the vote with declined candidates eliminated, but only fill as needed” - if rerunning after a withdrawal changes more than one finalist, which is possible (but probably uncommon), we'd need to publish both sets of data, and that will show who would have won a Hugo if the withdrawn work had been withdrawn before the initial selection of finalists. That could be messy. So maybe just take the next on the list is a better option, even if it's not as fair.

An example: if the winners are ABCDE, with F being the last eliminated, and B is withdrawn: the simple approach is to just promote F, for a revised ballot of ACDEF. With "rerun and fill as needed", the rerun could give us CAHDE, and a new final ballot of AHCDE (H filling the vacant spot left by B), which is fairer (the people who nominated B like H more than they like F). But the rerun could also give us CAHJK; with H as the highest ranked new result, we still get a new final ballot of AHCDE, but we know J and K should be there instead of D and E (because the votes from B were split between C, H, J, and K, improving the ranking for all of them). It's not fair to D and E to tell them "sorry, you're not finalists any more", but it's also not fair to tell J and K that they should have been finalists but aren't. Basically, withdrawing a work after the finalists have been notified is a bit of a nightmare.


Martin Wisse @2: "Okay, that makes sense. But it doesn't distinguish between people slate voting and those who naturally nominate five choices in a given category (and I could've nominated more this year). Not a problem if you assume slate voting isn't the issue, slate dominating is."

Slate voting is a problem, but trying to prevent it is a real can of worms. Slate dominating is a bigger problem, and one that it's easier to do something about. And it will probably lessen the incentive for slate voting as a bonus.

"What I'm struggling with is the assumption that in lesser voted categories slate voters would be at a disadvantage because their candidates have less points. Is the assumption here that in such a situation the odds are high two slate noms would be in the least popular situation and either one of them or both are eliminated, freeing up space for non-slate votes?"

Yes. But it's still possible for a slate to get multiple slots in a category with relatively few non-slate voters and few clear favourites (eg Short Story). SDV-LPE helps a lot, but it doesn't totally solve the slate problem on its own. But it's a good system even aside from the slate-fighting benefits.

Brian Z @4: "Result: your five votes all go to Piers Anthony."

Um... how?

#7 ::: Mike Scott ::: (view all by) ::: May 12, 2015, 05:32 AM:

Having struggled with my own proposal to change the threshold for eligibility to increase the number of nominees in categories where it looks like there's been slate activity, I support this proposal. It's better than mine. I would also lose the 5% rule, which is damaging the short story category since there are such a lot of short stories these days, but I believe there will be a separate proposal to do that. Sadly, I won't be at Sasquan to vote for it.

#8 ::: Annie Y ::: (view all by) ::: May 12, 2015, 05:33 AM:

felice@6

The problem with a rerun is calling F and telling them - "well, you are not nominated anymore because someone withdraw/we found someone else is ineligible". This is why I am for a straight "Pick the full order and stick to it". Yes - a popular choice withdrawal will move things around but do we really want to go down the "you are nominated, you are not" rigmarole? (and what are we doing if someone withdraws after the ballot is out - we change 3 places in the ballot because A decided they want out now? Or because it was found ineligible (that also can happen))? As fair as it is mathematically, it will cause way too much noise and headaches for any administrators.

#9 ::: Brian Z ::: (view all by) ::: May 12, 2015, 05:38 AM:

felice @6: Um... how?

if one of your nominations was eliminated, you now have fewer works on your nomination ballot — so each one gets more points

#10 ::: Annie Y ::: (view all by) ::: May 12, 2015, 05:42 AM:

Brian Z @9
But in your example, you did not mail your ballot at all? So you have 0 points to share around. How does a full point end up being for Anthony?

#11 ::: Brian Z ::: (view all by) ::: May 12, 2015, 05:51 AM:

Annie Y @ 10: the dog barked, jolting me out of my Toni Iommi trance, and I suddenly remembered that the stamps were sitting on the entry table. Phew!

#12 ::: felice ::: (view all by) ::: May 12, 2015, 06:06 AM:

Annie Y @8: "The problem with a rerun is calling F and telling them - "well, you are not nominated anymore because someone withdraw/we found someone else is ineligible". This is why I am for a straight "Pick the full order and stick to it". Yes - a popular choice withdrawal will move things around but do we really want to go down the "you are nominated, you are not" rigmarole? (and what are we doing if someone withdraws after the ballot is out - we change 3 places in the ballot because A decided they want out now? Or because it was found ineligible (that also can happen))? As fair as it is mathematically, it will cause way too much noise and headaches for any administrators."

Yeah, on reflection I think sticking with F is probably the only practical option. The "rerun and fill as needed" solution only delays the "you are nominated, you are not" rigmarole rather than avoiding it. Thanks for pointing that out!


Brian Z @9: "if one of your nominations was eliminated, you now have fewer works on your nomination ballot — so each one gets more points"

If Ubik, The Black Corridor, The Andromeda Strain, and Dune Messiah get eliminated before Macroscope, sure. That means more other people liked Macroscope; and it made the final ballot under the current system too, so you're hardly any worse off. If you really didn't want Macroscope to be a finalist, you shouldn't nominate it; but it's highly unlikely that leaving it off your ballot and only nominating the other four would make any difference (the last one left will be half a point higher up in the rankings, since your point is no longer split between it and Macroscope), and you'll probably end up with nothing you nominated on the final ballot. It is theoretically possible that that half point could mean the difference between 5th and 6th place for one of the other four, but it's quite improbable.

#13 ::: Annie Y ::: (view all by) ::: May 12, 2015, 06:15 AM:

Brian Z,

Great. So how does the point go to Anthony now with your ballot with 5 nominees? The only way to happen is if the other 4 get eliminated before Anthony - the same way as Anthony will get nominated under the current system only of these 4 are under it in the total ballots.

So... what am I missing in your example?

#14 ::: influxus ::: (view all by) ::: May 12, 2015, 06:32 AM:

Is it possible to handle withdrawals as follows?

Give authors a chance to withdraw when you reach, say, the final 13. Then if any do, rerun from scratch, call any new members of the final 13. When you have 13 willing nominees, finish the process.

#15 ::: Annie Y ::: (view all by) ::: May 12, 2015, 06:35 AM:

And how do you propose to deal with ineligibility found later? Or authors that decide that they changed their mind - for whatever reason?

#16 ::: influxus ::: (view all by) ::: May 12, 2015, 06:40 AM:

Annie Y,

As you suggested, keep it fixed once formally announced.

#17 ::: Brian Z ::: (view all by) ::: May 12, 2015, 06:47 AM:

felice @12

It is theoretically possible that that half point could mean the difference between 5th and 6th place for one of the other four, but it's quite improbable.

I hope it is improbable enough.

The Russians land a probe on Venus. WorldCon's done and "My Sweet Lord" is on the radio. Your next LoC goes "Ubik was trippy and all, but I mean, I'm glad I gave Piers Anthony all five of my votes. Maybe I'm the cat who got him onto the final ballot. He is every bit as good as Dick, Moorcock, and Herbert, probably better. If I could do it over I'd nominate Macroscope and nothing else. Piers Anthony is a genius and shouldn't have to compete with those has-beens. Next year I'll nominate only Ringworld. Larry Niven is practically the all-time greatest writer in history, second only to Piers Anthony." It is 2015 and you find that 'zine while cleaning your closet. Do you regret it?

#18 ::: Brian Z ::: (view all by) ::: May 12, 2015, 07:07 AM:

Self @ 17,

Come to think of it I remembered that wrong. The LoC actually said "Next year I will nominate only I Will Fear No Evil, since Mr. Heinlein literally can do no wrong."

#19 ::: albatross ::: (view all by) ::: May 12, 2015, 08:09 AM:

I love the way fandom's response to this mess is itself an SF trope--let's spin up a new technology that solves the problem.

#20 ::: J Thomas ::: (view all by) ::: May 12, 2015, 08:17 AM:

@6 felice

An example: if the winners are ABCDE, with F being the last eliminated, and B is withdrawn: the simple approach is to just promote F, for a revised ballot of ACDEF.

That is the simplest, but not much simpler and is unfair to the actual winners.

With "rerun and fill as needed", the rerun could give us CAHDE, and a new final ballot of AHCDE (H filling the vacant spot left by B), which is fairer (the people who nominated B like H more than they like F).

Yes. Of course the most likely result is that you get CAFDE and nothing is changed. But it could be different and then you get the better result.

But the rerun could also give us CAHJK; with H as the highest ranked new result, we still get a new final ballot of AHCDE, but we know J and K should be there instead of D and E (because the votes from B were split between C, H, J, and K, improving the ranking for all of them).

That could rarely happen.

It's not fair to D and E to tell them "sorry, you're not finalists any more",

But in fact they are not finalists any more. Better, when you call them to ask them whether they would decline, don't tell them they are finalists -- because after all, it isn't final. Ask them, "If it were to turn out that you were a finalist, would you accept?". Remind them that it would be premature to tell anyone they are a finalist, since it isn't true yet and might not work out. Then it isn't unfair to them in the rare case that it turns out otherwise.

but it's also not fair to tell J and K that they should have been finalists but aren't.

Yes.

Basically, withdrawing a work after the finalists have been notified is a bit of a nightmare.

It doesn't have to be. Just do the right thing, and it all works out.

That does raise a small issue with “rerun the vote with declined candidates eliminated, but only fill as needed” - if rerunning after a withdrawal changes more than one finalist, which is possible (but probably uncommon), we'd need to publish both sets of data, and that will show who would have won a Hugo if the withdrawn work had been withdrawn before the initial selection of finalists. That could be messy. So maybe just take the next on the list is a better option, even if it's not as fair.

If the winners of the last rerun are the winners, you can publish that version. Add a note that B withdrew. The more different versions you publish, the more people will be upset with the voting system after it's in use.

"If B had withdrawn 2 days earlier my favorite would have come in second, but instead it's off the ballot completely. What kind of crazy voting is this?"

Usually it won't be a problem because we'll get the same result with any of the three different methods.

But in the rare case that they're different, we do best by announcing the actual winners, and publish the numbers that made them the winners.

We can expect enough trouble already from people who look at those numbers.

"Hey, Slaughterhouse-five and Bug Jack Barron both got way more votes than Macroscope. How come Macroscope was a finalist and they weren't?"

"Practically everybody who voted for Bug Jack Barron also voted for Slaughterhouse-five and The Left Hand of Darkness. That proves they didn't care much about those, and they got part of what they wanted when The Left Hand of Darkness won. There weren't as many people who voted for Macroscope, but almost all of them voted for just that and nothing else. So it deserved to win."

It may take some people awhile to see that this is the best way.

#21 ::: junego ::: (view all by) ::: May 12, 2015, 08:31 AM:

The Google docs app isn't playing well with my iPad, so I'm putting my suggestions/votes/comments here.

I have been following this discussion almost from the beginning and want to commend and congratulate everyone who has participated. My non-expert opinion is that this is an understandable and viable fix for the nominating process. Not perfect, but probably good enough.

1) Suggest changing the explanation for the system from 'one vote (per category) that is divided' to '100 points (per category) that is divided' equally between all of a person's nominations in that category.

This is mathematically identical to SDV but may be more understandable/acceptable to the math or fraction phobic.

The current system allows up to 5 undifferentiated recommendations per category. I think translating that to 100 points per category divided equally is less jarring/more intuitive than reducing the whole category to 1 "vote" or point that is divided.

2) I also suggest that it would be better to de-emphasize "vote" language as much as possible and replace it with nomination, recommendation, choices, etc. Using the v-word may tend to stir subconscious cultural themes like "one person/one vote" combined with "hey, I don't want my vote DIVIDED!", which might add to resistance to the proposal. (I had this experience when changes were first proposed but had time to logic my way out of the knee-jerk response. There won't be as much time for people to get past this kind of thing at the business meeting.)

3) I may have missed it in the proposed explanation but in case I didn't, I suggest adding language that emphasizes that this proposal DOES NOT CHANGE what a member has to do to submit their nominations. It only changes how the counts are tallied.

Again, thanks to everyone for working so diligently on the proposal.

#22 ::: Brian Z ::: (view all by) ::: May 12, 2015, 08:37 AM:

J Thomas @ 20

"... There weren't as many people who voted for Macroscope, but almost all of them voted for just that and nothing else. So it deserved to win."

It may take some people awhile to see that this is the best way.

It may depend on what they see as the ultimate goal of the exercise.

#23 ::: Cassy B. ::: (view all by) ::: May 12, 2015, 09:12 AM:

junego @21, While I agree with all your points, I'd recommend 60 points-per-category-to-be-divided-equally, not 100. No fractions that way. One work on a ballot=60 points. Two works=30 points each. Three=20 points each. Four=15 points each. Five=12 points each.

Simple, no fractions, no rounding errors.

#24 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:20 AM:

@3: I am not the only person who has coded this up, but my code (and data) is visible at https://github.com/The-Center-for-Election-Science/HugoVotesim. I did it in R, the statistical computing environment. I realize that R is not at all the most accessible language for non-statisticians, and I'm sorry about that, but it would have taken more work to do bootstrapping in python.

I found:
1. SDV-LPE is reasonably similar to the current system; we can expect the nominee lists to overlap by about 4.5 out of 5 candidates on average.
2. If you break ties by points, then by previous round points, etc., you will essentially never have to fall back to eliminating all tied works, except early in the process for works with only a few votes each, which are going to be eliminated anyway. "Essentially never" means less than 1 in 1000 elections.
3. If somebody declines a nomination, running the system again from scratch will change one of the other 4 existing nominations only about 1 time in 100.

#25 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:38 AM:

@20: I don't think we should talk about Macroscope hypothetically "winning", but just about it getting a nomination. In the scenario you paint, it's pretty clear that The Left Hand of Darkness (or something else non-Xanthan) would get the actual Hugo. So Piers Anthony would get to go to the Hugo losers party, but that's all. (In fact, I suspect that Vonnegut and Spinrad would get invited too, because you can never have too many losers at the party, especially when we're talking losers like them.)

#26 ::: influxus ::: (view all by) ::: May 12, 2015, 09:39 AM:

Jameson Quinn @ 24:

Thank you, that's awesome.

Hey R is powerful, fit for purpose and free. I'm not complaining.

It would be good (for someone) to put a web app up (somewhere) if this proposal is getting broad acceptance as a viable reform.

#27 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:45 AM:

@23: Must... resist... bringing up my more complicated proposal...

Oh, what the hell.

From a voting-theoretic point of view, it would be even better if one work got 20 points, two got 8 points each, three got 4 each, four got 2 each, and 5 got 1 each. These numbers no longer add up to the same thing, but they approximate the Saint-Lagüe weights, and thus give better resistance to naive slate voting.

But that's hard to explain, so I think that your 60 point idea is better as a serious proposal.

#28 ::: J Thomas ::: (view all by) ::: May 12, 2015, 09:45 AM:

@23 Cassy B.

I'd recommend 60 points-per-category-to-be-divided-equally, not 100. No fractions that way. One work on a ballot=60 points. Two works=30 points each. Three=20 points each. Four=15 points each. Five=12 points each.

Simple, no fractions, no rounding errors.

That looks very good. My only concern is that while it works perfectly and it makes good sense, we are trying to come up with a description that will make sense to people who are not paying attention.

We have seen earlier that people who are not interested in the details prefer simple wrong explanations.

So our challenge is to find a simple overview which is not wrong, but which superficially looks so obvious that people don't notice that there are important details they are missing.

Maybe the important points to get across are:

1. It's entirely a process of elimination. We remove losers until there are no losers left to remove.

2. Each loser is selected because it had fewer votes than another nominee.

3. Nominees that have more shared ballots get more opportunities to be removed.

Since slates are maximally shared, they get lots of chances to be removed and it's harder for them to dominate the final results.

I'm not clear how to make a correct superficial short description of this, and I'm not certain it's what needs to be said.

#29 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:49 AM:

@24: "1. SDV-LPE is reasonably similar to the current system; we can expect the nominee lists to overlap by about 4.5 out of 5 candidates on average."

I meant, "on average with historical, mostly-slate-free voting patterns". If there are slates, the slates would get much less traction with SDV-LPE and so the overlap could be as small as 1.

#30 ::: Brian Z ::: (view all by) ::: May 12, 2015, 09:53 AM:

@ 25 Jameson Quinn

Spinrad would have been unable to get an invite to the losers' party whether Piers Anthony knocked him off the ballot or not, because it didn't exist yet.

If A Spell for Chameleon had knocked off George R.R. Martin's Dying of the Light in 1978, there might not be be a losers' party today.

#31 ::: J Thomas ::: (view all by) ::: May 12, 2015, 09:59 AM:

@25

In the scenario you paint, it's pretty clear that The Left Hand of Darkness (or something else non-Xanthan) would get the actual Hugo.

In reality, The Left Hand of Darkness did get the actual hugo, and Macroscope came in third. Slaughterhouse-Five and Bug Jack Barron came in fourth and fifth.

I think this was before Xanth. Piers Anthony put his heart into writing Macroscope and people didn't like it that much. Then he found he could crank out Xanth books and get a large steady income, and he settled for that.

I don't know whether any of those would get on the short list the new way, it would depend on the individual ballots.

#32 ::: junego ::: (view all by) ::: May 12, 2015, 10:11 AM:

Cassy B. @ 23
junego @21, While I agree with all your points, I'd recommend 60 points-per-category-to-be-divided-equally, not 100. No fractions that way. One work on a ballot=60 points. Two works=30 points each. Three=20 points each. Four=15 points each. Five=12 points each. Simple, no fractions, no rounding errors.

Sure, that works for me.

#33 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:17 AM:

Hi all, and welcome to the new voices!

I'm really glad to get some more input on the proposal and the system. I'm diligently taking notes on everyone's thoughts and suggestions. Completely optional, but if you want to make sure I don't miss a suggestion, you could put a header at the beginning of your message like [TIES], or [DELCINED NOMINATIONS] or whatever. Totally up to you, and probably not necessary for general conversation.

I wrote the code for the current simulator, but I'm not much of a web programmer or database specialist (I was a NASA engineer, so spacecraft programming I grok). I'd be glad to work with anyone who wants to make a web app though -- I can provide the algorithm if someone else can handle the back end. I don't think it's a high priority unless the proposal actually passes, though.

Until that time, I'd be glad to run any situations that people might be curious about. We now have the 1984 Hugo ballots that we can use as a base, plus some random data based on 2013. As Jameson mentioned, the system not actually all that sensitive to small changes -- usually you get the same results unless things were very close to begin with (which makes sense in any case). It is possible to manufacture unusual situations, so one of the previous discussions was engaged in how much we should worry about them. Jameson's simulations showed that those situations are quite rare indeed.

I'll respond to specific questions in another message.

Thanks,
Kilo

#34 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:22 AM:

For a name: how about Proportionally Allocated Pairwise Elimination (PAPE)? It's pronounceable, descriptive, and short. It's a bit technical-sounding, but that's not necessarily a bad thing.

Whichever name you favor, please vote on name, tiebreaker, and declined-nomination replacement procedure. We're using range voting for this poll, so rank each option 0-10. You can vote either here or there, as long as you give some recognizable nym with your vote there.

Here are the options again:

Name:
Single Divisible Vote, Least Popular (SDV-LP)
Single Divisible Vote, Least Popular Elimination (SDV-LPE)
Single Divisible Vote (SDV)
Least Popular Elimination (LPE)
Hybrid Divisible Vote
Quantum Divisible Vote
Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
Bottom Up Elimination (BUE, pronounced “byoo”)
One Notional Equally-divided Vote, with Oppositional Test Eliminations (ONEVOTE)
Proportionally Allocated Pairwise Elimination (PAPE)

My vote (edit with your ratings):

9: (SDV-LPE)
5: (SDV)
7: (LPE)
7: Hybrid Divisible Vote
3: Quantum Divisible Vote
5: (BUMPE)
5: (BUPE, pronounced “boopy”)
9: (BUE, pronounced “byoo”)
7: (ONEVOTE)
10: PAPE

....

Tiebreakers:

(T0) Eliminate all tied works (current version)
(T1) Break ties by score
(T1a) Scores from previous rounds may be used if the current round is not decisive
(T2) Random.
(T3) Earliest Ballot
(T4) Some other ranking form.

My vote:
10: T1a then T0.
5: anything else nonrandom
0: anything random

....

Declined nominations:

Options:


(DN1) Take the 6th place item.
(DN2) Recalculate the election without the declined entry and those are the new nominees. (Full recalculate)
(DN3) Recalculate the election without the declined entry, and add the first winner that is not yet on the ballot. (Incremental recalculate)

My vote:

10: DN1, 6th place
0: DN2, Full recalculate
5: DN3, Incremental recalculate

#35 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:27 AM:

ah, crap. I said this informal poll was using "range voting", I should have said "score voting". They're the same thing but the latter term is preferred these days. I sometimes forget.

#36 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:28 AM:

Regarding declined nominations:

For each round, the code reports the current surviving works, their current point total, and the number of nominations they received. It then reports which works were eliminated that round, along with their point total and number of nominations. It makes it pretty easy to see how the process went.

My intent for section 3.11.4 is to just publish the last fifteen rounds (which will generally also have the last 15 works that were elminated, barring ties). In this way everyonje can see how we eventually converged to a final set of works. I think this actually is easier and more informative than just listing the top 15 works as we do now. Of course, I personally think the whole text file of the process from beginning to end should be released, but that's up to the administrators and is what the "at least" language is for.

I don't see any inherent issues with just taking the works from the previous rounds. Some here think that is less fair, but consider that the voters' decisions were made assuming the declined work was actually an option. We have no way of knowing how they would have nominated if that work wasn't available, so I don't see any advantage to re-running the process, even in the very rare cases that it might make a difference.

As I read the current constitution, section 3.9.1 just says that the work is removed from the final ballot -- there is currently no explicit requirement to add the next work to the list, so I believe the administrators have the option to handle it however they want. Perhaps one of the experienced admins can confirm that. My thoughts are to just leave that part of the constitution unchanged. It's simpler, one less change to pass the business meeting, and pretty much solves all the issues. I would recommend to any Hugo admin to just take the work eliminated in the previous round, but it seems to me the constitution lets them do as they see fit.

Kilo

#37 ::: junego ::: (view all by) ::: May 12, 2015, 10:29 AM:

Jameson Quinn @ 27
@23: Must... resist... bringing up my more complicated proposal... Oh, what the hell.
From a voting-theoretic point of view, it would be even better if one work got 20 points, two got 8 points each, three got 4 each, four got 2 each, and 5 got 1 each. These numbers no longer add up to the same thing, but they approximate the Saint-Lagüe weights, and thus give better resistance to naive slate voting.

Yeah, I understand what your getting at but from a NOMINATOR standpoint I don't necessarily want to order my choices this early in the process. I'm mostly just recommending the coolest things I read/saw/heard last year. I could just nominate fewer items in the category to express preferences, but that doesn't satisfy your "theoretical" standpoint. 😋

But that's hard to explain, so I think that your 60 point idea is better as a serious proposal.

I don't think it's that hard to explain. IIRC, the main complaint in the previous discussion was more along the lines of my above. There is and was support for ranking nominations, but it seems to be less than those just wanting to nominate without.

#38 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:31 AM:

Jameson@34: By the way, my phone and tablet don't like Google docs much (it runs, just not in an easy-to-read fashion for aging eyes), so I would actually prefer that people post here, if that's okay. It makes it easier for me (and I would suspect others as well) to stay on top of what's happening. I do agree that it would streamline changes and make note-taking easier, but I'm often squeezing in time as I can.

Kilo

#39 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:33 AM:

junego@37: I think the concensus was that people want to rank the final voting, but that for nominations, we essentially want a controlled version of "throwing a name into a hat". Basically nominate anything that people think are Hugo worthy. The final voting then ranks which of those are the best.

Kilo

#40 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:40 AM:

@38: Would Google Docs work for you if we increased the font size? Those of us not on tablets could easily deal with a larger font.

#41 ::: DonBoy ::: (view all by) ::: May 12, 2015, 10:40 AM:

Delurking:

In FAQ 8, I think

Just appearing on the ballot isn’t a guarantee of winning a ballot.

is supposed to be

Just appearing on the ballot isn’t a guarantee of winning a Hugo.

#42 ::: Niall McAuley ::: (view all by) ::: May 12, 2015, 10:42 AM:

Keith, one wording issue. I think it would be better to re-order the first two points to emphasize that nominations are in the same form (for the nominator) as they are now. Many other possible systems require that nominators list nominations in preference order, and I think a strength of this proposal is that it does not:

- You can nominate as many works as you feel are Hugo-worthy, in the same way as you can nominate 5 works today, as an unsorted list.

- You have one nomination “vote”, which we’ll call one “point” to avoid confusion, and it will get divided among them equally. So, if you nominate two works, each gets half a point, if you nominate three works, each gets one third of a point, etc.

#43 ::: Jim Van Verth ::: (view all by) ::: May 12, 2015, 10:42 AM:

@20:

Having seen first-hand the stress that Hugo and Campbell nominees go through during awards season, I think calling people up and asking them "If you were given a nomination, would you accept?" is darn cruel, especially if you're that 1 in 100 person who got dropped off the list when the algorithm is re-run. Sure, it's rare, but it can happen.

As previously suggested, I rather see the algorithm run once to create a ranked list, and work down it when people decline. It may not be 100% mathematically fair, but it's kinder.

#44 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 10:43 AM:

[DECLINED NOMINATIONS] I strongly prefer the simple way of handling these: if ABCDE are nominated and then E withdraws, leave ABCD alone and promote F. My reason for preferring this is as follows:

The election (for nominees) is an algorithm that takes a pile of preferences in voters’ heads, translates those preferences into a pile of ballots, and translates those ballots into a ranked list of works in each category (with the first-eliminated candidates at the bottom of the list). Once that list is generated, the election is over and we should handle declined or disqualified nominees without regenerating the list.

It’s like, if Clinton and Bush run against each other in a Presidential election, and Clinton wins, but then says “screw this, I don’t want to be President after all”, the voters don’t get to run back to the polls and try again.

Also, if ABCDE are told they are the nominees, and E declines, and the algorithm is rerun, and D gets kicked off the revised ballot, then... I don’t want to see what happens when the authors of D and E happen to be in the same room.

#45 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:45 AM:

@Actually, it's 1 in 100 times, so that's 1 in 500 nominees. But yeah, I still agree with your point.

#46 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 10:55 AM:

I don't really have anything substantial to add, just a personal perspective: I have learning disabilities, thanks to neurological side effects of an autoimmune condition, and this is genuinely impenetrable to me. Not "this is a lot of work to understand", but "I am not able to grasp these details after any amount of effort in a short period of time". Maybe with years of repetition it would sink in, and maybe not.

(If I'd studied these things before that neurological damage developed, I could run on memories and suitably adaptive pathways. But I seem not able to learn new stuff involving a bunch of different kinds of math.)

So I'm in the position of having to go entirely on trust of others. That's scary and uncomfortable. Nothing about this is easy or good for my sense of security. I hope that when advocates work to round up support, you will remember that people like me exist, along with others who aren't going to quickly - or maybe ever - master the math for themselves, either.

I don't understand how this system doesn't end up rewarding single-choice voters throughout. It seems like that would be the sensible thing to do for anyone wanting Puppies-like domination: forget the "rule the nominations" part and just focus laser-like on a single pick in every category and carry the day. And it's deeply distressing that I can't make sense of any explanation about why that wouldn't happen, or why I shouldn't be bothered if it does.

#47 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 10:56 AM:

[WORDING] A phrase I would like to see, somewhere in the explanation or FAQs, is “the greatest good for the greatest number”. Relative to the current system, SDV-LPE increases the number of voters who can expect to see at least one of their preferences on the final Hugo ballot. (It decreases the number of voters who can expect to see more than one of their preferences on the ballot, but I think that’s a fair trade-off.)

#48 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 10:56 AM:

A phrase I would like to see, somewhere in the explanation or FAQs, is “the greatest good for the greatest number”. Relative to the current system, SDV-LPE increases the number of voters who can expect to see at least one of their preferences on the final Hugo ballot. (It decreases the number of voters who can expect to see more than one of their preferences on the ballot, but I think that’s a fair trade-off.)

#49 ::: junego ::: (view all by) ::: May 12, 2015, 10:59 AM:

Jameson Quinn @24

I found:
1. SDV-LPE is reasonably similar to the current system; we can expect the nominee lists to overlap by about 4.5 out of 5 candidates on average.
2. If you break ties by points, then by previous round points, etc., you will essentially never have to fall back to eliminating all tied works, except early in the process for works with only a few votes each, which are going to be eliminated anyway. "Essentially never" means less than 1 in 1000 elections.
3. If somebody declines a nomination, running the system again from scratch will change one of the other 4 existing nominations only about 1 time in 100.

Thank you for the stats.

I'm not sure what you mean at 2) about "break ties by points, then by previous round points"?

Your 3) finished convincing me that the new system should freeze the finalist list when the public announcement is made and any further withdrawals should be filled by either a) having frozen the entire list and taking next in line without recalculating or b) recalculating only for the "6th" spot. The nightmare (and "unfairness") of throwing a finalist off the list after the public announcement outweighs the "unfairness" of whoever might have replaced them if someone else hadn't withdrawn.

If the rule is clearly stated it will make the choices unambiguous. Well, hopefully...this is fandom after all 😆

#50 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 11:02 AM:

(sorry for the double-post above)

Re Bruce Baugh’s comment, I am wondering if someone with the appropriate skills can describe SDV-LPE in a graphic format, for the benefit of people who find pictures clearer than words.

#51 ::: Tim Illingworth ::: (view all by) ::: May 12, 2015, 11:07 AM:

Um. I was working through the proposed language to make it more consistent with the rest of the Constitution, and the first sentence of 3.8.8.2 puzzles me:

In the event that two or more works are tied for appearing on the fewest number of ballots, the tied works with the lowest point total will be eliminated.

But we're already looking at works with the same point total - that's how we picked them in 3.8.8.1. What am I missing?

#52 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:12 AM:

A diagram would definitely be helpful for some of us, yes. Thank you, Seth.

#53 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:12 AM:

@46: Thanks for adding your point of view.

The answer is "that wouldn't happen", not "that would actually be OK". The simplest explanation I can come up with is below. I realize it may not be simple enough. But explanation or no explanation, we have simulations to show that it wouldn't happen in any elections that are even vaguely Hugo-like. So I hope that if you can't trust the explanations, you can trust the sims, or trust the people trying to explain it.

Here's my explanation: Looking at actual Hugo ballots, we see that most voters only vote for 1 or at most 2 of the eventual nominees. In fact, the average is less than 1. That means that, by the end of the process, ordinary nonstrategic voters end up giving the same "points" to the works they like as your hypothetical bullet voters, even in the "matchup" (two lowest points) phase.

Furthermore, from the very beginning of the process, strategic and nonstrategic votes count equally in the "showdown" phase (eliminate whichever has fewest approvals). In fact. insofar as your hypothetical bullet voters had factional divisions, they would actually be at a disadvantage, not an advantage, over honest voters.

So bullet voters have no advantage when it counts, and even when it doesn't matter, they actually have disadvantages along with advantages.

I hope that some part of that is clear enough to help. If not, sorry; it's the best I can do.

#54 ::: junego ::: (view all by) ::: May 12, 2015, 11:14 AM:

Annie Y @8
felice@6
The problem with a rerun is calling F and telling them - "well, you are not nominated anymore because someone withdraw/we found someone else is ineligible". This is why I am for a straight "Pick the full order and stick to it". Yes - a popular choice withdrawal will move things around but do we really want to go down the "you are nominated, you are not" rigmarole? (and what are we doing if someone withdraws after the ballot is out - we change 3 places in the ballot because A decided they want out now? Or because it was found ineligible (that also can happen))? As fair as it is mathematically, it will cause way too much noise and headaches for any administrators.

I agree, Annie Y. Disenfranchising publicly announced finalists because :math!: would be a PR nightmare and would be perceived as completely unfair.

From Jameson Quinn's post @24 it would be an uncommon problem anyway, but if there's a clear rule about freezing (at least) the finalist list after the public announcement, if (and when) that uncommon problem crops up, there's clear criteria to resolve it.

#55 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:16 AM:

Jameson, that is helpful. I feel some nagging concern yet that I'll see if I can articulate later, but it helps.

#56 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:19 AM:

@51: we're looking at the two works with the lowest point totals, but one of them still may be lower.

@49: After eliminating ZYXW, say works U and V are tied with 10 points and 20 approvals each. Look at how many points they had before you eliminated W; say, 10.167 each. Then, look at how many they had before eliminating X. Aha! Now U is ahead with 10.667 to 10.167. So eliminate V. The point is this is almost certain to be decisive except in the very early eliminations (ie, Z vs Y), when both works are doomed anyway. So if this is not decisive, it's OK to just eliminate both.

#57 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:24 AM:

aak, my numbers were a bit backwards in the UV example @56. That is to say, in earlier rounds, they would have had fewer points, not more. The basic idea still stands.

#58 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:25 AM:

I will note one quick presentation thing, for the sake of folks who are dyslexic or have any of a wide variety of other learning impairments: single characters are much easier to confuse than any longer string of characters. If someone writes up examples, please consider doing it with 3- to 5-letter (or so) expansions. Alpha, Beta, Gamma, Delta, and so on are very, very much easier for many of us to reliably, accurately parse than ABCD.

#59 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:26 AM:

Bruce@46
I don't understand how this system doesn't end up rewarding single-choice voters throughout. It seems like that would be the sensible thing to do for anyone wanting Puppies-like domination: forget the "rule the nominations" part and just focus laser-like on a single pick in every category and carry the day. And it's deeply distressing that I can't make sense of any explanation about why that wouldn't happen, or why I shouldn't be bothered if it does.

It can indeed happen that if the slates focus on a single work, then they will probably get it on the final ballot. There are two reasons why that's okay. First, the whole point of this proposal is not to prevent slate works from getting on the final ballot. The purpose is to keep them from getting all five slots on the final ballot. When they get all five slots, they prevent any other work from having even a chance to win, and that's what is not fair about slates.

The second reason is that getting a work on the final ballot doesn't mean they will win the Hugo. The regular voting system is unchanged. They can present their one candidate (in this case), and the voters can decide if they like it better than the other candidates that were presented.

Now, if your concern is in the voting process itself, that's a separate issue and no change to the nominating system will affect that. However, I don't think that will be a problem either. The only reason that slates work is because fandom's interests are so varied. They end up nominating a lot of different works. But on the final ballot everyone is looking at the same five works, so that effect is gone. Slates will have a much harder time forcing a final win because the rest of fandom isn't as widely distributed anymore.

Does that help?

Kilo

#60 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:33 AM:

@59: "It can indeed happen that if the slates focus on a single work, then they will probably get it on the final ballot."

Yes. But it will not generally happen that a slate can get something on the ballot by bullet voting, unless they could also have gotten it on the ballot by voting more broadly.

There is one exception to this general rule. If there are two slates, and neither bullet votes, it is possible for the stronger slate to eliminate the weaker one entirely. In that case, the weaker slate could probably avoid that outcome by bullet voting. But this scenario is highly unlikely (less than 2% likely at worst) even in elections with 2 competing slates, so I think it's not worth worrying about.

#61 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 11:36 AM:

@60: ...and even if that scenario did occur, the bullet voting strategy would only restore the "fair" (proportional) result; it wouldn't give an unfair advantage.

#62 ::: Brian Z ::: (view all by) ::: May 12, 2015, 11:36 AM:

@59 Keith "Kilo" Watt

There are two reasons why that's OK.

Does everyone find this OK? I believe the purpose of the nomination process is to shortlist five works in each category that are identified as being among the best of the year by the most voters. The goal is to honor and reward excellence, not to feel fair or make as many people happy as possible.


#63 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:36 AM:

Kilo: That's bound to help someone else - I mean that; I love that kind of exposition - but I'm thinking at the moment of the final ballot, mostly. I don't like the idea of a cabal/slate being able to dominate the nominations, but I also don't really like the prospect of comparable effort being able to push one of the 5 (or 6, or whatever) final nominees to the win, either. I was unclear about that earlier, for which my apologies.

To be clearer, then:

So we have a final ballot. It's got 5 entries, give or take. If I'm understanding things, voting for 2 or more entries to be Best Thing spreads your point's worth of influence around, while voting for just 1 concentrates it all on that one pick. So if the United Jerks Of Flunkyvania focus on the one work in each category that Dear Leader has picked, it seems like they won't just get it on the final ballot, they'll likely get a win, unless there's some substantial strong convergence on a single other candidate by Everyone Else.

I'm not at all sure I like that. I'm not at all sure it's avoidable under any circumstances. I am sure that I can't assess the relative chances of such a thing working under the proposal versus under existing rules, nor if it's something I should be much concerned about.

#64 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:37 AM:

[DECLINED NOMINATIONS]

All: Is there a consensus to just take the work(s) that were eliminated in the previous round in the event of a declined or ineligible nomination? If so, do we want to explicitly add this to the proposal or leave the constitution as it is now and just make it a recommendation to the Hugo admins?

Kilo

#65 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 11:38 AM:

If someone writes up examples, please consider doing it with 3- to 5-letter (or so) expansions. Alpha, Beta, Gamma, Delta, and so on are very, very much easier for many of us to reliably, accurately parse than ABCD.

For a graphic presentation, would icons (rocket, blaster, propeller beanie, star, dragon) be more or less confusable?

#66 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:41 AM:

Seth, combining some good iconography with short text is probably ideal for reaching the widest range of folks.

#67 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 11:44 AM:

So if the United Jerks Of Flunkyvania focus on the one work in each category that Dear Leader has picked, it seems like they won't just get it on the final ballot, they'll likely get a win, unless there's some substantial strong convergence on a single other candidate by Everyone Else.

The final Hugo election involves preferential voting. If the majority of Hugo voters put the Flunkyvania choice at or near the bottom of their list of preferences, then it will lose, even if the United Jerks all rank it on top. (This is basically what happened to the Sad Puppy 2 candidates last year.)

#68 ::: Seth Gordon ::: (view all by) ::: May 12, 2015, 11:45 AM:

@64 [DECLINED NOMINATIONS] Best to make that rule explicit, I think.

#69 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:47 AM:

Seth@67: OK. This is the point where I'll wait for some exposition/examples that I may be able to follow, but I'm certainly willing to accept it as a true-looking answer.

#70 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:48 AM:

Bruce@63

Ah, so we're talking about the final election. This system doesn't change the final election at all. It's only for the nomination process, so the two are completely separate things. There aren't any votes to spread around under the current voting system.

Here's how the current voting system works once the final ballot is determined, hopefully someone will correct me if I've described something poorly.

In the current voting system (NOT the nomination system we are discussing in this thread), you take all five works on the ballot and you rank them order of preference (1 for your favorite, 2 for your next favorite, etc.) If you aren't familiar with one or more works on the final ballot, it's okay to just not rank them and leave them off. This doesn't give your higher-rank candidates any help, however. You also have the option of voting for "no award". This means that any work ranked above "no award" you consider to be Hugo-worthy, but anything below "no award" is not.

So, in short, what we're doing here won't change that. If slates voted for their candidate and put "no award" as their second choice, then the only thing they can do is say, "this one should win or nothing". Fandom can disagree, though, and there's nothing a slate can do about that.

Kilo

#71 ::: Vicki ::: (view all by) ::: May 12, 2015, 11:50 AM:

Bruce--

If I read this right, the proposal only changes how works get onto the ballot. The voting process after that is unchanged.

Once Alpha, Beta, Gamma, Delta, and Upsilon are on the list, everyone gets to rank them as we do now. Whether I vote "Alpha, Beta, Gamma, No Award, Upsilon, Delta" or just vote for Alpha and leave the rest of that section unranked, my whole vote goes to Alpha in the first round of counting.

#72 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 11:51 AM:

Kilo@70: Ah! Genuinely, I missed that this isn't supposed to change the final ballot round at all. That does make a lot of difference. Thank you.

#73 ::: P J Evans ::: (view all by) ::: May 12, 2015, 11:57 AM:

The current method of dealing with refused nominations, or withdrawn works, is to pick the next on the list.
felice needs to explain WTF we should run the entire process over again, when we have an ordered-by-popularity list already available.

(BTW: this isn't anything new. There was a nomination refused in 1984, as well as a nominee ruled ineligible, and both were handled in exactly this way.)

#74 ::: influxus ::: (view all by) ::: May 12, 2015, 11:58 AM:

@64 [DECLINED NOMINATIONS] Yes. This sounds sensible. @43 was very persuasive.

#75 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 12:02 PM:

OK. The open questions were naming, ties, withdrawals, and language/explanation/etc. It seems that we're coming to consensus on withdrawals. I think that my simulations show that we have a tiebreaker that works; to wit, use current round scores, then previous round scores, then the one before that, etc; and if you get all the way back to first round scores without breaking the tie (which would never happen in cases that matter), eliminate both.

So the remaining questions would be, name and language/explanation. Let's resolve naming. I like PAPE (Proportionally Allocated Pairwise Elimination). What do others think? (It seems my idea of a score voting poll is not catching on, so I'll drop it, unless somebody else picks it up.)

#76 ::: junego ::: (view all by) ::: May 12, 2015, 12:03 PM:

Here are my rankings for the proposed system names with 10 as highest and 0 as lowest:

0 - Single Divisible Vote, Least Popular (SDV-LP)
0- Single Divisible Vote, Least Popular Elimination (SDV-LPE)
0 - Single Divisible Vote (SDV)
2 - Least Popular Elimination (LPE)
0 - Hybrid Divisible Vote
0 - Quantum Divisible Vote
10 - Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
9 - Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
8 - Bottom Up Elimination (BUE, pronounced “byoo”)
0 - One Notional Equally-divided Vote, with Oppositional Test 0 0 - Eliminations (ONEVOTE)
5 - Proportionally Allocated Pairwise Elimination (PAPE)

#77 ::: influxus ::: (view all by) ::: May 12, 2015, 12:09 PM:

P J Evans @ 73: I think you'll find felice backed picking the next on the list @12. (Now that I've fully read and followed Annie Y and felice's conversation - sorry about earlier).

#78 ::: john, who is incognito and definitely not at work ::: (view all by) ::: May 12, 2015, 12:17 PM:

Sorry if this has been posted already--I missed it if so--but would it be possible to see how this sort of nomination system would affect the ballot in various years (both with and without slate voting, e.g. 2015, 2010, 2005, 2000, etc.)?

#79 ::: Tim Illingworth ::: (view all by) ::: May 12, 2015, 12:18 PM:

OK: I've added a possible revise of 3.8.8 to the Google Doc, which makes things more consistent with the rest of the Constitution.

#80 ::: influxus ::: (view all by) ::: May 12, 2015, 12:23 PM:

BUMPE is clearly the best acronym and avoids the slight propagandist overtones of ONEVOTE. However, I find Proportionally Allocated Pairwise Elimination more descriptive of the system.

So:

1 - Single Divisible Vote, Least Popular (SDV-LP)
1- Single Divisible Vote, Least Popular Elimination (SDV-LPE)
1 - Single Divisible Vote (SDV)
1 - Least Popular Elimination (LPE)
1 - Hybrid Divisible Vote
1 - Quantum Divisible Vote
7 - Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
3 - Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
3 - Bottom Up Elimination (BUE, pronounced “byoo”)
0 - One Notional Equally-divided Vote, with Oppositional Test 0 0 - Eliminations (ONEVOTE)
8 - Proportionally Allocated Pairwise Elimination (PAPE)

#81 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 12:24 PM:

Jameson@75:

That's not actually the way it is coded now. What I found was that looking at previous rounds never made a difference in the 1984 ballots, all the way back to the first round. So the tie-breaking scheme that is currently coded only looks at the current points for the current round. Basically if two or more works are tied for number of nominations, then eliminate the one with the fewest points in that round. If they are tied for number of nominations and number of points in that round, eliminate them both.

I actually think that's simpler to manage and seems to have the same effects. We need something like this no matter what since every tie I found went all the back to the beginning.

Kilo

#82 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 12:25 PM:

Tim@79: Would you mind posting your revision here as well?

Thanks,
K

#83 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 12:33 PM:

[NAME]

I actually don't really like any of the new names, I'm afraid. I think SDV-LPE is a good formal description, since it encapsulates what's actually going on: It's a two-part system which has a vote-distributing part and then a popularity-elimination part. I only suggested "Least Popular Elimination" as a short title, because I think we need a non-technical sounding moniker for discussion. I sort of don't think it's the best choice for an official name, though. What are the thoughts on having a formal name and a descriptive name?

Kilo

#84 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 12:57 PM:

@83: So post your ratings of the names. It's hard for us to turn your description of how you feel into numbers.

As for having two names... I don't think that should be in the proposal, but clearly if you say SDV-LPE anybody with google is going to be able to figure out what you're talking about if they really want to.

#85 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 12:59 PM:

@81: In my bootstrap, I found that it was rare that previous rounds mattered, but that it happened often enough to be worth doing. Basically, most ties happen early on, when there aren't a lot of previous rounds to go on; but the ties that matter happen later, and in those cases, previous rounds are decisive if the current round isn't.

#86 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:14 PM:

[NAME]

0 Single Divisible Vote, Least Popular (SDV-LP)
10 Single Divisible Vote, Least Popular Elimination (SDV-LPE)
0 Single Divisible Vote (SDV)
5 Least Popular Elimination (LPE)
0 Hybrid Divisible Vote
0 Quantum Divisible Vote
0 Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
0 Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
0 Bottom Up Elimination (BUE, pronounced “byoo”)
0 One Notional Equally-divided Vote, with 0 Oppositional Test Eliminations (ONEVOTE)
0 Proportionally Allocated Pairwise Elimination (PAPE)

K

#87 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:21 PM:

Jameson@85:

Okay, it pretty dramatically increases the data storage requirements, since right now the code just dumps the results of the current round to a text file and forgets about it (that's one advantage of the system: eliminated works vanish as though they never existed). That might not be such an issue with a web app + database combination, though. That's outside of my expertise.

Can you give some examples of where it made a difference?

Thanks,
Kilo

#88 ::: Annie Y ::: (view all by) ::: May 12, 2015, 01:25 PM:

Jameson Quinn@27

That would be a change in 3.7.1 as well and we are sticking to the 3.8.x changes (for now anyway) :)

More seriously - if something is so much less worth than something else, what is it doing on your ballot? The idea is to nominate the things you like and would not mind being a winner (or at least nominated) - with the margins we have even now, a single vote can tip the balance somewhere. With a weighted nomination, you are saying what you think is a better work...

@Niall McAuley @ 42
But we are still restricted to 5 (because of 3.7.1)? Your first bullet point sounds as if you are saying that you can go over 5 if you find more than 5 worthy.

@Jim Van Verth @43

Not to mention that in case of withdrawls, anyone that does not make the nominations after that will presume that it is because of whoever withdrew. Which as someone mentioned above - I really do not want to be in the same room with anyone not making the list and the guy that withdrew. :)

@64 [DECLINED NOMINATIONS]I think we should add it to the rules - or down the road someone will decide to complain from the administrators because they decided to rerun this one time.

#89 ::: Tim Illingworth ::: (view all by) ::: May 12, 2015, 01:26 PM:

KW@82: Sure. It's not that long...

3.8.8 The final Award ballots shall list in each category the five eligible nominees as determined by successive rounds of elimination.

(1) In each round, each nomination ballot is allocated a single point, divided equally among its surviving nominations.

(2) The current point total for each surviving nominee is then determined and the two nominees (or more, in the case of a tie) with the lowest point totals will be compared.

(3) Of those nominees, the one that appears on the fewest number of ballots will be eliminated.

(4) If two or more nominees are tied for appearing on the fewest number of ballots, the tied nominee with the lowest point total will be eliminated.

(5) If there is a tie for lowest point total as well as for appearing on the fewest number of ballots, then all tied nominees will be eliminated.

(6) If (due to a tie) elimination would reduce the number of nominees to fewer than 5, then none of the nominees will be eliminated and all remaining works will appear on the final Award ballot.


#90 ::: Duncan J Macdonald ::: (view all by) ::: May 12, 2015, 01:29 PM:

[NAME]

My list is shown below.

0 - Single Divisible Vote, Least Popular (SDV-LP)
10- Single Divisible Vote, Least Popular Elimination (SDV-LPE)
0 - Single Divisible Vote (SDV)
0 - Least Popular Elimination (LPE)
0 - Hybrid Divisible Vote
0 - Quantum Divisible Vote
0 - Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
0 - Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
0 - Bottom Up Elimination (BUE, pronounced “byoo”)
0 - One Notional Equally-divided Vote, with Oppositional Test 0 0 - Eliminations (ONEVOTE)
0 - Proportionally Allocated Pairwise Elimination (PAPE)

Please note that 'cutsey' pronunciations can be a douple-edged sword. For example, I can see, if PAPE is chosen, that the SP/RP Alliance will claim that the modifications are out to RAPE them.

Having waded through all the earlier discussions, using the term-of-art SDV-LPE makes web-searches easier.

#91 ::: Annie Y ::: (view all by) ::: May 12, 2015, 01:29 PM:

[NAME]
Do we really need a clever and/or mathematical name? Call it the Hugo Nominations System (or Algorithm)(and add explanation for the mathematicians if you must) but over-complicating with an acronym does not seem necessary

Plus the way thing works, someone will come up with an acronym at some point that will be cleverer than anything we can figure out now and that will stick despite what we put in the constitution.

#92 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:30 PM:

Brian@62:
The goal is to honor and reward excellence, not to feel fair or make as many people happy as possible.

Someone can correct me if I'm wrong, but I don't think that's actually the goal of the Hugos. The Nebulas, maybe. The Hugos are looking for fan favorites, not a measure of excellence. I don't think there's a real way to evaluate excellence with the Hugo process other than "A work is excellent if it was the one most people liked."

Kilo

#93 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:34 PM:

Jameson#40:

Would Google Docs work for you if we increased the font size? Those of us not on tablets could easily deal with a larger font.

I don't know. I suppose it can't hurt to try. But I think there's something to be said for keeping the discussion "out in the open" (I do realize the Google doc is public, but it's not a conducive place for discussion). It's completely true that we could post changes there and discuss them here, but that extra step is time-consuming. What would you think about waiting for the Google doc until we get to the actual proposal language crafting?

Kilo

#94 ::: junego ::: (view all by) ::: May 12, 2015, 01:34 PM:

Keith "Kilo" Watt @ 39
junego@37: I think the concensus was that people want to rank the final voting, but that for nominations, we essentially want a controlled version of "throwing a name into a hat". Basically nominate anything that people think are Hugo worthy. The final voting then ranks which of those are the best.

Ayyup, that's pretty much what I said. 😏

#95 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 01:36 PM:

[NAME]

I vote 10 for Hugo Nominations System (no acronym). No change to my previous votes.

#96 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 01:41 PM:

junego@94: Yes, sorry, in reading your quote of what I said, it's not clear that I was agreeing with you. :) I was just trying to reinforce your point with the group.

K

#97 ::: junego ::: (view all by) ::: May 12, 2015, 01:44 PM:

Jameson Quinn @ 56

@49: After eliminating ZYXW, say works U and V are tied with 10 points and 20 approvals each. Look at how many points they had before you eliminated W; say, 10.167 each. Then, look at how many they had before eliminating X. Aha! Now U is ahead with 10.667 to 10.167. So eliminate V. The point is this is almost certain to be decisive except in the very early eliminations (ie, Z vs Y), when both works are doomed anyway. So if this is not decisive, it's OK to just eliminate both.

Thanks, that's clear enough even with the slightly confused earlier point totals.

#98 ::: Brian Z ::: (view all by) ::: May 12, 2015, 01:53 PM:

Kilo @92

Then wouldn't they have called the categories Most Popular Novel, Most Popular Short Story, etc.?

Is the consensus here that rewarding excellence has never been a goal of the Hugos?

#99 ::: Annie Y ::: (view all by) ::: May 12, 2015, 01:57 PM:

Brian Z @98

Because most people will vote what they think is best. Which is why it is most popular basically - the best work may not win if enough people find something else to be the best (for some values of best). Finding undisputed best in a subjective field is impossible. Which is why allowing more people's nominations votes is crucial - it will push up the works that most people think are best

At the end of the day, the awards had chosen a lot of good works through the years.

#100 ::: J Thomas ::: (view all by) ::: May 12, 2015, 02:00 PM:

@46 Bruce Baugh

I don't understand how this system doesn't end up rewarding single-choice voters throughout. It seems like that would be the sensible thing to do for anyone wanting Puppies-like domination: forget the "rule the nominations" part and just focus laser-like on a single pick in every category and carry the day.

The way it is now, what slates do is like bullet-voting five different times with the same votes.

With the new rules, slates will get the same result as bullet-voting once, or probably twice, or maybe three times. It's only in categories that don't get a lot of votes or where the votes are especially scattered that the slate can win four times.

So it's an improvement.

It doesn't come close to solving the problem but it can be a big help.

#101 ::: J Thomas ::: (view all by) ::: May 12, 2015, 02:12 PM:

@49 junego

The nightmare (and "unfairness") of throwing a finalist off the list after the public announcement outweighs the "unfairness" of whoever might have replaced them if someone else hadn't withdrawn.

Do they make the announcment before all the authors have confirmed?

That's terrible! In that case it couldn't be OK to change any nominations.

@43 Jim Van Verth

Having seen first-hand the stress that Hugo and Campbell nominees go through during awards season, I think calling people up and asking them "If you were given a nomination, would you accept?" is darn cruel, especially if you're that 1 in 100 person who got dropped off the list when the algorithm is re-run.

Doesn't it depend? How about if it was "Because of our time pressure and the looming deadline, we are calling the first and second runner-ups to see if they would accept the nomination, in case someone else declines. Would you accept the nomination?"

Hugos are stressful, win or lose. But some writers will use the experience to write a great story.

#102 ::: CHip ::: (view all by) ::: May 12, 2015, 02:13 PM:

[ELIMINATION]: I'm unclear why the bottom two nominees by points should be compared for number of ballots instead of doing a straight elimination. Consider: work Bravo is nominated by 6 people who haven't had any choices eliminated yet (=> 1.2 points), while Charlie is nominated by 5 people who have already had a work eliminated (=>1.25 points); if I parse the rules correctly, it's "Sorry, Charlie!" despite Charlie having more points. It is unclear to me that this matches what I see offered as a feature, that nominating less-popular works doesn't hurt the chance that \something/ a nominator likes will end up on the final ballot.
      I gave up on the primal discussion some time ago, so links to the reasons for this tweak to simple elimination would be helpful if people want to be brief. I'd also be interested in the results of simulations, if any; i.e., did this tweak change the final results any? (I know the above numbers are simplified.)

#103 ::: Annie Y ::: (view all by) ::: May 12, 2015, 02:16 PM:

Because you want the work that had been nominated from more people to actually have a better chance.

This way in a case of Guy 1 moninating only A and Guys 2-4 all nominating 5 each, with B being one of them, in a direct A vs B, the work that actually had more nominations wins even if it has lower fractional points.

#104 ::: Pfusand ::: (view all by) ::: May 12, 2015, 02:25 PM:

How do slates do in votes-then-points relative to points-then-votes? About the same?

#105 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 02:36 PM:

Chip@102:

#796 in the previous thread has the results of SDV-LPE compared to the actual '84 Hugo results. Note that this was using an "eliminate all tied works" instead of the current "look at the points again in a tie" system. #843 has the what changed under the new system.

I'd be happy to run any variations on the actual ballots (one slate, two slates, someone drops out, etc.) that you might be curious about as well. The tie system that Jameson has proposed would require a fairly large re-work of the code, which is doable, but only after I finish grading final exams (so likely this weekend at the earliest).

Kilo

#106 ::: qforzy ::: (view all by) ::: May 12, 2015, 02:38 PM:

(Lurking lurker who has heretofore always lurked... rather nervous about decloaking now)

Brian Z @98

There was a phrase somebody* used in the last thread that I liked a lot: "the works most acclaimed by the fans" (with "fans", in context, being shorthand for members of the relevant WorldCon).

That makes excellence part of the equation (so far as the understanding is that fans will acclaim worthy works), but it's excellence in the eyes of the membership in all their multifarious glory. So "people being happy" and "perceiving the process as fair" also matter. If the final ballot is not broadly representative (as much as it can be!) of member opinion, then the award is losing the crucial "by these fans" element that makes it what it is. Hence, the focus on pushing slate-voter influence back to proportionate levels, not eliminating it entirely (which, separately, people have argued may be impossible anyway without terrible side effects).

I don't want to speak for others. This is only what I've taken from following the discussion. I offer it in case it's helpful, and am happy to be countered by those who are closer to being primary sources than me!

* Sorry to whoever it was for not going back to search... Tablet-posting is my excuse

#107 ::: Brian Gibbons ::: (view all by) ::: May 12, 2015, 02:45 PM:

Brian Z @98: Is the consensus here that rewarding excellence has never been a goal of the Hugos?

... and as soon as you come up with an objective measure for excellence, I'd love to take a look at it. Until then, we'll have to make do with proxies.

The current system could produce a ballot that includes works that 20% of the nominators thought were the second through fifth best works, while leaving out entirely works that 10% of the nominators thought were the best.

If we want to find the most excellent work, I think you have a better chance of doing so with a final ballot comparing a work thought by 20% to be the best vs. a work thought by 10% to be the best, rather than comparing a work thought by 20% to be the best with a work thought by that same 20% to be the fifth best.

#108 ::: Lighthill ::: (view all by) ::: May 12, 2015, 02:50 PM:

Recommendation from a programmer who sometimes makes mistakes: run the data through two or more independent implementations of the algorithm. Require that the results match.

#109 ::: junego ::: (view all by) ::: May 12, 2015, 02:59 PM:

J Thomas @ 101
@49 junego
The nightmare (and "unfairness") of throwing a finalist off the list after the public announcement outweighs the "unfairness" of whoever might have replaced them if someone else hadn't withdrawn.

Do they make the announcment before all the authors have confirmed?
That's terrible! In that case it couldn't be OK to change any nominations.

IIRC 1 or 2 were disqualified after this year's announcement, and of course 2 withdrew after (but hopefully these will remain very rare occurrences), but they still happened.

Doesn't it depend? How about if it was "Because of our time pressure and the looming deadline, we are calling the first and second runner-ups to see if they would accept the nomination, in case someone else declines. Would you accept the nomination?"

Personally, I think this is still cruel and stressful. It also might increase the possible leak of information.

#110 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 03:00 PM:

I have a meta-suggestion about naming: choose something that will turn up related work if someone does a Google search on it. It will be reassuring to research-minded people looking at the Hugos to find some prior art and other related implementations, as a buffer against the worry that this is basically a bunch of autodidacts playing Tom Swift And His Amazing Dean Drive Voting Machine.

Whatever name most readily connects to related work is going to be the best one at doing some self-propelled informing and reassuring.

#111 ::: Annie Y ::: (view all by) ::: May 12, 2015, 03:07 PM:

J Thomas @ 101

The announcement is after the authors had been notified but the notification itself means "yes, you made it". As little as these awards may mean to some, for some authors this is what will probably make their year.

We are talking real people here - some of them quite used to awards, some of them new to the field. To get a call saying "would you accept a nomination if we extend one" is just cruel. And not how the awards had worked.

Can it change? Probably. But I would not want to me the administrator that makes the "sorry, you did not get nominated" calls - because once you call for a "just checking", you need to call both winners and losers from that pool. Plus when the results become final, do you really want to see the guy that withdrew and the guy that would have made the ballot if he had not? Gaiman withdrawing knowing that the next on the ballot makes it; Gaiman withdrawing knowing that the ballot may change and he effectively can knock out someone from the ballot is a different story. And people did vote for Gaiman after all - him refusing under the new system (if we rerun) is not exactly a normal withdrawal.

#112 ::: J Thomas ::: (view all by) ::: May 12, 2015, 03:11 PM:

@73 P J Evans

The current method of dealing with refused nominations, or withdrawn works, is to pick the next on the list.
felice needs to explain WTF we should run the entire process over again, when we have an ordered-by-popularity list already available.

What's different, is that now it can make a difference.

The way we did it before, #6 will still have more votes than #7. If #3 drops out, there's obviously no better choice than #6. We had our answer and there was no other answer.

But with this voting system things change. Kind of like IRV for the finals -- if somehow the final winner was disqualified after the vote, there's no particular reason to think that the person who came in second would be the winner if you counted the votes over. The fifth-place guy might have been #2 on every single vote for the winner, but none of those counted for him because those votes counted only for the winner. Without the disqualified guy in there gumming up the works, the result might be very different.

Of course, for the nominations it usually won't matter. In recent years at most 2/3 of the nominators nominated any of the top 5, and more likely it was half the nominators who had any in the top 5 and less than 20% had two. About the same numbers spread among the next ten out of the top fifteen. It usually just won't make much practical difference, except to slates.

As a moral issue I think it's something that reasonable people can disagree about. Precedent doesn't rule because the new voting system changes this around. Whatever we decide now, fans WILL disagree about it later, maybe in a specific case where it makes a difference.

So it seems to me to be a good idea to make a strongly defensible choice now. Or else fuzz it over and let people decide later when it turns important.

#113 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 03:13 PM:

@104: The problem we're trying to solve is when a slate gets the top 5 vote counts. Votes-then-points wouldn't even consider a single member of such a slate for elimination until the very last elimination. So it would get 4 of 5 slots. So that doesn't work.

@102: Comparing them in votes is like asking "if everyone who had voted for either of these works, had voted for only that, which would win?" In other words, it makes it safer not to bullet vote. The fewer people bullet vote, the more non-slate votes we have, the better slate resistance the system has.

For all: imagine a world where each work had an intrinsic quality, and everyone votes based on their slightly-fuzzy perceptions of those qualities. That is, where any differences in taste are purely idiosyncratic, and there are no groups of people with systematic differences in taste from other groups. In that simplified world, the order of points and the order of approvals would be the same at each round; the points would climb as works were eliminated, but they'd do so essentially proportionally, so that if Alpha has twice the approvals of Beta, it would always have about twice the points too.

So in that world, using the two stage system here, with a matchup based on points and a showdown based on approvals, is just a matter of belt-and-suspenders "double robustness".

Now, obviously, the real world is not like that; there are real correlations and anti-correlations between voters in matters of taste. But with honest voters, you'd actually be surprised how weak those correlations are. Yes, we can all point to the one example of two people who enjoy utterly non-overlapping sets of works; but it's really much more common that if you give two random books to any two fans, they'll be able to agree which one is better. I mean, two really random books: I just went on Amazon and got "Swords Against Death
by Fritz Leiber" vs. "Tactics Of Mistake by Gordon R Dickson", neither of which I've read, but you get the idea. So sure, there will be correlations, but the "first principle component" of undeniable book quality will dominate the other, lesser dimensions that are more controversial matters of taste.

So the only time that points and approvals will seriously disagree is in cases of slates. Which is exactly what the system is designed for.

What that means in terms of strategy is that a strategy which focuses on points (ie, bullet voting) or one that focuses on approvals (ie, broad voting) generally won't work, because it will only move one of the two dials, and the other one will continue to give the same results. So this is one reason this system is generally strategy-resistant. (Although no system can ever be perfectly so.)

tld̦r: In general, using both points and approvals is just a way to double-check. It's only when slates are involved that it becomes important.

#114 ::: Brian Gibbons ::: (view all by) ::: May 12, 2015, 03:17 PM:

This might be a bit nit-picky, but is there a reason why this proposal is to add a 3.8.8 rather than just replace 3.8.1? 3.8.2 through 3.8.7 read better to me as additions to the nominating process rather than coming first. (And if you do want this section to appear at the end, the proposal should re-number the preceding paragraphs.)

3.8.6 and 3.8.7 assume that you can only make five nominations, whereas the explanation implies that this system allows for as many nominations as you want. The proposal itself omits any mention of whether there's a limitation on nominations or not, which I'm assuming means that no such limitation exists, but that seems like the sort of thing that should be called out explicitly (given that one exists now, and a non-trivial number of voters are likely to just assume it still will) and the other sections made to conform (3.8.6 and 3.8.7 seem entirely superfluous to me, with unlimited nominations).

On a bit of a broader nit-pick, the explanations seem pretty heavy on selling this proposal to someone who thinks that slate voting is a horrible problem to be fixed, and explaining (multiple times) how this change will solve the slate voting problem.

What about the member who thinks that slates are a temporary aberration, and is against any permanent solution to what they consider a temporary problem? Such members do indeed exist, and I would expect them to be solidly represented at the Business Meeting. Not only do these explanations not sell this proposal to such a voter, they would actually cause them to pigeon-hole it as an overreaction to slates and something to be opposed.

I'm assuming that this FAQ is going to be used as a reference for people to link to as the proposal starts getting wider circulation. I'd suggest that the explanation lead off with the benefits of this proposal over the current system, even in a world without slates: That (as I understand it) it increases the chance that a given nominator will see at least one of their nominations appear on the final ballot, and tries to produce nominees with a broad range of support.

If I were trying to sell this, I'd probably focus less on slates and use the perhaps more neutral example of BDP Short Form, since the situation and impact is essentially the same (replacing a system where one TV show could theoretically crowd everything else out with one in which a popular TV show would still get at least one episode on the final ballot, but is less likely to have the entire ballot to itself).

[NAME]
As to the name, I'm not sure I'm seeing the necessity of the proposal having one at all, beyond something generic like "Hugo Nominations System Improvements". It's useful to be able to throw around acronyms in discussions, as shorthand so everyone knows what systems are being referred to, but I think you're past that point now.

The purpose of a proposal title is (a) to quickly sum up the proposal for someone unfamiliar with it, and (b) to perhaps influence the mindset a member starts out with as they're reading your proposal. "Single Divisible Vote" has little meaning beyond making someone wonder if the proposal is about limiting voters to a single nomination per category; "Least Popular Elimination" does little beyond making such a member think about gym class dodgeball games.

The sort of member to whom titles like that are meaningful are likely already part of the discussion; as an introduction to the proposal for someone coming to this topic fresh, the only thing such a title means is "*technobabble*". I'd keep the more technical name in the explanations, but leave it out of the proposal title.

#115 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 03:20 PM:

@110: I am as versed in voting theory as anyone, and while I'd definitely say that this system is an essentially conventional voting system, it really combines aspects of at least 4 other systems, so it would be hard to make a single name that strongly harked back to all the relevant prior work. So I think that calling it "hugo nomination system", then making that a page on the Electorama wiki with all the relevant background, is the best we can do for google-friendliness.

#116 ::: P J Evans ::: (view all by) ::: May 12, 2015, 03:40 PM:

112
See, this is where you're not selling it.
The current system's problem is not in how it handles withdrawals and disqualifications, it's how it handles slates in nominating, and you're insisting on fixing the non-problem.

Don't break stuff that's not broken.

#117 ::: J Thomas ::: (view all by) ::: May 12, 2015, 03:44 PM:

@102 CHip

[ELIMINATION]: I'm unclear why the bottom two nominees by points should be compared for number of ballots instead of doing a straight elimination. Consider: work Bravo is nominated by 6 people who haven't had any choices eliminated yet (=> 1.2 points), while Charlie is nominated by 5 people who have already had a work eliminated (=>1.25 points); if I parse the rules correctly, it's "Sorry, Charlie!" despite Charlie having more points.

You could get a pretty good voting system doing it the other way. We got a pretty good system doing it this way. By some standards (the standards several of us chose) this way is better.

It is unclear to me that this matches what I see offered as a feature, that nominating less-popular works doesn't hurt the chance that \something/ a nominator likes will end up on the final ballot.

It's a trade-off. If you want to maximize the chance that something you vote for wins, you should pick the five that you think are most popular.

If you want to maximize the chance that one particular work you care about more than anything else will win, then you should just vote for that one.

If you want to nominate the best works you can think of, independent of strategies to win, you can't go wrong doing that. Each thing you nominate is more likely to win if you nominate it than if you don't.

In general, if you vote for something that's popular and something that isn't, your vote will help the popular one that doesn't need it more than it helps the unpopular one. It's hard to design a voting system that goes the other way round.

Jameson has found that more than 90% of the time, (without slates) the winners are the same as we used to get. I haven't studied that issue but my simulations are compatible with that result.

So when you nominate something that isn't in the top 5 in vote-getting, it will probably lose. And yes, it does hurt your other chances some. If other people voted only for it, it could last long enough to get your more-popular works eliminated. That can happen. It just isn't likely. More likely, your vote for something which is not popular will have not matter at all, just as it would not make any difference whatsoever in the old system. Unless you have misunderstood and it's more popular than you thought.

It's like this: If you nominate something that is unpopular, it will probably get eliminated quickly and have no effect on anything. Similarly anything else you vote for will have little effect on that one -- it's almost surely going to lose.

But if you nominate something that's on the edge of fifth place, you might hurt it by voting for any of the other popular works, particularly the ones that are *more* popular.

If you like two works that are competing for fifth place, it's possible that by voting for both of them you help each of them less than half as much as you'd help just one. That looks like it would be hard to test. It would depend on picky details. Chances are you'll never know precisely enough just how close to fifth place they both are, to predict that sort of thing. And the effect will usually be small.

What *is* clear is that every work you vote for, will do better than if you hadn't voted for it.

#118 ::: Richard Gadsden ::: (view all by) ::: May 12, 2015, 03:48 PM:

[DECLINED NOMINATIONS]
One possible approach would be to rerun the counting procedure, but to add a rule that already-announced nominees can't be eliminated (ie they automatically win a showdown, and if a showdown would otherwise involve two already-announced nominees then you back it out, and eliminate the work with the third-lowest number of points).

That way, you still get to use the system properly (e.g. if there are two works with similar voters and one is withdrawn, then the other will replace it, which wouldn't happen if you just take sixth place), but you don't get the annoying effect of telling someone they've been nominated and then snatching it away and giving it to two other people.

#119 ::: J Thomas ::: (view all by) ::: May 12, 2015, 03:54 PM:

@116 P J Evans

112
See, this is where you're not selling it.

You mean, I'm not doing a good job of selling my own preferences? I'm honestly not certain what's best. I would feel dishonest trying hard to push one solution. Maybe I ought to do that to balance the people who are dishonestly trying to push an alternative, but homey don't play that.

The current system's problem is not in how it handles withdrawals and disqualifications, it's how it handles slates in nominating, and you're insisting on fixing the non-problem.

The current system has an obvious best way to do this. The new system does not. We need to have a strong justification for whatever solution we choose, because it's likely to become a great big issue at some random time we can't predict yet, and now is the time we might head that off. (Of course, very likely by the time it goes bad people will have forgotten it was us that handed them that slimebucket. So it's understandable if you want to consider it Somebody Else's Problem.)

#120 ::: Richard Gadsden ::: (view all by) ::: May 12, 2015, 03:55 PM:

With this system, there's no need to limit the number of works a nominator can nominate - you could list 20 and it wouldn't be a problem.

#121 ::: Idumea Arbacoochee, Gardener of Threads ::: (view all by) ::: May 12, 2015, 04:09 PM:

(Note that I've moved most of the OP behind a jump text because it was...rather long.)

#122 ::: J Thomas ::: (view all by) ::: May 12, 2015, 04:14 PM:

@111 Annie Y

But I would not want to me the administrator that makes the "sorry, you did not get nominated" calls - because once you call for a "just checking", you need to call both winners and losers from that pool.

Yes, certainly you do.

Plus when the results become final, do you really want to see the guy that withdrew and the guy that would have made the ballot if he had not?

I am unclear about the emotional parts of this. If it was clearly understood that getting called in case you might win, does not mean you have won, then I just don't know. I imagine I would feel good to hear that I was at least a second runner-up, and disappointed if it turned out I wasn't on the short list, and I'd prefer that to never hearing anything. But I wouldn't really know how I felt unless it happened. (Which seems pretty unlikely!)

In another context we discussed calling people early when they were in the top 15. Everybody who makes his intentions clear at that point is a help -- if they decline then you can announce it while people are still voting. Of course, it's up to 3 times as much work to contact 15 people as to contact 5, even if it is on a more relaxed deadline.

I think it would help a lot to put out a public plea that authors who want to decline, do it early enough that fans can change their votes.

Gaiman withdrawing knowing that the next on the ballot makes it; Gaiman withdrawing knowing that the ballot may change and he effectively can knock out someone from the ballot is a different story.

In any case, the sooner he withdraws the less problems he makes. And he's very unlikely to knock someone off the ballot. We are discussing this because if we do the wrong thing that has rare consequences, those rare consequences can sneak up and bite somebody once.

As a practical approach, I think the easiest thing is to choose the top 15, contact the top 5, contact others as needed, and immediately destroy the ballots. If anybody asks anything about it, announce that all results are final and say no more. If they can't prove there's a problem, then there's no problem.

To me it seems obviously wrong, but it will usually get the right answer. And if it does go bad, and we stonewall well enough and nobody can get any evidence, then it will all blow over.

#123 ::: Joshua Kronengold ::: (view all by) ::: May 12, 2015, 04:16 PM:

Bruce @46: You've gotten a lot of responses to this, but I don't think anyone has managed to articulate two important points on why bullet voting isn't a dominant strategy in SDV-LP

First: Consider a non-bullet ballot, which because it voted for five different things, can expect to see at least four of them eliminated. As such, at some point, it will have all but one nominee (the strongest) eliinated -- and thus will be competing with the bullet ballots on entirely equal grounds. Because any ballot that doesn't have multiple entries make the final list will eventually become its strongest nominee and thus compete with a bullet ballot on equal grounds, listing fewer nominees doesn't strengthen your ballot against anything except the other nominees you would have otherwise put on your ballot.

Additionally, because victory in eliminations is based not on current votes, but overall popularity, bullet voting can be a serious disadvantage. It is possible that bullet voting a nominee will result in it never having to face a standoff. However, if the numbers aren't high enough for that test, a candidate that got mostly bullet votes will, all things being equal, lose to a candidate that got mostly shared ballots -- because they are pulling from a smaller pool of ballots. If there are three candidates, A, B, and C, facing elimination with equal numbers of adherents, and C's voters only like C, but A's voters like B and vice versa, then A and B will each handily beat C, although if they end up in a showdown against one another, one of them will (naturally) lose -- leaving a very strong candiate left at the end.

Re naming:


0 - Single Divisible Vote, Least Popular (SDV-LP)
9- Single Divisible Vote, Least Popular Elimination (SDV-LPE)
2 - Single Divisible Vote (SDV)
0 - Least Popular Elimination (LPE)
0 - Hybrid Divisible Vote
0 - Quantum Divisible Vote
4 - Bottom Up Matched Pairs Elimination (BUMPE, pronounced “bumpy”)
0 - Bottom Up Pairs Elimination (BUPE, pronounced “boopy”)
0 - Bottom Up Elimination (BUE, pronounced “byoo”)
8 - One Notional Equally-divided Vote, with Oppositional Test 0 0 - Eliminations (ONEVOTE)
7 - Proportionally Allocated Pairwise Elimination (PAPE)
10 - Don't name the voting system in the proposal.

I like ONEVOTE -- it's funny and hackery, and the spelled out version (however obviously contrived ["It means someone really wanted it to spell out 'one vote']) spells out the actual point of the system.

PAPE is a well done pronouncable acronym that efficently describes the system.

But of the names, SDV-LPE is in fact the name agreed upon by its originators (I should know), and SDV successfully calls back to STV in terms of the point of the system and basic logic.

However, we don't ever spell out that we use STV in the constitution, and I don't see any reason to specify a name for it here either except maybe in the commentary. Better to describe it, and let people use whatever names they like as long as they're all speaking the same language.

#124 ::: Joshua Kronengold ::: (view all by) ::: May 12, 2015, 04:33 PM:

Oh, also: I've moved my own opinion over time, and now think that we should just calculate a single list of 15 and promote from within.

1. It's easy to describe what's happening, and we don't need a lot of text to show what happened when we have a lot of withdrawals.

2. I don't anticipate that finalist hopefuls will be eliminated that early in the process, so it's not unlikely that something eliminated by a competitor will get another chance if there are a lot of declines.

3. There are obviously cases where eliminating one item would seriously change the ballot, Displaying an entirely different list of nominees and only taking the first one would create a lot of confusion in post-ballot noise.

4. Additionally just like promoting #6 can be a bit unfair, there are cases where calculating a new list of 5 can result in unfair results (removing #1 results in Seanan story #3 coming in first instead of Seanan story #1, so both stories make the ballot). So I don't see a good argument to using a system that still has a possibility of concentrating votes and does make the system far harder to understand. At least this way you make sure the votes are pretty well divided up, even if it's possible some will be eliminated when their candidate withdraws.

Brian Gibbons #114: There's not really a good way to replace section 1, which was my early draft didn't do so. Also, none of the current proposals drop the 5 card limit; there are good reasons to keep it even without it breaking nomination per se, and it's best to move one dial at a time.

Rewarding excellence is totally a goal of the Hugos, but that's a product of and goal of empowering voters. The rules are mostly there so the voters get the most say on what they think is excellent.

#125 ::: Bruce Baugh ::: (view all by) ::: May 12, 2015, 04:42 PM:

Joshua@123: Thank you, that is helpful - I can look back at earlier discussion and see the substance of it, but something about your exposition clicked with me.

#126 ::: J Thomas ::: (view all by) ::: May 12, 2015, 04:53 PM:

@78 john

Sorry if this has been posted already--I missed it if so--but would it be possible to see how this sort of nomination system would affect the ballot in various years (both with and without slate voting, e.g. 2015, 2010, 2005, 2000, etc.)?

Basicly, no. We have enough data to do that for one year, 1984. More than 30 years ago, back when a dedicated fan could almost read everything in contention that year. Things have changed.

We have some summary data for more recent years, but not the whole thing. I have done simulations using random data that is not too different from recent years. It sort of fits the given statistics for recent years. I think it's good enough for some purposes. You can tell the difference between common problems and rare problems, but not estimate just how common things are.

It would be possible to do that better, but I prefer to use distributions that are good enough, and firm them up if some particular purpose requires it. When I put too much effort into that I may get dogmatic about how good it is, and believe it too much, and so the extra time I spend on it is worse than useless.

I put a link to it on my URL line above, done in Python.

An ad hoc model could fit better, and might be worth doing later.

#127 ::: Cadbury Moose ::: (view all by) ::: May 12, 2015, 04:58 PM:

On Naming, this moose would also prefer to have it called by its proper name (SDV-LPE) rather that either a cutesy acronym or (worse still) something it actually isn't.

(The Hugo (or is it TAFF/GUFF) voting method is frequently incorrectly called "The Australian System", as I recall.)

#128 ::: felice ::: (view all by) ::: May 12, 2015, 05:03 PM:

[DECLINED NOMINATIONS]

Richard Gadsden @118: "One possible approach would be to rerun the counting procedure, but to add a rule that already-announced nominees can't be eliminated"

Actually, that's an idea - rerun and extend the ballot if necessary. If the winners on the first run are: Alpha, Beta, Gamma, Delta, Epsilon;
Then Beta is withdrawn, and the rerun gives us Gamma, Alpha, Sigma, Tau, Omicron;
We get a new final ballot with 7 finalists: Alpha, Gamma, Delta, Epsilon, Sigma, Tau, Omicron.

We already allow extending the ballot for a multi-way tie for 5th place, and in practice it's probably pretty rare for rerunning after a withdrawal to eliminate any of the other original finalists (and the new finalist may even often be Zeta/F). It's theoretically possible for this to result in a ridiculously large final ballot, if the original finalists included a 10-way tie for fifth, and the rerun produced a complete different set of finalists including another 10-way for 5th, but the chances of that actually happening are infinitesimally tiny (if the Hugo keeps running for a few million years, we still probably won't encounter such a situation).

P J Evans #73: "The current method of dealing with refused nominations, or withdrawn works, is to pick the next on the list. felice needs to explain WTF we should run the entire process over again, when we have an ordered-by-popularity list already available."

Because the current list isn't ordered by popularity. If B is withdrawn, and F is moved up to replace it, what if everyone who nominated B hates F, and most of them like G? Under the current system, G would already be a finalist, because it has more nominations than F, but under SDV-LPE, the vote is split between B and G, resulting in G getting eliminated earlier. If B had never been in the running, G would always have been a finalist.

#129 ::: Annie Y ::: (view all by) ::: May 12, 2015, 05:32 PM:

J Thomas @ 122

Noone is even hinting of improper treating - we all are discussing how to make it as open and accountable as possible

felice @ 128
Yeah reruns with guaranteed places will work I suspect. Either version is unfair to someone - any withdrawal could have meant someone nominating differently if they knew not to bother. So we are looking for the least unfair solution (that does not cause too much grief).

#130 ::: Joshua Kronengold ::: (view all by) ::: May 12, 2015, 05:55 PM:

Bruce@125: Thanks, glad it helped. Jameson actually hit both of the exact same points in #53, but I think my explanation was a bit simpler and more focused.

#127: It's worth noting that the "actual name" is very new, coming out of the previous thread (as the concept was invented by JQ riffing off a base idea I came up with). The concepts it's built upon are solid and well-explored (SDV is more or less another way of modelling RAV, for starters), but this particular way of putting them together is as far as anyone here knows original to Making Light.

#131 ::: felice ::: (view all by) ::: May 12, 2015, 06:06 PM:

Annie Y @129: "Yeah reruns with guaranteed places will work I suspect. Either version is unfair to someone - any withdrawal could have meant someone nominating differently if they knew not to bother. So we are looking for the least unfair solution (that does not cause too much grief)."

Well, everyone can already nominate up to four other works they think are Hugo-worthy, so they've already got their alternative choices in the running and nominating differently shouldn't be a big issue. I suppose you might get some people choosing not to nominate their second favourite work because it's too similar to their favourite, and nominating four more diverse works instead, but that's a dubious strategy - if fans of the first two books all do that but disagree over which is better, it could mean neither wins. And on the other hand, if the second book is widely agreed to be inferior to the withdrawn book, does it really deserve a Hugo even if it is popular? And there's nothing we can do about it anyway.

In short, there's no point worrying about how people might have nominated differently.

#132 ::: Annie Y ::: (view all by) ::: May 12, 2015, 06:11 PM:

:) Was just thinking aloud and going through variants in my head (and keyboard).

As long as
1. People are not contacted before they are actually nominees
2. A rerun cannot kick someone off the ballot once they have a space on it

I am fine with either flat single rerun or secondary reruns with no ability to remove someone.

There is no perfect system (and I think we all are acknowledging that)

#133 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 06:33 PM:

JT@122:

These are some of the more offensive intimations you've made so far. I do realize you aren't accusing anyone of doing any of these things directly, but it is simply completely false that we are trying to accomplish anything like the things you suggest. As Annie says, the goal is just the opposite: to make the system as open and accountable as possible.

You made a number of comments of this nature in the previous thread, and I advised you off-list that you weren't winning people to your point of view with them. You chose not to take that advice, which is your right, of course. But you need to understand that I've already seen some of your comments here quoted out of context on Puppy-aligned boards. It undermines everything we are trying to do here. We do -not- want to hide things or be gatekeepers or keep the Puppies off of the ballots. I'd actually like the Puppies to -support- this proposal since it meets their stated aims. Comments like the one made in 122 are just not helpful.

Kilo

#134 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 06:47 PM:

[DECLINED NOMINATIONS]

Jameson@24:

3. If somebody declines a nomination, running the system again from scratch will change one of the other 4 existing nominations only about 1 time in 100.

Would you mind describing the situations where it did matter? That might help us decide whether we need to worry about re-running the system at all, and if so, how to do it. The problem, of course, is that we are dealing with low-probability situations to begin with, so it makes it hard to anticipate what the effects would be.

Thanks,
Kilo

#135 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 06:57 PM:

[TIES]

Jameson@56:

I won't likely get to coding this until later this week (or more likely the weekend), but I wanted to make sure I've got the idea you're proposing as I'm thinking about it.

Basically, I need to keep an array for each work (it will be a two-dimensional array, but that's just a note for me) that continually grows with each round and records the total number of points received in that round, correct? We then go backwards through the array when have a tie for both points and nominations. If at any time one of the works has fewer points, then that work gets eliminated (the number of nominations will never change, of course, hence why we look at points). Is this a correct statement of the system?

Once I get it coded, I may be able to come up with a straightforward way to explain it. We can also then run simulations under both tie-breaking systems (with and without past-round lookback -- hey, that's not a bad short-title name right there!) and see the effects. It sounds like you may already have it coded, but as someone up-list said, it's a good idea to have two independent algorithms doing the same thing as a check.

Thanks,
Kilo

#136 ::: P J Evans ::: (view all by) ::: May 12, 2015, 07:07 PM:

128
You seem to be determined to settle it the hard way.
Explain, in simple terms, Why The Fuck it needs to be rerun to pick a replacement for a declined or ineligible nomination, when the current procedure, under the current rules, is to pick the next one from the list of counted names, and when this has not caused difficulties before?

(I refer you to 1984, where there is one withdrawn nomination, and one ineligible. Which you may not have noticed.)

#137 ::: Cadbury Moose ::: (view all by) ::: May 12, 2015, 07:38 PM:

P J Evans @ #136

I think it's a prime example of "the perfect is the enemy of the good" and that by attempting to get a mathematically rigorous nomination system, they are ignoring the potential human cost of a withdrawal/disqualification.

Once the initial nomination phase has been completed and the successful nominees have been notified that should be it. Any declined nomination must simply bump the entries below it up one place: rerunning the vote to select the mathematically perfect replacement is unacceptable if it could create conditions where the committee have to dash someone's hopes by withdrawing their nomination after they've been informed they are on the final ballot.

Run the numbers, produce a list of nominees + enough extras in case all the nominees decline(!) (so you want at least 10 slots) and notify the top 5. (If they all decline, then you might consider rerunning the nomination process with their votes reallocated, but I can't quite see that happening somehow.)

#138 ::: Annie Y ::: (view all by) ::: May 12, 2015, 07:42 PM:

@137

We do have the full list under the 5 anyway - it is the inverted elimination order (unless if we want to over-complicate that by running with them minus the ones already in the game or something along these lines to recalculate one more time for each place). So at the time when we know the top 5, we already know who is #6 and so on - last eliminated is first on the standby list and so on.

#139 ::: felice ::: (view all by) ::: May 12, 2015, 07:51 PM:

Cadbury Moose @137: "I think it's a prime example of "the perfect is the enemy of the good" and that by attempting to get a mathematically rigorous nomination system, they are ignoring the potential human cost of a withdrawal/disqualification."

No, my current proposal is don't remove anyone (other than declined/ineligible works), and extends the ballot to more than five works if needed (just like we do for ties). Call it "rerun, merge, and extend". There's no human cost there.

And it would be pretty rare for the ballot to need to be extended.


P J Evans @136: "Explain, in simple terms, Why The Fuck it needs to be rerun to pick a replacement for a declined or ineligible nomination"

If #128 wasn't simple enough, I can't help you. SDV-LPE is a weighted system, and removing a finalist changes the weights. The current system is not weighted, so removing a finalist doesn't affect anything else.

#140 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 08:18 PM:

@120: That's exactly right, but it's not part of the proposal, because we think that the minimal change that fixes the problem is what has the best chance of passing.

#141 ::: Brian Gibbons ::: (view all by) ::: May 12, 2015, 08:20 PM:

P J Evans @136: Explain, in simple terms, Why The Fuck it needs to be rerun to pick a replacement for a declined or ineligible nomination, when the current procedure, under the current rules, is to pick the next one from the list of counted names, and when this has not caused difficulties before?

Because the current procedure under the current rules isn't using weights and elimination.

Take a scenario in which the five final nominees in Best Dramatic Presentation, Short Form, are each individual episodes from five different TV shows, and it's discovered that one of the nominated episodes is ineligible for some reason.

The 6th place nominee after the initial run was an episode from a 6th TV show that was the sole surviving nominee on 8% of the nominating ballots. However, if you reran the analysis, excluding the ineligible nominee, 15% of the voters who voted for that nominee would have had another episode from that same show as their sole surviving nominee (but that got eliminated earlier in the original run).

By just going to the 6th place finisher rather than rerunning the analysis, you're essentially throwing away any ballots whose sole surviving nominee was the ineligible/withdrawn one. I think there's a reasonable argument that this is an unfair result.

I'm not in favor of removing any finalists from the list, but running the analysis again to see who should get added seems like a better option than just taking #6.

#142 ::: Annie Y ::: (view all by) ::: May 12, 2015, 08:23 PM:

felice @ 139

I think that everyone understands that it is a weighted system. But in order to keep everyone that was already nominated we already break the proper order in case of withdrawals. So why stick to the weighted at this point and do not go for the old rule (the #6 and down were already weighted).

Or think of this this way - usually eliminations can be led to their ultimate end leaving an ordered list of N elements (the inverted elimination order). We declare this the Hugo Nomination System result and we get as many of them as we need - 5 (if over 5%), more in case of ties or in case we need to get an additional one because of removal.

Don't get me wrong - I understand your logic and in a normal case I would agree with you. But we already need to break the standard case because we are dealing with people :)

#143 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 08:27 PM:

@134: I don't actually know, and it's not trivial to actually check. My best guess is that these are the cases where effectively a slate of (alpha, beta) was knocking out gamma before beta got knocked out; and then, when alpha withdrew, gamma never got knocked out by beta, and thus ended up knocking out delta.

If this is what's happening, then yes, gamma should be in there. But what I really want is to stop arguing about what color to paint this 1% of the bikeshed. I still think that the simple answer is better than the right one in this case.

Consensus is good, but when there's one person blocking it, at a certain point you have to move past them.

@135: Yes, you're right. Of course, you can know the final dimensions of the array beforehand, so there's no need to allocate memory on the fly.

#144 ::: P J Evans ::: (view all by) ::: May 12, 2015, 09:05 PM:

141
It hasn't been a problem in the past. Don't create one where it doesn't already exist - and it doesn't, really.

IOW: DON'T BREAK STUFF THAT WORKS.

Dealing with ineligible/withdrawn nominees is a whole 'nother thing from minimizing slate voting, which is the problem we're supposed? to be handling, and it doesn't have to be fixed Right Now.

#145 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 09:31 PM:

@133: "I've already seen some of your comments here quoted out of context on Puppy-aligned boards"

I understand that some puppies are just in it for the lulz, and that there's no talking to such people. But there are some who are sincere. If you know of boards where they are discussing this system, I'd like to make the case to them that this system is in their interests too.

#146 ::: felice ::: (view all by) ::: May 12, 2015, 09:57 PM:

Annie Y @142: "I think that everyone understands that it is a weighted system. But in order to keep everyone that was already nominated we already break the proper order in case of withdrawals. So why stick to the weighted at this point and do not go for the old rule (the #6 and down were already weighted)."

Almost everyone. And why not stick to the weighted? Rerun-merge-extend is simple and as fair as possible. Sometimes, it will be the same as promoting the last eliminated; sometimes, it will promote a work that was eliminated earlier, which deserves to be on the ballot - a better result; very rarely, it might promote more than one deserving work, in which case technically one or more of the existing works shouldn't really be there, but pretty much everyone agrees removing them after they've been notified is unreasonable, and they're popular enough that keeping them isn't unreasonable.

Actually breaking the proper order would be even more rare; generally speaking, if the rerun produces two new works instead of one (already unlikely), the ex-top-5 work will very probably be in 6th place, and with the final ballot extended to 6 works, it will be the correct top six. If the ex-top-5 work drops to 7th place, you could conceivably argue it's unfair on the work that comes 6th in the rerun, but I think that's pushing it; it wasn't in the top 5 before the rerun, it's not in the top 5 after the rerun, so it doesn't deserve to be a finalist.

#147 ::: nathanbp ::: (view all by) ::: May 12, 2015, 09:58 PM:

[DECLINED NOMINATIONS]

In the 1984 data a declined nomination may not have caused someone besides the last eliminated nominee to be nominated very often when rerunning the data. However, in the presence of a slate, if a slate nominee withdraws, it seems fairly likely that the last eliminated nominee would not be from the slate and thus the opinions of a large number of votes would being disregarded. You can imagine the same situation with correlated voters for other reasons as well, although it's much less likely. For this reason my preference would be to rerun the system and then replace the withdrawn work with the top work that's not on the ballot (2b from my listing of options at #174 in the previous post).

P J Evans@144: The solution that's been proposed here to deal with slate voting breaks the previously working system for dealing with withdrawn nominees, therefore discussing how to deal with withdrawn nominees is a relevant part of this discussion.

#148 ::: Laertes ::: (view all by) ::: May 12, 2015, 09:59 PM:

Here's Brian Z, over at file 770, characterizing @133:

But I just saw on Making Light, in a post ostensibly aimed at reaching consensus on a rules change, a regular participant raise a useful and fairly critical but also fairly dry, technical point, and then be chastised and put back in his place by a community leader on the grounds that one of his early remarks was supposedly quoted approvingly on a “pro-puppy” site. We can’t have that. The puppies should be approving what we are doing without being engaged in discussion because we are inherently right and good, and if they have ideas of their own then something must be wrong with them.
#149 ::: johnofjack ::: (view all by) ::: May 12, 2015, 10:19 PM:

J Thomas @ 122: If anybody asks anything about it, announce that all results are final and say no more. If they can't prove there's a problem, then there's no problem.

This is exactly the opposite of what I believe and what I want. I think the nominating and voting process needs to be open and transparent, not dismissive and authoritarian. There are people who trust people by default but I don't tend to be one of them.

#150 ::: Vicki ::: (view all by) ::: May 12, 2015, 10:21 PM:

I may be missing something, but:

This is a proposed way of selecting five nominees. It's possible that a tie might lead to six, seven, or even ten nominations. (Two-way ties have happened under the current system; it doesn't seem to be a major problem.)

If one of those 6-10 choices declines the nomination, there's no need to recalculate anything. The list without the person who declined would still include at least five finalists.

Can't we save recalculating for categories/years where the algorithm produces a list of five and then someone declines, or the (I suspect rare) cases where it produces six or more, and then enough people decline to bring it below five?

#151 ::: nathanbp ::: (view all by) ::: May 12, 2015, 10:35 PM:

Vicki@150: I don't know that what happens when the category already has more than 5 nominees and one of them declines has been considered. In that case it would seem reasonable not to replace them. But in the majority of cases where a nominee declines the system will have only selected 5 nominees for that category (since in the majority of cases the system will have only selected 5 nominees).

#152 ::: Brian Z ::: (view all by) ::: May 12, 2015, 10:41 PM:

@ 148, If you are trying to throw a bomb by bringing this discussion over here, could you please cite my entire comment in the context of the other discussion and link to the exchange about it. Thanks for your consideration.

@ 149, "the nominating and voting process needs to be open and transparent, not dismissive and authoritarian." Does anyone disagree with this? I have been following the discussion in this thread, and thought it was about when it is the appropriate time to say "this result is final" and not go back to recalculate the results again. That has absolutely nothing to do with whether or not someone trusts a WorldCon committee (who have won a bid!) to administer the Hugos in an ethical manner. Or did I misunderstand the topic under discussion?

#153 ::: Jameson Quinn ::: (view all by) ::: May 12, 2015, 10:50 PM:

I would like to include sad puppies in this discussion. I doubt that inviting them over here would end well though; there would be people on both sides looking to take offense. Any suggestions for neutral territory?

#154 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 10:51 PM:

Laertes@148:

Well, I guess I'm flattered that I'm a "community leader" now. Thanks Brian! :) But this is an example of how comments can taken out of context. As the folks reading here know, it wasn't a "dry, technical comment" it was an implication that we want to destroy the ballots and hide the evidence. Which Brian Z well knows, of course, but hey, that's okay if it fits your agenda.

Here's one quote of JT's that was posted on .Clue and commented on here. The specific quote is:

“My view is that when we specifically try to change the rules to exclude the Sad Puppies, and we judge how well the changes work by how well they would have excluded the Sad Puppies given historical data, we will have some difficulty explaining to journalists that we are not doing it to exclude the Sad Puppies.”

If you go back and look at the thread, JT is -not- saying this is what we shold try to do, even though the article makes it look like it is. But the point is that there are certain people (Brian Z, apparently as well) who are going to take things out of context to "prove" their point. In this case, JT's comment @122 looks -explicitly- like he is suggesting this is the goal. That's not okay, so it was necessary that we make it clear -- and also be cognizant of the fact that words matter. I don't know if he was joking or not, but it sure didn't come across that way.

Kilo

#155 ::: J Thomas ::: (view all by) ::: May 12, 2015, 10:53 PM:

@133

These are some of the more offensive intimations you've made so far.

I am not "intimating". I find it offensive that you would read that in.

I do realize you aren't accusing anyone of doing any of these things directly,

Or at all.

but it is simply completely false that we are trying to accomplish anything like the things you suggest.

Of course not. I do point out, though, that this approach in some ways would be the most practical.

As Annie says, the goal is just the opposite: to make the system as open and accountable as possible.

If we do make it as open and accountable as possible, we might find ourselves providing people with enough data that they can interpret it as the voting administrators doing something wrong.

Consider how it goes with IRV, where the top winner gets votes that don't go to anyone else, and then the 2nd place has votes that don't go to anybody else, etc. If the top winner got disqualified and you didn't recount the votes, that would be likely more than half the votes that didn't get counted. It would be a travesty.

It isn't that bad with our system since the votes do get counted to decide eliminations, and get partly counted for choosing who to compete.

But I can imagine the argument. "You guys say that this is the right way to choose the final nominations. But now here's somebody who isn't running after all, and you aren't using the method you think is the right one." You can say that you don't do it one time and not the next, it isn't arbitrary because you have a rule that you always follow. "So let me get this straight, you jumped the gun and announced the winners before you found out this guy was going to withdraw or get disqualified, and now you say it's too embarrassing to take it back from the guys you falsely announced were the winners so they get to keep their wrongly-acquired awards. While these other guys who really won get nothing."

I don't know that this argument is valid. I can easily imagine a reasonable-sounding argument that it's wrong. But it's better not to have to argue in the first place. We ought to have a really good explanation why it's right. "How would you feel, going to somebody who'd already been told he won the Hugo and having to tell him he didn't win after all?" will work with some people.

I know that nobody here wants to keep secrets. But if we did keep all the details secret we wouldn't have as many angry arguments about what's fair, as we can expect to get if people do find out details of individual elections.

It's predictable that most of the time, ties and withdrawals won't make any difference to the results. If people find out about one time when they do make a difference, then we need to have our superbly fair explanations ready.

#156 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:03 PM:

Brian@152:

"the nominating and voting process needs to be open and transparent, not dismissive and authoritarian." Does anyone disagree with this?

Certainly not me, which is why I felt JT's comment was out of line. And which certainly makes it curious that you would frame your comment on 770 as you did.

For the record, I don't think a "pro-Puppy" site is a bad thing at all, and in fact, I would really like to involve those who count themselves Puppies in the discussion, but I'm not sure there's many willing to have that discussion. I have been clear from the beginning that I am actually the Puppies' target demographic in my reading interests and I do -not- want any genre excluded. This system explicitly ensures that if there are enough people who like the types of works Brad T recommends, then they will appear on the final ballot -- but they won't also force other types of works off. Brad's complaint was that the types of fiction he enjoys was being ignored. I'm not sure if that's true or not, but one way or another, this system works to prevent -any- group -- Puppy, Kitten, or Monkey -- from being able to keep other groups' works from being considered. I'd like to think they'd welcome that, but I'm probably being naive.

But you know what, you're right that this -is- a post trying to reach a consensus on a Hugo nomination system. And this particular topic probably sidetracks that, so I'll stop.

Just my thoughts,
Kilo

#157 ::: Brian Z ::: (view all by) ::: May 12, 2015, 11:07 PM:

@ Kilo,

I would have made a similar comment over here too (not just on File 770) had there had not been that lightning-quick accusation from an alert reader. I am just back at the computer catching up on these threads, and things move quickly. I hope you get a chance to read my full comments on the other side and see that I am feeling disappointed with the totality of the situation, not with you personally.

I do think you seem like a "community leader" and I see you are doing great work. If I missed aspects of the full context of your criticizing the other poster that would have helped me better understand your remark, then I accept your clarification with thanks.

More generally, I believe that engaging more actively with those who people have sometimes referred to as if they were "the other side" (though I would rather think of all of us as first and foremost fans, and members of the WSFS) is a crucial step, and one that would be in what I consider to be the spirit of the award.

#158 ::: Annie Y ::: (view all by) ::: May 12, 2015, 11:10 PM:

J Thomas @ 155

"If we do make it as open and accountable as possible, we might find ourselves providing people with enough data that they can interpret it as the voting administrators doing something wrong."

And that is easily provable by releasing all the data if need be.

My problem with your comment in 122 were the last 2 paragraphs - you essentially say (or so I read it) that the easiest way is to build the system and then ignore any attempt at scrutiny by stonewalling and whatsnot. This is the worst thing we can do. I think you said it in jest and more to point to the worst alternative but... it is easily taken out of context and half of the conversations these few weeks were for things taken out of context. It does not help.

#159 ::: felice ::: (view all by) ::: May 12, 2015, 11:21 PM:

nathanbp @147: "my preference would be to rerun the system and then replace the withdrawn work with the top work that's not on the ballot (2b from my listing of options at #174 in the previous post)."

What do you think of "rerun-merge-extend" (see #128)? I originally supported your 2b, but that gives us trouble when it's time to publish the extended nomination data (see #6).


Vicki @150: "This is a proposed way of selecting five nominees. It's possible that a tie might lead to six, seven, or even ten nominations. (Two-way ties have happened under the current system; it doesn't seem to be a major problem.) If one of those 6-10 choices declines the nomination, there's no need to recalculate anything. The list without the person who declined would still include at least five finalists."

If one of the tied works is withdrawn, then yes, I'd agree there's no need to replace it. But if one of the higher-ranked works is withdrawn, fairness still requires a replacement. Of course it's possible that rerunning will promote one of the tied works above the other(s) to replace the withdrawn work, so the number of finalists will still drop. Eg if the finalists were ABCD[EFG], and B was withdrawn, a rerun might give us ACDE[FG], or even ACDEF; either way, the run-merge-extend result would be ACDEFG, ie no replacement. Though it is also possible that the rerun will promote something other than E, F, or G, so the final ballot could be AZCD[EFG].

#160 ::: Keith "Kilo" Watt ::: (view all by) ::: May 12, 2015, 11:27 PM:

Brian@157:
More generally, I believe that engaging more actively with those who people have sometimes referred to as if they were "the other side" (though I would rather think of all of us as first and foremost fans, and members of the WSFS) is a crucial step, and one that would be in what I consider to be the spirit of the award.

I agree wholeheartedly. I would dearly like to engage with "the other side" (I also hate that term, since I agree there shouldn't be "sides"), but I'm not sure how many people on that "side" are willing to do so. As I said in the other thread, I'm a retired Naval officer, a combat verteran, a Navy marksman, and Robert Heinlein is my hero. Honor Harrington rocks. I'm about as close to what the Puppies say they support as anyone, genre-wise (by the way, I loved Ancillary Justice, too). So, really, my problem is with slates in general, not with politics, and I actually think that's true of just about everyone here -- whatever their reading preferences might be. What I'm not sure is that there are Puppies who are willing to put aside politics and work together. I do hope there are, though.

At any rate, thanks for coming back and accepting the clairfication of what was happening. The more voices we get on this proposal, the faster we can make the Hugos work for everyone. That really is the goal.

Thanks again,
Kilo

#161 ::: nathanbp ::: (view all by) ::: May 12, 2015, 11:31 PM:

felice@159: Without having put too much thought or calculation into it, the potential issue I see with "rerun-merge-extend" is that it seems like it could potentially end up with a large number of nominees if multiple people drop out or are disqualified in in sequence (like happened this year). I'm not sure how likely that really is to happen but it is a concern. (2b) seems simpler and less likely to have odd side effects.

#162 ::: Brian Z ::: (view all by) ::: May 12, 2015, 11:31 PM:

Brad's complaint was that the types of fiction he enjoys was being ignored. I'm not sure if that's true or not, but one way or another, this system works to prevent -any- group -- Puppy, Kitten, or Monkey -- from being able to keep other groups' works from being considered.

Why wouldn't fans of Larry Correia, Jim Butcher, Kevin J. Anderson, John C. Wright, et al., be interested in that discussion? I bet a lot of them are.

My own perspective, FWIW, is different. I believe the culture surrounding the voting process has got to be a more vital ingredient in the long success of the Hugo Awards than some highly technical rules, and believe that the message sent to voters by the current rules (we will shortlist the works that are acknowledged as best of the year by the greatest number of a community of readers) is preferable to what I fear may be a subtly but crucially different message sent by the proposed rules (we will shortlist works in a way that maximizes voter happiness).

Whether I'm right about that or not, voting is explicitly open even to people who don't attend and were not regular voters in the past, so when thousands of new readers are motivated to participate, I still think the best way of ensuring a successful Hugo voting culture survives is to reach out to them, welcome them, and encourage them to read as much as possible and vote accordingly. Or perhaps, the Hugo voting culture (not necessarily the rules) might even evolve as the membership evolves - I don't think we can know for certain yet.

#163 ::: Annie Y ::: (view all by) ::: May 12, 2015, 11:43 PM:

Mike Glyer linked to this post in today's roundup. So let's see if someone will decide to join everyone here with fresh eyes and ideas. :)

#164 ::: P J Evans ::: (view all by) ::: May 13, 2015, 12:01 AM:

162
Dammit, I thought we'd cured these guys of trying to maximize voter happiness. That's not one of the things that's on the short list (see the previous post):

Whatever we choose should have these properties:

It should be fair.
It should be perceived as fair.
It should be relatively easy to explain, both to the voters and at the WSFS business meeting.
It should be relatively easy to administer.
It should encourage people to nominate.
It shouldn’t result in too many nominees for the electorate to reasonably read and rank by the Worldcon.
It should be resilient to some degree against strategic voting: i.e., minority voting blocs.

#165 ::: Brian Z ::: (view all by) ::: May 13, 2015, 12:21 AM:

@164 P J Evans

I didn't participate in that previous thread primarily because Bruce asked us not to discuss "whether it is a good idea to change."

But if you don't mind the late addition, I would support not abandoning one that is already codified.

Whatever we choose should have these properties:

It should represent the works acknowledged as best of the year by the greatest number of voters.

(If you can figure out a way to avoid slates and still do that, I'm all for it.)


"voter happiness"

You can call it "perceived fairness" too, I don't object, if maximizing the number of voters who get something from their ballot nominated is what both makes people happy and is perceived as fair.

For me, to use the thought experiment of 1970 above, if I had nominated Left Hand of Darkness, Slaughterhouse five and Bug Jack Barron, but Spinrad ended up getting knocked off the ballot by Piers Anthony and Michael Crichton (for example), an explanation that my acknowledgement of Bug Jack Barron as one of the best works of the year had been discounted because I felt the same way about LeGuin and Vonnegut would not strike me as fair at all. Not fair to Spinrad, that is. And being not fair to Spinrad would make me unhappy.

I don't mean to be too argumentative but I do hope you can see that some people may not be coming to this discussion from the perspective that "the greatest good for the greatest number" is particularly applicable to the question of how the Hugo Awards have on a fairly consistent basis managed to pick great SFF most of the time over the years.

#166 ::: Brian Z ::: (view all by) ::: May 13, 2015, 12:32 AM:

Self @165

If you can figure out a way to avoid slates and still do that, I'm all for it

Well... in fact I also do have a concern with limiting the number of slot to less than five on the grounds that encouraging voters to think outside the box and consider some things that are not their "top favorites" is a valuable signal sent to voters by the nominating ballot in its current form. But that is a slightly different discussion and may be off topic for this thread.

#167 ::: felice ::: (view all by) ::: May 13, 2015, 12:39 AM:

nathanbp @161: "Without having put too much thought or calculation into it, the potential issue I see with "rerun-merge-extend" is that it seems like it could potentially end up with a large number of nominees if multiple people drop out or are disqualified in in sequence (like happened this year). I'm not sure how likely that really is to happen but it is a concern. (2b) seems simpler and less likely to have odd side effects."

Theoretically possible, but the multiple withdrawals this time are mostly if not all slate-based, and RME is almost certain to just add the next work off the slate as a sole replacement. And ideally, wait as long as possible before rerunning, so if there is more than one withdrawal in the same category, one rerun can account for all of them. Even if multiple rounds of RME are required, I'm confident that the chances of the ballot growing much are pretty slim. At some point, the ballot is finalised and any late withdrawals just don't get replaced (eg Black Gate this year).

2b is pretty much guaranteed to cause a fuss the first time the rerun puts more than one new work in the top 5, and the other new work(s) don't end up as finalists. That information will come out when the extended nomination data is released.


P J Evans @164: "Dammit, I thought we'd cured these guys of trying to maximize voter happiness."

In this context, "happiness" is defined as "believing at least one and preferably more of the best eligible works have made the final ballot". That strikes me as a very reasonable goal.


Brian Z @165: "For me, to use the thought experiment of 1970 above, if I had nominated Left Hand of Darkness, Slaughterhouse five and Bug Jack Barron, but Spinrad ended up getting knocked off the ballot by Piers Anthony and Michael Crichton (for example), an explanation that my acknowledgement of Bug Jack Barron as one of the best works of the year had been discounted because I felt the same way about LeGuin and Vonnegut would not strike me as fair at all. Not fair to Spinrad, that is. And being not fair to Spinrad would make me unhappy."

The only way Spinrad could be knocked off would be if more people liked LeGuin and Vonnegut. If it's knocked off by other works that more people nominated, how is that unfair?

#168 ::: Chris ::: (view all by) ::: May 13, 2015, 12:43 AM:

I'm going to agree with Seth Gordon @44. The voting system takes a bunch of ballots, and outputs a ranked list. If one of the nominees withdraws, give the spot to the next work on the list. (If there are more than 5 nominees, and any one of them withdraws, do not replace.) It may not be perfect on the "should be fair" criterion for a mathematician's definition of "fair", but it's a hell of a lot better on "perceived as fair", "easy to explain", and "easy to administer".

#169 ::: Seth ::: (view all by) ::: May 13, 2015, 12:46 AM:

We should do re-runs.

The released data must include the calculations involving the declined nominee; that's required to show that the results are correct.

As for slates, a group of 20% of the nominators ought to be able to get one item on the ballot (whether they nominate a full slate or a single item). If there were five such slates they'd fill the entire ballot, which would annoy the remaining nominators except there aren't any.

I don't see any particular reason to limit people to 5 nominations (except perhaps the way it interacts with the 5% rule).

#170 ::: Steven desJardins ::: (view all by) ::: May 13, 2015, 12:48 AM:

PJ@136: The administrators don't rerun the counting in the current system because rerunning the counting wouldn't change any of the totals. Everything gets one nominating vote for each ballot it appears on, whether the other items they list are withdrawn or not.

On the other hand, if a bunch of voters nominate something like THE MARTIAN, which turns out not to be eligible, then I think the administrators should drop THE MARTIAN from their ballots before they even start the SDV elimination process. If the administrators find out after the fact that THE MARTIAN isn't eligible, then I think they should try to get the same results as if they'd realized that in the first place. And ideally we shouldn't require voters to research the eligibility of works or figure out what's likely to be withdrawn: people should be able to vote for what they like, without worrying that a miscalculation will devalue the rest of their choices.

I get that it's weird and counter-intuitive, but any option is weird and counter-intuitive in some way.

#171 ::: P J Evans ::: (view all by) ::: May 13, 2015, 12:50 AM:

167
I don't see how you're going to get there by futzing with unbroken stuff. What has people unhappy NOW is not the voting method, but slate voting. So you want to minimize the effects of slate voting. All else is frosting, and possibly unwanted frosting at that.

(How long have you been in fandom? Because this is stuff you should be aware of already.)

#172 ::: Brian Z ::: (view all by) ::: May 13, 2015, 12:50 AM:

felice @ 167,

Hypothetically, in my example, folks who liked Bug Jack Barron probably also voted for the LeGuin and Vonnegut, which both would have steamrolled onto the final ballot regardless. In contrast, in my example, fans of Crichton and Anthony hypothetically tended not to vote for Bug Jack Barron (perhaps finding it weird and challenging, not liking the authors politics, or for other reasons that seemed relevant to them in the summer of 1970). Why should a brilliant edgy writer who was ahead of his time have votes for him discounted because his voters also liked several more mainstream choices?

To extend the thought experiment: you are so blown away by reading LeGuin that you can think of nothing else and just want her to win at all costs. Why not just leave Spinrad's also-noble effort off the ballot entirely so that you don't detract from LeGuin's greater glory?

Ask whether voting strategies might change under the new rules, and if they might, ask if that is the best for the long term continued success and prestige of the award.

#173 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 01:06 AM:

Regarding final-place ties: the largest Hugo finalist field I recall there ever being was seven. I see no reason to try to break a last-place tie for finalists. I've been encouraging anyone writing rule changes that touch the ties language (this isn't the only proposal out there that might do so) to generalize the language into "ties involving last place" so that if we decide to modify the number of finalist positions, we don't have to change a bunch of hard-coded references elsewhere.

#174 ::: influxus ::: (view all by) ::: May 13, 2015, 01:10 AM:

Brian Z @ 172:

Under the current system there are only 5 slots, it is "top five past the post". If you are all about glory to the LeGuin then you are better off depriving any potential competition of extra votes, and just nominating one work.

Your objection applies equally to the current system. If we are going to use pathological voters as test cases we need to show that they would behave and impact on the current system in a better fashion.

#175 ::: Laertes ::: (view all by) ::: May 13, 2015, 01:12 AM:

I've been staying out of the rule change threads because they look like a lot of work, I have no unique expertise to offer, and it looked as though there'd be enough people involved that one more wouldn't be a make-or-break.

Just wanted to pop in to say I love what y'all have come up with. It's elegant and seems likely to achieve its aims.

It's very nice work. Bravo!

#176 ::: felice ::: (view all by) ::: May 13, 2015, 01:13 AM:

Brian Z @172: "Hypothetically, in my example, folks who liked Bug Jack Barron probably also voted for the LeGuin and Vonnegut, which both would have steamrolled onto the final ballot regardless. In contrast, in my example, fans of Crichton and Anthony hypothetically tended not to vote for Bug Jack Barron (perhaps finding it weird and challenging, not liking the authors politics, or for other reasons that seemed relevant to them in the summer of 1970). Why should a brilliant edgy writer who was ahead of his time have votes for him discounted because his voters also liked several more mainstream choices?"

Because more people nominated the mainstream choices; the Hugos are a popular award, not a judged award, and you opinion of a work's merit doesn't make a difference if other people don't agree with you. And SDV-LPE is just as likely to help Bug Jack Barron as hurt it; what if Macroscope had been released in two volumes, and most of its supporters has nominated both parts? Under the current system, it takes up two slots on the final ballot, leaving no room for Spinrad; SDV-LPE would probably eliminate one of them, and leave the 5th slot free for Bug Jack Barron.

"To extend the thought experiment: you are so blown away by reading LeGuin that you can think of nothing else and just want her to win at all costs. Why not just leave Spinrad's also-noble effort off the ballot entirely so that you don't detract from LeGuin's greater glory?"

That's exactly the same as under the current system. A vote for anything else in addition to LeGuin could be the difference between those two works tying for 5th, and the other one beating the LeGuin.

#177 ::: Laertes ::: (view all by) ::: May 13, 2015, 01:13 AM:

Also, seriously? Watch out for Brian Z. He's concern-trolling you, and at any minute is liable to run off to some other blog and spread vicious lies about what's going on here at ML.

#178 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:22 AM:

@174

"Your objection applies equally to the current system."

Are we sure? My gut reaction to this is that people may be given more incentive to nominate fewer works under the proposed system. In a perfectly well-meaning and innocent way. ("I'm a Jim Butcher maniac and the most important thing is for him to get as much support as he can, better not to split it with something else. I liked Goblin Emperor but Butcher is more important. And somebody told me Three Body Problem is a great book, but if lots of people like Goblin Emperor and 3PB they are probably going to win anyway and don't really need my vote, and if they did it would discount my support for Butcher and oh look at the time, why its already February anyway!")

#179 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:27 AM:

Laertes: I'm a knight, not a troll, and if you don't do exactly as I demand, I shall say "Ni" to you! :D

#180 ::: Jeffrey Smith ::: (view all by) ::: May 13, 2015, 01:31 AM:

Brian Z:

And just think how unhappy you'd be if some slate came in and knocked ALL of your choices off the ballot. That's the system you're advocating for here.

#181 ::: influxus ::: (view all by) ::: May 13, 2015, 01:31 AM:

Laertes @177:

I think Brian Z plays an important role as a hostile vetting process.

Brian Z @ 178:

That whole monologue you just imagined, works with the current voting system. See felice @ 176 for the precise case. By just voting once, currently, you cast 1 vote for Butcher and four for nobody but Butcher.

#182 ::: Annie Y ::: (view all by) ::: May 13, 2015, 01:35 AM:

Brian,

This explanation works for both the current and the proposed system. If anything, it is more likely to be a problem in the current system (I like A and B and C but a lot of people seem to like C so if I add a vote it can knock off B so let me skip it). Of course some people do not think strategically (which is a bad thing during nominations but that's a different story) and just nominate the works they will be perfectly happy to see on a ballot and winning - if I think less of book C than A and B, why would I put C on my ballot?

As for Butcher's fans - we can think, we do not go and just nominate Butcher because we really like him :)

#183 ::: Idumea Arbacoochee, who really is a moderator here ::: (view all by) ::: May 13, 2015, 01:35 AM:

I don't have time to untangle this in detail until tomorrow, but a few points:

1. I think we have come so far with this particular proposal that we should refine it rather than rewriting it in the light of late-coming commenters. I'd suggest focusing on practical objections rather than philosophical ones.

If other people want to make other proposals, I'd welcome links to similar discussion threads pursuing those. This is the one we seem to have settled on; let's run with it here. If a better proposal comes up elsewhere, it'll sweep the Business Meeting.

And that's also a win.

2. Someone please provide a link to the File 770 conversation so I can have a look at the context?

But I ask out of morbid curiosity as much as anything else. To a certain extent, we must accept that no matter what we say, there will be people elseweb who will spin it in the worst possible way, or flat-out lie about it. It's the nature of the conflict, because of the participants.

3. Brian, what I've seen of you elseweb does not fill me with confidence, but if you're participating in this conversation in good faith, that's fine. People who do conversation differently should also be able to use this system.


I will be back in a little while, when real-life obligations are not so pressing. I hate to sound like a mother when I say that I don't want to be ashamed on anyone's behalf when I do.

But there you go.

#184 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:38 AM:

influxus, thanks for the vote of confidence.

I agree some voters could (and do) already feel that way. I'm asking if the proposed system might incentivize more of it. I don't know the answer. Its a question about how voter strategy might change.

#185 ::: Annie Y ::: (view all by) ::: May 13, 2015, 01:40 AM:

abi, http://file770.com/?p=22470&cpage=5#comment-260835 is where it starts, go down from there.

#186 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:46 AM:

@ 182,

If I think less of book C than A and B, why would I put C on my ballot?

For example, I think Echopraxia is a breakthrough to a new level of science fiction, it is criminal that it doesn't even make the ballot. I think The Peripheral is a terrific return to writing stunning novels, but not a breakthrough of the same magnitude. I think Three Body Problem is a flawed book in some respects but also of tremendous importance to the field and absolutely deserving of a Hugo. So if I COULD rank my nominations in order of priority, that would be the order. Don't we all have such private rankings?

As for Butcher's fans - we can think, we do not go and just nominate Butcher because we really like him :)

Yes, up to a point, but if that starts to drown out more "traditional" voting practices of reading and nominating widely, the awards as a whole probably won't function as well (I think).

I could be wrong about this, but I didn't see anyone else asking on this particular how the proposed system will effect voters' strategies, and think that is an valid question in evaluating a new proposal.

#187 ::: Annie Y ::: (view all by) ::: May 13, 2015, 01:49 AM:

Brian @184

Well - we all (you included) may be SF fans but none of us can read minds (unless if someone steps up and say otherwise). So we cannot predict how fans will react and if the nomination patterns will change.

So let's take that one fan at a time. Would your vote been different last year if this was the rule? I've asked myself this question and my answer is "no". But I've already said that I nominate the works I will be happy to see win and not the ones I like and the once I really do not mind but rather not see win...

#188 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:51 AM:

185 thanks for posting the link.

Please note exasperation is aimed at the general tone (the File 770 thread is mostly meta-discussion) and I do apologize for leaving Kilo to be caught in the crossfire as it were by posting there and having people react to it - in retrospect I really should have posted my reaction here first.

#189 ::: Brian Z ::: (view all by) ::: May 13, 2015, 01:55 AM:

@187

Interesting question. Last year, I knew full well that none of my picks would make the ballot, and sure enough they didn't. Even altering my voting strategy wouldn't have helped. But if I feel all out determined for a particular thing to win and think it has an outside chance? I'm not sure.

#190 ::: Annie Y ::: (view all by) ::: May 13, 2015, 01:59 AM:

@186 Brian

Would you be happy if any of the three win the Hugo? I suspect the answer is yes - so you are fine nominating the three, right? Different books can make the nomination list for different reasons - as long as you are fine with them winning, it is your ballot.

But if you are putting a work that is not really at the same level and you know it but you just want to give it a nod, you may be responsible for this book knocking one of your other choices. In both systems :)

@188 Considering that I was the one that called you on it, I kinda knew how to find it fast. Citing (and reading) out of context - that's exactly what happened. Again. :) You (the generic you) start talking with the people, things can get resolved - as long as the good faith and the desire for that is there.

#191 ::: Brian Z ::: (view all by) ::: May 13, 2015, 02:06 AM:

For 2015, yes, I'd be fine with any of them making the ballot though I wouldn't have necessarily expected any of them to.

For 2014, I had wanted to vote my conscience. But I can at least imagine myself taking some kind Butcheresque tack ("Nicola Griffiths really deserves a nomination - this whole situation is hopeless and the best I can do is put my eggs in her basket.") I'd like to think I'd instead strive to be as "conscientious" as possible to all potential nominees but when the chips are down, who know?

"good faith and the desire for that"

If I didn't have some, I wouldn't be here concern trolling you. :D

And this highlights something else that I think is important. The Hugos should be a gesture that we the fans are collectively giving our authors - not something we are claiming for ourselves. Which is part of why it is not that important whether we get what we want are not.

But what do the authors want? To be honored as an example of excellence? To be ordained as Mr. and Ms. Popularity? Honestly these days it is getting hard to tell.

#192 ::: Brian Z ::: (view all by) ::: May 13, 2015, 02:16 AM:

180 sorry I missed this. But that's exactly my point. My personal happiness with having my personal opinion validated, or my belief the system if fair in the sense of giving as many people as possible something they personally like on the ballot, is not all that important to me. What's important to me is that the authors get a sense that all this hard work for almost no pay is worth something to us and we are grateful. That's the only reason not to let standards of excellence slip - the honor of being given this gift by fandom one might be lessened.

#193 ::: Annie Y ::: (view all by) ::: May 13, 2015, 02:25 AM:

@191/192 Brian
Ah, the authors. Most of them would love an award won cleanly - but not a lot of them are ok with being handed a rocket for being liked by 1 guy that just can organize a crowd. :) Which is what the current proposal is trying to solve - the Hugos are supposed to be a celebration of fandom and the authors and the works; not a celebration of 1 person's opinion of the genre. If the unwritten rules are not followed, we need to change the written ones.

Or do you think that a slate chosen by 1 person (or 2 or 3) is a fair way to nominate for Hugos? That it honors the authors? That it shows who fandom wants to honor? Fandom is everyone nominating; not just the ones that exploit a known crack in the process to make sure their ballots win the day. With your own words:
"My personal happiness with having my personal opinion validated, or my belief the system if fair in the sense of giving as many people as possible something they personally like on the ballot, is not all that important to me."

Then why are you supporting the slate voting when it is exactly the opposite of this? And if you are not supporting it (somehow), why are you willing to leave the system exposed to it again?

#194 ::: Annie Y ::: (view all by) ::: May 13, 2015, 02:31 AM:

OK - back on topic :)

Sounds like the biggest disagreement at the moment is how we treat withdrawals...

I am not really up to speed with all the rules but it is possible to change things during the ratification in Kansas, right? So why don't we set both possibilities, with Administrators set to be deciding at the start of the nominating period what they want to choose and then discuss again after Spokane when we know where we are with the rest of the proposal? Just... thinking aloud. That should allow us to get something that resembles an agreement together?

#195 ::: Brian Z ::: (view all by) ::: May 13, 2015, 02:42 AM:

@193,

Bear with me for a moment: In my opinion, Sad Puppies were originally trying to rally a fan base who doesn't often nominate to be more active in the process (which included a few individual authors or small groups of authors promoting themselves, like you are expected to do as an author these days.) Not to create slates or take over the award. But people started attacking them, some of them drank their own kool aid, and some seemingly started to believe the narrative that their silly campaign was actually some kind of vital front in the culture wars, which is ridiculous, and they let their rhetoric spiral out of control. Science fiction can never be reduced to any simplistic left-right split. In fact, very old and established elements of SFF debates (like the libertarianism on the right and also the left) are only now reaching the mainstream. We're light years ahead of the US culture wars, for crying out loud, why look backwards?


No, I don't support lockstep voting at all. I don't think anyone supports lockstep voting. In fact, if you proposed a rule with a clear and simple mechanism for preventing a group of lockstep voters from sweeping (Say, "in the event a set of identical ballots is large enough to place five works on the ballot in any category, the bottom two (or heck, maybe three) works shall be discarded and replaced with the works with the next highest numbers of ballots"), I'd have a lot more sympathy with it, personally speaking.

But the best solution is not to legislate - it is to welcome new voters into your culture and help them understand your traditions better - and get ready for a likely tidal wave of nominations in the future.

#196 ::: felice ::: (view all by) ::: May 13, 2015, 03:05 AM:

Brian Z @195: "if you proposed a rule with a clear and simple mechanism for preventing a group of lockstep voters from sweeping (Say, "in the event a set of identical ballots is large enough to place five works on the ballot in any category, the bottom two (or heck, maybe three) works shall be discarded and replaced with the works with the next highest numbers of ballots"), I'd have a lot more sympathy with it, personally speaking."

So the group put up a slate of four works, and sweep all but one slot; that's almost as bad. Or they tell their supporters to pick four out of the five slate candidates, and they might still sweep all the slots (because the ballots won't be identical). A voting bloc like this will just work around any rules that attempt to define a slate.

#197 ::: Brian Z ::: (view all by) ::: May 13, 2015, 03:11 AM:

196 felice, OK, fair point. Maybe you could say "in the event a set of ballots with either four or five identical works is large enough to place either four or five works on the ballot in any category, the bottom one or two (or else two or three) works shall be discarded and replaced with the works with the next highest numbers of ballots". Just an example of the kind of thing I'd personally be more receptive to.

#198 ::: Brian Z ::: (view all by) ::: May 13, 2015, 03:19 AM:

196 "tell their supporters"

Also, who is handing out marching orders to their minions? First, telling fans what to do is herding cats. Second, it seems Vox Day may have said to his little pirate crew that they should vote his straight ticket, but SP3 didn't say that. They made a mistake of perhaps inadvertently encouraging lockstep voting once, and I doubt they use that confused language again (I could be proved wrong I suppose). And if VD does, is he going to overpower the groundswell regular enthusiastic voters who are now motivated to nominate more vigorously? Seems highly unlikely to me. That's my personal take.

#199 ::: junego ::: (view all by) ::: May 13, 2015, 03:23 AM:

[WITHDRAWALS]

Annie Y @ 142
felice @ 139
I think that everyone understands that it is a weighted system. But in order to keep everyone that was already nominated we already break the proper order in case of withdrawals. So why stick to the weighted at this point and do not go for the old rule (the #6 and down were already weighted).
Or think of this this way - usually eliminations can be led to their ultimate end leaving an ordered list of N elements (the inverted elimination order). We declare this the Hugo Nomination System result and we get as many of them as we need - 5 (if over 5%), more in case of ties or in case we need to get an additional one because of removal.
Don't get me wrong - I understand your logic and in a normal case I would agree with you. But we already need to break the standard case because we are dealing with people :)

I've been wobbling back and forth about the "fairest" way to resolve this. I *think* there is consensus that once the finalists are notified, those nominees should not be removed by a later recalculation.

This year Worldcon "froze" the list to replacements when the ballots went to the printers. Was that "fair" to those who could have made the finalists otherwise? It's not like it would have been impossible to fix the problem, just really inconvenient and confusing.

Once the original calculations are completed we have a snapshot of the members' choices for the Hugos.

The present system uses the next-in-line replacement rule, and as mentioned upthread changing less is easier.

Given that we would be freezing the finalists list (and thereby being "unfair" to anyone who might have advanced in a full recalc) AND all of the above resons including Annie Y's points....I think it would be simplest to freeze the entire list and replace disqualifications/withdrawals #6, #7, etc. on the frozen list.

ps: I wouldn't be completely opposed to recalc'ing #6 down in case of withdrawal, but I think that might actually open up the process to even more rules lawyering and disagreement than just making a firm rule.

#200 ::: Soon Lee ::: (view all by) ::: May 13, 2015, 03:31 AM:

A suggestion Brian Z. and others: Can we stay on topic please? That is, the proposal for a change to the Hugo Rules to minimise the effect of slate or bloc-voting?

The wider ranging discussion is not germane here and to be honest, is an unnecessary distraction. There are other threads here & elsewhere where those sorts of discussions are being held.

#201 ::: Brian Z ::: (view all by) ::: May 13, 2015, 03:41 AM:

@ 200 Soon Lee,

Well, yes and no. The previous thread to discuss proposed rules changes carried an advisory at the top not to argue about whether changing the rules is a good idea or not, but this one doesn't (yet). I was sort of yanked back here and since then I've been mostly answering direct questions. But if no one else has questions, thanks for giving me this hearing, and I hope I've at least stated my case clearly.

#202 ::: Annie Y ::: (view all by) ::: May 13, 2015, 03:48 AM:

[SPECIFIC LANGUAGE]
(well... could not think of a better tag)

I was looking at 3.8.5
With the current rule, you need to count anyway so there is no point counting what is under 5% before you start the proper counting.

With the proposed change, doesn't it make sense to do straight counts first to eliminate all under 5% (because otherwise there is a chance that the elimination removes a work over 5%, leaves in the ballot one under 5% and then we are in the ineligible conversation again.

So shouldn't we first remove everything that we know cannot make the ballot (

#203 ::: influxus ::: (view all by) ::: May 13, 2015, 04:14 AM:

[SPECIFIC LANGUAGE] Annie y @ 202,

(Hopefully I'm following you this time.)

Given the exception to 3.8.5:

"except that the first three eligible nominees, including any ties, shall always be listed."

I don't think it is possible to rule the less than 5% out altogether. Also making something explicit around this would surely make the proposal contingent on keeping the 5% rule. I think you have just found another nugget for the withdrawal/ineligible conversation.

#204 ::: Annie Y ::: (view all by) ::: May 13, 2015, 04:34 AM:

Note to self - look before pressing post - sorry - one of the special characters ate the bottom part of my message and I managed to miss it in preview. Somehow. Need either coffee or sleep.

@203 influxus

Yup - this is what I was driving at. The rest of the message was talking about some other options besides full elimination early on - mainly to say to run all and grab the top 5 that are over 5% (maybe abi can fish it out from the backend?)

I still like the idea to just build the full list and then grab the first 5 that are over 5%. If we need replacements, we go down the list for the next one. It won't be mathematically perfect but we need to make allowances for things already on the ballot and for the 5% things which is a lot of non-perfect allowances already.

Otherwise we may need to rerun the thing a lot of times to fish the over 5% story out from the pull of the under 5% (and in some categories, it is a close call indeed on what is over 5 and what under 5 and the fractionals will allow some of the over 5 to slip down while the just under 5 raise up)

#205 ::: influxus ::: (view all by) ::: May 13, 2015, 05:14 AM:

@204 Annie Y

I did wonder about that final bracket.

IANAL, but... :-)
Does grabbing the first 5 over 5% violate 3.8.5 as it stands? Shouldn't it be the first 3 that are grabbed, and then 4 & 5 are selected from the next two works that have over 5%, if such works exist?

Maybe stating something along the lines of a general rule for removal due to late discoveries of ineligibility, withdrawals, and being below the threshold (5%) is easier.

I am also in favour of this rule being based on treating the first complete list as fixed and just moving down that list.

#206 ::: Annie Y ::: (view all by) ::: May 13, 2015, 05:26 AM:

That will depend if under 5% is considered a reason to make the work ineligible.

The way I read it is that it is there to guarantee that we have at least 3 works in any ballot. However - when 3.8.5 was written, the top 3 were always the real top 3 (so there was no way something with more than 5% to get under them if they were under 5% (or at least #3 was)).

However with the new proposal #3 can be 4.9% for example with 2 or 3 over 5% under it. So is #3 in this case protected under 3.8.5 and staying or is it eliminated because of the 5% rule as long as there is at least one over 5% work under it? No idea. We need someone that understand the rules a lot better to weight in (and we may need rewriting of 3.8.5 a bit to fit better). I think it is not protected but I am the first to admit that I am not good with legalese. :)

#207 ::: felice ::: (view all by) ::: May 13, 2015, 05:33 AM:

Annie Y @204: "I still like the idea to just build the full list and then grab the first 5 that are over 5%."

Anything below 5% needs to be eliminated upfront before feeding the nominations into SDV-LPE. Doing it the other way around would be deliberately introducing more of the same sort of unfairness caused by not rerunning after a withdrawal, and has absolutely no benefits. If there are fewer than 3 works above 5%, then don't eliminate anything upfront, but run SDV-LPE until three finalists are left instead of five. I believe that's a reasonable interpretation of how the existing 3.8.5 should be applied under SDV-LPE.

The 5% threshold is a bad thing and should be abolished, but that's a separate issue.

#208 ::: J Thomas ::: (view all by) ::: May 13, 2015, 05:44 AM:

158 ::: Annie Y ::: (view all by) ::: May 12, 2015, 11:10 PM:

@158 Annie YJ Thomas

"If we do make it as open and accountable as possible, we might find ourselves providing people with enough data that they can interpret it as the voting administrators doing something wrong."

And that is easily provable by releasing all the data if need be.

Well, no. The data will tend to show that we did not corruptly falsify the results.

But -- for example -- if we run the voting twice and get two different results, we can expect a controversy about which result is right.

Run it once, get Andrew, Betsy, Charlie, Drusilla, Edward. We publish this with numbers.

Andrew is removed, we run it again and get Betsy, George, Drusilla, Henry, and Inigo. We publish this with numbers too.

We should definitely declare Betsy and Drusilla winners. George almost certainly. Charlie, Henry, Inigo, and Edward are less certain.

If we publish both outcomes we have trouble. P J Evans argues that we shouldn't worry about it because today's problem is dealing with slates. I don't think we should mine our own future path. "Don't worry about it, it's only one little landmine and we're very unlikely to step on it."

One solution is to announce that we will never re-run the data. Take everything from the old data and never let anybody find out what would have happened if it got rerun. If people can't find out who would have won otherwise, then they won't argue about it.

A second solution is to rerun the data and declare that this is the correct result. Don't publish the old, incorrect data because it would cause trouble. Or publish it with the observation that Andrew threw it off.

Publishing two different ways to count the votes and including some winners from one and some from the other is just asking for argument.

We get the same problem though less if we publish two results, and either take the #6 from the obsolete result or the top new result from the new result, particularly if it's #4 or higher. Between those two choices, we do better to take the old #6 and never run the data again or allow anyone else to.

The most defensible approach is to rerun the data following the rules, and accept what it says. If people disagree, point out that this is the correct result without Andrew. But that's bad for authors who were told (or implicitly given reason to think) that they had won and who then lost.

The next-most-defensible approach is to run the data once and use those results, and never let anybody run it again.

This is not related to slates. This is a new problem that might never come up because with luck it will not happen before the rules get changed again.

#209 ::: Annie Y ::: (view all by) ::: May 13, 2015, 05:49 AM:

felice@207

In which case in most categories we will have an elimination round or 3 and we will be done. I really do not like removing all the under 5% before running the eliminations - it will kick a huge percentage of people's ballots out immediately, changing the vote distribution (especially when there is a lot of works around 5% but just under it).

Let's look at London:
Novellete has 7 works over 5% - that's 2 elimination rounds
Short story is at 4 (not too surprising really)
Best Related has 7 - so 2 here as well.
Even novel is with 11 over 5%. Which means 6 rounds if we start from these.

So what 15 rounds of elimination are we talking about (for the publication)? That will never happen (you need 20 nominees for that)...

#210 ::: influxus ::: (view all by) ::: May 13, 2015, 05:50 AM:

felice @ 207:

There is a big reason not to do that. In the short story category you could wipe out almost everything but a slate. IE reduce the field to 6 before SDV-LPE.

#211 ::: felice ::: (view all by) ::: May 13, 2015, 06:12 AM:

J Thomas @208: "Publishing two different ways to count the votes and including some winners from one and some from the other is just asking for argument"

What about including all the winners from both? Betsy, Drusilla, George, Charlie, Henry, Inigo, and Edward, in this example. It's going to be pretty rare for rerunning to change the outcome that much.

Anything that relies on keeping the data secret is a bad idea.


Annie Y @209: "In which case in most categories we will have an elimination round or 3 and we will be done. I really do not like removing all the under 5% before running the eliminations - it will kick a huge percentage of people's ballots out immediately, changing the vote distribution"

Changing the vote distribution is important! If works aren't eligible, people's votes need to be redistributed to works that are (if they nominated any). Intentionally having ineligible works eliminating eligible ones makes no sense at all.

"So what 15 rounds of elimination are we talking about (for the publication)?"

Obviously we can only publish the rounds that happen; if that's less than 15, it's less than 15. This should probably be a supplement to the existing publication of the top 15 nominations, rather than a replacement for it.


influxus @210: "There is a big reason not to do that. In the short story category you could wipe out almost everything but a slate. IE reduce the field to 6 before SDV-LPE."

I agree that's a bad thing, but the problem is the existence of the 5% threshold, not how it applies to SDV-LPE in particular. We've got exactly the same problem with the current system.

(As an alternative to abolishing the threshold, I've got a longlist proposal that would increase nominations for the most popular candidates, but that's on hold till after SDV-LPE is finalised - search for "Option 5b" in the previous thread if you're interested)

#212 ::: influxus ::: (view all by) ::: May 13, 2015, 06:13 AM:

J Thomas @ 208:

"One solution is to announce that we will never re-run the data. Take everything from the old data and never let anybody find out what would have happened if it got rerun. If people can't find out who would have won otherwise, then they won't argue about it."

Have you met humans?

Anyway, if we only run the data once, and

(1) this is part of the rules, and
(2) we release all the data.

Then any controversy about alternate histories can simply be referred to the rules.

My argument for not rerunning boils down to:
The rules are fair because a fair procedure was used to produce the ranked list. Removals were not removed as part of ordinary eliminations, hence they are a special case. Unless there is a clear reason otherwise the simplest rule should be used to fill special cases. Given the length of this debate over removals, it seems pretty clear that there is no definitive reason rerunning is fairer than keeping the list intact. Keeping the list intact is simpler.

#213 ::: Ben O. ::: (view all by) ::: May 13, 2015, 06:23 AM:

On [DECLINED NOMINATIONS], here's another vote for the "run once" option articulated in #43. While there's been much discussion of the nominal theoretical benefits of rerunning the nomination data in the event of a declined nomination, we also need to recognize that it will take a certain amount of time and effort to generate a revised calculation. What would rerunning the nominees entail - can we assume that there will be robust code that asks the user to enter one (or more) nominees to ignore, or will the nominee data need to be massaged? What if a nomination is declined at the last minute and the final list is due at the printers in a few hours - it would be a shame to mistakenly select the wrong nominee just because the exigencies of the moment didn't allow for a proper calculation. It may sound like a picky point, but all this will be on the shoulders of volunteers who probably have jobs and families.

On another point, it seems to me that it may be worth setting an upper limit on the number of nominations that a person could make in a category. Without one, odds are that some joker will nominate a thousand titles. A generous but reasonable limit shouldn't give cause for complaint - ten? twenty? If someone nominates more than the limit, their ballot should be disregarded for that category.

#214 ::: J Thomas ::: (view all by) ::: May 13, 2015, 06:32 AM:

@178 Brian Z

My gut reaction to this is that people may be given more incentive to nominate fewer works under the proposed system.

That depends on their gut reactions, which are hard to predict.

If one person does that, it won't have much effect at all. If a bunch of people do it, it has a bad effect on the system as a whole, and probably doesn't help them since they'll tend to act against each other.

In 2013, people nominated 475 novels. 5 of them won. If you care more about increasing the probably small chance that your most-favorite work wins than about increasing the chance that one of your five most-favorite works win, then you should vote for just the one.

It's always been that way. Quantitatively it isn't much more that way now. It's hard to quantify, and probably people will go by their gut feel, and I can't predict that but I hope they won't go that route.

#215 ::: Annie Y ::: (view all by) ::: May 13, 2015, 06:32 AM:

Ben O. @ 213

There is no proposed change in 3.7.1 which caps the nominations at 5 (exact text: "up to five (5) equally weighted nominations in every category.") so the above text need to be read in the context of this restriction.

#216 ::: influxus ::: (view all by) ::: May 13, 2015, 06:39 AM:

felice @ 211: "I agree that's a bad thing, but the problem is the existence of the 5% threshold, not how it applies to SDV-LPE in particular."

But we are discussing how it applies to SDV-LPE in particular. If SDV-LPE gets up and the threshold is not removed SDV-LPE should behave as robustly and simply as possible. I think folding ineligibilities, withdrawals and threshold eliminations into a single rule is important for simplicity.

#217 ::: J Thomas ::: (view all by) ::: May 13, 2015, 06:42 AM:

@211 felice

"Publishing two different ways to count the votes and including some winners from one and some from the other is just asking for argument"

What about including all the winners from both? Betsy, Drusilla, George, Charlie, Henry, Inigo, and Edward, in this example. It's going to be pretty rare for rerunning to change the outcome that much.

Yes, that sounds excellent to me. Usually it would make no difference. Rarely it would be different and you'd have six. Nobody is hurt much if it happens -- six works competing on IRV instead of five. If we assume that the two candidates that only won once will tend to be the weakest ones, they don't much hurt each other or anybody else when they both get their chance.

So the result would be six candidates rarely, seven candidates extremely rarely, you can publish any data you want and nobody should have a gripe with the result.

Beautiful!

#218 ::: influxus ::: (view all by) ::: May 13, 2015, 06:46 AM:

Ben O. @ 213, Annie Y @ 215:

Maybe this issue of capped nomination numbers should go in the FAQ. I made this mistake, as did another person on File770.

Can I nominate hundreds of works?

There is no proposed change in rule 3.7.1 which caps the nominations at 5 (exact text: "up to five (5) equally weighted nominations in every category."). SDV-LPE is still governed by this rule.

#219 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 07:20 AM:

I agree with @181: insofar as BZ is maintaining the appearance of legitimate debate, it doesn't matter if in his heart he is trolling. We should be careful about three-goalpost-monte, but if he raises something that others will worry about too, we should respond without name-calling.

So, what are the legitimate points he has raised?

He's brought up the thing about bullet voting. I think we've dealt with that one. Joshua @123 did a good job explaining why it's not actually a very powerful strategy. A few further points: in the end, if a few people do use this strategy, the system is robust to that; and they probably would have done it under the current system; and if they'd do it under any system, you can hardly even call it strategy, because it's just how they feel. I think if people come back to this point again, we can just point them to the same arguments, and not get distracted by restating them in different words.

He's brought up the question of what the goal is, and alleged that the goal of SDV-LPE is to maximize voter happiness. If he were a troll, this would be pretty sophisticated work; given a random member of my electology organization, probably the quickest way to troll them is to argue against voter happiness as a metric. But actually, even though voter happiness is largely the point of my organization, it was not the point when I designed this proposed Hugo nomination system.

The point of this system (SDV-LPE or whatever we call it) is to try to ensure that the eventual Hugo winner is excellent. The reason that the nominees should represent a variety of fans is not to make them happy, but because by representing a variety you avoid making the same mistake 5 times.

Are there cases where this system would pass over a more excellent work to nominate a less excellent one? Sure; there are several ways that could happen.

First, if the voters are all wrong about something, then there's no way a voting system can get it right. As far as I can see, there's nothing more than that to the "Macroscope" examples. If Spinrad and Vonnegut got votes only from people who'd already nominated "Left Hand", while "Macroscope" was supported by a separate but slightly smaller group of fans, then yes, there is a possibility that Spinrad and Vonnegut would end up together in a matchup, and that only one of those two would make the final ballot. There is not a possibility that both would be eliminated. So the final list might have LeGuin, Vonnegut, and Anthony, while Spinrad might be knocked out. If that's what the voters said, then that's the result.

Separately, assuming voters do NOT bullet vote, there is a chance that a well-crafted but essentially formulaic work could beat out an actual masterpiece. But imagine what that would look like on the ballots: "Formula 1" has support from 40% of the voters, while "Master and Apprentice" has support from 15%. So while I can imagine designing a voting system that would be more likely than SDV-LPE to elect "Master and Apprentice", I can also say that this hypothetical "pearls before swine" voting system would get things embarrassingly wrong (think "Chthon", by Piers Anthony) at least as often as it got a better answer than SDV-LPE.

I'd be interested to hear if other people think this is an adequate response to BZ's concern.

#220 ::: Louis Patterson ::: (view all by) ::: May 13, 2015, 07:30 AM:

Not a hugo voter, but:

If a work is determined to be ineligible countback/rerun is really the only option, I think. If work X shouldn't have been included in the count in the first place then including it is an error and the only way to fix that error is by rerunning the process properly, with ineligible entries excluded.

If the two runs generate different shortlists then it's pretty obviously the first one -- that counted the ineligible works -- that's in error, and it's the ones from the initial list that are getting concessionary treatment if [as I think they should be] they are included in the final shortlist.

[it's unclear to me how/if this applies to voluntary withdrawals]

#221 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 07:33 AM:

[DECLINED NOMINATIONS]

I think that each worldcon committee should have the freedom to predeclare whether they want to rerun and merge lists, or just take the last out, first in candidate. This predeclaration should be made at any time prior to the close of voting. If no such predeclaration is made, the default should be last out first in (under the assumption that a committee which misses the declaration deadline is one which can't handle complexity).

Remember, our best estimate, grounded in the 1984 data, is that this would only make any difference for under 1 in 100 cases of declined nominations.

#222 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 07:42 AM:

Oops. When I said "Chthon", I meant "Phthor".

#223 ::: J Thomas ::: (view all by) ::: May 13, 2015, 09:05 AM:

@219 Jameson Quinn

If Spinrad and Vonnegut got votes only from people who'd already nominated "Left Hand", while "Macroscope" was supported by a separate but slightly smaller group of fans, then yes, there is a possibility that Spinrad and Vonnegut would end up together in a matchup, and that only one of those two would make the final ballot. There is not a possibility that both would be eliminated.

This is a very minor thing, but they could both be eliminated as follows: If LeGuin also had a lot of shared ballots, LeGuin could also have a low enough score to eliminate both Spinrad and Vonnegut, or whichever of them survived their own matchoff. We can assume something else in 2nd and 3rd place, so this would require that Anthony and another both have scores higher than LeGuin has during the last elimination round. In the extreme case, if all of Spinrad's votes are shared with LeGuin and the 2nd place and 3rd place, and all of LeGuin's are shared with the 2nd and 3rd place, Spinrad, and some of the lesser works that haven't been eliminated yet, then Anthony and the others could have a little more than 1/4 as many votes as LeGuin while Spinrad and Vonnegut both have almost as many as Leguin.

But the extremely rare pathological cases don't really matter much. It could be argued that they aren't really even pathological, the system is doing what it's supposed to do. Just not quite what a naive voter might expect.

But a naive voter should expect slates to win big. If the voting system does everything he expects, that isn't good either.

#224 ::: JAB ::: (view all by) ::: May 13, 2015, 09:24 AM:

I like the idea of SDV. To be honest, though, it didn't really click for me until I started making a physical model in response to Bruce Baugh's request in @52. I don't see that anyone has posted one yet, so I'd be glad to make and share photos of it.

But I need an example to illustrate. Does anyone have a link to a very small example that I can implement? Is there an official small example? I'm just dipping in after not following the previous threads closely.

#225 ::: Seth ::: (view all by) ::: May 13, 2015, 09:39 AM:

If eliminating every candidate that appears on under 5% of the ballots leaves fewer than three, then don't do it (obviously). In that case, run the elimination rounds down to 3 (plus ties).

If eliminating every candidate that appears on under 5% of the ballots leaves 3-5, then done. Those appear on the final ballot (subject to declined nominations).

If we use re-run in the event of declines (which is the right thing to do), then there's a potential ambiguity with multiple declines. I would say that in the case of simultaneous declines, re-run with all of them eliminated. Else, use sequential re-runs. I define "simultaneous" as the new decline becomes known to the administrators prior to notification of the new potential nominee(s) after a re-run.

#226 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 09:42 AM:

@223: The scenario you paint, where both Spinrad and Vonnegut are knocked out despite being 4th and 5th place and not having significant overlap with 2nd and 3rd place, allowing 6th place Anthony and 7th place whatever as nominees, may be possible. But I'd bet that it would take me at least 3 tries to put numbers on it to make it work right. If somebody like me, mathematically sophisticated and steeped in voting theory, can't get it to work on the first try, I think that the probability of it happening by chance are effectively 0 — that is, less than one in ten thousand.

#227 ::: J Thomas ::: (view all by) ::: May 13, 2015, 09:59 AM:

If somebody like me, mathematically sophisticated and steeped in voting theory, can't get it to work on the first try, I think that the probability of it happening by chance are effectively 0 — that is, less than one in ten thousand.

I agree it's very unlikely. It isn't very important that it's possible, I just got caught up in the moment when I saw you say it was impossible.

What's impossible is for Leguin with the most votes to lose. No combination of improbable voting circumstances can get that result. Also, nothing with less than 1/5 of Leguin's starting score can possibly win, provided there are at least 4 others with more than 1/5 of Leguin's starting score.

I thought it wasn't completely irrelevant that the extremely rare event could happen. Noticing how it could happen might help people to better understand the system.

But it isn't worth a lot of talk unless someone doesn't understand it.

#228 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 10:14 AM:

JAB@224: I can run anything you need. Can you describe the type of set you're looking for?

[DECLINED NOMINATIONS]
All:
I've been reading all the posts on the subject, and I think we can all agree that there are two very distinct opinions on "fairness" with regard to this subject. I don't think we can say, therefore, that there is an objective definition of "fairness" in this case. My thoughts are this:

- The current constitution does not specify how to handle declined nominations. We have a method based on tradition, but that's it. Right now, it's ultimately up to the Hugo admins to handle it however they like. Strictly speaking, they don't have to replace -anyone- if the nomination is declined.

- Therefore, let's keep it this way, and let's allow tradition to grow and decide what's "fair". As a first cut, the Hugo admins could decide to stick with the current tradition and move the work eliminated in the next to last round to the ballot.

- If, after a year or so, fandom decides that they don't like this tradition, the Hugo admins can try something else (such as re-running the election) and see what fandom thinks about that. The important thing here is that as it stands, this would -not- require a constitutional amendment.

I think doing it this way gives the Hugo admins the agility to respond to what fandom decides is fair. More importantly, it let's fandom decide what's fair. Organically grown tradiations are kind of what our community is about. I think this is a perfect opportunity. If a new tradition is needed, it will develop naturally, so long as the Hugo admins have the freedom (as they do now) to handle it as they see fit.

Kilo

#229 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 10:30 AM:

[5% RULE]

I'd like to see the 5% rule eliminated, as I don't really think it serves its intended purpose. However, I believe Kevin said there is already a proposal to do that, so I don't think we should include it in ours. However, we do need to be sure we can handle it if the 5% rule is not removed.

If the 5% rule is not removed, I think the simplest way -- which draws back to the previous discussion -- is to treat any work getting less than 5% as an ineligible nomination. However the admins decide to handle those is used here.

That said, however, I will lobby extensively in support of those removing the 5% rule, as I think that's the most logical course of action.

Kilo

#230 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 10:43 AM:

Annie Y @194:

I am not really up to speed with all the rules but it is possible to change things during the ratification in Kansas, right?
Yes and no. Here is the exact wording, right out of the current WSFS Constitution:
Section 6.6: Amendment. The WSFS Constitution may be amended by a motion passed by a simple majority at any Business Meeting but only to the extent that such motion is ratified by a simple majority at the Business Meeting of the subsequent Worldcon.
This means that if you pass a change to the constitution this year, you can modify it next year, but only if doing so does not increase the scope of the amendment.

Example: Suppose we pass an amendment reducing the number of nominations per member per category from 5 to 3. Next year, you could move to change "3" to "4" and ratify it immediately because it would be less of a change than what was originally passed; however, if you changed it to "2", the change would be greater than what originally passed and the motion would have to lay over for one more year. (Possibly longer, if the Popular Ratification proposal is ratified this year.)

Wholesale reworking of a proposal is not allowed, and the more you try to change something at the ratification stage, the more likely it is to be ruled a greater change and thus require an additional round of ratification. I also suggest that anything that completely rewrites the existing nomination system, effectively throwing away the current system and starting over, would be very difficult to change at the ratification stage in any way that would not be considered a greater change.

#231 ::: Lydy Nickerson ::: (view all by) ::: May 13, 2015, 10:50 AM:

A quick data point, here. I am hugely math-challenged, but I don't find divvying up 60 points, or 100 points, easier to understand than divvying up one point into fractions. In some ways, I actually find the multiple points more confusing, not less. It requires an additional amount of explanation as to why you choose that many points, whereas one point divvied up between the total number of nominations seems easier to understand. If, in fact, it's numerically identical, then I'd go with the one vote thing.

Oh, and I _hate_ fractions.

#232 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 10:53 AM:

Keith @229:

I'd like to see the 5% rule eliminated, as I don't really think it serves its intended purpose. However, I believe Kevin said there is already a proposal to do that, so I don't think we should include it in ours.
That is correct; there is such a proposal already submitted. I'm discouraging anyone submitting additional proposals from trying to touch that section. Indeed, I've recommended that anyone doing omnibus changes try to break them into individual pieces so that the meeting can divide them into separate questions. (There's a proposal cooking up on Facebook, for instance, to combine the two "middle length" story categories and to carve out a "Best Saga" category for very long series works; I asked them to draft it as two clauses so they can be considered separately if the meeting so desires. They've put the revised Best Saga proposal on Google Docs for public scrutiny.)

We now have under the WSFS Business Meeting section of the Sasquan web site a list of New Business Submitted to the 2015 WSFS Business Meeting. We'll update this as we receive additional proposals.

#233 ::: Yarrow ::: (view all by) ::: May 13, 2015, 10:53 AM:

[DECLINED NOMINATIONS]

I'm writing mostly to support Kilo's "let the Hugo admins handle it" position.

I also think that, on one view, moving the next work up is not only less work than rerunning the process, but also more fair.

On that view, what a nominator gets when a work they like gets on the Hugo ballot is the opportunity for that work to be sent to the Hugo electorate. (Presumably this part is not controversial!) And what a nominator gets when a work they like is about to go on the ballot - that is, just before the work's author gets the call from the admins - is also the opportunity for the work to be sent to the Hugo electorate. But for that electorate, one voter (the author) gets a veto - if the author declines the nomination, the work doesn't get the Hugo.

So when an author declines, nominations for that work have not been wasted, any more than nominations have been wasted for works that turn out not to win the Hugo. In both cases, the work has passed the first gate, of getting on the short list. If an author declines the nomination, the work has not passed the second gate, and if the Hugo voters don't select the work, it has not passed the third gate; but in neither case have the nominators' choices been ineffective.

#234 ::: Lydy Nickerson ::: (view all by) ::: May 13, 2015, 10:56 AM:

And an additional thought: if it is described as divvying up some number of points, a lot of people are likely to want to know why they can't assign their points as they please. 35 points to choice A, 10 points to choice B, and so on, up to the number of points allotted. Which is so totally not what you want.

#235 ::: Buddha Buck ::: (view all by) ::: May 13, 2015, 11:02 AM:

Kevin Standlee, while you are here,

Would it be possible for the WSFS Business meeting to pass multiple, possibly contradictory, revisions to the nomination process, allowing those options to be discussed and experimented with until the following year, when only one (hopefully) would be ratified?

#236 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 11:04 AM:

Lydy@234

Regarding 60 points vs fractions and wanting to divide however the voter wants..

That's a good point. That argument actually has me convinced that we should just stay with a single point and divide it up.

Kilo

#237 ::: P J Evans ::: (view all by) ::: May 13, 2015, 11:24 AM:

233
THIS.

#238 ::: Cadbury Moose ::: (view all by) ::: May 13, 2015, 11:59 AM:

P J Evans @ #237 wrote:

233

THIS.

Indeed, and any declined nomination is not wasted since it will still appear in the published nomination stats flagged as (declined) after the awards have been made.

(So no rerunning of the selection is necessary (or, for that matter, desirable).)

#239 ::: P J Evans ::: (view all by) ::: May 13, 2015, 12:09 PM:

238
Andy Porter declined in 1984 for semiprozine (for 'Starship') - that's on the 'more details' page. There's some stuff going on there that I never knew or don't remember, since he doesn't seem to have had the votes anyway.

Timothy Zahn had the nominations for the Campbell, but he wasn't eligible, so they moved up the next two nominees (there was a tie).

#240 ::: Nancy Lebovitz ::: (view all by) ::: May 13, 2015, 12:26 PM:

I'm wondering whether an animated explanation-- watching the votes come in and seeing the ranks move up and down-- would help.

This is very tentatively suggested.

#241 ::: Bruce Baugh ::: (view all by) ::: May 13, 2015, 12:38 PM:

I spent some time last night thinking about suggesting a no-fractions approach, then had the same realization that Lydy describes in #234, and thought, nope, sit on that one. :)

#242 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 12:55 PM:

Buddha Buck @235:

Kevin Standlee, while you are here,
I've been watching this discussion, so assume I'm always here.
Would it be possible for the WSFS Business meeting to pass multiple, possibly contradictory, revisions to the nomination process, allowing those options to be discussed and experimented with until the following year,...
Possible, in the sense of not prohibited, yes. I personally think it's a bad idea and discourage it, but I recognize that there seem to be a bunch of people who are convinced that We Must Change Everything In Every Possible Way it Can Be Changed. *sigh* Also, I don't like dumping a parliamentary mess into my successor's lap.
...when only one (hopefully) would be ratified?
In case of contradictory proposals, it's axiomatic that only one can possibly be ratified. How we get to that point is not yet clear. In the absence of any other decision (see below), I'd rule (if it were my call, which it is not) that the last thing ratified is the most recent change, and that it overwrites anything ratified before it. That's not terribly satisfying, I admit, and therefore I really hope that people come up with ways to not have their proposals contradicting each other.

(One possibility would be to put all conflicting proposals up to a ballot vote by IRV, which would guarantee that only one of them could get ratified. But that has its own baby-with-the-bathwater problems.)

As far as "experimenting" with them goes: Nothing passed this year will be in effect next year. I don't know what you mean about "experimenting" with them. You can't compel next year's Administrator to do anything not in the current rules. If you think you're going to convince the 2016 Hugo Awards Administrator to run multiple simultaneous versions of a voting system in order to test theories, I suggest you contact him first to find out how plausible that is. (Not very, IMO, but I'm not Dave McCarty and therefore do not know for certain.)

#243 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 01:10 PM:

@242: "(One possibility would be to put all conflicting proposals up to a ballot vote by IRV, which would guarantee that only one of them could get ratified. But that has its own baby-with-the-bathwater problems.)"

Aaak, no! From a voting theory perspective, the clearly "right" thing to do is to use approval voting; that is, whichever passes with the most "yea" votes is the one that stands. That's what the state of Oregon did when they had that problem one year, and it avoids the complicated ballots and potential pathologies of IRV.

One way to do that would be, in the ratification year, add language to each of the proposals saying "If this proposal does not get more yea votes than either X or Y, it is null and void." Since that's clearly a reduction of scope, it is OK to add it in year 2.

But I agree with you that by far the best way to deal with this problem is not to have it.

...

Seperately: Is there currently a proposal to lift the 5-works-per-category-per-ballot limit? I think that such a proposal would go well with this voting system, but they should not be tied together.

#244 ::: Tim Illingworth ::: (view all by) ::: May 13, 2015, 01:41 PM:

@243 I think I'd treat them all as amendments-by-substitution to a null motion.

#245 ::: JAB ::: (view all by) ::: May 13, 2015, 01:51 PM:

Keith Watt @228:

Thanks for the offer. What kinds of ballots to include comes down to, what are the important parts of the system? What does the model need to show? What sorts of scenarios should be included? Definitely some slates; what else? How much should the ballots overlap on works? As I said, I don't know what would be most illuminating. Coming up with a good small but nontrivial numerical example to demo the system is hard, and right now I am too overwhelmed to spare the skull-sweat.

So, here's what I thought of: (and if someone else wants to do it -- especially if they can animate, go ahead with my blessing)

1. Ballots are equal-sized strips of thick cardboard, 1 per voter. The nominated works are represented as equal-sized stripes. Each work in the pool has its own color (+pattern for the colorblind folks). So a ballot for ABCD has 4 stripes, red, blue, green, and yellow. A ballot for CD has the top half green and the bottom half yellow. A ballot for E only is a solid black. And so on.

2. Each strip is cut apart on the color lines and the votes for each color are lined up (basically, a bar chart).

3. Each of the 2 shortest bars are picked up and their components stacked on top of each other. The shorter stack is out. (This is why chipboard/binder's board, not paper or cardstock).

4. Then the ballots are reassembled minus the missing strips. Some of these ballots are now smaller/shorter than others. The purpose of this picture is to show why it's fair to reweight votes and/or be a nice sound bite for "what's wrong with the existing system".

5. The ballots are shaved down so that they're the same size again (proportionally for each color). There's a sidebar that shows how equal amounts were removed from each ballot and it was split between remaining colors. Going back to the example above, if A is eliminated, the CD ballot loses 2 strips (one green and one yellow), each half the width of A. The E ballot loses a strip the size of A, so that now the BCD ballot is again the same size as the CD ballot and the E ballot. Etc etc.

6. Rinse and repeat until n nominees remain.

Basically, cutting is going to be a pain. So I'd want enough voters and works to show what's going on, but only 3-4 rounds to get winners.
It'd be nice if the fractions were easy to cut -- so fewer works per ballot, but not too few.

Tie-breaking and nominees declining is overloading the cardboard strips. But it wouldn't be hard to show the minimum nominations rule - just stack up by works (colors) and have a "You must be this tall to ride" sign. Is that important to include?

There's probably an easier way to do it; feel free to suggest how.

#246 ::: Brian Gibbons ::: (view all by) ::: May 13, 2015, 01:54 PM:

Jameson Quinn @243: Is there currently a proposal to lift the 5-works-per-category-per-ballot limit?

I would be surprised if there were not a proposal to simply have the final ballot include every nominee that cleared the 5% threshold. That seems to me to be the obvious philosophical opposite of this one: Rather than going to great lengths to determine who the 'best' top five nominees are, just put everyone who qualifies on the ballot.

That obviously comes at the expense of making 'Hugo nominee' a much less meaningful descriptor (10 nominees would become fairly commonplace, and even 15 not unheard of), but if you think that internecine fights are already doing that and will continue to do so, it might be a cost you're willing to pay. It also has obvious appeal to anyone who prefers simple solutions.

It's not clear to me that passing multiple contradictory proposals to Kansas City is the worst possible outcome.

Picking the One True Change at Sasquan means that next year's BM is going to be faced with a "take it or leave it" proposition. If you think that this a serious problem that needs to be solved (and I don't think we should be amending the Constitution if it isn't), then that's essentially giving them no choice but to ratify whatever this year's BM does.

In a sense, picking just one of the options this year is the "Adopt in Haste" path. I have to assume that a meaningful percentage of attendees at the Business Meeting are to be learning about the issues and possible solutions for the first time, and I'm not sure that it's a good idea to force an immediate choice as to which good option is the best one.

I don't think it would be a bad thing if this year's Business Meeting was treated, well, like picking nominees for a final ballot. Winnow through the candidates, of course, but if there are multiple options that the voters think are acceptable, I think that leaving for next the choice as to which is the best option is not a bad thing.

#247 ::: Fragano Ledgister ::: (view all by) ::: May 13, 2015, 02:19 PM:

It is correct that this system would weaken slates, though not completely eliminate them. It would, in effect, tend to benefit bullet voting.

Also, I want the time I wasted reading Macroscope back.

#248 ::: Buddha Buck ::: (view all by) ::: May 13, 2015, 02:24 PM:

When I talked about "experimenting" with them, I meant in much the same way we are doing here: Writing different implementations, testing them with variants of ballot data, seeing how resilient they are to various forms of strategic nominating (slates, bullet nominations, etc).

I would definitely consider it the responsibility of the promoters of the various proposals to do the experimenting, not the Hugo Administrators.

#249 ::: Chris ::: (view all by) ::: May 13, 2015, 02:27 PM:

JAB @245:

5. The ballots are shaved down so that they're the same size again (proportionally for each color). There's a sidebar that shows how equal amounts were removed from each ballot and it was split between remaining colors. Going back to the example above, if A is eliminated, the CD ballot loses 2 strips (one green and one yellow), each half the width of A. The E ballot loses a strip the size of A, so that now the BCD ballot is again the same size as the CD ballot and the E ballot. Etc etc.

Rather than shaving ballots down, I'd suggest stretching the ones that have lost candidates back to full size. Mathematically identical, but the visual metaphor (you haven't lost any of your voting power when you lost a candidate) is better.

#250 ::: Annie Y ::: (view all by) ::: May 13, 2015, 02:34 PM:

Kevin Standlee @230

Thanks. So it will go down to what the Kansas business meeting attendees believe to be narrowing in scope.

[5% RULE]
So if there are other proposals to get rid of that one (yey - it made sense when the number of ballots was very low but not anymore...) shouldn't this proposal not quote the articles that are not changed (3.8.2-3.8.7) but add somewhere in the 3.8.8 section that all works that are ineligible under any previous articles are removed before the elimination rounds?

This way we leave it open enough for a change in 3.8.5 and we do not end up with accepting the old 3.8.5 again on top of already accepted new one?

It may be me just not understanding the process, but if this text as it stands is accepted, the non-changing sections are also getting re-accepted? Or do we add them here only for clarification (put them in brackets then?)

#251 ::: JAB ::: (view all by) ::: May 13, 2015, 02:39 PM:

Chris @249:

Yeah, that does work better. If I could animate on the computer, I'd do that. Shaving is a workaround because I was going to use actual chipboard and magic marker. I wonder if there's a better (household) material? Maybe play dough and food coloring, if I could figure out how to mix enough different colors.

#252 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 03:37 PM:

@247: In a world without slates, there is essentially no benefit to bullet voting. Approval totals and honest point totals would agree on which candidates were better, so if your favorite candidate was losing, boosting its point total would only put off the inevitable (and if everybody did it, you could actually be worse off, because your favorite would be getting no second-choice votes).

In a world with slates, there is a small chance that boosting a work's points will let it avoid becoming collateral damage to a slate. But... if slates can't get more than their share of slots, they're much less likely to exist.

#253 ::: viktor ::: (view all by) ::: May 13, 2015, 04:05 PM:

The answer to FAQ #10 should be changed to Yes.

Question: why can't I list a work more than once on my ballot? If I want Alpha to get more of my 1.0 than another nominee, I should be allowed to nominate as follows:

Alpha
Alpha
Beta
Charlie
Delta

Thus giving Able 0.40 pts.

#254 ::: P J Evans ::: (view all by) ::: May 13, 2015, 04:08 PM:

253
Because that's Not Done: it's a form of ballot-box-stuffing, which is Really Not Done.

(In the past, people have put stuff on their list more than once, and it's been counted - but only as one.)

#255 ::: viktor ::: (view all by) ::: May 13, 2015, 04:15 PM:

Under the proposed rules, I should be allowed to allocate my 1.0 anyway I want.

#256 ::: Laertes ::: (view all by) ::: May 13, 2015, 04:31 PM:

viktor @255: Can you elaborate on why that's necessarily true? It's not obvious. I might, for example, have some idiosyncratic urge to break my vote up into chunks sized 2^-2 and 1 - 2^-2 because I've got a thing for irrational numbers, but it's not clear to me that the rules are flawed if they don't permit this.

#257 ::: Joshua Kronengold ::: (view all by) ::: May 13, 2015, 04:32 PM:

viktor@253: It's an interesting form of ranking. But it's also more complicated, so we decided to go with a simpler, less nuanced proposal.

Annie@250: No, we want to list the unchanged bits because it provides context for the rest. Functionally, an "as follows" amendment is a diff. Strikeouts mark the bits that are being removed; underlines mark the bits that are being added (we used italics here because you can't do underlines on ML).

#258 ::: Laertes ::: (view all by) ::: May 13, 2015, 04:37 PM:

Laertes @256: Notation fail. Those numbers should be 2^(-1/2) and 1-2^(-1/2). Or to make the point more simply, 1/π and 1-1/π.

#259 ::: Joshua Kronengold ::: (view all by) ::: May 13, 2015, 04:38 PM:

It seems to be another retread of the "the rules are flawed if they don't let me rank" argument.

Functionally, the reason the rules don't allow ranking on nomination (either by repeat votes, scoring, or ordering) is because they are following the tried and true coding principle of minimal change. The only thing that -needs- to change to fix the block nomination issue is to restrict people to (effectively) one vote at a time. One can, of course, make arguments about and ascribe value to various ranking forms [as well as dropping the 5% rule, opening or keeping closed nomination visiblity, etc ad infinum]. But none of those are part of the base change, so keeping them out of the argument makes the changes smaller and therefore better.

#260 ::: Annie Y ::: (view all by) ::: May 13, 2015, 04:46 PM:

viktor@255

The proposed change does not change 3.7.1 which states very clearly: "up to five (5) equally weighted nominations in every category."
So you cannot give different weights. Combined with 3.8.4 which is also not changing: "If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category", it means that you cannot give more power to one entry.

So, no, you cannot split your point anyway you want.

#257 ::: Joshua Kronengold

Thanks. So what is getting voted on are just the changes (add these, remove these) and not the whole proposal as it is listed above. (Sorry for asking weird questions - I am just trying to get the hang of what's going on). So if 3.8.2 (for example) changes independently, because we are not changing it here, that change gets incorporated with our changes (if both pass).

#261 ::: junego ::: (view all by) ::: May 13, 2015, 04:51 PM:

Keith "Kilo" Watt @ 236
Lydy@234
Regarding 60 points vs fractions and wanting to divide however the voter wants..
That's a good point. That argument actually has me convinced that we should just stay with a single point and divide it up.

I saw both of Lydy's objections to my proposal and will bow to the expertise of someone who is "fraction phobic". ;-)

I had also wondered about people wanting to redistribute their points if there was more than 1 but thought that there was enough resistence to the idea of ordering nominations that it wouldn't be a big issue. OTOH, if others think it will just make the issue more complicated, I'm ok with dropping it. I'm all for KISS as much as possible.

#262 ::: viktor ::: (view all by) ::: May 13, 2015, 04:51 PM:

@257
Ranking is built-in to the proposal as is. However, by eliminating my duplicate changes my "ranking" in favor of some predetermined "ranking". Under the proposal, I can only allot 1.0, 0.5, 0.33, 0.25, or 0.20 to my nominees (assuming 5 nom slots). By allowing dupes, trips, quads, my options open to include 0.8, 0.75, 0.67, 0.6.

Yes, I think it's a flaw. Never before in the nominating round has a nom been allocated fractional points based upon the number of other noms on the ballot.

Other than that, I have no problem with the proposal.

@256
I think you're being silly.

#263 ::: Joshua Kronengold ::: (view all by) ::: May 13, 2015, 04:52 PM:

Annie@260:

Yes, exactly. Because they're not modifying exactly the same parts, the two don't cause any sort of conflict and can happily co-exist.

Practically speaking, the WSFS chairman will avoid (with extreme prejudice) the meeting passing contradictory amendments -- instead, instructing the proponents to create a merged proposal or a base proposal followed by a series of proposed amendments (if there are some disagreements) rather than pass a nightmare of contradiction.

There is also a committee of last resort in the form of the nitpicking and flyspecking commitee (which near as I can tell, is empowered to make small wording tweaks to the Constitution as part of integrating amendments as long as they don't change meaning, and would likely get involved in coming up with something coherent if WSFS managed to nearly contradict itself. But this is for obvious reasons to be avoided.

#264 ::: Aan ::: (view all by) ::: May 13, 2015, 05:08 PM:

Keith "Kilo" Watt @228:

[DECLINED NOMINATIONS]
Isn't a problem with leaving this free, and letting a new tradition form, that there's only going to be a difference in the results in 1% of the cases?
So if the first couple of times there's a declined nomation, it's decided to do a full rerun, then those reruns would likely lead to the same result as just picking #6. That means no one is going to be unhappy, and the tradition forms of always doing a full rerun. And then 10 years down the road, we get that 1% case cropping up, and suddenly a previously nominated author is off the ballot, and all hell breaks loose.

A nominated author no longer being nominated seems eminently "perceived to be unfair" to me, and something the proposal should guard against.

(Also: hi all! Don't think I'll be posting much, but I peeked in to see the current progress based on the file770 mention, and got suckered into reading the entire discussion, so here I am for now. I mentioned to a friend a while ago how I was confident that all y'all would come up with a great proposal, and I'm gratified to see how right I was.) :)

#265 ::: Seth ::: (view all by) ::: May 13, 2015, 05:14 PM:

#262 There is no ranking in this proposal. All of my nominations are treated equally.

#266 ::: Kevin Standlee ::: (view all by) ::: May 13, 2015, 05:19 PM:

Jameson Quinn @243:

Is there currently a proposal to lift the 5-works-per-category-per-ballot limit?
No. The list of proposals submitted to the Business Meeting for this year is on the Sasquan web site and we will update it as we receive new proposals.

Brian Gibbons @246:

I would be surprised if there were not a proposal to simply have the final ballot include every nominee that cleared the 5% threshold.
Have you ever attended a WSFS Business Meeting?
That obviously comes at the expense of making 'Hugo nominee' a much less meaningful descriptor
That devaluation is why we stopped calling the finalists "nominees" a couple of years ago. People who got even one nomination started insisting that they were "Hugo Award Nominees" by the "plain meaning" of the term.
Picking the One True Change at Sasquan means that next year's BM is going to be faced with a "take it or leave it" proposition. If you think that this a serious problem that needs to be solved (and I don't think we should be amending the Constitution if it isn't), then that's essentially giving them no choice but to ratify whatever this year's BM does.
That's funny; I haven't heard of any proposals that include the phrase "no further Constitutional Amendments shall ever be allowed."

If next year's Business Meeting doesn't like what's submitted to it for ratification, there's nothing stopping them from rejecting the ratification of what's passed on and passing a new change that would have to be ratified again in 2017 (and by the members of the 2018 Worldcon, should the pending Popular Ratification amendment up this year be ratified).

Joshua Kronengold @263:

Practically speaking, the WSFS chairman will avoid... the meeting passing contradictory amendments -- instead, instructing the proponents to create a merged proposal or a base proposal followed by a series of proposed amendments (if there are some disagreements) rather than pass a nightmare of contradiction.
Yes, that's what I'll try to do; however, if the proponents can't agree, I can't force them to agree. And it will not surprise me a bit if we have multiple factions all convinced that they have the One True Answer. I will strongly suggest that people unable to reach compromises will not be in the majority. That, unfortunately, makes me a "politician," because I think in terms of something that can actually get enough support to pass as opposed to something that's ideologically pure.
There is also a committee of last resort in the form of the nitpicking and flyspecking commitee (which near as I can tell, is empowered to make small wording tweaks to the Constitution as part of integrating amendments as long as they don't change meaning, and would likely get involved in coming up with something coherent if WSFS managed to nearly contradict itself. But this is for obvious reasons to be avoided.
The NPFSC dislikes getting involved in pending legislation if it can be avoided. Our remit is mostly to try and codify existing practice and put it in writing, by maintaining the Rulings & Resolutions of Continuing Effect and to sometimes recommend rule changes that bring our written rules in line with what we actually do in practice.

#267 ::: junego ::: (view all by) ::: May 13, 2015, 05:26 PM:

Under the proposed rules, I should be allowed to allocate my 1.0 anyway I want.

Yoicks! Irony.

Obviously the 1 point rule isn't proof against someone wanting to rank their nominations. I still bow to Lydy's knowledge about those who dislike fractions. :-p

#268 ::: gin-and-whiskey ::: (view all by) ::: May 13, 2015, 06:01 PM:

I think it’s a bit facile to pretend that this is somehow more objective. It merely gives weight to a particular type of voting. The SP group you’re dealing with is a very strong willed minority, and this particular method is designed to basically disempower them. Sure, their votes would count even if they didn't make the ballot…. so long as they voted for stuff they don't like, thus contributing to disenfranchising themselves and keeping their items off the ballot. That’s a Hobson’s choice.

Also, on principle, weasel words like "genuinely popular" shouldn't be used to hide value judgments. What's more "genuinely popular:" something that is the absolute hands-down only favorite of 25% of the population, or something which is "OK, I guess" for 80%?

As a comparison, imagine a rule where you got 10 points, and you can spend your points as you wish, across as many categories as you wish…. but without the initial popularity screen.

In that scheme, if you want to spend all 10 points on “best novel,” you can do so. But you lose your say for all other categories. OTOH, if you care a bit about every category instead of caring strongly about only one, then you can spend one point on each… but not have as much effect. You can’t do both. And since there are multiple categories and multiple nomination slots, you basically have to choose how much you care about things–you can’t game multiple categories.

This would practically limit the effect of slates, but would not wholly eliminate minority viewpoints. It’s a bit like politics: if a group reeeeeeally cares enough about a very small and limited # of nominations, why shouldn’t they be able to get a win on a small field?

Also, it deters worthless voting. IOW it forces people who are nominating to actually consider what is important to them. You can certainly “give 1/100 of a point to everything that isn’t a Sad Puppies recommendation” if you want to, but if you were planning to do that for 40 books which you hadn’t read just because they were published by a house you like, perhaps you’ll reconsider.

You could include popularity if you really wanted to. For example: starting at the bottom and working up, for any two submissions within some range of points (say, 25%) you could make it a popularity “how many people spent at least 1/2 of their ten points on this book?” elimination. (This allows for some gaming of the popularity aspect, but limits it to 20 submissions across all categories.)

#269 ::: Joshua Kronengold ::: (view all by) ::: May 13, 2015, 06:04 PM:

268: You don't seem to understand the voting systems under discussion, and are arguing your conclusion, rather than anything under discussion in the thread. Come again?

#270 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 06:07 PM:

gin@268:
this particular method is designed to basically disempower them

That's actually not true. If you read the topic and the descriptions, this system is specifically -not- designed to keep them off of the final ballot. What it is designed to do is keep any one group (whoever that might be) from keeping everyone -else- off the ballot.

Kilo

#271 ::: felice ::: (view all by) ::: May 13, 2015, 06:08 PM:

[DECLINED NOMINATIONS]

Keith "Kilo" Watt @228: "I've been reading all the posts on the subject, and I think we can all agree that there are two very distinct opinions on "fairness" with regard to this subject. I don't think we can say, therefore, that there is an objective definition of "fairness" in this case."

That's not my impression. As I understand it, we all agree that it's unfair to tell people they aren't finalists after all due to someone else declining; and I don't think it can reasonably be argued that rerunning without the withdrawn finalist(s) isn't technically the right thing to do. My impression is the RME proposal is being objected to by people who want simplicity or are strongly attached to the status quo, not on grounds of fairness (and most of the objections to changing the process predate the RME proposal, or ignore it - eg #264).

"- Therefore, let's keep it this way, and let's allow tradition to grow and decide what's "fair". As a first cut, the Hugo admins could decide to stick with the current tradition and move the work eliminated in the next to last round to the ballot. - If, after a year or so, fandom decides that they don't like this tradition, the Hugo admins can try something else (such as re-running the election) and see what fandom thinks about that."

I think it's safe to assume they'll stick with the current tradition until it produces a blatantly unfair result and upsets a lot of people, and there's the potential for people to react badly (eg "this whole SDV-LPE thing is silly, we should revert to the old system"). And as long as there hasn't yet been a blatantly unfair result, more subtle unfairness will contine. Why not get it right from the start?

Yarrow @233: "I also think that, on one view, moving the next work up is not only less work than rerunning the process, but also more fair. On that view, what a nominator gets when a work they like gets on the Hugo ballot is the opportunity for that work to be sent to the Hugo electorate. (Presumably this part is not controversial!)"

I would say the primary purpose for nominating works is to get the opportunity for those works to be included in the voting process. Being "sent to the electorate" is pretty meaningless if it's withdrawn shortly afterwards. It doesn't go in the voter packet, nobody is going to read it in order to decide how they should rank it, it won't appear in most historical listings. And if a work is withdrawn and not replaced in accordance with the preferences of the people who nominated it, they might be left with nothing on the final ballot they consider worth voting for.


--------------------------

RME = Rerun-Merge-Extend: rerun the SDV-LPE process with any withdrawn finalists removed; merge the new set of finalists with the old set of finalists; if necessary, extend the ballot to more than five finalists (just as we do with ties). Nobody gets removed from the final ballot because of reruns. In most cases, RME won't actually need to extend the ballot, because the new finalists will include all of the remaining old finalists (and one new work to replace the withdrawn work). Occasionally, it will extend the ballot to six finalists. Very rarely, maybe more than six (particularly if we get withdrawals and ties involving last place).

#272 ::: Ken Burnside ::: (view all by) ::: May 13, 2015, 06:12 PM:

Speaking as someone who is the beneficiary of both the Sad and Rabid Puppies slate, I think the SDV-LPE solution is a good one.

This merely dilutes the votes - a bloc of voters can still have the effect of the slates given here, but it will, in a typical voting year, raise that to about 40% of the voters rather than say 23-28% as appeared to happen this year. (Numbers based on looking at historical nominating ballots).

Since Sad Puppies 2, the number of nominating ballots has increased by more than 100%. People who expect this to "eliminate" Puppies will be gravely disappointed (as, to be fair, has been pointed out three times in the proposal and in the FAQ).

Unrelated:

I am in the process of scoping out a project that should also help the process along: An automatically generated and hand-pruned publisher-neutral list of "What's Hugo Eligible In A Given Year." If there are any people who are good at building datascrapers and running databases, please contact me via my website.

I want to avoid situations where things like Three Body Problem get squeezed out (until someone drops out) because people didn't know they were eligible.

#273 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 06:17 PM:

felice@271:
I don't think it can reasonably be argued that rerunning without the withdrawn finalist(s) isn't technically the right thing to do.

Actually, it can, and a number of people have done just that. They haven't changed your mind, and you haven't changed theirs. This is the situation I was referring to earlier.

Both ideas have good and bad things to say for them. I've been watching the discussion, and I don't think we're likely to reach a definitive conclusion on the issue. The point of my post is that there is nothing in the constitution that deals with the issue now, and an argument can be made that, since there is genuine disagreement on what -should- be done, there's no compelling reason for mandating what -must- be done.

The system works either way. In fact, it seems to me the best thing to do is what Kevin suggests: write up your declined nominations idea as a separate proposal. Then they can all be voted on piecemeal. Since SDV-LPE doesn't require it, there's no reason it can't be separate. And since the default method doesn't require a constitutional change there's no real reason to discuss it in the SDV-LPE proposal.

What are the group's thoughts?

Kilo

#274 ::: abi ::: (view all by) ::: May 13, 2015, 06:21 PM:

Ken Burnside @272:
People who expect a rule change to eliminate any voters or nominators, Puppy or not, should look elsewhere. Without, if it needs to be said, the blessing of this community. Because bah.

With regard to the eligibility-listing process, are you in touch with Analee Flower Horne (@leeflower on Twitter)? I think she's started a similar thing.

#275 ::: Annie Y ::: (view all by) ::: May 13, 2015, 06:32 PM:

#273 ::: Keith "Kilo" Watt

My opinion is up in 194 (after the note for Kansas that Kevin Standlee clarified (Thanks Kevin!))

As long as the method of replacements is known before the nominations close (so there are no cries that the method was selected in order to kick a specific work (or add it)*), I am fine with it. Or we can just leave it alone and let the administrators decide how they want to handle this (and it ties again into the 5% mess and how that gets resolved anyway)

This year it is clear that it is either next on the list or no replacement at all. If this change gets approved, we have three options - two of them producing possibly different nominees.

* Before this year, I would not have thought that someone can complain in this situation. This year? My internal cynic is out in full power.

#276 ::: felice ::: (view all by) ::: May 13, 2015, 06:35 PM:

Keith "Kilo" Watt @273: "Actually, it can, and a number of people have done just that."

Can you point me at specific post numbers? My impression is it's a conflict of fair vs easy, not different interpretations of fair - what have I missed?

#277 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 06:45 PM:

Felice@276:

I think yarrow@233 has one of the clearest explanations.

Kilo

#278 ::: P J Evans ::: (view all by) ::: May 13, 2015, 06:45 PM:

273
I think they're very much separate issues, and should be handled separately. (I think I've made my views clear already.)

#279 ::: influxus ::: (view all by) ::: May 13, 2015, 06:48 PM:

#273 ::: Keith "Kilo" Watt

I'm not sure I'm part of the group, more of a recent blow to this elegant thing y'all put together, but I think this sounds like a perfectly reasonable compromise.

#275 ::: Annie Y:

"As long as the method of replacements is known before the nominations close (so there are no cries that the method was selected in order to kick a specific work (or add it)*)"

Unfortunately, I don't think before nominations close would prevent this. As the administrators would have access to the nominations as they roll in, to defeat all perceptions of bias the process would need to be declared before the nominations open. That being said, if we are at that level of distrust, I don't see why people wouldn't just make accusations of electoral fraud.

#280 ::: Darth Paradox ::: (view all by) ::: May 13, 2015, 07:09 PM:

[DECLINED NOMINATIONS]

I've been following this conversation on and off with interest, but haven't had a chance to catch up to present until now, and I feel like I need to defend the slate voters on this one. Specifically, I think that rerunning the system from the beginning is necessary to avoid disenfranchising people that voted for the declining nominee.

I think a slate-voting bloc consisting of one fifth of the total nominating voters should be reflected in one fifth of the final nominations, which is to say one nomination out of five; if I understand the proposed system correctly, that's basically what happens today. (Which of the slate's choices makes it on to the ballot will probably be determined by other voters whose ballots don't match the slate exactly, or minor deviations from the slate within the bloc.)

And I think should be true even if the nominee that turns out to be their top choice withdraws. Suppose roughly one fifth of voters nominate a slate of Victor, William, Xenia, Yorick, and Zola, and thanks to those minor deviations William ends up on the nomination ballot (and Xenia was the last one of the five to be eliminated, but another non-slate nomineed was eliminated later). If William then declines the nomination, Xenia should replace him on the ballot; as I understand it, that will happen when the system is rerun to replace the declined nomination but not when the sixth-place winner is promoted.

I don't have the math to back this up at the moment, but my intuition tells me that promotion of sixth place runs a high risk of disenfranchising the slate bloc entirely.

(That said, I do think it should be rerun-and-merge to avoid "unnominating" anyone; I think an "unnomination" is a sufficiently bad outcome (on an emotional and political level if nothing else) to warrant specifically being avoided.)

#281 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 07:43 PM:

I think that the "rerun and merge" contingent should make a separate proposal. It wouldn't have to involve lots of lawyerly language; just "It is the sense of the membership that if a nominee declines, and the ballots have not yet been finalized, then the award administrators should find all works which would have been nominees if the decliner were removed from all ballots, and offer them places as finalists, without withdrawing finalist status from any works which already have it."

#282 ::: felice ::: (view all by) ::: May 13, 2015, 07:56 PM:

Keith "Kilo" Watt @277: "I think yarrow@233 has one of the clearest explanations."

That seems to boil down to "nominations for declined or ineligible works aren't wasted", which strikes me as a very dubious proposition. I don't recall anyone else bringing it up beforehand (though a couple have endorsed it subsequently), and even Yarrow describes it as "one view" rather than their own view (Devil's advocate?).

I would certainly consider my nomination to have been wasted if the work in question was withdrawn after becoming a finalist, and extremely peeved if any of my other nominations missed out on being finalists because of the wasted nomination. Surely I'm far from alone in that? Having it mentioned in a footnote in detailed nomination statistics isn't much of a consolation.

#283 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 08:01 PM:

Felice, I understand you are sincere. Please accept that others, such as Yarrow, are too. You seem to be in the minority, but you seem to think that anybody who disagrees with you can't really believe what they say they do.

#284 ::: Soon Lee ::: (view all by) ::: May 13, 2015, 08:07 PM:

I think I agree with Darth Paradox #280: "That said, I do think it should be rerun-and-merge to avoid "unnominating" anyone".

It seems to me that it's more fair to the nominees, than "unnominating" someone.

#285 ::: P J Evans ::: (view all by) ::: May 13, 2015, 08:10 PM:

280 et seq
If the work was withdrawn, or the nominee declined, don't take it as aimed at you personally.

It's ridiculous to rerun the the counting just for that: it's not actually unusual (this year, only in the number of withdrawn/declined nominations). Besides which, any reasonable program will have counted all the names, and have an order which should go far past any point where this would be a problem. That's why there are names ready to fill in for declined/withdrawn nominees.

(I refuse to call them votes; that's where you're getting confused, I suspect: this isn't an election, it's a simple tally. Or should be.)

#286 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 08:14 PM:

(On my phone, so apologies for edits, our the lack thereof.)

felice@282:

Well, that's certainly a reasonable question of debate, as we've seen. But one way or the other, there's still no reason it has to be presented as part of the SDV-LPE proposal. It's an easy thing to draft as its own proposal, present it immediately after SDV-LPE, and if people feel it needs to be in there, it will. I really don't see why that's not the best route in this case. I'd even be willing to help you draft it, once we have the current proposal in the can (I don't think you'll have any trouble finding the required two sponsors). Jameson's 281 is real close to all you need, though.

Kilo

#287 ::: Brian Z ::: (view all by) ::: May 13, 2015, 08:19 PM:

@214 J Thomas

@178 Brian Z
That depends on their gut reactions, which are hard to predict... If one person does that, it won't have much effect at all. If a bunch of people do it, it has a bad effect on the system as a whole, and probably doesn't help them since they'll tend to act against each other... It's hard to quantify, and probably people will go by their gut feel, and I can't predict that but I hope they won't go that route.

If you are worried about more concerted efforts to hack the award in future, better be worried about something that "if a bunch of people do it, it has a bad effect on the system as a whole."

If it is hard to quantify, then people will go only by their gut feeling - which could have surprising consequences if they are being given conflicting messages that they should vote exactly as before and that the effect of their vote has probably changed.

#288 ::: Brian Z ::: (view all by) ::: May 13, 2015, 08:55 PM:

@ 219 Jameson Quinn

[RESPONSES TO CRITICISM]

"So what are the legitimate points he has raised?"

"I'd be interested to hear if other people think this is an adequate response to BZ's concerns.?

Classy.

Let's start by reviewing my "illegimate concerns."

1. Many "Puppies" want the same thing and you (this group) could engage better with others to build consensus.

Look I understand why a body would feel defensive when an outsider comes in and asks pointed questions. But take me. I'm sympathetic to your cause, and I've been called a troll and a liar repeatedly, had my words on another forum stripped from the topic sentence of the paragraph to paint my words in the worst possible light and cited out of context without a link, and it has been implied that I shouldn't engage further and that my questions are only useful in case any actual concerned person wants to know the answers in future. And you know what? Don't get mad: I'm pleasantly surprised by this reception. I had expected to be disemvoweled. Still, don't you think this sounds to outside ears like "we have enough attending at WorldCon that we don't particularly care what you think"? Not be the best way to repair a torn community.

2. Maybe it ain't broke. Possibly what happened this year won't happen again with the higher participation we are seeing.

3. Micro-managing the rules is going to be less effective than welcoming new blood and demonstrating through your words and actions that existing WorldCon culture is valuable and should be preserved.

4. The goal should be rewarding and honoring excellence, not fairness.

Now to your responses:

5. Bullet voting. I'm not convinced @123 has addressed the concern fully. Currently, if I think LeGuin and Spinrad are probably going to win already there is not much penalty to my other nominees to adding my voice to the chorus, but in the proposed system the reverse could be true. Might work out better for LeGuin than Spinrad. Note my focus is on honoring worthy nominees as a community not getting the most satisfactory outcome as an individual.

6. Goals and voter happiness. You may not have explicitly said in your mission statement that you are designing this to maximize happiness (which we agree is not the productive way to go) but you effectively have done so by emphasizing "fairness." Fairness doesn't choose excellent science fiction. The ONLY reason fairness might be important is that people feel happy when they get a piece of the pie (or in today's culture, apparently, when they perceive that everybody who wants one gets a little piece). I question whether that is a valid goal. It certainly doesn't achieve MY Hugo voter happiness, which comes when an author receives a heartfelt gift from their fans that they know very well is valuable because it consistently picks greatness.

6. There may be cases where mediocrity triumphs. Of course this is true of the current system as well (many fans were unhappy with the 2014 ballot, whether you think it was perfect or not). But 6a) the current system has a consistent, time honored message: if everybody thinks about what is the best and nominates it, a consensus shortlist will emerge. Your new message is "everybody's opinion is equally valid and we will try to be fair to everybody." I think that's the wrong message and in your own discussion of my 1970 thought experiment you have recognized that the danger of mediocrity is a real concern. 6b) whether this system is as effective at choosing excellence is a real concern and should considered to the best of our limited ability, considering the math but also changing voter strategies. My own feeling is: we know the old one did, that's a real treasure, so let's try to repair it without making drastic changes too quickly.

#289 ::: Seth ::: (view all by) ::: May 13, 2015, 08:56 PM:

#268 This doesn't disempower anybody, it gives any group strength proportional to its size.

20% of the nominators can get one work (that nobody else likes) onto the ballot. Or, depending on how the numbers work out, they might get the works that would (according to the other 80%) have been #6 and #8 onto the ballot. Either way, that seems about like the right amount of power for 20% of the nominators to have.

#290 ::: Walter Daniels ::: (view all by) ::: May 13, 2015, 08:58 PM:

As an ex-computer specialist (multiple injuries and permanent disabilities/pain caused giving that up), you have done a first class job. There is only one problem. The "Traditional (Big) 5 Publishers" all live/work in NYC. That means they all attend pretty much the same parties, know each other, etc. As a result, in an area with 8.5 Million people, you have 1,000 (at maximum), who can "informally agree" that "Author X, of Publisher Y, should win." At worst it's 5 agreed on. Giving the "group/slate") 200 to 500 assured votes.

Now, this is fine, and acceptable, as long as it's understood that employees publishers "block vote." The problem is that in the "universe" of 312 million other potential voters, a "slate" is nearly impossible. Fen are like herding cats, for "bloc anything." :-)

Those screaming about "bloc voting" fail to understand that the "slate" is, and always has been, "here are authors/artists/works to be considered by you (generic) for consideration for Hugo Nominations." If someone asks me. "What are some good Mil/SF authors to read?" I'll mention David Weber, Mike Z. Williamson (Note: I've known MadMike for almost 23 years, but I also like his writing), an ex(?)-Marine Officer female author whose name escapes me, and John Ringo. If they ask for possible Hugo Nominations (assuming all have books out in the relevant period), I would still give several names. Unless I know their individual tastes. If I do, I'll say. "I plan to vote for X, but these others are almost as good, IMO." That was, and is, the intent behind Sad Puppies, as I understood it. Had I become a "supporting member" early enough, I would have voted for the one I thought most deserving. I can't speak for what anyone else thought/did.

As I said earlier that puts me, and those like me, at a disadvantage. By population, I make up 0.00266 of an NYC "Cabal member's vote." That's a heavy disadvantage. That means 36.67 of me, to offset one of them. (Note: I used population to be as generic as possible. I know of no "count" for actual F&SF fans in the general pop.) So, if there is any "Bloc voting," going on, it's the publisher employees doing it. I doubt that they actually decide, "we'll all vote for X." It's more likely they all "like and talk up" X, better than any others. (Peer pressure at it's finest.) Nevertheless, it's the best way possible that you've come up with. Publishing (traditional)employees will always have an advantage. It's called, "Small area, where most live/work/play together." That some don't think that's enough, is a sign of their insecurity.

#291 ::: junego ::: (view all by) ::: May 13, 2015, 09:06 PM:

Keith "Kilo" Watt @ 273

The system works either way. In fact, it seems to me the best thing to do is what Kevin suggests: write up your declined nominations idea as a separate proposal. Then they can all be voted on piecemeal. Since SDV-LPE doesn't require it, there's no reason it can't be separate. And since the default method doesn't require a constitutional change there's no real reason to discuss it in the SDV-LPE proposal.
What are the group's thoughts?

I wouldn't object to leaving it up to the administrators. The less the proposal tries to modify current rules and practices, the better chance it has of passing.

I have a couple of concerns:

1) We might should warn administrators that this could be an issue?

2) When there is finally a need to make such a decision, those future administrators might benefit from this discussion of possible options. Would it be appropriate to include a list of what was discussed here for their consideration? I think there are 3 proposals a) No recalculations and just take the next in line from the final run. b) Freeze the finalists list and rerun the program for #6 down on the final run to replace only those removed/withdrawn. c) Keep the remaining finalists, rerun the entire list and add any qualifying nominees to the finalists if there are changes to the top 5 beyond the replacements.

#292 ::: Jameson Quinn ::: (view all by) ::: May 13, 2015, 09:09 PM:

@287: So, you're concerned that somebody might give voters conflicting messages? I wonder why you'd think that...

More seriously: "the effect of their vote has probably" not "changed". The Condorcet Jury Theorem still says that if enough voters define quality in similar enough ways, this system will get the right answer, which in that case is the same answer as the current system. The only difference is that any slate voters' ability to manipulate the system, and hurt other voters and the outcome in the process, will be reduced. Not entirely removed, but reduced. They'll become like concern trolls — actually kinda helpful — rather than like hate trolls — toxic.

#293 ::: P J Evans ::: (view all by) ::: May 13, 2015, 09:17 PM:

290
Except that it doesn't actually work that way in practice. That's the Big Lie involved with the juvenile canines and their slate.

How do I know this? I've been on a Hugo administrative subcommittee - that's how (and why) we have access to a set of actual data. And I can assure you that no one outside the subcommittee gets input to the counting, and they sure as hell don't get any input to the final ballot counting. (Nor do they get advance knowledge of who gets rockets. It's need-to-know, all the way.)

#294 ::: Yarrow ::: (view all by) ::: May 13, 2015, 09:20 PM:

felice@ 282: That seems to boil down to "nominations for declined or ineligible works aren't wasted", which strikes me as a very dubious proposition.

The case of declined nominations is different from that of ineligible works. Ineligible works are eliminated from (or moved to the right category on) all nominating ballots before counting begins, so there's no need to rerun the process.

Nominations that boost a work to the preliminary short list are not, I believe, wasted, even if the author declines the nomination - they get the work to the "make a phone call to the author" stage.

even Yarrow describes it as "one view" rather than their own view (Devil's advocate?).

It is my view: I believe it's the fairest choice among those available (some of which are also reasonable).

#295 ::: felice ::: (view all by) ::: May 13, 2015, 09:21 PM:

Jameson Quinn @24: "I am not the only person who has coded this up, but my code (and data) is visible at https://github.com/The-Center-for-Election-Science/HugoVotesim. I did it in R, the statistical computing environment. I realize that R is not at all the most accessible language for non-statisticians, and I'm sorry about that, but it would have taken more work to do bootstrapping in python."

I've got it somewhat running, but it throws the error:

Error in if (hugos[i, cat] < 7) { : missing value where TRUE/FALSE needed
In addition: Warning message:
In min(titdist) : no non-missing arguments to min; returning Inf

Any idea what the problem is?

#296 ::: felice ::: (view all by) ::: May 13, 2015, 09:39 PM:

Yarrow @294: "The case of declined nominations is different from that of ineligible works. Ineligible works are eliminated from (or moved to the right category on) all nominating ballots before counting begins"

Not if they're found to be ineligible after the finalists are announced, as happened in Novelette this year.

#297 ::: Chris ::: (view all by) ::: May 13, 2015, 09:49 PM:

Walter Daniels @290:

Those screaming about "bloc voting" fail to understand that the "slate" is, and always has been, "here are authors/artists/works to be considered by you (generic) for consideration for Hugo Nominations."

I quote:
"I encourage those who value my opinion on matters related to science fiction and fantasy to nominate them precisely as they are." (Vox Day, introducing the Rabid Puppies slate) (emphasis added)

That's not "authors/artists/works to be considered by you". That's "vote for exactly this list", a tactic that lets a coordinated 10% drown out the emergent consensus of the other 90%.

#298 ::: P J Evans ::: (view all by) ::: May 13, 2015, 09:53 PM:

296
It happens. Usually it's discovered before the ballot is sent to the printers.
Move up the next name on the list, and cope.

#299 ::: Yarrow ::: (view all by) ::: May 13, 2015, 09:59 PM:

felice@296: Not if they're found to be ineligible after the finalists are announced, as happened in Novelette this year.

Point. Now that I think of it, it would be massively more work to investigate the eligibility of every work that appears on any ballot, rather than what comes up on the preliminary short list.

#300 ::: Seth Gordon ::: (view all by) ::: May 13, 2015, 10:12 PM:

[DECLINED NOMINATIONS] Just adding my “vote” in favor of the rerun-merge-extend option. When faced with tricky boundary cases, erring on the side of inclusion sounds like the right way to travel.

#301 ::: Vicki ::: (view all by) ::: May 13, 2015, 10:18 PM:

Walter Daniels @290:

Would you care to rephrase that in the subjunctive, as appropriate for a hypothetical statement contrary to fact?

A few people here might even agree with you that collusion would be "fine and acceptable" as long as it was known about. (I say "a few" because if most of us did, this conversation wouldn't be happening, because the Puppies' activities are no secret.) But I hope that nobody here would consider it "fine and acceptable" for people to falsely believe that a few hundred people in the New York area are bloc-nominating in the Hugos.

Do you have any evidence for your charges? "Well, that's what I would do in their shoes" isn't evidence of anything except your psychology, which is not at issue here.

#302 ::: Brian Gibbons ::: (view all by) ::: May 13, 2015, 10:18 PM:

Keith "Kilo" Watt @273: Since SDV-LPE doesn't require it, there's no reason it can't be separate. And since the default method doesn't require a constitutional change there's no real reason to discuss it in the SDV-LPE proposal.

I'm not sure that I see why you think that the default method isn't in the constitution right now, or that the SDV-LPE proposal doesn't already remove it as an option.

3.8.1 tells us what works appear on the final ballot: the five eligible nominees receiving the most nominations.

When a nominee declines/is declared ineligible, my understanding is that the Constitution is agnostic as to what the Hugo administrator is required to do, but we know one thing: the only thing the final ballot can include is "the five eligible nominees receiving the most nominations" (ties aside).

The Hugo administrator can't replace a declined nominee with the nominee's friend or an ineligible work with another one by the same author. He can leave the ballot alone if he wants, but if he chooses to produce a new version of the final ballot, he must do so strictly in accordance with the rules for what can appear on a final ballot.

This is a pretty simple process (because the new member of the set of "five eligible nominees receiving the most nominations" will always be the one that was in sixth-place before), but this isn't just a default custom, it's simply producing a final ballot in accordance with 3.8.1.

3.8.8 in this proposal gives us an entirely new version of how to determine what works appear on the final ballot.

If a nomination is declined or a work declared ineligible, the Hugo administrator can similarly leave the ballot alone, aside from striking out those nominations, but if he does want to replace the ballot with a new version with additional nominee(s), he must determine what works appear on this new version of the final ballot by using 3.8.8.

Neither adding the work that was in the #6 spot in the last run, nor expanding the list to include extra nominees (as in a RME scenario) is an option, because they're simply not options the rules give. The administrator can't just decide to put those works on the ballot, any more than he can just decide to put his favorite there.

The administrator's only option (assuming he wants to replace the old version of the final ballot with a new one, and not just leave it alone) is to follow 3.8.8, wherever that might lead, which might include retroactively removing nominees.

... or, at least, that's how I understand the rules.

As I see it, SDV-LPE causes this problem and SDV-LPE should fix it. A separate proposal is kind of silly, since its necessity would stand or fall with this proposal passing.

#303 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 10:36 PM:

All -

First, hello and welcome to the new voices who have come by, particularly those who identify themselves with the Puppies! As Brian and I discused, I really don't like to think of "sides" in fandom. Ultimately, the Hugo process needs to work for everyone (to the greatest extent possible, of course) if it is truly to be the "fans' award". I'm glad that we can all rise above the politics of the situation and focus on getting the job done.

The proposal is nearing its final form, so I'd welcome any questions about how the system works, and if there are holes that we aren't seeing, it's great to have fresh eyes looking at it. We do have to keep in mind that there really isn't a perfect system -- that realization was one of the results of this discussion -- but in general, I do think this proposal is going to be a good way to go.

To that end, it's worth reiterating that the prime goal is not to keep anyone -- whoever they may be -- off of the final Hugo ballot. The goal is to prevent anyone - whoever they may be -- from preventing -others- from having a chance at appearing on the ballot. This is a win for everyone. For example, Walter@290 mentions a bloc of NYC publishers. Such a thing may or may not exist, but it realy doesn't matter: the idea here is for -no- bloc to be able to prevent works from being considered. So, I do think this proposal serves everyone's aims.

I think you'll find everyone here to be reasonable people and open to productive discussions. My only request is that you please do consider how far along we are in the process and understand that we really can't start from scratch at this point. My hope is that we can answer questions and address concerns as we put together the final proposal.

Thanks much,
Kilo

#304 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 10:47 PM:

Brian@201:

The previous thread to discuss proposed rules changes carried an advisory at the top not to argue about whether changing the rules is a good idea or not, but this one doesn't (yet).

Just to be clear, this is a continuation of the previous thread, so yes, the idea is to work on this proposal and isn't really to debate whether we should do it at all. That's certainly a worthwhile debate to have, but it's probably better put in its own topic. I don't feel we will have wasted our efforts trying to put this together if it turns out fandom decides they don't want to change anything. Unless we have something to show them, they certainly have no reason to do otherwise.

Kilo

#305 ::: JJ ::: (view all by) ::: May 13, 2015, 10:51 PM:

Keith "Kilo" Watt, #304:

Brian is quite fixated on trying to dissuade Worldcon members from making any changes to the current nomination process; he's made numerous posts in various threads not just here, but over at File770, trying to discourage everyone from making any changes.

Draw from that what conclusions you will.

#306 ::: P J Evans ::: (view all by) ::: May 13, 2015, 11:02 PM:

299
Well, some things are obviously ineligible. (The 1984 examples: Dune for Best Novel and Asimov for the Campbell Award. [/snicker])

The administrators check the top 10, I suspect.

#307 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 11:04 PM:

BrianG@302:

First of all, apologies for any confusion -- I didn't realize we had two Brians!

I see what you're saying, but the constitution deosn't actually mandate five nominees on the final ballot. It does mandate that there will be at least three (3.8.5). As you point out, he can leave the ballot alone -- not add anyone if someone declines, but the constitution does not actually give him the authority to do -anything- if someone declines. It doesn't even say that he can take the 6th place person. But he's not limited to the powers explicitly granted to him. Hopefully Kevin or someone will correct me, but my understanding is that anything not explicitly set out in the constitution is left to the administrator's judgment. I may be wrong about that, of course, but that is my understanding.

You say he can't replace someone with his friend. Well, most of us would agree that'd be an ... extremely ... poor decision for an administrator to make. But the constitution itself doesn't prevent him from doing so. I think that would universally be declared to be an unfair result, and who knows, an amendment might get passed to prevent it from ever happening again. It's never been a problem before (and isn't likely to be in the future, of course).

But as we've seen here, it's not quite so cut and dried in this case. An adminstrator could make an argument for either one. You say that SDV-LPE should fix it, and you may have a point, but I'm not sure that there is a clear and unambiguous fix. As I said, personally, I'd like to give the Hugo admins the freedom to figure out what the best course of action is. Changing the constitution may take up to three years in the very near future. For myself, I think that's too long a wait.

But if you really feel SDV-LPE needs to take a stand one way or the other, I still think the best way is with a separate proposal. In fact there's no reason not to propose -both- of the leading methods as separate proposals and let the business meeting decide which is best. Kevin has indicated he'd actually prefer that sort of thing in general. I'd be glad to help draft both of those proposals. If either one really is self-evidently more fair, then that will be made immediately apparent at the business meeting -- why bother theorizing now?

Just my thoughts,
Kilo

#308 ::: P J Evans ::: (view all by) ::: May 13, 2015, 11:05 PM:

305
I came to a conclusion earlier about his viewpoint, based on his comments at File 770.
I do wish those supporting the juvenile canines would say so, instead of leaving the rest of us to come to conclusions that may be incorrect, but are definitely unflattering to them.

#309 ::: viktor ::: (view all by) ::: May 13, 2015, 11:09 PM:

[DECLINED NOMINATION]
If a re-tallying is done after a declined nomination, are the ballots that included that nomination re-weighted? By that I mean, if I nominate Able and Baker -- thus assigning them each 0.5 pts -- and then Able declines, does Baker then get the full 1.0?

If so, then I would be against a re-tallying after a declined nomination. I weighted my ballot they way I wanted it. In this case, I think it's best to take the #6 finisher.

#310 ::: Cat ::: (view all by) ::: May 13, 2015, 11:11 PM:

Brian Z @ 288

O Hai! I remember you from File 770. You're the one who said we all jumped on someone for proposing something that Puppies quoted approvingly.

I'm so glad we got that straightened out, aren't you? It's great that several other people jumped in and pointed out that Making Light was trying to be fair to everyone, instead of trying to force Making Light commenters to disenfranchise the Puppies.

1. Many "Puppies" want the same thing and you (this group) could engage better with others to build consensus.

[snip]. But take me. I'm sympathetic to your cause,

I'm sorry--I haven't quite noticed this. How are the Puppies prepared to build consensus with us?

And what parts of what we are trying to do, exactly, are you sympathetic to?

Still, don't you think this sounds to outside ears like "we have enough attending at WorldCon that we don't particularly care what you think"? Not be the best way to repair a torn community.

The Puppies tore the community, right? Just pointing that out, to make sure nobody's getting the impression that we are responsible for what they did. I know the Puppies claim that, but it doesn't seem like a mistake you would make, being sympathetic to our cause. Just want to make sure nobody else makes it either.

Given that they tore the community, please explain to us how the Puppies are proposing to build consensus with us and help us repair it?

2. Maybe it ain't broke.

What an interesting idea. 17% of the nominators using a slate locked 83% of the nominators completely out of about 3/4 of the ballot including whole categories. That looks broken to me. Will the business meeting at Worldcon agree with you? Or with me? We'll see.

I wrote a couple of pieces about how a slate increases the nominating power of the group that uses it. The most relevant part is here. I think you are being far too optimistic.

I think we'd need ten to twelve times as many honest nominators as slate nominators to be able to get one or two of the non-slate group's favorites on the ballot. There were about 350 slate nominators, so we'd need a total of at least 3,850 nominators next year, and that's provided there isn't anybody left on Breitbart or in Gamergate to be drawn in. I do expect an increase in nominators next year. I don't expect it to be *that* large.

3. Micro-managing the rules is going to be less effective than welcoming new blood and demonstrating through your words and actions that existing WorldCon culture is valuable and should be preserved.

I for one welcome our new Puppy overlords--oh, wait, actually I don't. If they wanted to join as honest nominators, reading what appealed to them and individually choosing their particular favorites without resorting to a slate to guide their reading or nominating, of course I would welcome them, but then they wouldn't be Puppies anymore, would they.

4. The goal should be rewarding and honoring excellence, not fairness.

Currently the system is rewarding and honoring works the three or four curators of the slate (the Puppies have been remarkably opaque about who they actually were) choose to cram down our throats to make a political point. To the point where such works have forced more excellent works off the ballot in the view of the slatemakers themselves.

Rewarding and honoring excellence won't be possible until non-slate nominators can also get some work on the ballot, by individually reading what appeals to us and nominating our own particular favorites.

You may, of course, think that "excellence" consists of successfully catching the eye of one of these three or four slate curators. You are welcome to attempt to persuade the business meeting attendees of that.

You think that we mustn't worry about slate voters shutting us out of the nomination process because "excellence is better than fairness." This would be a consideration if there were any reason to think that the slate curators are successfully choosing excellence. They themselves admit they fell short this year, but perhaps you disagree with them; that is certainly your right.

But until you can make a good case that slates are better at choosing excellence than a fair process would be (a case you might do better to make to the business meeting than to us) I reject the dichotomy. We're not choosing between fairness and excellence but between fairness and a load of Puppy...chow...which so far has been almost uniformly abysmal leavened with the occasional lump of "meh."

Of course I still have four of the John C. Wright works to read. Perhaps you think I'll find excellence there? (shudder)

6. There may be cases where mediocrity triumphs.

This year, for example. Though there is always "No Award" thank goodness.

But 6a) the current system has a consistent, time honored message: if everybody thinks about what is the best and nominates it, a consensus shortlist will emerge.

That's still the message. We're just working to make it true again.

6b) whether this system is as effective at choosing excellence is a real concern [snip] My own feeling is: we know the old one did, that's a real treasure, so let's try to repair it without making drastic changes too quickly.

The old one did because nobody wanted to be the jerk who introduced slate nominating. It was a real treasure--until a big enough jerk came along. But slates certainly aren't going away now, and while a nominating pool of 10,000 Hugo voters that would overwhelm any resentment-driven political attempt to slate is a lovely dream, I don't see it happening anytime soon.

Besides, it takes two years to make changes. We can pass this this year (if the members at the business meeting approve). If the Puppies really are going to try to repair the community they tore and don't slate anymore and even stop being jerks blaming everybody but themselves for their bad behavior (I can dream), next year's business meeting can go, "oh, whew--it's so much easier to just count the usual way" and decline to ratify.

#311 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 11:12 PM:

junego@291:

We might should warn administrators that this could be an issue?

This might actually be a good argument for separate dual proposals for each method of handling declined nominations. Even if -both- fail, the next admins will be aware of the potential issue.

Still, like Jameson said, I don't really think we need to make more of this than what it is. The odds of this ever being an issue are so very, very small that we shouldn't get too much up in arms about it. The chance is not zero, however, so I think it's a good discussion, but we should make sure we keep things in perspective.

Kilo

#312 ::: JJ ::: (view all by) ::: May 13, 2015, 11:14 PM:

P J Evans, #308: "I came to a conclusion earlier about his viewpoint, based on his comments at File 770."

Likewise. At this point, I'm rather amused by the repeated protestations of "I can't believe what a nasty person you are, to accuse me of being a Puppy just because I keep trying to ram a bunch of Puppy Talking Points down everyone's throats while continually proclaiming that I'm really one of you!"

#313 ::: Keith "Kilo" Watt ::: (view all by) ::: May 13, 2015, 11:17 PM:

viktor@309:

By that I mean, if I nominate Able and Baker -- thus assigning them each 0.5 pts -- and then Able declines, does Baker then get the full 1.0?

If I understand felice's proposal correctly, yes, that's exactly what the case would be. Please correct me if I've got a misconception, felice!

K

#314 ::: Cat ::: (view all by) ::: May 13, 2015, 11:46 PM:

FWIW, In the case of declined or ineligible nominations I favor, in order

1) Generate a list of ten, take the top five, if one declines, take the sixth and so on. It has the virtue of simplicity and being unlikely to put too many extra nominees on the final ballot. (You might get the occasional tie for sixth.)

2) Generate a list of five, if one declines, just go with four. This is also simple, and does not risk ejecting anyone already on the ballot (which I think would lead to extremely bad blood.)

3) Generate a list of five, if one declines, rerun, merge, extend. This is arguably "most perfectly fair" but clunky and threatens to extend the ballot. Longer ballots (especially in the Best Novel and Best Related Work categories) make it more difficult to read everything.

In the case of nominations "wasted" on ineligible work, you wouldn't expect that to happen very often. One hopes that most of the people planning to nominate the work would check what year it was published. Being nominated by only a small percentage of those who loved it would see such a work dropped early and any ballots containing it re-weighted in its absence and thus not penalized by it. Large numbers of people would only nominate an ineligible work in the case where accurate information about its year of publication was withheld; this would be the only time such a work would reach the ballot. In that case the person to blame for "wasted nominations" wouldn't be the Hugo administrators, surely. So I don't see that the Hugo administrators would have to re-run the nomination process to make good on it.

#315 ::: felice ::: (view all by) ::: May 14, 2015, 12:00 AM:

[DECLINED NOMINATION]

viktor @309: "If a re-tallying is done after a declined nomination, are the ballots that included that nomination re-weighted? By that I mean, if I nominate Able and Baker -- thus assigning them each 0.5 pts -- and then Able declines, does Baker then get the full 1.0? If so, then I would be against a re-tallying after a declined nomination. I weighted my ballot they way I wanted it."

That's not how SDV-LPE works. If you nominate Able and Baker, they start off with 0.5 points each from you, but if either one gets eliminated during the initial tallying, the other gets the full 1.0. There's no way to indicate you want them both to stay at 0.5 points (and no reason to want it); all you can do is nominate the works you consider worthy, and the weighting and reweighting is an automatic process. If Able was found to be ineligible before the initial tallying, or got eliminated before Baker, then Baker would get the full 1.0 during the initial run. If Able declines or is found ineligible after the initial announcement of finalists, why should it be any different? There's no way to rerun the tallying without changing the weightings (other than not removing the withdrawn work, which would make the exercise pointless).

Though I don't see any problem with your suggestion in #253 to allow nominating the same work more than once; the only issue is explaining to people why it isn't a problem, and I don't think it's useful enough to be worth that hassle.

#316 ::: P J Evans ::: (view all by) ::: May 14, 2015, 12:05 AM:

3114
As I understand it, they'll go with 4 if there are too many tied in the next spot - meaning if it would put more than 7 on the ballot, counting No Award. (There used to be physical limits on ballots, as in 'only so much space available'.)

And, also, people have a harder time when there are more than about 5 or 6 to rank (as well as this multiplying the amount of time the counting takes - the final ballot program (AIUI) has to go through the process once for every nominee on the ballot, plus the check process for No Award.)

#317 ::: Annie Y ::: (view all by) ::: May 14, 2015, 12:07 AM:

felice @ 315
I don't see any problem with your suggestion in #253 to allow nominating the same work more than once

3.8.4 is the problem. So if you want multiples of the same work nominations to be allowed, you need to propose a change in it (that needs to be approved and ratified as well). So it is not just explaining - it also requires a change in the rules.

#318 ::: felice ::: (view all by) ::: May 14, 2015, 12:17 AM:

Annie Y @317: "3.8.4 is the problem. So if you want multiples of the same work nominations to be allowed, you need to propose a change in it (that needs to be approved and ratified as well). So it is not just explaining - it also requires a change in the rules."

True; I was thinking in terms of fairness rather than rules. We'd still need 3.8.4, since listing the same work multiple times would still only count as one nomination, but 3.8.8 could specify that 3.8.4 doesn't apply to division of points. I don't particularly care either way, though.

#319 ::: P J Evans ::: (view all by) ::: May 14, 2015, 12:35 AM:

felice, can you explain why counting the same work multiple times is more fair than the current rule of only counting it once?

How is this not like a slate, because it gives a some people more voting power than others?

(I have a lot of trouble with your suggestions, as they seem geared to making the process less fair than it currently is.)

#320 ::: Julie ::: (view all by) ::: May 14, 2015, 12:49 AM:

(slightly nervous lurker here who has been following the conversation):

[DECLINED NOMINATIONS]

As I understand it, this procedure does not produce a ranked list. All one can say about the final ordering is: #1-4 are ranked in order of points (they may never have been compared to anything on nominations); #5 has more nominations than any of #6, #7, #8... .

#6 and #7 may never have been compared (I'm think of a case where the eventual #5 starts low down on points, but works its way up the list via strong nomination counts).

So one could find a final ordering with roughly
#1 200 points, 800 nominations
#2 199 points, 199 nominations
#3 198 points, 600 nominations
#4 197 points, 200 nominations
#5 50 points, 401 nominations
-------------------------------
#6: 100 points (before elimination), 100 nominations
#7: 98 points (before elimination), 400 nominations

.... does that give another reason why taking the next one on the list might not be the best choice?

#321 ::: Darth Paradox ::: (view all by) ::: May 14, 2015, 12:54 AM:

[DECLINED NOMINATION]

It occurred to me after posting #280 that it was easy to construct an example to demonstrate my problem with the pick-the-last-eliminated method of replacing a declining nominee. Consider a situation where the electorate is comprised solely of five slate-voting blocs of roughly equal size, with just enough deviation in each bloc to avoid ties (and cause an ordering of each bloc's nominees). Specifically, suppose:

20.2% of the electorate nominates a slate of Abner, Adelaide, Ajay, Alice, and Arianna (and the deviations break ties in that order).
20.1% of the electorate nominates Bart, Bess, Blaine, Brianna, and Burns.
20.0% of the electorate nominates Cameron, Celeste, Chris, Colette, and Cyril.
19.9% of the electorate nominates Dawn, Deanna, Dina, Doris, and Dule.
19.8% of the electorate nominates Eaton, Edsel, Emily, Erica, and Eula.

Given that, the final nominated ballot would be Abner, Bart, Cameron, Dawn, and Eaton; the last five eliminated would be Adelaide most recently, then Bess, Celeste, Deanna, and Edsel.

Now suppose that Dawn declines the nomination. Replacing her with the sixth-place nominee, Adelaide, means the "A" bloc now has two nominees on the ballot and the "D" bloc has none, despite the fact that the two blocs are nearly the same size. But rerunning the process from the beginning (or even from the fifteen-nominees-left point) would replace Dawn with Deanna, the collective second-place choice of the bloc that got Dawn onto the ballot in the first place.

I acknowledge this example doesn't demonstrate the case when the rerun causes other changes to the top five besides simply replacing the withdrawn nominee, but I think the merge/extend discussion has the right of it.

Ultimately, there are two principles I'd like the replacement process to follow:
1) Anyone who would have been on the ballot if the withdrawal/decline was known prior to the votes being counted should still be on the ballot.
2) Anyone who was announced as being on the ballot (and has not intentionally withdrawn or been found ineligible) at any point should not be removed from the ballot.

If this ends up causing six nominees instead of five after a withdrawal, then so be it. As others have said, I'd rather be inclusive in this kind of situation. If it's likely to cause seven or more nominees instead of five after a single withdrawal, then maybe that's an indication that the results produced by the system are too fragile; it seems to me that slight changes to the field only causing slight changes to the results is a desirable property of a voting system.

Keith @ 307:

Is there a particular process for introducing proposals that are dependent on previous proposals, or proposals that are mutually exclusive? I suppose the former can just be handled by waiting to introduce the modifying proposals until the previous one has passed, but taking up competing proposals one at a time seems undesirable (compared to introducing both of them, debating, and then holding a vote between approving A, approving B, or rejecting both).

#322 ::: Ken Burnside ::: (view all by) ::: May 14, 2015, 01:01 AM:

I'm in favor of "Make a list of 10 nominees, use the top 5, if one declines, bring in number 6" if RME cannot be made to work acceptably.

#323 ::: Julie ::: (view all by) ::: May 14, 2015, 01:01 AM:

Actually the numbers I used don't make sense--- you can't have 50 points and 401 nominations, especially not at the end of the process.

Make it
#5 100 points 401 nominations
----------------------------
#6 120 points 120 nominations
#7 119 points 400 nominations


#324 ::: Nancy Lebovitz ::: (view all by) ::: May 14, 2015, 01:02 AM:

#319 ::: P J Evans

Each person gets the a total weight of one vote per category. In effect, if they only vote for one nominee, it's as though they voted a fifth of a vote five times.

I don't see the problem if, instead of voting their five fifths for one work or giving one fifth vote to each of five works, they give two of their fifths to one work, and three of their fifths to another.

#325 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 01:15 AM:

Darth@321:

Right, the two declined nomination proposals would only be introduced afte SDV-LPE passes (assuming it does, of course). My only reason for suggesting the two proposals would be to let the business meeting participants vote, since it wasn't looking like this group could come to a consensus.


But that said, a few people who were against RME now seem to be okay with it. I honestly don't have a strong opinion either way, as always, assuming we can explain what we come up with in a simple, straightforward manner.

Is it time to take another poll?

Everyone (whether you actively post or not, don't feel you can't express your instinct), if you wouldn't mind ranking from 0-10 (per Jameson's system) the following:

In the case of a declined nomination:

1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
3. Do not replace the declined nomination
4. Re-run the election with the declined nomination removed; accept the new results as-is
5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

I think these are the possible permutations that have been discussed. If I've missed any, please let me know.

Thanks,
Kilo

#326 ::: viktor ::: (view all by) ::: May 14, 2015, 01:22 AM:

#324
Long story short, what she said.
And this is why this proposal will ultimatley fail.

#327 ::: viktor ::: (view all by) ::: May 14, 2015, 01:29 AM:

How hard is it to say, "If you nominate something more than once on your ballot, it gets more points! So list it as many times as you like!" That explanation is simpler than the one that the current proposal will have to go through.

I think overall, changing rule 3.8.4 is simpler, clearer, more direct, and mitigates slate voting well enough. Nominate a work as many times as you like on your ballot. Each slot will count.

#328 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 01:32 AM:

viktor and Nancy:

What you are proposing has been discussed before. There's nothing inherently stopping us from allowing this in SDV-LPE, except for the fact that it is another rule change. This can be proposed, obviously. Not having this ability doesn't hurt the system, either, though.

Ultimately, we decided that the goal of the nomination stage was to suggest works that the fans think might be Hugo-worthy. That's all. In the final voting stage, voters actually rank their preferences. Not having ranked nominations also removes a large portion of strategy in voting, which was another design goal when we start all of this.

So, I don't think the proposal will fail because of this. You may disagree with the goals that were laid out, but we arrived at those goals after a great deal of very thoughtful discussion.

Kilo

#329 ::: Laertes ::: (view all by) ::: May 14, 2015, 01:43 AM:

I like option (2).

On its face, option (1) is more appealing, but this corner case isn't so cornery anymore. We can't see the future, but now that declining nominations has become more common, it seems proper to spell out how it's to be handled.

(3) would hurt the proposal's chances since replacing nominees who declined clearly improved the ballot this year. (4) invites catastrophe. (5)-(7) are excessively fiddly.

#330 ::: felice ::: (view all by) ::: May 14, 2015, 02:01 AM:

P J Evans @319: "felice, can you explain why counting the same work multiple times is more fair than the current rule of only counting it once? How is this not like a slate, because it gives a some people more voting power than others?"

It doesn't give anyone more voting power; the elimination comparison is based on how many ballots the work appears on, and the number of times it appears on each ballot is irrelevant. This really only matters in situations like "I like Alpha, Beta, and Gamma, but I like Alpha most, and I'm not sure I want to nominate the others because I don't want to help them beat Alpha". Nominating Alpha Alpha Alpha Beta Gamma would mean Alpha gets 0.6 points, and the other get 0.2 points; that nominator still only gets one point total. If they felt really strongly about Alpha, they could just list Alpha once and leave the rest of the ballot blank, so Alpha would have the full 1 point right from the start. This basically makes bullet voting a bit less appealing, which is a good thing, but the effect would be negligible.

#331 ::: P J Evans ::: (view all by) ::: May 14, 2015, 02:15 AM:

felice, dammit, pay attention: the process of choosing nominees is not preferential. You're trying to change that, and you have to convince people that it's better if we allow this kind of overvote.

So far, all you're saying that you think people should be able to put the same name in more than once in the same category, which currently is the same as putting it in ONCE. This is FAIR, it means that you can't stuff the ballot box by getting a group together to put the same name (or names) five times by each person. You see the problem? It's a slate in a slightly different form. We don't want that.

#332 ::: Darth Paradox ::: (view all by) ::: May 14, 2015, 02:28 AM:

P J Evans @ 331: If a nomination ballot of "Alice" (allocating 1.0 point to Alice) is fair, and a nomination ballot of "Alice, Bob" (allocating 0.5 points to Alice and 0.5 points to Bob) is fair, why isn't a nomination ballot of "Alice, Alice, Alice, Bob" (allocating 0.75 points to Alice and 0.25 points to Bob) fair? It doesn't violate the requirement that each person only has one total point to be allocated at any given time. And it doesn't enable slate voting any further; a bunch of people getting together to vote "Alice, Alice, Alice, Alice, Alice" is no different than a bunch of people getting together to just vote "Alice".

I agree that the lack of simplicity in describing such a system in the constitution is a strike against it, but I really don't see the argument on fairness that you're making here.

#333 ::: felice ::: (view all by) ::: May 14, 2015, 02:35 AM:

#331 - I really don't think you're getting how SDV-LPE works at all. With viktor's proposal, the ballots "A" and "AAAAA" would be exactly equivalent, just as they are now. The ballots "ABC" and "AAABC" would have slightly different effects on the relative rankings of A, B, and C, but they'd affect the rankings less than switching from "ABC" to "A" would. In no case would listing A more than once help it win an elimination comparison.

And I'm not interested in convincing anyone that viktor's proposal is a good idea; I don't care if it happens or not.

#334 ::: felice ::: (view all by) ::: May 14, 2015, 02:48 AM:

Keith "Kilo" Watt @325: " if you wouldn't mind ranking from 0-10 (per Jameson's system) the following: In the case of a declined nomination:"

3 - "Let the admins decide" - this would just result in the debate being repeated every year.
4 - "Add the works eliminated in the next-to-last round to the ballot" - easy but potentially unfair on those who nominated the declined work.
2 - "Do not replace the declined nomination" - easiest but even more unfair.
0 - "Re-run the election with the declined nomination removed; accept the new results as-is" - telling people they're finalists, then that they're not, isn't acceptable.
9 - "Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary" - not difficult, the fairest option, and the chances of the ballot getting unreasonably big are very small.
1 - "Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points" - if there are other new finalists, they'll find out when we publish the nomination data, and could quite reasonably be very upset about being excluded from the final ballot.
1 - "Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations" - ditto.

#335 ::: viktor ::: (view all by) ::: May 14, 2015, 02:49 AM:

@331
"the process of choosing nominees is not preferential."

Not now. However, the current proposal would make it so.

Simpler:

"The more you vote for your nominee, the more points it gets!"

#336 ::: Seth ::: (view all by) ::: May 14, 2015, 03:02 AM:

#325 In the case of a declined nomination:

8. Re-run the counting until there is only one finalist who was not on the original list, and add that one.

#337 ::: John M. Burt ::: (view all by) ::: May 14, 2015, 03:10 AM:

Does anybody have advice on how to deal with a parasite called http://www.general-ebooks.xyz/ ?

They are selling access to my writing without paying me. How can I stop them?

Sorry to be interrupting with such an elementary problem, but it is a new one to me.

#338 ::: Idumea Arbacoochee, Bad Cop ::: (view all by) ::: May 14, 2015, 03:12 AM:

Let me confirm what Keith "Kilo" Watt @304* has said: this thread is not about whether to change the process. That's a decision for the business meeting at Worldcon.

This thread is about one proposal to change it, which can then be brought to the conversation at Sasquan.

Given that, we're going to leave "whether" conversations aside from this point forward, along with a rehash of the behaviors that have led us to this point, except to the degree that they constitute valid input data.

Brian Z, if you're looking to get disemvoweled, now you know how. You can also achieve those heights by being personally insulting to other members of the conversation, trolling pointlessly, and ignoring moderators. It's actually not a very easy thing to do, for someone who is operating in good faith. Which, until I see evidence to the contrary, I assume everyone is.

My comparison of what you said on File 770 to what you say here is not yet sufficient evidence, but I would strongly suggest a better unity of message across sites that the same people read. It may be easier to remind yourself to do that if you include links back and forth, as you yourself advised others to do.

Also, Walter Daniels @290, you might have a better run in this community if you increased the spacing between your paragraphs (which I have amended), and maybe dropped the emotive language "screaming" and the accusatory tone. I know conspiracy theories are a delight to the intricately-oriented, but you either need evidence or objective probability, and I don't see either in your comment.

(And, for the record, I don't live in New York or work in publishing.)

------
* whom I esteem very highly indeed, though I have never previously had the honor of his acquaintance, online or off. Keith, do you write poetry?

#339 ::: Louis Patterson ::: (view all by) ::: May 14, 2015, 03:41 AM:

Declined nominations should use the same process as is used for handling late-detected ineligible entries, whatever that process is decided to be.

#340 ::: junego ::: (view all by) ::: May 14, 2015, 04:03 AM:

Keith "Kilo" Watt @ 325
In the case of a declined nomination:

I'd like to change and clarify the above condition for this proposal.

In the case of any change (elimination or withdrawal) in the finalist list after the finalists have been notified (and up to the point where the administrators determine that the ballet must be frozen to any further changes, at which point no new finalists will be added):

7 pts - Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
5 pts - Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
2 pts - Do not replace the declined nomination
0 pts - Re-run the election with the declined nomination removed; accept the new results as-is
10 pts - Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
0 - Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
0 - Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

Rationale: I've nuanced my previous positions. IF we are going to take any position on how to handle changes in the finalist list I now feel that felice's proposal is the cleanest and fairest. (I didn't originally grasp her point way up thread, so 'Oops'). If we are not going to address this question, then it would by default be left up to the administrators but I feel we should make them aware of the potential problem and our discussion/possible solutions.

Any eliminations or withdrawals *before* finalists are notified should entail a full rerun of SDV-LPE.

#341 ::: junego ::: (view all by) ::: May 14, 2015, 04:39 AM:

Nancy Lebovitz @ 324
P J Evans @ 319

Each person gets the a total weight of one vote per category. In effect, if they only vote for one nominee, it's as though they voted a fifth of a vote five times.
I don't see the problem if, instead of voting their five fifths for one work or giving one fifth vote to each of five works, they give two of their fifths to one work, and three of their fifths to another.

To the best of my understanding (and someone correct me if I'm wrong) the problems with this and all the permutations of proposing that members can/should prioritize their nominations are:

1) It would require additional constitutional changes because article 3.7.1 of the constitution says the nominations shall be "up to five (5) equally weighted nominations in every category." My emphasis.

2) After discussions in the last 2000+ comments, the consensus was that minimal changes had the best chance of being accepted. So the fewer ammendments and simpler the proposal, the better.

3) Those discussions also came to a consensus that more members than not disliked the idea of ranking the nominations, so any proposals to include weighting the recommendations was dropped.

And a clarification (I'm not sure from the above if there is confusion about this but better too much than too little explanation), If you nominate 1 thing it gets 1 point, 2 things means each gets 1/2 point, 3=1/3 pt, etc. up to 5 nominations.

As the Nominees 'challenge' each other and get eliminated, the proportions of your nominating point change if and when your nominees are eliminated. So if you had 3 and 1 loses its challenge, the remaining 2 go from having 1/3 point each to 1/2 point each.

#342 ::: junego ::: (view all by) ::: May 14, 2015, 04:40 AM:

Nancy Lebovitz @ 324
P J Evans @ 319

Each person gets the a total weight of one vote per category. In effect, if they only vote for one nominee, it's as though they voted a fifth of a vote five times.
I don't see the problem if, instead of voting their five fifths for one work or giving one fifth vote to each of five works, they give two of their fifths to one work, and three of their fifths to another.

To the best of my understanding (and someone correct me if I'm wrong) the problems with this and all the permutations of proposing that members can/should prioritize their nominations are:

1) It would require additional constitutional changes because article 3.7.1 of the constitution says the nominations shall be "up to five (5) equally weighted nominations in every category." My emphasis.

2) After discussions in the last 2000+ comments, the consensus was that minimal changes had the best chance of being accepted. So the fewer ammendments and simpler the proposal, the better.

3) Those discussions also came to a consensus that more members than not disliked the idea of ranking the nominations, so any proposals to include weighting the recommendations was dropped.

And a clarification (I'm not sure from the above if there is confusion about this but better too much than too little explanation), If you nominate 1 thing it gets 1 point, 2 things means each gets 1/2 point, 3=1/3 pt, etc. up to 5 nominations.

As the Nominees 'challenge' each other and get eliminated, the proportions of your nominating point change if and when your nominees are eliminated. So if you had 3 and 1 loses its challenge, the remaining 2 go from having 1/3 point each to 1/2 point each.

#343 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 04:46 AM:

On excellence and happiness: this system is designed to balance different kinds of evidence to find the 5 most-likely-to-be-excellent works. The fact that this is fair is no coincidence; unfair systems ignore evidence and promote strategic voting (bad evidence). The fact that this promotes happy voters is just a happy coincidence.

On multiple votes: there's nothing wrong with the idea except complexity. A separate proposal to allow this would be fine.

On withdrawn nominees : I vote 10 for "let the administration decide " or "let the BM decide (multiple proposals)". 7 for everything else (considered as proposals in isolation).

#344 ::: felice ::: (view all by) ::: May 14, 2015, 04:51 AM:

junego @340: "I'd like to change and clarify the above condition for this proposal. In the case of any change (elimination or withdrawal) in the finalist list after the finalists have been notified (and up to the point where the administrators determine that the ballet must be frozen to any further changes, at which point no new finalists will be added)

Any eliminations or withdrawals *before* finalists are notified should entail a full rerun of SDV-LPE.

Good points.

#345 ::: Aan ::: (view all by) ::: May 14, 2015, 05:11 AM:

[2] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[9] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[6] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[10] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[1] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#346 ::: Soon Lee ::: (view all by) ::: May 14, 2015, 05:31 AM:

[DECLINED NOMINATION]

[3]1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[2]2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[2]3. Do not replace the declined nomination
[0]4. Re-run the election with the declined nomination removed; accept the new results as-is
[8]5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0]6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[0]7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#347 ::: J Thomas ::: (view all by) ::: May 14, 2015, 06:04 AM:

@281 Jameson Quinn

I think that the "rerun and merge" contingent should make a separate proposal. It wouldn't have to involve lots of lawyerly language; just "It is the sense of the membership that if a nominee declines, and the ballots have not yet been finalized, then the award administrators should find all works which would have been nominees if the decliner were removed from all ballots, and offer them places as finalists, without withdrawing finalist status from any works which already have it."

That makes sense, and the SDE proposal should not mention what to do in the case of declines or removals.

If the BM rules allow it, maybe the second proposal should be a proposal-for-a-proposal. List the alternatives suggested so far and see which way the meeting chooses to amend.

It will rarely cause trouble any way we do it, and if the meeting considers it too unimportant to deal with, then it's settled.

And if the SDV proposal fails then it's moot.

#348 ::: Nancy Lebovitz ::: (view all by) ::: May 14, 2015, 06:41 AM:

#319 ::: P J Evans

I don't see how allowing multiple votes for the same thing makes the process less fair.

#326 ::: viktor

However, I also seriously doubt that whether preferential nominating votes are permitted is anywhere near a make-or-break for SDV-LPE.

#342 ::: junego :

Thanks for the explanation-- I'll quit pushing the idea in this round.

I have no idea how much people in general would prefer being able to vote preferentially.

#349 ::: J Thomas ::: (view all by) ::: May 14, 2015, 07:10 AM:

@280 Darth Paradox

I don't have the math to back this up at the moment, but my intuition tells me that promotion of sixth place runs a high risk of disenfranchising the slate bloc entirely.

The results will depend on circumstance. We can make up examples where a big variety of things happen, and the question is what's more likely, and that depends on lots of things. It's a hard question.

I have run simulations based on votes that are a lot like what happened in 2013. I can't be sure I got it right, but it probably isn't real far off. When I add a slate of 15% of the nonslate numbers, they average about 1.7 finalists. They are out of proportion to their numbers still, though not as much. This is because the other votes are very scattered.

Assuming that my simulated results are representative of later years, chances are if one slate work got removed late in the process there may quite likely already be another slate finalist.

Suppose though there's only one. Then it comes down to the details. While there are two slate nominees with around N votes, they will have scores around N/2. They will be paired up with nonslate nominees that have scores less than N/2 and beat every one of them, until they meet each other and one is eliminated. If there are only three nonslate nominees that have scores above N/2 at that time, then both slate items will be finalists. If there are four, then one of them will have the 6th spot and will be next in line if any of the top five is removed. If there are five or more, then the 2nd slate item is out of the running.

How can we predict that? If a nonslate work has less than N/2 votes total, then it will definitely be eliminated. In 2015, in a whole lot of categories, there were few works that had more than N/2 votes. If that's a precedent then you're home free.

Anyway, we can't expect to make it entirely fair to slate voters no matter what happens. If we go with "throw in whatever's in 6th place" and there's a 1/5 chance that a slate work will be in 6th place, then if an existing slate item gets removed that's a 1/5 chance we get another one. But if any of the other 4 items are removed we also get a 1/5 chance of a slate item. Is that close to fair?

Agreed that if we run the numbers again we definitely get at least one slate item the new way. It's more representative of the actual ballots.

All this may be academic. In 2013 the Puppy slate essentially did not exist. In 2014 it was a handful of people. In 2015 it was hundreds. By 2016 it could be well over a thousand. And by 2017 you guys may have moved on to something more important.

This whole exercise might turn into a case of locking the barn door after the horses have escaped, formed a commune, taken ownership of the farm, and burned the place down. But it's still worth preparing for some other slate.

#350 ::: J Thomas ::: (view all by) ::: May 14, 2015, 07:24 AM:

@250 Annie Y

[5% RULE]
So if there are other proposals to get rid of that one (yey - it made sense when the number of ballots was very low but not anymore...)

This is a minor picky point, but the issue is not how many ballots there are, but how many works that people might nominate. If they split their votes over 20 possible nominations, then somebody *has* to get 5%. If they split among 1000 works, then quite possibly none of them get 5%. That is true for 100 voters or a million voters. If they're limited to 5 votes each, and they split them among 1000 choices, then most choices will get very few.

#351 ::: J Thomas ::: (view all by) ::: May 14, 2015, 07:37 AM:

@252 Jameson Quinn

In a world without slates, there is essentially no benefit to bullet voting. Approval totals and honest point totals would agree on which candidates were better, so if your favorite candidate was losing, boosting its point total would only put off the inevitable (and if everybody did it, you could actually be worse off, because your favorite would be getting no second-choice votes).

I don't think that's quite fair, though it's pretty accurate.

If your preferred finalist is unpopular, it has little chance in any reasonable voting system.

If your preferred candidate is very popular, it has a very good chance in any reasonable voting system.

If it is popular enough to have a chance, you increase its chance by bullet-voting more than by voting for others also. If it is not one of the five with the most votes, its only chance comes if some among the top five have enough shared votes that they are selected to eliminate each other. The more of the ones with more votes than yours do that, the better chance yours has. The more shared votes your favorite has, the better the chance it will be eliminated.

However, usually it doesn't make much difference. And when you bullet-vote, you lose any chance to help out the other nominations you like. In general, when you vote for a second nomination you help the second one much more than you hurt the first one. So if you care about the second, vote for it.

I don't have the math for that and haven't thought about how to do it. Can you think of how? When you vote for a second work, how much on average does that hurt the first one's chances? How much on average does it help the second one's chances? How much does a third nomination hurt the first two compared to helping the third?

I can vaguely imagine a brute-force approach on that, but it seems ugly and it might take a long time. ;-)

#352 ::: Yarrow ::: (view all by) ::: May 14, 2015, 08:08 AM:

[10] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[6] 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[3] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[4] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[2] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[2] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#353 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 08:11 AM:

@295: it appears that something is going wrong with the loading of data. Try "tail (hugos)" to see what the data looks like.

#354 ::: nathanbp ::: (view all by) ::: May 14, 2015, 08:20 AM:

[2] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[5] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[2] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[8] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

I think it makes most sense to decide this clearly in the proposal rather than leave it to the admins later, that seems like a recipe for people being unhappy later.

J Thomas@351: That seems like the kind of thing that would require a lot of simulation with simulated voters with different voting styles. Your individual marginal vote is unlikely to change the results.

#355 ::: Cat ::: (view all by) ::: May 14, 2015, 08:43 AM:

For the purpose of weighting what I said upthread and also because two options are very close to even in my mind and right now I slightly prefer the other one.

[DECLINED NOMINATION]

[4]1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[9]2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[6]3. Do not replace the declined nomination
[0]4. Re-run the election with the declined nomination removed; accept the new results as-is
[7]5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0]6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[0]7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#356 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 08:45 AM:

No election theorist here, just a fan with gut feelings. But since you asked....

[9] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[5] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[7] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[6] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#357 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 08:52 AM:

Nancy Lebovitz @348, I think part of the problem here is terminology. Everyone gets to vote preferentially; that's what voting IS. What seems to be a sticking point is nominating preferentially. Which is a different thing.

And on a gut level, to me, nominations are "this is the thing that I loved; I hope you love it too!". Weighting nominations is "I'm going to throw three slips of paper with the same thing into the hat!" which, on a visceral level, strikes me as more unfair than "I'm going to throw three slips of paper with three things that I loved into the hat. Not necessarily rational, but I much much much prefer the second option. Please, tell me all the things you loved. But if you only loved one thing, you only have to tell me once.

Anyway, if you only loved one thing, then bullet-voting makes sure it gets 100% of your points in that category, so what's the point of pro-rating? If you loved it three times more than the other two things on your ballot, why are you nominating the other two things on your ballot?

#358 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 09:07 AM:

@351:

If your preferred finalist is unpopular, it has little chance in any reasonable voting system. Bullet voting doesn't help.

If your preferred candidate is very popular, it has a very good chance in any reasonable voting system. Bullet voting doesn't help.

If it is right on the edge - that is, if it comes in 6th place with your honest vote - then most probably it comes in 6th place in both points and approvals. Bullet voting can improve its points, possibly bringing it up to 5th place. But it still gets chosen for the final elimination round, and its approvals are still in 6th, so bullet voting doesn't help.

If the nonstrategic result is that it is in 6th place or below on points, but 5th place on approvals, then it would already be winning based on approvals, so bullet voting doesn't help.

If the nonstrategic result is that it is in 6th place or below on approvals, and 5th place on points, let's call it a "squeaker". Whether bullet voting helps still depends on some other factors.

If it's a squeaker, and the other things you might vote for are not in the top 5, then your ballot is already giving it full points when it matters. Bullet voting doesn't help.

If it's a squeaker, and without your ballot it is more than half a point below the fourth place work, then bullet voting almost helps, but isn't enough.

If it's a squeaker, and you would have voted for one of the top 5, and it is less than half a point from 4th place... then, bullet voting helps.

But even then, it's worth analyzing further. Why would a work end up being a squeaker? That is, why would it do better on points than on approvals?

There are basically two reasons that could happen. Either it could be getting more points than you'd expect from its approvals, or its closest rivals could be getting fewer points than you'd expect based on their approvals.

If it's the former — the work in question has more points per approval than most others — then that could be because of something about the work itself, or something about its fans. Perhaps the work is something that naturally makes some people say "it would be a crime if this one thing didn't get the Hugo"; that is, perhaps it's really outstanding, but it's a difficult or controversial work so it is still a squeaker. In that case, bullet voting could help, but it's entirely appropriate that it does; the work probably truly deserves a place among the finalists. Or perhaps its fans are just more inclined to bullet voting strategy than other fans. In that case, it probably doesn't deserve a slot.

So finally, after analyzing 8 different possible scenarios, each less likely than the last, we've come up with a situation where bullet voting strategy is helpful to the voters but harmful to the ideals of the Hugo.

But we're not quite done. What if your favorite is a squeaker not because it got more points per approval than average, but because two of its closest rivals got fewer? In that case, its rivals are probably on a slate. So if slates are common, bullet voting may help. But note that in this case, too, the bullet voting result is arguably "fairer" than the non-bullet-voting result.

So final tally: 9 scenarios, in order of increasing unlikeliness and specificity. Bullet voting only helps in scenarios 7, 8, and 9; the three least likely ones. Bullet voting leads to a worse overall result (less "excellence" among the finalists) only in scenario 8.

....

As to running simulations to see how likely all this is in practice: this kind of thing is the focus of much of my research. It's tricky to do; you have to make assumptions about what kind of voting is likely, and results can depend a lot on those assumptions. Based on the intuition I've gotten from running these sims, I think that you could make a sim where the average effect of bullet voting was non-negligible, but only by making assumptions I'd characterize as implausible.

#359 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 09:08 AM:

viktor @309, I don't understand your objection. Your nominating ballot *isn't* weighted in any way, either the "way you wanted it" or some other way. Each nomination in a category has exactly the same weight as every other nomination you make in the same category. As works you nominated get eliminated, the remaining works you nominate in that category get stronger. Equally. Until we're down to five candidates. If you nominated Alex, Barbara, Charlie, Dora, and Edward, and after all the tests Alex, Dora, Edward, Frances, and George end up on the ballot.... your 1/3 nomination each helped them get there. (But note, it started as a 1/5 nomination each, until Barbara and Charlie were eliminated.)

And if Alex declines (or is found invalid), and they rerun the results, you're starting with 1/4 nomination for Barbara, Charlie, Dora, and Edward. Which might be just enough of a difference to kick Barbara or Charlie into the finals, on a re-run. Is that a bad thing? I mean, you liked them enough to put them on your ballot... <smile>

(Note: I favor keeping Dora, Edward, Frances, and George on the ballot regardless of the results of the rerun. I like the "merge the two runs" option best, but I'd take "additional top point-getter added")

I hope this clarifies; it looks to me like you're operating from a fundamental misunderstanding of the process. I have every sympathy; I'm not a math or voting geek, and if I've misrepresented the system I hope someone corrects me right away.

I personally had real trouble following this discussion for a while until I realized that the word "vote" used about the nomination process was distorting my understanding of what was going on, because a "vote" implies ranking on a ballot, and nominators aren't ranking on a ballot. They're throwing names into a hat.

#360 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 09:12 AM:

This is a repost of my previous comment, with numbering added, so it's clear how I'm counting the scenarios.

@351:

1. If your preferred finalist is unpopular, it has little chance in any reasonable voting system. Bullet voting doesn't help.

2. If your preferred candidate is very popular, it has a very good chance in any reasonable voting system. Bullet voting doesn't help.

3. If it is right on the edge - that is, if it comes in 6th place with your honest vote - then most probably it comes in 6th place in both points and approvals. Bullet voting can improve its points, possibly bringing it up to 5th place. But it still gets chosen for the final elimination round, and its approvals are still in 6th, so bullet voting doesn't help.

4. If the nonstrategic result is that it is in 6th place or below on points, but 5th place on approvals, then it would already be winning based on approvals, so bullet voting doesn't help.

If the nonstrategic result is that it is in 6th place or below on approvals, and 5th place on points, let's call it a "squeaker". Whether bullet voting helps still depends on some other factors.

5. If it's a squeaker, and the other things you might vote for are not in the top 5, then your ballot is already giving it full points when it matters. Bullet voting doesn't help.

6. If it's a squeaker, and without your ballot it is more than half a point below the fourth place work, then bullet voting almost helps, but isn't enough.

If it's a squeaker, and you would have voted for one of the top 5, and it is less than half a point from 4th place... then, bullet voting helps.

But even then, it's worth analyzing further. Why would a work end up being a squeaker? That is, why would it do better on points than on approvals?

There are basically two reasons that could happen. Either it could be getting more points than you'd expect from its approvals, or its closest rivals could be getting fewer points than you'd expect based on their approvals.

If it's the former — the work in question has more points per approval than most others — then that could be because of something about the work itself, or something about its fans.

7. Perhaps the work is something that naturally makes some people say "it would be a crime if this one thing didn't get the Hugo"; that is, perhaps it's really outstanding, but it's a difficult or controversial work so it is still a squeaker. In that case, bullet voting could help, but it's entirely appropriate that it does; the work probably truly deserves a place among the finalists.

8. Or perhaps its fans are just more inclined to bullet voting strategy than other fans. In that case, it probably doesn't deserve a slot.

So finally, after analyzing 8 different possible scenarios, each less likely than the last, we've come up with a situation where bullet voting strategy is helpful to the voters but harmful to the ideals of the Hugo.

9. But we're not quite done. What if your favorite is a squeaker not because it got more points per approval than average, but because two of its closest rivals got fewer? In that case, its rivals are probably on a slate. So if slates are common, bullet voting may help. But note that in this case, too, the bullet voting result is arguably "fairer" than the non-bullet-voting result.

So final tally: 9 scenarios, in order of increasing unlikeliness and specificity. Bullet voting only helps in scenarios 7, 8, and 9; the three least likely ones. Bullet voting leads to a worse overall result (less "excellence" among the finalists) only in scenario 8.

#361 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 09:23 AM:

Can I request that the people voting in the [DECLINED NOMINATIONS] poll (including those who have already voted) rate a couple of additional options? To wit:

We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better

We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.

I'd vote 10 for either of the above options.

#362 ::: J Thomas ::: (view all by) ::: May 14, 2015, 09:26 AM:

@354 nathanbp

J Thomas@351: That seems like the kind of thing that would require a lot of simulation with simulated voters with different voting styles. Your individual marginal vote is unlikely to change the results.

Yes, exactly!

Your individual vote makes so little difference, and your careful strategy based on your estimate of what's popular makes so much less difference, that it's best to just pick the things you think most deserve the Hugo. Putting much thought about voting strategies is surely not worth doing unless you are part of a cabal that intends to work together to get a disproportionate effect on the nominations.

So it becomes a question how much can organized groups of voters affect the results by various strategies, and how large a group does it take to make much difference.

And yet, some people will go by their gut feeling of what works better, and then use their gut feeling about what's popular, to do strategic voting. And if there are a lot of them it will affect the results in unknown ways.

Jameson is right that if a lot of people bullet-vote at random, the result is that they individually have less effect (because their 2nd and 3rd choices are less likely to win, while their 1st choices don't get the benefit of other people's 2nd and 3rd choices). And by reducing the number of total votes, they make other voting strategies work better.

My guess is that bullet-voting by a cabal is likely to have the largest effect if the item they bullet-vote for is in 6th place, somewhat less in 5th place or 7th place. By the time it gets up to 3rd place or down to 9th place it will probably have little effect even for a large group.

How much effect it has depends on how spread the ballots are for works that are more popular, and on who their ballots are split with. You will not know that, and depending on how it comes out, a group's effort at bullet-voting could have no effect even in the best case. Your group could bullet-vote for something that would win anyway, or that would inevitably lose anyway.

The strategic voting rule that I think works, is:

If you like Annie and Bob, and you would hate for Bob to win while Annie loses, then vote only for Annie. Particularly if you think Bob is more popular.

There are voting systems that let you arrange to vote for Bob only after Annie has lost, or only after it's decided that Annie has won or lost. They tend to have other problems that got us to put them aside.

#363 ::: J Thomas ::: (view all by) ::: May 14, 2015, 09:27 AM:

@354 nathanbp

J Thomas@351: That seems like the kind of thing that would require a lot of simulation with simulated voters with different voting styles. Your individual marginal vote is unlikely to change the results.

Yes, exactly!

Your individual vote makes so little difference, and your careful strategy based on your estimate of what's popular makes so much less difference, that it's best to just pick the things you think most deserve the Hugo. Putting much thought about voting strategies is surely not worth doing unless you are part of a cabal that intends to work together to get a disproportionate effect on the nominations.

So it becomes a question how much can organized groups of voters affect the results by various strategies, and how large a group does it take to make much difference.

And yet, some people will go by their gut feeling of what works better, and then use their gut feeling about what's popular, to do strategic voting. And if there are a lot of them it will affect the results in unknown ways.

Jameson is right that if a lot of people bullet-vote at random, the result is that they individually have less effect (because their 2nd and 3rd choices are less likely to win, while their 1st choices don't get the benefit of other people's 2nd and 3rd choices). And by reducing the number of total votes, they make other voting strategies work better.

My guess is that bullet-voting by a cabal is likely to have the largest effect if the item they bullet-vote for is in 6th place, somewhat less in 5th place or 7th place. By the time it gets up to 3rd place or down to 9th place it will probably have little effect even for a large group.

How much effect it has depends on how spread the ballots are for works that are more popular, and on who their ballots are split with. You will not know that, and depending on how it comes out, a group's effort at bullet-voting could have no effect even in the best case. Your group could bullet-vote for something that would win anyway, or that would inevitably lose anyway.

The strategic voting rule that I think works, is:

If you like Annie and Bob, and you would hate for Bob to win while Annie loses, then vote only for Annie. Particularly if you think Bob is more popular.

There are voting systems that let you arrange to vote for Bob only after Annie has lost, or only after it's decided that Annie has won or lost. They tend to have other problems that got us to put them aside.

#364 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 09:34 AM:

@361, We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better

We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.

I'm not entirely sure what your second proposal means; is it (a) "let the administrators decide, and these are their two options to decide from"? Or is it (b) "Let the administrators decide" and then two separate proposals after the initial proposal which modifies that one way or the other? (On re-reading, I think this is what you meant, but I'm not positive).

If (b) is what you mean, I vote ten on both proposals. If (a) is what you meant, it just adds confusion, in my opinion.

And I forgot to put [DECLINED NOMINATION] in my 356. My bad.

#365 ::: Duncan J Macdonald ::: (view all by) ::: May 14, 2015, 09:36 AM:

In the case of a declined nomination:

10 -- 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
0 -- 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
5 -- 3. Do not replace the declined nomination
0 -- 4. Re-run the election with the declined nomination removed; accept the new results as-is
0 -- 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
0 -- 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
0 -- 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#366 ::: Duncan J Macdonald ::: (view all by) ::: May 14, 2015, 09:46 AM:

[DECLINED NOMINATIONS] Addendum

0 -- We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better

10 -- We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.

#367 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 09:50 AM:

@364: Yes, it's (b). Here's a better wording:

We create a base proposal which lets the administrators decide, and two supplementary proposals which would change that to "promote #6" or "rerun and merge". Let the business meeting decide which to pass.

#368 ::: J Thomas ::: (view all by) ::: May 14, 2015, 10:02 AM:

[10] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)

If they create a common practice the BM can confirm it later.

[0] 2. Add the works elininated in the next-to-last round to the ballot

If it's controversial here it may be controversial at the BM.

[4] 3. Do not replace the declined nomination

Not great but do-able.

[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is

If it's controversial here it may be controversial at the BM.

[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary

It doesn't hurt anybody much. If somebody becomes a finalist you think shouldn't, they don't hurt the other finalists much unless they win, and if they do win the Hugo then probably they deserved to be on the ballot.

[0] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points

bad compromise between two controversial proposals.

[0] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

bad compromise.

[1] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better

OK if the chair likes it.

[1] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.

OK if the chair likes it.

[10] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

I think that RME is the obvious best choice, while "add the old 6th" is one of the controversial also-rans which does not deserve equal billing. Others may disagree. ;-) Better to give only "let the administrators decide" top billing, since that's what they will want to do if it's controversial at the meeting and they don't want to spend much time on it.

#369 ::: albatross ::: (view all by) ::: May 14, 2015, 10:05 AM:

[0] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[10] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[5] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[5] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[5] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

This new voting scheme makes it hard to know how to deal with declined nominations or disqualifications after the nominations were announced, so I think it's important to nail that down up front. I don't see why the admins of the election will be in a better position to decide what to do later on.

I also think it's a disaster to announce someone's nomination and then withdraw it because of someone else's disqualification. That's pretty much optimal for creating bad feelings all around.

As I understand it, any way we rerun the election with removed candidates has the possibility of giving us weird results like A,B, C,D,E wins the first election, and after disqualifying A getting some very different outcome like B, F, G, H, I. Radically different outcomes are unlikely but not impossible, right? That leaves us with some ugly decision where we either end up with 8 nominees, or have to decide which winners of the second election get nominated.

It seems simpler and better all around, to me, to just get an ordered list from the original election, and then fill in removed nominees from that list. It's not a disaster to do this in a more complicated way, but I don't think it's a very good idea.

#370 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 10:12 AM:

abi@338:

Keith, do you write poetry?

Yes, but sadly nothing worth reading. One of the only downsides to learning to write is that you suddenly realize just how atrocious some of your stuff is. :) My prose is somwhat better. Stanley Schmidt said my writing was "quite good at times" -- but he still didn't buy the piece.

Kilo

#371 ::: J Thomas ::: (view all by) ::: May 14, 2015, 10:22 AM:

@282 felice

I would certainly consider my nomination to have been wasted if the work in question was withdrawn after becoming a finalist, and extremely peeved if any of my other nominations missed out on being finalists because of the wasted nomination.

We're arguing about what's fair, and there's fundamental disagreement.

It's something which will come up rarely. (Though with 17 or so different votes/year, and possibly declines will go up, it might come sooner than we'd expect.)

I don't think you will convince your heart-felt opponents, whatever you say.

So we need a way to put this off. When we argue about this topic we are delaying a proposal. (That's good for people who don't want a proposal.)

It will be hard for us to agree, and then if the business meeting is also disagreed then anything we decide on will be a reason for them not to pass it.

When you and I disagree about what's fair to you versus me, we can probably work out some compromise. When we disagree about what's fair to some third party in the future, that's a lot harder. We need to finesse this and get on with the other issues.

#372 ::: Laertes ::: (view all by) ::: May 14, 2015, 10:34 AM:

Okay, I think I see now how this is done. Here's another attempt at my @329, maybe formatted properly:

[7]1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[10]2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0]3. Do not replace the declined nomination
[0]4. Re-run the election with the declined nomination removed; accept the new results as-is
[4]5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[1]6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[1]7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations

#373 ::: Laertes ::: (view all by) ::: May 14, 2015, 10:42 AM:

Brian Z @288:

had my words on another forum stripped from the topic sentence of the paragraph to paint my words in the worst possible light and cited out of context without a link

You know, I let this go the first time (@152) you demanded a link that was already present, but since you're still banging on about it: In my @148, I omitted no meaningful context, and I provided a link.

You seem to be aware that "I was quoted out of context" is phrase with some power, but you're not getting how it works. Used correctly, it means that there was surrounding material that would make the quote less damning. In a case like this one, where you regret the quote because it turns out to have been a dumb thing to say, what you're looking for is "I apologize for lying about you all," not "I was quoted out of context.

#374 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 10:42 AM:

I've not been following this thread too closely, because, honestly, my eyes start glazing over after a while. My problem, not yours. I have, however, been ducking in every so often to check it out, and noticed the voting on how to deal with withdrawals/disqualifications. After scrolling back to get a sense of what each option means, here're my gut choices:

[8] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[5] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[4] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[3] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[5] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[10] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[8] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#375 ::: albatross ::: (view all by) ::: May 14, 2015, 10:44 AM:

I don't think the new voting scheme proposal is complete until it deals with withdrawn candidates, because that's an important feature of Hugo nominations, and it potentially interacts badly with SDV-PE. Worse, it's a little unintuitive and complicated to think about, which means there's a lot of opportunity for people to make dumb decisions and also a lot of opportunity for people to convince themselves some dumb decision was made from evil motives. Most peoples' experiences with voting would lead them to expect that dealing with a withdrawn candidate would be trivial, not something that could conceivably change the whole set of nominees.

I also don't see why kicking the can down the road makes it more likely to be solved well later--the worst case is to have the admins making the decision after the votes have been cast and the situaiton has come up, and that's an awful situation to leave them in.

#376 ::: Shane ::: (view all by) ::: May 14, 2015, 10:51 AM:

[FAQ / ARGUMENT]
"1.How does this eliminate slates?"
I think the FAQ should acknowledge the Puppies main concern (valid or not) by saying SDV-LPE limits the effect of hidden, informal or accidental slates the same as it does public ones.

And then say "With SDV-LPE, slates or blocs should get results in proportion to their numbers."


"15. Wasn’t this system just designed by Social Justice Warriors to block the Good Stuff?"

This answer should say, probably at the start "Those of us who worked on the system feel it is the best way to address the Puppies' concerns about hidden or informal slates. We are keen to discuss it and to run model data with any interested parties."

I think 15 spends too much time on TNH and PNH, (and Kilo's laudable bio ;). Better to say that all the discussions were held in the open and can be read at (link), as well as the bit about the goals and the developer's broad tastes in fiction.

#377 ::: Buddha Buck ::: (view all by) ::: May 14, 2015, 10:56 AM:

[DECLINED NOMINATIONS]

I've seen folks earlier talk about the chosen finalists as if there was some inherent order. I'd like to stress that there isn't, in my opinion, an order to the finalists (or to the nominated works in general), there's only a "not eliminated yet". So proposals which basically state "generate the top N (>5), take the top 5, and work down the list if a nomination is declined/ineligible" are, in my opinion, fundamentally flawed because there isn't a "top 5" and the a list to work down. Similarly with statements about #1-4 of the finalists have property A and #5 has property B. As such, I disagree with the parenthetical comment in option 2


[7] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[0] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[3] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[3] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[3] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[7] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[8] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[8] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#378 ::: J Thomas ::: (view all by) ::: May 14, 2015, 11:00 AM:

@290 Walter Daniels

The "Traditional (Big) 5 Publishers" all live/work in NYC. That means they all attend pretty much the same parties, know each other, etc. As a result, in an area with 8.5 Million people, you have 1,000 (at maximum), who can "informally agree" that "Author X, of Publisher Y, should win." At worst it's 5 agreed on. Giving the "group/slate") 200 to 500 assured votes.

That's a specific proposal, that could be somewhat testable.

We have the voting data for 1984. I assume that the big deal is novels (where the publishers would win if their last-year's wins keep selling, and for reprints, and having authors in their stables with Hugos could help some).

We could look at the novels from 1984, and measure correlations among ballots. Votes for winners by employees of one publisher would tend to favor just that publisher. We could get a sense how important that was, in that one year.

It would be pretty easy to see that for winners. With some effort it could be expanded to losers also -- see how much people voted for just one publisher. That requires finding the publishers for all the losers, which is a bigger effort.

We could do the same for later years if the anonymized data becomes available.

I will do this someday in my copious spare time. Right now my spare time goes into some other things that are higher priority for me and more time-critical, but I will put it on the list.


On rereading, it looks like you're saying that publisher employees don't just vote for their boss's products, but instead that they talk to each other and form a consensus about what's good, and then they tend to vote for the same things regardless who wrote them. That is much harder to test for. If you think there are 200-500 people like that, an average of 40 to 100 people working for each publisher and paying for Worldcon supporting memberships or better, I tend to have my doubts. Anyway there are more fans than that who go to a lot of conventions and discuss SF with each other. They don't slate-vote because they disagree a lot.

#379 ::: P J Evans ::: (view all by) ::: May 14, 2015, 11:09 AM:

348
They're all the same thing: putting it on your list more than once may be enthusiasm, but it shouldn't get you more say. And it's easier to game than the current system. (Think of this year, and imagine if they'd done that instead, with a system where each appearance of a name on a list counts as a separate name. Let's not do it that way.)

#380 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 11:09 AM:

Supplementary vote on [DECLINED NOMINATIONS]

[10] Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

....

As far as I can tell, the name issue is now resolved, with a consensus of "just call it the Hugo Nominations System, and if you want to refer to the discussions where it was created, SDV-LPE is OK too."

The tiebreaker issue is resolved to my satisfaction, though possibly others may disagree. Using points from all previous rounds will break over 99.9% of significant ties; in the less than 0.1% of cases remaining, eliminating both is an acceptable answer.

So as soon as we've got the declined nominations question resolved, we're ready to finish up the language and make the proposal formally. I think we should also have an FAQ, but somehow keep it dynamic, not freeze it and attach it to the formal proposal.

There are several other associated proposals that have been mentioned (aside from the stuff with declined nominations):

1. Remove the 5% cutoff.
2. Remove the 5-per-category-per-ballot limit.
3. Allow multiple votes for the same work, so people can tune how their points are distributed.

I weakly favor 1 and 2, and weakly oppose 3. I think that 1 and 2 should be proposals if they aren't already. With 2, I think there should still be some high cutoff, just to prevent annoying spam — 20 should be plenty.

#381 ::: Brian Gibbons ::: (view all by) ::: May 14, 2015, 11:14 AM:

Keith "Kilo" Watt @307:

That does make the point of disagreement a bit more clear.

You're saying that the Constitution doesn't say anything about what to do with the final ballot after a nominee declines or is ineligible, so if there's a gap left, the administrator can do anything he wants to fill it.

I'm saying that 3.8.1 defines what works can be listed on the final ballot, so if an administrator decides to issue a new version of the final ballot, they're simply using 3.8.1 to determine "the five eligible nominees receiving the most nominations" (with an updated list of 'eligible nominees') and coming up with a new final ballot according to those rules. The Hugo administrator can choose to leave the ballot alone, but the only way he can issue a new ballot is by using the rules that say what nominees go on the final ballot.

So, under your interpretation, changing 3.8.1 to 3.8.8 doesn't matter. If there's a decline or ineligibility, the administrator can still do whatever he wants to fill in the gap.

Under mine, changing the system to 3.8.8 changes everything, because it now means that if the administrator wants to issue a new final ballot after a decline/ineligibility, the only way he can do so is by following the directions that 3.8.8 gives as to how the nominees on the final ballot are determined (which means running the elimination procedure again, and potentially kicking someone off the ballot who was on it before), and that if we don't consider that an acceptable solution, then we need to include a clause saying so.

This means that, as I see it, "Let the admins decide" is a null option, because it assumes they have powers which they actually do not.

I have zero experience of WSFS rules interpretation or Hugo administration, so perhaps someone with more experience in that area might give their opinion as to which is correct.

It should be noted that while the number of withdrawals this year is unusual, that the ballot needed to be adjusted is not. This isn't some bizarre edge case that hardly ever happens. Six out of the last seven Hugos have had nominees withdraw or declared ineligible. Each of the last three years before this one have had three separate categories that needed to be adjusted, including several cases in which the nominators could not have predicted this and were entirely blameless (e.g., in 2012, Game of Thrones Season 1 was nominated in BDP Long Form and they chose to accept the nomination, which meant the three episodes nominated in BDP Short Form were declared ineligible; in 2013, the nomination for Game of Thrones Season 2 in BDP Long Form was declined, in favor of having a single episode nominated in BDP Short Form).

There is a very good chance that the first year that this is put into effect, some nominee will withdraw or be declared ineligible. Unless it is clear how the SDV-LPE system would handle this situation, I think it's likely that it will be seen as flawed and insufficiently detailed to deal with the nomination process.

#382 ::: J Thomas ::: (view all by) ::: May 14, 2015, 11:17 AM:

@375 albatross

I also don't see why kicking the can down the road makes it more likely to be solved well later--the worst case is to have the admins making the decision after the votes have been cast and the situaiton has come up, and that's an awful situation to leave them in.

Welcome to the world of politics.

One vital reason to kick the can down the road to the business meeting is that we are fundamentally disagreed about the issue and (at least I) see no reason to expect a consensus from further argument.

It looks to me like RME is gaining ground, but if it gets a solid majority with a minority who hate it, that isn't good. If we do get a consensus without a whole lot more argument, then great!

It might likely make sense for the business meeting to kick it down the road to later business meetings or to the administrators, if they can't agree quickly. They have limited time to argue. If we disagree they might easily disagree too. I'd rather see the voting system pass with this issue left to the administrators, than see it fail either pre-nomination or at the business meeting.

The administrators will probably be as good as anybody else at figuring out what to do. They have two years to think about it, and then probably more years before it turns into a crisis. If they can't reach a consensus, every year they have a leader who can decide it for that year so they can get on with their work.

The danger is mostly that they might decide it isn't a problem, or forget about it, and then at some unpredictable time it comes back and bites them. I don't know how likely that is. Right now the argument is getting in the way of getting anything else done.

I agree that it would definitely be better to agree on a great approach and go with it. But if in fact we can't do that, we do better to kick the can down the road rather than pick something a lot of people think is unfair and try to run with that.

#383 ::: Shane ::: (view all by) ::: May 14, 2015, 11:26 AM:

[DECLINED NOMINATIONS]
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary.

[0] for all other options.


Albatross @ 375 I agree, especially about not kicking the can down the road. Except I think re-running will rarely be different and when it is the difference is important.

Say 200 Making Light readers nominate two works, "Bitten Plums, Collected Poetry" and "Idumea's Favourite Tribute and Cookbook." Except I don't nominate the first because Ick, plums. You'll end up with the cookbook at 200 points and Bitten Plums eliminated at 99.5.

Say further Idumea's cookbook is nominated, along with four other works that have around 150 nominations each. If somehow cookbooks are found ineligible, then if we just take number 6, Making Light readers won't have any nominations, and the ML poetry book would be skipped. Whereas if we recounted it would be far the most popular. (which seems better to me).

#384 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 11:26 AM:

Brian Gibbons @381

If I understand you correctly, you're saying that "let the administrator(s) decide" means they'd they'd not be able to do anything but rerun the whole ballot, and potentially un-nominate something already nominated? If that is the case, then my vote above would change from "8" for let the admins decide, to "0". Because whether or not that is technically fair, emotionally it's horribly unfair.

If I'm misunderstanding you (or if you're misunderstanding the situation), then my vote stands. But given the choice between a) "rerun the whole vote" and either of b)"rerun and merge" or c)"add the last eliminated", I'm emphatically against a and for either b or c (though I prefer b).

#385 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 11:29 AM:

The dreaded Internal Server Error has struck again.

#386 ::: Yarrow ::: (view all by) ::: May 14, 2015, 11:33 AM:

Brian Gibbons @ 381: I'm saying that 3.8.1 defines what works can be listed on the final ballot, so if an administrator decides to issue a new version of the final ballot, they're simply using 3.8.1 to determine "the five eligible nominees receiving the most nominations" (with an updated list of 'eligible nominees')

This convinces me. I'd now say that yes (alas!) we need to specify what to do in the case of ineligible or declined nominations.

And the example of the Game of Thrones, where either the season or the episode(s) were going to be declared ineligible, but the nominators could not know which -- that has convinced me that rerun-and-merge is the best option.

The following is the libretto of my changed tune:

[0] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[5] 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[5] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[1] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[1] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[5] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[1] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[1] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#387 ::: J Thomas ::: (view all by) ::: May 14, 2015, 11:38 AM:

@380

As far as I can tell, the name issue is now resolved, with a consensus of "just call it the Hugo Nominations System, and if you want to refer to the discussions where it was created, SDV-LPE is OK too."

Agreed.

The tiebreaker issue is resolved to my satisfaction, though possibly others may disagree. Using points from all previous rounds will break over 99.9% of significant ties; in the less than 0.1% of cases remaining, eliminating both is an acceptable answer.

Say you go back five steps and one of them is ahead on points. But from six steps back to the beginning, the other is ahead on points.

I don't see that this is much better than nomination date to break a tie on. If they have the same number of votes and the same number of points when you're choosing between them, eliminate them both or keep them both.

But it isn't a giant issue that overrides everything else.

I think we should also have an FAQ, but somehow keep it dynamic, not freeze it and attach it to the formal proposal.

Yes. A note with the proposal might include a link to one or more FAQs?

1. Remove the 5% cutoff.

Separate proposal. This one works either way.

2. Remove the 5-per-category-per-ballot limit.

Separate proposal. This one works either way.

3. Allow multiple votes for the same work, so people can tune how their points are distributed.

It would change the results of this kind of voting, I don't know just how. I think it would let a smaller voting block win one nomination. But if you count it as five votes but 1/5 of a point, or vice versa, I don't know what to expect. If it's set up so a smaller voting block wins, then an existing slate could split into 5 groups that each nominates one item 5 times, and be just as good as a slate. I definitely don't want that. Since it has a big effect, and the effect varies by details of how the voting system handles it, maybe it can't be done as a separate proposal.

#388 ::: Brian Gibbons ::: (view all by) ::: May 14, 2015, 11:45 AM:

Cally Soukup @384: If I understand you correctly, you're saying that "let the administrator(s) decide" means they'd they'd not be able to do anything but rerun the whole ballot, and potentially un-nominate something already nominated?

Yes. It's entirely possible I'm wrong about the rules (and hey, if someone wants to light up the Kevin Standlee signal, I'm happy to be slapped down if I am), but the way I'm reading them, the Hugo administrator has no obligation to issue a new version of the final ballot after someone withdraws. However, if they do decide to do so, they can't just decide on their own who goes on the new version of the final ballot, they're essentially just following 3.8.1 which tells them who goes on the final ballot.

If you replace 3.8.1 with a new set of rules for determining who goes on the final ballot, then that also changes the rules for what a Hugo administrator needs to do if they want to issue a new version of the final ballot. They don't have any discretion beyond choosing whether or not to issue a new version; if they do decide to issue a new version of the final ballot, the only way to determine what goes on it is the procedure in 3.8.8 (which could potentially lead to nominees who were on the first version not appearing on the new version).

#389 ::: colin roald ::: (view all by) ::: May 14, 2015, 11:50 AM:

I find this proposal really interesting. It seems compelling to me as a way to reduce the power of slates while having minimal effect on the way the Hugos have always run.

I've taken a stab at rewriting the Simple Explanation to be clearer for nontechnical people or just ones who aren't really paying attention. But since I haven't participated in the discussion til now, it didn't feel right to make an edit as big as this to the Google Doc. If you guys think it's improvement, feel free to use it.

The most important change, I think, is clearing up some of the vocabulary. "Nominating" works to be "nominated" for a Hugo is confusing. I've fixed things to always refer to "proposed works" until the final ranking is established, at which point some of them become "nominated" works.

SIMPLE EXPLANATION

Least Popular Elimination (formally called "single divisible vote with least popular eliminated" or SDV-LPE for short) is straightforward and fair.

The goal is to allow people to simply nominate the list of works in each category that they feel would be deserving of a Hugo, very much as they always have. The procedure for a nominator remains exactly as it has been: in each category you can submit a list of up to five unranked “proposed works" for nomination. From the collection of all nomination ballots, the five works representing the widest selection of nominators will be selected to appear as the nominated candidates on the final Hugo ballot.

The procedure for ranking proposed works by the Hugo Administrators is the only thing being altered.

- Each nominator is given 60 “nominating points” for each category. (These are assigned transparently during the counting process. The nominators themselves never have to think about them.)
- Those points are divided equally among all proposed works. If you propose one work, it gets all 60 of your points. If you propose two, they initially get 30 each; if you propose five, they initially get 12 each.
- All the points for each work from all the ballots submitted are added together, and the two works that got the least number of points are compared with each other. The least popular is eliminated. (To account for possible ties, the rules for comparing are somewhat technical. See [reference] for details.)
- The eliminated work is “erased” from all submitted nominating ballots and the point totals for all remaining works are recalculated as if the eliminated work had never been present. For instance, if you had proposed “Dune”, “Tau Zero”, and “Ubik” on your ballot, and it was determined that “Ubik” was least popular and eliminated, then your 60 nominating points are reassigned with 30 to “Dune” and 30 to “Tau Zero”.
- We start over for the next round, and repeat the process until five works remain, which become the nominated works.

This means you have no penalty for “wasting” a vote by proposing an obscure work. If it turns out not to be popular enough to be nominated, your nominating ballot is recalculated exactly as if you hadn’t mentioned it.

Since points are reassigned as works are eliminated, they become concentrated in the final nominated list. If only one proposed work from a nominator survives, then it receives all 60 of your points in the tally, while if two survive, they are each receiving 30 of your points. It is therefore difficult for a slate pushed by a minority of nominators to take more than a minority of the nominations.

#390 ::: colin roald ::: (view all by) ::: May 14, 2015, 11:56 AM:

Brian@62:
The goal is to honor and reward excellence, not to feel fair or make as many people happy as possible.

Actually, I think "make as many people [sort of] happy as possible" *is* the goal, since it's the opposite of "let a minority push through a slate that everyone else has to live with."

#391 ::: P J Evans ::: (view all by) ::: May 14, 2015, 11:57 AM:

348
Or put it this way: the nominees are determined not by how many times they're named, but by how many people name them. So: one person, one time.

#392 ::: Yarrow ::: (view all by) ::: May 14, 2015, 12:00 PM:

Jameson Quinn @ 380:, J Thomas @ 387: 3. Allow multiple votes for the same work, so people can tune how their points are distributed.

Like Jameson, I'm mildly against this. (I see it as requiring a more complicated explanation.)

Requiring unique works on the same ballot allows (only) the following distributions of points:

1) Alpha, Beta, Gamma, Delta, Epsilon - 0.20 point each
2) Alpha, Beta, Gamma, Delta - 0.25 points each
3) Alpha, Beta, Gamma - 0.3333... points each
4) Alpha, Beta - 0.5 points each
5) Alpha - 1.0 point

Allowing multiple mentions (with a maximum of five total mentions) allows also point distributions like

a) Alpha 0.6 points, Beta 0.4 points (Alpha Alpha Alpha Beta Beta)
b) Alpha 0.6 points, Beta 0.2 points, Gamma 0.2 points (Alpha Alpha Alpha Beta Gamma)
c) Alpha 0.8 points, Beta 0.2 points (Alpha Alpha Alpha Alpha Beta)

(a) is hardly different from (4), while (b) and (c) diverge more definitely from the single-mention distributions. I'm not seeing the harm -- but neither am I seeing any great advantage for the nominators.

#393 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 12:04 PM:

All: I'm on the road, so will read the latest posts in detail this afternoon. I did want to say, though, that based on the previous arguments, I'm fairly strongly opposed to giving 60 points to be divided up just to avoid fractions. Also, as a result of our discussions in the earlier thread, I'm opposed to having voters rank their choices, which is what the AAABC idea does. The latter is against the principles we worked out for what the nomination process should be -- and I still strongly feel those principles are valid.

Kilo

#394 ::: Kevin Standlee ::: (view all by) ::: May 14, 2015, 12:11 PM:

Business Meeting Administration Request:

If your discussion results in you not being able to settle upon a single proposal, but instead on two variations of the same general process, I suggest you submit them both, in this way:

1. Main Proposal (whatever you perceive as having the most support)

2. Variation submitted as an Amendment by Substitution for the Main Proposal.

It will then be the task of the Preliminary Business Meeting to decide (in effect) whether to consider version 1 or version 2, where version 2 is a complete replacement for version 1.

I recommend this for any cases where you have heavily overlapping proposals that can't easily be reconciled by piecemeal amendments. Also, this gets around a mechanical issue with the WSFS Standing Rules that normally prohibits what are called "second-order" amendments. (I can explain the parliamentary neepery if anyone wants me to do so.)

#395 ::: Cadbury Moose ::: (view all by) ::: May 14, 2015, 12:14 PM:

This moose agrees with Shane at #383 now I've thought about it some more.

Reasons:

1) The nomination process must be nailed down tight, be easy to understand, and seen to be done fairly.

2) There should be no scope for anyone to get trampled on due to voting reruns triggered by a withdrawal.

3) There should be no "human judgement" involved that could possibly bias the result.

So:

1) Open nomination period. (Fandom gets busy nominating favourite stuff)

2) Nominations close.

3) Ballots are collected up, validated and entered into the system.

4) Works nominated are checked for elegibility and correct category, invalid ones are dropped or moved as appropriate.

5) Votes are then processed by SDV-LPE to produce the initial "ranked list", the top five (or whatever) are double-checked for elegibility (just in case), and then frozen as the first cut of nominees.

6) Nominees are notified and asked if they are willing to appear on the final ballot.

7) Once they have all answered, any declined nominations are removed from the pool and the SDV-LPE process rerun to identify potential replacements. The result of that run is matched with the accepted nominations and new nominees that make the 'cut' are merged in and notified (as at 6, above). Repeat as necessary until the nomination pool has the desired number of members.

8) Final check for errors, if required, then announce the nominees, prepare the ballot papers and send off for printing.

9) Do Hugo ballot, as per usual, award rockets.

10) Publish full nomination details (suitably anonymised) but with all the working shown. (i.e. elegibility eliminations, the initial SDV-LPE round, any withdrawn nominations (declines), and subsequent SDV-LPE rounds to select replacement nominees.)

11) Make the (anonymised) ballot data available for independent scrutiny if required. (That way, you've got the data for testing alternative processes in the future if anything needs to be adjusted to handle edge-cases or unforseen problems.)

Does that seem reasonable/make sense?

#396 ::: Aan ::: (view all by) ::: May 14, 2015, 12:18 PM:

J Thomas @ 382: It looks to me like RME is gaining ground, but if it gets a solid majority with a minority who hate it, that isn't good. If we do get a consensus without a whole lot more argument, then great!

Based on the voting so far, I don't think there's actually all that much hate for it. There's a bunch of people, myself included, who prefer options 2 and/or 6 (in my case mostly because I dislike the quite-rare-but-will-happen-at-some-point 7/8-work ballot which can result from RME), but everyone's giving at least some points for RME, with the sole exception of (if I tabulated correctly), Duncan, who was all for letting the admins decide (but that's presumably presuming that the admins _can_ decide, where I also follow Brian's interpretation that they can't).

I don't have any hope that consensus will emerge for 2 or 6 (too many 0 point votes for that), so RME seems like a good-enough middle ground to get behind.

#397 ::: Seth Gordon ::: (view all by) ::: May 14, 2015, 12:19 PM:

Allowing someone to mention a work multiple times on the same ballot, or letting them weight their nomination choices, would enable some interesting strategic-voting possibilities. Imagine a bloc of Enthusiastic Marmots submitting a hundred ballots that say “Alpha, Alpha, Alpha, Beta, Gamma”. Or imagine a voter scratching their head and agonizing over the question “I like Delta better than Epsilon, but do I like it twice as much?”

I would rather keep it simple, conform to existing practice, and make all the choices equally weighted.

#398 ::: colin roald ::: (view all by) ::: May 14, 2015, 12:22 PM:

CHip @102:
[ELIMINATION]: I'm unclear why the bottom two nominees by points should be compared for number of ballots instead of doing a straight elimination.

Comparing by number of ballots actually increases the slate resistance. Suppose we have a year with a slate with ten nominators all proposing the same five works, and fifty other nominators proposing a wide scatter of things. Consider the point when all the no-hopers have been eliminated. The scatter nominators have all probably had several of their proposed works eliminated, so they now only have a few works left on their ballots. Using the system where there are 60 nominating points per nominator, that means each of their proposed works is probably getting 20 or 30 points, while the slate nominators are still giving only 12 points each to all five of their proposed works. So when we reach the point where we're comparing the bottom slate-nominated work against a non-slate work, the slate work has 120 points from 10 nominators, but the non-slate work has 120 points from maybe four nominators, and therefore is preferred.

#399 ::: viktor ::: (view all by) ::: May 14, 2015, 12:34 PM:

391 et al

Yes. Under the current system. Believe me, I understand the current process.

With this proposal, however, all that changes. My ballot will get chopped up into predetermined fractions. This has never happened before. I'm suggesting that if my ballot is to be chopped up, I should be able to decide how fine the chopping. Rule 3.8.4 makes sense under the current process, but not under this proposal.

That's what I'm getting at.

The explanations of Not Done and Really Not Done are unsatisfactory to me.

Irony alert:
Please note that folks right here on this board are being asked to weight preferences on an issue.

#400 ::: colin roald ::: (view all by) ::: May 14, 2015, 12:39 PM:

J Thomas @117
If you want to maximize the chance that one particular work you care about more than anything else will win, then you should just vote for that one.

Actually, no, you can still vote for whatever you want. Suppose you love "The Left Hand of Darkness" and want to be sure it's nominated. But you really love the novelization of the movie version of "John Carter" even more, so you put it on your ballot just in case.

It turns out you were the only person who even considered "John Carter", so it's eliminated first. It's erased from you ballot, and now all of your nominating points go just to "Left Hand of Darkness" as if you'd never had that silly idea about "John Carter" in the first place. The same goes for any work that doesn't end up on the shortlist: go ahead and throw them on your ballot. If other people do too, then great. But if they're not popular enough to make it to the top, no harm done. Your least popular works will be eliminated and all your nominating points roll forward to the proposals you share with the most other people.

#401 ::: albatross ::: (view all by) ::: May 14, 2015, 12:41 PM:

Budda Buck:

The simplest way to get a ranking from SDV-PE is to rank the candidates by how many rounds they survived. If we ran to completion, Alice would have won. If we selected two, we'd have Alice and Bob. If we selected three, we'd have Alice, Bob, Carol, and Dave (because of a tie). And so on.

This falls naturally out of the vote counting process--at the end of each round of counting, you have a list of remaining candidates. It's easy enough to write the program to return that list for all the rounds where you had fewer than (say) 10 candidates.

There are other ways to get rankings (like counting points or number of ballots for the last 10 surviving candidates), but I think they're needlessly complicated.

#402 ::: Brian Gibbons ::: (view all by) ::: May 14, 2015, 12:43 PM:

I think the "Each nominator is given 60 nominating points" structure is the wrong way to describe this, because it almost begs for a discussion of "Well, what if I want to allocate my points non-uniformly?", which I think should be a completely separate discussion. (That's 3.7.1; this discussion is about 3.8.1, and combining the two discussions doesn't add anything.)

Indeed, I think the entire description of a voter having points that they decide to distribute among works is a poor fit. SDV-LPE doesn't change anything about the way a member nominates; it changes the way the final ballot is produced. The description should be coming from that perspective, rather than using "you distribute your vote" and "you divide your vote" language. You aren't doing anything different under SDV-LPE than you were before.

Something like:

* This system does not change the way a member submits their nominations. Assuming no other proposal changes that system, a member continues to simply submit a list of five unranked nominations in each category.

* In order to determine which nominated works appears on the final ballot, the Hugo administrator goes through a multi-round process in which he assigns points to nominees based on how many ballots they appear on and how many nominees on those ballots have not yet been eliminated.

* If, for example, a ballot has five nominees that have not yet been eliminated, the administrator gives each of those nominees a fifth of a point; if a ballot has four nominees that have not yet been eliminated, each of those nominees is given a quarter of a point, and so on, down to ballots in which only one nominee has not yet been eliminated, in which case that nominee gets one point.

* In each round of this process, the administrator adds up the points that each nominee has received that round, compares the two nominees with the least number of points to each other, and eliminates the nominee which appears on the fewer number of ballots.

* The administrator repeats this process again each round until he has eliminated all except the final five (or more, in the case of a tie) nominees. Each round, point totals start afresh and the points a nominee gains from a ballot are based on how many nominees have not yet been eliminated from that ballot. For example, if a ballot has five nominees that have not yet been eliminated, then each of those nominees are given a fifth of a point; when one of those nominees is eliminated, the four surviving nominees will each receive a quarter of a point instead.

#403 ::: Laertes ::: (view all by) ::: May 14, 2015, 01:02 PM:

viktok @399: "I'm suggesting that if my ballot is to be chopped up, I should be able to decide how fine the chopping."

Right. And at 256, I was suggesting that I should be able to decide how fine the chopping is too. I like irrational numbers, so why can't I split my vote into pieces sized 1/pi for Bob and 1-(1/pi) for Alice?

But I'm not finished yet. Because in the event that Alice is eliminated, I'd like for 2/3 of my vote for her to go to Carol instead, while the remaining 1/3 of it goes to Bob. There's this whole complicated thing where I like Carol's work a lot, but I think that if she were on the ballot along with Alice's work that I prefer, she'd draw primarily from Alice's voter base, so I want to nominate only Alice. But should Alice be eliminated, then my objections to Carol being on the ballot vanish with her and I'd like to swing my support to her.

Well, most of it. But some to Bob because I really like Bob's work.

If my votes are to be redistrbuted when one of my picks is eliminated, I should be able to decide how it's redistributed.

But at about that point I realize that you were correct @262, and I'm being silly. These additional levels of detail are fun to describe, but don't really deliver any value besides their inherent beauty and the gratification I get out of promoting them. Having my support be evenly distributed among the works I've chosen to nominate, and reshuffled to remain even when some are struck, isn't a perfect expression of my preferences, but it's simple and does the job.

#404 ::: Laertes ::: (view all by) ::: May 14, 2015, 01:03 PM:

viktor: Sorry about botching your name. I noticed it literally as I hit the POST button, and sat there hoping for a server error. No such luck.

#405 ::: P J Evans ::: (view all by) ::: May 14, 2015, 01:12 PM:

377 et seq

As it's been done for the last mumble years, the admins generate a list of 10 or more (actually, they count all the names in a category). So it should not be necessary to re-run the count.

#406 ::: P J Evans ::: (view all by) ::: May 14, 2015, 01:20 PM:

viktor, based on your comment, I don't think you quite understand the system as it is now. Nominees are not decided based on how enthusiastic the people naming them are, but on how many people name them: that's why there's a rule that says if you put the same thing in a category more than once, it counts as once only. Otherwise, it invites gaming the system, which we don't want.
Think of it as the difference between number of likes, and number of people liking. What we're after is liked-by-the-largest-number-of-people, not has-the-most-likes.

(This isn't the preferential-ballot part of deciding the Hugos: this is the find-out-what's-popular part. No ranking, no strategic voting, only 'what did you think was best from last year'.)

#407 ::: Yarrow ::: (view all by) ::: May 14, 2015, 01:23 PM:

Laertes @ 404: Having my support be evenly distributed among the works I've chosen to nominate, and reshuffled to remain even when some are struck, isn't a perfect expression of my preferences, but it's simple and does the job.

Hear hear!

#408 ::: junego ::: (view all by) ::: May 14, 2015, 01:24 PM:

Jameson Quinn @ 380

1. Remove the 5% cutoff.

This is already on the meeting agenda. We don't need to address it.

2. Remove the 5-per-category-per-ballot limit.

This requires a separate constitutional amendments and should be a separate discussion/proposal. Our only business might be to include language in our proposal that allows for this change.

3. Allow multiple votes for the same work, so people can tune how their points are distributed.

Again, this requires another constitutional amendment. My sense of previous discussion and my own opinion is that the nominations shouldn't be weighted.

#409 ::: J Thomas ::: (view all by) ::: May 14, 2015, 01:28 PM:

@400 colin roald

J Thomas @117
"If you want to maximize the chance that one particular work you care about more than anything else will win, then you should just vote for that one."

Actually, no, you can still vote for whatever you want.

....

Your least popular works will be eliminated and all your nominating points roll forward to the proposals you share with the most other people.

That's how it ought to be, and that's mostly how it is.

But there's a possibility it will result in the one you like best being eliminated.

It goes like this: Unknown to you, a bunch of people have bullet-voted for the one you like less. And a bunch of people have voted for the one you like better along with popular works. You voted for both of them, so your vote counts for them both equally. And unknown to you, something that is more popular than either of them also has a bunch of shared votes.

The one you like better gets chosen early to consider for elimination. So does the very popular one. It accidentally turns out that the extremely popular one gets matched up against the one you like, and eliminates it. Your other choice becomes a finalist.

Again, this is possible but unlikely. More likely when the one you want more is the less popular one than when it's the popular one. Also, there's a chance that when you give half a point to the more popular choice you are in fact *raising* its score and making it less likely to be matched against your favorite.

But in almost any unranked system, adding a vote for something else reduces the chance for your favorite by some small amount. If you hate for your second choice to win and not your first choice, don't vote for the second choice. It isn't good enough for you.

It's completely safe for you to vote for it, though, if you are completely sure that nobody else will vote for it.

#410 ::: Duncan J Macdonald ::: (view all by) ::: May 14, 2015, 01:56 PM:

Aan @396:

Duncan also voted 10 on "Create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide."

I'm more partial to "promote 6" than I am to "run it again".

The first run-through produces an ordered list of 1 - n. If there are withdrawals or late noticed ineligibilities, push those out of the list, apply pressure to the bottom, and squeeze the spaces out. Contact any new members of the top five club, and iterate as necessary until top five remain unchanged.

#411 ::: Annie Y ::: (view all by) ::: May 14, 2015, 01:56 PM:

@350 J Thomas
This is a minor picky point, but the issue is not how many ballots there are, but how many works that people might nominate.

Yes and no. My point is that with 100 ballots, the lack of the under 5% restriction will put on the ballot works with 4 nominations (or even 1 in slow years or categories). You increase the number of ballots, your 5% are a lot higher (50 votes on 1000). And you probably get more works nominated - so chances of someone getting 1 nomination and making it on the ballot is very small. So you do not really need the artificial protection. (this had always been my reading on why we have the rule)

#412 ::: junego ::: (view all by) ::: May 14, 2015, 02:01 PM:

Brian Gibbons @ 402
I think the "Each nominator is given 60 nominating points" structure is the wrong way to describe this, because it almost begs for a discussion of "Well, what if I want to allocate my points non-uniformly?", which I think should be a completely separate discussion. (That's 3.7.1; this discussion is about 3.8.1, and combining the two discussions doesn't add anything.)

I'm the one who originally proposed this idea (except with 100 pts). I later unproposed it after this and another objection were raised (although I will note that only having one point hasn't squashed the 'discussion' you describe...see viktor). I think colin is the only one still backing the proposal. Unless there's a sudden groundswell of support, I consider it doa.

Indeed, I think the entire description of a voter having points that they decide to distribute among works is a poor fit. SDV-LPE doesn't change anything about the way a member nominates; it changes the way the final ballot is produced. The description should be coming from that perspective, rather than using "you distribute your vote" and "you divide your vote" language. You aren't doing anything different under SDV-LPE than you were before.

YES! Excellent point. I totally agree and have stated similar upthread. I also think we need to stop slipping into "vote" language. We are polling/suggesting/nominating. (The constitution doesn't use "vote" wrt nominations and does use "vote" for the final, well, vote.)

#413 ::: Pfusand ::: (view all by) ::: May 14, 2015, 02:37 PM:

Why have the Hugo administrator wait until a withdrawal or disqualification before rerunning the election?

Run the election, and record the top six (or seven) finishers. Eliminate one of the top five finishers and rerun. Note the new top five (or six (for ties)) finishers. Repeat for each of the other top five. Are there any surprises? No? Then we're done.

Yes? Then there is the work to immediately check for validity, and the individual to call first to see if tey accept the nomination. Should this person be told that this is a linchpin nomination? Oh, what a good question.

#414 ::: colin roald ::: (view all by) ::: May 14, 2015, 02:59 PM:

J Thomas @409:
But in almost any unranked system, adding a vote for something else reduces the chance for your favorite by some small amount. If you hate for your second choice to win and not your first choice, don't vote for the second choice. It isn't good enough for you.

Okay, yes. That's true of the existing system, and it's also true of the LPE system. I don't see any scenario along these lines where LPE makes anything worse than it would have been under the current nomination system.

I have trouble thinking of scenarios where I'd be truly tempted to write down a second work -- call it Clifford -- in addition to my first choice -- call it Algis -- if I thought I'd actually be unhappy to see Clifford on the Hugo ballot. We're talking about the nomination process, right? Most nominations are a gamble, and the amount you can actually influence things by nominating only one work instead of five is pretty modest.

On the other hand, if I wanted to nominate five works that all seemed like longshots, and I didn't know which of the five had the best chance ahead of time, I'd definitely prefer a LPE ballot to a conventional one. Under LPE I could put down all five longshots, and my preferences would automatically roll forward to the one where it does the most good.

#415 ::: Darth Paradox ::: (view all by) ::: May 14, 2015, 03:08 PM:

[DECLINED NOMINATIONS]

[2] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[3] 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[2] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[10] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[5] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[5] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[9] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[5] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[2] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#416 ::: junego ::: (view all by) ::: May 14, 2015, 03:46 PM:

Pfusand @ 413
Why have the Hugo administrator wait until a withdrawal or disqualification before rerunning the election?
Run the election, and record the top six (or seven) finishers. Eliminate one of the top five finishers and rerun. Note the new top five (or six (for ties)) finishers. Repeat for each of the other top five. Are there any surprises? No? Then we're done.

This is a good procedural idea that the administrators could use and may very well prevent some potential grief.

The problem comes in when changes are made *after* the finalists have been notified. What if someone changes their mind after accepting and after the finalists are made public? What if a work is disqualified because of an issue not known or caught in time? What about the Game of Thrones example above?

Those are the cases we're trying to set rules for. The new system, unlike the old, has the property that such withdrawals/eliminations could change who the remaining finalists are and unnominating a finalist after notification because of a tally rerun would be all kinds of not good.


#417 ::: junego ::: (view all by) ::: May 14, 2015, 03:57 PM:

And I forgot an important point, what if more than 1 finalst in a category is removed?

#418 ::: Doire ::: (view all by) ::: May 14, 2015, 04:18 PM:

junego @ 417
Instead of extra 5 runs it's 31 (I think). Would that be a combinatorial function? Too, too many years ago

#419 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 04:46 PM:

viktor@399:
Irony alert: Please note that folks right here on this board are being asked to weight preferences on an issue.

Actually, no irony at all. In fact, it's a good illustration of what we have been talking about. Everyone threw out "nominations" for how they want the issue to be handled. Once all the nominations are on the final ballot -- unranked, I might add -- then we weight our preferences in the final vote. This is precisely how the Hugos are supposed to work and why we want to maintain that under SDV-LPE.

Kilo

#420 ::: colin roald ::: (view all by) ::: May 14, 2015, 04:54 PM:

junego@412:
I think colin is the only one still backing the proposal.

"Backing it" might be a little strong. I still think there's an argument in favour of being able to show people math done with whole numbers, but on the flip side there's the "60? why 60?" reaction that will sometimes need explaining.

In practice, all the arithmetic is going to be done by computers so it hardly matters.

#421 ::: Buddha Buck ::: (view all by) ::: May 14, 2015, 04:57 PM:

Doire @418: Technically it would be a combinatorial function/explosion, but with a small enough N that it's feasible. It's a different matter when you consider all the ways the 15 works on the short-list can decline nominations (32,767 possibilities).

#422 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 05:01 PM:

I've made a tally spreadsheet for the poll. I'm sure I've missed a supplementary vote or two but you can begin to see patterns.

https://docs.google.com/spreadsheets/d/1tZ9eSJ74TT_2A2bMVbuCoNwKJVS3QeV5ShCAap4ipCw/edit?usp=sharing

The current highest average is for RME. But: several people have not voted on the last three options (the ones regarding multiple proposals to be decided on at the BM). One thing that interests me is that the minimax of the last three is 5, so everybody seems to find at least one of those proposals at least minimally acceptable; but the maximin is 1, so each of those three options has at least one person who detests it. This is strange to me, as those three options seem to me to be broadly similar. It also suggests to me that if we could combine the advantages of those three and avoid their disadvantages, we could get consensus on an option in that direction.

#423 ::: Annie Y ::: (view all by) ::: May 14, 2015, 05:04 PM:

#422 ::: Jameson Quinn

OK - I got lost in what the current options are (you all talking too much when I am trying to sleep and then I need to work instead of reading what happened...) :)

In which post are the current options?

#424 ::: Shane ::: (view all by) ::: May 14, 2015, 05:04 PM:

#399:::viktor:::

No, I'm pretty sure you are incorrect. I believe some of the wording has misled you (in which case good, lets clean it up).

The proposal in this thread does not involve weighting of the works within a ballot.

There is a discussion here of weighting but it's one you started. I mildly favour it, btw.

#425 ::: Soon Lee ::: (view all by) ::: May 14, 2015, 05:15 PM:

[DECLINED NOMINATIONS SUPPLEMENTARY]

[7]8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[2]9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[2]10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default

#426 ::: Shane ::: (view all by) ::: May 14, 2015, 05:16 PM:

#420:::Colin Roald:::

1 point vs 60 points

Seems to me the 60 might be useful for explaining the system to some folks, and not needed anywhere else "Ok, if that's not making sense, let's say we counted each ballot as sixty points instead of one. ... [explanation with 60]... It works out the same either way. The designers decided to go with 1 point." To be clear, this would not be in the documentation anywhere, just use it if it helps you in discussion.

#427 ::: Shane ::: (view all by) ::: May 14, 2015, 05:17 PM:

[FAQ / EXPLANATION]
Per colin roald @414 - One of the things that delights me about this system is it gives your point wherever it can do the most good, i.e. to the least popular of your proposed works which your support can actually help. Colin's phrasing might be used verbatim:

"If I wanted to nominate five works that all seemed like longshots, and I didn't know which of the five had the best chance ahead of time, I'd definitely prefer a LPE ballot to a conventional one. Under LPE I could put down all five longshots, and my preferences would automatically roll forward to the one where it does the most good."

#428 ::: P J Evans ::: (view all by) ::: May 14, 2015, 05:21 PM:

I am tempted, when Frisbie can get to his LSI-11 and pull the stuff off the RX01s (and possibly the RK05s), to offer the 1984 version of the program that counted the final ballots.
(RX01: 8-inch floppy, holding slightly less than 256K; RK05: 5 or 10 MB hard disk - the 10MB version is the RK05F, where it's fixed, not removable.)

#429 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 05:46 PM:

Jameson@422:

I had just finished tallying results when your message came through. I'm getting the same totals you are, so I think we've covered everyone. However, you're right that few people expressed an opinion on the last three options (since they were in the original poll, most people likely missed them). For reference, they are:

8. Re-run the counting until there is only one finalist who was not on the original list, and add that one
9. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
10. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.


I believe 5 (RME) and 8 are functionally equivalent, if I'm interpreting 8 correctly.

9 and 10 are -almost- functionally equivalent, I think. Am I correct that the only difference is that 10 is the same as 1, just with a plan to have two more proposals afterwards to deal with the issue? And then 9 is the same thing, but we leave nothing to the adminstrators and just make two nearly-identical versions of the SDV-LPE proposal that includes the two major declined nomination schemes?

If we can agree on a scheme, then I don't think we need worry about 9 or 10. From Kevin's messages, I believe (correct me if I'm wrong, Kevin) that he would strongly prefer 10 to 9 -- separate out as much of the contentious issues as possible.

Kilo

#430 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 05:50 PM:

[DECLINED NOMINATIONS]

All:

Warning: Long philosophical post ahead...

It looks to me like 5 (RME) has the most broad support, with the fewest number of people hating the idea. 1 (let the admins decide) has the next highest support, and also has only a few people who hate the idea. 2 (add the works from the previous round) is nearly tied with 1, but even more people hate the idea.

1 requires us to do nothing, but if we were to go that route, I feel strongly that we should actually go for 10 -- do nothing in the SDV-LPE proposal, but have the two leading schemes (which would actually be 2 and 5) ready for voting.

I think this boils us down to three real possiblities:

2 - Add the work(s) eliminated in the previous round back to the final ballot.

5 - Re-run SDV-LPE without the eliminated work and add any new finalists to the finalists remaining from the first round.

10 - We can't decide. Do nothing, craft a proposal for both 2 and 5, and let the business meeting decide.


Pros and Cons:
In reality, the odds of eliminating someone who used to be on the final ballot as a result of a re-run are very, very small. In fact the odds are extremely high that 2 and 5 will produce identical results. So the stakes aren't very high here. We therefore need to consider two factors: general philosophy and fringe cases.

2 has the advantage of being very simply explained. It's what we do now. Pretty much everyone would accept 2 on face value. 2 says that just because the fans want something on the ballot doesn't mean that the author doesn't have a say -- he can veto the will of the fans, and they have to accept that veto without creating an "alternate history".

However, there is a good question as to whether this is truly fair. 5 maintains that the most fair way is to run SDV-LPE as though we knew that the declined nomination was essentially ineligible. If re-running SDV-LPE somehow would have swapped an original finalist for a new one, that seems odd and counter-intuitive, but only if you try to hold with both "alternate histories" at the same time. The power of 5 is that it neatly blends those two "alternate histories" in such a way that no one from the "original history" is denied, but the results from the "re-run history" are also recognized.

The downside to 5 is that you get results that seem counter-intuitive at a surface level, while with 2 you do not. The downside to 2 is that it is possible that works that might have made the Hugo might have been eliminated before the next-to-last round, and so won't be considered.

The disagreement, it seems to me, therefore boils down to whether you recognize the validity of "alternate histories" or not. You can definitely make a strong case (as 5 does) that the true history is the re-run ballot, since it was run with only eligible works. You can also make a strong case that because 2 does not cause any "cognitive dissonance", then it is the true history that needs to be respected.

So, to boil it down even further, it seems to me that:

2 has the advantage of simplicity, at the expense of compromise between/blending of the two histories.
5 has the advantage of compromise between/belnding of the two histories at the expense of simplicity.

Given the above, which do we value more? Blending the two histories or simplicity?

Thanks,
Kilo

#431 ::: P J Evans ::: (view all by) ::: May 14, 2015, 06:02 PM:

430
I prefer simplicity - it's easier to explain than rerunning the counting.
(and, especially if you're using a computer to count, there's no reason to stop when you get to five or six. Count until you hit the 5% limit, or run out of stuff to count, whichever comes first.)

#432 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 06:10 PM:

So the options are 6th place, RME, or punt. My meta-feeling is that if there are 2 or more strong votes against 6th place or RME, we should punt. I like RME better aesthetically, but am worried that it might make people wrinkle their noses, and stop them from passing anything. But if that's the only reason to prefer 6th place over RME, then punting is the worst of both worlds.

So I guess if I could get everyone here to agree, I'd prefer 6th place, under the theory that 1% probabilities that the BM doesn't understand won't hurt them. But that's not happening. Given that fact, I hope we can get consensus for RME; if not, we have to punt.

It looks as if Duncan J Macdonald, Laertes, and Cally Soukop are the ones who oppose RME. Given that it seems unlikely that we'll get consensus for anything but RME or punt, and if you oppose RME for reasons of complexity you may oppose punting even more, are any of you willing to rethink your stance on RME?

#433 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 06:13 PM:

Note to Duncan J Macdonald, Laertes, and Cally Soukop: even if you don't want to change your mind, a response would be helpful.

Note to Brian Z: if your goal here is to throw sand in the gears, you can make that pretty obvious right now by voting against RME.

#434 ::: Evil Rob ::: (view all by) ::: May 14, 2015, 06:28 PM:

While it may certainly be that I've missed it - I would think you would want to use separate methods for a declined nomination, and a disqualification.

@233 I would agree that bumping the last eliminated up in the case of a declined nomination makes sense.

In the case of a disqualification, I would think that RME is a better plan, since to at least avoid the appearance of impropriety, the work should never have been counted in the first place.

#435 ::: Buddha Buck ::: (view all by) ::: May 14, 2015, 06:38 PM:

[DECLINED NOMINATION]

Keith @430: I'm not sure it's correct to say that extremely unlikely that 2 and 5 will give different results. It's been tested that if one work drops out, the far most likely result is that the re-run will retain the other four works, with one new one. I don't know if it's been checked to see if, in that case, the new one was the last-eliminated in the original run. It could be that 'The Naked Now' was the last eliminated in the first run (and would thus get nominated under 2), but without the declined work, it would be eliminated earlier, leaving 'The Naked Time' as the replacement work. From the point of view of the problem of declined nominations, I don't see this as a problem; one declined/ineligible work was replaced by an eligible work not previously nominated. But it would be a difference between the two options.

If we are concerned about fidelity to the "true history", based solely on eligible works, we'd have picked the "rerun and accept new results" option (4), but it seems like we rejected that because of the logistical nightmare of recalling an accepted finalist slot.

Given the idea of "two histories" (one with "Spock's Brain" accepting the finalist slot, one with "Spock's Brain" declining the finalist slot), we (or at least I) don't want to (a) tell folks that they were a finalist, but now they are not, or (b) have to explain why "The Corbomite Maneuver" was not nominated when, without "Spock's Brain", it "should" have been. It seems more fair (to me, at least) to give a spot to those who would have earned it under either of the "two histories".

The only draw-back I can see is a potential unboundedness coming from multiple declined ballots. "Spock's Brain" turns it down; "The Corbomite Maneuver" and "The Tholian Web" get added, for a 6 work ballot. The Tholians decline the slot, and re-running again yields two more new finalists, for a total of three merged histories and eight finalists -- unless one of those two decline, etc. The chances are exponentially[1] low that it'll happen too many times.

[1] There's a 1/100 chance that eliminating a nominee will add multiple new nominees, and a low chance that the new nominees will decline, yielding a factor of 1/500 (or less) that another rerun will be required. So the chance of N reruns is on the order of (1/500)^N, so, yeah, exponentially low.

#436 ::: CHip ::: (view all by) ::: May 14, 2015, 06:44 PM:

[DECLINED NOMINATIONS]

my votes on handling, based on @325 and @361

option 2 (take last-eliminated): 10
option 5: 10
options 6-7: 5
(various patches to prevent RME from causing somebody to be tossed after we thought they were on)
option 9 (base proposal with supplements for handling declined nominations): 10
all others 0.

#437 ::: Joshua Kronengold ::: (view all by) ::: May 14, 2015, 06:45 PM:

[10] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[8] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[4] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary

6, and 7 are simply wrong, they should be:
[3] 42. Rerun the election without all eliminated nominations to completion (until no candidates remain). Add the last eliminated candidate(s) that did not originally make the ballot.

Regarding the procedural questions, Kevin is [obviously] right that if we have fundamentally two different amendments, we should do the amendment by substitution thing.

For additional supplementary amendments (that add but do not remove anything from the main proposal), we can simply have additional amendments to the main proposal, I believe.

For instance, we could have the main proposal without a way of handling withdrawals, and then an amendment that proposes that if any candidate becomes ineligible after nominee has been notified in that category resulting in four or fewer candidates, that the last eliminated candidate may be added.

And another competing amendment stating that instead, in that case, we should act as in 2 or 42 [42 has less chance than 2 of resulting in too many things on the ballot, but also more chance of one constituency being double-represented by a quirk of fate].

#438 ::: JJ ::: (view all by) ::: May 14, 2015, 06:45 PM:

I've not posted here yet, because I've been reading and thinking through what everyone else has to say.

I would be inclined to agree with Evil Rob at #434. I think RME is really needed in the case of a DQ (especially in the case of 2 items DQ'ed from the same category). I don't think the 6th Place method would be satisfactory in such a case. (I suspect, if we can persuade Sasquan to provide anonymized ballot data, we'll see a serious case of this example this year with Andy Weir's The Martian. I would be very curious to see how RME would affect those ballot results.)

In the case of a Declination, I could be persuaded to go with either 6th Place or RME -- but I think I prefer RME.

#439 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 06:56 PM:

[DECLINED NOMINATIONS]

Jameson's post reminded me that I didn't give any time to option 10 (leave the issue out of the main proposal and craft two separate proposals -- or amendments, referencing Joshua@437 -- for 2 and 5). Apologies for that; I'm trying to give an unbiased (as much as a human can be) summary of our options.

10 has a single, but powerful advantage: If there is disagreement here among those who are working on the system, then there is very likely to be a roughly equal amount of disagreement at the business meeting. If either 2 or 5 is encoded explicitly in the proposal, then there is a small chance that opposition to whichever was included would be enough to sink the whole the proposal.

Alternatively, 10 allows us to encode both options separately. When presenting the amendments, we can even state that a majority of the developers feel that [2 or 5] is the best way to go, but here are the options, and we will let you choose the one you're most comfortable with.

This option isn't quite a punt (as Jameson puts it), since it does allow that developer preference to be expressed, but it maintains the ability to get the main system passed safely.

Kilo

#440 ::: Joshua Kronengold ::: (view all by) ::: May 14, 2015, 07:03 PM:

FWIW, the meeting itself isn't shy about amending proposals, so some disagreement could be handled that way in any case regardless of what we propose.

By the same token, it's worth noting that -new- business is particularly hard (2/3 majority) to add, (but amendments are easy).

#441 ::: Brian Gibbons ::: (view all by) ::: May 14, 2015, 07:07 PM:

Keith "Kilo" Watt@430: 2 has the advantage of being very simply explained. It's what we do now.

Sure, it's what we do now in the simple first-past-the-post nomination process in which "nominee that would have been on the ballot if nominations for the ineligible nominee weren't considered" and "last nominee eliminated before the final five" are one and the same.

It's not what we do on the final ballot. After listing first place, the administrator then goes and (for reasons of completeness and, I suppose, to satisfy people's curiosity) runs the analysis again for each position. Second place isn't "last nominee eliminated before first place", it's "nominee that would have been in first place if we ignore all votes for the actual winner". Third place is the nominee that would have won if all votes for the first two are ignored, and so on.

If there's any group that can handle the concept of alternate histories, I think it's SF fandom.

I am surprised that Options 2 and 5 yield identical results. I would have assumed that, with highly-correlated data, you would end up with the exact opposite. For example, if the most popular Dr. Who episode nominated is ineligible for whatever reason, my assumption is that rerunning the analysis would cause the second-most popular Dr. Who episode to pop up as a nominee, even though that probably would have been eliminated earlier than the final elimination in the first run (due to it mostly appearing on ballots that also had the most popular one).

In any event, I don't believe the admins have the discretion that you believe they have, so if you are going to go with a "just leave it out" concept, I'd suggest making sure that the default is as you believe it to be, if Kevin Standlee or someone else is willing to opine about hypotheticals.

#442 ::: junego ::: (view all by) ::: May 14, 2015, 07:09 PM:

Keith "Kilo" Watt @ 430
2 - Add the work(s) eliminated in the previous round back to the final ballot.
.
5 - Re-run SDV-LPE without the eliminated work and add any new finalists to the finalists remaining from the first round.

I vote for 5.

#443 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 07:17 PM:

Brian G@441

In any event, I don't believe the admins have the discretion that you believe they have, so if you are going to go with a "just leave it out" concept, I'd suggest making sure that the default is as you believe it to be, if Kevin Standlee or someone else is willing to opine about hypotheticals.

Actually, I think that option was replaced by option 10 (not in the main proposal, but with amendments describing each scheme ready to go), so I think the question of how much discretion the admins have is now pretty much moot.

K

#444 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 07:22 PM:

Joshua@440:

A quick procedural question for you...

Which is easier to do, procedure-wise:
- Include one of the two schemes in the main proposal (RME, I'd think, based on the poll) and have an amendment for 6th place ready to discuss if necessary, or,
- Leave the schemes out of the proposal and have an amendment for both ready to dicuss.

THanks,
Kilo

#445 ::: Vicki ::: (view all by) ::: May 14, 2015, 07:27 PM:

Pfusand @413:

Realistically, the administrators will need to give the potential nominees some time to answer the question “do you accept the nomination?” Never mind genuine indecision, someone might not answer the phone immediately, and in fact might take several days to even check their messages before calling back.

Yes, you can set the computers to calculating what-ifs in those days, but any proposal should allow for reasonable turnaround time for accepting/declining a nomination.

#446 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 07:48 PM:

If RME is the one where we rerrun and merge the new results without kicking anybody who was previously nominated off, I'm actually for it. If it's not the rerun/merge one, umm, which is it exactly?
The one I'm against is "rerun and un-nominate the previously nominated", because that's a very cruel thing to do to people who had been notified they were nominated and had accepted.

It's quite possible I messed up and marked the wrong line on my "ballot"; if that's the case, I apologize for the confusion I've caused.

#447 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 08:04 PM:

Hi Cally -

Yep, sounds like you're in favor of RME (option 5 in the original list).

Thanks!
K

#448 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 08:05 PM:

Hi Cally -

Yep, sounds like you're in favor of RME (option 5 in the original list).

Thanks!
K

#449 ::: felice ::: (view all by) ::: May 14, 2015, 08:06 PM:

Cally Soukup @446: "It's quite possible I messed up and marked the wrong line on my "ballot"; if that's the case, I apologize for the confusion I've caused."

No, it's not your mistake - the spreadsheet at https://docs.google.com/spreadsheets/d/1tZ9eSJ74TT_2A2bMVbuCoNwKJVS3QeV5ShCAap4ipCw/edit?usp=sharing doesn't have the correct values from your post #374, and needs to be updated.

#450 ::: Cally Soukup ::: (view all by) ::: May 14, 2015, 08:17 PM:

felice: Ahh, thank you. I probably should have tried to find my previous post, but had to run off to deal with dinner. In any case, no harm, no foul!

#451 ::: felice ::: (view all by) ::: May 14, 2015, 08:30 PM:

Keith "Kilo" Watt @439: "10 has a single, but powerful advantage: If there is disagreement here among those who are working on the system, then there is very likely to be a roughly equal amount of disagreement at the business meeting. If either 2 or 5 is encoded explicitly in the proposal, then there is a small chance that opposition to whichever was included would be enough to sink the whole the proposal."

There is the other question of whether the people who oppose RME actually support SDV-LPE at all - participating in this discussion doesn't necessarily mean they'd vote in favour of it, irrespective of what we decide on handling withdrawals.

I'm reluctant to give the Business Meeting any more decisions to make than we have to; it's likely already going to be a nightmare with all the different puppy-inspired proposals, and passing off something too controversial for us to reach agreement on is unlikely to go down well. "This is SDV-LPE: it's good, vote for it" is going to have more chance of success than "Here are a couple of variants on SDV-LPE: which do you think is better?"


Brian Gibbons @441: "I am surprised that Options 2 and 5 yield identical results. I would have assumed that, with highly-correlated data, you would end up with the exact opposite. For example, if the most popular Dr. Who episode nominated is ineligible for whatever reason, my assumption is that rerunning the analysis would cause the second-most popular Dr. Who episode to pop up as a nominee, even though that probably would have been eliminated earlier than the final elimination in the first run (due to it mostly appearing on ballots that also had the most popular one)."

I think you're right; but the testing has been done with 1984 data, when the field was rather smaller and there weren't any awards given for TV episodes (technically they were eligible, but in the same category as movies). I suspect this would be a bigger issue with modern nominations, with fandom more divided into subgroups because it's just not possible to keep up with everything. RME and 6th place will still sometimes (maybe even often) get the same result, but I'd expect them to be different more often.

#452 ::: Annie Y ::: (view all by) ::: May 14, 2015, 09:03 PM:

[DECLINED NOMINATIONS]

After finally catching up with the reading

[8] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[8] 2. Add the works elininated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[5] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[0] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[10] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[10] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[10] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#453 ::: Laertes ::: (view all by) ::: May 14, 2015, 09:47 PM:

Jameson Quinn @433: I dislike RME on the grounds that it's complex--it's much simpler to lock down the finishing order and then just move down the list.

When, for instance, a runner in a foot race is later disqualified for some reason, they simply shuffle the other finishers up, this despite the fact that runners interact with one another during the race in complicated ways, and the results with the disqualified runner struck from the final standings isn't necessarily the same as the result would have been had the race been run without her.

But I (think I?) scored RME a 4, by which I meant to say that it's far from my favorite, but it's not a disaster like the 0s.

Short version: If a consensus is forming around RME, please consider me to be a fully paid-up member of that consensus.

#454 ::: Laertes ::: (view all by) ::: May 14, 2015, 09:55 PM:

Also? I tuned in to this effort very late in the game. The heavy lifting of hammering out this really terrific proposal was done entirely without me. If I manage to contribute something that's helpful, great. If not, please ignore me, and under no circumstances allow me to be any kind of impediment.

#455 ::: Laertes ::: (view all by) ::: May 14, 2015, 10:24 PM:

If we could all pretend that we didn't see that dreadful mixed metaphor in @454, I'd be eternally grateful. I'll be over here at the blackboard, writing "the Fascist octopus has sung its swan song, the jackboot is thrown into the melting pot" 500 times.

#456 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 10:57 PM:

felice@451:
There is the other question of whether the people who oppose RME actually support SDV-LPE at all

Of course. But I haven't really seen much objection from anyone, so I have faith that no one is intentionally being a bad actor here.


I'm reluctant to give the Business Meeting any more decisions to make than we have to

That's what the business meeting is for. I don't think they need to be protected from making decisions, so I'm not terribly worried about that.


I suspect this would be a bigger issue with modern nominations

I'm actually going to stand by my estimate. I ran well over 50 simulations using both the 1984 data and the 2013 data that Cheradenine modelled from the data that was published. I was only able to generate one situation in which the 6th place work would not have been on the ballot if the sim was re-run without a declined nomination. That case seemed to be exclusively when there was a slate present and it was one of the slate works that was declined. In that case, another slate work took its place instead of the 6th place work from the original run. You may have a point about TV shows, however -- if, for example, Dr. Who fans are going to nominate all Dr. Who episodes, then that will be a similar situation.


However, out of all of these sims that I ran, in no case did any of the other four original finalists change, so that's certainly good news.


My analysis is still that it doesn't matter much -- but much is not zero, and that's why I have no problem with either method, so long as we can come up with a clear and non-technical explanation for whichever one (and I think that we can).


===========
You may have noticed that I intentionally haven't voted to this point. That's because I don't think it matters much, and I'm much more interested in seeing what kinds of debates might come up at the business meeting. Subject to what Joshua/Kevin say about the procedural requirements for passing amendments, I think I have enough data to form an opinion and a suggested course:

What is the group's view on including RME in the SDV-LPE proposal and having an amendment ready to replace it with 6th place if there is significant resistance to RME? In my view, again, subject to procedural constraints, this is the smart play that will satisfy almost everyone (albeit to lesser and greater degrees).

Opinions?

Kilo

#457 ::: Keith "Kilo" Watt ::: (view all by) ::: May 14, 2015, 11:10 PM:

A quick P.S. to my #456:

In the one case where the two schemes make a difference (when a slate has its work's nomination declined), if 6th place is used, then slate loses its finalist slot to a non-slate work. If RME is used, it maintains its slot, just with a different work from the slate.

This fact alone is enough to make me lean towards RME over 6th place, even though I'm still not sure how I'd "sell" RME at the business meeting yet. Possibly with that argument right there, I don't know.

K

#458 ::: P J Evans ::: (view all by) ::: May 14, 2015, 11:16 PM:

456
That will satisfy me - I still prefer taking the next one on the list, but if it isn't usually going to matter, then it isn't that important a difference.

#459 ::: Cassy B. ::: (view all by) ::: May 14, 2015, 11:17 PM:

@456, What is the group's view on including RME in the SDV-LPE proposal and having an amendment ready to replace it with 6th place if there is significant resistance to RME?

I like either version being included, with an amendment ready to replace it with the other. I honestly don't know which would fly better at the Business Meeting, so I don't have an opinion on which should be in the original proposal and which should be the amendment... but, speaking personally, I prefer RME slightly over 6th place. For what it's worth.

#460 ::: Annie Y ::: (view all by) ::: May 14, 2015, 11:32 PM:

@456

Sounds like a good plan. I prefer #6 to RME but can support either and as it sounds like there is no much difference anyway, it does not make such a big difference anyway.

#461 ::: Jameson Quinn ::: (view all by) ::: May 14, 2015, 11:38 PM:

I think we should proceed with Keith's idea. An RME proposal, with an amendment ready to use 6th place. If my parliamentary understanding is correct, that would mean there would first be a vote to see who prefers 6th place over RME, then in the vote on the full proposal, it would include whichever one has more support. I think this would work well. Any actual admins who spoke in favor of either one would probably sway it, which is how it should be.

If that is the answer, then all that remains is to finalize language. I agree with the suggestion to remove the "you"s from the counting procedure. I also think that the two stages should have names (such as "matchup" and "showdown") and the numbers being counted should also have names ("points" and "approvals"; not "votes" or "nominations" in either case, because those are confusing). In fact, it's probably better to avoid "nominate" and "nominee" and all related words altogether, and stick to "finalist". I have various ideas for the FAQ too, but that can come later.

Is it time to go back to the google doc? I made the text size large; Keith, see if it works for you like that. I agree that we should be posting versions here, but it is probably good to work on it there...

#462 ::: junego ::: (view all by) ::: May 15, 2015, 12:03 AM:

I'm good with Kilo's proposal to present RME with the amendment and have #6 as the alternate.

Jameson - I had some trouble with my iPad and the Google doc, so I'll definitely observe but if it keeps freezing on me I may come back here to comment. Will try not to make it harder on you guys.

You've all done a tremendous job. Thank you, again.

#463 ::: felice ::: (view all by) ::: May 15, 2015, 12:05 AM:

Keith "Kilo" Watt @456: "Of course. But I haven't really seen much objection from anyone, so I have faith that no one is intentionally being a bad actor here."

I'm not accusing anyone of being a bad actor; just that taking an interest in the SDV-LPE proposal isn't necessarily the same as liking it more than the current system (it could pass whether they like it or not, after all). Is there anyone participating in this discussion who doesn't like SDV-LPE?

"I'm actually going to stand by my estimate. I ran well over 50 simulations using both the 1984 data and the 2013 data that Cheradenine modelled from the data that was published"

We don't have any information on how nominations for different works actually overlap between ballots for 2013, though, do we? Does Cheradenine's model account for cases like "what happens if "2312" is withdrawn, and everyone who nominated "Existence" also nominated "2312" and nothing else in the top 10, while everyone who nominated any of the other top 10 works nominated more than one? Or for short story, if we ignore the 5% threshold, I'd bet if "No Place Like Home" was withdrawn, RME would replace it with "One Hell of a Ride" while 6th place would replace it with "Robot".

[TIES]
Unless simultaneous elimination takes "One Hell of a Ride" out at the same time as "We Will Not Be Undersold!" (two Seanan McGuire shorts with the same number of nominations).

#464 ::: Vicki ::: (view all by) ::: May 15, 2015, 01:18 AM:

The business meeting doesn't have to be protected from making decisions. It does have finite time (realistically speaking, and given that everyone needs sleep, and nobody wants to spend their entire Worldcon on the business meeting). So it helps to have a coherent decision to make: "should we do this thing?" rather than "what would this thing do anyway?" My impression is that people here are doing a fair job of that.

#465 ::: Seth ::: (view all by) ::: May 15, 2015, 02:04 AM:

#400 Not quite. Putting my second choice on my ballot can cause my first choice to be eliminated: when it's down to 6, my first choice is #5, and loses to #6. But if I hadn't put that second choice on, my first choice would have had an extra half point, making it #4, and something else would have been eliminated.

#466 ::: Joshua Kronengold ::: (view all by) ::: May 15, 2015, 02:27 AM:

The reason I still support #6 over RME (or highest_new(RME), for that matter) is, in fact, my intuition that it will frequently not make a difference [that even the pairs to a winning entry will generally be high enough ranked that they have a good chance of either being in the final ballot or making it to #6], combined with the complexity. But it's very much not a deal-breaker, and while there are reasons to support hn(RME), RME is clearly fairer [if also more likely to result in an extended ballot].

I've implied this before, but to be clear, I've attended a bunch of BMs, made successful floor amendments, and drafted a couple of proposals including one that made it (with substantial amendments) to passage. Kevin is an acknowledged expert at this, with several times my experience just in BM time alone, regularly chairs WSFS committees, and is also a several time chairman of the WSFS business meeting itself (including this year). The gulf between our experience could hardly be larger.

That said, yes, I think if we think there's an optimal and dominant way to handle ties, we should include it in the motion -- with, if we see substantial room for disagreement (and I do), submitted amendments (or prepared ones -- though anyone who is eligible to add new business can submit the amendment by substituion (or by striking out words/paragraphs); it need not be the same parties that submit the main motion and any on the floor amendments).

There's also a slight procedural advantage in having one of the replacement proposals in the original proposal -- in that it provides a convenient handle to ammend by substituion to add the other replacement proposal -- and that in terms of floor time it's easier to move to amend to strike a paragraph out than to draft and insert one. So it's probably a more efficient use of debate time to have RME (or 6th) in the original proposal. [motions to amend consume debate time asigned to the main motion, so having a lot of motions to amend considered can eat a lot of the main motion debate time--although if we do end up moving the debate for this to Sunday, that will reset debate time, since it's only per-day).

#467 ::: Brian Z ::: (view all by) ::: May 15, 2015, 06:57 AM:

@433 JQ

Note to Brian Z: if your goal here is to throw sand in the gears, you can make that pretty obvious right now by voting against RME.

I'm not sure I would want to either "punt" or "throw sand." But I support a conversation about whether or not this change is a good idea. And I have not been convinced to support any rules change, especially if aimed at increasing fairness, which distracts us from selecting excellence.

I would support a very minor change which did nothing but allow replacement of the bottom two nominees in the event a set of lockstep voters swept a category. But even that is distracting from the real problem of encouraging new voters to value the traditional voting culture, which should be the priority especially until emotions subside and all heads are clearer.

Thanks.

#468 ::: Shane ::: (view all by) ::: May 15, 2015, 07:28 AM:

#467 ::: Brian Z :::

I think there have been a lot of good responses to your statement that we just need more voters, but you don't seem moved by them. Have you any response?

and I don't understand your distinction between "fair" and "excellent". We want a fair way of collating everyone's opinion on what's excellent. Making sure everyone gets equal consideration doesn't distract us from selecting excellence *at all*.

What we can't do is specify how people choose what is excellent, nor make them prove that excellence really is what they selected for. We take each person's opinion at face value, and then count them as evenly, as fairly as we can.


This last sounds so ridiculous to say that I kinda hope I've been successfully trolled. I can't see how you can oppose fairness in voting without being against democracy. What am I missing?

#469 ::: Duncan J Macdonald ::: (view all by) ::: May 15, 2015, 07:32 AM:

Jameson Quinn @ 432 & 433:

I didn't log back in until this morning. having reviewed posts up to #466, I have no problem with being a lone hold-out if needs be. More discussion follows.

It looks as if Duncan J Macdonald, Laertes, and Cally Soukop are the ones who oppose RME. Given that it seems unlikely that we'll get consensus for anything but RME or punt, and if you oppose RME for reasons of complexity you may oppose punting even more, are any of you willing to rethink your stance on RME?

I am a fan of simplicity. 6th place is the current solution, and it takes no special vote by the Board to make it the default in this case either. The best way to get this proposal through the Board (twice) is to make as few changes as possible.

The strengths that I see at this point are:
1) Nominating process does not change (from the membership point of view).
2) Nominations are counted/processed in a way that minimizes the impact of slates.
3) If a nomination is declined or found ineligible, the method by which the replacement is made does not change.

When I was a project manager in the Defense Contractor sphere, the watchword was always "Give the customer what they ask for. If they ask for a Volkswagen, don't design a Cadillac." The expressed objective here was to limit the effect of slate nominations. SDV-LPE in the counting the nominations stage does that. No need to change anything else at this time.

#470 ::: Seth Gordon ::: (view all by) ::: May 15, 2015, 08:54 AM:

May I propose a slight variation on #10 (base proposal plus amendment)?

Put a clause in the proposed constitutional amendment saying:

If a finalist withdraws or is disqualified after the votes have been tallied, the administrators shall exercise their discretion regarding what work or works to replace it with, subject to the following restrictions:

(a) No finalist who has already been notified shall be struck from the ballot without his or her permission;

(b) The WSFS may adopt a Standing Rule that specifies a procedure for replacement.

The Business Meeting can then vote the main SDV-LPE proposal up or down. If it passes, then there can be another debate on the standing rule for handling withdrawals (sixth-place vs. rerun-merge-extend vs. some other idea). If, by the time the amendment passes, the whole meeting is sick and tired of arguing about Hugo voting rules, then debate over the standing rule can be postponed to 2016.

#471 ::: Brian Z ::: (view all by) ::: May 15, 2015, 10:01 AM:

@467 I haven't replied except to direct questions because several folks have said some of my points are out of bounds for this thread and I can't really tell where the line is. But if you point me to the comments you mean I can respond.

It seems whether we need more voters or not, we have them.

Excellence vs democracy: I see no conflict. The WSFS, through democratic process, chose the rules which seek excellence. The Hugo nomination is not a democratic election: the business meeting is.

#472 ::: Steve Halter ::: (view all by) ::: May 15, 2015, 10:20 AM:

[DECLINED NOMINATIONS]

After finally catching up with the reading

[0] 1. Let the admins decide (don't mention declined nominations in the SDV-LPE proposal)
[10] 2. Add the works eliminated in the next-to-last round to the ballot (note that this is functionally equivalent to generating more than 5 potential finalists and replacing the top 5 as required)
[0] 3. Do not replace the declined nomination
[0] 4. Re-run the election with the declined nomination removed; accept the new results as-is
[5] 5. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep any new finalists, expanding the ballot as necessary
[0] 6. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total points
[0] 7. Re-run the election with the declined nomination removed; do not allow any of the original finalists to be changed, but keep only the new finalist with the highest total nominations
[5] 8. We craft two separate, nearly-identical proposals, with "promote #6" and "rerun and merge", and let the business meeting decide which they like better
[0] 9. We create a base proposal which lets the administrators decide, and two supplementary proposals with "promote #6" and "rerun and merge", and let the business meeting decide.
[0] 10. Create a base proposal which does not mention late removals, which implicitly lets administrators decide. Create a second proposal which invites amendments to decide what to do, with the explicit default being "let the administrators decide".

#473 ::: Steve Halter ::: (view all by) ::: May 15, 2015, 10:42 AM:

Is there a list of [TIE] proposals to vote on? It wasn't clear from the (lengthy) discussion to this point. Thanks.

#474 ::: Jon Lennox ::: (view all by) ::: May 15, 2015, 11:00 AM:

So Keith "Kilo" Watt in @456 demonstrates that in the 1984 data, there are no cases where a withdrawal would cause a previously-selected finalist to become unselected if the administrator just did a straight re-run of SDV-LPE (i.e., option 4).

Can someone construct a scenario in which this *would* happen? I'm having a hard time envisioning what the ballots would have to look like for it to occur.

Also, is there a standard term from voting theory for this sort of paradoxical outcome?

#475 ::: nathanbp ::: (view all by) ::: May 15, 2015, 11:23 AM:

Jon Lennox@474: It only makes a difference with heavily correlated ballots (such as might be found in the presence of a slate).

Suppose the following ballots in one of the last few rounds:
100x Alpha
95x Beta
90x Gamma
85x Delta
1x Psi
82x Psi, Omega
80x Epsilon

The second to last round is Omega vs Psi with Omega eliminated. The last round is Psi vs Epsilon with Epsilon eliminated. Now if Psi withdraws, under RME Omega would be the next nominee even though Epsilon placed 6th.

#476 ::: Joshua Kronengold ::: (view all by) ::: May 15, 2015, 11:31 AM:

Jon@474: This feels like something that requires real math.

My instinct is that this is impossible -- that something being removed will frequently cause it to be replaced by something else on the same ballot, but it won't usually cause anything else to be shifted out.

It's the LPE part that makes it hard here. If you look at the following ballots:

18: A
20: C, A
19: D, A
18: B

If we run these against SDV(2) [sdv with two winners], then in the initial ballot, C and D get eliminated and the final ballot is A, B, whereas if you remove A, then B gets eliminated first, leaving C and D as winners.

It's harder to construct stuff like this with LPE, because it will favor things with more votes -- you'd have to force two things that were eventual winners into an early showdown, so one eliminates the other (which then doesn't get to make the ballot.

But that requires a ballot where multiple things make the ballot due to the early showdown, so SDV-LPE(2) isn't going to work for this. SDV-LPE(3) might work--with the ballots:

18: A
20: C, A
19: D, A
18: E, A
17: B
16: F

With an early run of SDV-LPE, E and then D are eliminated, then F; the resulting ballot is A, C, B. But if you remove A from contention, then it's a straight popularity elimination, with C, D, and E making the balot, and B bumped.

#477 ::: colin roald ::: (view all by) ::: May 15, 2015, 11:32 AM:

Seth @465:
But if I hadn't put that second choice on, my first choice would have had an extra half point, making it #4, and something else would have been eliminated.

To be clear, that happens only if your second choice has not been eliminated yet, and since we're talking about the 5/6th place bubble, that means your second choice made the Hugo ballot.

Basically, LPE expects you (like now) to list works that you'll be happy to see on the Hugo ballot, and not to list works that you wouldn't. So seeing one of your choices make the ballot should be a good outcome for you, even if it was your second choice.

But yes, I acknowledge that I overstated slightly saying there's could never be any downside to listing a second, or a third, or a fifth choice that you like less well than your first. I *do* think the circumstances required for it to make a difference are exceedingly particular, and in the real world so rare as to not make a difference. And also no worse than the likelihood of the same thing happening under the current nomination system.

That is, LPE does not seem particularly vulnerable to "bullet voting", to the extent that anyone even thinks bullet voting is a bad thing.

(The initial problem at hand is that a minority voting a slate can shut their political opponents entirely out of a category. Bullet voting will never allow any minority group to do that, and I can't see it doing any other very bad thing either. My thoughts ramble on from here, but I think I'm going to break them out into a new comment.)

#478 ::: colin roald ::: (view all by) ::: May 15, 2015, 11:49 AM:

I don't know to what extent this stuff has been hashed through in previous thousand-comment threads, and will confess to not having kept up. If it's already been worked through, please feel free to reply just pointing me to the thread I should search through.

I'm willing to bet publically that next year there will be an "SJW" slate of some kind to oppose, presumably, SP4/RP2, and that the "SJW" slate will have a bigger group backing it. It's the obvious immediate response. So in my view, the goal of the LPE proposal is to make sure that nominators in 2017 who don't care to back either slate will have the option to nominate their own preferences without getting run over between mobs.

But *that* raises the question for me: have simulations been run with two or three competing slates amongst a majority still nominating whatever they like? I'm relatively confident that *one* group won't be able to lock everyone else out of a category, but how about two or three? Will anything that wasn't on *somebody's* slate have a chance? This is probably a question for Jameson's simulations.

#479 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 11:57 AM:

Colin@478:

I ran a number of multiple-slate scenarios, and in all cases there was at least one (and usually more, depending on the specific scenario) non-slate work on the final ballot. Obviously having a slate makes it more likely to get at least one of your works in the finals, but it didn't lead to sweeps.

Kilo

#480 ::: Jon Lennox ::: (view all by) ::: May 15, 2015, 12:31 PM:

Joshua Kronengold @476: Ah, that makes sense. So it sounds like finalist instability can happen when a broadly-popular work (nominated on a lot of semi-uncorrelated ballots) withdraws, leading to rejiggering the point counts in the early elimination rounds.

I'm tempted to call this a "Pratchett Withdrawal", though of course I don't know how the actual 2005 nominations were correlated.

I definitely like Pfusand @413's suggestion of having the software pre-calculate if any finalists' withdrawals would cause instability in the list of the other finalists, and letting the administrators use their judgment about whether to contact those finalists first. (Hopefully an administrator has some intuition about whether a withdrawal is likely.)

Obviously this doesn't help post-announcement withdrawals, but those were an unprecedented occurrence this year and only happened as a consequence of nomination slates, so hopefully they won't happen very often.

#481 ::: nathanbp ::: (view all by) ::: May 15, 2015, 12:45 PM:

Me@475: Oops, I misread Jon Lennox's request, I thought he was asking for a case showing a difference between RME and taking the 6th place.

#482 ::: Jameson Quinn ::: (view all by) ::: May 15, 2015, 01:41 PM:

@478: I agree with Keith. Multiple slates are problematic, but they have a hard time getting a sweep.

#483 ::: colin roald ::: (view all by) ::: May 15, 2015, 02:01 PM:

Buddha Buck @421:
(32,767 possibilities)

It's 2015. Not to be too flip, but I think running eliminations 32k times is what the supercomputer in my pocket does when I put down Crossy Road to go microwave a burrito.

#484 ::: Buddha Buck ::: (view all by) ::: May 15, 2015, 02:44 PM:

colin @483:

Yeah, 2^15 isn't too bad today, but it's still exponentially worse than 2^5.

Of course, the quickly-rejected proposal to find the 5 nominees which maximized some measure of total nominator happiness really showed a combinatorial explosion, as it scaled with N^5, where N was the total number of nominated works. With 100 nominated works, that's 72 million. Not impossible, but still a lot.

#485 ::: J Thomas ::: (view all by) ::: May 15, 2015, 03:01 PM:

@484 Buddha Buck

Of course, the quickly-rejected proposal to find the 5 nominees which maximized some measure of total nominator happiness really showed a combinatorial explosion, as it scaled with N^5, where N was the total number of nominated works. With 100 nominated works, that's 72 million. Not impossible, but still a lot.

You're probably talking about mine. It attempted to maximize diversity more than voter happiness. It didn't scale with N^5 because when one work had X bullet-votes then no work that had more than X total votes could compete with it, which cut down the number of combinations.

Combinations, not exponential.

Also, if we keep the 5% rule and the rule that you get only 5 nominations each, no more than 100 works can ever get 5%.

That proposal was shouted down before anyone understood it at all. I was deeply disappointed.

But what we have looks reasonably adequate.

#486 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 05:38 PM:

Jameson: I am off for the summer (finals graded, woo hoo!), so I can work on my laptop more. I reset the font size to a more reasonable size, but if someone else needs it larger, feel free. I appreciate your doing that for me.

That said, since some people still have trouble with the Google Doc, I think we should still paste a working copy here. In fact, it might be useful to have separate documents for the formal proposal language and the FAQ/commentary.

Kevin:
I'm assuming that all that can be submitted to the business meeting is the actual amendment and the FAQ's are just for discussion at the meeting itself. Can the FAQ's be submitted as well? Is there value in doing so?

Thanks,
Kilo

#487 ::: felice ::: (view all by) ::: May 15, 2015, 06:08 PM:

Jon Lennox @480: "Joshua Kronengold @476: Ah, that makes sense. So it sounds like finalist instability can happen when a broadly-popular work (nominated on a lot of semi-uncorrelated ballots) withdraws, leading to rejiggering the point counts in the early elimination rounds."

More or less, but it still requires a fairly large amount of unusual correlation. In #476, if A is so broadly popular, why didn't anyone who nominated B or F like it? One possibility is B and F are the last survivors of two different slates. So 6th place (or 4th place in this example) gives us a final ballot of CBF - ie both slates now get a spot, and nearly 3/4 of A's supporters miss out; while RME gives us CBDE - the B slate retains its spot, but the more popular D and E get on instead of F, and more than 3/4 of the people who nominated A get one of their other choices on the ballot instead (the other 18 either bullet voted A, or their other nominations were for unpopular/obscure works that got eliminated early).

In practice, there'd probably also be some overlap between C, D, and E, and a final ballot of CBD (no extension required) is more likely.

#488 ::: Tim Illingworth ::: (view all by) ::: May 15, 2015, 06:29 PM:

KW@486: it's normal to submit motions in the form:

Title: Change Something
Moved, to amend the WSFS Constitution by making this change.

Discussion: This change is really trivial and needs no discussion. We should just do it ;-)

#489 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 07:18 PM:

Tim@488:

Okay thanks. So if the discussion is also going with the proposal, we should probably keep that as tight as possible while still making the case.

K

#490 ::: felice ::: (view all by) ::: May 15, 2015, 07:47 PM:

Steve Halter @473: "Is there a list of [TIE] proposals to vote on? It wasn't clear from the (lengthy) discussion to this point."

The key question is: is it better to eliminate two works at once, or use an essentially random (but repeatable) method to pick one over the other?

Personally I'm strongly in favour of random when necessary; for the rare ties that make a difference (ie either one of the pair could become a finalist, but the tie eliminates them both at once and doesn't reallocate their points), it doesn't matter which wins because there have to be people who nominated both, so either way more people nominated the winner than didn't.

The system described in #135 will probably suffice to break important ties if we abolish the 5% threshold; but if we eliminate everything under 5% before running SDV-LPE, there may well be insufficient previous rounds to help. And if we keep the threshold but don't remove the works that fail to meet it before running SDV-LPE, a work below the threshold may cause a work above the threshold to be eliminated (eg C with 6 points and 3% causes A with 5 points and 10% to eliminate B with 4 points and 8%, when otherwise A and B would both be finalists).

#491 ::: Seth ::: (view all by) ::: May 15, 2015, 08:03 PM:

#477 colin, I could construct a case where putting my second choice on my ballot eliminated my first without itself making the ballot. Let's see:

At some point, my first choice (A) has 20.2. X has 20, Y has 20.4, my second choice (B) has 21. A gets eliminated. There are 8 left, and B also doesn't make the last 5. If B weren't on my ballot, A would have had 20.7, Y would have eliminated X (or vice versa) and that would have given more points to A, enabling it to survive to the last 5.

#492 ::: Seth ::: (view all by) ::: May 15, 2015, 08:06 PM:

#467 Brian Z., this entire proposal is simpler than any useful definition of "a set of lockstep voters swept a category."

#493 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 08:31 PM:

Seth@491:

I only see one problem with your analysis, though: You didn't look at the number of nominations each work received. That's a critical part of SDV-LPE and what makes it different from simple SDV. in your example, A and X would be -eligible- for elimination, but it might actually be that X gets eliminated if it had fewer nominations than A. It's true that if you hadn't nominated B then your A would have half a point more, but there's more to it than just points. Your nomination of B might actually have gotten it on the ballot since it's number of nominations that determines which of the eligible works gets eliminated, not points.

That said, I imagine there's still a way to construct a scenario where it happens, but it would be highly contrived and seems to me pretty unlikely. I dunno, you'd have to try it.

Kilo

#494 ::: Keith "Kilo" Watt ::: (view all by) ::: May 15, 2015, 08:53 PM:

[TIES]

Steve@473: In the previous discussion, it's true that felice spoke up for random elimination because of disapproval of ever eliminating more than one work per round (correct me if I'm mis-stating your case, felice). Random elimination seems to be a non-starter for a largish group of people (including me -- I won't support a non-deterministic nomination system), however.

According to Jameson's 1300 simulations, the method described in #135 only eliminates one work per round in all but three cases (and all of these were fairly rare):

- Works appearing on just a few ballots, eliminated early. (which couldn't possibly win anyway)

- Works appearing (almost) exclusively as bullet votes. (Two groups of exactly the same size each nominated only a single work and no one else nominated those works)

- Works appearing exclusively on slate ballots. (The slate had perfect discipline for its works, giving exactly the same number of nominations to two of its works.)

None of these are really a problem, when it comes down to it.


My only remaining quesiton is how necessary it is to look back to previous rounds (comparing points from previous rounds until there isn't a tie) before eliminating more than one work. I believe felice came up with a situation where it could matter, but it didn't show up in any of the simulations I ran without lookback.

In short, the tie system seems fairly settled:

(Note that if there is a tie for lowest or second-lowest POINTS then all tied works are eligible for elimination along with the second-lowest or lowest point total work(s).)

What we are really looking at is ties for total nominations. If there is a tie for total number of nominations, then we look at points again and eliminate the work with the lowest point total. Ties aren't common, but this is by far the most common tie that occurs. I've forgotten the exact number Jameson found, but it was 99% of the time or something like that. Correct me if I've got it wrong, Jameson.


So, the tougher case:

- If there are works that are tied in both points and nominations, then we look at point totals for those works in previous rounds until we find a difference -- we eliminate the one with fewer points whenever we find it.

- If we get all the way back to the first round and the works are still tied for points and nominations, we just eliminate them both (this is the part that felice doesn't like).


My original code doesn't do lookback. If they are tied for both points and nominations, then both were immediately eliminated. For all of the cases I ran with realistic data (no exceptions), the ties existed all the way back to the first round, so both methods would have eliminated both works no matter what. As I mentioned, I believe felice came up with a hypothetical situation where you broke the tie before round 1, so I'm not admantly opposed to lookback. It just makes tie breaking slightly more complicated, is all.

So, in summary, the only real voting left is for lookback or no lookback. I think the consensus is for lookback, but I don't know that we had a formal poll on it.

Kilo

#495 ::: Jon Lennox ::: (view all by) ::: May 15, 2015, 08:57 PM:

felice @ 487: So it sounds like a plausible scenario where this could happen is if you have two slates (each making up about 20% of the nominators) with one overlapping work -- this work ends up on 40% of the nominating ballots and so ends up easily as a finalist, eliminating the rest of those slates' nominations. If that work withdraws, each slate's second-place work ends up on the ballot, knocking some other finalist out of the top five.

#496 ::: Brian Z ::: (view all by) ::: May 15, 2015, 09:00 PM:

@492, Seth, if it were hard to define, why would we know what it means? I define a sweep by lockstep voters as when: without a set of identical ballots, a set of five works could not have been selected.

If we worry about an almost-sweep: it is when a set of ballots identical in four places causes four works to be selected.

If anyone wants to classify sets of ballots with less than four identical works as lockstep voting, we should push back against that. This is a crucial point. Using the WSFS constitution to regulate what somebody who may not even be a member says on the internet (about slates or anything else) is not going to work. We are discussing how to evaluate the actual ballots cast by members.

#497 ::: Steve Halter ::: (view all by) ::: May 15, 2015, 09:32 PM:

Keith@494:Thanks for the synopsis. The lookback method sounds good. The tied all the way back case would seem to be ultra rare enough that it will probably never occur.
felice@490:Thanks.

#498 ::: JJ ::: (view all by) ::: May 15, 2015, 11:54 PM:

Brian Z, #467: "But even that is distracting from the real problem of encouraging new voters to value the traditional voting culture, which should be the priority especially until emotions subside and all heads are clearer."

Firstly, you're saying that encouraging the Puppies to "value the traditional voting culture" is going to be an effective strategy? I think the Puppies have made it amply clear that that is not a viable strategy. But hey, you're sure welcome to give it a try, and let us know how much success you have.

Secondly, your comment "until emotions subside and all heads are clearer" (apart from being a direct quote from Brad Torgersen and being extremely condescending and patronizing to the people here) shows your unfamiliarity with the people with whom you are dealing. It's been 6 weeks. Emotions have subsided, and the heads of a great many people are very clear -- especially about the fact that there are changes which can be made to reduce the effect of bloc-voted nominations.

Seth's point at #492, which you apparently missed, is that there is no need to define "lockstep voting". That would be a subjective judgment -- and there is no need to resort to subjective judgments when what has been developed here is an objective method of counting nominations which does not rely on the subjective judgment of anyone.

#499 ::: felice ::: (view all by) ::: May 15, 2015, 11:55 PM:

Jon Lennox @495: "So it sounds like a plausible scenario where this could happen is if you have two slates (each making up about 20% of the nominators) with one overlapping work -- this work ends up on 40% of the nominating ballots and so ends up easily as a finalist, eliminating the rest of those slates' nominations. If that work withdraws, each slate's second-place work ends up on the ballot, knocking some other finalist out of the top five."

Yep, except RME extends the ballot in this case, so the other (non-slate) finalist doesn't get knocked out. And two equally powerful slates with one overlapping work isn't that likely a scenario (this year, the rabid puppies were significantly more powerful than the sad, and they overlapped by a lot more than one work per category).


[TIES]

Keith "Kilo" Watt @494: "I won't support a non-deterministic nomination system"

The proposal is for deterministic pseudo-random system (first appearance in nominations or pre-generated set of coin-tosses), so the results are repeatable.

"According to Jameson's 1300 simulations, the method described in #135 only eliminates one work per round in all but three cases"

All based on 1984 data. The 2013 ballots had a potential tie that could have been a problem - two Seanan McGuire short stories tied on 28 nominations (part of a four way tie for 5th), which probably very strongly overlapped with each other (maybe even 100%), so would have had relatively low points and could be subject to early elimination, and it's quite possible that they would have tied for points, too (but we don't know if they were or not). A slate-like effect, but presumably just the result of Seanan fans being very enthusiastic about her work, rather than any kind of organised voting bloc. If the threshold has been 4% instead of 5%, one of them could have been a finalist, and there wouldn't be any previous rounds for the #135 method to check. And Dan Dos Santos and Julie Dillon tied in Pro Artist, so depending on overlaps, it's theoretically possible John Picacio could have eliminated them both. There was also a tie in BDP Short Form, with only 6 nominations separating 1st from 5th

And in 2014, there were two pairs of works tied on nominations amongst the finalists for Best Graphic Story, and a three way tie for 5th in Best Fancast, plus ties in Best Fanzine, Pro Artist, and BDP Short Form; the degree of overlap is less clear there, but there's clearly at least the possibility of simultaneous elimination.

Multiple Doctor Who episodes regularly get close to tying, with probable significant overlap. A simultaneous elimination there one day is quite possible.

The only finalist ties in 1984 were Campbell Award (not tied on points), and non-fiction (tied for most nominations, so no risk of being eliminated).

The probability of two potential finalists being simultaneously eliminated is very low, but it's not zero, and it's bad if it happens. The lookback method is close to useless unless the 5% threshold is eliminated.

#500 ::: Keith "Kilo" Watt ::: (view all by) ::: May 16, 2015, 12:19 AM:

[TIES]
felice@499:
The proposal is for deterministic pseudo-random system

Fair enough. I won't support a pseudo-random system, either.

As for the 5% rule, I agree we should support those working to remove that rule. A proposal is already out there -- and Kevin doesn't want it in ours -- so I don't think there's anything to do there.

K

#501 ::: Keith "Kilo" Watt ::: (view all by) ::: May 16, 2015, 12:37 AM:

[FORMAL PROPOSAL]
All:

Here is the latest draft of the formal proposal. This is based on what I perceive to be the majority opinion on the system, and it includes the edits suggested for the Google Doc. I think in the main, we all agree, but some have some differences of opinion in some of the details. At this stage, I'd like for your thoughts and what, if any, dealbreakers you see in this version.

Note that we also need to write the "6th place" amendment as well...

Jameson: I didn't want to upload this on top of the Google Doc you have up there, as this just includes the formal proposal itself. Do you want to upload it as a separate document? I can email you the Word doc if that is easier for you, just let me know.

Kilo
========================================
Short Title: Change to Hugo Nomination System

Moved, to amend section 3.8 (Tallying of Nominations), section 3.9 (Notification and Acceptance), and section 3.11 (Tallying of Votes) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed.
3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.
3.8.3: Any nominations for “No Award” shall be disregarded.
3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.
3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.
3.8.6: The Committee shall move a nomination from another category to the work’s default category only if the member has made fewer than five (5) nominations in the default category.
3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.9 or 3.2.10, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category.
3.8.8: The final Award ballots shall list in each category the five (5) eligible finalists as determined by successive rounds of elimination, where each member gets a single vote that will be divided equally among their nominations.
3.8.8.1: In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots.
3.8.8.2: In the event that two (2) or more works are tied for both total points and for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, the point totals for each work in previous rounds will be compared. If the two works have had equal point totals and equal number of nominations since the first round, then all members of that tie will be eliminated.
3.8.8.3: If elimination would reduce the number of ballots to fewer than five (5), then instead none of the works should be eliminated and all remaining works shall appear on the final ballot.
3.8.8.4: After a work is eliminated, it is removed from the ballots of the members who nominated it, and those members’ votes are reallocated equally among their remaining nominations, if any.


Section 3.9: Notification and Acceptance.
3.9.1 Worldcon Committees shall use reasonable efforts to notify the nominees potential finalists, or in the case of deceased or incapacitated persons, their heirs, assigns, or legal guardians, in each category prior to the release of such information. Each nominee potential finalist shall be asked at that time to either accept or decline the nomination. If the nominee potential finalist declines the nomination, that potential finalist shall not appear on the final ballot. In this event, the nomination system shall be re-run as described in Section 3.8 without the declined nomination. The five (5) eligible finalists from this re-run shall be merged with the remaining potential finalists from the original run. If this merge would result in more than five (5) finalists, then the ballot shall be extended to include the finalists from both the original and the re-run of the nomination system. This procedure shall also be used in the event that a potential finalist is deemed ineligible.

Section 3.11: Tallying of Votes.
3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, … places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes. During the same period a record of at least the last fifteen rounds of the selection process for each category shall also be published.

Submitted by: (insert members here, ideally including at least one attending member who will be at all the relevant WSFS meetings. A proposing member gets the privilege of speaking first to the pro side of the proposal)

Commentary: In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn't penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate -anything- you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don't, that's okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

#502 ::: Annie Y ::: (view all by) ::: May 16, 2015, 01:03 AM:
at least the last fifteen rounds of the selection process for each category

What if there are less than 15 rounds? Shouldn't the rule account for that case?

#503 ::: Shane ::: (view all by) ::: May 16, 2015, 01:04 AM:

#501 ::: Keith "Kilo" Watt :::

[FORMAL PROPOSAL]
[DECLINED NOMINATIONS]

No dealbreakers. I'd support that as is.

One thing sticks out at me that we discussed but didn't focus on upthread. The way it reads now I picture rerunning for each declined nomination as we receive them. But actually we want to do the re-run only after all potential finalists have had a chance to decline, right? With any and all decliners removed.

Is it just me with that reading? Can we rewrite the new bit of 3.9 to make that clear?

"If any potential finalist(s) decline the nomination, those potential finalists shall not appear on the final ballot."

"In this event, the nomination system shall be re-run as described in Section 3.8 without any declined nomination or nominations."

#504 ::: Shane ::: (view all by) ::: May 16, 2015, 01:14 AM:

#501 ::: Keith "Kilo" Watt :::

[FORMAL PROPOSAL]
[DECLINED NOMINATIONS]

Is the sentence accurate where it says "The five (5) eligible finalists..."? Ties can mean either the first run, or the re-run, can end up with not 5.

Should we remove the number, and just say "The eligible finalists...".

And hold on, I lost this in the discussion before. What if we have 6,7 or 8 potential finalists, and some withdraw but we still have 5 or more? Do we only want to re-run when there's under 5?

No, y'know what, thinking about it, I want to re-run when anyone's declined because of the chance that some interest group's second choice is way down the rankings, but will make it on a re-run. Disregard my previous para.

#505 ::: colin roald ::: (view all by) ::: May 16, 2015, 01:16 AM:

BrianZ @496:
a set of ballots identical in four places causes four works to be selected

So, what happens if there are two slates? Next year, I virtually guarantee there will be some kind of Justice Kitten slate as a reply to the Puppies. So, suppose they get crazy and lock up all five short story ballots with turgid post-Marxist workshop exercises about hero/heroines with so many sex changes they don't know what gender they are anymore. Your rule kicks in, and bumps off the bottom two to be replaced by the erstwhile #6 and #7, which are neo-Nazi guns-and-boobs shoot-em-ups from Rabid Puppy Press.

Please go ahead and tell me that this seems like a good result to you, well representative of Excellence in Science Fiction.

#506 ::: JJ ::: (view all by) ::: May 16, 2015, 01:26 AM:

colin roald, #505: "Next year, I virtually guarantee there will be some kind of Justice Kitten slate as a reply to the Puppies."

I think you are vastly underestimating the number of non-Puppies who find slates, of any persuasion, to be utter anathema -- and that any non-Puppies attempting to spearhead such a movement would very quickly experience a tsunami of disapproval and disincentives from those people, just as the Puppies will.

I think most non-Puppies have come to the realization that the only acceptable response to slates this year, and next, until the WSFS nominating process is changed, is to suck it up and rely on Mr. Noah Ward.

#507 ::: nathanbp ::: (view all by) ::: May 16, 2015, 01:46 AM:

[FORMAL PROPOSAL]

Looks good overall. A couple comments:
3.8.8: The final Award ballots shall list in each category the five (5) eligible finalists as determined by successive rounds of elimination, where each member gets a single vote that will be divided equally among their nominations.
3.8.8.4: After a work is eliminated, it is removed from the ballots of the members who nominated it, and those members’ votes are reallocated equally among their remaining nominations, if any.
I think vote/votes should be changed to point/points here, to match the language in the rest of the proposed 3.8.8. Or change points there to votes, but I think points everywhere is clearer.

In the spirit of making the least changes possible, I think the proposal should leave the phrasing in 3.9.1 as "nominee" instead of changing it to "potential finalist".

In this event, the nomination system shall be re-run as described in Section 3.8 without the declined nomination
I think "with the declined nomination(s) removed from the ballots on which they appeared" might be clearer here?

We may also want to clarify 3.9.1 slightly to indicate that the results of the rerun should be combined with the original run and any previous reruns. Right now it could be interpreted to say that only the original run and the most recent rerun are combined.

#508 ::: colin roald ::: (view all by) ::: May 16, 2015, 01:51 AM:

Kilo @501:

3.8.8 says "each member gets a single vote", but then 3.8.8.1 goes right to talking about "works ... with the least and second-least total points". Points need to be defined, possibly simply by replacing "vote" in 3.8.8 and 3.8.8.4 with "nominating point".

3.8.8.2 says "In the event that two (2) or more works are tied for both total points and for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated." This is mis-stated, since obviously if two works are tied for total points, then neither one has a lower point total. I think it should be instead: "In the event that two (2) or more works being compared are tied for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated."

In the Commentary, it seems like it would be worthwhile to add two points:

- It is to be emphasized that this proposal does not change the nominating ballot from the perspective of Worldcon members: they still list one to five unranked works in each category they choose, just as they have previously.

- It is also an explicit goal of this proposal not to disenfranchise anyone, including members nominating from a recommended slate of works. Rather, this proposal seeks to ensure that a minority of members cannot disproportionately dominate an entire category, no matter how coordinated they are. It will still be quite possible for a motivated group to get one or perhaps two works onto the final ballot, but not to lock all other perspectives out of the running.

#509 ::: colin roald ::: (view all by) ::: May 16, 2015, 02:02 AM:

JJ@506:

You are an optimist. I mean, I think you are right that the majority of traditional fandom despises slates and won't join them, but I bet there is at least another similar-sized minority for whom political ends are more important than Hugo culture, and they won't be able to resist promoting something. Tradition or social disapproval didn't stop the Sad Puppies, did it? I doubt it will stop their enemies, either.

Also, the Puppies (especially Rabid, I think) stacked their vote by drawing in a lot of outside activists who never really cared much about the Hugos other than as a way to "show" the SJWs. I bet there are at least as many feminist activists out there willing to come in and stuff ballot boxes on the Justice Kitten side.

#510 ::: felice ::: (view all by) ::: May 16, 2015, 02:30 AM:

[FORMAL PROPOSAL]

Keith "Kilo" Watt @501: "If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, the point totals for each work in previous rounds will be compared."

Perhaps "If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, the point totals for each work in each previous round will be compared until a round with different point totals is reached."

"3.9.1 Worldcon Committees shall use reasonable efforts to notify the nominees potential finalists"

Is "potential" necessary? They are definitely finalists unless they actively decline, right? Ie if there's no response, they go on the final ballot anyway?

"During the same period the nomination voting totals shall also be published"

I'd suggest leaving this in, since unless the 5% threshold is removed, there could be a lot fewer than 15 SDV-LPE rounds (none, in some cases). And add instead "Additionally, a record of at least the last ten rounds of the selection process (or all rounds, if fewer than ten) for each category shall also be published." Ten rounds shows ten works eliminated plus the 5 finalists, ie is equivalent to the current top 15.

"Commentary"

I'd suggest including something along the lines of "This system increases the chance that one of the works you nominate will make the final ballot, but decreases the chance that more than one will" to simply explain how this system differs from the current system in practical terms.


Shane @504: "And hold on, I lost this in the discussion before. What if we have 6,7 or 8 potential finalists, and some withdraw but we still have 5 or more? Do we only want to re-run when there's under 5?"

I would suggest something like "In the event that there is a tie involving last place, and one of the tied works declines or is found ineligible, it should not be replaced." Though actually, maybe that's not necessary; I would guess the most likely result of RME in this situation is to not add a replacement anyway, eg finalists are ABCD[EFG], E withdraws, and the rerun produces ABCD[FG]. But saying tied works don't need to be replaced does no harm, and would be helpful to have already there if the "take the 6th" amendment beats RME.

#511 ::: Shane ::: (view all by) ::: May 16, 2015, 02:38 AM:

#510 ::: felice :::

Does it work out differently if one or more of the non-tied works (ABCD) declines?

#512 ::: felice ::: (view all by) ::: May 16, 2015, 02:51 AM:

Shane @511: "Does it work out differently if one or more of the non-tied works (ABCD) declines?"

It's more likely to; any second choice for E nominators is going to be less popular than F and G (because it's less popular than E by definition, and F and G are equal to E)*, but second choices for A, B, C, or D could still be more popular than E, F, and G, but eliminated earlier due to point splitting.

#513 ::: junego ::: (view all by) ::: May 16, 2015, 04:21 AM:

3.8.8: The final Award ballots shall list in each category the five (5) eligible finalists as determined by successive rounds of elimination, where each member gets a single VOTE [Please remove the word "vote" and "voter" and "voting" from any section of the proposal dealing with the nomination. The current Constitution does, in fact, differentiate the language between the nomination process and the final voting process. I don't know if this was due to some earlier discussion/disagreement or not, but it makes a lot of sense not to confuse the two things we are doing. We are not voting (in a strict sense) at this stage, we are recommending/suggesting/polling. We should comply with the current Constitution's useage unless there's some compelling reason not to.]. that will be divided equally among their nominations.

#514 ::: junego ::: (view all by) ::: May 16, 2015, 04:37 AM:

@ #513 ::: junego ::: (view all by) ::: May 16, 2015, 04:21 AM:
Forgot to put header on.

[FORMAL PROPOSAL]

3.8.8: The final Award ballots shall list in each category the five (5) eligible finalists as determined by successive rounds of elimination, where each member gets a single VOTE [Please remove the word "vote" and "voter" and "voting" from any section of the proposal dealing with the nomination. The current Constitution does, in fact, differentiate the language between the nomination process and the final voting process. I don't know if this was due to some earlier discussion/disagreement or not, but it makes a lot of sense not to confuse the two things we are doing. We are not voting (in a strict sense) at this stage, we are recommending/suggesting/polling. We should comply with the current Constitution's useage unless there's some compelling reason not to.]. that will be divided equally among their nominations.

#515 ::: junego ::: (view all by) ::: May 16, 2015, 05:59 AM:

colin roald @ 508 and
Kilo @501:

[COMMENTARY]

colin, I agree with your recommendations for changes to the formal proposal.

- It is to be emphasized that this proposal does not change the nominating ballot from the perspective of Worldcon members: they still list one to five unranked works in each category they choose, just as they have previously.

I think this needs to be the first point in the Commentary and emphasized. Our proposal doesn't affect the way members nominate, it just changes the way the nominations are tallied.

- It is also an explicit goal of this proposal not to disenfranchise anyone, including members nominating from a recommended slate of works. Rather, this proposal seeks to ensure that a minority of members cannot disproportionately dominate an entire category, no matter how coordinated they are. It will still be quite possible for a motivated group to get one or perhaps two works onto the final ballot, but not to lock all other perspectives out of the running.

1)I really don't want to put any language anywhere in the proposal the explicitly or implicitly condones, supports or even accepts slates nor do I think we want to enter into arguments about what a slate is, what recommendation lists are, etc.. There may be people at the meeting motivated to oppose the proposal, we should reduce points of contention. So suggest changing the first sentence to "It is an explicit goal of this proposal not to disenfranchise anyone."

2) Suggest rewording second sentence to "Rather, this proposal seeks to ensure that a motivated, coordinated minority of members cannot disproportionately dominate an entire category." Just for brevity (which I don't notice when editing my own verbosity for some reason :-p ).

4) Now for just nitpicky edit-foo(lishness). Could change sentence to "A coordinated group can still get one, or maybe two, works on the final ballot, but they will not be able to lock all other perspectives out of the running.

#516 ::: J Thomas ::: (view all by) ::: May 16, 2015, 06:01 AM:

@493 Keith "Kilo" Watt

That said, I imagine there's still a way to construct a scenario where it happens, but it would be highly contrived and seems to me pretty unlikely. I dunno, you'd have to try it.

Given a "normal" collection of votes, the only likely result is that the five with the most votes will win. That's about 90% right there. *Any* other result is unlikely, and the more specific you make it, the less likely it is.

If we were doing approval voting, you would vote for the works you thought should be finalists. You are saying that the ones you vote for are worth being finalists and you say that for all of them equally. If it's more important to you to vote for Able over Baker than to vote for both of them, then you should vote only for Able. If you think they're both good enough, then vote for both.

This other way, the total votes count more than 90%. If you particularly care about the choice between voting for Able versus Able+Baker, the chances are probably around 96% that any variation from "most votes win" will not affect those two.

If you want to get a different result, you need to be part of a large cabal that tries to have more influence than anybody else. This voting system reduces the effect of slates to be no more than that of bullet-voting.

Bullet-voting is the equivalent of single-issue voting. You get more influence on your single issue, and you lose all influence over anything else. The effect is likely to be small, because if your cabal is large enough to get your work into the finals with bullet-voting, it's probably large enough to do that even if all of you also vote for your own individual other choices. You give up all influence on the rest of the nominations, in exchange for an improved chance (total improvement well under 10%) to get this one in.

The reason I say under 10% is that it's 90% the top 5 will win, and so if your cabal is 20% of the voters and you all bullet-vote, that increases your work's chance from better than 90% to 100%.

That reasoning isn't exactly correct, but it suggests the order of things. If your work is not in the top 5 for votes, by bullet-voting you can increase the chance that if the top 5 don't all win, it will be yours that replaces one of them. And that chance is probably not over about 10% whether your whole group bullet-votes or not.

The odds are worth thinking about while considering whether to adopt the system. It's worth thinking about while we argue whether the system is fair enough. When peculiar results are *possible*, does that make it bad?

It's probably not worth building a cabal to have that sort of effect. You'd do better to just write reviews telling whoever will listen how great the work is.

#517 ::: J Thomas ::: (view all by) ::: May 16, 2015, 06:57 AM:

@494 Keith "Kilo" Watt

So, in summary, the only real voting left is for lookback or no lookback. I think the consensus is for lookback, but I don't know that we had a formal poll on it.

I have a moderately strong preference against lookback because it is harder to explain. Particularly, if you want to make an animation describing the process, it's harder to draw it.

Still possible, just harder. So I don't like it.

And the farther you have to look back to find a difference, it seems to me the less difference it makes.

Imagine you're explaining it to the guy whose work lost. "It was a tie, so we looked back, and at the beginning you were almost tied, but one of your votes was shared with four others, while he had a vote that was shared with only three others. The first round one of those four dropped out and you were both tied from that point on. We decided that that extra vote for somebody else meant that his deserved to be a finalist and yours didn't."

So I prefer the simpler system. If they tie on votes, eliminate the lowest score. Eliminate all that tie on lowest votes and lowest scores both.

For the last round, better to have 6 finalists than 4, better to have 4 than 7, better 7 than 3, better 3 than 8 or more.

If we have a 5-way tie on the last round, something has gone seriously wrong.


#518 ::: Jameson Quinn ::: (view all by) ::: May 16, 2015, 08:24 AM:

@494 Re: ties: I did 1300 bootstrap runs, and there were no unbreakable ties with fewer than 15 candidates remaining. (It's not that there was one with 16 remaining; it's just that I arbitrarily only started checking when there were 15. There were unbreakable 1- and 2-vote ties, I don't know exactly when the latest one was.

"Unbreakable" means same points all the way back to the first round.

This was without a 5% threshold.

In theory, you could make a scenario where eliminating or not eliminating the 5% works could make a difference as to who was a finalist. But really the only way it could matter is in deciding which was the second slate candidate finalist. And that would have to mean that a 20% slate was getting two slots, which wouldn't happen unless non-slate voters were really all over the map.

So anyway... if you need the point counts from the pre-5% rounds to break a tie, you can just run the system without doing the 5% elimination to get those numbers, and make sure that it doesn't eliminate an over-5% candidate before an under-5% one, which except in case of failed slates won't happen anyway.

As a voting theorist, I consider the question what to do in case of ties resolved, unless somebody (who otherwise supports the proposal) has an objection to the "points and historical points, then mass elimination" option.

#519 ::: Jameson Quinn ::: (view all by) ::: May 16, 2015, 08:56 AM:

So it seems that people are doing the edits here rather than on Google Docs. That's a big slower but more inclusive. If people want to use Google Docs instead, say so, and I'll curate the document. If the consensus is that slow and steady wins the race, then I won't, and I'll just leave it here.

I think that @516 puts it very well; well enough to go into the commentary. Something like:

"Under this system, looking at computer-generated elections based on historical data¹, it seems that if a work has one of the top 5 vote tallies, it will win about 90% of the time. That means that in the absence of slate voters, the results of this system will be 90% the same as historical results. It also means that unless you have near-perfect knowledge of exactly how everyone will vote, the additional advantage to be gained from coordinated strategic voting is no more than about 10% better chances of getting the desired result, and even that is only true if the result is reasonably popular already. Given the chances of a "no award" backlash in the following round, a less than 10% chance of an advantage in the nominations is probably not worth the work of coordinating a strategy.

¹ We made 1300 simulated elections, generated by resampling ("bootstrapping", in statistical parlance) from the 1984 elections. (100 simulations from each category). The simulated elections had only 100 ballots each, so these results are likely to be conservative; that is, if anything, they will exaggerate the possibility of unusual results."

....

I'm just going to put this out there; I don't expect it to be picked up, but it can't hurt to try.

I'd enjoy going to Sasquan, but I can't justify it financially on a graduate student's earnings. (Yes, I'm a graduate student who's old enough to have read "tea with a black dragon" when it first came out in paperback.) I've also done an amount of work on this proposal which would have easily earned me enough for the necessary tickets if I had been consulting. (Though if I did earn money on this, I'd probably donate all of it to electology.org) Would anybody be willing to sponsor me? Obviously, I'd go to the BM, and be available to answer questions there; it has been suggested that would have value.

And similarly: if I deserve being sponsored to attend, then Keith surely deserves it twice over.

If you're interested in sponsoring either one of us, you can email me, using my first and last names separated by a dot, on google's email service (or using my names with no dot in between, at Faculty of Arts and Sciences [FAS] dot harvard dot edu).

#520 ::: Jameson Quinn ::: (view all by) ::: May 16, 2015, 09:00 AM:

@517: But the idea is that if there is a tie at the last step, you just keep both. So the explanation you're giving is why a work lost when it lost, but unless it's part of a slate, the fact that it was up for elimination at all probably means it would have lost a round or two later anyway. The tiebreaker is just a way to allow the eliminations to proceed; it mostly won't make a difference. And finally, the alternative, as I see it, is to eliminate both; so the person who is eliminated by the tiebreaker hardly has grounds for complaint.

#521 ::: Jameson Quinn ::: (view all by) ::: May 16, 2015, 09:02 AM:

Note that my post @518 was in limbo (database error) for some time, so it may no longer have been really germane when I finally posted it.

#522 ::: J Thomas ::: (view all by) ::: May 16, 2015, 09:53 AM:

@520 Jameson Quinn

@517: But the idea is that if there is a tie at the last step, you just keep both. So the explanation you're giving is why a work lost when it lost, but unless it's part of a slate, the fact that it was up for elimination at all probably means it would have lost a round or two later anyway.

We're talking about a rule that applies from the 2nd to next-to-last step. At the next-to-last step, it could make the difference between being a finalist and not.

The tiebreaker is just a way to allow the eliminations to proceed; it mostly won't make a difference.

Eliminations proceed slightly faster if both are eliminated. Agreed, it doesn't make a big difference. It's simpler and also easier to explain without the lookback. I think it's fairer without that, but not more merciful. It won't usually make much practical difference.

And finally, the alternative, as I see it, is to eliminate both; so the person who is eliminated by the tiebreaker hardly has grounds for complaint.

He does, but he reveals himself as small-minded if he talks about it. "My work is just as good as his, but he's a finalist and I'm not." I could imagine Correia and Scalzi complaining about that, whichever one lost, but it wouldn't be a big surprise if they just congratulated the other instead. It's a bad loser who complains that somebody else didn't lose too. We probably have some of those.

#523 ::: junego ::: (view all by) ::: May 16, 2015, 10:09 AM:

[SPONSORSHIP]

Jameson

I would be willing to contribute to a couple of available, interested and knowledgeable people going to Sasquan. Maybe you can discuss this with whoever is interested in representing the proposal and start something like a kickstarter campaign?

#524 ::: junego ::: (view all by) ::: May 16, 2015, 10:32 AM:

[TIE-BREAKING]

I am almost positive what I'm about to ask/propose was covered in the previous thread. (I can't find it and don't remember details.) If so, and it is just too something-or-other to work, would someone explain quickly and simply why it won't work and/or point me to the discussion? tia

Wait, was the reasoning that the method below would favor slates?

PROPOSED
If 2 works are tied on points and nominations both they challenge the work with the next lowest points. At the challenge, if their number of nominations don't exceed that work, both are eliminated. If their nominations exceed the challenged work, that work is eliminated. If the tied works win, after the recalculation the tied works will either no longer be tied or will still be tied and will challenge the next lowest work.

#525 ::: colin roald ::: (view all by) ::: May 16, 2015, 11:10 AM:

junego@515:
I think I like felice@510's wording way better than my too-verbose attempt. How about:

"It is an important goal of this proposal not to disenfranchise anyone. This system increases the chance that one of the works you nominate will make the final ballot, but decreases the chance that more than one will."

#526 ::: colin roald ::: (view all by) ::: May 16, 2015, 11:44 AM:

junego @524:
If their nominations exceed the challenged work, that work is eliminated

I believe that will favor slates, since they will tend to be allocating fewer points per approval -- or conversely, will have more approvals per point. So tied slate works would be likely to win this challenge.

#527 ::: junego ::: (view all by) ::: May 16, 2015, 12:01 PM:

colin roald @ 525
junego@515:
I think I like felice@510's wording way better than my too-verbose attempt. How about:
"It is an important goal of this proposal not to disenfranchise anyone. This system increases the chance that one of the works you nominate will make the final ballot, but decreases the chance that more than one will."

So, no statements about reducing the power of coordinated minority groups at all? Why should anyone want to go to the trouble of changing the constitution if it's not going to help with the problem we've got? I can see the point of putting as positive a light on this as possible, but shouldn't we mention the elephant in the room? Or is that going to be somewhere else?

First sentence is fine. I'd rephrase the second sentence.
"This system increases the chance that each member can get one nominee on the ballot, but decreases the chance that anyone can get 2 or more on the ballot."

And add something like the following.
"This also generally reduces the possibility of coordinated minority groups getting more than 1 ballot slot per category."

#528 ::: J Thomas ::: (view all by) ::: May 16, 2015, 01:05 PM:

@524 junego

PROPOSED
If 2 works are tied on points and nominations both they challenge the work with the next lowest points. At the challenge, if their number of nominations don't exceed that work, both are eliminated. If their nominations exceed the challenged work, that work is eliminated. If the tied works win, after the recalculation the tied works will either no longer be tied or will still be tied and will challenge the next lowest work.

We discussed this a little bit, before. I'll say more now.

Without slates, this works maybe a bit better than discarding both. After awhile either they will both lose, or they will stop being tied and things go back to normal, or they stay tied while they work their way to the top -- if they win against the #5 then presumably they become tied for 5th place. Maybe you'd want to run them against #4 and if they win then #4 is gone and they become #4 and #5. It's better for them and worse for their competitors, no big deal. And it would be very very rare.

With slates, it is not so rare. There's a good chance that people who vote slates will make some mistakes and not all the slate nominees will get the same number of votes. But it only takes two, and so there are 10 chances for a tie, with the ten different combinations of 5 nominees.

If it's the top two slate nominations that are tied, then there is a small chance they will lose to the top nonslate candidate if it has enough shared votes to compete against them, and if it has more votes than they do.

Otherwise, if there are four or more nonslate works that have more than half their points, they will be tied for #5. If there are only three they will be tied for fourth place.

If both tied works were discarded, then in this particular case the slate would lose out when they collided. This rare event would make a tiny step toward balancing the extra slate items that win because the nonslate votes are so scattered. Remember that they deserve to win one when they have 20% of the vote, but with 15% of the vote they will often win two final places, and this gives them a slight chance to not get any.

The case of tied slate works that have fewer votes than some other slate work doesn't make as much difference. The other work will eliminate them if there are enough nonslate works that have scores above 1/3 its number of votes. Otherwise they get 3 slate wins.

It doesn't really make a lot of difference. It lets us be nice to the very rare tied nonslate nominations, and it lets us be even nicer to the somewhat-rare tied slate nominations.

I don't see that it's worth doing, and the net result looks negative to me, but if people really wanted it I'd go along. It's a small effect, not very important except possibly on principle, about fairness. I think the argument would be that given the choice, we try to be nice to a few specific people while hurting everybody else a little, rather than be mean to a few people to help everybody else a little.

#529 ::: colin roald ::: (view all by) ::: May 16, 2015, 02:06 PM:

junego @527:

I take your point, but on the flip side, being too specific about a current issue can make the proposal sound too specifically tailored to today's circumstance. People may be rightly suspicious of an overreaction. This is not an "anti-Puppy" proposal, and I don't think we want to make it sound like it is.

And I might as put the Commentary back together and suggest a couple tweaks to the wording again while I'm here:


Commentary: In the past we have simply counted the number of nominations and the top five works were put on the final ballot. Because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

We regard it as destructive to the tradition of the Hugos if a minority of members using coordinated strategic voting is able to overwhelm the collective input of all others who simply nominate their own diverse preferences.

Using this proposed system, fandom isn't penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate anything you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don't, that's okay - when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

It is an important goal of this proposal specifically not to disenfranchise anyone. This system actually increases the chance that one of the works you nominate will make the final ballot, while decreasing the chance that more than one will.

It is to be emphasized that this proposal does not change the nominating ballot from the perspective of Worldcon members: they still list one to five unranked works in each category they choose, just as they have previously.


Under this system, looking at computer-generated elections based on historical data¹, it seems that if a work has one of the top 5 vote tallies, it will win about 90% of the time. That means that in the absence of slate voters, the results of this system will be 90% the same as historical results. It also means that unless you have near-perfect knowledge of exactly how everyone will vote, the additional advantage to be gained from coordinated strategic voting is no more than about 10% better chances of getting the desired result, and even that is only true if the result is reasonably popular already. Given the chances of a "no award" backlash in the following round, a less than 10% chance of an advantage in the nominations is probably not worth the work of coordinating a strategy.

¹ We made 1300 simulated elections, generated by resampling ("bootstrapping", in statistical parlance) from the 1984 elections. (100 simulations from each category). The simulated elections had only 100 ballots each, so these results are likely to be conservative; that is, if anything, they will exaggerate the possibility of unusual results.

#530 ::: Keith "Kilo" Watt ::: (view all by) ::: May 16, 2015, 02:48 PM:

[FORMAL PROPOSAL]

Sitting at my daughter's ice skating practice...

One important stylistic factor we need to keep in mind is that I think the commentary section needs to be brief and therefore very tightly written. Take a look at the proposals that have already been submitted on the business meeting site, and you'll see what I mean. I don't think we need to be minimalistic, but I think we need to use the commentary to make our case convincingly and save the really detailed explanations for the meeting itself, if required.

It's like when my astro students are presenting their research projects. You don't put the kitchen sink in your presentation, but that doesn't mean that -you- don't have to know the topic in minute detail. So while we need to have all these persuasive arguments worked out, they don't necessarily need to be in the commentary.

Just my thoughts,
Kilo

#531 ::: Brian Gibbons ::: (view all by) ::: May 16, 2015, 02:48 PM:

As was discussed upthread, I think the way this proposal is worded is slightly flawed. It's coming from the wrong perspective.

"Each member gets a single vote." No, they don't. Members don't get votes. Members don't allocate their votes. Members aren't doing anything under this section.

Members submit their nominations, and that's under a completely separate section (3.7.1). This section is all about telling the administrator how to tally those nominations.

Also, the "two (2)" language isn't required (see, e.g., Sections 2.4, 2.8, and 3.8.1), and really comes across to me as trying too hard to look all official and lawyerly. It's not even recommended for lawyers in modern practice.

I'm also not sure why we would want to leave a blank 3.8.1 and make a new 3.8.8. Striking out and replacing 3.8.1 seems a much better option.

So, here would be my suggestion for wording:

Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed. determined by the following multi-round process:

3.8.1.1: In each round, each non-eliminated nominee shall be assigned points based on the number of ballots on which they appear and how many nominees on those ballots have not yet been eliminated: For each ballot which contains a single non-eliminated nominee, that nominee shall receive one point; for each ballot containing multiple non-eliminated nominees, each of those nominees shall receive a fraction of a point equal to one divided by the number of non-eliminated nominees on that ballot.

3.8.1.2: After points have been assigned to all nominees, the two nominees with the lowest point totals for that round (or more than two, in the event of a tie) shall be compared to each other, and the compared nominee which appears on the least number of ballots shall be eliminated. If there is a tie as to the number of ballots on which the compared nominees appear, then both compared nominees (or all, if more than two are being compared) shall be eliminated. Should an elimination of tied nominees result in fewer remaining nominees than the number to be listed, then no nominees will be eliminated and all tied nominees will be listed on the final ballot.

3.8.1.3: After a nominee is eliminated, it is treated as though removed from all ballots on which it was listed, for purposes of determining how many points the non-eliminated nominees on that ballot receive.

3.8.1.4: Should a nominee decline nomination, be declared ineligible or otherwise be removed from the final ballot, and should a Committee, in its discretion, choose to add nominees to the final ballot to replace the removed nominee, it may do so by using the process described in this Section to determine who the nominees listed on the final ballot would have been with the removed nominee(s) eliminated at the start of the process, and then adding to the final ballot all nominees who would have appeared on the final ballot under that condition, even if this requires listing more nominees than described in 3.8.1. This procedure may only result in the addition of nominees, not the removal of otherwise-eligible nominees.

--------------------

I'm sure there's room for improvement. There are a few redundancies in here (e.g., specifying one surviving nominee gets one point; specifying that a rerun of the final ballot can't delete nominees) that make it come across as less elegant to me, but are probably necessary to increase the likelihood of immediate comprehension.

I haven't included 3.11.4 here, because I think that's worth a separate discussion.

#532 ::: Kevin Standlee ::: (view all by) ::: May 16, 2015, 03:57 PM:

I agree that you want extensive commentary. Commentary included with a proposal becomes part of the "legislative history," giving people who come later an opportunity to understand what the people who proposed something meant.

It is unfortunately rather common for people to only read the bare technical rules without comprehending that there is an entire body of practice and commentary built up around them. This is true of "mundane" law as well as what we do in WSFS; witness the rather wooden, context-less readings of portions of the US Constitution for examples.

#533 ::: Keith "Kilo" Watt ::: (view all by) ::: May 16, 2015, 04:00 PM:

Kevin@532:
OK excellent. That actuality makes the job much easier. Would you also include the FAQs we've written?

K

#534 ::: Steve ::: (view all by) ::: May 16, 2015, 05:05 PM:

I coded up a webpage to let people visualize how SDV-LPE works. It's at: http://www-users.cs.umn.edu/~damer/hugos/SDVLPE.html
I'd appreciate any feedback people have - especially bugs. I think it accurately implements what is outlined in the proposal, but then I never think that code I write has bugs until someone points them out to me. There isn't much in the way of error checking, but if something goes wrong you can reload the page to start over.

#535 ::: David Goldfarb ::: (view all by) ::: May 16, 2015, 05:37 PM:

Steve: I didn't notice any specific errors, but I will say that your interface is a bit obscure. In particular, when I clicked "Load Votes", I couldn't at first make head nor tail of the chart that appeared. Possibly you could put line numbers next to the nominating ballot data? Or at least put in some sort of explanatory note. Also, when you put a new entry in the activity log, you really really need it to have a trailing newline. (I.e.: separate log entries with a line of whitespace.)

I don't think I could swing a full sponsorship for someone, but I'd happily kick in $10 or so to some sort of crowdfunding for it.

#536 ::: felice ::: (view all by) ::: May 16, 2015, 05:46 PM:

Steve @534: "I coded up a webpage to let people visualize how SDV-LPE works. It's at: http://www-users.cs.umn.edu/~damer/hugos/SDVLPE.html"

Nice work! It would help if you could order the Current Votes by points, and add text explaining that the numbers in the grey boxes are voter IDs, and the width of the box is determined by how many of the voter's nominations are still in the running. And add "box-sizing: border-box" to the css.

#537 ::: felice ::: (view all by) ::: May 16, 2015, 06:18 PM:

[FORMAL PROPOSAL]

Keith "Kilo" Watt @501: "Short Title: Change to Hugo Nomination System"

That's really not very informative, particularly since there are already two other change to the Hugo Nomination System on the agenda, and are likely to be more. Can we stick to "Short Title: SDV-LPE (Single Divisible Vote with Least Popular Eliminated)"?

"3.8.8.1: In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared."

We should probably be more specific about how many works should be compared. How about "all works with the least or second-least total points will be compared (at least two works, but more if there are ties for least points and/or second-least points)"? The current wording isn't clear on what happens when there's a tie for least - it could mean only more than two if there's a three-or-more-way tie for least.


Keith "Kilo" Watt @533: "OK excellent. That actuality makes the job much easier. Would you also include the FAQs we've written?"

It might help to look at the New Business section at http://loncon3.org/wsfs-business.php for examples of more detailed commentary sections.


junego @527: "So, no statements about reducing the power of coordinated minority groups at all? Why should anyone want to go to the trouble of changing the constitution if it's not going to help with the problem we've got?"

I think that's covered by "This system increases the chance that one of the works you nominate will make the final ballot, but decreases the chance that more than one will." The problem of coordinated minority groups is that they currently get too many of the works they nominate on the final ballot. But that sentence was intended as an introductory summary of what SDV-LPE does; there's certainly room to elaborate afterwards.

#538 ::: colin roald ::: (view all by) ::: May 16, 2015, 06:35 PM:

Steve @534:

That's really cool. For educational purposes it would help a lot to highlight what's *going* to happen before you hit the Eliminate button -- otherwise you end up with a "well, *something* just happened, but I'm not sure what" reaction.

Like felice@536 suggests, it would help a lot to sort the current candidates by points, and also to explicitly list the current points. Then you could highlight the works that are going to be compared (usually the bottom two, but sometimes more), and an annotation in red next to the one that's going to be eliminated that says something like "Next to be eliminated: has same approvals (3) but lower point total".

It would also look sharper if you changed the code for your chart elements to <span class="vote-element" style="width: 60px;">10</span>

And added a stylesheet

.vote-element {
display: inline-block;
box-sizing: border-box;
background-color: rgb(177, 214, 234);
border: 1px solid gray;
line-height: 1.2;
}

But don't take this as complaining -- this is a cool thing.

#539 ::: Seth ::: (view all by) ::: May 16, 2015, 06:56 PM:

#493 I left out the number of ballots each appeared on, under the assumption that those numbers were such that the eliminations I suggested were the ones that would happen.

#540 ::: Seth ::: (view all by) ::: May 16, 2015, 07:11 PM:

junego@515: I think it would be better (and more accurate) to specify that under this proposal, a bloc can get a number of finalists proportional to the percentage of nominators in the block. 20% can get one item, 40% can get 2 items, 60% can get 3 items.

#541 ::: J Thomas ::: (view all by) ::: May 16, 2015, 07:29 PM:

@540 Seth

junego@515: I think it would be better (and more accurate) to specify that under this proposal, a bloc can get a number of finalists proportional to the percentage of nominators in the block. 20% can get one item, 40% can get 2 items, 60% can get 3 items.

Except that isn't true. It's possible for 10% to get 3 nominations, or 4, just less likely than with approval voting.

#542 ::: felice ::: (view all by) ::: May 16, 2015, 07:29 PM:

Seth @540: "I think it would be better (and more accurate) to specify that under this proposal, a bloc can get a number of finalists proportional to the percentage of nominators in the block. 20% can get one item, 40% can get 2 items, 60% can get 3 items."

It's not quite that simple, though. Some non-slate nominators will have all their nominations eliminated early on (because they only nominated works not many other people nominated), for example, which will increase the representation of the slate. The number of slate finalists will tend to be significantly closer to proportional than under the current system, but we can't give exact figures.

#543 ::: J Thomas ::: (view all by) ::: May 16, 2015, 10:04 PM:

I have an animation to help show how the voting system works. You can click on my name.

http://www.glowscript.org/#/user/J_Thomas/folder/hugo/program/Nominations

It's currently less flexible than I'd like, it cycles through the same steps and your only choice is to decide when to click forward to the next step. I'd do better to lay it out like a webpage so you can click on any part of it when you like, etc. But I should get some feedback before I put too much work into it.

The language is supposed to work on all modern computers with modern graphics cards, but it doesn't. I hope it works on enough to be useful.

It doesn't quite match the current leading version of the voting system; it would take some work to get it to.

#544 ::: Brian Z ::: (view all by) ::: May 16, 2015, 10:35 PM:

@505 colin roald

"Next year, I virtually guarantee there will be some kind of Justice Kitten slate as a reply to the Puppies"

First, more old and new members must and will nominate more vigorously, especially in short fiction categories. That's a given at this point. I think.

If something like half the voting members then proceed to organize themselves to attack and subvert the system a la Game of Slates, of course that is not going to lead to an optimal outcome. The award depends on most voters participating in the spirit of the award. If they don't, the award will not be worth much. This new proposal could lead to Game of Slates even more quickly: it ensures each faction gets something they want. Lowest common denominator. I would almost rather see rival slates battling each other on their merit.

But why don't we see if we can save the true spirit of the award first, before capitulating.

#545 ::: JJ ::: (view all by) ::: May 16, 2015, 10:48 PM:

Brian Z., : "But why don't we see if we can save the true spirit of the award first, before capitulating."

Capitulating to what? To the Puppies' repeated insistence that we should just accept their forced nominees as valid despite the obvious lack of quality, and not use "No Award"? Or to their repeated insistence that we should not try to change the nominating process?

I don't think there are very many people who are interested in or willing to capitulate to either of those things.

#546 ::: Seth ::: (view all by) ::: May 16, 2015, 11:03 PM:

#541 and #542: True, I didn't mean to say that a slate supported by 20% will always get exactly one finalist. It deserves to get exactly one finalist, and will typically get around one finalist (if no non-slate nominators like any of the slate's candidates). It will do a much better job of accomplishing that than the current method.

#547 ::: P J Evans ::: (view all by) ::: May 16, 2015, 11:10 PM:

544
But why don't we see if we can save the true spirit of the award first, before capitulating.

That's what we're doing here. The true spirit is things readers-who-are-members-of-Worldcon think are 'Oooh shiny!'

#548 ::: Brian Z ::: (view all by) ::: May 16, 2015, 11:15 PM:

@546

Do you expect many voters will follow slates under this new system? How many slates?

#549 ::: Steve Halter ::: (view all by) ::: May 16, 2015, 11:30 PM:

Brian Z@548:The general idea is to render slates moot. In the past slates were not a problem as they didn't happen. The new voting system will act to remove the leverage of minority slates.

#550 ::: Brian Z ::: (view all by) ::: May 16, 2015, 11:34 PM:

@549 render them moot, or help ensure they'll succeed in getting something on the ballot?

#551 ::: J Thomas ::: (view all by) ::: May 16, 2015, 11:56 PM:

@550 Brian Z

render them moot, or help ensure they'll succeed in getting something on the ballot?

If you have enough people to run a successful slate, you don't need a slate to get something on the ballot. Get everybody to vote for one thing, and then they have four more votes to vote for anything they like.

It cleans up the evidence, too. If you have a slate, a cluster analysis will reveal it to anybody who has the data and the expertise. But if a bunch of people happen to vote for one candidate, statistics from the ballots won't say much about that.

#552 ::: Brian Z ::: (view all by) ::: May 17, 2015, 12:11 AM:

@551 Yes. But in addition to disliking slates, traditionalists object to all rallying of Hugo voters through all campaigns. I'm wondering whether this proposal further incentivizes the crass campaigning that has been increasing ever since the internet hive mind was first awoken. (Future fan historians may end up pointing to that behavior as a major cause of dilution of the award.)

#553 ::: Keith "Kilo" Watt ::: (view all by) ::: May 17, 2015, 12:32 AM:

Brian:

The ultimate goal is to prevent any one group from keeping everyone else off of the ballot. That's really our -only- goal. Yes, it will also make slates basically pointless, because you don't need a slate to get a single popular work on the ballot.

You don't think the nomination process should change. We get that. But you are talking to a group of people in a thread expressly created to change the nomination process. We aren't likely to change your mind, I understand that. But you aren't likely to change ours either, given the whole purpose of this thread. There are a great many people who -do- think the process needs to be changed. We think this is the best way to meet those needs.

Kilo

#554 ::: colin roald ::: (view all by) ::: May 17, 2015, 12:49 AM:

Brian Z, I think you're probably actually a much sharper guy than the relentlessly thick-headed one you're presenting yourself as here. Do you seriously think we're trying to "help ensure" slates can get something on the ballot, or is that just the latest convenient way to pretend to misunderstand us?

#555 ::: Brian Gibbons ::: (view all by) ::: May 17, 2015, 12:58 AM:

Colin Roald @505: Next year, I virtually guarantee there will be some kind of Justice Kitten slate as a reply to the Puppies

I take a statement of absolute belief that there will be some sort of SJW slate next year as akin to stating a certainty that this winter's snow will taste like chocolate.

It belies a view of the world entirely at odds with my own, and indicates premises so far removed from what I consider reality that I can only shrug and suppose that we'll find out in due course. It's not really worth arguing over, and if it does turn out that you're right then, hey, free chocolate.

I would have an easier time believing that a group of actual cats will gather together, form a paramilitary organization and/or musical band, and head out into the streets to right wrongs under the name Justice Kittens, than I would in believing that "SJWs" could manage to get a meaningful number of people organized and all pointed in the same direction long enough to put together a counter-slate.

What I expect from next year is that there will be an awful lot more people nominating. This will mean that the categories that people care about will largely be Puppy-free, and ones that usually receive less attention will have at least a nominee or two that's sufficiently promoted (self- or otherwise) to stand out enough from the crowd to make it on the ballot even with Puppies still around. Presuming they are, of course. I would give at least 50/50 odds on VD simply declaring victory and moving on, now that he won't have the advantage of surprise, and the Sad variety are likely to retreat back into irrelevance as public scrutiny means they have to actually live up to their talk of democracy and public participation.

Oh, there will no doubt be people promoting works (their own or otherwise) more heavily, and I'd give odds on at least a dozen people (none of whom will be particularly high profile or with a meaningful following) trying to push different recommendation lists and/or slates. Some unified "SJW slate" on the level of the Puppies, however, is simply a laughable concept.

... none of which is necessarily that relevant to the SDV-LPE discussion. I think that this is a system that's more robust and will produce better results that the simple first-past-the-post nomination system being used now, and so hope that it ends up being adopted despite the fact that I expect the Puppies to become nothing more than a historical curiosity to explain to future Hugo historians what the asterisk next to this year's awards means.

#556 ::: Brian Z ::: (view all by) ::: May 17, 2015, 02:34 AM:

@553, 554

I fear some may be hearing things I'm not saying. Please indulge me in one last summation:

We stand at crossroads. In one direction, Hugos remain a charmingly retro fannish tradition in which a community of avid readers come together to identify the works most widely acclaimed as best of the year.

The other road leads to more internet tribalism and us vs. them mentality, twitter feuds, bestelling and wannabe bestselling authors setting themselves up as glorified thought leaders and opinion makers, cliques that may include some of the cleverest people on the planet organizing to hack a process that is by its very nature wide open, uncountable public and secret single work campaigns and slate campaigns, entrenchment of a culture in which fans are perpetually offended when they don't get what they want, and enough eligibility posts to sink the Titanic.

I'd like to see us go with charmingly retro.

Failing that, the Brave New World is at least interesting. We can do our best to help shape the voting culture that emerges. But it is going to be beyond anyone's control.

If nearly or more than half of the voters were to subscribe to multiple slates, then the game has probably already changed so much that rearguard action is pointless.

Ditto if multiple factions start going to battle with each other by rallying around favorite authors/works and driving other deserving potential nominees off the ballot. We don't "win" by getting a nominee we like most on the ballot. We "win" by giving a meaningful gift to an honored author representing the collective wishes of the members.

A new rule that does nothing but prevent a set of identical ballots from sweeping, period, is fine. If it were passed, I'd vote to ratify it.

Changing the goals to prioritize the "fairness" of everybody getting a thing they want over the spirit of fans reading widely and striving to reach consensus about what is excellent is not fine, in my view. It sends the wrong message, it will be that much harder to stay retro, and if the second road is taken will not make much difference and might make things worse.

#557 ::: P J Evans ::: (view all by) ::: May 17, 2015, 02:54 AM:

552
the crass campaigning that has been increasing

I think you need to be a little more explicit here, because the only really successful campaigns most of us have seen have been run by the juvenile canines.

(Authors saying that 'these are eligible' is not a campaign, nor is a recommended list.)

#558 ::: Laertes ::: (view all by) ::: May 17, 2015, 03:00 AM:

I'm honestly surprised that so few concern trolls have showed up.

#559 ::: Jeremy Leader ::: (view all by) ::: May 17, 2015, 03:07 AM:

Brian Z. @557: A new rule that does nothing but prevent a set of identical ballots from sweeping, period, is fine. If it were passed, I'd vote to ratify it.

Were you paying attention during the 2000+ comments on this thread and its 2 predecessor threads, where some very smart knowledgeable participants attempted to do exactly that, and came to the conclusion that this proposal was the best way to achieve that goal? Are you asking that the discussion be restarted from the beginning, because the consensus it reached wasn't what you wanted?

It seems that you're the only person who thinks the current proposal doesn't support "the spirit of fans reading widely and striving to reach consensus".

I've seen no evidence that doing nothing as you seem to want will lead to the "charmingly retro" outcome you claim to want. Given that the SP/RP supporters have said they don't want that, and their actions this year have shown that under the current system they have in fact prevented the Hugos from operating that way...

On the other hand, I've also seen no evidence that the SDV-LPE proposal will lead to the "Brave New World" you fear. Since SDV-LDE provably makes slates less effective, why do you believe SDV-LDE will lead to more slates, rather than fewer?

If you want to discuss other ways to improve the Hugos, there have been 4 threads so far about the Hugos that haven't been dedicated to coming up with and refining a proposal to improve the nomination mechanism; perhaps you should propose how you think the "crass campaigning" can be reduced on one of those threads?

(on preview, Laertes @558 was far more succinct than I!)

#560 ::: felice ::: (view all by) ::: May 17, 2015, 03:20 AM:

Brian Z @556: "We stand at crossroads. In one direction, Hugos remain a charmingly retro fannish tradition in which a community of avid readers come together to identify the works most widely acclaimed as best of the year."

There is no "coming together" with the current system. People nominate what they like, and the biggest tiny minorities win the finalist spots. Last year a whopping 23.1% of nominators backed the most popular novel; that means the other 77.9% didn't express an opinion on it. The least popular novel finalist had no support from 93.9% of nominators. 90.9% of nominators gave no support to the most popular short story.

"Changing the goals to prioritize the "fairness" of everybody getting a thing they want over the spirit of fans reading widely and striving to reach consensus about what is excellent is not fine, in my view."

There's never been any consensus. SDV-LPE aims to represent a range of opinions on what's excellent rather than just one minority's opinion, which is a good thing irrespective of whether or not there are any slates involved. And it does nothing to encourage "the other road" any more than the existing system; on the contrary, it removes the need for alternative slates to counter the puppies.

#561 ::: Brian Z ::: (view all by) ::: May 17, 2015, 03:27 AM:

No, I didn't say that this proposal would be responsible for leading us down one path or the other.

I intended that summation to be my final word on this thread - so thank you all for your additional questions (and accusations) but I think I've said my piece. Best of luck with your proposal, whatever form it takes.

#562 ::: JJ ::: (view all by) ::: May 17, 2015, 06:00 AM:

I would be able to contribute $20 to sponsor sending Keith or Jameson to Sasquan. Is someone in a position to set up a Kickstarter / indiegogo / GoFundMe campaign? (I am not, unfortunately.)

#563 ::: Jameson Quinn ::: (view all by) ::: May 17, 2015, 09:12 AM:

Keith has told me that he's interested in going, but not interested in fundraising for himself. I'm going to set up a crowdfund campaign for myself this afternoon, when I have time.

#564 ::: Jameson Quinn ::: (view all by) ::: May 17, 2015, 09:13 AM:

Cheradenine, if you are interested in being in on the crowdfund, please contact me today.

#565 ::: Shane ::: (view all by) ::: May 17, 2015, 09:48 AM:

I was going to say not Kickstarter, because it only pays if fully funded. But it might be that's what we want - decide the minimum you need to make it affordable, and if we can't reach it we can't.

I expect to subscribe, but I'd like to read your costings and reasons.

#566 ::: J Thomas ::: (view all by) ::: May 17, 2015, 10:18 AM:

In case this got lost in the argument with Brian....

I have an animation to help show how the voting system works. You can click on my name.

http://www.glowscript.org/#/user/J_Thomas/folder/hugo/program/Nominations

I'd like to get feedback whether something along these lines is worth doing.

#567 ::: Shane ::: (view all by) ::: May 17, 2015, 10:38 AM:

For Brian Z, from Jameson Quinn @ 219

The point of this system (SDV-LPE or whatever we call it) is to try to ensure that the eventual Hugo winner is excellent. The reason that the nominees should represent a variety of fans is not to make them happy, but because by representing a variety you avoid making the same mistake 5 times.

#568 ::: P J Evans ::: (view all by) ::: May 17, 2015, 11:14 AM:

Shane
Having a variety of nominees is what you get when there are lots of excellent works available; slates kill that. Look, for example, at the three non-slate novels: that's the kind of variety that we want.

#569 ::: junego ::: (view all by) ::: May 17, 2015, 11:58 AM:

J Thomas @ 566

http://www.glowscript.org/#/user/J_Thomas/folder/hugo/program/Nominations

That looks pretty cool, JT. The basic visual concept really gives a good graphic of how the system works. Obviously, a few more refinements would be great, like text to explain each event and maybe tie it to the sections of the proposal, slow or stop as the votes travel from the ballot to the nominees to show how many points are being dropped in, etc.

As a proof of concept I'd say "Do more." Good job.

#570 ::: junego ::: (view all by) ::: May 17, 2015, 12:53 PM:

Brian Gibbons @ 531
As was discussed upthread, I think the way this proposal is worded is slightly flawed. It's coming from the wrong perspective.
"Each member gets a single vote." No, they don't. Members don't get votes. Members don't allocate their votes. Members aren't doing anything under this section.
Members submit their nominations, and that's under a completely separate section (3.7.1). This section is all about telling the administrator how to tally those nominations.

Good point. I agree and support this approach. We should emphasize that there is NO change in how members nominate, only in how the nominations are tallied.

I suggest the following modification: I think that it makes more sense to leave the current 3.8.2 thru 3.8.7 as is. There are important instructions there that logically come before describing the new tally process. I've modified 3.8.1 (shown in bold) and will work on combining your, my and the original verbiage in a separate post. (Doing this on a tablet is a pain!)


Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations determined by the multi-round elimination process described in 3.8.8. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

#571 ::: junego ::: (view all by) ::: May 17, 2015, 12:55 PM:

Brian Gibbons @ 531
As was discussed upthread, I think the way this proposal is worded is slightly flawed. It's coming from the wrong perspective.
"Each member gets a single vote." No, they don't. Members don't get votes. Members don't allocate their votes. Members aren't doing anything under this section.
Members submit their nominations, and that's under a completely separate section (3.7.1). This section is all about telling the administrator how to tally those nominations.

Good point. I agree and support this approach. We should emphasize that there is NO change in how members nominate, only in how the nominations are tallied.

I suggest the following modification: I think that it makes more sense to leave the current 3.8.2 thru 3.8.7 as is. There are important instructions there that logically come before describing the new tally process. I've modified 3.8.1 (shown in bold) and will work on combining your, my and the original verbiage in a separate post. (Doing this on a tablet is a pain!)


Moved, to amend section 3.8 (Tallying of Nominations) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations determined by the multi-round elimination process described in 3.8.8. If there is a tie including fifth place, all the tied eligible nominees shall be listed.

#572 ::: colin roald ::: (view all by) ::: May 17, 2015, 01:28 PM:

J Thomas @566:

It's a promising start, but I fear still pretty opaque to anyone who hasn't carefully read the proposal. I do think something along these lines is worth doing, if it's taken to the point where it's clear to someone brand new to it (and who is not an election theorist).

Suggestions:

(1) I think it would help to have the ballots always shown, maybe in a split-screen kind of view. I think you want to animate the changes to the ballots when a work is eliminated, also.
(2) I think listing the works vertically and the scores horizontally (like the way Steve's simulator does it) would be a big improvement. You could write longer names for works (at the moment I'm not sure it's clear to a newcomer that "A", "B", "C" etc are meant to be the names of proposed stories), and you could write the scores as a simple decimal instead of the awkward improper fractions.

I'm not familiar with Glow, but it doesn't look too bad. If I have some time this evening I might be interested to poke a bit and see if I can help, if you like. Do you have this in version control anywhere? Poking around glowscript, I don't see anything, but maybe it's hiding.

Steve@534: I'll also volunteer to help work on your simulator, if you would like to share.

#573 ::: junego ::: (view all by) ::: May 17, 2015, 02:46 PM:

[PROPOSAL LANGUAGE]

OK, here's my shot at combining the original proposal, Brian Gibbons suggestions and my own. It is way too tedious trying to do strike through and italics on a tablet. Cutting and pasting loses all formatting. So I've just rewritten Section 3.8.8 and added 3.8.9 (Brian's 3.8.1.4, which covers withdrawals and I thought needed its own para.)

My rewrite of 3.8.1 is @
#571 ::: junego ::: (view all by) ::: May 17, 2015, 12:55 PM:

Here 'tis
*******************************************************
3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees on the ballot. If there is one nomination it gets 1 point, two nominations get 1/2 point each, three get 1/3 point each, etc. The total number of nominations for each separate work in the category on all ballots are totaled, each nomination counting as one tally. Points for the nominees change as each elimination round is started, the nomination tally for each nominee does not change.

3.8.8.1 Elimination rounds are as follows: At the beginning of the first round, all points for each work nominated are totaled. The two works with the least total number of (fractional) points will be compared. Of those works, the one that appears on the fewest ballots (the lowest nomination tally number) will be eliminated from contention.

3.8.8.2 Subsequent rounds begin by reallocating points as follows: The work eliminated in the previous round is treated as though it had never been nominated by the ballots in which it appeared. For example, if a ballot had 4 nominations at 1/4 point each in that category and one is eliminated, the remaining nominations are now allocated 1/3 point each. All points for all remaining nominees are recalculated and the next round proceeds as in 3.8.8.1. These rounds continue until there are 5 remaining nominees (except as described in 3.8.8.4 and 3.8.9).

3.8.8.3 In the event that two or more works are tied for lowest points and nomination tallies, the tie will be broken by looking back to previous rounds...[SOMEONE NEEDS TO DESCRIBE THIS METHOD, I DON'T FEEL I CAN DO IT JUSTICE]

3.8.8.4 If (due to a tie) elimination would reduce the number of remaining nominees to fewer than 5, then none of the works should be eliminated and all remaining works should appear on the final ballot.

3.8.9 Should a nominee decline nomination, be declared ineligible or otherwise be removed from the final ballot, and should the administrators choose to replace the removed nominee(s), they may do so by using the process described in 3.8.8 to determine who the final nominees would have been without contending with those removed. The administrators may then add to the final ballot all nominees who would have otherwise qualified, even if this requires listing more nominees than described in 3.8.1. This procedure may only result in the addition of nominees, not the removal of the previously qualified nominees.

#574 ::: J Thomas ::: (view all by) ::: May 17, 2015, 03:42 PM:

3.8.8.3 In the event that two or more works are tied for lowest points and nomination tallies, the tie will be broken by looking back to previous rounds...[SOMEONE NEEDS TO DESCRIBE THIS METHOD, I DON'T FEEL I CAN DO IT JUSTICE]

This is an argument for not doing it that way.

It's hard to explain.

Should we go back until the latest time they're different, and throw out whichever one had the lower score then?

What about going back to the beginning and throwing out the one that had the lower score more often?

What about going back to the beginning and throwing out the one which had the lower average score?

It isn't simple, it's hard to explain, there are multiple ways we could do it with no indication which is fairer. It doesn't get us anything useful.

The two which are tied for last place right now, are in fact tied for last place. They are the two which most deserve to be eliminated. It's possible if we gave one of them a reprieve it would do better and go on to win later, but that's true of a lot of the ones that get eliminated. That's true of any of the five with the most votes, which will all win unless one of them gets eliminated and doesn't get a reprieve.

I don't see the advantage of doing it.

Except for one minor advantage. If the last two slate members go head to head and tie, it would be fairer to the slate if they don't eliminate each other and have no slate members left among the finalists. No, wait, if the slate members are tied at the end then they will have been tied all the way through. Either they'll both be eliminated anyway, or they'll both be finalists. So not even that dubious advantage is real.

I must have missed the advantage somewhere.

#575 ::: Steve Halter ::: (view all by) ::: May 17, 2015, 04:20 PM:

Brian Z@550:

render them moot, or help ensure they'll succeed in getting something on the ballot?

Since you left @561, this is probably also moot, but helping to ensure slates get something on the ballot is clearly not the intent of the proposal.

Wind attacks the sand
Balance of voting impinged
Sand abides in dunes

#576 ::: J Thomas ::: (view all by) ::: May 17, 2015, 04:31 PM:

@572 colin roald

(1) I think it would help to have the ballots always shown, maybe in a split-screen kind of view.

That ought to be possible. It's a complication but probably not that difficult.

I think you want to animate the changes to the ballots when a work is eliminated, also.

That's probably harder for me, but I think possible.

(2) I think listing the works vertically and the scores horizontally (like the way Steve's simulator does it) would be a big improvement. You could write longer names for works (at the moment I'm not sure it's clear to a newcomer that "A", "B", "C" etc are meant to be the names of proposed stories),

I can write a description at the start for that. I let the layout and the code get co-adapted and did what looked simplest for both, and so changing it would require me to change my thinking around. That might be better done by someone else.

... and you could write the scores as a simple decimal instead of the awkward improper fractions.

That's valuable. I thought of them as more obvious to people who weren't math-inclined, and I didn't bother to reduce the fractions because there was always something higher priority. But if decimal approximations are clearer, that's an easy fix once it's laid out sideways to make room for them.

I'm not familiar with Glow, but it doesn't look too bad. If I have some time this evening I might be interested to poke a bit and see if I can help, if you like.

You'd be very welcome! It's very easy to use as long as you don't do anything complicated. You get a choice between javascript, Coffeescript, and Pyjeon. I find javascript excruciating. Pyjeon is just like Python that runs in javascript, except when it isn't. Lots of Python is missing including the min function, the sort function, sets, inheritance is subtly different, etc. It's all easy and obvious as long as you don't try to do anything that isn't easy and obvious.

Do you have this in version control anywhere?

My coding style makes me ashamed to show anybody my version control. I try out big variations, trying to do that mostly early in the process but also later when I find I've gotten to a dead end. I code faster than I can write workable specs without coding.

I could give you something that starts with the current version, but you could pick that up from the site about as easily.

Poking around glowscript, I don't see anything, but maybe it's hiding.

The site has no version control at all, but they keep a copy of your latest version for you. It's designed for visual demos. You can roll your own by installing the sources. If you want to putter around with code on their site, using Chrome gives by far better error messages. They host your code for anybody to look at, running their software, for free.


@569 junego

a few more refinements would be great, like text to explain each event

I can do more of that, but it's probably better to stretch it out into a website with a series of canvases you can click on to see animations. I did it with one canvas and changing text because that was what I got working quicker.

and maybe tie it to the sections of the proposal,

Interesting idea! It's still a moving target now, but....

slow or stop as the votes travel from the ballot to the nominees to show how many points are being dropped in, etc.

I set the slider so it can go slower whenever you want, down to almost stopped. I'll look at putting in a stop button, but I don't know ahead of time how to do that which leaves me wanting to do easier things first.

Thank you for the feedback!

#577 ::: felice ::: (view all by) ::: May 17, 2015, 05:02 PM:

J Thomas @574: "Should we go back until the latest time they're different, and throw out whichever one had the lower score then?"

Yes.

"It's possible if we gave one of them a reprieve it would do better and go on to win later, but that's true of a lot of the ones that get eliminated"

No, it's not; getting a reprieve isn't a random act of benevolence by the administrator, it's a product of sharing a ballot with another work that gets eliminated, and obtaining extra points from the eliminated work. If you randomly decide not to eliminate a work in one round, then the next round will be identical and it will be eliminated anyway. If you randomly eliminate the more popular work instead, that's obviously unfair. There is no way to give a reprieve to a non-tied work that should get eliminated.

"Except for one minor advantage. If the last two slate members go head to head and tie, it would be fairer to the slate if they don't eliminate each other and have no slate members left among the finalists. No, wait, if the slate members are tied at the end then they will have been tied all the way through. Either they'll both be eliminated anyway, or they'll both be finalists. So not even that dubious advantage is real."

Not necessarily. What if they're tied for 150 nominations each, but 148 people nominated both, and four other people each nominated just one of them (either because they were following the slate but chose not to stick to it exactly, or they weren't following the slate at all and coincidentally overlapped with it)? Lookback at previous rounds will likely find a difference for those four nominators. Lookback won't help with 100% overlap ties, but such ties between potential finalists should be pretty rare (they're still a problem on those rare occasions, though). And it's not just slates that could be affected; I listed a bunch of finalist ties from the last couple of years in #499, any of which could theoretically have tied for points and been compared for elimination under SDV-LPE. We don't have the data, so we don't know for sure.

#578 ::: Cally Soukup ::: (view all by) ::: May 17, 2015, 05:15 PM:

Junego @573

[PROPOSAL LANGUAGE]

May I suggest that in 3.8.8 instead of:

3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees on the ballot. [etc.]

it say:

3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees in that category on the ballot. [etc.]

thus making it extra-clear that the point is per category, not per ballot? For a second when I read it, I read that a point would be divided equally among all nominees on the WHOLE ballot, and it took me aback. On a second reading, I understood it, but personally, I'd rather err on the side of stating the obvious in a document like this when there's even a small chance of ambiguity.

#579 ::: J Thomas ::: (view all by) ::: May 17, 2015, 05:52 PM:

@577 Felice

"It's possible if we gave one of them a reprieve it would do better and go on to win later, but that's true of a lot of the ones that get eliminated"

No, it's not; getting a reprieve isn't a random act of benevolence by the administrator, it's a product of sharing a ballot with another work that gets eliminated, and obtaining extra points from the eliminated work.

It might get points from the other work that happens to have the same number of nominations and points. It might not. If it does, the other would get the same points if we chose that one instead, because we found that the other had been ahead on points more of the time.

But then, it might get points if it beats its next competitor, that would never have faced it except you decided it should be spared. What's good for one is bad for the other. If it had been eliminated then the competitor who shares ballots with it would have gotten those points and might not have had to compete on the next round after all.

If you randomly decide not to eliminate a work in one round, then the next round will be identical and it will be eliminated anyway.

Yes, if you put it up against exactly the same competitor again.

If you randomly eliminate the more popular work instead, that's obviously unfair. There is no way to give a reprieve to a non-tied work that should get eliminated.

Yes, it's obviously unfair to eliminate the more popular work. I'm not sure how it's more fair to look back in history to find some tiny difference to use to eliminate one and not the other. I see that it looks that way to you. I just don't see it myself, and I haven't seen an explanation that made sense to me.

Is there some technical reason why it works out better to keep one and not the other, apart from fairness?

"Except for one minor advantage. If the last two slate members go head to head and tie, it would be fairer to the slate if they don't eliminate each other and have no slate members left among the finalists. No, wait, if the slate members are tied at the end then they will have been tied all the way through. Either they'll both be eliminated anyway, or they'll both be finalists. So not even that dubious advantage is real."

Not necessarily. What if they're tied for 150 nominations each, but 148 people nominated both, and four other people each nominated just one of them (either because they were following the slate but chose not to stick to it exactly, or they weren't following the slate at all and coincidentally overlapped with it)? Lookback at previous rounds will likely find a difference for those four nominators.

Well yes, that's possible. You might look back and find that at the very first round, one nominator picked slate work Alpha and also Evil Clowns of Gor, while another nominator chose slate work Beta, Evil Clowns of Gor, and also Evil Dentists of Gor, and both of the extra nominations were eliminated the first round, having gotten no other votes. So that difference means that slate work A is better than slate work B?

I can sort of see it if there's some important technical reason we must minimize ties, and other people won't let us flip a coin or go by first nomination date. In that case sure, find some random criterion you can base on the votes to say that one is a tiny tiny sliver of a defunct point better than the other. But what's wrong with ties that we have to go to such extremes?

And it's not just slates that could be affected; I listed a bunch of finalist ties from the last couple of years in #499, any of which could theoretically have tied for points and been compared for elimination under SDV-LPE. We don't have the data, so we don't know for sure.

There's on particular reason to expect two nominees that have the same number of nominations to also have the same number of points. It seems pretty unlikely, when the points start to add up. Unless they have the two lowest numbers of points they won't be compared, and won't be tied.

Without the data we can't be sure it wouldn't have happened, but it looks unlikely.

Anyway, it's very likely that finalists who tie wouldn't have been paired up for competition.

And what's wrong with the reasoning that if the rules are that we use SDE and they do get paired up for competition and eliminated, then that's how it's supposed to come out?

We have a collection of SDE voting systems that are pretty similar and get pretty similar results. What method should we use to choose among them? I like the simplicity of eliminating them both. (But then, I didn't like the simplicity of just using #6 if #1 got disqualified. I thought it was better to actually find out who won.) If it's basicly arbitrary, let's keep it simple. But if there's an important reason, I want to understand it.

#580 ::: junego ::: (view all by) ::: May 17, 2015, 06:14 PM:

Cally Soukup @ 578
Junego @573

[PROPOSAL LANGUAGE]

May I suggest that in 3.8.8 instead of:
3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees on the ballot. [etc.]
it say:

3.8.8 The nomination ballots for each category shall be counted as follows: Each ballot is allocated one point per category. That point is divided equally among all nominees in that category on the ballot. [etc.]

thus making it extra-clear that the point is per category, not per ballot? For a second when I read it, I read that a point would be divided equally among all nominees on the WHOLE ballot, and it took me aback. On a second reading, I understood it, but personally, I'd rather err on the side of stating the obvious in a document like this when there's even a small chance of ambiguity.

Excellent! Thank you, this is a great help. It can be difficult to proof your own writing because your brain knows what you *meant* to say. :^}

#581 ::: Jameson Quinn ::: (view all by) ::: May 17, 2015, 06:17 PM:

Personally, I think that debating back and forth about the tiebreakers is pretty much the epitome of bikeshedding. In more than half of the cases, a tiebreaker rule will never even be invoked; in the overwhelming majority of the remaining cases, it will not affect the final outcome; and in the overwhelming majority of the tiny remaining fraction cases where it does matter, it only matters in terms of deciding which of two essentially-equivalent works from the same voting slate ends up as a finalist. So really, who cares. Let's just decide on something and move on.

It would be inappropriate for me to argue the above, and then also say, "...so just do what I say." So you all know which option I prefer, but I'm not going to argue for that again here. I say: let Keith decide. Or, if he'd rather, he can decide how to decide, as long as there's a time limit. If he wants us to have a poll, so be it. If he thinks he knows best and he's just going to call it for tiebreaker X, I support him 100%, no matter what X is.

If spending extra time now on the decision helps us build support for the eventual proposal, then it's well worth it; but if we're all just arguing over the aesthetics of an aspect that will almost certainly never make a difference that matters, it's a waste of time. For me, I suspect it's more of the latter, and I say we should leave it up to Keith. If he thinks I'm being too impatient, I'll accept that answer too.

#582 ::: felice ::: (view all by) ::: May 17, 2015, 07:09 PM:

J Thomas @579: "It might get points from the other work that happens to have the same number of nominations and points. It might not. If it does, the other would get the same points if we chose that one instead, because we found that the other had been ahead on points more of the time."

True, but that's no reason to not give either a chance. If it was a work you nominated, would you rather a 50% chance of your nomination surviving, or a 0% chance? If it was two works you nominated (and it will be for some if not most nominators, if the tie matters) would you rather a 100% chance of one of your nominations surviving, or a 0% chance? If you were the author, would you prefer a 50% chance of your work becoming a Hugo finalist, or a 0% chance? For everyone directly affected, using a tiny difference to distinguish between then is better than eliminating both.

"Is there some technical reason why it works out better to keep one and not the other, apart from fairness?"

Not aside from fairness, no. But fairness seems like quite sufficient justification to me.

"You might look back and find that at the very first round, one nominator picked slate work Alpha and also Evil Clowns of Gor, while another nominator chose slate work Beta, Evil Clowns of Gor, and also Evil Dentists of Gor, and both of the extra nominations were eliminated the first round, having gotten no other votes. So that difference means that slate work A is better than slate work B?"

Not really, no, but keeping one is better than eliminating both, and it's an acceptable deterministic method of choosing one.

"There's on particular reason to expect two nominees that have the same number of nominations to also have the same number of points. It seems pretty unlikely, when the points start to add up. Unless they have the two lowest numbers of points they won't be compared, and won't be tied. Without the data we can't be sure it wouldn't have happened, but it looks unlikely."

Most of the time, yes, they'll have different points, but it's not that unlikely they'll be the same. By the time potential finalists are up for elimination, it's likely that everything else on the ballots that nominated the tied works will have been eliminated, in which case they'll definitely have the same number of points. And if some of them also nominated another finalist, the numbers are likely to be low enough that matching numbers aren't unlikely.

If two potential finalists in short story (A and B) are tied on 40 nominations each, and overlap by 20 nominations, it doesn't matter what else the overlapping ballots nominated, and of the 20 who nominated just A and the 20 who nominated just B, maybe 5 from each nominated another surviving work? Those don't seem like unrealistic numbers to me. A and B would be tied on 27.5 points or less, and could well be up for elimination against another potential finalist with relatively low points but higher nominations - one of the last two works on a slate would be an obvious possibility. I'd say it's better to have one slate work and either A or B as finalists than it is to have two slate works and neither A nor B.

In 1984, 69% of people who nominated one of the best novel finalists didn't nominate any of the others. 54% didn't nominate anything else in the top ten. Only 5% nominated more than two of the finalists. The chances of potential finalists which are tied on nominations being tied on points as well when they're up for elimination is not that low.

#583 ::: Seth ::: (view all by) ::: May 17, 2015, 07:20 PM:

Slight disambiguation:

3.8.8.2 Subsequent rounds begin by reallocating points as follows: The work eliminated in the previous round is treated as though it had never been nominated by the ballots in which it appeared.

Change to

3.8.8.2 Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are treated as though they had never been nominated on the ballots in which they appeared.

That covers the case in which the previous round eliminated more than one (due to ties), and makes it clear that eliminated works don't reappear.

Also, if the 5% rule remains in force, I still think there should be a Round 0 that eliminates everything that isn't eligible, whether due to actual ineligibility (e.g. published in the wrong year), the 5% rule, or early withdrawal.

#584 ::: felice ::: (view all by) ::: May 17, 2015, 07:27 PM:

Jameson Quinn @581: "Personally, I think that debating back and forth about the tiebreakers is pretty much the epitome of bikeshedding"

Given that we're pretty much agreed on everything except the bikesheds, I don't see that as a huge problem. But I'd say it's more like discussing a safety feature to deal with a highly unlikely reactor meltdown.

"it only matters in terms of deciding which of two essentially-equivalent works from the same voting slate ends up as a finalist"

It matters in terms of deciding whether one of two essentially-equivalent works from the same voting slate ends up as a finalist. I don't care how ties are broken, as long as they are broken; J Thomas is arguing against tie-breaking.


Seth @583: "3.8.8.2 Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are treated as though they had never been nominated on the ballots in which they appeared."

Yep, that's good.

"Also, if the 5% rule remains in force, I still think there should be a Round 0 that eliminates everything that isn't eligible, whether due to actual ineligibility (e.g. published in the wrong year), the 5% rule, or early withdrawal."

Saying "points are divided between all eligible works on each ballot" at the relevant point in the rules should be sufficient to cover that, rather than an explicit round one. Works below 5% currently aren't eligible unless there are fewer than three works above 5%.

#585 ::: felice ::: (view all by) ::: May 17, 2015, 07:27 PM:

Jameson Quinn @581: "Personally, I think that debating back and forth about the tiebreakers is pretty much the epitome of bikeshedding"

Given that we're pretty much agreed on everything except the bikesheds, I don't see that as a huge problem. But I'd say it's more like discussing a safety feature to deal with a highly unlikely reactor meltdown.

"it only matters in terms of deciding which of two essentially-equivalent works from the same voting slate ends up as a finalist"

It matters in terms of deciding whether one of two essentially-equivalent works from the same voting slate ends up as a finalist. I don't care how ties are broken, as long as they are broken; J Thomas is arguing against tie-breaking.


Seth @583: "3.8.8.2 Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are treated as though they had never been nominated on the ballots in which they appeared."

Yep, that's good.

"Also, if the 5% rule remains in force, I still think there should be a Round 0 that eliminates everything that isn't eligible, whether due to actual ineligibility (e.g. published in the wrong year), the 5% rule, or early withdrawal."

Saying "points are divided between all eligible works on each ballot" at the relevant point in the rules should be sufficient to cover that, rather than an explicit round one. Works below 5% currently aren't eligible unless there are fewer than three works above 5%.

#586 ::: J Thomas ::: (view all by) ::: May 17, 2015, 07:46 PM:

@581 Jameson Quinn

If spending extra time now on the decision helps us build support for the eventual proposal, then it's well worth it; but if we're all just arguing over the aesthetics of an aspect that will almost certainly never make a difference that matters, it's a waste of time.

That sounds good to me.

When I try to make an animation of it, the lookback is a great big extra hassle that I'm not sure how to present.

I don't know which woykd pass easier, I suspect that the lookback complication is a liability but for all I know for people new to it, the double elimination would look bad.

Maybe there's a way to ask them what they'd prefer, or whether it matters to them.

#587 ::: J Thomas ::: (view all by) ::: May 17, 2015, 08:20 PM:

@582 Felice

If it was a work you nominated, would you rather a 50% chance of your nomination surviving, or a 0% chance?

I would prefer that mine had a 50% chance, of course. But if it was somebody else's and not mine, then of course I'd prefer it had a 0% chance so my own nominations' chances were improved. And it's not mine far more often than it's mine.

"Is there some technical reason why it works out better to keep one and not the other, apart from fairness?"

Not aside from fairness, no. But fairness seems like quite sufficient justification to me.

I don't see that it's fair at all. But usually when people argue fairness they don't reach an agreement unless they're willing to spend an effort to look at deep issues. And it is not very central to the proposal unless people at the board meeting take sides on the fairness issue.

So my first thought is to just do something and stop talking about it and hope everybody forgets it, because if the people at the meeting notice and argue about what's fair and can't come to an agreement then they're likely to vote it down. And if it comes to an argument they're far less likely to agree than we are.

But my second thought is that somebody like Brian Z is likely to do his level best to start a controversy about it, taking whichever side we don't take, hoping for that exact outcome.

So we do better to have simple clear arguments why the way we do it is fair.

Here's my argument why double elimination is fair:

At each step we look at the two with the lowest scores and we eliminate the one with the lowest number of nominations. That's how we do it. If they're tied on nominations and tied on points there isn't a ha'penny worth of difference between them. They are the two lowest. Of course we eliminate both.

What else would we do? Flip a coin? Rummage around in old records to see how they used to do? We don't do that for anything else. There are lots of ways to choose between them -- we could for example choose the one that's first in alphabetical order if they have an even number of nominations, and reverse alphabetical order if the votes are odd. But there's no obvious reason to do anything except eliminate the two lowest when they tie for last place.

#588 ::: Brian Z ::: (view all by) ::: May 17, 2015, 08:39 PM:

557-60,75

I didn't mean to stomp off, I just worried taking up too much of the thread was inconsiderate. I'll try to be brief:

We seek consensus to honor a widely acclaimed author: I loved Moorcock. But I know everyone wants to honor LeGuin and it is just and right to nominate her too. You think Herbert should be on there, but you agree LeGuin should be too. That guy over there loves Michael Crichton, but even he grudgingly admits The Left Hand of Darkness was also a breakthrough. We all nominate LeGuin without being afraid supporting her will work against getting our own personal favorite one of the other slots. We have failed to agree on Moorcock, Herbert and Crichton, but we achieved consensus on LeGuin.

I don't know why people have lost sight of this - maybe in the last decade or two we have spent too much time voting people off the island and being team this or team that, and we've lost sight of the spirit in which the award is intended.

This still have a chance of the system working for all fiction categories except short story, which is broken due to the fragmented readership. We need to address that through informal mechanisms: passing around shortlists and reviewing and discussing them. We should probably be spending more money to buy magazines. But fixing this mess with a technical rules change? No chance.

Don't worry: With all this enthusiasm, would-be slate voters won't have a prayer.

#589 ::: JJ ::: (view all by) ::: May 17, 2015, 09:31 PM:

Brian Z., #588: "I don't know why people have lost sight of this - maybe in the last decade or two we have spent too much time voting people off the island and being team this or team that, and we've lost sight of the spirit in which the award is intended."

You're showing your Puppy side again. This has not been going on in the last decade or two in the Hugos.

We have not lost sight of anything here.

You have bought into the Puppy lies about the Hugo Awards hook, line, and sinker. You can accuse the Puppies of "voting people off the island and being team this or team that" -- because that's exactly what they've been doing.

But your making that sort of accusation of the rest of the Worldcon members, who have been participating in good faith, goes beyond the pale -- and you owe everyone here an apology for the fact that you've just accused them of this.

#590 ::: Duncan J Macdonald ::: (view all by) ::: May 17, 2015, 09:36 PM:

Brian Z @588

We seek consensus to honor a widely acclaimed author...

No, we don't seek consensus. Nowhere in any of these discussions or in the history of Hugo nominating or voting has there been consensus on either the slate or the losers.

I really don't care who you loved -- I'm nominating my loves, and when it comes time for the final voting, I'll vote for the one I like the best, or dislike the least, or No Award if I believe that all the offerings are dreck.

And none of that is consensus.

... maybe in the last decade or two we have ... sight of the spirit in which the award is intended.

No, it is a popularity contest plain and simple. The rules change as proposed restores the pop in popular.

Don't worry: With all this enthusiasm, would-be slate voters won't have a prayer.

A devoutly to be desired result.

#591 ::: Keith "Kilo" Watt ::: (view all by) ::: May 17, 2015, 09:40 PM:

Hi all -

I'm having surgery on my knee tomorrow (I took a rapier shot under the kneecap -- I don't recommend the experience...), so I'll be out of pocket pretty much all day. On Tuesday I'll combine all of your suggestions into the next draft of the proposal and see if we can't start to lock it down.

Regarding lookback or no lookback, for me, so long as we can explain them simply, either one works about as well as the other. As Jameson says, it's really pretty much a non-issue from a practical standpoint. Since simplicity (or the lack thereof) seems to be the major issue with lookback, let me come up with a description on Tuesday for each method, and then I'd like to get a sense of what the general group thinks -- just a "this one, that one, or I don't care". I don't think a formal poll is really necessary. I have some ideas as to how to handle the issue either way during the actual business meeting, so I think we'll be fine no matter what.

Regarding sponsorships, as Jameson mentioned, while it's a little bit of a financial issue for my family, I'm not comfortable asking for money, crowdsourced or otherwise. We will make it happen so that I can be there at least for the three days of the business meeting. So, I think we need to discuss who is able to go to Worldcon and who can be at the meeting -- even if you are there just to support the proposal, the more bodies we have, the better. Even if you can't go in person, if you are a supporting member, you are eligible to sign on as a submitter. If the proposal meets your satisfaction, I'd be honored if all of the people here -- even if you've been just following along -- would consider being a signer. We only need at least one person to present at the meeting, and if the group is okay with it, I'm willing to do that. But how many others of you are going to Worldcon and would be willing to attend the business meeting as well? If you'd post under the [SIGNING] or [ATTENDING] tag (or both, obviously), I'll add your name to the next draft of the proposal. Please include your actual name, as some of us have different handles. if you aren't comfortable posting your real name here, just email me -- apparently Googling my handle gives my email address most readily. :)

Thanks all, and I'll check in with you again on Tuesday.

Kilo

#592 ::: Tammy Coxen ::: (view all by) ::: May 17, 2015, 09:47 PM:

[SIGNING] [ATTENDING]

I'll be at the meeting, and would like to be a cosigner for the proposal. (And that's my real name)

#593 ::: Tim Illingworth ::: (view all by) ::: May 17, 2015, 10:04 PM:

KW@591: you should probably plan for a Sunday Business Meeting as well, as the nomination numbers aren't usually released until after the Hugo ceremony, which is Saturday this year.

I expect all the Hugo business to be postponed until after that, which will give us the Preliminary BM on Thursday, Main on Friday, Site Selection on Saturday and Hugo Proposals on Sunday.

#594 ::: junego ::: (view all by) ::: May 17, 2015, 10:08 PM:

Seth @ 583

[PROPOSAL LANGUAGE]

3.8.8.2 Subsequent rounds begin by reallocating points as follows: The work eliminated in the previous round is treated as though it had never been nominated by the ballots in which it appeared.
Change to

3.8.8.2 Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are treated as though they had never been nominated on the ballots in which they appeared.

That covers the case in which the previous round eliminated more than one (due to ties), and makes it clear that eliminated works don't reappear.

The language change is fine with me, it covers all possible cases that way. I may be misunderstanding, but I think the new tie breaking rule (LookBack) would do away with tied works challenging another work? Either way, this change covers it.

Also, if the 5% rule remains in force, I still think there should be a Round 0 that eliminates everything that isn't eligible, whether due to actual ineligibility (e.g. published in the wrong year), the 5% rule, or early withdrawal.

Section 3.8.5 already says this, so we don't have to. If the proposal to get rid of the rule is passed I,d assume that's a section that would disappear.

#595 ::: Soon Lee ::: (view all by) ::: May 17, 2015, 10:20 PM:

Keith "Kilo" Watt #591:

Best wishes with the surgery.

(At least it wasn't an arrow in the knee?)

#596 ::: junego ::: (view all by) ::: May 17, 2015, 10:26 PM:

Keith "Kilo" Watt @ 591

[SIGNING]

I can't attend but would be honored to sign even tho I've barely done anything. Name is CJ Cabourne

I'm having surgery on my knee tomorrow (I took a rapier shot under the kneecap -- I don't recommend the experience...), so I'll be out of pocket pretty much all day. On Tuesday I'll combine all of your suggestions into the next draft of the proposal and see if we can't start to lock it down.

Ouch on the kneecap!

Yes, we need to consolidate.

Regarding lookback or no lookback, for me, so long as we can explain them simply, either one works about as well as the other. As Jameson says, it's really pretty much a non-issue from a practical standpoint. Since simplicity (or the lack thereof) seems to be the major issue with lookback, let me come up with a description on Tuesday for each method, and then I'd like to get a sense of what the general group thinks -- just a "this one, that one, or I don't care". I don't think a formal poll is really necessary. I have some ideas as to how to handle the issue either way during the actual business meeting, so I think we'll be fine no matter what.

I think simplicity and appearance of fairness are key.

Regarding sponsorships, as Jameson mentioned, while it's a little bit of a financial issue for my family, I'm not comfortable asking for money, crowdsourced or otherwise. We will make it happen so that I can be there at least for the three days of the business meeting. So, I think we need to discuss who is able to go to Worldcon and who can be at the meeting -- even if you are there just to support the proposal, the more bodies we have, the better. Even if you can't go in person, if you are a supporting member, you are eligible to sign on as a submitter. If the proposal meets your satisfaction, I'd be honored if all of the people here -- even if you've been just following along -- would consider being a signer. We only need at least one person to present at the meeting, and if the group is okay with it, I'm willing to do that. But how many others of you are going to Worldcon and would be willing to attend the business meeting as well? If you'd post under the [SIGNING] or [ATTENDING] tag (or both, obviously), I'll add your name to the next draft of the proposal. Please include your actual name, as some of us have different handlle.

If you're going to represent me at the meeting, I would feel better if I could help defray your costs. Hope you will reconsider.

#597 ::: Steve Halter ::: (view all by) ::: May 17, 2015, 10:31 PM:

[SIGNING]

Steven Halter

I haven't decided on attending yet or not.

#598 ::: P J Evans ::: (view all by) ::: May 17, 2015, 10:57 PM:

[SIGNING]
Supporting, and already signed on the 4/6 proposal.
But can do this one also, I think.

The only way I'm likely to have convention budget again is with a lottery win. (I don't play the lottery: the times I've bought tickets, I've been lucky to get one number. This is The Universe Speaking.)

#599 ::: colin roald ::: (view all by) ::: May 17, 2015, 11:23 PM:

J Thomas @586:
When I try to make an animation of it, the lookback is a great big extra hassle that I'm not sure how to present.

It's not obvious how to visually display what's going on, but it's not that difficult to track internally for the purposes of tallying votes. I think it's the kind of detail that could be left for those who care to deal with in text, but glossed over in tutorial demos.

#600 ::: David Goldfarb ::: (view all by) ::: May 17, 2015, 11:27 PM:

Keith "Kilo" Watt @591: [SIGNING]
I'm a supporting member and will not attend, but I'd be happy to be listed as a signer of the proposal.

#601 ::: felice ::: (view all by) ::: May 18, 2015, 02:32 AM:

J Thomas @587: "I would prefer that mine had a 50% chance, of course. But if it was somebody else's and not mine, then of course I'd prefer it had a 0% chance so my own nominations' chances were improved. And it's not mine far more often than it's mine."

If it's not one of yours, why should your opinion be taken into account? How about we make all works by authors born in January ineligible? Should we assume the majority of authors will approve, because it improves their chances, or should we agree it's a bad idea because it's unfair to authors born in January, irrespective of what authors born in other months might think?

And it's just as likely that it will be one of yours as that it will make a difference to you if it isn't; in the overwhelming majority of cases, simultaneous elimination of someone else's nominations will not change whether or not any of your nominations make the final five. Actually, no, it must be twice as likely (approximately) - there are two works each with a 50% chance of making the final five, and just one work that one of them could replace. Ie, it's in everyone's interest to prevent simultaneous elimination; if it affects you at all, it will hurt you twice as often as it will help you.

"At each step we look at the two with the lowest scores and we eliminate the one with the lowest number of nominations. That's how we do it. If they're tied on nominations and tied on points there isn't a ha'penny worth of difference between them. They are the two lowest. Of course we eliminate both."

And if the two lowest aren't tied on nominations or points, they're still the two lowest; why not eliminate both of them anyway? SDV-LPE doesn't work that way. Simultaneous elimination is akin to a divide-by-zero error.

The most common tie that matters would be a tie for 6th place between works with overlapping support. Eliminate one, and the other gets enough points to place it above the current 4th; then the former 4th and 5th are compared for elimination. There are very few other works left at this stage, so the chances that works tied for nominations will also tie for points is pretty high, and ties for nominations aren't that rare. It's an issue that won't happen every year, but it will happen. And it's not just slates that will be affected; you could expect significant overlap for different episodes of the same TV show, stories by the same author, stories published in the same magazine, standout books in the same subgenre, etc.

#602 ::: J Thomas ::: (view all by) ::: May 18, 2015, 09:39 AM:

@601 felice

"I would prefer that mine had a 50% chance, of course. But if it was somebody else's and not mine, then of course I'd prefer it had a 0% chance so my own nominations' chances were improved. And it's not mine far more often than it's mine."

If it's not one of yours, why should your opinion be taken into account?

For that matter, if it is one of mine why should my opinion be taken into account? We're trying to do what's fair, we don't need to listen to special pleading from the people who individually stand to benefit.

The whole argument is bogus.

Suppose that we reprieve one of two tied, and then it loses later. That was no real benefit to it. But during the time it survived maybe it knocked out some others, so the final winners were different. That's good for the ones who won, and bad for the ones who lost. It comes out exactly even.

OK, suppose we reprieve one and later it does win. Because the order of challenges has changed, it and maybe some others are finalists that otherwise would not be, and others are knocked off. It's good for the ones it helped and bad for the ones it hurt. Again it comes out exactly even.

Why are the ones that win due to lookback more deserving than the others that lose because of lookback? I see that you want to be nice to one of the two that tied for last place, but if it makes a difference it will be at the expense of some who were not tied for last place.

"At each step we look at the two with the lowest scores and we eliminate the one with the lowest number of nominations. That's how we do it. If they're tied on nominations and tied on points there isn't a ha'penny worth of difference between them. They are the two lowest. Of course we eliminate both."

And if the two lowest aren't tied on nominations or points, they're still the two lowest; why not eliminate both of them anyway?

Because the rule is to eliminate the lowest.

SDV-LPE doesn't work that way.

We're in the process of deciding what rules SDV-LPE will actually follow. If you want to propose eliminating the two lowest when they aren't tied then go ahead, but I won't support it.

The most common tie that matters would be a tie for 6th place between works with overlapping support. Eliminate one, and the other gets enough points to place it above the current 4th; then the former 4th and 5th are compared for elimination. There are very few other works left at this stage, so the chances that works tied for nominations will also tie for points is pretty high, and ties for nominations aren't that rare.

If the two lowest both get eliminated at 6th place, then we're done.

I understand that you want to give one of the last-place works a break so it will have a chance to beat out one of the winners. I don't see why you favor one of the tied last-place nominees over one of the winners.

#603 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 10:09 AM:

@602: The problem with that argument is that it proves too much. Any decision between two works is zero-sum in some sense; one wins and one loses. And if you go back and recapitulate the reasons why the voting system decided which one wins, then it is "fair", because the voting system was designed using impartial rules. But that's circular reasoning.

There is a valid reason to prefer not to eliminate both: the system without dual elimination is Droop-proportional, and under certain assumptions of voter utility, any Droop-proportional system gives higher utility than any non-proportional system.

Of course, you can argue the other way, too. If dual elimination happens, it usually harms slates; and if you thing slate winners are inherently illegitimate, then that's a good thing. But if you want to make that argument, you have to make it explicitly, and not rely on circular reasoning.

I want to reiterate that I don't think the tiebreaker matters much and I support whichever solution can resolve the issue for the community. The argument I'm making here is not that J Thomas is wrong, but rather that you're both right, and it doesn't really matter.

#604 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 10:29 AM:

So, I'm figuring out which crowdfunder to use. There are a few questions.

Number one, what if Keith changes his mind, and decides to accept some money? I would not hesitate to split the money with him 50/50 if that were to happen. No offense, Keith; I respect your stance, but I also respect your right to keep an open mind.

Number two, what happens if the funding goes over or under the budget I've established? I'd be inclined to give the money to Electology.org in those cases (or, if Keith asked for it, 100% to him). The work I've done on this proposal, and the time I've spent here, was motivated by my desire to further understanding of and comfort with election reform. I think that some money going to support a charitable organization to further that cause is reasonable. Do others strongly object to that stance? (I want to respect all opinions here, but in the end, I can't end up like the boy and the old man and the donkey, so I'm going to have to make the call myself.)

#605 ::: Seth Gordon ::: (view all by) ::: May 18, 2015, 11:00 AM:

[SIGNING] (but not, alas, attending)

Good job, everyone!

#606 ::: Tammy Coxen ::: (view all by) ::: May 18, 2015, 11:08 AM:

I think as long as you are clear on where any over-donations go, then people can decide if they want to give money after you've funded.

#607 ::: Idumea Arbacoochee, Bad Cop ::: (view all by) ::: May 18, 2015, 11:19 AM:

Brian Z @ 550, 552, 556, 561, and 588:

So remember when I said back in 338 that This thread is about one proposal to change it, which can then be brought to the conversation at Sasquan, and that we're going to leave "whether" conversations aside from this point forward?

Also, remember that I said Brian Z, if you're looking to get disemvoweled, now you know how?

I let you ignore these things once, because you said you would have your say and then leave. It was disruptive to a conversation that was about something else, but calling you on it would have been more disruptive. So I was being nice, at the expense of the other people here who are trying to have a particular, focused conversation. I apologize to them for permitting it; it was an error in judgment.

Because then you came back to do it again. So now hear this. The very next time you try to derail this conversation, you're losing vowels.

I know you will then go back to File 770 and wave your martyr's palm in triumph. I double-dog-dare you to link to this comment when you do, because if you don't, I'm going to. I will also include a detailed discussion of why it's not appropriate behavior, neither in a moderated environment in general nor in this conversation.

You will not enjoy the experience.

#608 ::: Ginger ::: (view all by) ::: May 18, 2015, 11:36 AM:

[SIGNING]

Been following along since the first post, and admiring all the hard work put into the proposal. Can't attend, but signing with my real name, Ginger Tansey.

#609 ::: J Thomas ::: (view all by) ::: May 18, 2015, 11:39 AM:

@603 Jameson Quinn

@602: The problem with that argument is that it proves too much. Any decision between two works is zero-sum in some sense; one wins and one loses. And if you go back and recapitulate the reasons why the voting system decided which one wins, then it is "fair", because the voting system was designed using impartial rules. But that's circular reasoning.

Yes, exactly. If we want to say that one way is fairer, we need some reason why that nominee deserves to win more than the other does. My reasoning is simple, at every other step we care only about the number of nominations and the current score. Why should we add an exception and a complication to change the results?

There is a valid reason to prefer not to eliminate both: the system without dual elimination is Droop-proportional, and under certain assumptions of voter utility, any Droop-proportional system gives higher utility than any non-proportional system.

You've proved that this system with tie-breaking is Droop-proportional? That's interesting! I'm sure nobody has ever proven that before. I'd be interested to see your proof. It would probably change my mind.

I wonder what assumptions of voter utility it's true for.

The argument I'm making here is not that J Thomas is wrong, but rather that you're both right, and it doesn't really matter.

It matters most if it affects the proposal passing. We disagree about which way is fairer, and if other people disagree about that, it could affect the vote. That is its main practical significance.

If you want to argue that the Hugo nominations are won by small splinter groups with little consensus, none of whom have read that much of the literature and so the results are kind of random, and it really doesn't matter whether the results are fair as long as we stop the slates that people are upset about, I guess I could go along with that. But I don't think we should tell the business meeting that we don't care whether it's fair.

#610 ::: Steve Wright ::: (view all by) ::: May 18, 2015, 12:07 PM:

OK. I've not been chiming in on this one, but I have been following with interest.

It is clear that Something Must Be Done; this is a fair and well-reasoned Something, and I would be happy to support it.

[SIGNING] - Steve Wright is my real name, but I'm "Steve Wrigh" to Sasquan (and they have bigger things to worry about than fixing that typo).

#611 ::: Tom Whitmore ::: (view all by) ::: May 18, 2015, 01:33 PM:

I'm attending, may or may not be at the business meeting at various times. Have not followed closely enough here to sign. I'm interested in the discussion, but it's a lower priority than some other issues for me right now.

#612 ::: Annie Y ::: (view all by) ::: May 18, 2015, 01:45 PM:

[SIGNING] [ATTENDING]

At least the plan for now is to be in Sasquan and to be in the business meetings. :)

Full name is Yoana Yotova

Keith "Kilo" Watt @ 591:

Ugh. Good luck with the knee and hopefully you will recover fast. :)

#613 ::: Chris Battey ::: (view all by) ::: May 18, 2015, 02:00 PM:

[SIGNING] [MAYBE ATTENDING]

Which is to say, I'll be at Sasquan, and I intend to attend the business meetings, but my wife and I will have a three-year-old in tow, and I'm sure most of you know how a toddler tends to randomize plans.

#614 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 03:34 PM:

OK, one final funding question, before I set it up:

Would people rather have the money be tax-deductible, by going through electology.org; or would you rather have the option of giving it to Keith if he changes his mind? I realize I can't do both, and I have to decide up front. I think tax-deductible is better, because I don't actually expect him to change; but I've said I would keep that option open, so I want to check before I have to close it.

(Note that aside from Keith, there's also Cheradenine, who also gave well-informed help in the early stages of developing this proposal, and whose presence in the BM would I think be helpful, but who doesn't seem to still be keeping tabs on the discussion. I'd be willing to split any monies with them too if it came to that.)

In the interest of actually telling people what the fundraising will involve: I'll be looking for money for a Sasquan ticket, a round-trip Amtrak ticket there from Boston (yes, that's 1 week of just travel time, but since I can work on the train I'd rather do that than a plane trip with attendant CO2.), and a bit of money for spartan expenses while I'm there. So that's about $1000, which, based on the number of people who have responded positively to even an indefinite idea, seems probably doable.

#615 ::: felice ::: (view all by) ::: May 18, 2015, 03:53 PM:

J Thomas @602: "Because the rule is to eliminate the lowest."

Not the two lowest. Eliminating more than one at a time means potentially throwing out all of a person's nominations at once without ever reallocating any of their points.

"I understand that you want to give one of the last-place works a break so it will have a chance to beat out one of the winners. I don't see why you favor one of the tied last-place nominees over one of the winners."

Because it would have more points than the winner it replaces. If F and G are tied, and have fewer nominations than E, but would have more points than D if one was eliminated, that means E must share a significant number of nominations with the other finalists. Replacing a work nominated by the same people as other finalists with one nominated by a different set of people is the whole point of SDV-LPE. (Or if E beats D in the elimination, D either has fewer nominations than F or G, or also overlaps with other finalists, but to a lesser degree than E - either way, getting rid of it is still the correct result.)

Eg if ABCDE is a slate that's swept the category, replacing E with F or G is obviously a good thing.

If either F or G was declined or found ineligible before the initial count, one of them would be a finalist. If either F or G had 1/5 of a point more or less, one of them would be a finalist: if one of the F or G nominators added D and E to their ballot, that would make D or E lose - a clearly pathological result.

#616 ::: Cat ::: (view all by) ::: May 18, 2015, 04:34 PM:

I'm a supporting member and would be happy to sign. My name is also on the 4/6 proposal as a co-sponsor but I believe I am allowed to sign this one also.
[SIGNING]
I can be reached at ["handle" under which I am posting]faber @ toast.net if you need to contact me in e-mail to arrange it.

#617 ::: Joshua Kronengold ::: (view all by) ::: May 18, 2015, 04:42 PM:

[SIGNING] [ATTENDING]

Good luck on the surgery, Keith. I ended up giving up fencing [classical and rapier] after my miniscus surgery, although frankly I'm not sure exactly what happened to cause it to tear (fencing is hardly a bad bet, though).

#618 ::: Annie Y ::: (view all by) ::: May 18, 2015, 04:50 PM:

@614 and the whole funding question

I can help with a plane ticket or 2(provided that it is not too late and awards are not available anymore) - have enough miles so I can spare some so keep this as an option as well together with the funding campaign.

#619 ::: Andy H. ::: (view all by) ::: May 18, 2015, 04:56 PM:

[SIGNING]

As a continual lurker on these threads, I admire and endorse this proposal. I am a supporting member as 'Andy Holloway'.

#620 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 05:06 PM:

@618: I suggest you make that offer to Keith. He's said he doesn't want to take people's money, but miles may be a different story. Meanwhile, I'm looking at doing it by train, since (a) I've always wanted to take that train and (b) I can afford the time at that moment, as long as I do work while on the train and (c) I want to minimize my carbon footprint.

#621 ::: Duncan J Macdonald ::: (view all by) ::: May 18, 2015, 05:14 PM:

Keith @ 591

[SIGNING]

That's my real name up there.

#622 ::: Annie Y ::: (view all by) ::: May 18, 2015, 05:28 PM:

@620 Have fun with the train ride. :)

Keith @ 591

Keith, let me know if that would help?

#623 ::: J Thomas ::: (view all by) ::: May 18, 2015, 06:01 PM:

@615 felice

Eliminating more than one at a time means potentially throwing out all of a person's nominations at once without ever reallocating any of their points.

That sounds reasonable. It might be somebody's last two surviving nominations. If he has another that would get the points that's fine, but maybe he doesn't.

Of course, after they are spared they might knock out various other people's only surviving nomination. It's hard to predict. Any time a work gets eliminated it might be taking away some nominator's last surviving nomination.

It does seem like a bad thing when a ballot loses its last nomination, but that does happen. In 2013, for novels there were 1113 ballots cast and 717 votes for finalists. At least 396 ballots had no finalist on them at all, more than a third, and probably rather more than that since some ballots would have two finalists.

If we want to make that happen less, maybe instead of looking back to find some slight edge, we should instead look at how many ballots are down to their last chance, and eliminate the nominee that has fewer of those ballots.

"I understand that you want to give one of the last-place works a break so it will have a chance to beat out one of the winners. I don't see why you favor one of the tied last-place nominees over one of the winners."

Because it would have more points than the winner it replaces. If F and G are tied, and have fewer nominations than E, but would have more points than D if one was eliminated, that means E must share a significant number of nominations with the other finalists. Replacing a work nominated by the same people as other finalists with one nominated by a different set of people is the whole point of SDV-LPE. (Or if E beats D in the elimination, D either has fewer nominations than F or G, or also overlaps with other finalists, but to a lesser degree than E - either way, getting rid of it is still the correct result.)

That makes sense. We'd like to have the best combination of winners. So if F is spared and then it beats E, F must be better than E.

Either F has more votes than E and therefore F deserves to win.

Or maybe when G is eliminated that gives F more points so D and E must compete and one of them loses, and even if the loser has a lot more votes than F, still F is better because F has more points.

Wait a minute, though. One way, we say that F is better because it has more votes, and the other way we say it's better because it has more points. What combination of votes and points does it really take to make it better?

If the important thing is to have the best ones win, according to some combination of votes and points, shouldn't we look at ABCDEFG and find the combination of them that is best?

See, if we paired them off at random we could get lots of different finalists, and some of them would be better than others. When we have a way to tell which ones are best, shouldn't we just use that instead of using the haphazard ordering we get from sequential redistribution of points?

Well, but we have a voting system that works OK. I think that to claim that one way of handling ties gets a better result than the other, we'd need some way to tell what's better, and then we could work out which is better on average. But we don't have anything like that.

It's easy to tell which approach is simpler, but simplicity isn't the only thing we care about. It's just the only one that's easy to measure.

#624 ::: P J Evans ::: (view all by) ::: May 18, 2015, 06:41 PM:

623
There's no reason for you to worry about x being the last item left in someone's list for a category. That's Not Hugo's Problem.

It happens very, very frequently - or hadn't you noticed, with the 1984 data, how very many lists people submitted had one or two items in a category?

Choices: YMMV, and you shouldn't expect to get what you want.

#625 ::: Cassy B. ::: (view all by) ::: May 18, 2015, 07:07 PM:

[SIGNING] [SUPPORTING]

My name is Pynhqvn Ornpu, rot13ed to prevent casual googling.

#626 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 07:34 PM:

I've made a GoFundMe campaign. Since all funds will be going through the Center for Election Science, donations are tax deductible.

Thanks a lot for any support you can afford! It's really great to feel that the work I've done here is appreciated.

#627 ::: felice ::: (view all by) ::: May 18, 2015, 08:26 PM:

J Thomas @623: "It does seem like a bad thing when a ballot loses its last nomination, but that does happen"

Losing the last nomination isn't a problem, it's losing last and second last in one go without ever increasing the last's points.

"Wait a minute, though. One way, we say that F is better because it has more votes, and the other way we say it's better because it has more points. What combination of votes and points does it really take to make it better?"

The combination specified by SDV-LPE. And F will never have more nominations than E, otherwise E would lose the elimination comparison between E, F, and G (if I'm right about how we pick the works to compare - it's ambiguous in the various drafts).

"If the important thing is to have the best ones win, according to some combination of votes and points, shouldn't we look at ABCDEFG and find the combination of them that is best?"

Now you're arguing against the whole SDV-LPE system. And what happens if ABCFD and ABCGD are equally good, which seems very likely?

"When we have a way to tell which ones are best, shouldn't we just use that instead of using the haphazard ordering we get from sequential redistribution of points?"

The SDV-LPE order isn't haphazard; popular works are only considered for elimination when they share a lot of nominations with other popular works. If everyone bullet voted or nominated one popular work and a bunch of unpopular works, it would produce the same order as the current system.

#628 ::: J Thomas ::: (view all by) ::: May 18, 2015, 08:54 PM:

@624 P J Evans

There's no reason for you to worry about x being the last item left in someone's list for a category. That's Not Hugo's Problem.

I agree. Felice suggested that one reason her preferred method to handle ties is better, is that it is less likely to remove a ballots' last choice along with the second-last choice.

I say that if preserving ballots' last choices a longer time is what we want, there are better ways to get it. But I don't think that's a particularly important goal. I could be wrong, and I might see it her way when I understand better.

#629 ::: Brian Z ::: (view all by) ::: May 18, 2015, 09:02 PM:

@590 No, we don't seek consensus. Nowhere in any of these discussions or in the history of Hugo nominating or voting has there been consensus on either the slate or the losers.

Exhibit A: http://file770.com/?p=22247

@607 wave your martyr's palm in triumph.

This screen name is Abi Sutherthand? Hi Abi. No, it's fine. I was replying to others' specific questions and concerns, but of course you don't have to host a discussion of criticisms of the proposal if you don't want to. Hope to have a chance to chat with you again another time.

#630 ::: P J Evans ::: (view all by) ::: May 18, 2015, 09:21 PM:

629
The screen name on 607 is Idumea Arbacoochee, not abi. You need to pay attention to things like that.

(We also care about capitalization.)

#631 ::: P J Evans ::: (view all by) ::: May 18, 2015, 09:23 PM:

also 629
So your 'evidence' is from 1953? That's really solid and convincing. Not.

#632 ::: JJ ::: (view all by) ::: May 18, 2015, 09:39 PM:

Obviously my definition of the word "consensus" differs vastly from Brian Z's.

Maybe he misunderstood, and thought you were talking about antonyms of "consensus".

#633 ::: Laertes ::: (view all by) ::: May 18, 2015, 10:08 PM:

The concern troll thing was getting pretty old, so the transition to super snotty is a refreshing change.

#634 ::: Brian Z ::: (view all by) ::: May 18, 2015, 10:25 PM:

@630, 631

I think the moderator has appeared to warn that this discussion is off topic. Isn't that Abi Sutherland? If I have misidentified the moderator, my apologies.

That's "evidence" that early Hugo awards saw people trying to convince each other about what was worthy. But we've been told that the history of the Hugos is off topic, so would you like to join me over on File 770 to discuss it? Another place is fine too.

#635 ::: Carrie S. ::: (view all by) ::: May 18, 2015, 10:41 PM:

Brian Z: Since PJ Evans appears to have been too subtle for you, perhaps it should be made explicit that abi's screen name around here does not begin with a capital letter. It's considered polite to spell a person's name in the way that they spell it.

One also generally attempts to use the name a person gives, rather than any other name one may be aware they use. Insisting on linking two identities as if one were bravely making a revelation ranges from faintly amusing (as here, where we are all perfectly aware of the nature of Idumea's relationship to abi) to actively harmful.

#636 ::: Carrie S. ::: (view all by) ::: May 18, 2015, 10:42 PM:

Brian Z: Since PJ Evans appears to have been too subtle for you, perhaps it should be made explicit that abi's screen name around here does not begin with a capital letter. It's considered polite to spell a person's name in the way that they spell it.

One also generally attempts to use the name a person gives, rather than any other name one may be aware they use. Insisting on linking two identities as if one were bravely making a revelation ranges from faintly amusing (as here, where we are all perfectly aware of the nature of Idumea's relationship to abi) to actively harmful.

#637 ::: P J Evans ::: (view all by) ::: May 18, 2015, 11:01 PM:

If the gnomes will allow, from 'Worldcon' on Wikipedia:
11th ... 1953 ... Philcon II ... Philadelphia, Pennsylvania ... United States ... GOH: Willy Ley ... Attendance: 750

That's totally comparable to Sasquan. /sarcasm

#638 ::: Brian Z ::: (view all by) ::: May 18, 2015, 11:12 PM:

@635, I'm not a regular here. I only read the blog occasionally when a topic of special interest comes up. In trying to understand who is the moderator of this thread (and follow moderators' instructions) I noted the name "Abi Sutherland" is listed (capitalized) just above this text box I'm typing in now as "the moderator most frequently onsite" in addition to four others, but if I have inadvertently mis-addressed or misidentified anyone in any way, I do apologize.

#639 ::: JJ ::: (view all by) ::: May 18, 2015, 11:27 PM:

Brian Z.: "I'm not a regular here. I only read the blog occasionally when a topic of special interest comes up."

Since you've been showing a similar sort of disrespect over at File770, it seems clear that you've never bothered to educate yourself at Netiquette. You would be well-served to do so, instead of just assuming that you know everything already.

Rule 3: Know where you are in cyberspace


#640 ::: Laertes ::: (view all by) ::: May 18, 2015, 11:29 PM:

You suck at flouncing.

#641 ::: Jameson Quinn ::: (view all by) ::: May 18, 2015, 11:57 PM:

Hey, I have an awesome idea! Since we have a troll in the thread, it would be a nice change of pace if I turned into a spammer, and started posting regular updates (in which I just happen to mention my organization's URL) every time somebody kicks in another few dollars.

Or, I could not do that, even though I'd kinda like to, because even in my short time here, I've learned that it would be cosidered insufferably rude, even if I tried to be clever or humble or whatever.

Brian Z, you are obviously capable of googling the people you're talking to, to help you figure out what you can say. If we googled you, and we had your social engineering skills, what could we figure out to say to you that would make you carry out your terrible threat to leave us alone?

#642 ::: abi, who is also Idumea ::: (view all by) ::: May 19, 2015, 01:07 AM:

Actually, I don't care about capitalization for myself (though I try to be meticulous about it for others). I do care about good conversation, and let's all let Brian Z actually leave without poking him on the way out, shall we?

#643 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 03:02 AM:

Re tiebreakers: I'm not sure what's unclear here. The issue is that of removing entire demographics--possibly ones that would make the ballot otherwise with a strong showing, just because of an edge case.

That's not desirable. If you consider the following ballot set:

10 x A
11 x B
12 x C
13 x D
14 x E
15 x F
40 x GHIJK

Without a tiebreaker, G, H, I, J, and K will all be eliminated at the first elimination -- removing 34% of the ballots (and all their nominations) at the first pass. This is obviously not proportional -- those ballots should get 1-2 nominations--they have a third of the votes!

Add a bulletproof tiebreaker -- (although frankly, I know of no bulletproof one beyond coinflips other than "first encountered on any ballot") and the non-bullet slate (even if it's random happenstance--which while unlikely, is still totally possible) will take three of the entries (given the terrible voting all around [just because I needed a simple example], its fairest calculatable share).

#644 ::: Shane ::: (view all by) ::: May 19, 2015, 04:19 AM:

I couldn't see how you donate with a nickname, so an onymouse. There ya go.

Moderator - Would you find Cheradenine's email address and give her/him a heads up?

#645 ::: J Thomas ::: (view all by) ::: May 19, 2015, 06:12 AM:

@643 Joshua Kronengold

10 x A
11 x B
12 x C
13 x D
14 x E
15 x F
40 x GHIJK

Without a tiebreaker, G, H, I, J, and K will all be eliminated at the first elimination -- removing 34% of the ballots (and all their nominations) at the first pass. This is obviously not proportional -- those ballots should get 1-2 nominations--they have a third of the votes!

I find that argument kind of convincing. That is a bad outcome. Of course, by far the easiest way to get this result is when GHIJK is a slate.

What happens the other way?

GHIJK get a score of 8, and H is removed.

GIJKA then have a score of 10. K is removed.

GIJ have a score of 13.3. A, B, C, D are removed.

Winners GIJFE.

If A is removed instead of one of the others the outcome is the same. They deserve a third of the slots and they get nearly two thirds! The slate doesn't deserve 3 slots, but I guess denying them one is worse. It's more important that the slate gets at least one finalist than that it doesn't get three.

You give a rare case where our voting system gives a bad result either way. That will happen sometimes. Of course, if the slate isn't perfect and G and I get a few more votes than the rest, then there's a strong chance that balance is restored and we will get the normal outcome of GIJFE, with three slate winners.

tl,dr----------------

Felice presented an interesting result earlier. The numbers 1 4 11 29 have a special relationship which has the result that whenever the votes are tied and the scores are tied, both nominees have the same number of votes with only themselves surviving, the same number of votes with one other, the same number with two others, and so on.

So double ties might be common among the rare ones --there might be two that each have one vote shared with four others, or two that each have one vote shared with four others, etc. They get less common for nominees with more votes. But then they start to get more common late in the process as there are fewer and fewer ballots that have more than one surviving vote. So while it will be rare apart from slates, when it does happen it's likely to be important.

So in Felice's example with F and G tying for 5th place, in the reasonable case that F and G each have about 8% of the vote, when G is eliminated that will increase F's score by something less than 8% of the ballots F shares with others. Unlikely this will bring the score above D and E both, so in most cases F will lose to D or E and the outcome is the same.

Sorry for the digression. I was interested and didn't want to just delete it.
--------------

You have a plausible argument that is likely to affect people's feelings about the issue independent of the facts. I can't think of any argument that would affect them more. So you are right.

How should we break ties?

Add a bulletproof tiebreaker -- (although frankly, I know of no bulletproof one beyond coinflips other than "first encountered on any ballot") and the non-bullet slate (even if it's random happenstance--which while unlikely, is still totally possible) will take three of the entries

The current proposal is to look back for a minor past difference. Here is another idea -- look at the results of each outcome. When one is eliminated (or both are) that will increase other scores. Of the three choices, pick the one that has the least effect on the other nominees.

That is, pick the one that changes the relative order of scores the least.

Say it's a tie between F and G, and the scores are now ranked ABCDE. If you eliminate F and the scores are then ranked ABCED, but when you eliminate G the scores are ranked ABDEC, then eliminate G because it has affected three others instead of two.

We want to be fair to the tied nominees F and G by treating them different, but we should also try to be fair to all the others. Eliminating F and G will change the same *number* of ballots, but they're likely to have different effects. Why should C be hurt by arbitrary resolution of a tie that had nothing to do with C? Better to reduce that effect.

And maybe we don't need a bulletproof tie-breaker. If we apply a reasonable tie-breaker and it doesn't work because they're from a slate and identical except ballot order, why bend over backward to pick one? They still have a great big advantage over everybody else because the rest waste most of their votes on things that can't win, while the slate can take things that couldn't win and win with them -- because they're focused. The outcome where 30% of ballots gets 60% of finalists is not implausible. It can easily happen that 15% of ballots gets 60% of finalists. If slates have a very rare disadvantage, what's wrong with that?

#646 ::: Jameson Quinn ::: (view all by) ::: May 19, 2015, 08:39 AM:

So I lied about SDV-LPE without a tiebreaker being Droop proportional. It's just Dummet (Hare) proportional, not Droop. I also think, but have not yet thought of a proof, that if it fails Droop proportionality for some group, it must also succeed for some other group. But anyway: with simultaneous elimination it's clearly not any of those things, because slates can be eliminated wholesale.

#647 ::: J Thomas ::: (view all by) ::: May 19, 2015, 10:03 AM:

@646 Jameson Quinn

So I lied about SDV-LPE without a tiebreaker being Droop proportional. It's just Dummet (Hare) proportional, not Droop.

I'd be interested in seeing an argument for that, too. My email address is easy to find.

But anyway: with simultaneous elimination it's clearly not any of those things, because slates can be eliminated wholesale.

Sure, which says that maybe those are criteria we don't want. Being nice to slates in rare corner cases is not my highest priority, anyway. Slates still have a giant advantage with SDE-LPE and will get finalists out of proportion to their numbers, because of the way our nonslate nominators vote.

Anyway, thinking over the arguments by Felice and Joshua Kronengold, I think I see a reasonable argument against double elimination.

SDE-LPE is already quasi-chaotic. A small difference in votes can lead to a great big difference in outcomes. I'm reasonably sure that a pathological case could be constructed where a change in one vote would result in all five finalists being different. (It would be a lot of work to construct it, though. You agree it's possible, right?)

Similarly, with double elimination, we could get a case where the 2nd and 3rd nominees by votes go up against each other fairly early, and both are eliminated leaving #1, #4, #5, #6, and #7 to be the finalists. But with just one vote different #2 survives and then eliminates #4, #5, #6, and #7 leaving #1, #2, #8, #9, and #10 for finalists.

This sort of thing is slightly less likely if we have some way to avoid ties. Any reasonable thing we can do to make the system a tiny little bit less quasi-chaotic, is good.

So we should look for a good criterion to break ties. Currently I think rather than finding some insignificant difference between the tied candidates, we should choose one based on the principle of least chaos. Make the winner be the one that has the least significant effect on the other surviving nominees.

Instead of saying "This is fair because one of the tied nominees was a tiny tiny bit worse" we can say "This is the one that's most fair to all the others."

#648 ::: Derry Earnshaw ::: (view all by) ::: May 19, 2015, 10:07 AM:

[SIGNING]
I'm a supporting member because I'm too far away to attend practically - and too time-zone displaced to comment before someone else has said what I think. I have been following these discussions and think this is a good balance of fairness, robustness and understandability.

Jameson Quinn @620 I believe the Business Meeting Chair is also very fond of trains.

#649 ::: Pfusand ::: (view all by) ::: May 19, 2015, 10:40 AM:

J Thomas @ 643

I think you made a mistake in your computing. You started with Joshua's example:

10 x A
11 x B
12 x C
13 x D
14 x E
15 x F
40 x GHIJK

[Joshua's example deleted.]

What happens the other way?

GHIJK get a score of 8, and H is removed.

GIJKA then have a score of 10. K is removed.

This is where I think the error is. With H removed, its points are redistributed, and now G, I, J, and K have 10 points, sure. (As does unchanged A.) But they still have 40 nominations, and should win over A with only 10 nominations.

#650 ::: Cally Soukup ::: (view all by) ::: May 19, 2015, 10:43 AM:

Jameson Quinn:

It probably won't mesh with your schedule, but just in case it does (and for anyone else who likes trains and Worldcon) I thought you might like to know that a group of fans has put together "Traincon", for people who are planning on taking the Empire Builder to Worldcon. http://sasquan.org/wp-content/uploads/2014/10/Traincon2flyer.pdf

#651 ::: Jameson Quinn ::: (view all by) ::: May 19, 2015, 10:50 AM:

@647: "(It would be a lot of work to construct it, though. You agree it's possible, right?)"

Yes, I agree it's possible. But if it were Brian Z making that claim, I'd challenge him to give me an example, because I also agree that it would not be easy to construct one (I expect it would take more than one gimmick to build it, and getting more than one gimmick to work in the same scenario is not easy.)

#652 ::: J Thomas ::: (view all by) ::: May 19, 2015, 11:01 AM:

@649 Pfusand

I think you made a mistake in your computing. You started with Joshua's example:

....

This is where I think the error is. With H removed, its points are redistributed, and now G, I, J, and K have 10 points, sure. (As does unchanged A.) But they still have 40 nominations, and should win over A with only 10 nominations.

You're right. A loses now, instead of having a 1/5 chance of being the one that loses. So at the next step GIJK still have 10 points and one of them loses (unless we do it the other way and throw in B in which case B loses.) Then B wins against A, C wins against B, D wins against C, GIJ win against D, leading to three slate wins plus EF.

If we do it the other way and take everything with the lowest two scores, GIJK wins all the way leading to finalists FGIJK. Four slate wins.

I included an extra case that can't happen, but which leads to the same result.

#653 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 11:43 AM:

Hi all! Made it through surgery with no problems, though I won't be walking much for the next couple of days. Thanks for all the good wishes!

[TIES]

@643 (and following)
The reason these types of scenarios are so implausible is that you're only likely to get the distributions you cite when there are very few total ballots. For example, in 2013, the category with the fewest ballots counted was Best Fanzine, with 820 ballots (the lowest literary category was 1067 ballots). To get Joshua's situation, we would need to multiply all those ballots by a factor of 10. How likely is it that you're going to get 100-150 independent people bullet voting a single work with no overlap whatsoever? And how likely are you to get 400 slate ballots with absolutely perfect discipline? This is why none of these cases show up in realistic data. Without explicit coordination among the non-slate nominators, it's very close to statistically impossible for it to occur. And if even 5 of those 400 ballots differ from the slate, then there are no ties at all.

If someone really wanted to be fair about it, you could just advise slate coordinators to make sure at least 5 people change their nominations. But I think odds of perfect slate discipline among those 400 ballots is pretty small.

Kilo

#654 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 11:54 AM:

It's not untrue that 60% slate wins in my constructed example isn't ideal -- but it's one of the less non-ideal of the possible exaples--and since the other demographics don't have more than 15%, probably no better than any other plausible solution (RAV/N (linear weights) would get us exactly the same answer -- two slate wins off the bat, followed by two non-slate wins, and the slate would take another win at 40/3). If you consider the slate broken up into three rough demographics of 13.3 people each, it seems entirely fair that they'd get 3 wins alongside E and F -- as if you eliminate the no-hope demographics, they're three of the biggest pluralities. One of the features (and penalties) of SDV-LP is that it tries to give neither an advantage nor a disadvantage to people voting in a larger clump, after all.

I think "past scores" is a reasonable tiebreak for most cases. The place where it fails, though, is slates that are too successful -- which is going to happen on occasion, if not very often. Obviously, in straight voting ties in an early round, there may be no past scores to use.

The reason I referred to "first seen on first ballot" is that it's actually bulletproof -- the ballots are entered in a particular order, and while the order on them doesn't normally matter, there are no ties.

Of course, while I forgot to mention it, random is also a bulletproof tiebreaker by that measure.

I'd be pretty happy, fwiw, with what amounts to random without spelling it out:

Ties not including last place should always be broken (rather than removing all tied candidates), since if the tie is not moot (with the winner being eliminated regardless of the result), it is the same voters being affected if both are removed. The administrators should therefore use a fair and arbitrary method of breaking such ties in cases where no other method is specified.

#655 ::: J Thomas ::: (view all by) ::: May 19, 2015, 12:14 PM:

@651 Jameson Quinn

"(It would be a lot of work to construct it, though. You agree it's possible, right?)"

Yes, I agree it's possible. But if it were Brian Z making that claim, I'd challenge him to give me an example, because I also agree that it would not be easy to construct one

It turns out to be surprisingly easy with double-elimination, given one different ballot.

ABCDE 50 ballots
F 10
G 10
H 10
I 10
J 10
FGHIJ 1 ballot

ABCDE each have score 10.
FGHIJ all have scores 10.2.

So ABCDE are all double-eliminated, leaving FGHIJ.

Without the ballot FGHIJ, every nominee has score 10 and FGHIJ are eliminated, leaving ABCDE.

It can happen by changing one vote.

ABCDE 50 ballots, 10 points.
FG 9 ballots, 4.5 points for each
GH "
HI "
IJ "
JF "

FK 2 ballots, worth 1 points for each
GK
HK
IK
JK

K 5 ballots
So everything has 10 points. With simultaneous elimination, everything but ABCDE is eliminated the first round.

Suppose that there were only 4 K ballots.

Then K has 9 votes and 9 points. K is eliminated first. Then FGHIJ each have 11 points, and in the second round ABCDE are all eliminated.

One single changed vote gives an entirely different result.

I don't see in detail how to do that with invent-a-way-to-break-ties. The one with the most votes would win either way. We can turn that into a tie with the 2nd most votes, but one of the two of them would still win. So we'd need to make #2 lose the first way, and win the tie the second way, and the details would depend on precisely which arbitrary way we chose to break the tie. It looks tedious.

SDE-LPE is quasi-chaotic generally. A small difference in votes can lead to an extreme difference in outcomes. But usually it doesn't. And yet it could. One mailed ballot with the right postmark that arrives late, could give you everything we've worried about declined nominations.

#656 ::: P J Evans ::: (view all by) ::: May 19, 2015, 12:32 PM:

One mailed ballot with the right postmark that arrives late

Don't worry so much about it. You can't plan for every situation that might come up. Deal with the likely ones first, then the more probable of the unlikely ones.
(The post office is more of a constant. One of the final ballots showed up in December 1984. With black marks that seemed to say it had been stuck in a machine.)

#657 ::: J Thomas ::: (view all by) ::: May 19, 2015, 12:40 PM:

@653 Keith "Kilo" Watt

The reason these types of scenarios are so implausible is that you're only likely to get the distributions you cite when there are very few total ballots.

Sure. We look at examples with a few ballots because we can understand them, and they demonstrate some of the strange things that can happen.

When there are a whole lot of ballots we can't expect to understand it at all. Lots of weird things will happen that we don't even notice. But usually they won't call themselves to our attention in any way that forces us to notice them. So that's good.

#658 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 12:44 PM:

@655

With small-number statistics, you will almost always have problems in any field. This is the big problem in education research, for example, and makes it very difficult to draw any conclusions from a small data set. It's why education research is so hard. Again, let me reiterate: Getting these distributions when scaled up to statically-valid sample sizes is -highly- improbable.

It is not chaotic (which means "sensitive dependence on initial conditions"); "quasi-chaotic" doesn't have any rigorous meaning that I know of.

I think at this point, we should stick with one of the two remaining tie break choices, with lookback or without. The number of cases where this is going to make a difference in real data is vanishingly small as it is. The cases you are talking about are even less likely. If it really bothers someone in the meeting that there is even a chance to eliminate more than one work at a time, then they can propose an amendment based on some additional criteria at that time. I really see no value in complicating the system for cases that may literally never occur.

Kilo

#659 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 01:18 PM:

Kilo: I don't see any value in overcomplication, but I do see some value in specifying priority (ie, "break ties some other way" rather than "throw out tied potential nominees if you can't break a tie").

#660 ::: J Thomas ::: (view all by) ::: May 19, 2015, 01:29 PM:

@658 Keith "Kilo" Watt

With small-number statistics, you will almost always have problems in any field.

This is not statistics. We haven't particularly come up with names for pathological cases, much less defined them well enough to look for statistics about how often they will arise in plausible data. Our single real data set is more than 30 years old and doesn't represent current conditions. Our simulated datasets have potential flaws. So our efforts at statistics have been entirely amateur-3-minutes. What we've done with small understandable examples is to look at what's possible, not what's likely.

Getting these distributions when scaled up to statically-valid sample sizes is -highly- improbable.

We'd have to see, wouldn't we? It does look rare. Ties are rare. In 2013 there weren't any among finalists, and in 2012 there were only 2. Felice has made an argument that vote-ties in late stages are likely to be score-ties too. There just aren't many others left to share ballots with, and most of the ballot-sharing was done with the ones that are already gone. If the #5 and #6 works both have N nominations, not that unlikely they will both have N points, or possibly both have N-0.5 points or N-1 points etc.

I haven't tried to quantify that yet, but there could be something to it. I personally believe that you're probably right that any problems of any sort are unlikely, but I'd rather we make a serious effort to test that before we claim it's true.

It is not chaotic (which means "sensitive dependence on initial conditions"); "quasi-chaotic" doesn't have any rigorous meaning that I know of.

[sigh] Are you feeling belligerent? Five minutes of websearch would have given you some meanings. The one I'm using is given here.

http://www.hawaii.edu/uhmfs/documents/2010_11/20110216_reso_votingmethodsalternatives.pdf

quasi-Chaos: Small changes in voting can lead to large changes in outcome.2

I think at this point, we should stick with one of the two remaining tie break choices, with lookback or without. The number of cases where this is going to make a difference in real data is vanishingly small as it is.

I believe and hope that you're probably right.

If it really bothers someone in the meeting that there is even a chance to eliminate more than one work at a time, then they can propose an amendment based on some additional criteria at that time.

I want to have a simple clear argument ready for why the way we propose it is fair. If they get into an argument over two versions that neither one sound good, they may throw the whole thing out.

So far the best argument I've seen in favor of double elimination is that lookback is basicly arbitrary and complicated and may have unpredictable results.

And the best argument I've seen in favor of lookback is that double elimination is unfair to slates and may eliminate things disproportionately, plus it may have unpredictable results.

Maybe we should go with a variation on Joshua Kronengold's suggestion and punt. "The administrators should therefore use a fair and arbitrary method of breaking such ties."

#661 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 01:33 PM:

[TIES]

All:

As promised, here are the descriptions of the system with and without lookback. As noted, with lookback, the only time we eliminate more than one work per round is if the works have been tied for both points and nominations from the very beginning. As we have discussed, about the only way this can matter is if there is a slate with perfect disciplne. It would be trivially easy for a slate organizer to avoid this situation by just having five people not vote a perfect slate. I'm still not convinced we need anything more than that, but propose your "last-ditch" tie-break rule, and we'll discuss it. I am so far of the opinion that this should be handled via amendment if necessary. My thoughts are that we should explain the possiblity to the business meeting and if they see it as a problem, then fine.

I think I've described both systems (with and without lookback) in a fairly simple manner. Without lookback is simpler, obviously, but with lookback isn't too bad, in my opinion. What are your thoughts on these two, and then we'll hash out the last-ditch tie-breaker issue.

[I will post the text in the next message.]

Kilo

#662 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 01:43 PM:

[TIES]

Hugo Nomination System FAQs

1.Can you explain the system in plain language?
The important thing to remember is that nothing changes in how you nominate. If you think a work is Hugo-worthy, then nominate it. That’s all. All we are doing at this stage is throwing names into a hat. The final voting system, which actually chooses the winner, is unchanged. We could, in theory, simply put everyone’s nominations on the final ballot, but that would make for a very long ballot indeed. We therefore need to narrow the nomination list down. This system narrows down the list by eliminating the least popular works until only five (under current rules) finalists remain. Here are the basic steps to the elimination process:

a.You have one nomination “point” for each category that will be divided equally among the works you choose to nominate. So, if you nominate two works, each will get half a point; if you nominate three works, each will get one-third of a point, and so on.

b. All the points given to each work from all ballots are added together. The two works that got the least number of points are eligible for elimination. One of these works is the least popular and will be eliminated.

c. To determine which of these two works is least popular, we compare the total number of nominations they each received (that is, the number of ballots on which each work appears). The work that received the fewest total number of nominations is the least popular and now completely vanishes from the nomination process as though it never existed.

d. We start over for the next round and repeat the process, however, if one of your works was eliminated, then you now have fewer works on your nomination ballot. This means that each work gets more total points, since you aren’t dividing your point among as many works.


2. How are ties handled? [LOOKBACK]
First, there are two types of ties. If there is a tie for least or second-least number of points, then all the tied works are eligible for elimination. That’s straight-forward. It is possible, however, for two works to also have the same total number of nominations. In this case, we look at points again: the tied work with the fewest points is eliminated.

What if there is a tie for both total number of nominations and for total points? In this case, we just look back through the previous rounds to the first round in which they were not tied for points. The work which had the fewest points in that round is now eliminated. If the two works are tied for both points and total number of nominations all the way back to the first round, we just eliminate them both.

2. How are ties handled? [NO LOOKBACK]
First, there are two types of ties. If there is a tie for least or second-least number of points, then all the tied works are eligible for elimination. That’s straight-forward. It is possible, however, for two works to also have the same total number of nominations. In this case, we look at points again: the tied work with the fewest points is eliminated. If there is a tie for both total number of nominations and for total points, then we just eliminate both works.


#663 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 01:52 PM:

JT@660:
[sigh] Are you feeling belligerent? Five minutes of websearch would have given you some meanings. The one I'm using is given here.

No, I'm not feeling belligerent. My master's research is chaos theory, studying with Jim Yorke, who invented the term. I'm think I'm fairly knowledgable about the mathematical definitions of chaos. The definition in the article you cite is not mathematically rigorous, as I said.

If you want to cite reference articles, then you need to give complete references. This one isn't even signed. The footnote doesn't give the reference to support the claim -- situations matter, particularly with statistics (and yes, we are dealing with statistics).

That said, I'm not really interested in debating the minutia of terminology with you. We have a proposal to write. But you don't get to accuse me of being belligerent in order to discount something I've said, sorry. If I come across as being short with you, then I apologize.

Can we get back to work?

Kilo

#664 ::: Laertes ::: (view all by) ::: May 19, 2015, 02:04 PM:

Been following this, but not contributing, since big brains are clearly on the job and I've nothing unique to add. I'll open my mouth now that it's voting time.

It's not entirely clear to me which of the options (LOOKBACK or NO LOOKBACK) generates better results. It seems likely that the difference in results quality, whatever it is, is very small.

LOOKBACK is substantially more complex than NO LOOKBACK, and doesn't obviously offer substantially better results. I think it's more trouble than it's worth. I prefer NO LOOKBACK.

#665 ::: P J Evans ::: (view all by) ::: May 19, 2015, 02:25 PM:

[NO LOOKBACK]

KISS. I think some people keep forgetting this principle.

#666 ::: junego ::: (view all by) ::: May 19, 2015, 03:01 PM:

Kilo @ 662

2. How are ties handled? [LOOKBACK]
First, there are two types of ties. If there is a tie for least or second-least number of points, then all the tied works are eligible for elimination.That’s straight-forward. It is possible, however, for two works to also have the same total number of nominations. In this case, we look at points again: the tied work with the fewest points is eliminated.

I'm a bit confused about the bolded terminology. Not sure what you mean. Let me run down what I think would happen and see if I understand the process.

a) If there is a point tie at the least position, these two works challenge, nomination tallies are compared. If still tied, whatever tie-breaker is decided on is employed.

b) If there is a point tie at second-least position, nomination tallies for all works in least and second least position are compared. One should be eliminated (on points or nominations) and a new run calculated.

The tie-breaker would only be needed for works tied for last place in points that are also tied in nominations. Right?


#667 ::: J Thomas ::: (view all by) ::: May 19, 2015, 03:19 PM:

@663 Keith "Kilo" Watt

The definition in the article you cite is not mathematically rigorous, as I said.

Yes, that's true. It does give the sense I was using the term. The mathematically rigorous uses aren't appropriate here.

I guess it could be said that we have dealt with statistics a little bit, kind of. I've given hardly any description of what I've done, you've given a little more but hardly anything about what you've done, Jameson likewise. I haven't noticed any definitive results, but we've all gotten some that are suggestive, that imply it ought to sort of work.

If I come across as being short with you, then I apologize.

Accepted!

Can we get back to work?

Yes!

#668 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 03:23 PM:

junego@666:

Suppose works A and B tie for the least number of points and work C has the second-least number of points. We'll compare the number of nominations for all three works. Whichever has the fewest (even if it's C) gets eliminated.

Similarly, if A has the least number of points and C and D are tied for the second-least number of points, then all three will be compared for total number of nominations.

If A and B are tied for least number of points, and C and D are tied for second least-number of points, then we compare all four.

The tie breaker is needed when there is a tie for number of nominations, so that is what the other cases are referring to. Being tied for points can't cause a double elimination (unless they are also tied for number of nominations), so isn't really a problem for people as far as I can tell.

Does that help?

Kilo

#669 ::: abi, who is also Idumea ::: (view all by) ::: May 19, 2015, 03:24 PM:

Shane @644:

Forgive me, but I'm a little confused what, precisely I should point Cheradenine at. (Assuming the email address they used is a working one.)

#670 ::: J Thomas ::: (view all by) ::: May 19, 2015, 03:25 PM:

@662 Keith "Kilo" Watt

It is possible, however, for two works to also have the same total number of nominations.

I want to suggest something more like this:

It is possible, however, for two works to also be tied for the lowest total number of nominations.

I think the way you said it was good enough, and this leaves even less to think out.

#671 ::: Annie Y ::: (view all by) ::: May 19, 2015, 03:31 PM:

Keith "Kilo" Watt @ 668

I have a slow day so I will just ask - why would we touch C. We have two bottom work, we work only with them. Why grab C as well? Unless if we have a 3-way (or more) tie on points at the bottom, why wouldn't we just get the last 2 works at the bottom?

Now - if we have A with least points and B&C tying for second-least, then I can see us grabbing them all...

#672 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 03:39 PM:

Annie@671:

In truth, we can do it either way. My thoughts are to just handle everything the same way -- it reduces the number of "cases" that we have to deal with.

Kilo

#673 ::: J Thomas ::: (view all by) ::: May 19, 2015, 03:51 PM:

@668 Keith "Kilo" Watt:

Suppose works A and B tie for the least number of points and work C has the second-least number of points. We'll compare the number of nominations for all three works. Whichever has the fewest (even if it's C) gets eliminated.

I had thought we had a consensus on this. If we already have two works A and B with the lowest number of points, why do we need to add a third?

Here's my practical issue:

Say there is a slate that has more votes than most nonslate works.

If two or more slate works are tied with a score of N/5 or so, and all works with score

#674 ::: P J Evans ::: (view all by) ::: May 19, 2015, 03:55 PM:

The idea is to have the same procedure for all the names, regardless of whether they're on a slate.

#675 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 04:11 PM:

[FORMAL PROPOSAL]

All:

Okay, as promised, here's the next draft of the proposal. junego, it was recommended to us that we not include exmaples in the actual amendment, and instead include those in the explanatory text. I just wanted to make sure you knew why those examples weren't included.

Next post will have the text...

Kilo

#676 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 04:17 PM:

Short Title: Change to Hugo Nomination System
Moved, to amend section 3.8 (Tallying of Nominations), section 3.9 (Notification and Acceptance), and section 3.11 (Tallying of Votes) as follows:

Section 3.8: Tallying of Nominations.
3.8.1: Except as provided below, the final Award ballots shall list in each category the five eligible nominees receiving the most nominations. If there is a tie including fifth place, all the tied eligible nominees shall be listed. determined by the following multi-round process described in 3.8.8.
3.8.2: The Worldcon Committee shall determine the eligibility of nominees and assignment to the proper category of works nominated in more than one category.
3.8.3: Any nominations for “No Award” shall be disregarded.
3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.
3.8.5: No nominee shall appear on the final Award ballot if it received fewer nominations than five percent (5%) of the number of ballots listing one or more nominations in that category, except that the first three eligible nominees, including any ties, shall always be listed.
3.8.6: The Committee shall move a nomination from another category to the work’s default category only if the member has made fewer than five (5) nominations in the default category.
3.8.7: If a work receives a nomination in its default category, and if the Committee relocates the work under its authority under subsection 3.2.9 or 3.2.10, the Committee shall count the nomination even if the member already has made five (5) nominations in the more-appropriate category.
3.8.8: The final Award ballots shall list in each category the eligible finalists as determined by successive rounds of elimination, where each member gets a single nomination “point” for each category that will be divided equally among their nominated works in that category.
3.8.8.1: In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots for subsequent rounds as if they had never appeared on the ballots.
3.8.8.2: In the event that two (2) or more works are tied for appearing on the fewest number of ballots, the tied work with the lowest point total will be eliminated. If there is a tie for appearing on the fewest number of ballots as well as for lowest point total, the point totals for each work in the previous rounds (in reverse order) will be compared until the first round in which they differ in points is found. The work with least number of points in this round is then eliminated. If the two works have had equal point totals and equal number of nominations since the first round, then all members of that tie will be eliminated.
3.8.8.3: If elimination would reduce the number of ballots to fewer than the minimum number of finalists, then instead none of the works from that round should be eliminated, and all remaining works shall appear on the final ballot.
3.8.8.4: Subsequent rounds begin by reallocating points as follows: All works eliminated in previous rounds are removed from all ballots and treated as if they had never appeared on a nomination ballot. Members’ points are reallocated equally among their remaining nominated works, if any.

Section 3.9: Notification and Acceptance.
3.9.1 Worldcon Committees shall use reasonable efforts to notify the nominees finalists, or in the case of deceased or incapacitated persons, their heirs, assigns, or legal guardians, in each category prior to the release of such information. Each nominee finalist shall be asked at that time to either accept or decline the nomination. If the nominee any finalist(s) declines the nomination, that nominee finalist(s) shall not appear on the final ballot. In this event, once all finalists have had the opportunity to decline, the nomination system shall be re-run as described in Section 3.8 with the declined nomination(s) removed from the ballots on which they appeared. The eligible finalists from this re-run shall be merged with the remaining potential finalists from the original run. If this merge would result in more than the maximum number of finalists, then the ballot shall be extended to include the finalists from both the original and the re-run of the nomination system. This procedure shall also be used in the event that a finalist is deemed ineligible.

Section 3.11: Tallying of Votes.
3.11.4: The complete numerical vote totals, including all preliminary tallies for first, second, … places, shall be made public by the Worldcon Committee within ninety (90) days after the Worldcon. During the same period the nomination voting totals shall also be published, including in each category the vote counts for at least the fifteen highest vote-getters and any other candidate receiving a number of votes equal to at least five percent (5%) of the nomination ballots cast in that category, but not including any candidate receiving fewer than five votes. During the same period a record of all rounds of the selection process for each category shall also be published.

Submitted by: Keith “Kilo” Watt, Tammy Coxen, Yoana Yotova, Joshua Kronengold, Chris Battey, CJ Cabourne, Steven Halter, P.J. Evans, David Goldfarb, Seth Gordon, Ginger Tansey, Steve Wright, Catherine Faber, Andy Holloway, Duncan J. Macdonald, Claudia Beach, Derry Earnshaw

Commentary: It is to be emphasized that this proposal does not change the nominating process from the perspective of Worldcon members: They still list works, unranked, they feel are Hugo-worthy (up to the maximum permitted) in any category they choose, just as they have previously. In the past we have strictly counted the number of nominations and the top five works were put on the final ballot. However, because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Using this system, fandom isn't penalized for nominating a wide variety of works. If you nominate something that ends up not having a chance to make the final ballot, then your remaining choices automatically get more of your support instead of just being wasted. In other words, you can safely nominate anything you feel is Hugo-worthy. If enough people agree with you, it will make the final ballot. If they don't, that's okay – when that work is eliminated, your other choices will have a greater chance of making the final ballot. In this way, by eliminating the least popular candidates, fandom slowly converges to a consensus as to which works should be voted on to be the final Hugo winner.

It is also an explicit goal of this proposal not to disenfranchise anyone. Rather, this proposal seeks to ensure that no minority of members can disproportionately dominate an entire category. Instead, this system allows the broadest range of works that are popular with fandom-at-large to be considered for the Hugo Award.

#677 ::: Keith "Kilo" Watt ::: (view all by) ::: May 19, 2015, 04:20 PM:

Okay, the boss says it's time for pain meds, so I'm off for a bit. Had to scramble to get that uploaded, but hopefully it's formatted correctly.

Let me know of changes and if I forgot any signed (It just occurred to me that I did forget one person who emailed me -- I'll get you on the next draft!)

Kilo

#678 ::: Annie Y ::: (view all by) ::: May 19, 2015, 04:30 PM:

Kilo@672

I'd vote for the simplest case. Always lowest two. If there is a tie with the higher (or with both), grab all that tie.

Otherwise we may end up with a 3-way tie at 2.0 and 5-way split at 1.0 (for example). Why force it when we can just go 5-ways instead of 8-ways? :)

#679 ::: Tim Illingworth ::: (view all by) ::: May 19, 2015, 04:46 PM:

KW@676: I 3.8.8.3 should "ballots" not be "nominees"?

It could also be worth breaking out 3.8.8 as a new Section 3.X: Nominee Tallying Procedure. S 3.8 is getting a bit unwieldy...

#680 ::: felice ::: (view all by) ::: May 19, 2015, 04:52 PM:

[TIES]

Keith "Kilo" Watt @662: "If there is a tie for least or second-least number of points, then all the tied works are eligible for elimination."

That should be "then all works with least or second-least number of points are eligible for elimination" - the current text could be interpreted as "if there is a tie for second-least points, only the works with second-least points are eligible for elimination, and the single work with least points is automatically safe", or less blatantly unreasonable, "if there is a tie for least points, the single work with second least points is automatically safe".


Annie Y @671: "I have a slow day so I will just ask - why would we touch C. We have two bottom work, we work only with them. Why grab C as well? Unless if we have a 3-way (or more) tie on points at the bottom, why wouldn't we just get the last 2 works at the bottom?"

It allows tied works with more nominations than any of the other low point works to bubble up together to tie for 5th place. The alternative is more frequent simultaneous elimination; we agreed that having tied works eliminating each other was worse than having them both eliminated by a third work with more nominations. Note that in this case, the tied works might be entirely unrelated, just happening to have the same number of nominations rather than being nominated by the same people.

#681 ::: Annie Y ::: (view all by) ::: May 19, 2015, 04:54 PM:

@676
OK - someone with English as a native language should weight in.

Does "the least and second-least" in the following case 1,1,2 means pick 1 and 1 or does it mean pick all three. If the latter, we need other wording here. Actually does it not matter what it means - if it is confusing readers, we probably need to restate it. I am confused and I had been here all along :)

So why don't we replace


In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared

with

In each round, the two (2) works with the lowest number of points (more than two if there are works with the same number of points as at least one of the two) will be compared.


More - when I read the whole thing again :)

#682 ::: Annie Y ::: (view all by) ::: May 19, 2015, 05:05 PM:

felice @ 680

OK - how is a 40.1, 40.2, 40.3 split and 40.1,40.1,40.3 so materially different that it is worse to leave the 40.3 alone in one case but not in the other?

Or in the case of 40,40,40,40,40,40.1. Let the 40s cannibalize one of themselves, the redistribution will change it again. The only case when this may be bad if the 40s are all slate works that noone else (at all) selected and it is a full slate of 5 (so they won't ever change - they will always stay at 40, cannot be resolved and so on). How moving the whole slate to start eliminating the next up work will help in that case? They will have the most nominations at each step.
This will lead to the slate eating ALL other works. Which frankly is the last thing we want.

Yes - I know we are not trying to exclude the slates. But we are also not trying to give them a clear strategy. And I would argue that this is a viable problem considering the quality of works that they nominate.

We are over-complicating it here I think. Get the two lowest and all that tie with them; don't try to pull more than 2 if the lowest two are a tie but do not have a third to tie with them... It also is a lot easier to explain to people :)

#683 ::: P J Evans ::: (view all by) ::: May 19, 2015, 05:25 PM:

681
1 would be 'least' and 2 would be 'second least' (or next-to-least').
(English. Drives everyone a bit crazy.)

#684 ::: J Thomas ::: (view all by) ::: May 19, 2015, 05:30 PM:

@682, from Annie Y,

So why don't we replace


In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared

with

In each round, the two (2) works with the lowest number of points (more than two if there are works with the same number of points as at least one of the two) will be compared.

Get the two lowest and all that tie with them; don't...

I think you did better the next post.

"In each round, two (2) works with the lowest point scores (and all that are tied with them) will be compared.

Assuming this is what we want to say, and not something else.

... may be bad if the 40s are all slate works that noone else (at all) selected and it is a full slate of 5 (so they won't ever change - they will always stay at 40, cannot be resolved and so on). How moving the whole slate to start eliminating the next up work will help in that case? They will have the most nominations at each step.
This will lead to the slate eating ALL other works. Which frankly is the last thing we want.

My comment saying that got cut off, but I think you said it better. Thank you.

#685 ::: Doire ::: (view all by) ::: May 19, 2015, 05:37 PM:

Kilo @672 Firstly, take care of yourself and recover soon.
From the commentary -

However, because SF fandom typically nominates a diverse range of works, it was easy for an organized slate to make it so that no other works made the final ballot.

Might "diverse" be a dog-whistle word? I can see it being used to suggest that this proposal assumes diversity in SFF is good, and that this is enough reason to oppose it. How about "wide"? That should be neutral.

Is there any advantage to submitting this proposal early? New, innocent eyes might see gaps in the wording that are almost impossible to spot when (generic) you've been working on the details for so long.

#686 ::: felice ::: (view all by) ::: May 19, 2015, 05:46 PM:

Keith "Kilo" Watt @653: "Hi all! Made it through surgery with no problems"

Glad to hear it!


[TIES]

#680: Ah, I see that's fixed in #676.


J Thomas @660: "We'd have to see, wouldn't we? It does look rare. Ties are rare. In 2013 there weren't any among finalists"

Aside from BDP Short Form and Best Professional Artist, and single-nomination differences in five more categories. There would have been a four-way tie for fifth in Short Story too, if not for the 5% threshold. And last year, there were finalist ties in five categories. Ties are not rare!

"Felice has made an argument that vote-ties in late stages are likely to be score-ties too. There just aren't many others left to share ballots with, and most of the ballot-sharing was done with the ones that are already gone. If the #5 and #6 works both have N nominations, not that unlikely they will both have N points, or possibly both have N-0.5 points or N-1 points etc."

And lookback probably won't help if works below 5% are excluded from the process; there won't be enough rounds to look back at. The 5% threshold might (hopefully) be abolished, but that's a separate proposal and we can't rely on it. And frankly I don't think lookback is really any less arbitrary than looking at first appearance on any ballot, which has the advantage of always working and being easier to explain. I mean, it's choosing the winner based on whose nominators supported the fewest semi-popular-but-already-eliminated works; entirely arbitrary, and it encourages bullet voting, because bullet-voting nominators are more likely to win a lookback tiebreaker. Lookback was designed for STV elections (ranked, many voters rank all candidates); it just doesn't make any sense for SDV-LPE (unranked, nobody nominates more than a tiny fraction of candidates).

"quasi-Chaos: Small changes in voting can lead to large changes in outcome."

As I understand it, those possible large changes are only due to simultaneous elimination. Can you come up with any examples of single votes drastically changing the outcome that don't involve ties?

#687 ::: Annie Y ::: (view all by) ::: May 19, 2015, 05:55 PM:

J Thomas @ 684

I had been trying to figure out how to say it. :) I know what I mean, I am not always able to say it in a concise sentence.

So what is the consensus then. In the case of 1,1,2, do we get 1,1 or 1,1,2 into eliminations? I was under the impression that is 1,1, sounds like not everyone thinks so.

#688 ::: felice ::: (view all by) ::: May 19, 2015, 06:02 PM:

Annie Y @682: "OK - how is a 40.1, 40.2, 40.3 split and 40.1,40.1,40.3 so materially different that it is worse to leave the 40.3 alone in one case but not in the other?"

It's to minimise the chance of a tie for points and nominations. If the two 40.1s are tied for nominations as well, there's no non-arbitrary way to pick which one to eliminate. Comparing to the 40.3 as well means we might not need to break the tie.

"Or in the case of 40,40,40,40,40,40.1. Let the 40s cannibalize one of themselves, the redistribution will change it again. The only case when this may be bad if the 40s are all slate works that noone else (at all) selected and it is a full slate of 5 (so they won't ever change - they will always stay at 40, cannot be resolved and so on). How moving the whole slate to start eliminating the next up work will help in that case? They will have the most nominations at each step.
This will lead to the slate eating ALL other works. Which frankly is the last thing we want."

They will only eat all other works up till they tie for 5th place, in which case the four other finalists will be non-slate works. A slate tied at 40 means they have 200 nominations each; but we only need four works with more 41 or more unique nominations each (more to the extent that they overlap) to stay above them right to the end. And if we don't have any works with 41 or more unique nominations each, then all the non-slate works will eliminate each other early on and the slate will never even be compared for elimination, and the fact that they're tied is irrelevant.

Having the entire slate tied for 5th is a nuisance - it means 9 finalists instead of 5 - but it's not really that awful an outcome, and it's incredibly unlikely to happen (it requires no slate supporters deviating from the slate in any way, and nobody outside the slate supporters coincidentally nominating any of the same works).

#689 ::: Annie Y ::: (view all by) ::: May 19, 2015, 06:11 PM:

felice @ 688

Unless if someone else also ties at the 40 with them somewhere along the way (especially high up), allowing these 5 and that 6th one to eat everyone else. Which is as likely as having the full slate not matching anyone else's single nominations (which is the only case we are getting here). If we are in that case, the slate will end up eating more than one of the top 5 works more likely than not.

And picking up the 2 in the 1,1,2 is saving the slate only in this case as well (all 1s will go away otherwise) - in all other cases, there will be some of them a bit off step because of other ballots. So I still do not see why we should not go for the 1,1 in this case.

PS: Not a deal breaker. I will support it either. I just prefer the simpler solution when there is no real reason to go for the more complicated one. Makes it cleaner to explain. And easier to convince people that it is the proper way.

#690 ::: felice ::: (view all by) ::: May 19, 2015, 06:56 PM:

Annie Y @: "Unless if someone else also ties at the 40 with them somewhere along the way (especially high up), allowing these 5 and that 6th one to eat everyone else. Which is as likely as having the full slate not matching anyone else's single nominations (which is the only case we are getting here). If we are in that case, the slate will end up eating more than one of the top 5 works more likely than not.

A perfect slate and other potential finalists tied for second least points is even more incredibly unlikely. This must be a tie involving at least two of the last five non-slate works, otherwise there'd still be four non-slate works above them to stay on the final ballot, or the elimination would already be over. Simultaneous elimination of the tied works with fewest nominations could theoretically leave fewer than four non-slate finalists, but in practice this is never going to happen; and also, I'm strongly opposed to ever doing simultaneous elimination. Only one of the tied works should be eliminated, which leaves us with the same 9 finalists as without a tie between non-slate works.

"And picking up the 2 in the 1,1,2 is saving the slate only in this case as well (all 1s will go away otherwise) - in all other cases, there will be some of them a bit off step because of other ballots. So I still do not see why we should not go for the 1,1 in this case."

The works tied for last aren't necessarily slate works, though! Eg the ballots below have A, B, C, and D tied for 1 point and 3 nominations each, even though no two ballots are the same, while E is second-last with 2 points and 2 nominations. This isn't a particularly realistic scenario, but more complicated sets of ballots with two partially overlapping works tied for last place aren't implausible.

ABC
ABD
ACD
BCD
E
E

#691 ::: J Thomas ::: (view all by) ::: May 19, 2015, 06:56 PM:

@688 Felice

"This will lead to the slate eating ALL other works. Which frankly is the last thing we want."

They will only eat all other works up till they tie for 5th place, in which case the four other finalists will be non-slate works.

I think I followed that. You have 5 slate works that each have 40 points, and you don't eliminate any of them when they are tied, instead you compare them to the work with the next highest score. They eliminate everything until they are in 5th place and at that point you stop. Even though there are 9 survivors and you could compare them to the next lowest as you've been doing all along, instead they are tied for 5th place so that's where you stop.

Neat trick!

The other 4 works are the ones with the highest point scores at that time, which have had high point scores all along. None of them have competed since the slate started competing every turn; if they had they would probably have lost. (Though maybe the one with the most or second-most number of votes might have won.)

That outcome doesn't look as bad as I was thinking. There might not be that many of them tied, and if there's a tie and there's one slate member that has more votes, presumably it would eliminate all the others. When there are 6 left and it has the 5th highest score and another slate member is at #6, it will beat that one.

I dislike including items with the second-lowest score when there's a tie at the lowest score, and I dislike keeping all members of a tie when there's a tie, but the combination of these can work better than I thought.

#692 ::: Jameson Quinn ::: (view all by) ::: May 19, 2015, 07:18 PM:

I agree that it should be only 1,1 not 1,1,2. @688 is, I think, wrong; they could in fact eat everyone else, because the system doesn't realize they're tying for 5th, it thinks they're tying for 8th/9th. The question remains, how to word it simply and unambiguously; I think "In each round, two works with the lowest point scores (and all that are tied with one of them) will be compared." (This is not my suggestion of course, but I'm not sure whom to credit.)

(I also think we should avoid the "two (2)" tic.)

...

In terms of "alerting cheradenine", it had to do with the fundraiser. But now that I've promised that donations will be tax deductible, I will not be able to split them with cheradenine. If Cheradenine is interested in getting some of the fundrisen money, they should contact me, and I'll set up a separate gofundme and encourage people to donate to both equally.

...

Moderators: I want to promote my gofundme campaign so I can go to sasquan (already over $200 raised, over 1/7 of the way there), but I don't want to spam. Can I get some guidance as to what would be in bounds?

...

Keith: Good that the surgery went well. Did you see @618/@622?

#693 ::: Annie Y ::: (view all by) ::: May 19, 2015, 07:35 PM:

"In each round, two works with the lowest point scores (and all that are tied with one of them) will be compared." is still ambiguous though. Or at least sounds ambiguous to me.

What about:

In each round, all works with the lowest point scores are compared. If only one work is selected this way, all works with the next lowest points score are added to the comparison.

Or something along this lines... Makes it clear enough for anyone reading.

#694 ::: Jeremy Leader flags a moderator ::: (view all by) ::: May 19, 2015, 08:16 PM:

Jameson Quinn @692 above has a question for the moderators which has some degree of time-sensitivity (he'd like to promote his GoFundMe while people are still engaged with this thread).

#695 ::: J Thomas ::: (view all by) ::: May 19, 2015, 08:25 PM:

@686 Felice

"quasi-Chaos: Small changes in voting can lead to large changes in outcome."

As I understand it, those possible large changes are only due to simultaneous elimination. Can you come up with any examples of single votes drastically changing the outcome that don't involve ties?

Sure, but it's more complicated. Here's a simple example:

A 1 vote 1 point
AB 100 votes 50 points
ACDE 1 vote 1/4 point

C
D I don't care how these go but they should be more than 50.34 points and less than 51.25 points
E

CDF 1 vote 1/3 point
F 50 votes 50 points

There are 6 works left so one should go. The low scores are B with 50 and F with 50.33.
B has 100 votes while F has 51 votes, so F is eliminated and the winners are ABCDE.

But if the single vote for A is removed, then for the last round,

A has 50.25 points while B has 50 points. A has 101 votes and B has 100 votes. B is eliminated. The winners are ACDEF.

It's easy but tedious to extend this. Have some BG pairs in place of some of the AB pairs. Normally B eliminates G and gains points, leaving F in 6th place losing to E. But when B is instead eliminated, G vaults into 4th place, and E (which had some EG pairs) into 2nd, giving F the chance to beat C or D.

Etc.

Does it seem plausible to you that one vote different could possibly result in four of the finalists changing, without involving any ties?

#696 ::: J Thomas ::: (view all by) ::: May 19, 2015, 08:33 PM:

@692 Jameson Quinn

I agree that it should be only 1,1 not 1,1,2.

I like that better but I'm no longer sure the other way is very bad.

@688 is, I think, wrong; they could in fact eat everyone else, because the system doesn't realize they're tying for 5th, it thinks they're tying for 8th/9th.

Are you sure? If it was #6 and #5 tied, they would be tying for 5th place. If it was #7 and #6 and #5 tied, wouldn't that be a 3-way tie for 5th place? Etc. How else would you get a tie for 5th place?

I can kind of see this one both ways.

#697 ::: J Thomas ::: (view all by) ::: May 19, 2015, 08:37 PM:

@693 Annie Y

"In each round, two works with the lowest point scores (and all that are tied with one of them) will be compared." is still ambiguous though. Or at least sounds ambiguous to me.

With human beings, if it *sounds* ambiguous then it *is* ambiguous.

What about:

In each round, all works with the lowest point scores are compared. If only one work is selected this way, all works with the next lowest points score are added to the comparison.

Much better!

How about:

In each round, all works with the lowest point score are compared. If only one work is selected this way, all works with the next lowest point score are added to the comparison.

#698 ::: felice ::: (view all by) ::: May 19, 2015, 08:54 PM:

J Thomas @691: "The other 4 works are the ones with the highest point scores at that time, which have had high point scores all along."

Not necessarily all along - some of them might have had fewer points than the slate early on, but won an elimination comparison against another non-slate work and thereby gained enough points to put them above the slate; and each work eliminated by the slate could change the relative positioning of works ranked above the slate; but that's a minor technicality.


Jameson Quinn @692: "@688 is, I think, wrong; they could in fact eat everyone else, because the system doesn't realize they're tying for 5th, it thinks they're tying for 8th/9th."

You're right, that's a slight bug in the system; I think the way I described it is the way it should work, though. Perhaps 3.8.8.3 should be something along the lines of: "If fewer than four remaining works are not being compared, then instead none of the works from that round should be eliminated..." or "If elimination of all but one of the compared works would reduce the number of ballots to fewer than the minimum number of finalists, then..."

Though that would mean a tie between 4th and 5th would always make the 6th a finalist too (or more if there's a tie for 6th too), which we don't really want. Though with simultaneous elimination, that will sometimes happen anyway: we always have to compare all three of 1,2,2, and if eg 4th and 5th are tied on 30 points, 40 nominations each, while 6th has 29 points and 41 nominations, we have to keep all 6 to prevent the ballot dropping to four finalists. And I don't think we want to break ties involving 5th place.

How about this?
1, 1, 2 - compare all
1, 2, 2 - compare all
1, 1, 2, 2, - compare 1s only

That way, a tie involving 4th and 5th won't let more than one extra finalist in. There could theoretically be a 5-way tie for 4th, but that's effectively only possible with a perfect slate, and just means we get four non-slate finalists again; not a bad outcome.

Or if there's a tie for second-least points, only compare the one with fewest nominations against the work(s) with least points, eg in the example below, only compare E, F, and G, so if D and E are tied on points for 4th, E gets eliminated rather than all four being compared and nothing eliminated:

D: 10 points, 15 nominations
E: 10 points, 12 nominations
F: 9 points, 18 nominations
G: 9 points, 18 nominations

#699 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 09:14 PM:

[NO LOOKBACK] I don't think lookback gains us much and it's worth having a last ditch tiebreak that can't result in a tie, but lookback isn't it.

Jameson, I'm not a lawyer or accountant, but I do have some familiarity with non profit law due to my experience working with and running sf cons. I don't know of anything that would prevent Cherardine being covered by the same gofundme you use.

The measure looks good, but the title is awful -- dry as dust, and completely indistinguishing this measure from, well, any other measure that is trying to change the Hugos this year.

A WSFS short title should ideally be funny, memorable, and remind one of what the measure is about. It need not be totally descriptive, nor need it be dry.

Keep in mind that the short titles for new business this year are:

4 and 6
The Five Percent Solution
I remember the Future

The business passed on for ratification from last year is:

Popular Ratification (2 plus 1)
A Story by any other Name
Hugo Finalists
and WSFS Membership Types and Rates

Obviously, not all of these are great, and the last one isn't even good -- but particularly in a year with multiple hugo change proposals, this simply won't do.

I propose:

Put Proportion into Nomination

But frankly anything memorable and evocative would do:

Nomination by Pie
Sharing the Hugo
E Pluribus Hugo
Single Divisible Vote
Single Divisible Nomination

The last two aren't funny, but has the advantage of being -immensely- memorable, so we should probably do one of them.

That said, I think I favor "E Pluribus Hugo" as long as it won't cause a derail based on my non-existent Latin.

#700 ::: Joshua Kronengold ::: (view all by) ::: May 19, 2015, 09:17 PM:

Oh, also, I agree that we should bias towards two entries enter, one entry leaves. So if there's a tie for lowest, only the things tied for lowest should be compared.

In terms of stopping at a tiebreaker, if there's a tie that -includes- fifth place, the tie should not be resolved and all of the tied works shall be considered finalists.

#701 ::: Annie Y ::: (view all by) ::: May 19, 2015, 09:26 PM:

felice,

There would have been a four-way tie for fifth in Short Story too, if not for the 5% threshold. And last year, there were finalist ties in five categories. Ties are not rare!

OK - missed that somewhere in the shuffle - sorry.

Agree. In the number of nominations. But with a double system (points and nominations), ties will happen a lot less often. We are in a real tie only if both the nominations and the points match exactly and then going back all of them keep matching. This will happen a lot less often than the ties under the current system. Comparing them directly is not exactly called for -- you introduce second variable here.

Even if these ties come against each other, chances of them tying on points as well (especially with the looking back as a third variable) are small unless if we are talking very similar ballots...

I still have the feeling that we are over-complicating for the sake of small sets of data designed to show tying. These sets are good for looking at corner cases but selecting a strategy based on corner cases is not what we are trying to do I think. Still thinking on all this though...

#702 ::: Annie Y ::: (view all by) ::: May 19, 2015, 09:37 PM:

felice @ 698,

Now describe that example with the 1s and looking forward to see if we also want the 2s in a way that does not require 3 paragraphs (and does not make people wonder why we need to do that and if we are not actually manipulate data)? :)

And what are you doing for:

1,1,1,2,2
1,1,2,2,2
1,1,2,2,2,2
1,1,1,2,2,2
(and so on)
Just the 1s? If so - that first 1,1,2 is like a sore thumb. Grab the 2s somewhere - what is the common element that makes us grab them?

If you want to look at nominations to figure out what you are grabbing (your last paragraph)

Points: 1,1,1,2,2,2
Nominations: 2,2,2,2,2,2

Why do we need to bring the 2s with us?

Same example:
Points: 1,1,1,2,2,2
Nominations: 2,2,2,4,4,4
Same question?

Points: 1,1,1,2,2,2
Nominations: 2,2,2,6,2,4
The 1s are still the ones loosing a member.

Points: 1,1,1,2,2,2
Nominations: 3,4,5,6,2,3

This is a case where the 2s are losing one of their members if we bring it in the elimination but that outlier will get kicked on a next round anyway because it is lower than all other 2s. Plus this is the case with a single nomination anyway - these will always track high at the start.

It is not just about building a model here - it is also about being able to explain it clearly enough so people can buy into it and support it.

#703 ::: P J Evans ::: (view all by) ::: May 19, 2015, 09:52 PM:

702
It's also important that we not get hung up on less-likely conditions and edge cases. You cannot foresee everything, and this isn't a production system, but a test system, being tested with, AFAICT from what people are saying, minimal sets of test data. (Has anyone actually run this model with a full category of data?)

#704 ::: felice ::: (view all by) ::: May 19, 2015, 10:14 PM:

[TIES]
Ok, how about this:

"3.8.8.1: In each round, the two works with the least and second-least total points will be compared. In the event of a tie for least and/or second-least distinct total points, all out of those works having the least or second-least distinct total nominations will be compared. Of the compared works, the one that appears on the fewest number of ballots will be removed from all nomination ballots for subsequent rounds as if they had never appeared on the ballots.

...

3.8.8.3: If fewer than four remaining works are not being compared, then instead none of the works from that round should be eliminated, and all remaining works shall appear on the final ballot."

Eg A: 10/20, B: 10/15, C: 9/12 - only compare B and C.
A: 10/20, B: 10/15, C: 9/13, D: 9/12 - only compare C and D.
A: 10/20, B: 10/15, C: 9/18, D: 9/18 - compare B, C, and D.
A: 10/20, B: 10/15, C: 9/27, D: 9/27, E: 9/27 - compare A and B.
A: 10/15, B: 10/15, C: 9/18, D: 9/18 - compare all.

Comparing only the works with least or second-least distinct total nominations makes absolutely no difference except when there's a tie involving 5th, but stating it as a general principle rather than additional special handling for ties involving 5th seems tidier to me.

That way, if there's a points-only tie between 4th and 5th, only one of them (at most) will be compared against the work(s) with 6th most points. Though if there's a points-and-nominations tie between 4th and 5th, and all 6th placed works have the same number of nominations, they're all compared and nothing is eliminated. This is a pretty unlikely situation, but will happen occasionally. It's the best way I can see to let tied works bubble up to 5th place together but prevent them swallowing up the 4th or higher places.

If we break all ties and don't ever allow simultaneous elimination, then this process isn't necessary. But I think we agreed that allowing tied works to eliminate each other was worse than letting them be eliminated simultaneously by a third work with more nominations, which is the reason we introduced the "compare all of 1,1,2" rule.

#705 ::: Annie Y ::: (view all by) ::: May 19, 2015, 10:25 PM:

@703 - agree, see my last paragraph in 701 :)

A set can be designed for any eventuality. The work with the most votes always* makes the ballot - it beats anyone in a straight elimination showdown no matter how low the points are. Actually in a normal year, the nominees will be close enough...

*OK - there is a special case here.
The top 2 (or more works - can be from a slate, can be just unlucky pair of works) gets the same number of votes and tie on points lower than at least 5 other works. This can kick them both out as an unresolved tie. The looking back should clear them from this case (I think?) but we are still loosing one of them. We can add an explicit rule that the work(s) with the most nominations are always added to the ballot at the end if they are not there already. This will put a clean slate full on the ballot (so too many nominees, ugh) but only if it is a real slate - 300 people voted it straight; noone else voted even one of the works (1 person picking up a work will kick the points and votes and deal with the whole situation)

It is such an outlier case but if the proposal makes it through, Murphy's laws will make sure we get that in at least one category. And kicking out the most popular nomination is just ugly.

OK - just thinking aloud. Did I miss where we discussed that earlier?

#706 ::: felice ::: (view all by) ::: May 19, 2015, 10:42 PM:

Joshua Kronengold @699: "[NO LOOKBACK] I don't think lookback gains us much and it's worth having a last ditch tiebreak that can't result in a tie, but lookback isn't it."

Agreed.

"The measure looks good, but the title is awful -- dry as dust, and completely indistinguishing this measure from, well, any other measure that is trying to change the Hugos this year. A WSFS short title should ideally be funny, memorable, and remind one of what the measure is about. It need not be totally descriptive, nor need it be dry."

Agreed.

"That said, I think I favor "E Pluribus Hugo" as long as it won't cause a derail based on my non-existent Latin."

That works for me. Or "Hybrid Divisible Vote", or just "Single Divisble Vote (LPE)".

Annie Y @701: "Agree. In the number of nominations. But with a double system (points and nominations), ties will happen a lot less often. We are in a real tie only if both the nominations and the points match exactly and then going back all of them keep matching. This will happen a lot less often than the ties under the current system."

Real ties will happen more often than you think, because by the time we're down to the last 6 or 7 candidates, there are very few other nominees left for points to be split with. Only a small fraction of the people who nominated one or more of the tied works will also have nominated one or more of the other 4 or 5 remaining works, and the chances of it being the same fraction aren't insignificant (it could quite possibly be zero, ie none of the people who nominated D and/or E also nominated A, B, C, F, or G). Maybe a tie that matters will only happen every few years; it's still a bad thing that we should prevent happening.

Lookback will usually break the ties, unless the 5% threshold stays in which case it may well be useless, and as I discussed in #686 it seems to be an utterly arbitrary tiebreaker, and if we're going to break ties arbitrarily (and I think we should), we may as well use something simpler and 100% effective ("first appearance on any ballot" works nicely).

J Thomas @695: "B has 100 votes while F has 51 votes, so F is eliminated and the winners are ABCDE. But if the single vote for A is removed, then for the last round, A has 50.25 points while B has 50 points. A has 101 votes and B has 100 votes. B is eliminated. The winners are ACDEF."

That's an extremely unlike unstable situation, requiring the highest and lowest scoring of the last six works to be separated by only 1.25 points. Rather less probable than two works tying for points and nominations, I think.

#707 ::: Jameson Quinn ::: (view all by) ::: May 19, 2015, 10:45 PM:

@699: It's not that it would be illegal for electology.org to (hypothetically) buy tickets for cheradenine. It's that it's no longer my decision. I have clearance to say that buying my own tickets is a valid use of money to further the organization's goals; and while I personally think that would also be true for cheradenine, I don't have the authority to decide that by myself, and thus I can't collect that money under the same gofundme.

It's probably a moot point, though, because cheradenine has yet to express any interest in this possibility.

...

I still think that if it's 11,11,12, only 11,11 should be considered. If 12 is eliminated in this case, then 13, then 14, while 14 could have beaten 15 and been a finalist... I think the point of the SDV-LPE process is defeated, and the voters for 14 have too much of a bullet-voting incentive.

...

I like "E pluribus Hugo". Or is it too US-centric? If we want something dry and descriptive, then "Proportional Hugo Nominations System".

#708 ::: colin roald ::: (view all by) ::: May 19, 2015, 10:56 PM:

To clarify 3.8.8.1: "In each round, the two (2) works (or more, in the case of a tie) with the least and second-least total points will be compared. Of those works, the one(s) that appear on the fewest number of ballots will be removed from all nomination ballots for subsequent rounds as if they had never appeared on the ballots."

Is it the intention of this proposal that if two works are tied for lowest point total (say Alpha and Beta both have a score of 0.45) you nevertheless *also* select Gamma (which has a score of 0.5) for showdown, and then if Gamma has only one approval while Alpha and Beta each have two, then Gamma is eliminated and both Alpha and Beta survive the round?

That appears to be the meaning of the wording here, but it does not match my expectation from the casual phrasing "select the two works with the lowest point totals for showdown". I'd expect you'd only expand to three in the showdown if you "have to" – ie, if Alpha is lowest and Beta and Gamma are tied for second-lowest.

#709 ::: J Thomas ::: (view all by) ::: May 19, 2015, 11:09 PM:

@704 felice

If we break all ties and don't ever allow simultaneous elimination, then this process isn't necessary. But I think we agreed that allowing tied works to eliminate each other was worse than letting them be eliminated simultaneously by a third work with more nominations, which is the reason we introduced the "compare all of 1,1,2" rule.

I'm more unsettled about all this than I was day before yesterday.

I thought double ties would be very rare, so we needed something that would sound fair to people at the meeting. It didn't really matter how well it worked if it didn't actually happen.

But we shouldn't decide how to do things that *do* happen based on how they affect things that *don't* happen. Ties for lowest score will happen every year, and we should handle them based on what works for them, not on what might hypothetically happen someday.

But now I don't know how common double-ties will be. It was rare in the 1984 data. That was a third the number of ballots today, and there weren't as many works to dilute the vote. Also we lack data about slates.

If double ties really are important, important enough to change common routines to affect them, we need to be careful. It isn't just about saying something that sounds go