Go to Making Light's front page.
Forward to next post: “You’re asking me to agree that my great-grandparent and great-great-grandparents were monsters.”
Subscribe (via RSS) to this post's comment thread. (What does this mean? Here's a quick introduction.)
After over 3000 messages and nearly two and half months of discussions, we have put together a simple Hugo Award nomination system that meets our primary goal:
No group — whoever that group may be — should be able to absolutely prevent nominees from having the chance to be considered for the Hugo Award.
This works both ways. Not only should a slate not be able to force all other nominees off the final ballot, but just because a nominee appears on a slate, it should not be disqualified from appearing on the final ballot. No “shadowy cabal” should be able to prevent a nominee from being considered, either. All nominees deserve a fair chance. E Pluribus Hugo accomplishes this goal.
The system was developed by fans with a wide range of interests in science fiction and fantasy, including those who prefer the genres supported by the Sad Puppies 3 slate. We hope that it will truly meet the needs of all varieties of SFF fans.
The purpose of this proposal is explicitly not to change the rules so as to prevent the “new voices” of 2015 from being heard. There is no shadowy cabal here. All discussions, including our missteps and paths not taken, have been conducted in a completely open forum. You can find all of those discussions in the following threads:
On voting systems: a guest post from Bruce Schneier
Discussing Specific Changes to the Hugo Nomination Election: Another Guest Post By Bruce Schneier
Discussing Specific Changes to the Hugo Nomination Election: A Post Not By Bruce Schneier
E Pluribus Hugo: Out of Many, A Hugo
This thread is for questions and answers (and hopefully voices of support!) from the community about the system. We hope that the FAQ attached to the proposal will answer many of your questions, but we want to be available for any other questions that may arise. We sincerely hope that you will give this proposal a thoughtful and fair hearing. In this effort we are not Social Justice Warriors, we are not Puppies, and we don’t hate anyone. We are simply fans who want to save the Hugo from sinking into irrelevance as a result of the flaw in the nomination system that has been pointed out this year.
The official proposal can be found here once it has been posted by the Sasquan business meeting chair. We look forward to hearing from you.
Hello and welcome!
As the top post indicates, this is the place to ask questions, comment, or offer support for E Pluribus Hugo. We've been working on this for a long time, so we're very much looking forward to hearing from the rest of the community.
For those that don't know me, I'm Keith Watt, but I'm called "Kilo" (my callsign when I was in the Navy) by essentially everyone. While I've provided some leadership to our work, it really has been a group effort, so any of us will be glad to answer any questions you might have.
Welcome aboard!
Kilo
As a fellow swabbie, may I just say that while it has been a long road to get here, I do believe that we've got something that is viable. Your efforts to keep us on track have been greatly appreciated.
I haven't contributed more than the odd nitpick (Abi – are you really happy with that "whomever" in the original post?) but am deeply impressed by what others have wrought. Wreaked? No, wrought. Onward!
While things were being hammered out, folks did various kinds publicly viewable simulations, which were interesting to look at. I wonder, in the stretch where I needed to put attention elsewhere, did anyone do a simulation of reasonably historical data plus a this-year-sized slate? Being able to compare and contrast with the current process in some reasonably visual way would probably be handy.
Doire on June 20, 2015 at 7:31 am saidBrian, would you object to me quoting your remarks made here at ML, so that those who are interested can discuss them?
Brian Z on June 20, 2015 at 7:41 am said:
Doire, this forum gets people lots of perspectives dropping by, and here is where I’d hoped to have a discussion just at this moment! But I’m interested in having points I raise discussed at ML as well, as I mentioned to the moderator a little while ago.
As Brian is interested in having his points discussed at ML, but not enough to post them, I’ve copied those relating to EPH below.
1 Brian Z on June 19, 2015 at 7:52 pm said:So there we are, the case against EPH. How do we persuade other readers that these are not problems?
[snip Gene Wolfe discussion]
EPH
The tragedy of “E Pluribus Hugo” is that it replaces the original goal of excellence – shortlisting the five works that the greatest number of fans identify as among the very best of the year – with the goal of fairness – selecting five works that make the greatest number of interest groups happy, while at the same time penalizing the voter quite harshly for choosing more than one thing should one of them be something that many others also agree should win. The creators, proponents, and cheerleaders will tell you it is not about fairness and nothing will change, but they are shading the truth, deluding themselves, and have not thought it through carefully, respectively.
Kevin Standlee (@10:02am) hasn’t clarified what he means by “spread the votes out smoothly.” Will Kate Paulk “spread the votes out smoothly,” or will her SP4 book group gleefully squabble and debate and try to sway each other as they reach some degree of limited consensus about a bunch of works they share a common interest in? Will NESFA and many many others “spread the votes out smoothly” with shortlists? Will GRRM or Scalzi “spread the votes out smoothly” by plugging the next The Goblin Emperor or Leckie? Will those who don’t buy Tor because they stand with real warrior for social justice and prison chaplain Peter Grant against being called all those terrible things “spread the votes out smoothly”?
Does anyone honestly think Vox Day is going to fight the next war using the strategy of the last one? Of course if there are new nomination rules in 2017 he is going to do something different. Clown shoes or no, he has a few smart people on his side. He may not “capture all five slots” but he and others can still capture a whole lot of them, and if you dare him to he surely he will find new ways to ruin your experience of what should be a joyous fan award.
And do you honestly believe your voting behavior won’t change if 1) you are penalized for selecting a work that already has a reasonable chance of making the ballot because its success sucks the strength from your vote for an equally or more beloved but less-popular work, 2) the field becomes even more politicized than it already is with these culture wars, and/or (presumably and) 3) the “ravening hordes” [andyl @10:58] are amassing at the gates planning a fresh assault? Do you really?
Vastly expanding the number of nominators won’t prevent all “disaster scenarios”, but it is going to help a lot. It also means that the Hugos of the future will never again look like the Hugos of the past. Sorry.
2 Brian Z on June 19, 2015 at 11:46 pm said:
JJ
you’d like to derail any possible rules changes which might diminish the effectiveness of slates (Any guesses why? Bueller? Anyone?).
Ad hominem.
advice from an expert in how voting systems works — as well as actually tested with real and mocked-up ballot data.
Appeal to authority. And what on earth do 1984 ballots cast under old rules in a vastly different situation have to do with potential 2017 conditions and voter behavior?
The proposal prevents a slate from being able to monopolize the ballot (as happened this year) while not compromising the integrity of any individual’s votes.
Has it been tested against one to multiple slates and other attacks? I’ve already explained why I think it compromises the integrity of an individual’s votes – would you care to comment?
3 Brian Z on June 20, 2015 at 12:07 am said: [snip]JJ’s “Appeal to authority” was deployed in order to avoid responding to the specific concerns I raised.
I’m glad to know you are concerned about impact on voter behavior. We all should be. And I’m no expert but I think this could result either in more mediocre final ballots, a direction I think we are already headed in over the past several years, personally – and it has nothing to do with cabals, puppies or left-right politics. I also think there are probably a number of ways it could be attacked that haven’t got full attention, and that’s something I’d like to see an independent groups of experts – not the ones who designed it or support it – look at and tell us more about.
4 Brian Z on June 20, 2015 at 12:18 am said:
Soon Lee, I’ve stated in this comment
http://file770.com/? p=23227&cpage=17#comment- 290452
and my subsequent comments above what my specific concerns are.
Competing slates are not “a proposed solution.” They are a pattern of voter behavior that may or may not emerge and we may or may not have control over. It is certainly worth thinking about how voters will behave under these proposed conditions and in this general climate (rather than rerunning irrelevant 1984 ballots). I’m not an expert but hopefully there is an expert who was not involved in drafting EPH who can devise ways to attack it and give us a report on what might happen. Hopefully that will happen before ratification (at least).
5 Brian Z on June 20, 2015 at 4:07 am sais:
Abi Sutherland, thanks for your kind invitation and if folks would like to discuss the several concerns I mentioned here over there, I’m willing to join them.
I’ve mentioned before that I have no complaints about the moderation of Making Light, aside from a general (personal) distaste for the disemvoweling thing that I recognize may not be shared by others. [snip]
There may or may not be time between now and August for an independent review of EPH and possible vulnerabilities. At least, maybe something like that might be arranged before ratification, if passed.
Best of luck with presenting the case for the proposal.
6 Brian Z on June 20, 2015 at 4:20 am said:
influxus, I feel a “shading” was pretty explicit in for example Question 7 of their FAQ, “Isn’t it true that any voting system can be gamed (or strategized, etc.)?” An earlier post pointing to that vanished – multiple links? – so let me reiterate. There are many ways there might be potential for changes in either individual members’ voting behavior, or groups of members looking for ways to game the system, and I would like to see more evidence for the FAQ’s conclusion:
There are many ways for your strategy to fail, or even backfire, and only one, highly-specific way for it to work.
Nominating what you think is Hugo-worthy really is your best strategy.
7 Brian Z on June 20, 2015 at 4:54 am said:influxus, it appears we agree on several of these points in principle, but disagree on how comprehensively the FAQ covered the ramifications of potential changes in voting behavior of individuals and (potentially multiple) groups in 2017, and perhaps also on whether File 770 is a suitable venue for discussing that concern.
8 Brian Z on June 20, 2015 at 5:09 aminfluxus, I take your point that “shading” was harsh. OTOH, when something like this is introduced one might wish to see potential exploits and negative outcomes discussed more thoroughly than they were in that particular FAQ example. I hope they can be considered as carefully as possible in the time available and as you also mentioned that provisions are made for outside expert scrutiny.
9 Brian Z on June 20, 2015 at 5:15 am
Abi Sutherland, specifically I’m concerned with the FAQ’s implication that because “voting strategically” is deemed in the FAQ to be “technically risky”, members will choose to vote the way considered normal/traditional.
Yes, I hope for the best outcome for the Hugos, thanks for asking.
10 Brian Z on June 20, 2015 at 6:25 am said:
influxus, you thought of one valid concern, but other implicit assumptions worry me too.
How can we accept the premise that people will uniformly perceive an incentive to go back to behaving as straight and narrow, “traditional” or “normal” Hugo voters, and return to the “correct” pattern of nominating a set of things they each individually find most worthy, as a result of the rules have been changed to make alternatives apparently “too risky”? Actual Hugo voters, including those who thought they didn’t have a prayer of getting their stuff on the ballot, have done things like 1) loosely congregate and each nominate a couple of things they liked from a list of suggestions, and 2) follow the advice of a favored author/publisher to vote a straight ticket “as is.” At a gut level (at least) I don’t feel penalizing this behavior by making “non- acceptable” voting strategies more risky will be any more likely to convince people to pipe down and go back to behaving the way they are “supposed to” than launching a campaign to No Award their beloved authors sight unseen.
I’d also wonder how the risks of voting strategically can be mitigated by cooperation together with others and increasing one’s knowledge (or a group’s knowledge) about the likely behavior of others. We have several ready-to-wear factions already in existence, possible incipient ones, and social and technological means by which one might learn more, broker that information, and influence behavior.
If everyone had a set mental list of 1-5 things most desired to “win” we wouldn’t have a problem in the first place. (Actually, it is a fan award, so it isn’t the nominated work or author that “wins” – it is the fans that “win” when they see the happiness of an author cherished by the community as a whole, beaming when their name is read out, but maybe that is for a separate discussion.) So what if many voters and groups are motivated by desire to support a wider set of works/authors/community- members without caring exactly which one wins? What if some are just interested in making a political statement and are happy as long as they feel they’ve made their point?
Just a couple questions off the top of my head, sorry if I am rambling slightly.
11 Brian Z on June 20, 2015 at 6:57 am said:
Ray,
More mediocre than the bland to outright crap stuff Brad and Theodore picked for this year’s ballot?
No, I meant more mediocre than in past decades when the Hugo system did a pretty good job at identifying excellence in most years. (I personally think it started to slide noticeably a couple years ago – before, admittedly, kind of going off a cliff in 2015 – but that’s my personal opinion.)
But we just can’t directly compare a time when a couple hundred people routinely licked postage stamps to the coming years when thousands from the cyberelectorate will nominate in a hypercharged atmosphere where you have to read 1000+ comments on each day’s roundup. Whatever happens, we are not going back to the past/
David@3:
Abi – are you really happy with that "whomever" in the original post?
That one's on me. That's what I get for trying to dash something off before heading out on a road trip...
Kilo
5
Brian Z hasn't said anything, anywhere, that leads me to think he can contribute usefully on any subject that isn't either filking or puppy-dominated.
Doire@5:
The comments your quoted were a bit hard to follow, since we're only getting half of the conversation (you didn't have permission to post the other half; I understand). As such, I replied to Brian over on File770. I'm not sure he actually makes much of a case against EPH, so much as he's trying to make a case for doing nothing at all.
I do agree that we need to be clear to everyone that the best "strategy" is to nominate as many works as you think are worthy. Nothing else is likely to help you, and it might hurt you. I think that will resolve itself naturally if the proposal passes, since some will try strategy no matter what we say. When the first results come out, it will be obvious that it doesn't work. Still, we need the proposal to pass for them to see that! So we need to be clear on the strategy-resistant nature of EPH.
Thanks for link, I might not have seen it on File770.
Kilo
David Langford @3:
Fixed. I was betwixt and between tasks and mostly checking formatting issues when I posted. Thanks for catching it.
I find Brian's main point — that EPH is looking for "fairness" while the old system was maximizing "excellence" — to be laughably wrong in both regards. His ancillary aspersions are no better.
The point of EPH is not fairness, but using the evidence of the ballots most efficiently and consistently to maximize the probability that the best work of the year is among the finalists. Period. "Fairness" is not the point.
"Efficiently and consistently" are statistical terms of art, and as a doctoral student in statistics, I believe that under a reasonable model these properties of EPH could be proven (that it is more efficient than non-proportional alternatives, that is; not that it is maximally efficient, because it almost certainly isn't), similarly to the Condorcet Jury Theorem. No, I'm not going to actually work up such a proof (that would be months of work), but I'd be happy to discuss what I think it would involve with anybody who shows an understanding of the issues involved. I've seen no indication that Brian Z has any such understanding.
My reply to Brian was approved, but it ended up being the last post in the thread, so I'm not sure anyone actually saw it. I don't think there's much point in reposting it over there as the discussion seems to have (mostly) moved on. But I'll repost it here in case someone else wants to see it:
=======================================
Hello Brian –
I’m certainly happy to answer any questions that I can, here or anywhere else. As others have said, it’s true that we have set up a thread specifically for people to ask questions about EPH, but I don’t want to imply in the slightest that it shouldn’t be discussed anywhere else. In fact, I’m hoping it’s discussed -everywhere-. However, the down side of that is that I can’t read and respond to every post out there. That’s why some of the others were pointing out the Q&A thread — it’s a great place where you can get a response to your concerns, since it’s one place that is being checked regularly. That said, there’s absolutely nothing wrong — of course! — with discussing EPH absolutely anywhere. I’m not sure how we can address concerns that aren’t asked of us, however.
Moving to some of your concerns…
The tragedy of “E Pluribus Hugo” is that it replaces the original goal of excellence – shortlisting the five works that the greatest number of fans identify as among the very best of the year
Well, I’m not sure I agree with your definition of “excellence”. What you propose, I would define as “popularity”. And fundamentally, that’s what a “People’s Choice” award such as the Hugo really is: It’s a measure of popularity (as an aside, I’ll note that’s not necessarily equivalent to “sales”, though the two certainly often overlap).
Continuing with the second half of that phrase:
– with the goal of fairness – selecting five works that make the greatest number of interest groups happy
Again, the issue isn’t fairness as such, it’s popularity. First, here again I want to be explicit: I -like- the “rayguns and rockets” genre; I have no problem with the types of works many Puppies and similarly-minded folks are pushing for. Robert Heinlein is, in my opinion, one of the greatest science fiction authors who has ever lived (he’s certainly -my- absolute favorite). I have Lazarus Long quotes above my desktop. So please believe me when I say that, for me, at least, this isn’t about the Puppies politics or philosophy. Honestly. But the problem with slates — of any sort — is that they cannot measure popularity. Because they work by forcing all other nominees off the final ballot, there’s no way to know if those slate nominees were the most popular works or not. If “rockets and rayguns” truly are the most popular type of story, I want to win in a fair fight. Because the slate represents only a small percentage of the fans, we have no idea how popular their recommended works really are. So, yes, there is a fairness issue there, but perhaps not in the way you are indicating.
Continuing…
while at the same time penalizing the voter quite harshly for choosing more than one thing should one of them be something that many others also agree should win.
That’s not actually how it works, so I’m not really sure how to address that. Outside of an organized slate, I can only think of a few instances in which they could be “penalized” at all. One example that comes to mind is Dr. Who in the Dramatic Short Form category. They aren’t an organized slate. But there are a great many devoted fans (I was myself until Tennant left, but that’s another story), and they will all tend to choose the same five episodes without the need for explicit coordination. But is Dr. Who -truly- the best thing on television? It might be, but I’ll wager there are some fans who (apparently) like Orphan Black. So let’s take the very best Dr. Who episode and put it against the very best Orphan Black episode. Do you see that this is -increasing- excellence and not decreasing it? Instead of five Dr. Who episodes, we have the very best Dr. Who episode. We also have other works to compare it to. Is Dr. Who -really- the most popular science fiction show on television? It just might be, but now we can find out. In the past, we couldn’t. So it’s not -just- about fairness, nor is fairness really the major issue.
The creators, proponents, and cheerleaders will tell you it is not about fairness and nothing will change, but they are shading the truth, deluding themselves, and have not thought it through carefully, respectively.
Well, you are entitled to your opinion. I think you’re wrong on all three counts. But I’m not sure there’s anything we can objectively discuss there.
Does anyone honestly think Vox Day is going to fight the next war using the strategy of the last one?
Of course not. That’s why EPH was designed to be as strategy-resistant as possibly within a nomination system. Is it perfect? Definitely not — perfection is unobtainable in a voting system. Is it the best we can come up with that addresses the problems we see and what problems we can foresee? Absolutely.
And do you honestly believe your voting behavior won’t change if 1) you are penalized for selecting a work that already has a reasonable chance of making the ballot because its success sucks the strength from your vote for an equally or more beloved but less-popular work,
Please see the FAQ for why what you're suggesting isn’t true. We address this question specifically.
2) the field becomes even more politicized than it already is with these culture wars,
I’ll be honest with you. Prior to my work on EPH, I was despondent that the Hugos’ only course of action was exactly this: endlessly competing political slate wars. At that point, the Hugo has no relevance whatsoever. The Hugo, as an award, will have ceased to exist. But I was heartened to learn that the vast majority of fans -don’t- want politicized slate wars. So, with a few people — eventually growing to a large number of people — who felt the same way, we put heads together in the best of SF tradition, and tried to figure out a way to keep the Hugos from becoming irrelevant. Sure, there are those that will say it was irrelevant a long time ago.,because it was controlled by shadowy cabals pushing their own agenda. I disagree, but even if that were true, EPH works against -those- people as well. This is what I mean when I say we are trying to “save the Hugos”. We aren’t trying to “save” it from any one type of fiction or any one type of group. We’re trying to save it from becoming irrelevant.
and/or (presumably and) 3) the “ravening hordes” [andyl @10:58] are amassing at the gates planning a fresh assault? Do you really?
Yes, I do. Really.
Thanks, Brian, I’d welcome any other questions you might have. If I understand your posts, I think your preferred solution is to do nothing. For the reasons I’ve outlined above, that’s not an option, as far as I am concerned. We may have to agree to disagree on that, but I’m happy to discuss any of your perceived flaws in EPH.
Regards,
Kilo
Bruce@4:
I'd be happy to run any simulations you might want to see. Can you narrow down the type of data you want to look at?
Thanks,
Kilo
Hi Kilo,
That Dr Who / Orphan Black example you gave to Brian is an extremely clear articulation of why EPH is an improvement on the nomination process, regardless of the puppy fiasco.
Thank you!
Keith "Kilo" Watt #11: One definition I've heard of a "concern troll", is someone who shows up to try and talk you out of a winning position. We've had a few of those wander through the EPH threads here (and prior Puppy-related discussions as well), so I'm unsurprised to see them elsewhere..
Kilo: The #1 thing I'm thinking of, and that I believe others are too, is a demonstration of the claim that EPH can weaken slates without significant other effect. So what I'd hope to see is side-by-side tallies, from the same pool of data - some past year plus a fictional slate, or something like it - of current practice and EPH. The idea would be to show parallel rounds, leading to the final outcomes.
An obvious concern is that this would be taken as a direct slam on the Puppies crowds. Hence the suggestion of a purely fictional slate, possibly the League of Plergb Voters or the Crottled Greeps Liberation Front or some such. :)
Bruce@15:
Okay, sure. We've done that with the actual 1984 data set we have, plus with some computer-generated data sets that match the distribution of the 2014 data (though in this case, we don't have the actual ballots, of course). Do you have a preference?
Kilo
Kilo: The crucial thing I want is that it include a slate element, for the comparison. Beyond that...I'm hardly clueful enough to begin to have other preferences. Anything that makes for a good simple demonstration, really.
Bruce@17:
Okay, no problem. I'll run that tomorrow (my wife is having a pool party here soon).
TTYL..
K
Brad@819: Your proposed system would not have stopped the puppies this year. There is no basis in the constitution for throwing out the puppy slate nominations. The review process would have to find actual impropriety, breaking of the rules, to override the nominations, right? Well, there were violations of community norms, but there were *not* violations of the rules.
If your intention is for the review to remove things somebody (who?) just "doesn't like", then HELL NO! That is abhorrent.
20
Plus write-ins on the final ballot would increase the work for the administrators, as well as making it not very final. Let's not do that, please?
Bruce@17:
Apologies for the long data post, but you were interested in seeing every step. The code reports the current point total and number of nominations (which doesn't change) for each surviving nominee. It also reports which nominee was eliminated in each round. The nominees are just numbered 1-32 for simplicity's sake. #1 is the non-slate work with the most number of nominations; #27 is the non-slate work with the least number of nominations. #28-32 are the slate works, which I added manually. The data is designed to have the same statistical distribution as the 2013 Hugo Awards (I said 2014 before, sorry about that).
If a slate has -perfect- discipline -- that is if all slate works get exactly the same number of nominations (which is -highly- unlikely), then all slate works will eliminate each other due to the tie-break rule and none will survive. To simulate a more realistic case, I had #28 be the slate work with the most number of nominations, and #32 be the slate work with the least number of nominations.
In the case below, there are 1313 ballots, 200 of which are slate ballots. If you run the code without the slate ballots, then the final ballot is 1,2,3,4,5 under both the current system and under EPH, as you would expect. With the slate present, the winners are 28,29,30,31,32 under the current system -- the slate sweeps the category. Under EPH, the winners are 1,2,3,4,28 -- the slate only takes one slot.
Keep in mind that as the slate gets a larger percentage of the total ballots, they can potentially get more slots on the final ballot. And of course, if non-slate ballots also nominate a slate work independently, that work will get a boost as well.
So, with all that preamble, here are the results under EPH:
Current Works for Round 1:
1 Points: 90.6166666666667 Nominations: 193
2 Points: 67.3333333333333 Nominations: 138
3 Points: 63.35 Nominations: 135
4 Points: 62.5 Nominations: 133
5 Points: 61.3166666666667 Nominations: 118
6 Points: 48.5166666666667 Nominations: 101
7 Points: 43.9666666666667 Nominations: 91
8 Points: 42.7333333333333 Nominations: 90
9 Points: 31.2333333333333 Nominations: 74
10 Points: 30.65 Nominations: 69
11 Points: 36.2333333333333 Nominations: 68
12 Points: 33.9833333333333 Nominations: 62
13 Points: 27.2333333333333 Nominations: 61
14 Points: 26.8166666666667 Nominations: 58
15 Points: 26.7 Nominations: 56
16 Points: 23.2333333333333 Nominations: 55
17 Points: 27.7166666666667 Nominations: 54
18 Points: 26.7 Nominations: 53
19 Points: 21.65 Nominations: 52
20 Points: 21.8666666666667 Nominations: 50
21 Points: 24.3166666666667 Nominations: 50
22 Points: 23.6833333333333 Nominations: 50
23 Points: 21 Nominations: 50
24 Points: 22.8333333333333 Nominations: 50
25 Points: 24.9833333333333 Nominations: 50
26 Points: 0.583333333333333 Nominations: 2
27 Points: 0.25 Nominations: 1
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 1
Work(s) Eliminated:
27 Points: 0.25 Nominations: 1
Current Works for Round 2:
1 Points: 90.6166666666667 Nominations: 193
2 Points: 67.3333333333333 Nominations: 138
3 Points: 63.4333333333333 Nominations: 135
4 Points: 62.5 Nominations: 133
5 Points: 61.4 Nominations: 118
6 Points: 48.5166666666667 Nominations: 101
7 Points: 43.9666666666667 Nominations: 91
8 Points: 42.7333333333333 Nominations: 90
9 Points: 31.3166666666667 Nominations: 74
10 Points: 30.65 Nominations: 69
11 Points: 36.2333333333333 Nominations: 68
12 Points: 33.9833333333333 Nominations: 62
13 Points: 27.2333333333333 Nominations: 61
14 Points: 26.8166666666667 Nominations: 58
15 Points: 26.7 Nominations: 56
16 Points: 23.2333333333333 Nominations: 55
17 Points: 27.7166666666667 Nominations: 54
18 Points: 26.7 Nominations: 53
19 Points: 21.65 Nominations: 52
20 Points: 21.8666666666667 Nominations: 50
21 Points: 24.3166666666667 Nominations: 50
22 Points: 23.6833333333333 Nominations: 50
23 Points: 21 Nominations: 50
24 Points: 22.8333333333333 Nominations: 50
25 Points: 24.9833333333333 Nominations: 50
26 Points: 0.583333333333333 Nominations: 2
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 2
Work(s) Eliminated:
26 Points: 0.583333333333333 Nominations: 2
Current Works for Round 3:
1 Points: 90.7000000000001 Nominations: 193
2 Points: 67.3333333333333 Nominations: 138
3 Points: 63.6 Nominations: 135
4 Points: 62.5833333333333 Nominations: 133
5 Points: 61.4 Nominations: 118
6 Points: 48.5166666666667 Nominations: 101
7 Points: 43.9666666666667 Nominations: 91
8 Points: 42.7333333333333 Nominations: 90
9 Points: 31.3166666666667 Nominations: 74
10 Points: 30.65 Nominations: 69
11 Points: 36.2333333333333 Nominations: 68
12 Points: 33.9833333333333 Nominations: 62
13 Points: 27.2333333333333 Nominations: 61
14 Points: 26.8166666666667 Nominations: 58
15 Points: 26.7 Nominations: 56
16 Points: 23.2333333333333 Nominations: 55
17 Points: 27.7166666666667 Nominations: 54
18 Points: 26.7 Nominations: 53
19 Points: 21.7333333333333 Nominations: 52
20 Points: 21.8666666666667 Nominations: 50
21 Points: 24.3166666666667 Nominations: 50
22 Points: 23.6833333333333 Nominations: 50
23 Points: 21.1666666666667 Nominations: 50
24 Points: 22.8333333333333 Nominations: 50
25 Points: 24.9833333333333 Nominations: 50
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 3
Work(s) Eliminated:
23 Points: 21.1666666666667 Nominations: 50
Current Works for Round 4:
1 Points: 92.5833333333334 Nominations: 193
2 Points: 68.9666666666667 Nominations: 138
3 Points: 65.4 Nominations: 135
4 Points: 63.55 Nominations: 133
5 Points: 61.7833333333333 Nominations: 118
6 Points: 48.9833333333333 Nominations: 101
7 Points: 44.4833333333333 Nominations: 91
8 Points: 43.4833333333333 Nominations: 90
9 Points: 31.7 Nominations: 74
10 Points: 31.1166666666667 Nominations: 69
11 Points: 36.8666666666667 Nominations: 68
12 Points: 34.3666666666667 Nominations: 62
13 Points: 27.4833333333333 Nominations: 61
14 Points: 27.15 Nominations: 58
15 Points: 26.75 Nominations: 56
16 Points: 23.9833333333333 Nominations: 55
17 Points: 28.4333333333333 Nominations: 54
18 Points: 27.3333333333333 Nominations: 53
19 Points: 22 Nominations: 52
20 Points: 22.4166666666667 Nominations: 50
21 Points: 24.8166666666667 Nominations: 50
22 Points: 24.0333333333333 Nominations: 50
24 Points: 23.25 Nominations: 50
25 Points: 25.0666666666667 Nominations: 50
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 4
Work(s) Eliminated:
20 Points: 22.4166666666667 Nominations: 50
Current Works for Round 5:
1 Points: 94.0833333333334 Nominations: 193
2 Points: 70.3833333333334 Nominations: 138
3 Points: 67.15 Nominations: 135
4 Points: 64.3 Nominations: 133
5 Points: 62.2 Nominations: 118
6 Points: 49.9833333333333 Nominations: 101
7 Points: 46.2333333333333 Nominations: 91
8 Points: 44.15 Nominations: 90
9 Points: 31.8666666666667 Nominations: 74
10 Points: 31.2833333333333 Nominations: 69
11 Points: 37.0333333333333 Nominations: 68
12 Points: 34.95 Nominations: 62
13 Points: 28.15 Nominations: 61
14 Points: 28.0666666666667 Nominations: 58
15 Points: 28 Nominations: 56
16 Points: 25.0666666666667 Nominations: 55
17 Points: 28.6 Nominations: 54
18 Points: 28.3333333333333 Nominations: 53
19 Points: 22 Nominations: 52
21 Points: 24.9 Nominations: 50
22 Points: 24.45 Nominations: 50
24 Points: 23.5833333333333 Nominations: 50
25 Points: 25.2333333333333 Nominations: 50
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 5
Work(s) Eliminated:
24 Points: 23.5833333333333 Nominations: 50
Current Works for Round 6:
1 Points: 96.5 Nominations: 193
2 Points: 71.3833333333333 Nominations: 138
3 Points: 68.0666666666667 Nominations: 135
4 Points: 66.3833333333333 Nominations: 133
5 Points: 63.45 Nominations: 118
6 Points: 50.2333333333333 Nominations: 101
7 Points: 47.8166666666667 Nominations: 91
8 Points: 44.8166666666667 Nominations: 90
9 Points: 33.6166666666667 Nominations: 74
10 Points: 31.8666666666667 Nominations: 69
11 Points: 37.0333333333333 Nominations: 68
12 Points: 35.45 Nominations: 62
13 Points: 28.7333333333333 Nominations: 61
14 Points: 28.0666666666667 Nominations: 58
15 Points: 28.0833333333333 Nominations: 56
16 Points: 25.15 Nominations: 55
17 Points: 29.4333333333333 Nominations: 54
18 Points: 28.5 Nominations: 53
19 Points: 22 Nominations: 52
21 Points: 24.9833333333333 Nominations: 50
22 Points: 25.1166666666667 Nominations: 50
25 Points: 25.3166666666667 Nominations: 50
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 6
Work(s) Eliminated:
21 Points: 24.9833333333333 Nominations: 50
Current Works for Round 7:
1 Points: 98.2666666666667 Nominations: 193
2 Points: 72.05 Nominations: 138
3 Points: 70.0666666666667 Nominations: 135
4 Points: 66.9333333333333 Nominations: 133
5 Points: 64.95 Nominations: 118
6 Points: 51.0666666666667 Nominations: 101
7 Points: 47.9 Nominations: 91
8 Points: 45.95 Nominations: 90
9 Points: 34.1666666666667 Nominations: 74
10 Points: 32.6166666666667 Nominations: 69
11 Points: 37.6166666666667 Nominations: 68
12 Points: 36.0333333333333 Nominations: 62
13 Points: 29.2333333333333 Nominations: 61
14 Points: 28.5333333333333 Nominations: 58
15 Points: 28.3 Nominations: 56
16 Points: 25.8166666666667 Nominations: 55
17 Points: 29.65 Nominations: 54
18 Points: 28.6666666666667 Nominations: 53
19 Points: 22.25 Nominations: 52
22 Points: 26.0333333333333 Nominations: 50
25 Points: 25.9 Nominations: 50
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 7
Work(s) Eliminated:
19 Points: 22.25 Nominations: 52
Current Works for Round 8:
1 Points: 100.516666666667 Nominations: 193
2 Points: 73.4666666666667 Nominations: 138
3 Points: 71.9 Nominations: 135
4 Points: 68.1 Nominations: 133
5 Points: 65.7833333333333 Nominations: 118
6 Points: 51.9833333333333 Nominations: 101
7 Points: 48.4833333333333 Nominations: 91
8 Points: 46.95 Nominations: 90
9 Points: 35.4166666666667 Nominations: 74
10 Points: 33.0333333333333 Nominations: 69
11 Points: 38.2 Nominations: 68
12 Points: 37.0333333333333 Nominations: 62
13 Points: 29.8166666666667 Nominations: 61
14 Points: 29.45 Nominations: 58
15 Points: 28.4666666666667 Nominations: 56
16 Points: 26.4833333333333 Nominations: 55
17 Points: 29.9 Nominations: 54
18 Points: 29.0833333333333 Nominations: 53
22 Points: 26.2833333333333 Nominations: 50
25 Points: 26.65 Nominations: 50
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 8
Work(s) Eliminated:
22 Points: 26.2833333333333 Nominations: 50
Current Works for Round 9:
1 Points: 102.933333333333 Nominations: 193
2 Points: 73.6 Nominations: 138
3 Points: 72.65 Nominations: 135
4 Points: 70.0166666666667 Nominations: 133
5 Points: 66.1166666666666 Nominations: 118
6 Points: 52.95 Nominations: 101
7 Points: 49.65 Nominations: 91
8 Points: 47.95 Nominations: 90
9 Points: 36.5833333333333 Nominations: 74
10 Points: 34.5333333333333 Nominations: 69
11 Points: 38.5 Nominations: 68
12 Points: 37.8666666666667 Nominations: 62
13 Points: 30.45 Nominations: 61
14 Points: 29.7833333333333 Nominations: 58
15 Points: 28.8 Nominations: 56
16 Points: 26.4833333333333 Nominations: 55
17 Points: 29.9 Nominations: 54
18 Points: 29.1666666666667 Nominations: 53
25 Points: 27.0666666666667 Nominations: 50
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 9
Work(s) Eliminated:
25 Points: 27.0666666666667 Nominations: 50
Current Works for Round 10:
1 Points: 105.766666666667 Nominations: 193
2 Points: 76.15 Nominations: 138
3 Points: 73.1166666666667 Nominations: 135
4 Points: 70.8166666666667 Nominations: 133
5 Points: 67.6166666666667 Nominations: 118
6 Points: 53.2833333333333 Nominations: 101
7 Points: 50.4 Nominations: 91
8 Points: 48.0833333333333 Nominations: 90
9 Points: 36.75 Nominations: 74
10 Points: 34.7833333333333 Nominations: 69
11 Points: 39 Nominations: 68
12 Points: 38.6166666666667 Nominations: 62
13 Points: 30.6166666666667 Nominations: 61
14 Points: 30.45 Nominations: 58
15 Points: 29.65 Nominations: 56
16 Points: 27.2833333333333 Nominations: 55
17 Points: 29.95 Nominations: 54
18 Points: 29.6666666666667 Nominations: 53
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 10
Work(s) Eliminated:
16 Points: 27.2833333333333 Nominations: 55
Current Works for Round 11:
1 Points: 107.4 Nominations: 193
2 Points: 78.3666666666667 Nominations: 138
3 Points: 74.6166666666667 Nominations: 135
4 Points: 72.9 Nominations: 133
5 Points: 68.3666666666667 Nominations: 118
6 Points: 53.8666666666667 Nominations: 101
7 Points: 51.9 Nominations: 91
8 Points: 50.25 Nominations: 90
9 Points: 37 Nominations: 74
10 Points: 35.2833333333333 Nominations: 69
11 Points: 39.1666666666667 Nominations: 68
12 Points: 38.7833333333333 Nominations: 62
13 Points: 31.2 Nominations: 61
14 Points: 30.9166666666667 Nominations: 58
15 Points: 30.2833333333333 Nominations: 56
17 Points: 30.2 Nominations: 54
18 Points: 30.5 Nominations: 53
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 11
Work(s) Eliminated:
17 Points: 30.2 Nominations: 54
Current Works for Round 12:
1 Points: 108.7 Nominations: 193
2 Points: 80.95 Nominations: 138
3 Points: 76.3666666666667 Nominations: 135
4 Points: 73.6166666666667 Nominations: 133
5 Points: 70.1166666666667 Nominations: 118
6 Points: 54.5 Nominations: 101
7 Points: 52.4 Nominations: 91
8 Points: 51.0833333333333 Nominations: 90
9 Points: 37.0833333333333 Nominations: 74
10 Points: 35.45 Nominations: 69
11 Points: 39.6666666666667 Nominations: 68
12 Points: 39.3333333333333 Nominations: 62
13 Points: 31.7 Nominations: 61
14 Points: 31.5 Nominations: 58
15 Points: 31.6166666666667 Nominations: 56
18 Points: 30.9166666666667 Nominations: 53
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 12
Work(s) Eliminated:
18 Points: 30.9166666666667 Nominations: 53
Current Works for Round 13:
1 Points: 111.7 Nominations: 193
2 Points: 82.6166666666667 Nominations: 138
3 Points: 77.2833333333333 Nominations: 135
4 Points: 75.0333333333333 Nominations: 133
5 Points: 72.6166666666667 Nominations: 118
6 Points: 55.25 Nominations: 101
7 Points: 52.9 Nominations: 91
8 Points: 52.9166666666667 Nominations: 90
9 Points: 37.9166666666667 Nominations: 74
10 Points: 35.95 Nominations: 69
11 Points: 40.3333333333333 Nominations: 68
12 Points: 39.75 Nominations: 62
13 Points: 32.2 Nominations: 61
14 Points: 32.6666666666667 Nominations: 58
15 Points: 31.8666666666667 Nominations: 56
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 13
Work(s) Eliminated:
15 Points: 31.8666666666667 Nominations: 56
Current Works for Round 14:
1 Points: 112.5 Nominations: 193
2 Points: 84.6666666666667 Nominations: 138
3 Points: 78.25 Nominations: 135
4 Points: 76.3666666666667 Nominations: 133
5 Points: 74.7833333333333 Nominations: 118
6 Points: 56.1666666666667 Nominations: 101
7 Points: 54.2833333333333 Nominations: 91
8 Points: 54.3333333333333 Nominations: 90
9 Points: 39.0833333333333 Nominations: 74
10 Points: 36.7 Nominations: 69
11 Points: 40.75 Nominations: 68
12 Points: 39.8333333333333 Nominations: 62
13 Points: 32.8666666666667 Nominations: 61
14 Points: 34.4166666666667 Nominations: 58
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 14
Work(s) Eliminated:
14 Points: 34.4166666666667 Nominations: 58
Current Works for Round 15:
1 Points: 115.916666666667 Nominations: 193
2 Points: 86.1666666666667 Nominations: 138
3 Points: 80.4166666666667 Nominations: 135
4 Points: 78.7833333333333 Nominations: 133
5 Points: 76.7833333333333 Nominations: 118
6 Points: 56.8333333333333 Nominations: 101
7 Points: 55.3666666666667 Nominations: 91
8 Points: 55.75 Nominations: 90
9 Points: 41.5833333333333 Nominations: 74
10 Points: 37.2833333333333 Nominations: 69
11 Points: 41.3333333333333 Nominations: 68
12 Points: 39.8333333333333 Nominations: 62
13 Points: 32.95 Nominations: 61
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 15
Work(s) Eliminated:
13 Points: 32.95 Nominations: 61
Current Works for Round 16:
1 Points: 118.083333333333 Nominations: 193
2 Points: 88.0833333333333 Nominations: 138
3 Points: 81.6666666666667 Nominations: 135
4 Points: 80 Nominations: 133
5 Points: 78.4166666666667 Nominations: 118
6 Points: 58.8333333333333 Nominations: 101
7 Points: 57.9166666666667 Nominations: 91
8 Points: 57.9166666666667 Nominations: 90
9 Points: 43.25 Nominations: 74
10 Points: 38.4166666666667 Nominations: 69
11 Points: 41.75 Nominations: 68
12 Points: 40.6666666666667 Nominations: 62
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 16
Work(s) Eliminated:
10 Points: 38.4166666666667 Nominations: 69
Current Works for Round 17:
1 Points: 121.333333333333 Nominations: 193
2 Points: 91.4166666666667 Nominations: 138
3 Points: 84.5 Nominations: 135
4 Points: 82.8333333333333 Nominations: 133
5 Points: 79.75 Nominations: 118
6 Points: 60.5 Nominations: 101
7 Points: 60.1666666666667 Nominations: 91
8 Points: 59.25 Nominations: 90
9 Points: 44.8333333333333 Nominations: 74
11 Points: 42.9166666666667 Nominations: 68
12 Points: 41.5 Nominations: 62
28 Points: 41.2833333333334 Nominations: 200
29 Points: 40.2833333333334 Nominations: 199
30 Points: 39.7833333333334 Nominations: 198
31 Points: 39.45 Nominations: 197
32 Points: 39.2 Nominations: 196
Results of Round: 17
Work(s) Eliminated:
32 Points: 39.2 Nominations: 196
Current Works for Round 18:
1 Points: 121.333333333333 Nominations: 193
2 Points: 91.4166666666667 Nominations: 138
3 Points: 84.5 Nominations: 135
4 Points: 82.8333333333333 Nominations: 133
5 Points: 79.75 Nominations: 118
6 Points: 60.5 Nominations: 101
7 Points: 60.1666666666667 Nominations: 91
8 Points: 59.25 Nominations: 90
9 Points: 44.8333333333333 Nominations: 74
11 Points: 42.9166666666667 Nominations: 68
12 Points: 41.5 Nominations: 62
28 Points: 51.0833333333333 Nominations: 200
29 Points: 50.0833333333333 Nominations: 199
30 Points: 49.5833333333333 Nominations: 198
31 Points: 49.25 Nominations: 197
Results of Round: 18
Work(s) Eliminated:
12 Points: 41.5 Nominations: 62
Current Works for Round 19:
1 Points: 124.5 Nominations: 193
2 Points: 92.5833333333333 Nominations: 138
3 Points: 86.8333333333333 Nominations: 135
4 Points: 84.75 Nominations: 133
5 Points: 80.5833333333333 Nominations: 118
6 Points: 62.0833333333333 Nominations: 101
7 Points: 61.4166666666667 Nominations: 91
8 Points: 59.9166666666667 Nominations: 90
9 Points: 44.8333333333333 Nominations: 74
11 Points: 44.5 Nominations: 68
28 Points: 51.0833333333333 Nominations: 200
29 Points: 50.0833333333333 Nominations: 199
30 Points: 49.5833333333333 Nominations: 198
31 Points: 49.25 Nominations: 197
Results of Round: 19
Work(s) Eliminated:
11 Points: 44.5 Nominations: 68
Current Works for Round 20:
1 Points: 129.583333333333 Nominations: 193
2 Points: 94.4166666666667 Nominations: 138
3 Points: 89 Nominations: 135
4 Points: 86.4166666666667 Nominations: 133
5 Points: 81.5 Nominations: 118
6 Points: 64.6666666666667 Nominations: 101
7 Points: 63 Nominations: 91
8 Points: 60.9166666666667 Nominations: 90
9 Points: 45.5 Nominations: 74
28 Points: 51.0833333333333 Nominations: 200
29 Points: 50.0833333333333 Nominations: 199
30 Points: 49.5833333333333 Nominations: 198
31 Points: 49.25 Nominations: 197
Results of Round: 20
Work(s) Eliminated:
9 Points: 45.5 Nominations: 74
Current Works for Round 21:
1 Points: 134.75 Nominations: 193
2 Points: 97.4166666666667 Nominations: 138
3 Points: 93.0833333333333 Nominations: 135
4 Points: 87.5833333333333 Nominations: 133
5 Points: 83.1666666666667 Nominations: 118
6 Points: 67.4166666666667 Nominations: 101
7 Points: 64.6666666666667 Nominations: 91
8 Points: 62.9166666666667 Nominations: 90
28 Points: 51.0833333333333 Nominations: 200
29 Points: 50.0833333333333 Nominations: 199
30 Points: 49.5833333333333 Nominations: 198
31 Points: 49.25 Nominations: 197
Results of Round: 21
Work(s) Eliminated:
31 Points: 49.25 Nominations: 197
Current Works for Round 22:
1 Points: 134.75 Nominations: 193
2 Points: 97.4166666666667 Nominations: 138
3 Points: 93.0833333333333 Nominations: 135
4 Points: 87.5833333333333 Nominations: 133
5 Points: 83.1666666666667 Nominations: 118
6 Points: 67.4166666666667 Nominations: 101
7 Points: 64.6666666666667 Nominations: 91
8 Points: 62.9166666666667 Nominations: 90
28 Points: 67.5000000000001 Nominations: 200
29 Points: 66.5000000000001 Nominations: 199
30 Points: 66.0000000000001 Nominations: 198
Results of Round: 22
Work(s) Eliminated:
8 Points: 62.9166666666667 Nominations: 90
Current Works for Round 23:
1 Points: 139.333333333333 Nominations: 193
2 Points: 99.9166666666667 Nominations: 138
3 Points: 96.3333333333333 Nominations: 135
4 Points: 92 Nominations: 133
5 Points: 86 Nominations: 118
6 Points: 68.4166666666667 Nominations: 101
7 Points: 68 Nominations: 91
28 Points: 67.5000000000001 Nominations: 200
29 Points: 66.5000000000001 Nominations: 199
30 Points: 66.0000000000001 Nominations: 198
Results of Round: 23
Work(s) Eliminated:
30 Points: 66.0000000000001 Nominations: 198
Current Works for Round 24:
1 Points: 139.333333333333 Nominations: 193
2 Points: 99.9166666666667 Nominations: 138
3 Points: 96.3333333333333 Nominations: 135
4 Points: 92 Nominations: 133
5 Points: 86 Nominations: 118
6 Points: 68.4166666666667 Nominations: 101
7 Points: 68 Nominations: 91
28 Points: 100.5 Nominations: 200
29 Points: 99.5 Nominations: 199
Results of Round: 24
Work(s) Eliminated:
7 Points: 68 Nominations: 91
Current Works for Round 25:
1 Points: 145.75 Nominations: 193
2 Points: 100.833333333333 Nominations: 138
3 Points: 98.25 Nominations: 135
4 Points: 95.4166666666667 Nominations: 133
5 Points: 90.0833333333333 Nominations: 118
6 Points: 69.6666666666667 Nominations: 101
28 Points: 100.5 Nominations: 200
29 Points: 99.5 Nominations: 199
Results of Round: 25
Work(s) Eliminated:
6 Points: 69.6666666666667 Nominations: 101
Current Works for Round 26:
1 Points: 152.083333333333 Nominations: 193
2 Points: 106.75 Nominations: 138
3 Points: 102.666666666667 Nominations: 135
4 Points: 100.583333333333 Nominations: 133
5 Points: 94.9166666666667 Nominations: 118
28 Points: 100.5 Nominations: 200
29 Points: 99.5 Nominations: 199
Results of Round: 26
Work(s) Eliminated:
5 Points: 94.9166666666667 Nominations: 118
Current Works for Round 27:
1 Points: 157.5 Nominations: 193
2 Points: 111.166666666667 Nominations: 138
3 Points: 105.666666666667 Nominations: 135
4 Points: 107.666666666667 Nominations: 133
28 Points: 100.5 Nominations: 200
29 Points: 99.5 Nominations: 199
Results of Round: 27
Work(s) Eliminated:
29 Points: 99.5 Nominations: 199
Current Works for Round 28:
1 Points: 157.5 Nominations: 193
2 Points: 111.166666666667 Nominations: 138
3 Points: 105.666666666667 Nominations: 135
4 Points: 107.666666666667 Nominations: 133
28 Points: 200 Nominations: 200
Final Hugo Ballot:
1 Points: 157.5 Nominations: 193
2 Points: 111.166666666667 Nominations: 138
3 Points: 105.666666666667 Nominations: 135
4 Points: 107.666666666667 Nominations: 133
28 Points: 200 Nominations: 200
Because parts of this discussion were copied/commented on in several places, I’m putting my response to Kilo both here and there.
Hi Kilo,
I had to rejoin my quest over in mundanity for a bit. I’m glad you came over. File 770 roundups attract Star-bellied Sneetches as well as those whose bellies have no stars, so I hope you will understand my reasons for talking there.
The ML excerpts glossed over my review of Gene Wolfe’s The Sorcerer’s House (Tor, 2010) and Home Fires (Tor 2011), both of which should have been nominated for a Hugo. They were published (to my eyes) at the time the Hugo ballot began passing over more of the truly excellent work in favor of more of the passable-to-mediocre. Please buy them.
Also, I’m not a puppy. I currently have a filk zine, temporarily Canine Daze until I get a better name. (I had settled on The Outmarches, but RedWombat out of nowhere mentioned War Beaver Apocalypse and it’s tempting.) I’ve been linking stuff including great filk from ML, so let me know what I missed.
Now:
your definition of “excellence”. What you propose, I would define as “popularity”. And fundamentally, that’s what a “People’s Choice” award such as the Hugo really is: It’s a measure of popularity
So my vote is for radically expanding the community of readers – however geographically dispersed and sub-culturally fragmented they may be thanks to the internet – who take this yeoman's work seriously. I think Spider Robinson once advocated doing that by mail and it sputtered, but it couldn't hurt to try again.
If “rockets and rayguns” truly are the most popular type of story, I want to win in a fair fight… So, yes, there is a fairness issue there, but perhaps not in the way you are indicating.Rayguns are a red herring (I mocked them in my filking), and I see fairness from another angle. We “win in a fair fight,” in this fan award, by honoring an author cherished in our community. We can win bigger the more fans buy and read as much as they can and squabble over it before putting pen to ballot. The “puppies” put up a slate because they thought our system was broken and drawing large numbers of new fans into nominating would help. There are various perspectives on whether or how broken the system was, but they are fans with a right to their point of view, and that was at least the kernel of a useful idea, however dumb or hamfisted the actual slate campaign was. EPH is equally the kernel of a good idea, but equally hamfisted – like preemptively scheduling collective brain surgery and then telling everyone a talking cure wouldn’t have worked anyway.
But is Dr. Who -truly- the best thing on television?We are in total agreement. But the reign of Dr. Who is over, and to bring it up now makes EPH seem like rearguard action by a retreating army. Do you know why, some petty griping aside, nobody objected too terribly strenuously when Dr. Who kept winning? Because Whovians were a group of hardcore fans who really, really, really wanted things they loved to win fan awards.
EPH was designed to be as strategy-resistant as possible…Is it perfect? Definitely not... Is it the best we can come up with that addresses the problems we see and what problems we can foresee? Absolutely.Three campaigns taking three spaces would suck too, and anyways, slates of five things is so February 2015. I mentioned just one thing that evolved since: there might be an attempt to keep all work by certain publishers off (or on) a 2017 ballot, or to push up (or down) any and all authors fitting political criteria. Our community will have two full years to think of something even more ghastly to do to ourselves than that, and we are, um, creative people.
When I say we aren’t really facing the truth, I mean that at its heart this is (as noted on the other thread) a squishy human problem. Not a math problem. EPH seems less vulnerable than some other systems, and thank you for this hard work. But there is already a mechanism to counter bloc voting – No Award. And to the extent that EPH puts on training wheels that we’ve never needed til now and allows us to go back to playing in our favorite sandbox without attempting to read (while nominations are open) the widest sampling that we feasibly can of what is being written in our field, discuss it, and find whatever common ground we can about what is great SFF, we are failing in our mission.
In addition to culture wars and fragmentation of readers’ tastes there are other structural problems that hinder us, like the chaotic state of short fiction markets and the explosion of the 18-book series where one book or even a trilogy would do. Math won’t fix those either.
Please see the FAQ for why what your suggesting [voting behavior won’t change] isn’t true.
Your FAQ assumes people have pre-drafted a list of 1-5 things they individually want to see “win” and will act “rationally” as an individual voter to give at least one those things the best shot, statistically, at winning. Will they? Even if some feel this is all being shoved down their throats, and All Fandom is Again Plunged into War?
We aren’t trying to “save” it from any one type of fiction or any one type of group. We’re trying to save it from becoming irrelevant.
Me too. I tried to discuss the human dimensions, and I recommend more consideration of how your proposal’s technical strengths might hold up in a gale of fannish wrath. Good luck.
Oops - like Brian I mistakenly posted this in the wrong thread at first. Moderators, please feel free to delete it from there.
I endorse E Pluribus Hugo. I think it is the best fix to prevent slates from dominating the process in future without irreparable damage to the credibility of the awards.
Behind the principle that that "No group - whoever that group may be - should be able to absolutely prevent nominees from having the chance to be considered for the Hugo Award" lies an assumption with which I agree: that the Hugo ballots should resemble a recognisable reflection of the state of the genre in each year. This year's ballots for most of the fiction categories and several others, notably Best Related Work, are not a recognisable reflection of the state of the genre, and fail that test. The slates have filled them with finalists which are based only in one corner of the field (and also, for the most part, aren't very good). A wider selection of finalists - including the slate candidates, if they have the support - must be a good thing.
To be honest, I'm not sure that recent slate-free Best Short Story ballots, with fewer than five finalists in three of the last four years, have really managed that reflection of the state of the genre either. The nominations for Best Short Story were so smeared out among a large number of candidates that not enough of them crossed the 5% threshold to produce a full ballot. My instinctive reaction at the time was that the 5% threshold itself should be abolished, but I can see that this is not satisfactory: it leaves you with nominees on the final ballot who got 20% of the available slots (one of the final five) with only 4.5% of the vote, or less. It's not quite in the territory of the Australian Sports Party, who managed to elect one of the six senators from Western Australia in 2014 with 0.23% of first preference votes, but it's not brilliant either. E Pluribus Hugo will produce a more consensual ballot, comprising finalists which have broader support than simply being the fifth-least obscure of a number of obscure works.
The downside of E Pluribus Hugo is that it does reduce the value of the votes of those who nominate more than one work. At present, if you nominate five candidates, each of them gets a full vote. Under the proposed system, your vote gets full value only when it is cumulated onto the most popular of those candidates. It means that the choice of the candidates who get through to the final ballot will be largely determined by those who nominate a single candidate, and give it enough votes to last to the end of the counting of nominations. I'm a bit sad about this. In previous years, one of the values of the Hugo process for me was that I was able to take recommendations from people better-read than me, and I did feel that it was fair enough that someone with enough knowledge to cast five nominating votes effectively had five times the fire-power of someone who cast only one. But the slates broke that system this year, and I guess we must discard it.
In general, I'm a fan of and evangelist for the Single Transferable Vote in multi-member constituencies, for all public elections where more than one person is to be elected. It seems to me to get the best balance possible between voter impact on the result, proportionality of outcome between different political groups, and encouraging accountability from elected representatives. However, I can see that two of those three considerations don't matter much in this case.
Until recently, there were no slates for Hugo nominations, and the designers of E Pluribus Hugo specifically don't want to encourage their formation for future ballots. So it is not advantageous to have a system like STV that rewards political party-like behaviour, if that behaviour is something you want to discourage. Some of the slate-mongers actually do seem to desire a situation where politically oriented slates duke it out through the awards process, to the extent that they claim that this has already been happening for years. I don't want this to happen, and I think that E Pluribus Hugo is better than the present system for discouraging it.
As for accountability, while I wish that some of this year's finalists were behaving in a more dignified or indeed decent way, the fact is that this is a yearly vote for an award, not a choice of public representatives who will carry out the work of one or other branch of government. So there is no point in designing a system that rewards responsiveness from the nominees to the voters.
There's a process issue as well which makes STV unsuitable for this purpose. By definition, it gives your first preference candidate 100% of your vote, unless and until it is transferred. The ranking of candidates is hugely important. For the final stage of the Hugos, that matters; for the nominations stage, which is about assembling a decent ballot rather than electing representatives, it isn't. One would also have to do imaginative things with quotas and the values of surplus votes; while I understand that similar tweaks have been implemented elsewhere without complaint, I think that E Pluribus Hugo scores on transparency and ease of implementation.
What does still matter is maximising voter impact on the result. As I said above, I regret that the voters whose impact is reduced by E Pluribus Hugo are those who nominate several candidates, and those whose impact is increased are those who nominate only one. But I think that the system does mitigate that a bit through the serial cumulation process. Those who have run simulations report that the difference to actual ballots is not very large. What I fear is that the quirky candidate that got a lot of people's fourth or fifth nominating votes will lose out to the popular candidate which the fourth or fifth largest number of voters have heard of. But in fairness, the quirky candidate was unlikely to win the award anyway, and the Hugos are a popularity contest rather than a quirkiness competition.
So with a slightly heavy heart - I regret that small-minded slate-mongers have killed off a large part of the wisdom-of-crowds aspect of the Hugo nominations process - but with thanks to those who have worked on it, I hope that E Pluribus Hugo is adopted by the WSFA business meetings at Sasquan and MidAmeriCon II.
Brian Z: The question of whether the electorate should be expanded is orthogonal to the question of whether EPH should be adopted. Both strike me as good ideas; and there is nothing stopping us from doing both.
In general, a lot of what you're talking about in your long post is stuff that EPH won't help. But EPH won't hurt it either, and none of it is a reason not to adopt it. Yes, there are human factors outside the math. That isn't a reason not to have better math.
You talk about campaigns to target certain publishing houses and authors. I'm having trouble imagining how that would even work. Please be more specific. And show how such campaigns would be worse, or have a stronger effect, under EPH than under the current system -- if they have the same effect, or a lesser effect, then they're not an argument against adopting EPH.
@25:
I regret that the voters whose impact is reduced by E Pluribus Hugo are those who nominate several candidates, and those whose impact is increased are those who nominate only one.
This is not an accurate summary of how EPH would work (though I recognize that it's offered in good faith). In order to have reduced influence, a voter must:
1. Vote for several works...
2. ...of which several are among the top few vote-getters...
3. ...and which have significantly above-average overlap in their support.
In other words, the only people who lose out are slate voters (though some may be inadvertently so).
Imagine a voter who does 1 but not 2. Their extra nominees are soon eliminated and their ballot counts full strength for their strongest choice. Now imagine 1&2 but not 3. Their most marginal choice has half a point, or maybe up to .8 point, less; but it still has the same nominations, and faces other works with fewer nominations.
... In terms of your general pro-STV stance, I agree that it's a good system, but I could debate some of your points and offer alternatives that I think are even better. And I love to talk about voting systems. But that's a bit off topic.
Nominating does always involve a decision about where to draw the line -- at what point on the ordered (by my preferences) list of works in a category (that I have an opinion about) do I stop listing them on my nominating ballot? Or, if I have too many works to fit on the ballot, sometimes that decision is made for me by the limit on number of nomination.
The current, simple plurality system just counts up all the nominations on all the ballots for each work in each category, and the top 5 go on the final ballot (with some tie-resolution rules that can expand the ballot, and a 5% check that can shrink the ballot but only somewhat).
If you know of only one work that you consider Hugo-worthy, it's simple -- you just nominate that work.
If you have a bunch of works that you consider broadly equally worthy, it's also simple -- you nominate all of them.
However, if you know of one outstanding work but also four other works that you think belong on the ballot (even though you have a strong preference for the outstanding work), what should you do under the current system?
If you list just your top favorite, then if it fails to make the ballot, you've passed up the opportunity to suggest the other four works that you think belong on the ballot. In fact, even if your favorite *does* make the ballot, you have passed up the opportunity to suggest the others.
And, if you do list your 4 lesser favorites, it's possible that your listing them may provide the nomination that knocks your top choice off the final ballot.
And you can't know in advance which problem is more likely; you can at best guess (and the conditions for the second problem actually happening are extremely precise; most of the time it won't).
This problem or some close analog exists with any nominating scheme that uses simple lists rather than taking rankings of each work. Since the list doesn't encode any preference you have among the nominees, the tallying system can't take account of it. Your only choice is "nominate" or "don't nominate".
So, the new EPH system being proposed does have this problem. But we've been living with this problem since the beginning; it's not something new.
Under EPH, the greatest difference choosing how many to nominate can make in the point count for a work at any stage is 4/5 of a point. That difference is smaller than the full 1 vote difference the choice can make in the current nominating system. In both cases, it only makes a difference in terribly close cases. And under EPH, the final decision to eliminate is made by the same raw count used currently to rank the results (the point system is only used to pick the candidates for elimination, the actual elimination decision is made on raw count; this balancing of two measures of popularity is the source of the system's robustness, methods of gaming one measure tend not to work on the other measure).
Claims that the ballot will be more dominated by "bullet votes" under EPH than under the current system are definitely incorrect.
@27,
I invite you over to my LJ for discussion of STV, which I agree is inappropriate here!
However, I think we both agree that the group of voters whose influence is reduced by EPH are is a subset of those who nominate multiple candidates; and I agree further with your point that they are a subset of those who nominate multiple popular candidates. In past years I have been happy to trust such voters to have a larger voice than me in determining the appearance of the final ballot. Obviously, this year that trust was misplaced.
And you're right to say that those voters who support works with significantly above-average overlap in their support are those whose influence is most reduced by EPH; but I disagree that those who fall into your 1&2 category have nothing to worry about. It's a price they'll/we'll have to pay, and we should be clear about that.
I do think it's important to note, also, that the voters whose power is increased are those who make single nominations for the most popular works. My experience of STV (and I'm sure yours as well, if you have dealt with it in the wild) is that an early strong foundation of first preferences is much more desirable than a lot of second places. EPH is a bit different, but the point still remains that a work nominated by 100 people, each of whom has also made another four nominations, is less likely to make it though to the final stage than one which is nominated by 100 people as their sole nomination, because it is more likely to be overtaken during the elimination phase by a work with 25, or 30, or 40 sole nominations. Again, I think it's a price we have to pay, but we should be clear that it is a price.
Wouldn't a work with 25 solo nominations also have to have at least 75 other nominations to eliminate a work with 100 mixed nominations in the elimination phase? Or do I not understand the elimination phase?
Influxis @30,
My reading is that, sure, a work with 25 full-value nominations will lose out to a work with 100 quarter-nominations in the event of a tie-break. But a work with 100 quarter-nominations will lose to a work with 26 full-value nominations.
I read eliminations as being done on nominations first, if these tie then that is resolved with points. Is that not what it says? By full value nominations you seem to mean points as per the EPH proposal.
Influxus,
No, that's not what it says. It's precisely the other way round - points are calculated first, then ties are split on the number of nominations.
See section 3.a.1 (3)
(3) Elimination Phase: Nominees chosen in the Selection Phase shall be compared, and the nominee with the fewest number of nominations shall be eliminated and removed from all ballots for the Calculation Phase of all subsequent rounds. (See 3.A.3 for ties.)
Nicholas Whyte@34
Points are used to determine which nominees are compared for possible elimination. Elimination is done based on nominations (that is, the number of ballots the candidates for elimination appear on). Ties in the elimination phase are broken based on points.
It's not one first then the other, exactly.
Two or more candidates for elimination are chosen based on points in the "selection phase".
Then, the one of those with the fewest nominations is eliminated in the "elimination phase".
(Complexities for tie-breaking omitted).
This is the core of the system; using two measures of popularity make it harder to game, since optimizing one often hurts the other.
influxus #32: For each comparison, the one with fewer nominations loses, but both of them were chosen for comparision because they had the two lowest point totals in the pool.
Nicholas Whyte #31: Thus, the work with 26.0 points from 26 nominations might well be compared against one with 25.0 points from 100 nominations, but it will lose.
At that point, the 26 bullet ballots are "fallen soldiers" -- they did their best, and lost. But not only is the more broadly-supported work still in play, so are its ballot-buddies.
Addendum to me #38: Note also that the 25/100 work would itself be beaten by one with 25/125, but the other works on its ballots (likely including its opponent of the moment) would benefit thereby.
David #38: yes, sorry I should have been a bit clearer about there being two phases. #37's explanation was much more elucidating than mine.
Thanks, all. I see that I had missed the important point that the elimination phase in principle between the two works with fewest points, but is itself decided by the number of nominations. This is fairly clearly explained in the FAQ, but less clearly in the body of the proposal which is what I was concentrating on.
Kyra over at File770 has outlined a case where my scenario could still happen, but I have to admit that it's pretty improbable.
So my major objection to the EPH proposal is substantially mitigated by new information, and I can endorse it more whole-heartedly than before.
@29: So you agree that steps 1 and 2 are necessary for getting extra influence, but disagree that step 3 is necessary.
Let's make a statistical model of what's going on. I'll start as simple as possible, and add things until some voters have extra influence.
Say each voter reads everything and votes for a random subset. In this case, everything is noise with no actual signal, but chances are very high that EPH gets the same answer as traditional voting (as number of voters goes to infinity, the probability goes to 1).
Say each work has an inherent quality, and each voter sees that quality plus some random "error" term (or "personal taste"; it's the same thing mathematically), and then each voter votes for what they saw as the best 5 works. Again, the chances that the two systems get the same answer is asymptotically 1, and EPH is in no sense "overweighting" anyone's vote.
Say each voter reads a random selection of works, reading each work with a constant probability p. Still no "overweighting".
Say p is weighted by some monotonic function of work quality. Still no. OK, weight p by "popularity", which correlates imperfectly with quality. Still, all is good. OK, weight p by the product of popularity and a voter's "well-read-ness"; that is, well-read voters read more works. And let's say that the variance of the "voter error" (that is, the imprecision of a voter's taste) is higher for less well-read voters.
Finally, a model with enough structure to potentially trip up EPH. So, well-read voters are more likely to have read the best candidates, and also more likely to realize that they're the best; so, as the non-viable candidates are eliminated, the well-read voters are likely to be spreading their points over several works. But then, the highly-popular medium-quality stuff is also getting clusters of votes from the poorly-read voters. The end result depends on whether popularity or quality is more diffuse, on how they correlate, and on the variance of the error terms. It's almost as if there's two overlapping slates, based on quality and popularity. EPH might down-weight the quality one relative to the old system; or it might up-weight it. In general, the chances that the best-quality work is among the finalists are higher and more robust under EPH.
And still, we haven't found a case where EPH is downweighting one set of votes for something that can't be interpreted as slate voting; that is, for something where 1, 2, and 3 don't all hold.
There is, in fact, a possibility of that happening. Say you have 2 slate works "Alpha" and "Beta" getting 51 and 49 points with 100 and 98 nominations, and you have a non-slate work "Gimmel" in the same point ballpark with 60 nominations. If Gimmel has 50 points, it will face off against Beta, and be eliminated. If Gimmel has 52 points, Alpha and Beta will face off first, with Beta being eliminated and Alpha's points going up to 100. So Gimmel survives.
But even then, 1 and 2 still have to hold for Gimmel. Also, there still has to be a significant amount of slate voting, and a large dose of bad luck, for Gimmel to be eliminated in a case where it could have been a finalist. So I still say that in practice, the only works which will be negatively affected by EPH are those where 1, 2, and 3 hold; which is basically saying, only slate works are harmed.
Whoops, cross-post. I see you basically have seen my point already. Kyra's example is essentially a special case of my Alpha, Beta, Gimmel example.
One very useful term I learned from Bruce Schneier is “threat model”. A high-school locker, a bank vault, and a cell phone screensaver are designed with different threat models in mind.
So in considering the relative merits of the current Hugo nominations algorithm, EPH, and any other system someone might come up with, a useful focusing question is “against what threat model?”
EPH was designed to resist the threat model of (a) a minority bloc of between 10–20% of voters (b) who have limited information about who the rest of the electorate prefers (c) voting with near-unanimity for a particular slate of candidates (d) in the hope of that slate sweeping the category entirely.
There may be other threats to the integrity of the Hugos. For example, some people think that the small number people who actually vote in Hugo elections have idiosyncratic tastes that and do not represent “fandom” broadly defined. EPH doesn’t solve this problem (if indeed it is a problem) but doesn’t exacerbate it, either.
I think Brian Z is expressing concern that if EPH passes, the Hugos will become more vulnerable to a new threat model, but I am having trouble understanding what that model is.
One more question, and this might be FAQ-worthy for the benefit of those who program little or (more likely) not at all:
How big is an implementation of EPH - in lines of code, size of spreadsheet file, or something like that? How long does it take to execute once data's put in?
Suggested addition to the FAQ:
Q: Are there other benefits to the integrity of the Hugo from EPH?
A: Yes, EPH also works against slates on a social level. In the past, slatemakers have advertised for their slates by suggesting that the Hugo voters packet which includes a number of slated works compensates for the cost of the Wordcon membership. This is only an advantage for people who don't have the slated works in question and thus slightly against the spirit of the nomination process which asks nominators to name works they have read. By removing the undue influence of slates EPH also weakens the argument above and thus discourages people to join the voting process for the Hugo voters package.
Seth@44: I've been following the comments over at File 770, where Brian Z has a history. I'm pretty sure that he's not actually concerned about the Hugos being more vulnerable to some other threat model; rather, he likes the idea of Theodore Beale getting to own the awards, and is doing his best to resist anything which might stop that, by sowing FUD.
@David: Please note that I said “I think Brian Z is expressing concern...” Whether the concern is heartfelt is another question entirely.
I think the E Pluribus Hugo method will be excellent for the Hugo nominations. Even when there isn't slate voting, it will be a big improvement over the current method.
I am not concerned about my influence being reduced in categories where I nominate a full list. To the contrary, EPH enhances my influence in the way that matters most to me: The ability to maybe get some of my recommendations onto the final ballot.
I have never expected or wanted to get all of my recommendations onto the final ballot. If somehow the final ballot ended up with exactly the same candidates I nominated, I would not consider that to be a victory condition. More like a bizarre coincidence. It would have me nervously checking if a can of UBIK has appeared in my medicine cabinet.
I get a lot of value from the Hugo nominations as must-read recommendations. It's nice if some of my recommendations make it onto the final ballot, but I also want to see recommendations from other fans, as many and as different as possible. I'm a science fiction fan. I'm always interested in new and different stuff. I don't want my recommendations to sweep a category, even accidentally, because it would be boring. EPH will do a better job of finding me new and interesting stuff than any of the alternatives.
Of course, the Hugos are not just a recommendation engine. They are awards with all the attendant foofaraw and egoboo. "It is an honor just to be nominated." My concern as a fan and voter is first and foremost the integrity of the Hugos. The honor of being nominated is greater when they are seen as recognition based on a robust survey, not as something to be captured by gaming the system. When I make multiple nominations in a category, my goal is not to maximize the number that get onto the final ballot, but rather to maximize the prestige for those that do make it. For that purpose, I think that EPH will be better than the current method and may be the best possible method.
One can also reframe the slightly lessened influence of people who nominate more than one work as a feature: Under EPH there is an incentive to only nominate work deemed Hugo worthy, whereas under the current system there is an incentive (however irrelevant) to fill up the list with stuff your friends like or which got good buzz even if you are not 100% behind the nomination.
BrianZ@24:
File 770 roundups attract Star-bellied Sneetches as well as those whose bellies have no stars, so I hope you will understand my reasons for talking there.
I do understand your reasons, and as I mentioned, I hope EPH is discussed everywhere -- I want there to be a buzz about the system. But I just spent the entire morning trying to keep up with the pace of File770 comments. It's fun, but it's just not possible for me to do much else during the day if I were going to try to stay on top of that. So, odds are, I will miss a lot of questions directed at me over there.
So my vote is for radically expanding the community of readers – however geographically dispersed and sub-culturally fragmented they may be thanks to the internet – who take this yeoman's work seriously.
Absolutely, and you'll get no disagreement from me on this. But I'm not sure how EPH would discourage this. From the voters' end, EPH doesn't change the nomination process at all, so any initiative to increase the community of readers will still be effective with EPH.
We “win in a fair fight,” in this fan award, by honoring an author cherished in our community.
Well, that's an odd use of my quote, since what I was referring to was the fact that unless we have a good, broad sampling of works to compare to, we can't know if an author is cherished or not. I once won an archery competition because I was the only archer in my class to show up. Am I really a championship archer? (For the curious, the answer is a resounding "No!" :) ) If an author was put on the final ballot due to a slate, he or she might actually be one of the best we have -- but we can never know because we don't have the other authors to compare him to. So, I'm not sure how what you are saying here is related.
When I say we aren’t really facing the truth, I mean that at its heart this is (as noted on the other thread) a squishy human problem. Not a math problem.
I've seen this as the crux of your argument against EPH elsewhere, but it still doesn't quite explain your opposition. I think what you're saying is that EPH would remove the need for people to think about ways to address the cultural issue that is dividing fandom, correct? I'm not sure that's true, because the two issues aren't mutually exclusive. I don't want to draw too many conclusions since I may still be misunderstanding your opposition. Of course, if EPH does remove the need to deal with these issues, then I'd consider it a success.
other structural problems that hinder us, like the chaotic state of short fiction markets and the explosion of the 18-book series where one book or even a trilogy would do. Math won’t fix those either.
Well, no. But that's not the problem we're trying to solve. And there's nothing in EPH that will prevent you from solving those problems if you like.
I tried to discuss the human dimensions, and I recommend more consideration of how your proposal’s technical strengths might hold up in a gale of fannish wrath.
Hmm. Well, we have in some respects, though it's far from clear what you mean exactly by "fannish wrath". But EPH is designed to be as strategy-resistant as possible -- whatever that strategy may be. Will that be all that's needed? I don't know, but it will definitely help. I would think you would support EPH, if for no other reason than it will give us the breathing room to have that dialogue you want to have. And if, as a result of EPH, people no longer feel that dialogue is necessary, is that a bad thing? Ultimately, if people are nominating the works they think are Hugo-worthy, then everything is as it should be, at least in my mind.
I would like to point out that you have expressed some opposition to EPH, but haven't really proposed anything else. That may be because -- and again, I'm just trying to fairly interpret what you're saying, so I may very well be wrong -- you think EPH will dissuade fandom from coming up with something truly new and amazing that ends the culture wars by expanding everyone's reading tastes. That may be a worthy goal, but I disagree that EPH is a factor in whether or not that will ever come to pass. EPH was designed to do one thing, and one thing only: No group — whoever that group may be — should be able to absolutely prevent nominees from having the chance to be considered for the Hugo Award. Everyone working on proposals were specifically asked by the chair of the business meeting to separate out parts of our proposals into discrete elements for easier voting. There's nothing stopping you from proposing anything you want at the business meeting. We've designed EPH to be compatible with essentially any ideas that people can come up with because we are focused on this one issue. Is this the only issue in fandom? Of course not. But it's the one we've chosen to deal with, and the only one this group will deal with. We'll let other groups take a shot at the other issues. Some of us may even join them. But right now, I don't have a squishy human answer to the squishy human problem you describe. I do have a mathematical answer to a very specific mathematical problem. That answer is EPH, and I think it answers that question very well.
Thanks for your thoughts, but as I mentioned in my last message to you, if your proposal is to do nothing (for whatever reasons), then we may simply be at an impasse and will have to agree to disagree. For me, and many others here, doing nothing is simply not an option.
Kilo
All this concentration on Points & Distribution and Balloting makes my head hurt. It may well be useful, but I hope it's okay for me to skip it and approach from another direction.
By 1960, after I'd been hanging around Fans for two years, I figured out that the Hugos were a Popularity Award -- based on popularity among a group of people (Fans who read a lot of science-fiction and participated in WorldCons) who mostly had a serious understanding and appreciation of the genre. I respected the Hugos accordingly, and took seriously my participation in voting for them. (I note that up until about that time it was quite possible for a Fan to read [practically] _all_ of the science-fiction published in any given year, and that many of us had done that for a decade or more.)
Yes, there certainly were times when I wondered "Why on earth did X win?', and some when, a few years later, I wondered "Why in the world did I vote for Y when Z had been nominated?, but stuff like that happens.
I'm not sure how many regular science-fiction Fandom conventions there were in the early 60's in the U.S. -- WorldCon, Westercon, and MidWestCon, and possibly some third. Now... you might well be able to find at least two for every weekend of the year. So ConFandom has strongly influenced even WorldCons, only a small percentage of its members now vote for the Hugos... and the "puppies" attempt to increase that number does not seem likely to improve the expertise behind the Hugo voting.
Sadly, I've not been favorably impressed by the quality of the Hugo selections for the past few years, nor (even as much so) by the material in the puppies' Slate, though I admit that some of the latter reaches to the level I'd consider "Mediocre". Whether there are some ways -- preferably not too difficult for the Administrators to handle -- to improve the situation in ways that aren't likely to be Gamed by people who want to cop all the Awards for themselves isn't clear to me. I wish the people who are trying to straighten all this out the best of luck.
Bruce@45:
How big is an implementation of EPH - in lines of code, size of spreadsheet file, or something like that? How long does it take to execute once data's put in?
I don't know that there's much need to put this in the FAQ, since I'm not qualified to write a web-based database app. I'm more than happy to work with the folks that end up writing the final version, though.
The code that I've written has 404 lines, including comments (and I comment heavily). The simulation I ran for you took 23 seconds to run, though I haven't even attempted to optimize the code.
Kilo
Kilo@53: Thanks!
This is one of those things where those of us who aren't well-versed in a craft may have tremendous difficulty getting any sense of what it takes to do a task. Itch now scratched. :)
Kilo,
Thanks for this conversation. It was the earlier insults from some of your esteemed friends and colleagues that had led me to feel there may be a desire around here to deflect or minimize concerns about the proposal, so I can assure you that your reply is very sincerely appreciated.
Also, last time I was over here, I did say that I would consider supporting, or at least voting to ratify if passed, an amendment that did nothing but prevent a group of lockstep voters from sweeping the ballot, if I was convinced it did not have adverse effects.
We seem to all agree (to quote from Quinn at 27 above) that under EPH:
In order to have reduced influence, a voter must:
1. Vote for several works...
2. ...of which several are among the top few vote-getters...
3. ...and which have significantly above-average overlap in their support.
But I disagree with his assessment that "in other words, the only people who lose out are slate voters (though some may be inadvertently so)," and the implication that that should be OK with everyone, for the following reasons.
1. There are no "slate voters" – I feel calling WSFS members that is something of a scare tactic. There are "lists of eligible works posted by a number of people on the internet." There are "fans who cast ballots." Brad and Larry asked their readers not to vote for Brad’s recommendations unless they had read and agreed with them. In my opinion, a certain number of lockstep-voting fans among Vox Day’s blog readership probably did "screw up" (as I would define it in my own view) by following his suggestion to vote his list "as is." They had their reasons for doing it, and I can see what those reasons were, but I myself don't agree with them. The reason I am still here wondering whether talking with them about what happened and how we can all avoid it in the future might be a possible strategy to consider is that if we insult them, ridicule them and "no-award" them, and then on top of that dare them to go ahead and try even harder to do some damage next time, it would be only too human to want to take us up on our challenge. What I’ve seen you do here (correct me if I’ve misunderstood) is consider how your model holds up against the threat of some like-minded people voting together strategically in support of a slate of five things, but that is not even a very good characterization of the mess in 2015, much less a likely future scenario if a) this thing is ratified and b) disgruntled voters and potential voters are even more pissed off than they already are and become hellbent on gaming the new system.
2. We are attracting thousands of fans to start participating in the selection process for this fan-run award. That’s a beautiful and joyous thing. But we are not doing a very good job so far of welcoming some of them to the fold, and reducing and circumscribing their voting power because we don’t trust them "feels" like "icing on the cake."
3. When you look at the criteria above describing who "loses influence," it isn’t hard at all to see yourself looking back. Even putting aside the disgruntled voters who are not "behaving correctly" for a moment, of course this would have an impact even on what you consider a "rational, right-thinking" voter. Suppose EPH had been ratified in time for 2014. Oooh! Look! There’s Wheel of Time on my shelf right there! MUST NOMINATE. And I thought Ancillary Justice was every bit as great as they said it was in Strange Horizons, io9, Tor.com, NPR and the Guardian! There’s the new Scalzi! I love everything about Scalzi! Mmm… bacon! Must vote for Scalzi! But look over here, I could hardly forget Self-Reference Engine by Toh Enjoe, one of the most mind-blowing SF novels I’ve read in years! I want to support that most of all, and it got a little buzz and a couple reviews but I don't know how many people actually read it. Hmm… maybe I should bullet vote for Toh EnJoe? But… Robert Jordan! Ancillary Justice! Wait, if I choose two or three, how much difference does that… I need a drink!
Maybe that’s enough FUD for now. Again, thank you for your hard work, and please accept my concerns in the spirit of vigorous discussion before a constitutional amendment.
"if I was convinced it did not have adverse effects"
Every change in the voting system will have effects, and by your standards they all could, might perhaps have unforseen and unforseeable consequences. So you'll vote against them all. Duly noted.
1) It is a good characterisation of 2015. There is no way to game the system anyone could think of so far. Unless you have one, please refrain from dire predictions.
2) EPH does quite the opposite: It ensures that their votes will be counted and not rendered meaningless by a slate of 10% of the electorate. There is no way to stop people lying about that.
3) You still pretend not to understand the system. Or you are arguing from ignorance. If so, ask questions _about the system_ until you get it. Don't make up nonsense.
In your hypthetical scenario, the rational thing to do is nominate up to five worthy works. That hasn't changed from the last time to offered up this particular argument.
Brian Z @55, if we insult them, ridicule them and "no-award" them, and then on top of that dare them to go ahead and try even harder to do some damage next time, it would be only too human to want to take us up on our challenge.
If the Puppies (sad or rabid) had nominated anything that I felt was better than mediocre, I'd be happy to vote for their nominees. As it is, I'm reading everything in the Hugo packet, and for the first time I can recall, it's a joyless chore.
This makes me very, very sad.
I'm not completely through reading the Hugo packet. I profoundly hope there's something in there that I like (other than The Goblin Emperor, which I nominated). But at this point, I'm honestly not holding my breath. The ONLY category in which I've found several actually Hugo-worthy entrants so far is the graphic work category. I've not finished reading the novels yet, so I can make no definitive statement on that category. But I stopped reading The Dark Between The Stars halfway through, even though I was running an online book discussion on the novel, because every single person in the discussion said that the book was boring, the characters were cardboard, and could we please read something else. I've run dozens and dozens of book discussions over the years. This has never happened before.
(I've not seen most of the movie or TV show nominees, so I don't intend to vote in those categories.)
tl/dr: I'm not "no awarding" them because they're puppy slate choices; I'm "no awarding" them because I genuinely do not believe any of them that I've read so far deserve a Hugo. If they wanted my vote, they should have nominated better stories.
Brian Z.@55:
Brad and Larry asked their readers not to vote for Brad’s recommendations unless they had read and agreed with them.Did they? Where? I just re-read the post with the Sad Puppies 3 slate, and it says no such thing. It does say "If you agree with our slate below" but it goes on with a nudge-nudge-wink-wink "and we suspect you might".
(Beale, of course, went right ahead and said to nominate his list exactly as it was.)
I honestly can't make head or tail of Brian Z's latest. Insofar as it seems to mean anything, it sounds to me exactly like a supervillain's rant as the hero wins.
"And now, who has won? Fool. Did you really think that by all this you could save the human traitor? After you're dead, what's to stop me from killing him as well as the rest of his kin? You've given your own life, and you have not saved anybody. In that knowledge, despair! And DIE!"
57
Knitting a baby blanket in garter stitch is more interesting than reading the puppies' choices. (Certainly it's a more productive use of my time.)
I found TBP, with all its flaws, to be more interesting than TDBTS.
For a certain type of detail minded individual (a type common in fandom) I think the data from post 23 is a very convincing argument for the proposal and makes the English description easier to follow. Would it be possible to add it to either the FAQ or as some kind of appendix?
Jameson Quinn #59: Worse -- he's so cocky that right at the end he admits he's just spreading FUD. Definitely a concern troll. Also, several puppy partisans have already been by doing the same thing, from Brad Torgersen himself back at the beginning, through various derailing attempts right to the end.
Basically, they're spewing bullshit and trying to make us refute it at the usual 10:1 rates, which is meant to distract us from our own purposes.
David Harmon and Jameson Quinn, I'm pretty sure neither of you clicked through to see the last entry at the bottom of this post. It might be worth a quick look.
Brian Z @63:
If you want to drive traffic to your blog, you already have it linked to your name. But you could also just say what you want to say here.
(Personally, I have my doubts whether one can both filk about this situation and maintain the kind of respect for all parties that you so fervently insist on. The two spirits are antithetical to one another. I can certainly see elements in your works that do not further the constructive tone you're very keen to promote in lieu of changing any voting rules.)
abi, I thought it would be more polite to reply to those two comments indirectly. As for filk, I'm doing my best.
Brian @65:
The best way to refute accusations of being a concern troll is not to link to your rather strained rewrite of Le Guin; it's to stop acting like a concern troll.
Discuss details. Follow points through instead of introducing new ones as soon as your old ones are addressed. Listen to others and show evidence of taking their points on board.
And if you're going to do the "why is no one talking nicely to the Puppies?" thing, well, go over there and build those bridges you keep telling everyone they should drop everything to work on instead.
Your reputation is a bed that you have made; now you're lying in it. Don't like that? Do better.
Thank you, Abi. Yes, I'm looking for specifics. "The next time, the puppies will be different bwahaha" isn't scaring me. And filk is nice as far as it goes but doesn't make up for a lack of infodumps.
Keith@23: I think this example could be turned into a series of PowerPoint graphics to illustrate how the system reduces the power of slates. Represent each work as a bar graph with two bars in different colors - one to show the current points of the work, and one to show the total nominations (with both bars scaled to the final maximum values). Arrange these in (say) 4 rows of 8. Then at each step, highlight the two works being compared, do a closeup overlay of those two works showing the points and nominations at the top of the bars, and then return to the overall graphic with the losing work eliminated, and the points redistributed (losing works could be grayed out, have a red X over them, or just blanked out, but surviving works should be in the same relative position so people can track them from slide to slide). You could probably skip a bunch of the preliminary rounds - show the first couple of steps to give people a sense of how the system works, and then skip to the point where the first slate work is selected for elimination, around round 16. I think that could be done in a 20-minute presentation of how the system works.
It makes me really happy to see others finding value in the data Kilo presented at my request. I was hoping. :)
abi, let's review.
In that previous EPH thread, you asked me to leave when I raised what I thought were important concerns about whether changing the rules right now was really such a good idea, and when I agreed with you and left, in your own words, I was "kicked on the way out" by your own "voting systems expert" who has been kickstarted to Sasquan and a few others.
Recently, you made the point of going over to File 770 to specifically urge me to come back to Making Light and raise my concerns here, not elsewhere.
Now that I have done so, your star PhD student and his friends have again dismissed me as a quote, troll, unquote, and just now, you have very forcefully seconded their motion.
That's enough. Thanks for your invitation, and goodbye.
Kilo, thanks again, very much, for your (apparently less disingenuous) engagement with the points I raised, and I look forward to talking with you again somehwhere about an issue that is clearly very important to the two of us, and a lot of other people. And good luck.
Staging notice: While the room where the Business Meeting is behind held has screens and I expect to be able to show PowerPoint presentations (you must work with the Parliamentarian in advance; I won't have the meeting held up with people mucking with the tech), note that the screens are going to be along the left wall of the room, not directly behind the head table. This is a technical restriction of the way the convention center rooms are built.
Okay, stoopid question time.
Suppose I REALLY like "Story of the Century" and decide to put it on my ballot five times - using all five of my Short Story slots for it in hopes of giving it a boost.
Do the duplicated slots on my ballot get tossed away (turning it into a bullet vote), or does that tactic actually have an effect under either system?
Links and quotes Brian, or it didn't happen. I'm not going to bother to track these things down and bring the texts into the conversation to parse where you're accurate and where you're not; you're wasting enough of my time already.
I will point out that I said you could come back if you behaved yourself. And you still can! All you have to do (apart from links and quotes) is what I said above: Discuss details. Follow points through instead of introducing new ones as soon as your old ones are addressed. Listen to others and show evidence of taking their points on board.
I'm betting that you're flouncing because that's precisely what you don't want to do; it doesn't serve your aims.
And if when you start telling any version of this in on File770, I'm sure there will be people there saying exactly the same thing: links and quotes, or it didn't happen.
Your record in these things is not good enough for unsupported cross-site narratives.
Rev. Bob @72, not a stupid question; I remember it coming up in one of the many, many, many discussion threads. The rules are (as I understand them) that duplicate entries are thrown away, so you end up with one nomination for "Story of the Century" and the rest of the ballot in that category is blank. This is an existing rule that E Plurbus Hugo would not change.
(I welcome correction if I have any of this wrong.)
72
Someone did that in 1984, and it was counted as one nomination.
Rev. Bob addendum: I should add that, under E Plurbus Hugo, it doesn't matter if you nominate one work once, or one work five times. If you nominate one work once, it gets 1 point. If you were allowed to nominate one work five times, it would get (1/5) point each times five... or 1 point.
(I suppose one could add complexity by hypothetically giving "Story of the Century" 3 noms and "Almost As Good A Story" two noms... but, as I said above, this is a moot point; the practical result under current rules is that you've given each one nom, and 1/2 point each, until one is eliminated, in which case the other gets your whole point. It would take a rule change (in addition to E Plurbus Hugo) to change this.)
(Again, corrections welcome.)
abi, nobody is flouncing, I'm just done with you. There have been a handful of voting systems threads, which everybody can read, and they are all linked here. The File 770 thread is from yesterday. You invited me here, and I obliged, and now I guess I see what you had in mind. Would you like to disemvowel me on the way out? Actually, if you did, I'd appreciate the gesture.
Brian,
(a) comment 77 is a lapidary flounce*, and
(b) every single word in comment 73 remains true and unrefuted, including the fact that you're welcome here if you behave.
-----
* including the fact that it's the second announcement of your departure
Bruce@45, Keith@53:
It's not a hard algorithm to implement. For unrelated reasons, I wanted to learn a new programming language (Rust) a few weeks ago, and decided to code EPH as an exercise. It took about 180 lines of not-very-efficient, not-very-experienced, certainly-not-very-well-commented code, and most of an afternoon. I would give about 40% odds that it's correct, since I didn't test it, but the key concept isn't hard, and the implementation is fairly efficient.
If I were doing this for a real deployed system, I bet that I would spend far more code in normalizing inputs, so as to ensure that e.g. a nomination for _Ancillary Sword_ by Anne Leckie and a nomination for _The Ancillary Sword_ by Ann Leckie have a reasonable chance counting as the same book.
As I said in @67, I'd also welcome Brian's participation if he would address details and defend his points, rather than running a Gish Gallop. As he does so, he can call me snide names all he wants; he seems to at least keep the aspersions reasonably on-point, as I hope I have in stating my opinions of him.
I'm just done with you
Overdone, more like.
Lighthill@80:
Yes, the basic algorithm is very simple and straight-forward, so I'd bet that your code is pretty good. If you want to test it, just send me an email (Google my handle here and you'll find my work email easily enough), and I'd be glad to send you the sample ballots I used for Bruce's case. It's not a problem at all, just let me know.
Thanks again,
Kilo
I'm all for E Pluribus Hugo and tried to follow the development threads but got snowed under by sheer volume. I have a question that may have been answered in those.
How specific is the available data from the 2014 nomination ballots? Do we have enough information to find out which works would have been nominated under E Pluribus Hugo? If it's not that specific, do you have enough for a statistically distributed estimate?
Basically, I've been very curious about what the shadow ballot would have looked like. I've had trouble even finding who the next non-puppy runner-ups would have been in the current system. If this info is available, I hope someone will point me toward it. Thanks!
Gement @84, as I understand it (and I may be wrong), information as to the runners-up in the 2014 Hugo nominations will not be released until after the Hugo ceremony.
The data that people have been working with to test this proposal is from (if memory serves) a hard-copy printout of a old nomination set from 1984, with and without a hypothetical "slate" added. Even anonymized raw data of Hugo nominations is apparently very difficult to come by.
I have been reflecting on the details of the system, and playing with the 1984 figures which Paul kindly shared with me. My conclusions are not yet firm, but I want to test them out here in case anyone else has already tried this point and in hope that someone can run some simulations to test it.
My question is this: What is the threshold, under EPH, at which slates start to win places on the final ballot?
Looking at Keith's worked example, 101 slate voters rather than his 200 would be able to get a single place on the ballot paper. At round 26, a slated work with 101 nominations and 101 points would be ahead of both non-slate 4 and non-slate 5, and non-slate 5 would lose out. The slated work would therefore get on the ballot ahead of works with more individual nominations - 101 to non-slate 5's 118.
My initial dabblings with the 1984 data suggest to me that this is an inherent problem; while EPH does make it very difficult for slates to win more than one place on the ballot, it actually lowers the threshold for a group of voters whose tastes differ from everyone else's to get one of their preferred works onto the slate. If anything, the RL figures suggest to me that the problem is a bit worse than in Keith's simulation.
Now, some may see this as a further advantage of EPH - that a small group should get some representation on the ballot fairly easily, and their preferred candidate then gets subjected to the same scrutiny as all the non-slate candidates. I have some sympathy with this, but I feel the bar for a single slate candidate may actually now be set too low, that the system as currently proposed comes close to offering candidates with a very narrow support base an unfair advantage in winning their first (if probably only) seat.
I would love it if those who have better coding skills than me, and access to the 1984 data, could look at these questions:
1) on each final ballot for each category, what is the number of points needed to beat the fourth-placed candidate?
2) if there had been a single slate candidate (with individual votes cast for, or cumulated on, only that candidate), which had enough nominations and points to beat the fourth-placed RL candidate, how many candidates with more actual nominations than the slate candidate would have failed to get on the ballot?
I do have a potential solution for this as well, but if I can be reassured that the numbers are not as bad as I thought, I won't need to unveil it!
Gement@84:
How specific is the available data from the 2014 nomination ballots? Do we have enough information to find out which works would have been nominated under E Pluribus Hugo? If it's not that specific, do you have enough for a statistically distributed estimate?
We don't have the actual ballots from 2013 (I had said 2014 initially, sorry), but one of our group generated a statistical set of ballots that matched the distribution of nominations for that year. We do have the actual ballots from 1984, however.
In the 2013 case, EPH gives exactly the same results as the current system (without the slate I added, of course). As I recall, the 1984 set was -almost- exactly the same, but it seems there was a tie in one category that got resolved differently with EPH than with the current system. I will re-run that data and let you know, since it's probably easier than searching the old threads.
Kilo
Lighthill, thanks for additional info about difficulty (or, in general, lack thereof) in implementation.
Nicholas@86:
I'll see if I can get the data you're asking for, though I'm not sure I'm parsing exactly what you're describing. I think it'll be more clear once I step through what you're saying.
Regarding thresholds, though, one thing to keep in mind is the distribution of non-slate ballots. It seems to me it's not quite as simple as a slate should get 20% of the vote in order to qualify for one slot. The more diffuse the nominations of non-slate ballots, the lower the percentage of the total ballots a slate should need. If you think about it, that's because if lots of non-slate ballots have all their candidates eliminated (because they chose all obscure works), then even if the number of slate ballots doesn't change, they effectively have a higher percentage of the number of ballots with nominees that actually had a chance. That is, they may get (say) 20% of the surviving ballots without necessarily having 20% of the initial ballots. So, the "cut-off" may depend more on the distribution of the non-slate ballots than anything else.
Jameson is the expert on this, though, so I'll defer to him for the theoretical interpretation -- believe what he says over what I say! :)
Kilo
Keith,
I entirely take your point. As far as I can see, the diffusion of ballots makes percentage comparisons pretty worthless.
My concern is that the diffusion of votes among non-slate works may be sufficient to give slate voters an unwarranted advantage in getting on - in your own example, a slate work with 101 nominations will easily beat a non-slate work which also has 101 nominations, and even one with 118.
This was what I worried about previously; I just had't previously worked out that the slate needs to beat the fourth-placed non-slate work as well as the fifth.
@89: That's very well-put, I don't have much to add. Only that lifting the 5-nominations-per-category limit on voters might slightly decrease the effective "dispersion", or decrease the strategy that voters need to use to get a given dispersion. But it's probably not a big deal, because probably few voters would vote for more than 5 anyway.
91
I did arithmetic on the 1984 ballots, and it averaged less than two names per category per ballot.
At the risk of giving Brian Z even more attention, I'd just like to say...
I only started lurking on File770 fairly recently (I'm glad I don't remember who I got the link from, because I might be tempted to curse them for giving me even more interesting stuff to read online) and so I'm now extremely familiar with Brian's rhetorical behavior in the last couple of months, and with everyone else's attempts to get him to substantively address the subjects he brings up. This is someone who, over dozens or possibly hundreds of posts, has repeated variations on "Anti-puppies are saying X" and "The proposed voting changes won't help with Y", and each time people have told him "Please provide some links to X, and examples of Y." And yet, when he complained over there that he'd been poorly received over here, and Bruce Baugh said basically "You might do better if you'd provided some links and examples"... Brian replied that he didn't know what Bruce meant, since he'd done just that: he had provided a link to his own comment!
I always hesitate to say "No one could honestly be that dense"; people are capable of so much. But... no one could honestly be that dense. It's a game.
On topic: I've been following the hammering out of EPH with great interest, and what I've seen makes sense to me. I've never had any involvement in Worldcon, but have considered being a supporting member before, partly as a way to give myself a kick in the ass to read more widely in current SF. I don't feel like this change would make my hypothetical voting decisions any more difficult, and the rationale for the change seems pretty clear. Take that for whatever it's worth, from someone who knew basically nothing about the nominating process prior to this year.
Hob@93:
I don't feel like this change would make my hypothetical voting decisions any more difficult, and the rationale for the change seems pretty clear. Take that for whatever it's worth, from someone who knew basically nothing about the nominating process prior to this year.
I just wanted to say how much I appreciated your comment. This is exactly what we've been striving for: fix the problem, but don't change how people nominate. We've all been immersed in this for a couple of months now, so we're probably too close to it at this point. It's very gratifying to see that someone on the "outside" responds in just that way.
Thanks much!
Kilo
Kilo @87:
Thanks. That answers the body of my question, which is that we don't have actual specifics for recent years. I don't mind relying on statistical models, just wanted to be clear one way or another. And after the final awards, hopefully we can apply the same models to this year's nominations.
Nicholas Whyte @86:
I see the ability for a group of motivated people to apply point pressure and get _something_ on the ballot to be entirely valid. I can think of occasions where that would have been very useful for any number of underrepresented groups, and I'd be glad to see those more-different options no matter what quarter they're coming from.
This year's motivated voting bloc swept the nominations, excluding anyone else. That's not cool. But weeding out a motivated voting bloc from getting representation at all? Even more uncool.
Kilo @94, good! To be more precise, I don't think EPH is instantly clear. It's just complicated enough (the counting part; not the act of voting, which is for all intents and purposes unchanged) to make one wonder why it's done that way instead of the old way. But to me that's a feature, not a bug— since an answer is easily available ("it's done this way so that if a relatively small number of people vote for the exact same 5 things, everything else doesn't get pushed off the ballot") and, I think, pretty easy to understand with an example. Whereas in the old system, which looked simple, it was easy for someone who wasn't thinking in terms of gaming the system to completely miss the possibility of such a thing happening.
(I realize that lack of understanding was less of an issue in this case than deliberate exploitation, and an unenforceable social norm that people would refrain from using that exploit because people should not be jerks. But I think lack of understanding, among those who just hadn't realized the system was so vulnerable, meant that it took a lot of time and effort to explain to people why this mattered, and made it easier for the exploiters to spread FUD about 1. whether they had intended this result and 2. whether something like it had already been going on.)
Cassy B. @76: Actually, under EPH there is a difference between bullet voting and putting the same thing in all five slots. It's only one selection point either way, but in the elimination phase a bullet vote would count as one nomination while 5X would count as five.
IMO a 5X should be condensed down to a bullet vote, and my guess is that the existing rules support that.
David Goldfarb #97:
The existing rule is:
3.8.4: If a nominee appears on a nomination ballot more than once in any one category, only one nomination shall be counted in that category.
David Goldfarb #97:
Given that the current wording of the proposal for the calculation phase is:
"(1) Calculation Phase: First, the total number of nominations (the number of ballots on which each nominee appears) from all eligible ballots shall be tallied for each remaining nominee."
It seems that EPH does prevent an individual ballot from adding more than one count to any work's nomination tally. Regardless of the other rule.
Or is each item on someone's nomination a ballot?
I think @97 is at best an unhelpful counterfactual, and arguably wrong.
David Goldfarb @97, but they'd still all be on one ballot, so only count as one nomination (because if memory serves that explicitly counts ballots, not instances) as well as being one point. Not that it matters, because as I and others have said, the rules eliminate duplicate votes, so it's all hypothetical unless there's a rule change.
Lighthill @80: decided to code EPH as an exercise. […] I would give about 40% odds that it's correct, since I didn't test it, but the key concept isn't hard
Is there a set of test data files that can be used to confirm that an independent implementation works correctly, even in the weird corner cases?
Julian@102: I'd say "no". Because no implementation has been through a complete audit process yet; we haven't done the work to validate either code or test files to the level needed for production use.
103
Should I post the numbers for 1984?
Julian@102:
It kind of depends on what you mean. I'd be happy to provide you with ballot files we've been using if you want to verify that your code gives the same results as mine. As David mentioned, we haven't done a full production-level audit of everything, though. If EPH passes, we'll need a web database app for the real system. I'm not qualified to write that type of code, but for demonstration proposes (which I think may be what you're looking for), what I have works fine.
If you (or anyone else) wants those ballot files, just shoot me an email (Google is your friend :-) ) and I'd be glad to send them out. As I said over on File770, the more people we have doing their own versions of the algorithm, the better, as far as I'm concerned.
Kilo
Soon Lee and David W:
By the way, I think your ideas for a graphical representation of the post 23 results is a great idea. I just wanted to let you know that I'm working on it. I'll post when I make some progress, but of someone else has this underway, let me know.
K
All:
One persistent misconception that I've seen on the various blogs is the idea that EPH encourages bullet voting. We've come up with some fairly involved proofs of why it doesn't, but I think we need some simple, intuitive explanations. I've seen some good ones here and over at File770, so it might be worth gathering and refining those as talking points.
So here's the situation: Someone comes up to you at Worldcon and says, "If I really want to be on the ballot, it still seems to me I should only nominate that one so that it gets my full point."
Can you convince her otherwise in a way that makes sense intuitively?
I have some thoughts, but I'll hold off until some others have chimed in.
Kilo
And that should be:
"If I really want [WORK] to be on the ballot, it still seems to me I should only nominate that one so that it gets my full point."
(Gotta remember not to use angle brackets...)
K
PJ@104:
I'm starting to think posting a ballot file here would be a good idea, just so that anyone can test their own implementations. I'm concerned about the amount of data in the 1984 set though. Maybe if we just posted the 2013 sample we made, that would give people a set to get started with. If they wanted more they could contact you (or me, since I have the 1984 files as well, obviously). What do you think?
Kilo
109
I was thinking about the results, rather than the data files - the data would certainly be too much for here! (I myself am curious about the one category where the EPH results differed from the original - I suspect that's the Campbell award, where one of the top ten was ineligible.)
So far my favorite "bullet voting" refutation is that it is slightly better than the current system there. The argument is simple: at worst, if you nominate 5, any one of your nominees has 1/5 point and one nomination (that gets better if some get eliminated). Under the old system, each of your choices gets one nomination. So, in one phase they're the same, and in the other phase they differ by *at most* 4/5 point.
So, it shows incremental improvement, and emphasizes that this is NOT a big change.
(Any nominating system that doesn't use ranking requires you to make a decision at what point to cut off your nominations -- at what point do you stop listing your favorites on the nominating ballot? So deciding whether to list the three "very good" choices after your "outstanding" favorite is necessary in all cases. Well, except in the cases when you have more favorites than slots, in which case the decision is made for you.)
Another way to argue it:
If you really only care about one thing, then you should only nominate that one thing (true both in the current system and in EPH).
If you have 5 things you feel strongly (and roughly equally) deserve consideration, you should nominate those 5 things (true both in the current system and in EPH). (And similarly if it's only 4, 3, or 2 things of course.)
If you have one thing you consider outstanding, far better than everything else this year, but also think some other works are worthy of consideration, what should you do? Basically, you far prefer the one thing, but if due to democracy in action it has to lose, you prefer it loose to one of these other things. To me, when expressed that way, the question answers itself -- you should nominate all of them. It's possible to construct rare cases where adding the others can cause your favorite to fail, but those cases are very rare, and knowing that they are happening is impossible at the time you nominate (it requires knowing exact, not just rough, counts for all the nominees). This is true both in the current system and with EPH.
I'm pretty busy and/or offline for the next couple of weeks, but after that I can certainly work on doing a graphical representation based on the 1984 data.
Also, the 1984 data is available inside my Hugo github project: https://github.com/The-Center-for-Election-Science/HugoVotesim
As to convincing people that bullet voting isn't favorable, I'd make 3 arguments:
1. If your [BABY] turns out to be more popular than some [OTHER] work you also like, then leaving off [OTHER] won't help [BABY], because [BABY] would never come up for elimination until [OTHER] is already eliminated.
2. If [OTHER] is more popular, then it's probably going to come down to [BABY] against some given [RIVAL]. In that competition, what counts is nominations, not points so bullet voting makes no difference.
3. The only time bullet voting is worthwhile is if you expect it to come down to a choice between [BABY] and [OTHER] for 5th place. In that case, yes, if you prefer [BABY] it could make sense not to vote for [OTHER]. But that's exactly the same situation you're in with the current system, and it's about nominations. This is not about your ballot being "weaker" because the points are getting divvied up; if it's between two works, nominations count, not points.
Now, those arguments leave out two possibilities where bullet voting could actually make a difference, because I think (and my simulations show) that those possibilities are much less likely than the situations above. One possibility is that bullet voting moves your favorite work above a pair of slate candidates, as in my Alpha, Beta, Gimmel example above. This should be rare because I think under EPH slate voting is not rewarded and will die out naturally. The second possibility is that bullet voting moves your favorite work from 6th place all the way up to 4th. This will be rare because it requires your faction to be markedly more strategic than other factions. While some slight differences in strategy level between factions are possible, I think that the situation where the 6th-place work's voters are decidedly more strategic than the voters for 4th and 5th place is an unlikely, low-entropy situation.
But explaining how those two scenarios are possible only to then explain why they're unlikely seems to me to be a poor way of arguing. I think it's perfectly honest to rely on arguments 1 2 and 3 above, and if somebody seems to be groping towards the "pro-bullet" scenarios, to try to steer them away from that rather than helping them get there and then showing them why it's wrong.
Edge Cases
For the current nominating system (not EPH), in what ways is it possible for your nominating a second item to bump your first item off the ballot?
I can't find one that doesn't involve a tie, such as this:
If the counts are A:200 B:199 C:198 D:197 E:100 F:100 (and thus a 6-work ballot because of the tie for 5th place) when you nominated E and not F, if you add a nomination for F then you get A:200 B:199 C:198 D:197 F:101 [ E:100 ] (with E in square brackets because there is no longer a 5th-place tie, and the ballot goes down to 5 entries including F and not including E).
Are there any other ways for the current system to do that that are substantively different?
@114: For any single vote to make a difference in the current system, it needs to either break or make a tie, so yes, any example involving one vote will involve ties somehow. Probably the better way to think of it is you and a handful of like-minded individuals all trying to make the same calculation (without formal consultation). If you and three others who love E all decide to put F on your ballots as well, it's possible that F has 98 votes without your ballots and 102 with, so it would displace E at 100.
Oh, good, David Dyer-Bennet @ 111/112 said stuff much better than my draft while I set mine aside for a bit. But echoing his posts, I think the goal shouldn't be to try to explain that EPH doesn't give some benefit to bullet nominating but rather to explain that EPH doesn't materially change the mental calculus (strategic implications?) of nominating versus the current system, which includes issues like bullet nominating.
@111-113:
Ooo, I like these. Very good explanations. One other point I've thought to make is that -points- are not going to get your favorite eliminated or on the ballot either one. It's -nominations- that make that determination -- and you can't increase the number of nominations you give something by bullet voting.
I could then follow up with the fact that it's true that points determine whether the work will be -eligible- for elimination, but just giving it points won't affect its fate if it comes to that.
Your only hope is to try to give so many points to a work that it never ends up eligible for elimination. You can't do that with only 4/5 of a point extra, so you've got to get all your friends to do the same thing -- and you do that by talking up your favorites to all your friends, just as fandom has always done. Unless you want to force them to give up all of their other choices as well (in which case you just made a "single-work slate"), there's really not much you can do to make -sure- your work gets on the ballot.
And the price for even trying is that you are giving up everything else you think might be worthy to do it.
Kilo
@117: You say "4/5 of a point extra"; but note that the average number of winning candidates per ballot is less than 1. That means that in most case strategy wouldn't even affect the point total of your most pivotal choice (the one closest to the finalist threshold); sometimes, it would mean half a point extra; in very rare cases, 2/3 of a point extra; and essentially never 0.8 points extra. Even if you could somehow know which of your choices is most pivotal, the "expected points extra" you could give it through bullet voting is probably under 1/3 of a point; really not worth it, especially given that chances are that nominations, not points, will be decisive.
(Apologies for poor formatting of tables below. Slightly better version here.)
I've spent more spare time than is healthy over the last few days musing on EPH. As previouslty stated, I am in sympathy with the intention of preventing any group - whoever that group may be - from absolutely excluding nominees from having the chance to be considered for the Hugo Award. I think that the proposal as it currently sits achieves that aim, but at a cost of making it too easy for a group which is otherwise utterly unconnected with Hugo voters to get a single work onto the ballot by "bullet votes" (ie votes for their candidate[s] and no other). I explore this problem below, using data from the 1984 Hugo nomination ballots, and propose a partial solution, which is to use square roots as divisors when weighting nomination votes.
Detail
I'm tremendously grateful to Paul Evans for providing me with the 1984 data. Having spent a couple of evenings crunching figures, I now feel huge sympathy and admiration for the Hugo administrators trying to make sense of the variant titles and spelling submitted by voters. Administering what are essentially thousands of write-in ballots is not exactly straightforward, and I am not sure that I would have the patience to do so in an RL setting myself. Not surprisingly, my tallies vary a bit from Paul's. He has taken more time over it, so his numbers are probably right.
I've picked three different ballot categories from 1984 to analyse mainly because they were relatively easy to process, with less name and category confusion than some of the other options would have presented.
First, Best Fan Writer - in some ways the easiest, because fewest nominations were submitted in this category (481 nominations of 174 candidates by 181 voters). My figures differ from Paul Evans' totals - he found a couple more votes for some of the candidates than I did - but it doesn't make a lot of difference to the story. This is a case where EPH clearly works, and a slate candidate with bullet votes would have had to get enough support to win under the old system as well.
The top seven candidates by nominations were:
56 Mike Glyer
36 Richard Geis
33 Dave Langford
28 Arthur D. Hlavaty
18 Teresa Nielsen Hayden
12 Ted White
10 Claire Anderson
A nice big gap between the fifth and sixth placed candidates, and indeed between the fourth and fifth. Under EPH, the points for each candidate on the final ballot would have been:
Mike Glyer (56 nominations): 28 + 21/2 + 4/3 + 3/4 = 40.583
Richard Geis (36 nominations): 16 + 13/2 + 4/3 + 3/4 = 24.583
Arthur D. Hlavaty (28 nominations): 14 + 7/2 + 4/3 + 3/4 = 19.583
Dave Langford (33 nominations): 6 + 20/2 + 4/3 + 3/ = 18.083
Teresa Nielsen Hayden (18 nominations): 7 + 9/2 + 2/3 = 12.167
To knock Teresa Nielsen Hayden off the ballot, a single slate candidate would have needed to also have 18 nominations which were bullet votes (or at least shared less with other candidates than hers), which would have beaten her 12.167 EPH points. I must say I'm not completely content with this; precisely because TNH had more voters who also supported other popular candidates, she is in a disadvantage in a head-to-head against a more sectarian candidate. But I think any variation of a single divisible vote system delivers this result, and what it basically means is that the last place may go to a representative of a small minority. The question is, how small?
I looked at Best Non-Fiction next because there was a more even spread of candidates at the lower end. 206 voters made nominated 86 different works here. The top 9 were:
45 Dream Makers, volume ii
43 The High Kings
29 The Fantastic Art of Rowena
26 Staying Alive: A Writer's Guide
19 The Encyclopedia of Science Fiction and Fantasy, vol iii
16 Worlds Beyond: the Art of Chesley Bonestell
11 Amber Dreams
11 The SF Book of Lists
11 Uranian Worlds
If we apply the EPH system, it doesn't change much in ranking:
Dream Makers, volume ii (45 nominations): 18 + 13/2 + 9/3 + 4/4 + 1/5 = 28.7 points
The High Kings (43 nominations): 17 + 14/2 + 8/3 + 3/4 + 1/5 = 27.617 points
The Fantastic Art of Rowena (29 nominations): 12 + 8/2 + 4/3 + 4/4 + 1/5 = 18.533 points
Staying Alive: A Writer's Guide (26 nominations): 9 + 6/2 + 8/3 + 2/4 + 1/5 = 15.367 points
The Encyclopedia of Science Fiction and Fantasy, vol iii (19 nominations): 4 + 7/2 + 4/3 + 3/4 + 1/5 = 9.783 points
But because the successful candidates shared a relatively large number of supporters, a single slated candidate with 16 bullet votes would have got onto the ballot, and would have knocked off the actual winner, vol 3 of The Encyclopedia of Science Fiction and Fantasy.
41 slate voters, ie fewer than voted for either of the two front-runners in the category, who voted for two candidates, and only those two, could have knocked out Staying Alive out as well.
The most striking results I've found are for the 1984 John W. Campbell Award, which had a very lopsided vote distribution - one front runner far ahead of the field, and a number of candidates jostling around the edge of the ballot. 247 voters made 519 nominations here. The top 15 candidates, with their total nominations, were:
121 R.A. MacAvoy
19 Joseph H. Delaney
19 Joel Rosenberg
18 Timothy Zahn
18 Sheri S. Tepper
17 Lisa Goldstein
17 Warren Norwood
15 Barbara Hambly
12 Robin Wayne Bailey
12 Dan Simmons
10 P.C. Hodgell
10 Kim Stanley Robinson
10 Lucius Shepard
9 David Brin
9 John De Chancie
Some of these were not exactly new writers in 1984. Timothy Zahn had first published in 1980 and I guess was excluded from the final ballot for that reason. That brought in both Lisa Goldstein and Warren Norwood, tied on 17 nominations.
If we apply the EPH points system, the final six (Zahn having been excluded) end up in the following slightly different ranking:
R.A. MacAvoy (121 nominations): 89 + 23/2 + 9/3 = 103.5 points
Joel Rosenberg (19 nominations): 19 points
Warren Norwood (17 nominations): 9 + 7/3 + 1/4 = 11.583 points
Lisa Goldstein (17 nominations): 7 + 3/2 + 6/3 + 1/4 = 10.75 points
Joseph H. Delaney (19 nominations): 5 + 6/2 + 7/3 + 1/4 = 10.583 points
Sheri S. Tepper (18 nominations): 2 + 14/2 + 1/3 + 1/4 = 9.583 points
Sherri S Tepper would have been eliminated at this stage, although she had more nominations than two other surviving candidates (Warren Norwood and Lisa Goldstein). 165 of the original 247 nominating ballots are still in play. (Wow, MacAvoy made a pretty big splash, didn't she!)
After Sherri S Tepper's votes are redistributed, the point scores for what would have been the final ballot under EPH are as follows:
R.A. Macavoy (121 nominations): 103 + 10/2 + 8/3 = 110.667 points
Joel Rosenberg (19 nominations): 19 points
Warren Norwood (17 nominations): 9 + 1/2 + 7/3 = 11.833 points
Lisa Goldstein (17 nominations): 7 + 3/2 + 7/3 = 10.833 points
Joseph H. Delaney (19 nominations): 5 + 6/2 + 8/3 = 10.667 points
163 of the original 247 ballot papers would have remained in play.
A slate candidate with 11 nominators, none of whom supported any of the other surviving candidates, would have been ahead of Goldstein and Delany on points and would have made it to the final ballot (Goldstein would have lost due to having fewer nominations than Delaney). Such a candidate would have had fewer nominations than five excluded candidates - Lisa Goldstein, Sherri S Tepper, Barbara Hambly, Robin Wayne Bailey and Dan Simmons. These seems to me very unsatisfactory.
It's not irrelevant to note that Joel Rosenberg had 19 bullet votes at this stage (a few of whom had voted for other less popular candidates as well), and that these included ten voters with consecutive membership numbers who cast nominating votes identically for him in this category and for a novel called The Sleeping Dragon and a short story called "The Emigrant". You'll never guess who those works were by.
25 slate voters would have been able to get two works onto the shortlist here (if you eliminate Delaney, Norwood ends up with 12.5 points in third place). Again, that seems to me to be, simply, too low.
Conclusion and recommendation
I hope it's fairly clear that while EPH does, as advertised, make it very difficult for a small set of voters to dominate entire ballot categories, as has happened this year, it also actually lowers the barrier to a small detached group getting their first candidate onto the list. Of course, minorities should not have insurmountable barriers placed in front of them, but for my taste, EPH as presently constructed goes too far the other way. A slate candidate which had fewer nominations than ten more popular candidates could still have got onto the John W Campbell Award ballot. A slate candidate could have knocked the eventual RL winner off the Best Non-Fiction Work ballot despite getting fewer nominations. I don't think that's quite right.
My modest proposal is that the divisor for calculating points should not be the number of candidates supported by a voter, but the square root of that number. Square roots have a venerable place in political calculus, particularly in the apportionment of seats, both in the U.S. Congress and for enlightened commentators on the European Union. Basically, instead of weighting your vote by 0.5 if you still have two candidates left in the race, 0.333 if you have three, and 0.25 if you have four, your vote gets weighted instead as 0.707, 0.577 and 0.5. Looking at my test cases above, the new weighted points would be:
Best Fan Writer:
Noms EPH modified
Mike Glyer 56 40.583 46.659
Richard Geis 36 24.583 29.002
Dave Langford 28 19.583 22.759
Arthur D. Hlavaty 33 18.083 23.952
Teresa Nielsen Hayden 18 12.167 14.519
A slate candidate would still need 18 bullet votes to displace Teresa Nielsen Hayden from the ballot; no change.
Best Related Work:
Noms EPH modified
Dream Makers, v2 45 28.7 34.836
The High Kings 43 27.617 33.466
The Fantastic Art of Rowena 29 18.533 22.413
Staying Alive 26 15.367 19.309
The Encyclopedia of Science Fiction and Fantasy, v3 19 9.783 13.206
A slate candidate would now need 19 bullet votes, rather than 16, to displace the Encyclopedia. That seems to me an improvement.
John W. Campbell Award:
Before we get to considering the slate, there's that matter of the fifth and sixth places to resolve. The top six now look like this:
Noms EPH modified
R.A. MacAvoy 121 103.5 110.460
Joel Rosenberg 19 19 19
Joseph H. Delaney 19 10.583 13.784
Warren Norwood 17 11.583 13.541
Lisa Goldstein 17 10.75 13.085
Sheri S. Tepper 18 9.583 12.977
Lisa Goldstein would drop off due to having fewer nominations than Sheri S. Tepper, and the final ballot would look like this:
Noms EPH modified
R.A. MacAvoy 121 105.833 111.987
Joel Rosenberg 19 19 19
Joseph H. Delaney 19 11.333 14.380
Warren Norwood 17 12.167 14.008
Sheri S. Tepper 18 9.667 13.054
A slate candidate would now need 15 bullet votes, rather than a mere 11, to get on the ballot. That would be the same number of votes as the candidate who just missed in 1984, Barbara Hambly. For me, that's a more equitable outcome.
So, basically, EPH can be improved by using square roots as divisors. That is all.
In fact, there was some 'slate' voting in 1984, and the candidate did get on the ballot. (Actually, there was one obvious slate, and one that was invisible to the administrators: each got their candidate on, and neither won.)
Jameson@118:
Ah, good point. That makes sense to me -- I hadn't thought of that aspect.
K
Nicholas@119:
I'm working through your analysis, but there's one thing that's not clear to me. When you list the EPH point totals, are those the initial point totals or the final point totals after the non-finalists have been eliminated?
Thanks,
Kilo
Keith,
The final point totals. Much easier to calculate!!!!
so people have results for 1984 to compare:
Best Novel
Millennium by John Varley [Berkley, 1983] . . . 52
Moreta: Dragonlady of Pern by Anne McCaffrey [Ballantine Del Rey, 1983] . . . 54
Startide Rising by David Brin [Bantam, 1983] . . . 137
Tea with the Black Dragon by R. A. MacAvoy [Bantam, 1983] . . . 55
The Robots of Dawn by Isaac Asimov [Doubleday, 1983] . . . 75
Best Novella
"Cascade Point" by Timothy Zahn [Analog Dec 1983] . . . 71
"Hardfought" by Greg Bear [Asimov's Feb 1983] . . . 62
"Hurricane Claude" by Hilbert Schenck [F&SF Apr 1983] . . . 42
"In the Face of My Enemy" by Joseph H. Delaney [Analog Apr 1983] . . . 47
"Seeking" by David R. Palmer [Analog Feb 1983] . . . 80
Best Novelette
"Black Air" by Kim Stanley Robinson [F&SF Mar 1983] . . . 39
"Blood Music" by Greg Bear [Analog Jun 1983] . . . 46
"Slow Birds" by Ian Watson [F&SF Jun 1983] . . . 33
"The Monkey Treatment" by George R. R. Martin [F&SF Jul 1983] 36
"The Sidon in the Mirror" by Connie Willis [Asimov's Apr 1983] . . . 38
Best Short Story
"Servant of the People" by Frederik Pohl [Analog Feb 1983] . . . 28
"Speech Sounds" by Octavia E. Butler [Asimov's mid-Dec 1983] . . . 37
"The Geometry of Narrative" by Hilbert Schenck [Analog Aug 1983] . . . 34
"The Peacemaker" by Gardner Dozois [Asimov's Aug 1983] . . . 31
"Wong's Lost and Found Emporium" by William F. Wu [Amazing May 1983] . . . 32
Best Non-Fiction Book
Dream Makers, Volume II by Charles Platt [Berkley, 1983] . . . 45
Staying Alive: A Writer's Guide by Norman Spinrad [Donning, 1983] . . . 26
The Encyclopedia of Science Fiction and Fantasy through 1968, Vol 3: Miscellaneous by Donald H. Tuck [Advent, 1983] . . . 19
The Fantastic Art of Rowena by Rowena Morrill [Pocket, 1983] . . . 29
The High Kings by Joy Chant, Ian Ballantine, Betty Ballantine, George Sharp, and David Larkin [Bantam, 1983] . . . 45
Best Dramatic Presentation
Brainstorm (1983) [MGM] . . . 74
Return of the Jedi (1983) [Lucasfilm] . . . 226
Something Wicked This Way Comes (1983) [Bryna/Disney] . . . 51
The Right Stuff (1983) [The Ladd Company] . . . 87
WarGames (1983) [MGM] . . . 86
Best Professional Editor
David G. Hartwell . . . 83
Edward L. Ferman . . . 129
Shawna McCarthy . . . 96
Stanley Schmidt . . . 82
Terry Carr . . . 83
Best Professional Artist
Barclay Shaw . . . 49
Don Maitz . . . 59
Michael Whelan . . . 123
Rowena Morrill . . . 73
Val Lakey Lindahn . . . 31
Best Semiprozine
Fantasy Newsletter/Fantasy Review ed. by Robert A. Collins . . . 49
Locus ed. by Charles N. Brown . . . 185
Science Fiction Chronicle ed. by Andrew I. Porter . . . 94
Science Fiction Review ed. by Richard E. Geis . . . 70
Whispers ed. by Stuart David Schiff . . . 23
Withdrawn - Nomination Declined: Starship ed. by Andrew I. Porter
Best Fanzine
Ansible ed. by Dave Langford . . . 15
File 770 ed. by Mike Glyer . . . 84
Holier Than Thou ed. by Marty Cantor and Robbie Cantor . . . 25
Izzard ed. by Patrick Nielsen Hayden and Teresa Nielsen Hayden . . . 22
The Philk Fee-Nom-Ee-Non ed. by Paul J. Willett . . . 20
Best Fan Writer
Arthur D. Hlavaty . . . 28
Dave Langford . . . 33
Mike Glyer . . . 58
Richard E. Geis . . . 36
Teresa Nielsen Hayden . . . 19
Best Fan Artist
Alexis Gilliland . . . 35
Brad W. Foster . . . 32
Joan Hanke-Woods . . . 28
Stu Shiffman . . . 34
William Rotsler . . . 37
John W. Campbell Award for Best New Writer [Not a Hugo Award]
Joel Rosenberg . . . 19
Joseph H. Delaney . . . 19
Lisa Goldstein . . . 17
R. A. MacAvoy . . . 123
Sheri S. Tepper . . . 18
Warren Norwood . . . 17
Just to add that Paul obviously found a couple more votes for The High Kings and MacAvoy, and one more for TNH, than I did; and that he probably looked harder and got the right answer.
Nicholas@119:
One thing to keep in mind is that even though cut-off percentages are difficult to discuss (due to non-slate distributions), it's still probably better to talk about percentages instead of absolute numbers. For example, in your Best Non-Fiction example, it's true that 16 bullet votes would make it to the ballot. That's about 8% of the initial ballots, though it's probably a significantly higher percentage of the final ballots. You found that a "two-work slate" would need 41 ballots, but that's 20% of the initial ballots. That's a lot, especially since again it's likely an even higher percentage of the final ballots. It's not quite as much for the Campbell Award, but as you noted, that was kind of a weird case.
We actually considered several different types of proportionality when we were designing the system. The main reason we went with evenly dividing the point was that it is intuitively easier to understand. EPH has been characterized as complex (it's really not that bad once you've been through the process for a couple of rounds -- all the more reason for a graphic demo, I think), so we didn't see a lot of advantage to doing anything more complicated. It definitely could be done, obviously, but to a certain extent, there's still a value judgement in "how much is too much" that ultimately comes down to opinion. We decided that going too obscure and mathematical would hurt our chances of passage, and I still think that's the case.
So, I guess what I'm saying is that I agree with you that going with what you suggest would make it even harder for a slate to get a work on the ballot. But I'm just not sure that the trade-off in complexity is really worth it.
That's just my thoughts, though...
Kilo
Just to add that Paul obviously found a couple more votes for The High Kings and MacAvoy, and one more for TNH, than I did; and that he probably looked harder and got the right answer.
Those are the numbers that were given at the time. (Note that the official list I have is just numbers, no names.)
Sorry for repeating previous comment. Moderators, feel free to delete the second one.
Keith,
I disagree. I don't think percentages are actually all that helpful here. Partly that's because of my own background in STV and other proportional systems, where they are a lot more relevant to the outcome than in this case. But mainly it's because you need to be clear at every stage whether you mean percentage of total ballots, of total ballots in this category, of total continuing ballots in this category, or of the ballots as modified by this or that counterfactual development. I'm preferring raw numbers in this discussion because I think they supply better information in this case.
@122, 123: I haven't gone back and run my numbers, but it certainly seems wrong that you're getting 1/5 points after elimination; that would mean that some people voted for every one of the finalists, which I don't think is very likely. Can you double-check your numbers?
Keith again,
As for the weighting, it seems to me that once you persuade people to accept the principle that votes get weighted at less than full value if you are supporting more than one candidate in the same category, it's less of a stretch to propose an alternative weighting, especially if the intent is to create a reasonable (but not impossible) threshold for newcomers.
@130 Jameson: One person voted for all five finalists in Best Related Work. I was surprised too.
132
It was counted as one nomination at the time. So: one nomination, one point.
(You think that's strange? Someone wrote 'Abstain' on the back of the final ballot and returned it. It was considered to be a valid ballot: the person didn't vote for anything.)
@132, I think you misunderstand. Jameson was querying, and I was confirming, that there was one voter whose nominations perfectly reflected the Best Non-Fiction final ballot, improbable though that may seem.
134
Okay - I wasn't looking at that kind of thing.
Nicholas Whyte @119: "My modest proposal is that the divisor for calculating points should not be the number of candidates supported by a voter, but the square root of that number. Square roots have a venerable place in political calculus, particularly in the apportionment of seats, both in the U.S. Congress and for enlightened commentators on the European Union. Basically, instead of weighting your vote by 0.5 if you still have two candidates left in the race, 0.333 if you have three, and 0.25 if you have four, your vote gets weighted instead as 0.707, 0.577 and 0.5"
Yes, that would make it harder for a small bloc to get a single work on the ballot; but have you looked at how it affects the ability of a bigger bloc to take more slots? Eg a bloc of 40 would be enough to take four slots for the Campbell.
Nicholas@131:
Well, I see what you're saying, but I'm still trying to make sure we convince people to accept the idea of weighting in the first place!
It seems to me that changing the weighting might be a tweak rather than an essential feature of EPH, so maybe should be considered after the system has been used for a few years. We'd then have a better idea if it were needed or not. I certainly don't think we're broken with the weighting we have. As I said, to a certain extent, it's only a matter of opinion as to how much is too much.
Kilo
Better not to deal with weighting - it's going to confuse people. At this stage, confusion is not what's wanted. (We'd need a lot more data before we could know if it's even needed.)
Felice,
It would take a lot more than 40 to take 4 slots at the Campbell! Basically they would each have to equal or beat Joel Rosenberg's 19 points in second place, and that means 76 voters, whether by EPH or my modified proposal (since Rosenberg's score remains the same either way). Or if we treat Rosenberg himself as the potential foundation for such a slate, he needs enough additional voters to outpoll Delany and Norwood, I don't have access to the numbers but I reckon that will be a vote total in the high 50's under the EPH proposal as is, and the mid-60s under my modification.
Paul and Keith,
I'm happy to leave it there; I won't be at the Sasquan Business Meeting and I'm not going to prime anyone else to bring this up on the floor. As I have said, invert much endorse the proposal and hope it passes; this proposed modification would fix a fairly minor hole which I hope we do not see exploited.
"Invert much endorse" -> "I very much endorse". Sorry about that.
Nicholas Whyte @139: "It would take a lot more than 40 to take 4 slots at the Campbell! Basically they would each have to equal or beat Joel Rosenberg's 19 points in second place, and that means 76 voters, whether by EPH or my modified proposal (since Rosenberg's score remains the same either way)"
Rosenberg's score remains the same, yes, but under your proposal the slate's score changes. You're suggesting 0.5 points for four works, rather than the current 0.25 points; that would give a bloc of 40 people 20 points for each of the four surviving works on their slate, enough for all of them to beat Rosenberg's 19. Am I missing anything?
Under the current weightings, three surviving slate works supported by a bloc of 40 would each have 13.333 points, enough to beat everyone except MacAvoy and Rosenberg; still not a great outcome, but not as bad as the slate taking four slots.
Felice,
D'Oh, you're right, of course. My system doesn't penalise multiple-member slates quite as heavily as EPH does. But EPH hits them very hard indeed. Perhaps that's what we need right now, but I think there is a balance to be struck.
So if it understand correctly, this nominating process is the Thunderdome, where Aunty Entity chooses who fights next, and Aunty Entity is Anti Strategy. So the only right way to nominate is Clap If You Believe In Fairies. The Wisdom of the Crowd prevails, through the most Claps.
spacechampion @144:
You might think that. I couldn't possibly comment.
143
That was the point: to limit the damage caused by slates.
Spacechampion @144, the numbers choose who fights next. The numbers choose who wins. There's no "Aunty Entity". E Plurbus Hugo is structured so that there are no gatekeepers, period. It's all math.
Where do you see an "Aunty Entity"? Or is that your nickname for computer programs?
The only thing the administrators need to do is confirm eligeability of the nominees, and make sure all the names and titles are spelled right. And they have to do that anyway.
Do you have a specific concern about the way this works?
Yes, Aunty Entity = the algorithm.
No, no concerns from me. I support the proposal. I'm trying to convince doubters who seem to think this is too complex to get support, and challenged me to state what it is in simple terms they could understand. I figured using SF geekery would do that. The doubter in question claimed that those not understanding it would decline to nominate, and think the process is illegitimate if they don't understand the process.
149
I'd wonder if they'd understood the process to begin with.
I've seen the odd comment from people having trouble understanding how E pluribus Hugo works.
People have been offering different explanations to help explain the concept, so hopefully one of the explanations does the trick.
Soon Lee@151: Well, I hope so. I've now heard from one person that the language is "atrocious" or some such (not looking it up to be sure it was that exact word). I apologized and explained I'd worked on that part a lot.
Soon Lee/David:
On the other hand, I've seen a number of people who weren't part of the effort explaining it to others, so it's not -too- bad. One friend of mine basically said it was too long for her to read, though. I wonder if some of the confusion comes from the length of it. I don't regret posting all the FAQs, however.
Still, the basic idea is simple, it's just the various details that get involved. I'm glad we didn't chase too many more edge cases, though. I think if we focus on the basics (essentially FAQ #1), we can get the message across. But it's probably not immediately obvious that FAQ #1 -is- the important part.
Kilo
Kilo,
Your worked example #23 is extremely helpful as it shows round by round how EPH would work when a slate is in play.
I'm looking forward to a graphical version would aid those of us who are more visual learners.
154
I took his example and turned it into a spreadsheet, with works as lines in the sheet and points/nominations as columns. The eliminated works turn into gray-filled cells (with 'x' in each one), and works that are eliminated have bold red text. It helps when visualizing: you can see them disappear, and see how the other works pick up points.
PJ@155:
I was working on the presentation PowerPoint this afternoon, and did something similar in Excel to what you did (which then got pasted into PP). It sounds like yours is much better than mine; would you be willing to send that to me?
Kilo
156
Should be in your mailbox Real Soon Now.
Accidentally deleted semi-colon:
I'm looking forward to a graphical version; would aid those of us who are more visual learners.
@156: What's the basic outline for your powerpoint? Can we see a draft?
Jameson@159:
Yep, once I've got it in a presentable form, we can start refining it. My wife and oldest daughter have a Girl Scout event this weekend, so I'll be working more on it then.
Kilo
All:
Thanks to PJ's excellent spreadsheet, I was able to make much more rapid progress with the PowerPoint than I thought I would. The presentation has a graphical representation of each round of the elimination process, showing the current works, which are selected for elimination, and then which one is actually eliminated. It also follows the fate of "Voter #44" throughout the process. In between, the tie break rules and how slates are handled are also illustrated.
The PowerPoint can be downloaded here and a PDF (which can be viewed in a browser, but doesn't have the transitions) can be downloaded here.
Please take a look and let me know what you think!
Thanks,
Kilo
The PDF was entertaining as well as pretty clear on how it works.
I've got an infinitely better idea (no, not original to me): People only get to nominate 5 works each. Take top 20 nominees in each category and have a second round of voting, paring down to the top 5.
The purpose of EPH seems to be to make sure that people can still game the system. My response to that is "if you don't want to make the system ungameable, I can't see any reason why we should change the rules at all."
Two rounds of nominating voting means the only way something can make it on the final ballot is if it has real widespread support, not just 40 people who are strongly interested in it.
No "group" has a "right" to see their favorite on the final ballot. At least, if that is the case, then it's also the case that the Hugos have become a sad joke, no longer worth any attention, because they'll be about stroking egos, not about rewarding quality.
I'm open to all sorts of compromises on time periods, when validation of eligibility takes place, where cutoffs go, etc. Where I'm not willing to compromise is this: If we're going to change the way the Hugos are nominated, it must be to a system that blocks everyone from gaming it.
EPH isn't that system.
163
The people running this are volunteers, doing this in their copious spare time. Let's not add to the workload any more than absolutely necessary.
Greg@163:
We actually discussed an idea very similar to that way back when we were first looking at potential solutions. The Hugo admins and others with experience in the matter were resoundingly opposed to the extra workload, so it looks like it has zero chance of passing at the business meeting. It's a good idea, just a bit too impractical. EPH, at least, has the advantage of not changing how the Hugo nomination process works as viewed from the "outside".
Kilo
Keith "Kilo" Watt @165:
As I recall we agreed to put the discussion on hold till after EPH was finalised...
This obviously isn't the thread for such a discussion, though. Would the mods be amenable to opening a thread for discussing longlist proposals? EPH is a good proposal, but a) it's not sufficient on its own to eliminate the slate problem in some categories, and b) it might not pass, so it is still worth looking at additional ways to deal with slates.
Keith "Kilo" Watt@161: I really like the animation effect. One minor point: the bullet points alongside the chart consistently show the works in the opposite order than they appear in the chart, which makes me, at least, have to work harder to match the numbers in the text with the bars on the chart.
166
EPH will minimize the slate problem, which is what we want.
(Anyone who wants a longlist or an added step in the process needs to figure out a time-stretcher for the admins. Srsly, people, it's bad enough figuring out what people are trying to nominate without adding more of it.)
Wait until we've had at least a couple of years of using it before trying to fix problems that haven't come up and may not exist.
@163:
The purpose of EPH seems to be to make sure that people can still game the system. My response to that is "if you don't want to make the system ungameable, I can't see any reason why we should change the rules at all."
According to the Gibbard-Satterthwaite theorem, any system that lets more than two people decide between more than two options can be gamed in at least some cases. Within those fundamental constraints, EPH is state-of-the-art; one of the best systems for avoiding that in most real-world scenarios.
The current system has a demonstrated weakness. If you don't want to fix that weakness until you can find a perfect system, you don't want to fix that weakness, because a perfect system doesn't exist.
dotless@167:
I can easily flip the text descriptions, no problem. I'll do that next version.
Thanks,
K
Keith @ #161
Referring solely to the PDF: On Slide 63 (Round 15) it looks like #10 and #13 have the lowest point total, and should go head to head. (With #10 having about ~38 points). Instead this slide picks #13 and #32. Is the visualisation of the point total of #10 wrong in that slide, or...?
I'd also recommend starting the entire visualisation with a graphical representation of how things would work under the current system. So between slides 3 and 4, insert a slide with bars for each nomination, with width equal to their total nominations. "This is the current system. This is why just ~200 people could totally dominate the ballot."
Two reasons for reminding people of this:
1) Not everyone present might actually have a good understanding exactly what happened under the current system. If that's the case, then one visualisation should prevent quite a bit of discussion with the purpose of bringing everyone up to speed.
2) Near the end of the presentation, I found myself (despite having a very good grasp of this proposal!) thinking how it was totally unfair that the elimination of one slate nomination gave the other slate nominations such a huge boost, and that something should be done about that. Extrapolating there, I can see people not favouring the proposal because of it. So, better to _start_ with something closer to the end situation, and thus show people that the slate ends up with just a single long bar, rather than 5 long ones, and that _this_ is the improvement.
- On slide 10 the text in the graph is in the wrong order. It should be 27 on top, 26 below, just like on the y-axis. Same for slides 13, 14, etc. it really is very confusing if the top thing in the middle is not the top thing on the axis.
- I suggest including a slide listing the number of nominations as per Aan above
- Slide 27 the fourth bullet should IMO start with "This would also have to give ..." to make it clear that we're still talking about the same strategy and that there's no ohter way to add points.
Aan@171:
You're right, it looks like the slate is first eligible for elimination in round 16, not 15. The code tells me which nominee was eliminated (which is in fact #13 in this case), but then I have to manually scan through the data for that round and find the second-lowest point total. I just missed #10. The visualization is just me drawing a box around it -- there's nothing automatic there. :)
I've added the nominations slide and swapped the text boxes as well. The new version is up; this afternoon I'll go through and do another check on the second-lowest point totals, since I missed one. The code, again, is calculating the eliminated work correctly.
Thanks for the feedback!
Kilo
Keith,
Would it make it impossibly complicated to code your visual explanation to show each of those bars broken up by their votes? Black-gray-black-gray or some such; longer bits for bullet votes; shorter bits where divided on a ballot? (I'm afraid that short of assigning a different color per ballot, this would be impossible to implement, but I'm not a coder so I figure it's worth asking.) Maybe have the bars' votes listed by elimination order so the eliminated ones come off the end...? If it *is* possible, it will make what happens in the elimination phase more obvious, as some (but not all) fragments of the bars lengthen each round...
*headesk* Oh-no-second; of course nothing needs to be "in order" because nothing comes off the end. It's either there, or eliminated all together.
*headdesk*
174/175
You mean, have them displayed in order of elimination, from the first round to last, instead of ordered by work?
@176, actually, ordered by work is fine; I was somehow thinking (wrongly) that some fragments of some bars would be eliminated... when that's not the case at all. The fragments of bars get longer and longer, until the work itself is eliminated, but it always has the same number of fragment.
Example: A work with four nominations has 1 point (bullet), 1/2 point, and two 1/5 points.
Then it has (2) 1 points, 1/4 point, and 1/5 point.
Then it has (2)1 points, 1/3 point and 1/4 point.
Then it's eliminated...
Cassy@176:
The only problem is that you have 1300 ballots, so (potentially -- it won't actually be that many) 1300 segments, most of which are only a fraction of a point. On the scale of the graph, I'm not sure you could see anything useful.
Kilo
@178, Good point. It was just a thought. Hadn't considered the sheer number of ballots involved.
@178: you could color the bars by grouping full points, half points, third points, quarter points and fifth points together, so you can see the effect as smaller fractions convert into larger ones.
1890
Bright colors for the smaller fractions, and darker for the larger ones?
All:
Okay, I took Tim's suggestion, so the bars are now colored and show what percentage of each nominee's points came from 1-point ballots, 1/2-point ballots, and so on. I don't know if it tells much more, but it's certainly more colorful! :) Please let me know which version you prefer.
As a special bonus -- no extra charge! -- I uploaded a version that has just the bar graph animation so that you can scan through it without being interrupted by the text.
Here are the links again:
Full PowerPoint
Full PDF
Animation PowerPoint
Animation PDF
Please let me know your thoughts!
Kilo
182
It's certainly fun to watch the bars changing!
I prefer the first version because it's simpler. For someone seeing it the first time, the multi-coloured animation is more likely to confuse because of too many things happening at once.
But I would keep the new animation as supporting material, because it shows something the original doesn't.
I put the originals back up as well for comparison. You can find them here:
Original PowerPoint
Original PDF
I'd like to add a new entry in the FAQ that links to the graphic demo. Once we've made a decision on that, I'll have Linda create the new page for the FAQ on the Sasquan site. Let me know which you prefer.
Thanks,
Kilo
Keith "Kilo" Watt, #182:
That looks fantastic, and very effective!
My only comment is that, on my computer, the blues for 1/5 point and for 1 point look nearly identical.
But awesome presentation -- I think it will really help people understand EPH.
I mostly prefer the new version. I find it very effective in highlighting the distinction between "regular" ballot distribution and slate ballot distribution.
I'm looking at animation-less slides, though (LibreOffice doesn't like what you've done here), so it might very well be that you've reached a point where animations plus colors gets too busy. (I suspect you might be having each bar grow from nothing? If so, that's something I'd cut - only animate the changes from slide to slide.)
Maybe also consider changing the specific colors you've used? Right now they've been picked for being maximally distinct, which contributes to how busy it looks. Instead pick 5 similar colors from a single gradient/hue-shift; just different enough to be easily distinguishable, but not more than that. This distinction is after all not the primary information you're conveying with the slide - it's okay if it's less obvious.
I mostly prefer the new version. I find it very effective in highlighting the distinction between "regular" ballot distribution and slate ballot distribution.
I'm looking at animation-less slides, though (LibreOffice doesn't like what you've done here), so it might very well be that you've reached a point where animations plus colors gets too busy. (I suspect you might be having each bar grow from nothing? If so, that's something I'd cut - only animate the changes from slide to slide.)
Maybe also consider changing the specific colors you've used? Right now they've been picked for being maximally distinct, which contributes to how busy it looks. Instead pick 5 similar colors from a single gradient/hue-shift; just different enough to be easily distinguishable, but not more than that. This distinction is after all not the primary information you're conveying with the slide - it's okay if it's less obvious.
Pedant here. You are comparing 2 things, but are using the term "fewest." It should be "fewer" and someone will spend much too much time at the Business Meeting telling you/us so if you don't change it. (Yes, sometimes 3 or more things are being compared, but they're being compared pairwise, aren't they?)
I appreciate all of the time and effort that people are putting into this. Thank you.
Keith "Kilo" Watt @ #182:
As another note on the colours used in the presentation, it looks like you have red and green next to each other -- at least that's how they appear on my screen -- and some people who are colour-blind (such as some of my male family members) have difficulty distinguishing between those colours. I'm not sure how common it is, but it's something I try to avoid.
One thing I'm not entirely comfortable with in regards to the color version is that I think there is too much emphasis one the 1-point bar getting longer and not enough emphasis on the fact that the point total is increasing. I'm out of room on that initial slide, though. I may need to just add another text slide.
Regarding colors, I'm open to suggestions. I would have to go through and change all 80 or so graphs by hand, so if we're going to change that, it'd help me if we settled on colors beforehand! :)
K
I've uploaded a small update to all the versions, changing the "fewest" to "fewer" and also changing the round 1 slide. In the PDF, that slide is way too busy, though in the PowerPoint, where bullets are appearing one at a time, it's not too bad. I'd prefer to explain things while they are looking at the graph, so I'm loathe to make a separate text slide for it.
Incidentally, for the business meeting I plan to use a similar PowerPoint, but without quite so detailed text, as I'll be explaining it verbally as we go along.
Kilo
Here's a quick 'n dirty mockup of the type of color scheme I think we should aim for:
http://tinypic.com/r/2d8hms2/8
My aim here was to make the individual colors just distinct enough to be easily distinguishable (including, if I didn't make any mistakes, by most color-blind people), but blending together enough that the emphasis returns to the length of each bar, and not so much to the individual sections.
All:
New version uploaded with Aan's color scheme. Pleae let me know what you think as soon as you can!
Thanks,
Kilo
EPH PowerPoint
EPH PDF
EPH Animation PowerPoint
EPH Animation PDF
I looked at the EPH PDF, and it works for me, still.
Keith: I think the new color scheme works well. One point that I found a bit confusing on my first click-through: I think it would be better to say you will next "fast forward" to round n rather than "skip ahead" to round n. The latter description led me to expect that you would go immediately to the designated round, rather than having an abbreviated version of what happens in the intervening rounds. By analogy with fast forward on a DVD or VCR, I think the "fast forward" terminology matches better what the slide show is doing.
Hi Keith and team,
I appreciate that you have put a lot of work into the worked example, but can I recommend very strongly that you experiment with some of the 1984 real data? I think it will convey the message in a much more meaningful way. My top tip on this is not to include any votes for the simulation that don't list any of the top ten candidates.
Keith in particular,
I've only now spotted that on an earlier thread you crunched some of the 1984 numbers as I did here for three categories. We come to different conclusions for two of those threee, and I think it's because of identifiable (and understandable) error on your part, but just wanted to check in with you.
For Best Fan Writer, you have Ted White taking the final place instead of Teresa Nielsen Hayden. But you found only 10 nominations for her in the 1984 data, which of course puts her behind White's 12 on the old system as well; I found 18, taking into account all variations of her name (which I won't annoy her by listing here), which is easily enough to keep her on the ballot.
For the John W. Campbell Award for Best New Writer you seem to have simply lost Joel Rosenberg's 19 nominations (most of which were bullet votes), which is why he isn't on your simulated final ballot. I seem to have found another few nominations for some of the others than you did also.
Please don't take this as in any way a personal criticism - having spent a couple of evenings last week trawling through the data myself, I know just how frustrating and difficult it is to do this, and I only looked at three of the categories! But I thought I should record our differences here and see if you agree with my proposed explanation.
One of the complaints about EPH is that it's going to be too complex to code. No number of statements here from the proposers saying "It's simple really" is going to convince anyone who dislikes maths or statistics.
Would it be helpful to contact the 2017 site bidders, one of who would be the first to implement it, to get their impressions, maybe do a one-to-one run through?
I think their support would be essential; on the other hand their opposition could be a problem. And their personnel will include a fair few attendees.
So should we contacts the bid chairs to ask who to invite here to have their fears, if any, allayed?
David@197:
Good suggestion; I'll do that for the next version.
Nicholas@198:
You're right, I didn't attempt to go through and manually change names (though I think I may have removed initial "The" in titles, if I recall). At the time I wasn't sure if Hugo admins fixed spelling or not. I'm sure that's the difference between results. I'd be glad to send you the 2013 ballots (and the 1984 ballots I actually used) if you want to do another comparison.
As for using 1984 data with the current demonstration, I'm not sure that's a good idea. I read a number of comments along the lines of "30 year old data doesn't tell us anything about modern voting patterns", so I'm not sure it's any more convincing than the 2013 statistically-generated ballots. For the demo, the point is to show how the system works and how it limits the effects of slates. That can be done with pretty much any realistically-distributed set of ballots. I'm concerned about making the demo more complicated by including titles and such. One thing I learned in teaching is to not hit your student with everything at once. Nothing you teach them should be -wrong- (you don't want them to have to unlearn anything) but sometimes throwing the kitchen sink at them hurts more than it helps. I could be convinced otherwise, I suppose.
Doire@199:
One thing we should remind people of is that EPH is much simpler -- both to explain and to code -- than the current instant runoff voting system used for the final award. I think a lot of people lose sight of that fact.
I worked on the pointing software for Hubble and Chandra, so coding is fairly easy for me, but this was all pre-web, so I know virtually nothing about writing web-based code (particularly when it comes to security aspects). However, it occurs to me that we could use the existing code for gathering the Hugo nominations (assuming such code exists -- I've never nominated). As long as it outputs to a comma-delimited text file, the code I've already written is perfectly fine for running the actual nomination process. So I think there may not be any need for anyone to write any code at all. I had been thinking that it all had to be done in a single application, but that's silly. There's no reason not to break apart the data gathering from the processing (and quite a few good reasons to do so).
Kilo
Doire@199:
Forgot to add: But in spite of the fact that we may not need any new code, I agree it's a good idea to get the 2017 bids on board with the idea. That makes sense no matter what.
K
200
I ran 1984 through a spreadsheet and did my own count. I can't get a 100% match on the numbers, but the difference is no more than one for any given nominee. (One nominee appeared in three categories, all the shorter fiction.)
Keith,
I'd be glad to send you the 2013 ballots (and the 1984 ballots I actually used) if you want to do another comparison.
No, no, it's fine thank you!!! I don't especially want to try crunching the numbers again - once was enough! - I just wanted to make sure we had a shared understanding of the differences between our figures, which I think we do.
As for the demo, I'm sure you have more experience in training people on this kind of thing than I do; it's just that I personally found the lack of titles a bit of a barrier to understanding what was going on. But I'm not the one who would have to rewrite the powerpoint deck if my suggestion is implemented, so I'll drop it.
Was out of town and didn't see the bright-colored version, but I have to say I like the shades-of-blue version. The fact that all the slate candidates stay pale blue while the other bars get (on average) darker and darker... does point up the difference between slate nominations and more typical fannish chaotic nominations, and I think that's a relevant difference that's easy to see in this version. (NOTE: I didn't run the power-point; I just went to the .pdf. I assume they're largely similar.)
Kudos!
All:
In the "To-Do" thread I have posted some proposed language for a new FAQ entry for the graphical demo, as well as expanded two other entries in response to concerns I've seen in the community. If you would, let's do some quick wordsmithing on those so that we can have Linda post the new FAQ page for the system.
Thanks!
Kilo
Here's a thought: is there any technical reason preventing the 2016 administrators from calculating next year's nominations both (as they must do) under the current system and as they would be under EPH, and publishing both sets of lists? (Obviously only the list produced by the current rules would be valid.) Perhaps, if EPH passes (as I hope it does), the Business Meeting can invite the Midamericon committee to pursue this. I imagine it's within the committee's powers to decide whether it should proceed with such an experiment or not.
Nicholas Whyte @206:
Or the 2015 admins publishing the EPH list for this year's nominations, before the Sasquan Business Meeting? (though obviously that invitation couldn't come from the Business Meeting!)
207
They can't do that before the Hugos are announced.
@206:207 - the only constraint on the Admins is to execute the rules faithfully, which includes the requirement in 3.11.4 to publish some of the workings by 90 days after the con.
They could, without violating the rules, publish all the nominating and final ballots in detail with names, but I wouldn't expect them to...
So no, I wouldn't expect them to do that. They could, but it would be Controversial...
I wouldn't expect the full ballots with names. That's way too far. (There's enough flak with releasing anonymous data.)
The Business Meeting could ask the Administrators to do the things you suggest. The Administrators would not be required to do any of those things, and would be completely within their authority and within all tradition to refuse to do any of those things.
The WSFS Business Meeting can only compel Administrators by amending the WSFS Constitution. Everything else is a request, and need not be complied with. (This is pretty typical of the seriously decentralized nature of WSFS governance.)
All:
I've uploaded a new version of the PowerPoint with a slide that transitions between the description of the old system and the new system. It does something else, however. One thing that I think we need is an "elevator pitch" to very quickly explain the system to people here online and at the con. I can imagine there are a number of different "level" of pitch we need, for example, someone who has no interest in the inner workings at all, someone who who wants a broad overview of the details, etc.
On the new slide, I've put what I think might be one version of an elevator pitch that could be useful. I'll copy the text here for discussion:
"Why are slates a problem?"
If someone then says, "Well how does that stop slates?" The simple answer is that any set of nominees that have both the same point total and the same number of nominations will tend to compete against each other. Slates tend to meet that condition, so will generally eliminate each other until only one slate nominee is left.
Thoughts?
Kilo
Me@212:
In the second bullet point, is it better to say "none of their nominees individually has enough support to compete with a dedicated slate" or "none of their nominees individually has enough nominations to compete with a dedicated slate"? I can see someone saying, "Well, if people don't support them, they shouldn't win!" But the point is that people -do- support them, they just like wide variety of other things too. We generally use "support" to mean the combination of nominations and points (which is what I mean here), so I don't want to confuse things, however.
Kilo
212
In the fourth bullet point, I'd leave out 'five', because people shouldn't be thinking they have to fill all five in. (And then, 4/6 might pass....)
PJ@214:
Yeah, I can see that. I think I was trying to subliminally coach people away from bullet voting. :-)
Kilo
Suggestion re: "Why are slates a problem?" text: Where it says "divided among your five nominees", change it to "divided among all your nominees". That way, you get the don't-need-to-bullet-vote thing, and you also don't urge people towards any particular number of nominations.
Cubist@216:
Okay, that works for me. Good idea!
What do people think about #212 as a 20-second explanation?
Kilo
I kinda like #212 for the way it says things but I think it is a bit too complicated/wordy for an easy explanation. May be just me. :)
Annie@218:
Maybe the answer is for the first approach to be just the first three bullets:
"Why are slates a problem?"
If the person says, "Okay, but -how- does it redirect their support?" only then would the last two bullets be needed:
And then, finally, if they ask, "So how does that limit the effectiveness of slates?"
I think you're right that it really has to be tight. I'm way, way too close to it to avoid giving too much detail, I think. :)
Kilo
...and if we want the shortest possible answer to the question "So what is this E Pluribus Hugo thing?" we could just say, "EPH is a change to the nomination system that allows the least-supported nominees to be eliminated, and then automatically re-directs the voters' support to the remaining works on their nomination ballots."
If they're interested at that point, more details could follow.
Kilo
I agree with Annie Y in that the"elevator pitch" @219 is too wordy & takes too long to say to be an effective elevator speech. Unfortunately I don't have a good alternative to propose.
I'm grasping here:
"E pluribus Hugo only changes the way nominations are tallied, to minimise the effectiveness of slate nominating. As a Worldcon member, your nominating process remains unchanged."
Then we go to the next level of more detail.
Soon Lee@221:
That's a good one, too. Having several "on hand" and ready to use for answering questions is a probably a good idea, and I agree, having a really short one for those who really don't want to talk about it much is definitely the way to go. I think what you propose would be very reassuring to someone who hasn't been following the controversy at all (and I bet there will be many).
Kilo
Kilo,
A thought: would it be worth putting the elevator pitch (whichever version works best) in the Power Point? I was thinking after the title slide & before the summary slide. It would orient the audience/reader as to what they will be looking at in subsequent slides.
For the Power Point - maybe as a first slide
- You do not change what you nominate
- You do not change how you decide how many books to nominate
- The tallying of the nominations changes to minimize the effectiveness of slates.
OK - this is clunky but this is the message I am thinking - nothing changes for anyone that nominates today (so if you never knew how they were counted, it does not even make a difference); the outcome changes if you are supporting a slate basically.
Aaaand... Brian Z. is off trying to spread as much disinformation and FUD as possible on other people's blogs.
Yup, he's been reduced to vaguely implying things that would be both false and mutually contradictory if he said them outright. And then defending himself by reminding everyone that the people refuting him once called him a troll so they are obviously ad hominators who can't be trusted.
He's been doing that over at Westeros for a while, too, but Felice has been doing a great job of debunking him there.
I just have to shake my head at someone seeming to get so much pleasure out of being a wrecker.
And, Faithful Reader reports, Brian Z. has been busy at File 770, too.
What the hell does he get out of this? Does he really think we don't notice what he's up to?
JJ@227 - that's Brian? Oops; I should probably have recognised the style...
I just saw this and wanted to mention that I'm not whoever it is on Westeros - though if you can point me to "me" I'll go have a look at what I'm saying. :) My actual latest thoughts are on my blog.
Good luck with your proposal.
After viewing the PDF posted by Kilo, it's plain to see this will neither prevent "gaming" the system nor eliminate "slate" voting. In the example in the PDF, a voting block of 110 voters with a single nominee would be guaranteed a slot on the finalists' ballot. With 110 "points", it would be immune to the "selection" process. There would be ten nominees with more nominations, but six of them would be eliminated.
This is not a system for fans. It's a system for people with narrow tastes who follow trends. The fan who reads twenty novels a year and nominates five will have a vastly decreased chance of seeing ANY of his nominations on the final ballot than someone who reads one SF novel a year and nominates that one.
Ironically, one of the biggest complaints against the Sad/Rabid Puppies (which mostly isn't true) is that they "oppose" diversity and want to stifle it. So our proposed solution is one that ACTIVELY DISCOURAGES diverse voting and gives a weighted advantage to monolithic voting and publishers who put a big publicity push behind their Hot New Authors.
The Hugo Awards have now officially become Animal Farm.
231
Nice try, but it's already been tested and found to work as described.
John Cairnes @231: "In the example in the PDF, a voting block of 110 voters with a single nominee would be guaranteed a slot on the finalists' ballot. With 110 "points", it would be immune to the "selection" process. There would be ten nominees with more nominations, but six of them would be eliminated."
Four of the six that get eliminated are from another voting bloc; no loss there! One of the remaining two is eliminated anyway by the survivor from that other bloc. The addition of a bloc of 110 does cause one work with about 135 nominations to be eliminated; that's because nearly 60 of the people who nominated it also nominated other finalists. So putting the 110 bloc's work on the final ballot means representing more people (110 vs about 78).
"The fan who reads twenty novels a year and nominates five will have a vastly decreased chance of seeing ANY of his nominations on the final ballot than someone who reads one SF novel a year and nominates that one."
That's completely wrong. If a fan nominates five novels, and four of them get eliminated, the remaining nomination is exactly equal to the nomination of the fan who only nominated one novel to start with; there's no advantage to only nominating one. On the contrary; if one of the five-nominator's novels is eliminated, they still have four left, while if one of the one-nominator's novels are eliminated, they have nothing left. Using all five slots maximises the chances of getting something you nominated on the ballot.
John@231:
As others have pointed out, your analysis basically isn't correct. At all. I'd normally think this would mean that we need to modify the PowerPoint, because we must not be explaining things correctly. However, nearly your exact post here was made on June 23 at https://sfkittens.wordpress.com/2015/06/22/kittenpuppy-dialogues-on-pizza/ by a poster named "Xephon". I should point out that post is dated before I posted the first version of the PowerPoint.
Of course, I'd be willing to give you the benefit of the doubt that Xephon is not you, and you just read that post and liked its phrasing, but even on that thread, the post was pretty thoroughly debunked. As a result, I'm not 100% sure there's anything we should do in response to your criticism. On the surface, at least, it just seems to be a modification of a post made by someone who just didn't understand the system -- or understood it well enough to realize that it prevents slates from being able to dominate the ballot and therefore is something to be adamantly opposed.
Kilo
Damn, there are a lot of folk out there trying to kill EPH, aren't there?
Hmph...wouldn't be trying that tactic unless THEY'RE afraid of it.
Bwa-hah-ha...
235
Brian Z was still trying this week, over at File 770. He wasn't having any luck. (I think he's afraid of people here.)
I caught his act over there. How stupid can you get? The more frantic they get about it, the more determined I am to see it pass.
Lori@235:
In reading around the various blogs (Puppy-aligned blogs included -- yes, I try to keep to up with what everyone is saying as best I can), I think things are looking quite good for EPH. A lot of people are realizing that the system really does what it's designed to do. It's been described as "elegant" in a number of places. Even VD said he would neither support nor oppose it (of course, he predicts EPH will cause the Hugos to implode, so he gives that as his reason). We still need to convince the business meeting attendees who have not been following all the discussions, however.
I think a lot of the remaining opposition is people who just don't want any rule changes at all. Of course, that would of necessity mean that slates would continue to have the power they had this year, so one would have to be a bit suspicious of the true motives there...
And by the way, hat's off to all of you who suggested the PowerPoint and contributed to it. I think that's gone a long way towards helping people understand the system. I think our various "elevator pitches" will further that goal, as well. The details are what they need to be, but the underlying concept really is pretty straightforward.
Kilo
For what it's worth, although I'm not going to Sasquan, I just accidentally found out that a friend-of-a-friend is going. I asked him, tentatively, if he'd heard of the Puppies; he gave a disgusted look. I gave him the elevator pitch, strongly urged him to research EPH for himself, and he told me that if, after looking at it in detail, he agreed with me that it would limit the influence of slates, he'd go to the Business Meeting. So there's that...
Cassy@239:
That's outstanding. As always, feel free to direct him here or to me personally (online or at the con) if I can answer any questions he might have. I'd be interested in the take of someone coming at it "cold", as it were.
Kilo
Just a note that the Hugos are not the only awards processes we at electology.org have helped: the Webby awards just wrote a blog post about how we helped them too.
For the Webby awards, what was needed was a lot more technical than EPH, but at heart, it's about the same thing: building a system to make good, robust use of the voters' judgment as expressed on the ballots.
Kilo, I have a vested interest in this sort of thing because I helped set up the Pegasus Awards for OVFF. I used the Hugo Award system as the template, back in the mid-1980s.
Not too many years ago, Steve Macdonald refined the award process and brought it into the 21st century, for which the whole committee was exceedingly grateful.
Filking fandom is small enough that the measures we have in place are working. I'm hoping they always will. But it's nice to know there are folks available who will help should changes become necessary.
@232
Nice try, but it's already been tested and found to work as described.
Please reread my comment. I never said it didn't work as described. I said it discourages fans who read and nominate multiple works and give weighted advantage to monolithic voting blocs supporting a single nominee.
@233
Four of the six that get eliminated are from another voting bloc
Only because the example is set up to to present a non-varying voting bloc. A novel with 200 votes that appeared coincidentally on 200 otherwise-dissimilar ballots of five nominees would have only 40 points, would be equally penalized, would have at best a 50/50 chance of making it past the slate vote and would STILL have less chance of getting on the final ballot as a nominee with 110 single-vote ballots. That novel would not be a "slate" vote, but would be collateral damage in this process.
The addition of a bloc of 110 does cause one work with about 135 nominations to be eliminated; that's because nearly 60 of the people who nominated it also nominated other finalists. So putting the 110 bloc's work on the final ballot means representing more people (110 vs about 78).
No, if it appeared on 135 ballots, then it got 135 nominations, not 78. Is this really what it's coming down to, pretending nominations don't exist?
@234
As others have pointed out, your analysis basically isn't correct.
What part did I get wrong? You neglected to metion that. Of the others you mentioned, one puts words in my mouth and the other wants me to believe that a work that got 135 nominations actually only got 78 (or only 78 of them actually "count" - I'm not sure which).
@235
Damn, there are a lot of folk out there trying to kill EPH, aren't there?
That's because we know it's unfair; we know that in the zeal to punish the Wrongfans it will penalize nominees (and voters) who don't deserve it and, no matter how much you spin it, you can't hide the fundamental unfairness of this system.
243
You are still incorrect.
You might want to actually read what people are saying, instead of repeating what the various juvenile canines have said about EPH, as you seem to be doing here.
A novel with 200 votes that appeared coincidentally on 200 otherwise-dissimilar ballots of five nominees ... would STILL have less chance of getting on the final ballot as a nominee with 110 single-vote ballots.
This is false. (It's also ambiguous, but anything it could possibly mean is equally false.)
Of the others you mentioned, one puts words in my mouth and the other wants me to believe that a work that got 135 nominations actually only got 78...
Now that's chutzpah. "Don't listen to him, he's saying "ni!"; and as for her, ni! ni! ni! ni!"
That's because we know it's unfair; we know that in the zeal to punish the Wrongfans it will penalize nominees (and voters) who don't deserve it and, no matter how much you spin it, you can't hide the fundamental unfairness of this system.
And the veil comes off. It's a matter of faith; "we" know it's unfair, just as Homer knew Zeus could win a tug-of-war against the other gods. (Also, "we" know that EPH supporters think in terms of "Wrongfans", despite any disavowals of such thinking.)
With that much knowledge, who needs evidence?
John, why don't you produce a representative dataset that would work the way you're worried about? Set up the ballots in a Google spreadsheet, process them in an open and auditable way, link it here. Show the comparative results to the current system; demonstrate how it does what you say it does.
Pony up.
abi, I'm still trying to understand his claim that a change in counting nominations would change how people nominate. That one ... makes no sense, especially with Kilo's worked example.
(The complaints do remind me of previous visitors.)
PJ Evans @247:
I see two claims:
(a) it discourages fans who read and nominate multiple works and
(b) give[s] weighted advantage to monolithic voting blocs supporting a single nominee
(a) is indeed as provable as the subtle damage that marriage equality will somehow do to straight people and the fabric of society as a whole (in other words, not at all; it's FUD), but
(b) can be proven by a dataset.
He's certainly late to this party with those claims.
John Cairnes @243: "A novel with 200 votes that appeared coincidentally on 200 otherwise-dissimilar ballots of five nominees would have only 40 points, would be equally penalized, would have at best a 50/50 chance of making it past the slate vote and would STILL have less chance of getting on the final ballot as a nominee with 110 single-vote ballots. That novel would not be a "slate" vote, but would be collateral damage in this process."
It would start with only 40 points, but most of the other works on its ballots would get eliminated early on (they'd have a lot less than 40 points), so its points total would quickly increase. Depending on how much it shared ballots with the other finalists, it would probably end up with over 150 points, and there's absolutely no chance it would get eliminated.
"No, if it appeared on 135 ballots, then it got 135 nominations, not 78. Is this really what it's coming down to, pretending nominations don't exist?"
It's counting people rather than nominations. Adding the 135!work to the final ballot means 78 extra people get at least one of their nominations on the final ballot, and another 57 people increase their number of finalists to two or more. Adding the 110!work instead means 110 extra people get one of their nominations on the final ballot.
felice,
It's counting people rather than nominations. Adding the 135!work to the final ballot means 78 extra people get at least one of their nominations on the final ballot, and another 57 people increase their number of finalists to two or more. Adding the 110!work instead means 110 extra people get one of their nominations on the final ballot.
Now you've given math-challenged minions a direct hint. Next they'll filibuster the meeting by making everyone watch The Search for Spock while hung over. :D
But seriously, if the "wisdom of the crowds" question or indeed the "needs of the many" question might trouble the soul of John Cairnes, or Sam Harris, or Brother Guy Consolmagno, or any troll willing to start brushing their teeth by about 8AM on August 23rd, there is an outside chance it might be a good topic for discussion. Even with liberal arts majors.
(Sorry to come back and trouble you, folks. In fairness, you do keep invoking me.)
John@243:
Given that you are using the same rhetoric I've seen posted elsewhere, I'm not sure you're actually sincere in wanting to have your concerns addressed. Nevertheless, I feel an obligation to try.
Felice has already explained to you why no one is discouraged from reading and nominating multiple works -- and in fact, it is to their advantage to nominate as many works as the Worldcon constitution will allow them to. The key here is that under EPH, the number of points a work has increases over successive rounds. Many of the arguments I've seen on the Puppy-related blogs have used the same argument you have. But just because they start with fewer points, doesn't mean they will stay that way. As Felice pointed out, you are much more likely to get one of your nominees on the final ballot if you nominate the maximum number of works in a category. If you had actually walked through the example in the PowerPoint without pre-judging it, you'd have seen that is true. I'm not at all certain you're interested in the truth, but it's there if you want it.
That's because we know it's unfair; we know that in the zeal to punish the Wrongfans it will penalize nominees (and voters) who don't deserve it and, no matter how much you spin it, you can't hide the fundamental unfairness of this system.
We know nothing of the sort. It's not unfair, and no one is spinning anything. But the main thing I want to respond to in this comment is that there is no such thing as "Wrongfans". Period. EPH has no way of knowing the political affiliations of anyone who nominates. I won't support any system that does. EPH does one thing and one thing only: It makes it so that no small group of organized fans can keep all other works from appearing on the ballot. That's it. Nothing more. To that extent, it works to answer the original justification that was given for Sad Puppies slate to begin with. If there had been a shadowy cabal controlling the Hugos, they will have no power once EPH is implemented. If you are a Puppy, I would think that would be exactly what you want. All of the rest of the social aspects of the Great Hugo Debate of 2015 have to be handled by some other proposal -- and EPH will happily work with anything fandom wants to do in that regard.
It may be that you feel preventing an organized bloc of voters from controlling the Hugo ballot is inherently unfair. That's your right. That, however, is the only reason not to support EPH. It's your right, but I think you will find a significant number of fans disagree with you, even those who count themselves Puppies, if they were being honest about their motivations.
Kilo
Jameson@241:
Thanks for linking that article -- really cool stuff!
K
As a matter of historical interest for Kilo and others: Very early on in discussions leading to EPH, there were a few comments that sounded (to me and others) like "If a work you nominated in a category is discarded, the rest of your nominations will also be discarded." Clearly, EPH doesn't do that, and the visualizations make it even clearer that when a work is discarded, your other nominees all get a larger fraction of your point.
I'd chase down the comment(s), but I'm on the second day of a nasty fever and complications. Just wanted to note the point.
Kilo,
you are much more likely to get one of your nominees on the final ballot if you nominate the maximum number of works in a category.
At this point I'm half-expecting that a screen will roll back as Standlee bangs his gavel and Hari Seldon himself will explain that EPH ushers in a dark age of 30,000 years. But having said that, I find your willingness to discuss your proposal with all comers a credit to you and your arguments, even though I haven't been convinced that wanting to give more people something they want on the ballot is the most appropriate goal. Thank you.
Bruce@254:
That's interesting, and you're right EPH definitely does not toss out your entire ballot if a work on your ballot is eliminated. As you noted, the whole point is to let your support become more and more focused on your surviving nominees. But it's useful to know that some people got that impression, as it shows us a possible misconception that we'll need to look out for and explain as needed. I appreciate your giving us that perspective!
Kilo
Keith: It's my social history training. Much of social history is basically "So how did people get hung up on that, anyway?" And like I said, the current visualizations make it perfectly clear how the actual proposal works.
Brian Z @251:
But seriously, if the "wisdom of the crowds" question or indeed the "needs of the many" question might trouble the soul of John Cairnes, or Sam Harris, or Brother Guy Consolmagno, or any troll willing to start brushing their teeth by about 8AM on August 23rd, there is an outside chance it might be a good topic for discussion. Even with liberal arts majors.
Citations and links to Sam Harris and Brother Guy's views on EPH, please. For your convenience, here is Brother Guy's commenting history on Making Light. I will also call his attention to this thread and he may comment himself (he may not; he's a busy man).
Sam Harris has never logged onto Making Light, to my knowledge.
Furthermore, please explain how EPH is a "needs of the many" solution. Be explicit and clear. Use citations and details.
And please unpack what you mean about "liberal arts" majors. I am interested to hear what, precisely, you're getting at with that comment.
Comment honestly and with full citations or your vowels are in play, as I have warned you.
Also, peripherally, the system that most thoroughly embodies "the needs of the many" and "the wisdom of the crowds" is... democracy.
Brian, do you object to the fact that the WSFS uses voting to determine both whether the bylaws will be changed and who gets the Hugo?
Asking Brian Z to "Be explicit and clear", and "Use… details"? Sheesh! You might just as well start disemvowelling the disingenuous troll right now…
As I have said repeatedly both here and at File 770, Brian is welcome to comment on Making Light if he obeys the moderators and follows the rules of the community. Because of his behavior in the past, I'm simply making it very clear precisely what he needs to do.
We try to keep a wide door here, and I'm a firm believer in the redeemability of humanity.
abi @ 261... I'm a firm believer in the redeemability of humanity
That Catholic upbringing, eh? :-)
Are there any plans to submit a request to the Business Meeting to ask the administrators to release anonymised ballot data, as Kevin Standlee mentions as a possibility?
I would appreciate some comments in the other thread.
Jameson, I left you a little something on the other thread -- have at it!
@244
You might want to actually read what people are saying
I did. Here are some statements that support my assertions:
Nicholas Whyte @29: "However, I think we both agree that the group of voters whose influence is reduced by EPH are is a subset of those who nominate multiple candidates;"
and
"I do think it's important to note, also, that the voters whose power is increased are those who make single nominations for the most popular works."
and
"I think it's a price we have to pay, but we should be clear that it is a price."
(Of course, no price is too great when someone else is paying it, right?)
And then there's Nicholas Whyte @119: "I hope it's fairly clear that while EPH does, as advertised, make it very difficult for a small set of voters to dominate entire ballot categories, as has happened this year, it also actually lowers the barrier to a small detached group getting their first candidate onto the list."
Okay. I stand corrected. It's not Animal Farm. It's Affirmative Action. Not that I consider that much of an improvement.
@245
Now that's chutzpah. "Don't listen to him, he's saying "ni!"; and as for her, ni! ni! ni! ni!"
Well, it didn't take long for the ad hominem attacks to begin. Childish mockery doesn't strengthen your position. It weakens it.
@246
John, why don't you produce a representative dataset that would work the way you're worried about?
What would be the point, when all I've gotten so far is deflection and insults? Mr. Whyte has done an adequate job of stating my case that the coincidental appearance of two or more nominees on two or more long list ballots inevitably leads to the devaluation of both. The only variable is one of scale. That devaluation will occur whether the voters are "gaming" the system, whether they are just a bunch of fans who are trying to get recognition for their favorite authors, or whether the voters have no connection beyond coincidental interests. I have no feelings whatsoever about the first, but the second and third don't deserve to get marginalized as the price for assuaging other people's outrage.
And no one has yet refuted my assertion that a coordinated effort by, say, a publishing company with a substantial marketing budget, to promote a one-vote slate will create immunity from both the selection and elimination phases.
@252
Given that you are using the same rhetoric I've seen posted elsewhere, I'm not sure you're actually sincere in wanting to have your concerns addressed.
Oh, so now I am incapable of thinking for myself? I don't expect you to believe this, since you have obviously made up your mind that I am some kind of sock puppet, but my understanding of this process comes ENTIRELY from the PDF you posted and the comments of Nicholas Whyte on this page who, I admit, clearly understands it better than I do.
no one is spinning anything
Well, here's an example: On Slide #8 of the Powerpoint, you assert "Points don't cause elimination!" Perhaps, not directly, but they are the element that leads to works being selected for elimination, so they are at least indirectly responsible. You also neglected to mention that a sufficiently high point value provides nominees with a degree of immunity to selection/elimination that may be significantly disproportional to the number of nominations they actually receive.
It may be that you feel preventing an organized bloc of voters from controlling the Hugo ballot is inherently unfair.
I'm still on the fence about that. But from what I'm seeing so far, it seems the cure is worse than the disease. Sasquan will be my 26th Worldcon. For most of the last decade, I've listened to, and participated in, discussions about how to keep the Worldcon "relevant" and how to boost interest for a convention that traditionally has drawn 4000 - 8000 attendees in an age when many conventions draw 100,000 - 200,000 fans. I supported the Sad Puppies only to the extent that they have made the first genuinely successful effort to increase interest in the Worldcon and the Hugos (I didn't participate in the slate voting; I sent in my ballot before I'd even found out what they were nominating). And the amount of crap that's been thrown at them as a result has horrified me. I came here in good faith with my reservations about this process and had only posted two comments before people started impugning my honesty, sincerity, and intelligence (that is, when they weren't dismissing me entirely). If I've "come late to the party" as one commenter said, it's because, unlike you, I DON'T have any inclination to "keep up with what everyone is saying". I'm finding the signal-to-noise ratio in fan discourse is at an all-time low right now.
My questions about this process will not the last ones asked. If the typical response is to make insinuations and spit bile at the people asking, you won't win a lot of converts. EPH is not transparent; it is not intuitively easy to understand and it appears to be a knee-jerk reaction to a manufactured crisis. The fans who have not yet taken a side in this fight (and yes, I'm sure there are many) will find that last one to be particularly onerous.
I hope you will consider running the program on this year's nominees after Sasquan. I would be interested in seeing how it addresses an actual slate vote, rather than one that has been crafted to illustrate the process. The actual slate nominees have a much broader vote range than your example (i.e., from 145 to 338 for the novella category) and will probably have a much wider range of point values. They are much less likely to eliminate each other than you've predicted.
266
The program has been run with data that matches the distribution this year. It's been run with the nomination data from 1984. Those tests have been run by multiple people, multiple times, and the results are visible, for a limited set of data, in Kilo's files that you don't seem to have paid any attention to.
Possibly you could read what people here and elsewhere have actually said about EPH, instead of going after straw problems seen only by puppies. Or, just possibly, you could have come in two months ago, when your input would have been timely if still hlepy.
@266:
Here are some statements that support my assertions:Nicholas Whyte @29: .......
If you had actually read the thread carefully, you would realize that Nicholas Whyte@41 said that Nicholas Whyte@29 was based on a misapprehension.
Well, it didn't take long for the ad hominem attacks to begin. Childish mockery doesn't strengthen your position. It weakens it.
The words you quoted from me do not make any characterizations of your intellect or motives. I simply pointed out that within the span of a single sentence, you accused one person of putting words into your mouth, then put words into the mouth of another.
What would be the point, when all I've gotten so far is deflection and insults?
The point is that you said "A novel with 200 votes that appeared coincidentally on 200 otherwise-dissimilar ballots of five nominees ... would STILL have less chance of getting on the final ballot as a nominee with 110 single-vote ballots" and I responded "This is false. (It's also ambiguous, but anything it could possibly mean is equally false.)" You are claiming that something happens in some vaguely described case. I am point-blank denying that there is any such case. In order for you to prove me wrong, you need only one scenario. In order for me to prove you wrong, I would need to develop enough notation to write a mathematical proof, which is a much larger undertaking. So I really think the burden of proof is on you.
(I am so confident that there is no way for you to add and/or modify 200 non-slate votes in Keith's scenario such that they all vote for a single work and yet that work is not a finalist, that I'm willing to make a one-sided bet: If you show I am wrong about this, I pledge that if I'm given time to speak at the BM, I will read and/or sing any statement written by you, not to exceed the length of "I'm a little teapot". I'll also do the dance if you want me to.)
If the typical response is to make insinuations and spit bile at the people asking, you won't win a lot of converts.
Not a single person has responded to you without substantive, germane points. I admit that a few people have made such responses to Brian Z; but most people, including abi, Keith, and (I hope) myself have remained willing to engage with his arguments and not just his person.
The actual slate nominees have a much broader vote range than your example (i.e., from 145 to 338 for the novella category) and will probably have a much wider range of point values. They are much less likely to eliminate each other than you've predicted.
This is a substantive point, and I think we should run a scenario to show that you're wrong. What happens is that even though two slate works have disparate point totals, eventually all the works "in between" are eliminated, and so they still eliminate each other.
John Cairnes:
And no one has yet refuted my assertion that a coordinated effort by, say, a publishing company with a substantial marketing budget, to promote a one-vote slate will create immunity from both the selection and elimination phases.
I look forward to seeing your motion on the Sasquan Business Meeting page.
John Cairnes @266:
I hope you will consider running the program on this year's nominees after Sasquan. I would be interested in seeing how it addresses an actual slate vote, rather than one that has been crafted to illustrate the process.
So will you be supporting the Hugo Nominating Data Request that Kilo has suggested in the "Post-Proposal Planning and To-Dos" thread (sister to this thread)?
John Cairnes @266: "no one has yet refuted my assertion that a coordinated effort by, say, a publishing company with a substantial marketing budget, to promote a one-vote slate will create immunity from both the selection and elimination phases."
They can do the same thing under the current system. Except at the moment, they'd be better served by promoting a full slate of five rather than a single work (multiple finalists is better for them than a single finalist), while EPH makes it harder to get more than one finalist. And under EPH, they'd have to promote "don't nominate anything except this one work we recommend" to have a slightly better chance of making that work a finalist than under the current system, and seriously, who's going to pay attention to a campaign like that? Positive attention, anyway; there could well be a backlash against any work promoted in such a fashion, causing people to withdraw their nominations for it. Heavy marketing might get you more nominations, but it won't stop people nominating other stuff they like.
@268
Not a single person has responded to you without substantive, germane points.
Really? What is "substantive" and "germaine" about: "You might want to actually read what people are saying, instead of repeating what the various juvenile canines have said about EPH, as you seem to be doing here."
or
"With that much knowledge, who needs evidence?"
or
"Oh, is that what Mr. Beale is planning on doing next year?"
I'm not really seeing the "substance" in those statements. They are snide, dismissive and insulting. I am particularly sick and tired of getting tarred with the Beale-brush.
@269
Guilt-by-association - even when that association exists only in your imagination? Is this what you need to feel morally superior to people with whom you disagree? If so, I feel sorry for you.
@270
So will you be supporting the Hugo Nominating Data Request that Kilo has suggested in the "Post-Proposal Planning and To-Dos" thread (sister to this thread)?
Yes, I would very much like to see that. The 2013 run presented here contains manufactured data that is intended to illustrate the process. I don't believe the true effect of the process can be charted unless it's tested on a dataset that is both relevant to the problems it's supposed to solve and is free of fudge factors designed to produce specific illustrative effects.
All:
The new FAQ page is up on the Business Meeting site. Linda is also hosting the PowerPoint and PDF there as well (links are in FAQ#2). They are the same files as are on my website, but I will leave those up for people who follow the links from here.
@271
Positive attention, anyway; there could well be a backlash against any work promoted in such a fashion, causing people to withdraw their nominations for it.
Within the past few days, I've become aware of at least two persons or agencies that are giving away Sasquan memberships. One is a vocal opponent of the Sad Puppies; the other is remaining anonymous, but is actively seeking out people who are likely to oppose the Sad Puppies. I'd like to know how these memberships are being transferred from purchaser to recipient - and how the accompanying voting privileges are being handled.
I'd ask the people involved, but I'm quite certain getting whacked with the Beale-brush would the least offensive response I'd receive. And I'm certainly NOT seeing any of the "backlash" you mentioned.
Of course, they may be completely innocent but, if I had the money and the desire to steal a Hugo Award ceremony, this is how I'd do it.
274
Ask the people doing it: Not Our Problem.
(By the way, this is something that puppy-supporters have a tendency to do: show up on unrelated sites and demand that other people fix stuff they think is wrong somewhere else. Don't do it, mmkay?)
As for your claims of 'unsupported': you're the one who showed up at the end of the discussion and wants everything explained all over again, never mind the explanations and links to graphic presentations that explain everything and will actually answer your questions. We've explained and demonstrated and shown how it works: it's your problem, not ours.
John Cairnes: If you do any programming, you can get the existing data sets - or set up your own - and set up your own implementation of EPH to test results. (Presumably, sharing your source code so that others can check it for errors.) If you work with spreadsheets, you can do it that way. (Ditto.) And you can even do small data sets by hand. You can see for yourself things that do and do not happen.
Even a sort of toy universe would be illuminating. Say, a couple dozen voters, six of whom all vote a slate, a couple of their choices also get a few other votes, the rest spread around ten or so other works. Verify, when you cannot trust.
John, following up on Bruce's@276:
Should you decide you want to verify things for yourself, I'd be glad to send you all of the various test cases that I ran. There were dozens, but you might find it useful to look at them and run the results. I'm more than happy to do that for you, just let me know.
Incidentally, I think you are mis-characterizing the 2013 data that was run. It was not manufactured to illustrate the process. There are no fudge factors. The example does have slate ballots manually added, but that's only because there was no slate in 2013. I ran it both with the original distribution and with the slate ballots to compare the results. It was designed to give the same distribution of nomination numbers as the actual ballots -- because we don't have the actual ballots. But it was generated using statistical methods, not "manufactured" in the sense you mean. We also ran it with the -actual- 1984 ballots. I'd be glad to share that with you as well, as I indicated. I have to point out that the whole reason we used the 2013 data over the 1984 data is that there were some people who were saying our analysis using the 1984 data had no meaning since voting patterns have changed in 30 years. As always, we have continually tried to be responsive to critiques and suggestions, especially from those opposed to the system at the outset. I, too, hope that this year's ballots will be released. I think it would be very informative.
Cally(269) and felice(271) directly answered your assertion that EPH will allow mega-publishers to dominate the ballot. Just to highlight the salient parts, as Cally pointed out, why would they bother? By definition, the mega-publishers are likely to get a work on the final ballot even if they do nothing at all. If they pushed a single work, they'd face a backlash, both at the nomination stage and at the final voting stage. It wouldn't be a smart move for them at all. As felice points out, they would also have to convince a -large- bloc of voters to give up nominating -all- of their other preferences. That's just not likely to happen. But let's say it does. How would we know the difference? Tor and Baen and all the others are very likely to get at least one work on the final ballot no matter happens.
Responding to some of your other points:
You also neglected to mention that a sufficiently high point value provides nominees with a degree of immunity to selection/elimination that may be significantly disproportional to the number of nominations they actually receive.
Please look at slide #32, where I do exactly what you say I neglected to do. If you can give a work enough points to put it into 4th place, then it is safe, otherwise, it all comes down to number of nominations -- exactly as I said. And given that you can't give more than 4/5 of a point and typical finalists have over 100 points, the odds of you being able to do that are very, very small.
This type of statement is an example of why it seems to me that you haven't read the PDF. It's why it doesn't seem -- at least on the surface -- that you're interested in truth. It's also why others have directed your attention to what we've worked so hard to put out for you.
And the amount of crap that's been thrown at them [the Puppies] as a result has horrified me.
I don't believe you've seen any of that here. I've been very clear -- to everyone -- that this is about slates and not the Puppies. As I've said several times, I am pretty much exactly the Puppies' demographic. The Puppies aren't the point. Slates are the point. I can't help but notice that you complained about "tarred with someone else's brush", yet you seem to have no compunction about doing the same thing to me.
Oh, so now I am incapable of thinking for myself? I don't expect you to believe this, since you have obviously made up your mind that I am some kind of sock puppet
No, I believe I specifically said that you deserved the benefit of the doubt and that you just liked the phrasing you had read elsewhere. At no point did I say you couldn't think for yourself. You did, however, use phrasing and comments that have been used, if not by you, then by others. I even linked to one example. This is the rhetoric that I have seen elsewhere and to which I was referring. You can choose to be offended by that, but there was no offense intended -- it was simply observable facts. Regardless, please don't put thoughts or implications in my mouth that I didn't say.
I'd like to know how these memberships are being transferred from purchaser to recipient - and how the accompanying voting privileges are being handled.
I don't know. I have no way of knowing. We aren't empowered to even consider that. We aren't "Puppy kickers" or "SJWs" or even "the Other Side". We are a group of fans who feel that bloc voters forcing all other works off of the ballot destroys the Hugos. That's all. Any other arguments really have to be taken up with those who have a say in it. We aren't the anti-Puppies you're looking for, I'm afraid.
I think if you read the posts in how I've led this effort, I have been careful to be respectful to all parties. But your tone has come across as angry and bitter from the start. Perhaps that wasn't your intent, but it's certainly not conducive to a discussion of your opinions, and I think that is some of what you're feeling in the responses you've received.
In the end, though, I'd like to help you better understand what EPH does, because in spite of what you've said, some of your comments indicate that you really don't understand it. As was suggested, I think one of the best ways to make sure we aren't "fudging" anything is to run it yourself. I'd be glad to help you -- or anyone else -- do that. I've made that offer here, File770, and other places several times. i'm not sure how much more open I can be.
Regards,
Kilo
John Cairnes @274: "@271 Within the past few days, I've become aware of at least two persons or agencies that are giving away Sasquan memberships."
That has absolutely nothing to do with #271. Would you care to respond to the actual point under discussion, as raised by you in #266?
Also, what exactly are the requirements for obtaining these free memberships? Are people being required to promise to vote a particular way? In any case, these memberships are totally unconnected to EPH since it's a proposal which won't come into effect before the 2017 Hugos at the earliest.
Keith: I also want to highlight and signal boost felice's point that "one publisher could push a single work and get it on the ballot" is NOT an argument against adopting EPH. In order to make an argument against that, you have to highlight a thing you think is a weakness and then demonstrate that the weakness would be worse under EPH than under FPTP.
And -- as felice said -- while it's true that under EPH a coordinated effort can get one work on the ballot, under FPTP a similar coordinated effort can dominate the entire ballot.
Oh, and Mr. Cairnes? If you're really "late to the party" and "on the fence"...tell me where you picked up that word "Wrongfans"? I have yet to see anyone use it, besides Puppies desperately feeding their own persecution complex. So when I see you use it, it doesn't incline me to believe that you are really honestly neutral.
David@279:
In order to make an argument against that, you have to highlight a thing you think is a weakness and then demonstrate that the weakness would be worse under EPH than under FPTP.
That's an extremely good point, and something that I think has been missing in the posts of many people opposed to EPH. It's worth re-iterating when these things come up.
Kilo
John Cairnes came in cynical and negative, and complained when people called him on it. He cherry-picked Nicholas Whyte's comments, ignoring that he later saw that he was on the wrong track. He insisted that people here be accountable for the actions of people elsewhere. Her used talking points instead of substantive conversation.
The evidence points to the idea that he (unlike, ironically, Brian Z) does not, in fact, want to discuss EPH and its effects. He just wants to have a fight.
Congratulations, John; you've had as much of one as you're going to get. I hope you enjoyed it.
If you (or anyone) genuinely have a problem with EPH as a process, please discuss the specific scenarios data that you think would cause a problem, in detail, either with Keith's kind offers or on your own. Do it neutrally. Community, please give a neutral response when people do this.
To help that along, from this point forward, I will disemvowel everything from John, from other newcomers, and from locals responding to such comments, that I deem to be about the Puppy kerfuffle rather than about the mathematical model and electoral process described in EPH.
There may be a day or two's lag, because I'm actually on vacation right now. But it will happen. Rely on it, everyone, and act accordingly.
@272: Would you rather talk about your perceived flaws in the system and back up your statements? Or would you rather talk about how we're being mean to you?
You have claimed "A novel with 200 votes that appeared coincidentally on 200 otherwise-dissimilar ballots of five nominees ... would STILL have less chance of getting on the final ballot as a nominee with 110 single-vote ballots". Please show a scenario where that happens or withdraw your claim. If it's the former, my earlier commitment holds. I really don't want to do more of the who said what when thing; this should be about the proposal and its merits or flaws.
I'm going to presume that Jameson didn't see my 281 when he hit "post"on 282. The grace period for this kind of thing is not, however, very long at all.
Right, that was a cross-post. Sorry.
So I do have one question - it's about the slate, on slide 77 the presentation says
"Due to the tie breaker rules, a slate with
perfect discipline will eliminate all nominees
on the slate in one round!"
But then the next sequence of slides shows round 23 to 26? It looks like I am missing something?
I'm not sure the point wouldn't be better made with the slate at 201,202,203,204,205 votes? Or would that be more confusing? It's unlikely that all the slate nominees would be exactly the same, I'd expect more of a clustering effect?
I'm having trouble, coming into it cold, with how the slate votes kind of vanish in the event of a tie. It seems that the likelihood of that is remote.
I'd also like to thank everyone for working on this - I think it is a great solution to prevent lock-out of the final ballot by a slate. I don't have the math to follow exactly along, but the presentation is heading in the right direction.
285
The slides have to compress several intermediate steps (eliminating several items, for example) to get to the next point. You can tell by the titles: each of the succeeding slides is a different round.
Chris: I think the underlying point is that perfect slate discipline is really unlikely. The Puppies aren't the only people who'd have nominated Butcher and Anderson for novels, for instance. Even in the poor ghastly short fiction categories, some of the works there likely got some non-Puppy votes. A one-fell-swoop five-way elimination would require that every slate voter voted for all five works (which apparently didn't happen this year) and that nobody outside the slate supported any of the works. The worked example reflects what we know of actual voter behavior around slates.
Chris:
What Bruce says is right. The tie breaker rules say that if two nominees are tied for both lowest point total and for number of nominations, then all of the tied works are eliminated. If a slate has perfect discipline, then this will be the case for all five nominees, all at the same time. The slate won't have enough points yet to be in at least 4th place (and therefore be safe), so the first time they come up for comparison, they will all be eliminated together.
But as Bruce as says, the odds of that happening are vanishingly small, because not only does the slate have to have perfect discipline, but also -everyone- else has to avoid nominating even one of the slate works. One nomination from anywhere is enough to break the perfect discipline. We fooled around with trying to handle this case early on in the development, but we soon realized that the odds of it happening are just too small to worry about. And if does, well, slates are something we'd like people to avoid anyway...
Kilo
Chris@285:
I forgot to also address your question about the slides between the "instructional slides". I wanted to show every single round of the process, but there are rounds where nothing really illustrative happens. That's why I put in the "let's fast-forward to round XX" bit. You can then just blast through the next few rounds until you get to the one with a title in red. But if you want to follow those elimination steps, you can. In fact, I have a version of the PowerPoint that has nothing but the bar graphs. It's kind of interesting to fast-forward through the PowerPoint and watch how things are selected, eliminated, and the total point bars grow.
Kilo
Hey Brian Z, I see you're active on File770. Please note that I asked you some questions in comments 258 and 259.
If you reappear on Making Light without answering them, I will disemvowel the comments I am seeking clarification on, and any further comments you make on this site.
You're still welcome if you follow the rules. You're just increasingly unwelcome not to.
I'm copying, with permission, a comment made by Rev. Bob on file770 http://file770.com/?p=23595&cpage=32#comment-307085 It was made in reply to a persistent questioner but has useful illustrations and ways of explaining EPH. I know some of you will have seen it, but I thought it worth having here too.
@Brian Z: “Finally, you ignore the concern that voters have the right to make five nominations and EPH can winnow their number of nominations down, even if their nominees had more votes than others, reducing their voting power; instead, you simply deflect the point by claiming that voting behavior shouldn’t change.”Ah, Brian, you lying bastard. I would ask just how stupid you think we are, but judging by the transparent problems with the “arguments” you post while JAQing off, the answer would appear to be somewhere around flatworm level.
Still, you have managed to say something interesting. I’m not responding for your benefit, because I think you’re a troll who’s just trying to get attention, but because I don’t think anyone’s tried this particular illustration yet. In fact, should anyone associated with the EPH proposal wish to appropriate any part of this explanation as part of their presentation, I give them permission to do so with my blessing.
Let me try the simple approach first. I live in Tennessee. In the 2012 GOP Presidential primary election, my state voted for Rick Santorum. At the GOP convention, Mitt Romney became the nominee. That doesn’t mean the ballots for Santorum weren’t counted, or that the Tennessee delegates were excluded from the convention – only that there weren’t enough of them to win. Likewise, in the Hugo nomination process, there are only a few winners (usually five) in each category. If nothing on your ballot is in that top echelon, your selections were considered but did not win. Such is the nature of the “first past the post” (FPTP) nominating process. All EPH does is modify the way works with low support are eliminated from consideration. It does so by broadening the definition of “low support” to include “heavy support from few people” in addition to “light support from many people.” It’s the same concept either way, that a worthy nominee must reach a certain “critical mass” to become a finalist – but FPTP can’t recognize one of them and is therefore flawed. EPH recognizes both versions. It does that by using math. You do not appear to understand the math, but that’s what it does. If you get that, congratulations! If not, keep reading…
Now, the long version, in which there are algorithms and other complex scary things. Yes, EPH commits the Unforgivable Sin of deleting properly-nominated works from (copies of) voter ballots for the purpose of compiling the final ballot. Horrors! Oh, woe! Someone, think of the emotions of small animals!
Except, of course, that the current system does the same thing. It just does so differently. The current system can be interpreted with absolutely no change in function or results as working in either of the following two fashions:
1. A central figure, either a human or a piece of software, creates a blank tally list for each category. That entity looks at each nominating ballot and, for each work listed, increases that work’s count by one (if the work was already on the list) or adds the work to the list with a starting count of one. This task is repeated until there are no more ballots. The list is then sorted in descending order by count, and the top five (or more, if there is a tie for fifth place) items become the finalists.
2. Each nominating ballot’s data is entered into a computerized data structure, such as a database, which can be queried to yield information. A central entity, as defined above, queries the structure for the work with the fewest number of nominations. If there is a tie, all tied works are selected. If there are five or more unselected works, the selected works can be (and are) deleted from the structure; otherwise, report all remaining works as the final ballot. Repeat that query-and-delete step until the final ballot is all that remains.
Those two algorithms will give the same results every time, guaranteed. Granted, I didn’t add in the 5% threshold check, but that’s because I didn’t feel like looking up that step’s precise details. I also left out the validation step, where nominated works are examined to determine that they are eligible. Anyway, the first method is FPTP, and the second is… call it Last Man Standing (LMS). All that’s different is that one picks the top and discards the rest, while the other discards from the bottom until it preserves the top. It’s like describing a dime as heads-up versus tails-down; it means the same thing.
Either way, the top is the top and the bottom is the bottom and the top stuff goes on the ballot. Truth be told, although I haven’t looked at the current software, I wouldn’t be a bit surprised to find out that it’s actually more like LMS than FPTP. LMS is kind of roundabout in some ways, but using a data structure that can generate reports is very useful. The 5% threshold is also easy to set up as the first LMS step, whereas with FPTP it means running a test on the presumed victors… but that’s a tangent in an already long post.
What matters is that it’s the same thing. FPTP and LMS carry out precisely the same task. You’re still making five nominations, and the ones that weren’t on the top-five-overall list get the axe either way. That’s the nature of culling a longlist down to a shortlist. Your ballot isn’t getting modified, but the software slices its data a few different ways in the calculation process. Some of those steps involve removing some – or all! – of your nominations from consideration, on the grounds that what you’ve nominated had insufficient support from the rest of the community… like the guy who writes in Ron Paul when he votes in the 2016 primaries.
You’re looking at FPTP and concluding that the only solution is social. The rest of us have realized that by tweaking LMS’s removal process, we can plug the vulnerability that bloc voting exploits. Since we disapprove of that hole, and since LMS is the same as FPTP anyway, we’ve decided that we like this new LMS. We like it so much, in fact, that we gave it a new name: EPH.
You still get your five nominations! either way, and the ones with insufficient support still won’t make the ballot. All that’s going away is a known bug that the overwhelming majority of Hugo voters want to see exterminated: the ability of any relatively tiny group to control the nominations. It doesn’t matter to us who’s in that group, what their motives are, or what their politics are – we don’t like that loophole and want it closed. Period, done, end of story.
Doire #291: Thanks for passing that along, it's cogent and interesting. Also, I like that phrase "JAQing off" -- I hadn't heard it before, but figured it out pretty quickly.
P J Evans @124,
I've been playing around with the HugoVotesim data and your numbers for the past few days on my morning commute. I've found a few differences:
- In Professional Artist I get 124 votes for Michael Whelan (there was one slight misspelling of his name)
- in Semiprozine I get 186 votes for Locus, after moving some ballots from category 10.
- In Fan Artist I get 36 votes for Stu Shiffman after cleaning up name differences and moving some from category 11.
- In Campbell I get 124 votes for R. A. MacAvoy after cleaning up name difference.
- In Campbell I get 18 votes for Timothy Zahn.
Are your numbers from the official results? Was Zahn disqualified for some reason?
Thanks for all the effort. This is a great proposal.
Zahn was disqualified because he was past the eligibility period.
I ended up sorting by name rather than category, to find all the misplaced entries. It was interesting as an exercise.
P J Evans @294,
Zahn was disqualified because he was past the eligibility period.I ended up sorting by name rather than category, to find all the misplaced entries. It was interesting as an exercise.
The data is pretty dirty. Hopefully I've cleaned it enough to make my implementation of EPH useful as a comparison.
Thanks for the additional info.
Patrick@295:
Yeah, the data is quite dirty. I didn't do any cleanup before my first runs (way back in one of the earlier threads) with the 1984 data -- it's amazing the amount of variation in spellings you get. Kevin Standlee was recently making the same point on File770. If we do manage to get the 2015 data, I imagine there's going to be quite a bit of cleanup there as well.
Please let me know if you run into any problems with your EPH implementation. Happy to help!
Kilo
Keith "Kilo" Watt @296,
Is this thread where most EPH discussion is happening or is there a better forum?
Hi Patrick -
This is probably the easiest place, just because the signal-to-noise ratio is a lot higher than other places. I and several others check here pretty much every day, so it's probably the best place to get answers to questions.
Kilo
Here are the results of my implementation of EPH on the 1984 data. As a side effect, my code found several duplicate ballots (voters 1444, 4112, and 4554) which changed the total ballot count in a few categories slightly from that provided by P J Evans.
Can anyone confirm these numbers?
Novel
Startide Rising: 105 1/4 points, 136 ballots
The Robots of Dawn: 52 3/4 points, 75 ballots
Moreta: Dragonlady of Pern: 41 3/4 points, 54 ballots
Tea with the Black Dragon: 40 1/6 points, 55 ballots
Millennium: 33 5/6 points, 52 ballots
Novella
Seeking: 43 19/20 points, 80 ballots
Hardfought: 40 17/60 points, 62 ballots
Cascade Point: 38 37/60 points, 71 ballots
Hurricane Claude: 27 11/30 points, 42 ballots
In the Face of My Enemy: 24 47/60 points, 47 ballots
Novelette
Blood Music: 34 1/6 points, 46 ballots
The Sidon in the Mirror: 27 1/6 points, 38 ballots
Black Air: 26 11/12 points, 39 ballots
The Monkey Treatment: 25 1/4 points, 36 ballots
Slow Birds: 19 1/2 points, 33 ballots
Short story
Speech Sounds: 29 1/2 points, 37 ballots
Wong's Lost and Found Emporium: 25 5/6 points, 32 ballots
The Geometry of Narrative: 25 1/2 points, 34 ballots
The Peacemaker: 25 1/3 points, 31 ballots
Servant of the People: 22 5/6 points, 28 ballots
Non-Fiction Book
The High Kings: 29 37/60 points, 45 ballots
Dream Makers Volume II: 28 7/10 points, 45 ballots
The Fantastic Art of Rowena: 18 8/15 points, 29 ballots
Staying Alive: A Writer's Guide: 15 11/30 points, 26 ballots
The Encyclopedia of SFF Volume 3: 9 47/60 points, 19 ballots
Dramatic Presentation
Return of the Jedi: 145 1/3 points, 225 ballots
The Right Stuff: 47 1/12 points, 87 ballots
War Games: 42 5/12 points, 86 ballots
Brainstorm: 35 1/12 points, 74 ballots
Something Wicked This Way Comes: 20 5/6 points, 51 ballots
Professional Editor
Edward L. Ferman: 64 7/12 points, 129 ballots
Shawna McCarthy: 45 5/6 points, 95 ballots
hartwelldavid: 45 5/12 points, 83 ballots
Stanley Schmidt: 43 1/6 points, 82 ballots
Terry Carr: 41 3/4 points, 83 ballots
Professional Artist
Michael Whelan: 89 1/2 points, 124 ballots
Rowena Morrill: 40 1/2 points, 73 ballots
Don Maitz: 32 11/12 points, 59 ballots
Barclay Shaw: 28 points, 49 ballots
Val Lakey Lindahn: 24 1/12 points, 31 ballots
Semiprozine
Locus: 120 19/60 points, 185 ballots
Science Fiction Chronicle: 43 2/5 points, 94 ballots
Science Fiction Review: 33 3/20 points, 70 ballots
Fantasy Newsletter: 21 2/5 points, 49 ballots
Whispers: 11 7/30 points, 23 ballots
Fanzine
File 770: 69 5/12 points, 84 ballots
The Philk Fee-Nom-Ee-Non: 18 1/2 points, 20 ballots
Holier Than Thou: 16 1/12 points, 25 ballots
Izzard: 14 11/12 points, 22 ballots
Ansible: 7 1/12 points, 15 ballots
Fan Writer
Mike Glyer: 41 5/6 points, 58 ballots
Richard E. Geis: 24 1/2 points, 36 ballots
Arthur D. Hlavaty: 19 1/2 points, 28 ballots
Dave Langford: 18 points, 33 ballots
Teresa Nielsen Hayden: 13 1/6 points, 19 ballots
Fan Artist
William Rotsler: 24 1/4 points, 37 ballots
Brad Foster: 21 5/12 points, 32 ballots
Stu Shiffman: 19 5/6 points, 35 ballots
Alexis Gilliland: 18 3/4 points, 35 ballots
Joan Hhanke-Woods: 16 5/12 points, 28 ballots
John W. Campbell Award (Not a Hugo)
R. A. Macavoy: 113 2/3 points, 124 ballots
Joel Rosenberg: 19 points, 19 ballots
Warren Norwood: 11 5/6 points, 17 ballots
Lisa Goldstein: 10 5/6 points, 17 ballots
Joseph Delaney: 10 2/3 points, 19 ballots
This is the only difference from the actual 1984 nominees. Sherri S. Tepper with 18 votes was eliminated on points (6 19/20)
Thanks,
Patrick
Just to clarify my comment on Tepper, in the final selection phase Delaney and Tepper had the lowest point total and Tepper was eliminated for being on fewer ballots. Norwood and Goldstein ranked higher despite being on fewer ballots.
Patrick
I renumbered the ballots to fix that problem - they're not duplicate ballots, but duplicate tracking numbers.
P J Evans @301,
Ah, good to know. I don't think it affects the final numbers much. If anyone else has results I'd love to compare.
Keith "Kilo" Watt @016206.html#669: "If the community doesn't mind, I'd really like to finish up the SDV-LPE proposal before we get too embroiled in a second debate. Would it be acceptable to table the debate on this for just a bit?"
MODS: the SDV-LPE proposal is finished up; would it be possible to have a thread started for discussing potential longlist proposals? EPH is a good proposal, but a) it's not sufficient on its own to eliminate the slate problem in some categories, and b) it might not pass, so it is still worth looking at additional ways to deal with slates.
303
Why do we need a longlist?
It's probably a great idea when you're talking a dozen or so works, or a small group of nominators, but that's not where the Hugos are.
P J Evans @304: "Why do we need a longlist?"
Because even with EPH, it's still possible for a slate to get disproportionate representation in at least some categories, because the non-slate vote is so diverse; eg they could still get three or even four spots in Short Story (it would have taken at most 156 puppies to get four Short Story slots in 2013, even though 662 people nominated).
As I see it, the theoretical ideal for the Hugos would be for all voters to read every single eligible work in ever category and rank them all. Obviously that's completely impossible in practice, so we use the nomination process to produce a shortlist and only rank the top five. But the nomination process is pretty crude; each person only gives an opinion on a very small number of works, so the support for each of the finalists is measured in just tens even though there are hundreds of people nominating. Just a handful of people (or even a single person) decide that the 5th placed work gets to be a finalist and the 6th doesn't, while hundreds of people who didn't nominate either work don't get a say in the matter. A longlist lets everyone see all the works that have a lot of support and allows more people to have an input into which ones become finalists. It's an effective measure against slates and potentially produces a better set of finalists even in the absence of slates.
felice #305:
I don't like the idea of a running-total longlist being publicised before nominating deadline, so I have no interest in supporting any such proposal. Such a proposal will guarantee tactical nominating by members, which I don't think is in keeping with the spirit of the Hugos.
Fan initiatives like the Wikia and the Google doc collating eligible works are a better alternative in my opinion.
Soon Lee @306: "I don't like the idea of a running-total longlist being publicised before nominating deadline, so I have no interest in supporting any such proposal. Such a proposal will guarantee tactical nominating by members, which I don't think is in keeping with the spirit of the Hugos."
My preference is for a one-off publication of a longlist (the top 15 or so, unordered and without nomination counts) about half way through the nomination period, not a running total. This does let people change their nominations to works that have a chance of becoming finalists, but I don't see that as being any more tactical than choosing to rank a work you didn't nominate above No Award in the final vote. How is it not in keeping with the spirit of the Hugos?
It would also make the Five Percent Solution proposal unnecessary. The 5% threshold is indeed currently a problem, but it was originally put there for a reason, and increasing the share of of nominations for the finalists seems to me to be a better option than getting rid of the threshold.
"Fan initiatives like the Wikia and the Google doc collating eligible works are a better alternative in my opinion."
Those initiatives are certainly good things, but they don't help with the issue of nominations being overly spread out. There just isn't that much consensus over what's best, which lets a relatively small group of people pick the finalists just by being slightly less small than all the others. A longlist helps build consensus, by letting people see what's popular with others, and choosing the works they think are best from that list.
How would you feel about a formal three stage process? 1) run nominations exactly the same as the current system, but select 15 semifinalists instead of 5 finalists; then 2) a new stage where voters approval-vote for up to 5 of the semifinalists, to select the five finalists; then 3) the final stage is ranking those finalists, just as under the current system. Both of the first two stages could be tallied using EPH.
Obviously this would be more work for both voters and admins, but aside from that, would it be a bad thing? Contrary to the spirit of the Hugos?
Leaving aside that it would be more work for nominators & admins (two more reasons against the idea), I still don't like adding the semifinalist (one-time reveal of 15 leading contenders) stage, because that will encourage tactical voting, which is contrary to what I think is the spirit of the award: nominate works that you think worthy & let the chips fall as they may.
With this, it becomes 'vote for these 15 that are more likely to make the final ballot', not to mention having that additional information tempts fans for contenders 6-15 to campaign. I think there is already enough low-level campaigning without a rule change that encourages escalation of campaigning.
Could someone who is articulate and knowledgeable about the testing that's been done (more so than me) please go over and respond to the bullsh*t that Brian's posted on File770? Thanks.
@310 JJ
There's no point in doing so, apart from warning new readers of his reasons for commenting.
It is, however, a useful demonstration of the points that may be laid against the proposal. Forewarned is forearmed and all that.
Personally, I think he's trying to elicit statements that can be lovingly polished and stored up as possible ammunition.
So I’m paging again through all that discussion on the “Q&A thread”.Waves to Brian.
[LONGLIST] I'd like to see a list of everything that's been nominated, even if the nominator has subsequently changed their mind and withdrawn it, without any figures. Make it public after, say, the first 50 nomination ballots are in.
If nothing else it would allow the Hugo administrators to note the preferred speling.
I'm unsure if it would widen or condense the nomination pool. Get your obscure favourite in early and it may pick up nominations - in the end stages people may not see any point in nominating something new.
And a start on checking eligibility with lots of eyes.
Soon Lee @309 "that will encourage tactical voting, which is contrary to what I think is the spirit of the award: nominate works that you think worthy & let the chips fall as they may."
So why not get rid of the nomination stage entirely, and just vote for the one work you think is most worthy, and let the chips fall as they may? Isn't the finalist stage and voting for one of the popular top five instead of whatever you think is actually the best work contrary to the spirit of the Hugos? Doesn't it encourage campaigning for those five finalists? </devils' advocate>
"With this, it becomes 'vote for these 15 that are more likely to make the final ballot'"
Are you replying to #307 or #308? They're two distinct proposals. For #308, the 15 are the nominated semifinalists, there's no "more likely" about it.
JJ@310:
I went over there and read through it, and people really didn't seem to want to engage Brian in that thread -- which is probably a good call. I'm not sure there's anything to be gained by it at this point.
Brian looked at the old thread where I first ran the 1984 data, but as Nicholas and I discussed, I did no clean up on that the data for that run at all, hence the differences. When I cleaned up the data and re-ran it, I get the same results Nicholas does. Essentially, Brian is looking over a necro-post from early discussions. Nicholas replied at File770, so I think he's explained it well.
Kilo
felice #313:
My personal opinion is that EPH doesn't change the culture of nominating & voting in the Hugos.
Your ideas in #307 & #308, as well as #313 ("So why not get rid of the nomination stage entirely, and just vote for the one work you think is most worthy, and let the chips fall as they may?") represent changes to the nominating/voting culture of the Hugos which I do not support.
I don't understand why publishing a long list would help damage slates. Seems to me that what it does is advantage people who are voting tactically. I don't see that this is beneficial, since it encourages people to view the Hugo as a game to be played, rather than an honor to be bestowed.
Lydy Nickerson @316: "I don't understand why publishing a long list would help damage slates."
Slates are effective because the non-slate vote is spread out amongst so many different candidates. 150-ish bloc voters could have taken four of the finalist slots in Short Story 2013 even under EPH, because less than 40 out of 650-ish non-bloc voters nominated any single work other than "Immersion". A longlist narrows the field down from "everything eligible" to "works with a significant amount of support", and that means that the non-bloc voters' nominations are divided between far fewer works, so each one gets a lot more nominations, and so it's much harder for a voting bloc to get their slate on the final ballot. Eg in 2013 if all the finalists had gotten say an extra hundred nominations (easy with 650 people choosing up to 5 from 15, and even easier if 5 of the 15 are slate works that non-bloc voters won't nominate), a bloc of 150 would only have taken one slot instead of four. An extra 150 nominations, and the slate gets nothing.
How is selecting your top five from a longlist of 15 semifinalists more tactical or gamey than selecting your top one from a shortlist of five finalists?
Soon Lee @315: "Your ideas... represent changes to the nominating/voting culture of the Hugos which I do not support."
They represent changes to the process; I don't think that necessarily equates to changes to the culture.
felice, the point of EPH is not to prevent slates entirely, but to minimize the number of nominations they get. This has been said many times already, and it's been demonstrated.
P J Evans @318: "felice, the point of EPH is not to prevent slates entirely, but to minimize the number of nominations they get. This has been said many times already, and it's been demonstrated."
And do you really think letting them win up to 80% of the finalist slots in some categories is minimizing their impact? Reducing, yes, but not enough. They shouldn't be getting more than 20% of the finalist slots, based on their share of the total number of nomination ballots. EPH helps, but it isn't sufficient.
felice #319:
I'm a bit confused by "And do you really think letting them win up to 80% of the finalist slots in some categories is minimizing their impact?"
RP/SP getting the bulk of finalist slots is what happened this year without EPH. The worked example with 20% bloc-voters shows that with EPH in place, the bloc would have been reduced to getting one out of five finalists (a 20% representation which would be fair and commensurate with their voting strength). So in that sense, EPH is sufficient as a technical fix.
A possible problem is competing slates, which is sort of what we had this year. Competing slates would be a bad thing IMO because they more likely shut-out non-slate works, but given the opprobrium generated by slate/bloc-voting this year, I highly doubt any new group would organise slates/bloc-votes in future. The objection the slates is the cultural component of the solution that complements the technical fix EPH provides. IMO. YMMV.
So on that basis, I don't think more needs to be done at this time.
Soon Lee @320: "The worked example with 20% bloc-voters shows that with EPH in place, the bloc would have been reduced to getting one out of five finalists"
Which example and in which category? Best novel, yes, they'd probably only get one slot (maybe two) because lots of people nominate and the finalists get a relatively high level of support. But in Short Story, the second finalist had just 5.74% of non-bloc support in 2013 - add in 20% bloc voters, and that drops to 4.592%, so four slate finalists. 2014, a 20% bloc would have taken three slots in Short Story.
And that's just 20% relative to the number of ballots in each category; if it was 20% relative to Best Novel ballots applied to all categories (a more reasonable approach, since there's no reason for bloc voters not to nominate in every category), that's enough to take three finalist slots in most categories, because most other categories would be lucky to get half as many non-bloc nominations as Best Novel.
@320: Since next year must be run under the current rules, and the Sad Puppies have announced their co-ordinator, I firmly expect to see a Happy Kittens slate next year.
So 2 months before the deadline, a long list of 15 works is published (just example numbers).
A LOT of people will just decide to concentrate on those 15 instead of reading things they would have discovered otherwise - the field is huge, if those 15 made the list already, why not check them? People that nominate early are the people that have access to the new books via work or whatever means or the funds to buy them (and libraries tend to buy more known authors and publishers than the new things - at least the ones I have access to).
I really do not see what purpose will a long list at any point serve...
As for the idea of the semifinalists - I read one of the books I nominated days before the deadline. Changing this to a two step nomination system will mean that months earlier, I will have the list of semi-finalists instead. Books that are published later in the year or did not get a lot of attention will get lost in the shuffle this way.
Now - a full list of what had been nominated so far with no numbers attached to it and without ever removing a title - that may work out. :) Will be fun to see as well I guess.
felice #321:
I refer to the worked example in the Power Point & pdf linked in the EPH FAQ which uses a generated dataset matching the 2013 Hugo novel data distribution but which includes 200 slate ballots out of 1313 total ballots. It's an attempt to model bloc-voting & a pretty good example of it IMO. It shows that under FPTP (the current system) ~15% bloc-votes gets 5/5 of the finalists but under EPH, bloc-votes gets 1/5 finalists.
I agree that Short Story is the most problematic prose category, but I would rather wait and see how EPH (and the removal of the 5% threshold) work out before suggesting more rule changes. I would prefer to make small changes, and monitor outcomes, rather than make bigger changes. I think this is more in keeping with the WSFS constitution which is an accretion of gradual rule changes over the decades of its existence.
But next year will be different because:
- the opprobrium toward organised slates/bloc-votes will likely deter others from organising their own. I understand that Kate Paulk will not be doing a SP4 slate. Maybe the backlash against slates is working.
@Tim Illingworth: If there is a Happy Kittens Slate next year, I will oppose it too.
- Theodore Beale whose RP has been the more effective this year, will probably keep trying to vandalize the Hugos. But this is where EPH comes in: if the cultural solution doesn't work, the technical solution (EPH) will.
- the increased membership this year will be able to nominate next year, maybe in sufficient numbers to dilute bloc-voters.
- initiatives like the Wikia and Google doc collating Hugo eligible works to help members find their nominees.
Shorter me: let's see what happens first.
@ felice 317: Your description of how the long list works makes it sound remarkably like competing slates. That does not seem like an improvement, to me. It seems to me that it very much increases the tendency towards campaigning, and again, that is not a good thing. You assert that it will not change voter behavior, but the mechanism hinges on changing voter behavior.
You also say, "How is selecting your top five from a longlist of 15 semifinalists more tactical or gamey than selecting your top one from a shortlist of five finalists?' is so baffling to me that I am not sure how to respond except to state the obvious; stuff I think you must already know. It conflates nominating with voting, which are two quite different choice-spaces with different goals.
You also seem to be misrepresenting how one votes on the Hugos. Your description makes it sound like one votes for only one thing, a FPTP decision, when, of course, one actually ranks. Preferential voting _reduces_ tactical voting, since you can vote based on your preference without worrying overmuch about how other people are voting. If you vote for something that has little support, you aren't "throwing away" your vote, and you still have a say in the final outcome. The longlist seems to embody exactly the opposite sensibility, basing ones voting on other people's behavior rather than one's own preferences.
Soon Lee @324: "I refer to the worked example in the Power Point & pdf linked in the EPH FAQ which uses a generated dataset matching the 2013 Hugo novel data distribution"
And that does show how EPH helps, but novel is by far the least representative category, and doesn't show the limitations of EPH.
"but which includes 200 slate ballots out of 1313 total ballots... I agree that Short Story is the most problematic prose category, but I would rather wait and see how EPH (and the removal of the 5% threshold) work out before suggesting more rule changes."
We know how EPH would work out in the presence of slates. Removing the 5% threshold is irrelevant to the slate issue, and Short Story isn't the only problematic category.
For 2013, under EPH with 200 slate ballots added, the results would be:
Best Novel: at least 1 slate work, maybe 2
Best Novella: at least 2 slate works, maybe 3
Best Novelette: at least 3 slate works, maybe 4
Best Short Story: 4 slate works
Best Related Work: 3 slate works
Best Graphic Story: 3 slate works
Best Dramatic Presentation (Long Form): 1 slate work
Best Dramatic Presentation (Short Form): 2 slate works
Best Editor (Long Form): at least 2 slate works, probably 3
Best Editor (Short Form): at least 1 slate work, maybe 2
Best Professional Artist: at least 3 slate works, maybe 4
Best Semiprozine: at least 1 slate work, maybe 2
Best Fanzine: 3 slate works
Best Fancast: at least 3 slate works, maybe 4
Best Fan Writer: at least 3 slate works, maybe 4
Best Fan Artist: 4 slate works
The John W. Campbell Award: 3 slate works
The maybes and probablies are for categories where the slate almost has enough points to take the next slot as well; whether it does or not depends on how much overlap there is between the surviving non-slate works. Eg for Best Novel, if a lot of the people who nominated "Captain Vorpatril's Alliance" and "Throne of the Crescent Moon" also nominated "2312", "Blackout", or "Redshirts", that could put them both below 100 points, and mean they're eliminated by a second slate work. We don't have that data, so don't know for sure, but even the best case scenario has two categories with only one non-slate work, eight more with only two non-slate works, and only four categories with just one slate work.
Results for 2014 would be broadly similar.
Lydy Nickerson @325: "Your description of how the long list works makes it sound remarkably like competing slates."
No, it's the exact opposite of a slate. It's saying "here are the five finalists you've collectively chosen; are you really sure they're the ones you want? Or maybe you'd prefer one of these other ten works?" (it doesn't actually identify which are the finalists and which are the runnerups, but the choice is the same). The longlist is an expression of the collective will of the nominators, not an external list being pushed by any particular faction.
"It conflates nominating with voting, which are two quite different choice-spaces with different goals."
How are the goals different? A longlist is just introducing a new intermediate choice-space to produce a smoother progression from "everything" to "the best single work".
"You also seem to be misrepresenting how one votes on the Hugos."
Simplifying, not misrepresenting. Ranking the candidates does involve picking one as your first choice, and the process certainly selects one winner rather than the five from the nomination stage.
"Preferential voting _reduces_ tactical voting, since you can vote based on your preference without worrying overmuch about how other people are voting. If you vote for something that has little support, you aren't "throwing away" your vote, and you still have a say in the final outcome."
And that's exactly the problem with the current nomination process; unlike the final voting, most nominations do get thrown away, and only a small number of people have a say in the final outcome. Personally I'd have no problem with preferential ranking of nominations as well, though that would require an entirely new tallying system instead of EPH.
Annie Y @323: "A LOT of people will just decide to concentrate on those 15 instead of reading things they would have discovered otherwise - the field is huge, if those 15 made the list already, why not check them?"
Exactly; that's the point. It means more people checking out stuff that has a chance of becoming a finalist, and making meaningful nominations. The field is huge, so if everyone is reading random stuff instead of concentrating on the top 15, none of that random stuff has a chance of making the Hugo ballot (because too few people will read any one item to promote it from below 15th to above 6th in the nomination rankings).
"As for the idea of the semifinalists - I read one of the books I nominated days before the deadline. Changing this to a two step nomination system will mean that months earlier, I will have the list of semi-finalists instead. Books that are published later in the year or did not get a lot of attention will get lost in the shuffle this way."
And did the book you nominated make the final ballot? If not, how does it matter whether you nominated it or not? And if it did, do you think it was likely to have been outside the top 15 a month earlier? An extra month or two isn't generally going to make that much difference to late-in-the-year, neglected works.
"Now - a full list of what had been nominated so far with no numbers attached to it and without ever removing a title - that may work out. :) Will be fun to see as well I guess."
Fun to see, sure, but it won't do much to concentrate the nominations.
327
If we'd wanted that kind of juried list, we'd have chosen it years ago. (It's great if you're a bunch of literary critics, but it's not how fans do things: three fans, five opinions.)
What you're pushing is an extra step which lengthens the amount of time between the start of nominations and the final count (adding to the workload for the people running it) and does zero to fix the problem of slates.
felice @328, I don't want to nominate what other people love. I want to nominate what *I* love. I have a visceral dislike for the longlist system you propose because it disincentivises me from nominating my own personal preferences; the incentive is to nominate as if I were voting, for what other people like. And I really, really don't like that.
felice @328
It means more people checking out stuff that has a chance of becoming a finalist ... If not, how does it matter whether you nominated it or not?
And this shows exactly where we really differ in opinions. I do not care if the books I nominate make the final ballot or not. If enough other people like them, they will. If not - they won't. That is the whole point of the nominations process. Trying to play strategy is as bad as the slates this year - and against the unwritten agreements in my book. I nominate what I loved last year - not the best 5 from a list of 15 that some other people loved.
Once the Hugos are given, the next ones that did not make the ballot are published and you can catch up all you want. The idea of the nominations is quite different. I do not care what the people that vote early think is the list of the best (if we need a list, use the Locus one for example? Why bother ask people to nominate early at all)
I think I mentioned it before - I nominated Nina Allen's "The Race" because I loved it. I knew it cannot make the ballot but that's fine. If I was wrong and somehow it made it in? Wonderful. But when I am nominating, it is the books that made MY year. Changing that will change the whole point of the nominations cycle.
P J Evans @329: "If we'd wanted that kind of juried list, we'd have chosen it years ago."
It's not juried! It's just the top 15 nominations instead of the top 5!
"What you're pushing is an extra step which lengthens the amount of time between the start of nominations and the final count (adding to the workload for the people running it) and does zero to fix the problem of slates."
It doesn't have to lengthen the amount of time, though personally I would advocate opening nominations earlier. The only way it could fail to work against slates is if it doesn't actually change how anyone nominates, which seems unlikely.
Cassy B. @330: "I don't want to nominate what other people love. I want to nominate what *I* love. I have a visceral dislike for the longlist system you propose because it disincentivises me from nominating my own personal preferences; the incentive is to nominate as if I were voting, for what other people like."
You start off by nominating what you love just as you do now; if enough other people share your opinion, it will make the longlist (and if it doesn't make the longlist, it certainly won't make the top 5!). After the longlist is published, you can change some or all of your nominations if you want.
I thought this was a thread for Q&A about EPH, not for debating various systems? Especially since many, many different systems were discussed at length in previous threads?
felice, we've been through this already with you, more than once. No 'longlist', no three-stage process, no huhu.
Right now, anyone can nominate anything, and eligibility gets sorted out by the admins. This is how we've been doing it for the last 45 or so years, and until the juvenile canines decided to game the rules, it worked fine. EPH is intended to fix that so that slates can't easily lock out everyone else. That's all we're looking at.
felice: You appear to be falling really, really hard into the fallacy of "if I can just find the right words to explain it to them, they'll understand!"
We do understand. We understand perfectly well. We don't agree, which is an entirely different thing. And, as has been repeatedly pointed out, this was all hashed out during the process, and your suggestion was rejected. Bringing it up again and continuing to hammer on it at this late date is not going to change that.
Xopher Halftongue @333: "I thought this was a thread for Q&A about EPH, not for debating various systems? Especially since many, many different systems were discussed at length in previous threads?"
The longlist proposal is complementary to EPH, not an alternative to it. The longlist discussion was ended with "table the debate on this for just a bit", not rejection of the proposal. The mods have not yet responded on whether there should be a separate thread for it, and the deadline for proposals isn't that far off.
P J Evans @334: "felice, we've been through this already with you, more than once. No 'longlist', no three-stage process, no huhu."
That's not your call to make.
"EPH is intended to fix that so that slates can't easily lock out everyone else. That's all we're looking at."
EPH is not enough. Do you have any comment on #326?
felice, Lee said what I think. Accept that you lost that argument a month ago.
Annie Y @331 "I do not care if the books I nominate make the final ballot or not. If enough other people like them, they will. If not - they won't. That is the whole point of the nominations process."
I assume you mean you don't want to take your estimate of a work's chances into account when deciding whether or not to nominate it? Because if you actually don't care, then what would be the point of nominating at all?
"I nominate what I loved last year - not the best 5 from a list of 15 that some other people loved."
But you vote on a list of 5 that some other people loved. I'm struggling to understand the difference there.
What if the nominating process remained exactly the same - pick up to five works you love, and the finalists are announced after voting closes; but we had 15 finalists instead of 5? That's the same principle as the "4 and 6" proposal. It's a somewhat unwieldy set of finalists, though; too many to read them all, and difficult to rank, so maybe have a two-stage voting process? Start by approval voting for up to five finalists, then rank the winners of the first voting round to get the overall winner.
"(if we need a list, use the Locus one for example? Why bother ask people to nominate early at all)"
Because it's our list. If who comes up with it didn't matter, we may as well award the Hugos to the Locus winners and skip the whole voting process!
"I think I mentioned it before - I nominated Nina Allen's "The Race" because I loved it. I knew it cannot make the ballot but that's fine. If I was wrong and somehow it made it in? Wonderful. But when I am nominating, it is the books that made MY year. Changing that will change the whole point of the nominations cycle."
I don't want to change that. If there'd been a longlist this year, you should still have nominated it before the longlist was published. And maybe it gets into the top 15 (much more likely than getting into the top 5!), and a whole lot of people hear about it who'd otherwise not have know it existed. It might even get enough extra nominations to become a finalist, because of the longlist. Or if you read it between the publication of the longlist and the close of nominations, well, either enough other people had already nominated it to make the longlist, or it never had a chance of winning. But even if it has no chance, you can still nominate it if that's what you want to do; you have five spots on the ballot, after all. Unless you read five new novels between longlist and closing and they're all better than anything on the longlist, you can nominate a mix of the works you love and the best out of the longlist.
There were thousands and thousands of eligible novels published last year; over the rest of your life, you could end up reading many 2014 novels that you would have wanted to nominate if you'd read them in time.
felice @328
Annie Y @323"Now - a full list of what had been nominated so far with no numbers attached to it and without ever removing a title - that may work out. :) Will be fun to see as well I guess."
Fun to see, sure, but it won't do much to concentrate the nominations.
I thought it would concentrate the nominations and that was one of my worries. Do we want to concentrate nominations? Isn't that what the nomination tallying is for?
It would make little difference to the well/widely known works (apart from maybe standardising spelling), but it could shift nominations from the not-on-the-list-yet work of your heart to this-one-has-a-chance-but-it-will-do second best. Of course that's a reason to get work of your heart on the list so that others will join you in nominating it.
felice @305: "Two-stage nominations with longlist."
Let me say at the top that I view a longlist as no different form a slate to begin with, so your proposal is a non-starter from my viewpoint.
Next, the status quo is for fans who have nomination rights to nominate their favorite works in each category (up to five), and then vote (with rankings) on the top five (or more, if ties at 5th place exist).
This has been disrupted by block nominations which tended to drown-out other nominations.
You are suggesting to add an intermediary step of listing the top 15 'from the heart' nominations, and asking the fans who have nomination rights to now choose up to five works which they believe might have a possibility of being included in the top five (or more, if ties at 5th place exist).
That step moves the nomination process from "what I really like" to "what might reasonably win" -- essentially tactical nominations. That is what is not in keeping with Hugo traditions.
Also, looking at the data from the PowerPoint included with the FAQs for EPH, the top 15 nominations is reached after round 18 -- at which point 4 of the slate works are still in the running. The slate director/proponent/cabal then rallies the troops and nominates their remaining four, vastly improving their chances, since they now know what they are up against, and can start the propaganda machine in earnest.
Don't forget the fan with nomination rights who doesn't see any of their nominated works on the longlist and thinks, "The hell with this, I'll go have a beer and see what happens next." rather than nominating a different set of five?
@ Felice 388:
Quote:
"I nominate what I loved last year - not the best 5 from a list of 15 that some other people loved."
But you vote on a list of 5 that some other people loved. I'm struggling to understand the difference there.
----
Possibly I can explain. When nominating, what I am doing is nominating a work that knocked my socks off, something that I adore. I am not comparing it to other works directly. Instead, I am reacting to it in the context of my total experience of the genre. If I am so lucky that I have more than five works in a category that totally knock my socks off, then I might have to do some head-to-head comparison, but the usual thing is to nominate the one, two, three really amazing things, and then if there's some really nice stuff, maybe fill out the ballot, or maybe not, depending on a lot of things.
When voting, one is doing a direct comparison between the nominated works. I didn't like The Three Body Problem all that much, and I'm comparing and contrasting The Goblin Emperor with Ancillary Sword, to determine ranking. This is a very different cognitive task, and uses very different metrics. In this latter case, I'm thinking not just about my socks, but about the fact that Leckie's already won the Hugo last year, that Addison (Monette) has never won, that Sword, while wonderful, is wonderful in a very similar way to Justice, and that while I don't like The Goblin Emperor quite as much as I liked Sword it is new in a way that Sword isn't. A very, very different cognitive task from saying, "Wow, I loved the Leckie and the Addison, my socks, they are in fucking orbit, I will nominate both of these. And also, I will nominate Jo's The Just City because damn, there go my socks again." This is also why I don't like ranking nominations. Ranking requires that head-to-head comparison which is just not the same thing.
The fact that they are two different tasks, and that they use two different metrics, suggests that they will result in potentially different results. It changes the way one thinks about the works from "Is this wonderful?" to "Is this better?" For the noms, I want the first, for voting, I want the second.
328
Not just no, but hell, no.
You proposed this before, and we told you no, and why it's not going to go anywhere.
It's against the spirit of the awards, just as slate voting is. People should, at the nomination stage, put in what lit up their lives in the previous year's works, not what they think can win. The nomination stage is selecting the works that lit up the most lives.
If all it was, was about selecting works most likely to win, we could have only ten or a hundred people doing it, or choose them from Amazon sales or Goodreads rankings, like some of the canines wish. That's not what we want; it isn't asking fans, which is the point of the process as it's set up.
Another question for felice:
Seems to me that the long list allows people to develop slates after the long-list is published, and indeed encourages them to do so. Why would the long-list not lead to various people with agendas to try to get people to focus on 5 leading contenders in a category? What would prevent campaigning based on the long list to get to the short list?
Lydy Nickerson @341, In this latter case, I'm thinking not just about my socks, but about the fact that Leckie's already won the Hugo last year, that Addison (Monette) has never won, that Sword, while wonderful, is wonderful in a very similar way to Justice, and that while I don't like The Goblin Emperor quite as much as I liked Sword it is new in a way that Sword isn't. A very, very different cognitive task from saying, "Wow, I loved the Leckie and the Addison, my socks, they are in fucking orbit, I will nominate both of these. And also, I will nominate Jo's The Just City because damn, there go my socks again."
This. Exactly this. Nominating is about what made me go, WOW! Voting is about comparing what made most people (not necessarily including me, but one can always hope) go WOW!. I don't *want* to take into account what made other people go WOW! in the nomination phase. That's what the voting phase is for.
Nor, in the nomination phase, do I want to start comparing the relative orbits of my socks. (I *love* that turn of phrase; thank you, Lydy!) Again, that's what the voting phase is for. It's enough, when nominating, to say, "gosh, I need to buy more socks because the ones I'm wearing are fucking ORBITING!"
felice,
I have the feeling that you think that nominations and voting is the same action...
Nomination is: "This is what I loved. It may be one book or 5 but I am not ordering them in any order, I am just pointing them out. I do not really care (despite you thinking that this is not what I mean) how anyone else is voting or if this work has a chance in making the ballot - I am just pointing out the works I loved. What happens after that will be irrelevant to how I point out works - my first novel nomination for next year had been known since early this year. The second from a few weeks ago - and unless something else really amazes me enough to bump them, they are in. No matter what the critics and other people think of them"
Voting is: "There are 6 works (No award stands in for the best work I read last year that did not make the ballot) here that I need to order. They may not be the 5 works I loved the most but they are the 5 works I need to order. And if I think the most awesome book last year did not make the ballot, I have the No Award to use of course."
These are two very different actions. If you believe them to be the same, then I guess I see where you are coming from. Feel free to draft your long list proposal and send it to the business meeting. If it comes to a vote, I will vote against. But maybe you will find supporters.
I still do not understand why you want to concentrate nominations before the tallying. Isn't that exactly what the slates did and we are trying to find a way not to see happen again?
You won't see much traction for the 15 finalists either - that is way too many to read in each category and honestly, do we have 15 really eligible and somewhat competent entries outside of the fiction categories?
PS: And exactly what Lydy Nickerson @341 said :)
For those intending to go to the Business Meeting(s), Kevin Standlee recorded a short Youtube video about the basics of Business Meeting procedures: https://www.youtube.com/watch?v=fcQSRxE57o8
felice,
As a moderator, as a curator of conversations, I have to ask. What do you want from this conversation? Do you think that this is the venue that will give you what you want? Do you think you're going about the process in an effective way?
From where I'm sitting, this is looking more like an argument than an invitation to collaborate. It's possible that this is not the best venue for this discussion, but it's certain that you haven't taken a very good approach to the disagreement you've met.
So. What do you want? Start there, and then figure out how, where, and with whom you can get it.
abi @347: "What do you want from this conversation? Do you think that this is the venue that will give you what you want? Do you think you're going about the process in an effective way?"
I was hoping to find some people who agreed with me on the basic concept, in order to refine a formal proposal and as far as possible address the concerns of those who object to it. As I recall there was rather more support for the idea before it got put on hold (not rejected) to focus on finalising EPH, but it seems only the opposition is still around. I'm not aware of any more suitable venue, and I don't know how else to go about it.
#326 addresses why I think it's necessary; I note that nobody has replied to that.
"it's certain that you haven't taken a very good approach to the disagreement you've met."
What would have been a good approach?
felice @348,
#326 addresses why I think it's necessary; I note that nobody has replied to that.
I thought I had. More succinctly... a longlist is just another way of making a slate. I don't want to nominate according to a slate. I don't want my nominations to become part of a slate. I want everyone to nominate what they love, without looking over anyone else's shoulder, or anyone looking over mine.
Is that clearer?
Hit post too soon. <wry> Sorry 'bout that.
To continue... if slate voters make up 20% of the electorate, they DESERVE 20% of the positions on the final ballot. Math is wiggly; sometimes they'll get 40%. They might even get 60%... but they won't get 100% as in this year. I don't see how a longlist will change this, especially since slaters will be in the vote-early-and-often camp anyway and will therefore likely make up most of your hypothetical longlist anyway.
Felice: #326 addresses why I think it's necessary; I note that nobody has replied to that.
I doubt that next year the influence of slates will be as damaging as you suggest in #326, because of the reasons I list in #324.
But even if it is, I don't think your longlist idea is the solution, for the reasons others have articulated: it would change my nominating process (I like the current thought process behind nominating & don't want it to change), and it would promote gaming the system (see Duncan J Macdonald #340).
Cassy B. @349: "I thought I had. More succinctly... a longlist is just another way of making a slate."
#326 has nothing to do with the longlist proposal; it's entirely about the limitations of EPH.
"if slate voters make up 20% of the electorate, they DESERVE 20% of the positions on the final ballot. Math is wiggly; sometimes they'll get 40%. They might even get 60%... but they won't get 100% as in this year."
In 2013, a bloc of 200 would have got 60% or 80% in more than half the categories! They don't deserve that much.
felice @352, In 2013, a bloc of 200 would have got 60% or 80% in more than half the categories! They don't deserve that much.
Since we don't have the raw data for 2013, I'm not sure how you know that. And even if that's true, I don't think a longlist will fix it, for the reasons I cited.
felice @352
And how does publishing a long list solve that issue? By concentrating the vote? Well - this is what slate do. So you are advocating a slate-like approach to fight slates?
Concentrating the vote is the worst thing we can do. This is what we are trying to stop -- because this is what slates do. It does matter if the slate is published by puppies, the secret cabal or is a "here is the 15, choose from them" - it still serves as concentrating votes that would otherwise go somewhere else - so I do not find it a good idea in the nominating stage. Nor it does actually help with the slates if they show up - even with a long list, the slates can sweep the vote anyway in most categories under the current rules - if anything, they can get more powerful by dropping things that do not seem to be making it into the list and adopting something from the list in its place (in the Best Novel category for example).
As I mentioned, if you feel that strongly about it, draft the proposal. Sign it, find one more member to co-sign (sounds like noone in this conversation agrees with you but I suspect there will be someone somewhere that will?) and submit it and see how it will fare in the BM.
Cassy B. @353: "Since we don't have the raw data for 2013, I'm not sure how you know that"
We know how many nominations each of the top five works in each category got (see http://www.thehugoawards.org/content/pdf/2013HugoStatistics.pdf), which is the maximum number of points they could have. We know how many points a voting bloc of 200 gets for each surviving work. If the second-highest scoring non-slate work has less than 50 points, then four slate works become finalists (200 bloc nominations split amongst four slate finalists gives them 50 points each). If the third highest scoring non-slate work has less than 66.6 points, then three slate works become finalists (200/3 = 66.67), etc.
The raw data will only tell us if any of the non-slate finalists have lower points than nominations (the answer is almost certainly yes); a slate of 200 being able to take 60% or more of ten categories in 2013 is a best case scenario.
Does anyone else have any idea of what to do about this, if the number of slate voters remains sufficient to overwhelm EPH?
felice @355,
The raw data will only tell us if any of the non-slate finalists have lower points than nominations (the answer is almost certainly yes); a slate of 200 being able to take 60% or more of ten categories in 2013 is a best case scenario.
A quick read through the 2013 statistics shows that 200 slate voters under the current rules could have swept every category except for the first two spots in Best Dramatic Presentation, Long Form. That makes EPH a superior alternative.
Does anyone else have any idea of what to do about this, if the number of slate voters remains sufficient to overwhelm EPH?The answer that I see is to implement EPH and also encourage more SFF fans to participate. Every additional non-slate voter makes it that much harder for slates to have an impact. When you have only 400-600 voters in a category, a 200 person slate is going to be successful regardless of the voting algorithm.
Felice, vehement support of The One True Solution is all well and good, but if few/none of the people you're talking to agrees with you that your One True Solution is a One True Solution—nor yet that your One True Solution is even any solution at all—you may want to reconsider talking to that particular group of people about your One True Solution. So far, you're merely a faint echo of Brian Z, and that only because of your vehement (not to say 'tunnel-visioned', nor 'monomaniacal') contrarian support of the One True Solution which no one else thinks is a good idea; you do not display any great degree of dishonesty, let alone the unrelenting, all-pervasive dishonesty of the lyng sck f sht sometimes called 'Brian Z'.
Please, felice: Before you post anything further regarding your pet longlist proposal, do consider whether such posting will heighten your resemblance (which is, as I said, currently quite weak) to the aforementioned lyng sck f sht. Okay?
[Disemvoweled namecalling. Just nope. — Idumea Arbacoochee]
Patrick May @356: "A quick read through the 2013 statistics shows that 200 slate voters under the current rules could have swept every category except for the first two spots in Best Dramatic Presentation, Long Form. That makes EPH a superior alternative."
Yes, EPH is superior to the current system; that's not in dispute. It's a good proposal and should be passed. The problem is that it doesn't help enough.
"The answer that I see is to implement EPH and also encourage more SFF fans to participate. Every additional non-slate voter makes it that much harder for slates to have an impact. When you have only 400-600 voters in a category, a 200 person slate is going to be successful regardless of the voting algorithm."
More non-slate nominators would be good, but they only help by a fraction. For every 100 new non-slate nominators, a 20% slate only needs to recruit 25 new supporters to counter them and retain the same percentage. And that's if we're lucky and the new non-slate participants nominate in the same distribution as the existing nominators. It's possible that most of these new nominators end up nominating different things, resulting in a bigger tail for the distribution without lifting the head much (ie not substantially increasing the nomination count for the top 5 non-slate works).
Cubist @357: "Felice, vehement support of The One True Solution is all well and good"
I'm open to other solutions. But as Patrick says, voting algorithms won't do it (EPH is about as good as we can get in that area). And getting more people participating seems more like a hope than a plan to me (I do think there's likely to be something of a surge in nominations next year, but I'm not confident it will be sustained long term, and I'm not confident it won't be matched by a surge in rabid puppies bolstered by their success). And I haven't posted about the longlist at all today.
358
Yes, EPH is superior to the current system; that's not in dispute. It's a good proposal and should be passed. The problem is that it doesn't help enough.
Don't let unattainable perfection be the enemy of the good-enough-to-use, especially before it's even put into actual use.
You seem to be headed in that direction, with your insistence that slates be prevented completely by methods we don't want to use. We don't want to prevent slated works from getting nominated; we want to make sure they don't get all the nominations.
felice @348:
If there are other people who supported the longlist, it's true that I'm not seeing them in this conversation. I'm also aware that a front-page post might get their attention.
The problem is that I also see ways that a front-page post and subsequent discussion could very well turn into a flamewar, and I'm short of time to moderate such a thing just at the moment.
What would have been a good approach?
You're matching the tone of the people who disagree with you. Or worse, you're matching how their tone comes across to you; people read disagreement as an element of hostility. You (and others in this conversation) are also taking refuge in reductio ad absurdum: there are a number of statements that reduce to "if [thing I support] does not happen then [disaster]", where the disaster is either not actually possible or the least likely of a number of outcomes.
To run such a conversation—to see such an idea through—requires that you have (a) a clear and well-articulated vision of what you want, and (b) a statesmanlike approach to the inevitable disagreement. Kilo Watt has been an excellent example throughout this; see also the epic Hugo discussion on File 770.
Shorter me: don't argue. Respect, ask thoughtfully why people disagree and listen to the responses, build consensus where possible, agree to disagree where not.
P J Evans @359: "Don't let unattainable perfection be the enemy of the good-enough-to-use, especially before it's even put into actual use."
Good advice, but I'm not opposing the good-enough-to-use at all; I think EPH should be approved. I just think we should consider additional measures as well.
"You seem to be headed in that direction, with your insistence that slates be prevented completely by methods we don't want to use. We don't want to prevent slated works from getting nominated; we want to make sure they don't get all the nominations."
I don't want to prevent slates from getting any finalists either; we're in agreement there. But I don't think "not all the finalists" goes far enough; they shouldn't be getting most of the finalists either. It's ok if they get one or two of the finalist slots in a category (not ideal, but ok), but to me it's not ok if they're getting three or four of the finalist slots. And using the example from the EPH FAQ Powerpoint, of adding 200 slate ballots to the 2013 data, that happens in many categories as shown in #326 (if I made any mistakes in the math, please let me know!).
Does anyone else at least agree that the outcome shown in #326 is undesirable?
@felice: At one point in the earlier discussions, I was ready to support some form of longlist proposal, on the grounds that it was the one thing that could possibly salvage the 2016 Hugo nominations from another slate attack. The reason being that since it's not actually prohibited by the current rules, the Hugo administrators could go ahead and implement it under current rules for 2016. I no longer think that a longlist is so desirable.
For one thing, there's enough opposition to the proposal (as evidenced in replies to you in this thread) that it is now clear to me that even if the Hugo administrators technically could do that, they won't. That takes away the strongest argument in favor of it. I also notice that many people who support EPH feel that the longlist is just another form of (slightly bigger) counter-slate. Since I don't want to see Hugo voting degenerate into slate vs. counter-slate (which is one reason I support EPH), I'm wary of pushing something that looks to so many people like what I oppose.
Finally, it's no longer so clear to me that it is needed to save the 2016 Hugos. Slates have become so toxic that the Puppies may not even put up a formal slate for 2016, and may have a lot more difficulty getting people to agree to be on it if they do. And there are a whole bunch of new voters (like me) who have become aware of the importance of nominating good stuff in the shorter fiction categories, so that it will be somewhat harder for a slate to dominate so thoroughly as the Puppies did in those categories this year.
There is one thing that I plan to do as an individual nominator next year that looks something like some kind of longlist - in addition to nominating stuff that I've read naturally that orbited my socks, I'll try to read some stuff on major shortlists to look for additional sock-orbiting potential. In that way, I hope to boost at least a few stories that already have significant fan support as a defensive measure against possible slates next year. But that's just something I hope to do as an individual, and it doesn't require any formal proposal. At this time, I see no need to support a formal longlist proposal, and I think it might be counter to the goal of getting EPH passed.
felice @326: Could you provide the data-set you used to generate your "2013 plus slate" results? Given that, first, EPH eliminates one work at a time, and second, with each work eliminated from any given nomination-set, the remaining works in that nomination-set increase their point-totals, it's far from clear that the point-totals for each work at the start of an application of EPH are a reliable indicator of what the point-totals will be when there's only five works left in a category.
Cubist @363: "Could you provide the data-set you used to generate your "2013 plus slate" results?"
As described in #355. We know the point totals the slate works will have at each stage (starting at 40 for all five, ending at 200 when there's just one left), and we know the maximum possible points for the non-slate works (the same as nominations). If the most popular non-slate work has fewer than 40 nominations, then all five slate works must be finalists. If the second most popular non-slate work has fewer than 50 nominations, then at least four slate works must be finalists, etc. It's possible that the non-slate works will have lower points due to overlap (as long as there's more than one left!) but not higher points.
(This is a slight oversimplification; the slate works in the EPH FAQ PDF actually have between 200 and 196 nominations each, so the starting points technically range from 39.2 to 41.283; that doesn't make a significant difference, though.)
My view on longlists:
If people are restricted to nominate 5, then I oppose them.
If there is no such restriction, then I see how the longest kind of longlists could be useful. That is, anything with two or more votes (with automated suggested matching using an algorithm that doesn't fit in this margin). It could help jog people's memory and marginally increase the number of things people vote for (I expect it would add 10% or so on average, which is less than half a work per voter) thus increasing the overall resistance against slates.
I think that there is plenty of time to think about this idea next year, when tempers are not so ragged and trolls not so interested. (I'm not implying that every puppy is a troll, just that, to mix metaphors, there's blood in the water right now.)
365
If you can only throw a maximum of 5 names into the nominating bin for a category, you have to think about what you're going to put on your short list.
@366: If that's supposed to be an argument for limiting the number people can vote for, then it's subject to a pretty obvious reductio.
My feeling is that if some people have higher standards, and some people have lower ones, it actually helps the system find the best candidates. This is shown clearly in simulations, as long as level of standards and overall genre/taste issues aren't ridiculously tightly correlated.
So it would be great if some people vote for only the 1 or 2 very best works they've seen in a year, while others vote for everything that's even halfway good. You wouldn't want everyone to vote like the latter group, but since that's not going to happen no matter what you do, constraining the voting system to prevent it is counterproductive.
It's probably time to resolve the issues of T-shirts in the other thread.
367
You can be an admin if you want, for an award where it's 'all the names you want to put in', but that's not how the Hugos are done.
@369: Changing the rule would not change the social norms, so the number of extra nominations it would cause/allow would be minimal. The main reason to change it would be to remove the incentive for dishonesty, where if there are more than 5 things you'd like to nominate, you would strategically choose the more-viable ones even if they're not actually your top 5 choices.
But again, I'm not actually proposing this for this year, when the BM is likely to go into overtime anyway. This is more a minor fix that I'd support in an off year, not something worth worrying about in the current situation.
Jameson Quinn @ 370
I would disagree. Now I need to decide which were the 5 works that really made the biggest impact on me last year. Allow me to nominate more, I will be listing all books I liked last year. It changes the dynamics of the nomination process. Not for everyone - not everyone nominates 5. But for the people that read a lot in some categories. May not make a difference statistically but it will make difference to the process - and that one vote somewhere may change things (some of the differences are that small).
Increasing the number of nomination for something like the 4/6 proposal is fine - it still asks you to nominate what you think is the best. Making it unlimited or changing the number to a very high one will change things. And it will not solve any issue - perceived or not. So why break what is not broken?
371
People keep thinking that fixing the problem with slates should mean changing the whole way it's done, even after we've said that's not at all what we want.
If they can't understand 'no' when it's that loud, why should we bother trying to explain it to them again?
(Cranky. Third week of shingles, with limited sleep because of the pain and no comfortable position possible. Those who have had chickenpox vaccines, be thankful, because you [probably] won't have to deal with shingles.)
@372
Had a few minutes, decided to try to respond again in between writing documents :)
Get better, P J Evans!
P J Evans @372, sympathies. Hope it clears up soon.
374
It's the third week; I certainly hope so. The visible marks are healing, but the !@#$%^&*() neuralgia is going to take a while longer.
Ok, one last attempt at a longlist variant. I think this preserves the integrity of the current nomination system, while minimising the extra work for both voters and admins. See #326 for why I think it's necessary. And if you don't think it's any improvement, there's really no need to reply; this isn't going to go anywhere unless people like it. This is a three stage system rather than a change to nominations:
1) Nominations: Exactly as now, nominate up to five works you think are Hugo-worthy in each category.
2) Longlist Voting: The top 15 from the previous stage are published, and voters approval-vote them (ie tick works that you'd rank above No Award - this could be limited to at most your top 5, or it could be unlimited; the latter would be stronger against slates). Anything you nominated that makes the longlist would be ticked automatically - if nobody participates in the longlist stage, then the outcome would be exactly the same as if there had been no longlist stage. But if you want to, you can approval-vote additional works that you didn't nominate. This could be a short stage; say just one week, since participation is optional and it's not intended as a reading list.
3) Final Voting: Exactly as now, the top 5 from the previous stage are published and voters rank them.
If EPH passes, it would be used to tally both Nominations and Longlist Votes.
The advantages of this system would include crowdsourcing of ineligibility issues and giving potential finalists the opportunity to decline early, reducing the chances of the admins having to make changes to the finalists after they're announced. There's no need to compromise on nominations; nominate what you love, and it either makes the longlist or not. But if none of your nominations make the longlist, then you can still have a chance to affect the finalists by approving whatever you like best out of other people's favourites, if you've read and liked any of them. There shouldn't be much of a problem with campaigning, because the longlist stage is too short. If a slate makes the longlist, the other ten or so non-slate works on the longlist will probably get a lot more extra approvals, meaning fewer slate finalists (eg for the 200 voter slate in 2013 example, just a dozen extra approvals for "Mono no Aware" would result in it beating the 4th slate work in Short Story; 33 extra approvals each for "Mono no Aware" and "Mantis Wives" would mean three non-slate finalists).
The main disadvantage of this over the previous longlist proposal is that there's not really any time to check out unfamiliar works from the longlist; approvals will mostly be limited to works you've already read but didn't nominate.
Felice, what would stop Puppies and other slaters from upticking all their slate nominees? Indeed, if they're automagically ticked up for each nominator, that'll happen anyway. Not seeing an advantage here; just more work for the admins. Maybe I'm not understanding what you mean.
Honestly, the various Wikis and Google Groups set up to show what's eligible for nominations seems like it does this job better, with no slating, and no additional work for the Hugo administrators (since they have nothing to do with it). If everything everyone loved is listed, it's the longest of longlists, and not a slate...
378
If everything everyone likes is listed, you might as well just collate the lists of forthcoming books in Locus.
Which is another reason to not have a long list: felice, have you seen the lists of forthcoming books? They go for pages, just listed by publisher.
I think that this debate is not going anywhere. Even if a longlist or lift-the-limit proposal is worth doing (which I happen to believe), there's no harm at all in waiting a year until we at least know whether EPH has passed. At that point, both sides can make their points on clear ground without having to pose multiple hypotheticals or risking giving ammunition to the anti-EPH concern trolls.
(That is, as long as EPH passes or fails with a clear enough margin this year so that the revote next year is pretty clearly foreordained. If not, we should wait two years to talk about longlists and such.)
I'll let the anti-longlist folks have the last word now if they want, and felice, I think you should do so too.
I think Jameson@380 has some wisdom here. I've been following the discussion, and I'm not sure how I feel about longlist, to be honest. The biggest concern I have apart from voting patterns is the extra work for the admins, but since I don't have to actually do the work, I suppose that wouldn't be my call to make. But, as Jameson said, it seems to me that it would be much easier to think about the pros and cons once we see what happens with EPH. I think at that point it'd be much less hypothetical.
Kilo
Cassy B. @377 "Felice, what would stop Puppies and other slaters from upticking all their slate nominees? Indeed, if they're automagically ticked up for each nominator, that'll happen anyway. Not seeing an advantage here"
The puppy nominations will probably get exactly the same number of approvals as they did nominations, because all the puppies will have already nominated everything on their slate, and nobody else will want to add their support at the longlist stage. But the non-slate works will get extra approvals, improving their position relative to the slate works. As well as approvals from everyone who nominated them, they'll get approvals from people who didn't nominate them but think they're better than the slate works. Eg "I nominated W, X, Y, and Z, and none of them made the longlist; that's a pity. But A and C are good stories; maybe not my first choices, but certainly Hugo-worthy; I'll approve them." or "Yay, my nominee F made the longlist! And A, of course; I didn't nominate that because I knew it was so popular and didn't need my support, but I may as well approve it. Oh, and D and G too - I would have nominated those if I'd been allowed seven nominations!"
Most of the top 10 non-slate works will have been nominated by well under 10% of nominators, which leaves more than 90% who could add their approvals at the longlist stage. Plus voters who didn't nominate at all could join in at the longlist stage. The approvals for most non-slate longlisted works could easily be double their nominations, even with 80-90% of voters still not approving them.
Does that make sense?
Felice, ok, I think I see your point, but you're still generating a LOT of work for Hugo administrators, and I don't actually *want* to know that that perfect gem that I loved and nominated has no chance at all. And in some ways, I think it might work against those perfect gems; people will nominate strategically and ignore those gems that don't show up on the longlist. I think this strategy will concentrate votes on the fifteen early-popular books. Nor should they.
That doesn't necessarily sound bad at first blush, but over at file 770 there's a fun little "which book wins" contest going on, and the early leaders do not always win their brackets.
Odd, I checked preview, but my post is still munged. The last line should be:
That doesn't necessarily sound bad at first blush, but over at file 770 there's a fun little "which book wins" contest going on, and the early leaders do not always win their brackets. Nor should they.
Felice@382:
Just to be clear, you are proposing to close nominations before the longlist is released correct?
Kilo
Keith "Kilo" Watt @385: "Just to be clear, you are proposing to close nominations before the longlist is released correct?"
Yes, that's correct for my new proposal in #376. So there's no strategic nomination to worry about, and perfect gems aren't disadvantaged.
Do you agree that my analysis of extending your Powerpoint example to categories other than Best Novel in #326 is correct? Or did I miss something?
Cassy B. @383: "ok, I think I see your point, but you're still generating a LOT of work for Hugo administrators"
There's some one-off work to add the new stage to the voting code. An extra round of publicising each year ("the longlist is out now! Go look at it and add some extra approvals if you want"). Anything else? The nomination data needs to be cleaned before tallying, but that's the same irrespective of whether the next stage is the longlist or it goes straight to the shortlist as now. The longlist works don't need to be checked for eligibility or nominee approval, and making them public could save the admins work through earlier withdrawal of ineligible or declined works.
I do think the longlist discussion has been confused by the fact there are two different types that have been discussed. One is the early release of nomination data, to allow people to possibly edit their nomination ballots before nominations close. The other is an intermediate voting stage added after nominations have closed, to reduce the top 15 from the nominations stage down to 5 for the final ballot. That is what felice is talking about at the moment, I believe, but some of the arguments against it seem to be for the early release type, so not actually on point.
I do agree with felice that the intermediate additional voting stage would help to further reduce the influence of slates, but I am not sure it would be worth the extra trouble, both to the admins and to voters. I think Jameson is right that we should wait on this proposal to see if it is actually needed.
I can see some advantages to an intermediate approval-voting stage, but I fear it would also risk blandifying the ballot, and make it harder for lesser-known but worthy works to make it on the ballot. For example, suppose Dr. Who snagged three of the top five slots, and eight of the top fifteen: I would be completely unsurprised if the eight Dr. Who episodes crushed everything else in the approval voting stage. Or if a small but passionate group of fans gets something like Digger, which most of the nominators haven't read, it could get knocked back into oblivion by works which aren't as good but which more people have already read. On the whole, I think the longlist proposal is bad even if you don't find its similarities to slate voting objectionable.
Steven desJardins @388: "I can see some advantages to an intermediate approval-voting stage, but I fear it would also risk blandifying the ballot, and make it harder for lesser-known but worthy works to make it on the ballot."
That might possibly an issue with a brief longlist stage, yes. A longlist could also have the opposite effect, though, bringing lesser-known works to more people's attention and increasing their chances of making the top 5. The longer the longlist stage, the more opportunity people have to check out unfamiliar works. My latest proposal is based on minimising the changes to the existing system; an alternative version where the nomination period is say 1-31 January, and the Longlist is up for most of Feb and March, would I think be excellent for promoting quality lesser-known works to the final ballot, but disadvantage works published late in the year because fewer people will have the chance to read them before the nominations deadline. It's a balancing act. Does a given work have a better chance of getting enough support to make the top 15 by the end of Jan, or enough to make the top 5 by mid March?
And note that if all the Doctor Who episodes are approved by the same people, it's a de facto slate, so EPH makes it harder for them to get multiple finalists. And in both 2013 and 2014, there were six DW works in the top 15 for BDP:SF, and the gaps between most popular DW and least popular DW were pretty huge; that suggests that most people who nominate the best episodes of DW aren't motivated to also nominate episodes they didn't like as much. Also, a longlist gives "anything but DW" voters some options to choose from :)
389
We have lists like that: recommended book lists, like the ones that Locus publishes every year. (Why re-invent the wheel?)
My apologies for commenting so late, but I had been deliberately avoiding reading discussions of Hugo voting until after ballots were finalized.
I will first comment that I can appreciate how much work has been put into this proposal.
If the intention is to limit the power of organized voting, and to insure that the largest number of nominators have an influence on nomination, then 3.A.1(3) has been designed incorrectly.
3.A.1(3) compares the two lowest point total options, and then eliminates the one on the fewer number of nominating ballots. (For simplicity, this will be identified as Work P, with Work Q referring to the one with the larger total.)
Case #1: If Work P has points greater than or equal to Work Q's total, ballots belonging to supporters of Work P necessarily have a smaller average number of remaining works in consideration.
Under most statistical assumptions, this will include a larger number of nominators for whom P is their only remaining nominated work, as opposed to Q. Thus, in this case, 3.A.1(3) will typically eliminate the voice of more nominators than a simple point-based comparison.
Case #2: If Work P has points less than Work Q's, a point-based comparison achieves identical results to 3.A.1(3) as written.
Accordingly, 3.A.1(3) produces a narrower nomination process than simple elimination of the lower point total, introducing a bias that actually favors organized slate voters over haphazard individuals.
Similarly, a reversal of 3.A.1(3) would favor individuals whose nominees have been mostly eliminated, over slate voters who still have all their nominees still in the race.
In general, using multiple criteria within the same elimination process (here, point total and nominating ballots) is a recipe for an unintended mess, and my general recommendation as an expert on voting systems would be not to use one except as a tiebreaker. If you are using point totals to select two works for consideration of elimination, you will introduce fewer complications by simply selecting the lowest point value work for elimination.
However, if you want to use an alternate elimination method after selection of two "weak" candidates via point totals, there are other options in that step that would actually function to limit, instead of functioning to preserve, the power of bloc voting. Two particular examples:
(3-K) Resolve the elimination contest by eliminating the option whose voters have, on average, fewer surviving alternatives. (Which is not quite the same thing as being on fewer nominating ballots.)
(3-L) Minimize eliminated 1-voters who become 0-voters. If tied, minimize eliminated 2-voters who become 1-voters, et cetera. (This is more complicated in book-keeping terms, but takes direct steps to locally maximize the pool of nominators whose nominations are continuing to affect the process.)
Actually, the mess is intended. I think you're making a crucial misinterpretation, by assuming that the point total is meant to be the primary selection criterion. Actually, the total number of nominations is meant to be the most important factor, just as it is now, and only in rare circumstances (mainly, the presence of slate voting, or slate-like concentrations of votes, such as a large minority nominating five episodes of the same TV series) does the point comparison change that outcome, by in effect saying, "Hey, this set of nominators has too much overlap, one of their choices has to go."
#392 - really, that last sentence should be added prominently at the top of the Commentary.
> Actually, the mess is intended. I think you're making a crucial misinterpretation, by assuming that the point total is meant to be the primary selection criterion.
The net effect of 3.A.1(2-3) is to make the point total the primary selection criterion within the iterative process, and to make the appearance on number of nomination ballots a secondary selection criterion within the iterative process. This does not go quite as far towards proportionality as making the iterative point total the only selection criterion.
My understanding of the overall intention of this proposal is given by:
(A) "No group — whoever that group may be — should be able to absolutely prevent nominees from having the chance to be considered for the Hugo Award."
(B) Statements to the effect that a "slate" of nominators consisting of approximately 20% of active nominators should obtain roughly 20% of nominations.
The evaluation offered above was offered in the light of (A-B) and the actual mathematical properties of the proposed algorithm.
As I said, I appreciate the effort collectively put into this process. Speaking as an expert on the subject (mathematics of voting systems), it is not unusual for reform efforts to make problems worse; and the proposal as submitted does, in fact, provide a net improvement of Hugo nomination policy w.r.t. (A-B).
394
Did you look at the pdf/powerpoint files that Kilo posted? Because they make it clear that points are not the primary method.
Hi Tomas -
As others have said, the intent is to keep number of nominations as the primary cause of elimination, exactly as it is currently. We really want to change as little of the Hugo nomination process as possible. Point total determines which works are eligible for elimination, but they do not cause elimination themselves.
All that said, it's always possible that real world experience will make us want to tweak EPH. The deadline for submitting proposals is upon us, however, so we aren't going to be able to even consider changes at this point.
Thanks for the input!
Kilo
I (suspicious and uncharitable soul that I am) can't help but wonder if our new friend Tomas might be an old 'friend' posting under a new nym. Can Abi (or anyone else with access to IP addresses and suchlike) dig up data to confirm or deny this notion?
Cubist @397:
Here is a statement of principle:
I regularly check IP addresses of new commenters when I have my suspicions aroused. What I do with the information, however, may not include disclosing it to the wider world.
Beyond that, I have no comment to make.
Well, one more thing: if you comment here, I expect you to take people in good faith unless or until proven otherwise. And even when you feel that they are dealing in bad faith, please don't indulge in namecalling again (you'll note that one of your previous comments is now bereft of a few vowels).
> Did you look at the pdf/powerpoint files that Kilo posted? Because they make it clear that points are not the primary method.
>As others have said, the intent is to keep number of nominations as the primary cause of elimination, exactly as it is currently. We really want to change as little of the Hugo nomination process as possible. Point total determines which works are eligible for elimination, but they do not cause elimination themselves.
I have looked now. It contains a number of statements of intention, which do not change the mathematical properties of the system.
It is unfortunately the case that intentions often do not reflect reality when it comes to the actual behavior of voting systems. (Weighted voting systems provide some particularly striking examples, e.g., see Banzhaf's famous Nassau County case.) As I said, you have done very well overall, considering how these things often end up going.
Reading the presentation *does* suggest that 3.A.1(3) as written helps make the proposal more palatable to some who would otherwise oppose it. The proposal as written is considerably more effective than the lack of any policy change, and I wish you the best of luck with it.
Certainly, if the deadline has passed, you have no options other than to go forward as written for the current business meeting, leaving any further reform for later (at which point the impetus for reform may have passed).
Regarding comments on my identity & expertise: My e-mail address makes it trivially easy to pull up my full name, which in turn is unique enough to make it trivially easy to find things like evidence of the existence of my dissertation on the mathematics of voting systems.
I do not think I will be following the comments on this thread further, but I can be contacted via that e-mail address if you want my input, expert testimony on the record, etc for something related to how a particular voting system works, whether in the near future or a year or two down the road.
Please order your E Pluribus Hugo t-shirts ASAP using this link. They cost $8.04 per shirt, if you're there at the convention to pick them up. If not, please still order now, and we'll figure out what shipping costs later (probably a reasonable amount; but it certainly won't be free, since we have no profit margin on the shirts themselves).
In the notes on your order please tell us:
-Your name and email.
-If you will be picking the shirt up.
-If so, which day you'd like to get it.
-If not, your mailing address. We'll contact you by email to work out shipping payment.
[shirts]
Ordered shirts, forgot to put in the shipping information. I'll email you, as it's probably easier than cancelling and reordering. [braining not good]
Ordered my shirt. BTW, that's a fine choice of blank -- we use the Gildan Cotton in Metro Blue for a couple of our shirts. See you at the con!
I finally got some free time to run some tests on EPH vs Slates. For those not wanting to click, the results from the 1984 data in the Novel category are:
With 43 slate ballots (10% of the number cast) added, the result would have been identical to the actual 1984 result.With 85 slate ballots (20% of the number cast) added, one slate work would make the list, bumping off “Millennium”. This is quite different from the current rules where only “Startide Rising” would remain out of non-slate works.
With 128 slate ballots (30% of the number cast) added, two slate works would make the list, bumping off “Millennium” and “Tea with the Black Dragon”. Again this is quite different from the current rules where the only non-slate work remaining would be “Startide Rising”.
Even with 170 slate ballots (40% of the number cast) added, both “Startide Rising” and “The Robots of Dawn” would remain on the nomination list under the EPH rules. Under the current rules, slate works would sweep the category.
In the Short Story category:
With 28 slate ballots (10% of the number cast) added, the result would have been identical to the actual 1984 result.With 57 slate ballots (20% of the number cast) added, two slate works would make the list, bumping off “The Peacemaker” and “Servant of the People”. This is very different from the current rules where less than 15% of the voters could sweep the category.
Even with 113 slate ballots (40% of the number cast) added, “Speech Sounds” and “The Geometry of Narrative” would remain on the nomination list.
Patrick, may we crosspost this to file770? There's a person over there claiming EPH would only knock one puppy nominee off the short fiction ballot.
It would be nice to be able to show actual numbers to refute him.
Cassy B @404,
Please feel free to share it wherever you like. What's the topic at File770?
Patrick May @405, a fellow named Brian Z; all over file770, but most recently at The 2015 Hugo Awards - A Perspective
He says he won't vote for EPH because it won't eliminate ALL slate nominees. Instead, he explicitly prefers to make no change at all (thus guaranteeing that any future hypothetical slate will get all the nominees. Again.) He's the classic case of "the perfect is the enemy of the good". And he's fixated on felice's assertion that in short story categories slates will get four out of five nominees. Your evidence seems to suggest that felice was in error...
Cassy B. @406,
'He's the classic case of "the perfect is the enemy of the good"'
He's going to hate Arrow's Impossibility Theorem, then.
406
Brian Z is fixed on 'anything that will allow the puppies to win', I think.
Cassy B: Your friend and his, Brian Z, is known to Making Light of old. He's left his Puppy-propagandizing bullshit spoor all over the recent Hugo-related threads hereabouts; a site-specific google search of Making Light will reveal more of Brian Z than you ever wanted to know.
Brian is welcome to come here and discuss these matters himself, but he will have to clarify some previously unclear comments first. My comment at 258 is relevant in this context; he has made some sketchy attempts at answering elseweb, but I need complete answers here.
I'm not usually this strict, but Brian has used up his credit on Making Light. Now his information has to be linked and quoted, up front, cash on the nail.
410
He doesn't like to answer questions elsewhere, either, as I'm sure you've noticed.
Patrick May @403: "I finally got some free time to run some tests on EPH vs Slates. For those not wanting to click, the results from the 1984 data in the Novel category are:
With 43 slate ballots (10% of the number cast) added, the result would have been identical to the actual 1984 result.
In the Short Story category:
With 28 slate ballots (10% of the number cast) added, the result would have been identical to the actual 1984 result."
It would be more realistic to stick with 43/85/128/170 slate ballots in Short Story too. If people are nominating works according to a slate, they've got no reason to abstain from the Short Story category, unlike genuine voters who just don't all read short fiction.
Cassy B. @406: "he's fixated on felice's assertion that in short story categories slates will get four out of five nominees. Your evidence seems to suggest that felice was in error..."
I'm talking about specifically a slate bloc of 200 in 2013; other years and other bloc sizes will give different outcomes. The participation rate for Short Story nominating in 2013 is down slightly compared to 1984, and crucially, the range of stories nominated is much wider in 2013. in 1984, the 2nd and 3rd ranked works got around 33 nominations each from 280 ballots; in 2013, the 2nd and 3rd ranked works got just 34 and 38 nominations respectively, from 662 ballots. So proportionately, a slate in 1984 would be up against much tougher competition than one in 2013.
EPH is obviously still a huge improvement on the current system, though.
felice,
EPH is obviously still a huge improvement on the current system, though.
This is important to emphasize, going into Sasquan. I can't make it to the convention, but I hope there are enough supporters to get EPH approved. That will give us a year to test more scenarios before final approval next year.
abi #410,
Just followed a link back here.
You pursued me "elseweb" to demand that I engage with you, yet didn't want to continue the conversation when I replied to your questions. To call that a "sketchy attempt at answering" is self-serving.
I see you are a generally nice person, and I'm sure I'd enjoy chatting with you in real life, but if a blog moderator insults me and allows others to insult me and threatens to "disemvowel" if I don't "redeem my humanity" by meeting a list of demands, I'm not going to be hankering for more of that. I have no objection to your moderation style - your house, your rules, and if one is a spectator it could be amusing to watch the drama. By the same token, I'm not at your beck and call. I think some of the behavior here illustrates how this discussion might have been more productive if it had taken place on neutral ground, but that's just my personal perspective.
Your proposal has a chance of passing. BTW, Mike Glyer just commented that he is leaning towards voting for it, though he didn't say that with any noticeable enthusiasm. In my view, it has not been shown to work as advertised in the smaller categories, and it seems to have likely unintended consequences. If you must act out of conviction that something should be done immediately to fight the slate menace, it would be logical to pass all the other Hugo-related proposals, since potential problems with EPH may emerge over the coming year. (See my Hugo prediction # 10. My predictions have never been wrong before. Then again, I've never made any before.)
Anyway, no hard feelings about the discussion here, and best of luck at the business meeting.
abi #410,
Just followed a link back here.
You pursued me "elseweb" to demand that I engage with you, yet didn't want to continue the conversation when I replied to your questions. To call that a "sketchy attempt at answering" is self-serving.
I see you are a generally nice person, and I'm sure I'd enjoy chatting with you in real life, but if a blog moderator insults me and allows others to insult me and threatens to "disemvowel" if I don't "redeem my humanity" by meeting a list of demands, I'm not going to be hankering for more of that. I have no objection to your moderation style - your house, your rules, and if one is a spectator it could be amusing to watch the drama. By the same token, I'm not at your beck and call. I think some of the behavior here illustrates how this discussion might have been more productive if it had taken place on neutral ground, but that's just my personal perspective.
Your proposal has a chance of passing. BTW, Mike Glyer just commented that he is leaning towards voting for it, though he didn't say that with any noticeable enthusiasm. In my view, it has not been shown to work as advertised in the smaller categories, and it seems to have likely unintended consequences. If you must act out of conviction that something should be done immediately to fight the slate menace, it would be logical to pass all the other Hugo-related proposals, since potential problems with EPH may emerge over the coming year. (See my Hugo prediction # 10. My predictions have never been wrong before. Then again, I've never made any before.)
Anyway, no hard feelings about the discussion here, and best of luck at the business meeting.
#414:
Brian Z.:"Just followed a link back here."
Given that Brian Z grabbed a comment from this very thread on July 23 to post on File770, colour me unconvinced by his lie.
He's been not answering questions there to such an extent that we now have this at File770:
[RUBBER STAMP BRIAN Z RESPONSE]
Hey Brian, are you going to to reply to Oneiros’ question or are you going to keep hand-wringing and trying to spread FUD (Fear, Uncertainty and Doubt)?
[/RUBBER STAMP BRIAN Z RESPONSE]
Brian Z.,
"In my view, it has not been shown to work as advertised in the smaller categories, and it seems to have likely unintended consequences."
This is the opposite of what has been demonstrated to be true (see here http://patrickmay.tumblr.com/post/126023308724/e-pluribus-hugo-vs-slates and follow the links for more details). You know this from our discussions at File770.
Please support your claim or retract it.
Patrick,
I could also ask you to retract your claim that I support disenfranchising Hugo voters, simply because I think that the perhaps 70-80 percent of WSFS members who voted in the Hugos this year yet are denied a vote at the business meeting should be consulted and given at least some kind of voice in the process of amending the rules.
Even your own data supports my conclusions, as I already explained. You did not answer me - preferring to repeat the assertion that I am de facto against democracy.
But if you would like to continue our discussion at File 770 where you left off, I'm happy to do so.
Brian Z @414:
You pursued me "elseweb" to demand that I engage with you, yet didn't want to continue the conversation when I replied to your questions. To call that a "sketchy attempt at answering" is self-serving.
It's not "pursuing" you to comment on File770; I have done so many times without it being about you. I think you have an exaggerated sense of how much I care about you. Might want to see to that.
The comment there was relevant because we were discussing me writing something that would logically go on Making Light. But your answers on File770 were sketchy. To refresh everyone's memory, I asked the following questions up at comment 258. (I'll interleave your answers for everyone's convenience):
[AS] Citations and links to Sam Harris and Brother Guy's views on EPH, please.
[BZ] I thought of the names Sam Harris and Brother Guy as people who might have thought about perspectives on utilitarianism.
commentary: Missing any actual data on what they do think about either EPH or utilitarianism. Missing any link between EPH and utilitarianism.{*}
[AS] Furthermore, please explain how EPH is a "needs of the many" solution. Be explicit and clear. Use citations and details.
[BZ] EPH might choose a nominee with fewer nominations over one with more more nominations by selectively eliminating nominations made by certain WSFS members (even though they had more votes), reducing the voting power of those members.
commentary: Missing any examples that would support this contention.{*} Misleading allegation that elimination is based on the identity of the member making the nomination.
[AS] And please unpack what you mean about "liberal arts" majors. I am interested to hear what, precisely, you're getting at with that comment.
[BZ] (no response)
commentary: Question stands.{*}
Additional BZ comment: I support democracy.
commentary: No explanation of how to reconcile this with the negative statements about "wisdom of the crowds" and "the needs of the many".{*}
I see you are a generally nice person, and I'm sure I'd enjoy chatting with you in real life,
Very kind.
but if a blog moderator insults me
Citations and insults, please.{*} I try very hard to focus on what people do rather than who they are in my comments.
and allows others to insult me
It's always a fine line between commentary on behavior and insults, and a judgement call where to draw the line. I have disemvoweled someone calling you names here (at 357).
I do note that you comment quite happily on File770, where many more and worse things have been said about you than anything here. That reduces my concern that this is the real issue.
and threatens to "disemvowel"
Those scare quotes are unneccessary. Disemvoweling is the accurate term for what I will do to your previous comment if you don't do a better job of answering my questions.
if I don't "redeem my humanity"
I don't even know what this means. I said in comment 261 that I I'm a firm believer in the redeemability of humanity.. You're human. That means you're capable of redeeming yourself.
Do it or do it not not. You're human either way. Any other reading of my comment is deliberately obtuse. (Get off the cross. We need the wood for bridges.)
by meeting a list of demands, I'm not going to be hankering for more of that.
And yet here you are.
I have no objection to your moderation style -
Did I hallucinate the preceding paragraphs?
your house, your rules, and if one is a spectator it could be amusing to watch the drama. By the same token, I'm not at your beck and call.
True. You're free to never come back to Making Light; I have no power to summon you (nor desire for such a power). But if you do, then yes, you do have to play by the rules of the site.
I think some of the behavior here illustrates how this discussion might have been more productive if it had taken place on neutral ground, but that's just my personal perspective.
Links, comment numbers, clear and detailed explanations of which, please.{*}
Your proposal has a chance of passing. BTW, Mike Glyer just commented that he is leaning towards voting for it, though he didn't say that with any noticeable enthusiasm. In my view, it has not been shown to work as advertised in the smaller categories, and it seems to have likely unintended consequences. If you must act out of conviction that something should be done immediately to fight the slate menace, it would be logical to pass all the other Hugo-related proposals, since potential problems with EPH may emerge over the coming year. (See my Hugo prediction # 10. My predictions have never been wrong before. Then again, I've never made any before.)
You are of course entitled to your opinions; you can even continue to state them here if you address all of the comments I have marked with a {*}.
Anyway, no hard feelings about the discussion here, and best of luck at the business meeting.
None whatever! I think you have expanded many people's understandings of the variety of commenting styles that occur on these here internets. I confess that you haven't brought anything to the party that I haven't seen before, but it's been an educational package of rhetorical behaviors nonetheless.
Brian Z @418:
Where did Patrick May say this? {*}
(Hint: if it was on File770 or elseweb, tackle it there. Unlike his comment to you, which was based on what you said here. But even that, do not pursue till you've answered me.)
abi, I don't think we have much of a quarrel, nor should we. I read your #410 and was moved to respond, but that doesn't mean I want to get in the weeds arguing about EPH either. My purpose was to set my reasons for not wanting to discuss it further here. You don't have to agree with them, of course.
I'm sorry if the thing about liberal arts majors offended you - I thought your resident statisticians could be a bit rough on critics who didn't have the same math background, but in hindsight I might have phrased that better.
I do think we've become too wrapped up in selfish interests, and that the important goals are on the social side - to recapture the spirit of coming together to collectively honor our authors. But I've said that before.
So I didn't come here to re-ignite the great EPH debate of summer 2015. I just wanted to explain my perspective and say no hard feelings. It was nice stopping by and I appreciate your response.
ETA: abi, yes, I agree - if Patrick May would like to discuss further that we might continue talking on File 770.
I hope to maybe drop in again someday for a less controversial chat on books or filking. Best of luck.
416
Also over at File770, Aan created a stylish script so we can ignore him (and some of the other trolls) more easily.
Brian Z.,
I could also ask you to retract your claim that I support disenfranchising Hugo voters, simply because I think that the perhaps 70-80 percent of WSFS members who voted in the Hugos this year yet are denied a vote at the business meeting should be consulted and given at least some kind of voice in the process of amending the rules.
Please point out where I made that claim or retract your accusation.
Also, please address your demonstrably incorrect claim that you attempted to distract from with this bit of nonsense.
Finally, if you are genuinely interested in continuing a conversation, answer the simple yes or no question that I asked you repeatedly at File770. Until you do so, I have no reason to treat you as an honest interlocutor.
For those who don't follow File770, Brian Z seems to think that supporting members are somehow cheated out of the right to vote at the Business Meeting. Despite the demonstrable fact that they're not paying for that right, unlike attending members, who DO have that right, because they pay for it. I honestly can't understand his argument, here.
425
It's just another way to try to discourage people from supporting EPH and not the juvenile canids he like so much.
I think there's enough information here for people not yet acquainted with Brian to come to a balanced assessment of his character. I'm not convinced we need to dissect his assertions in any further detail.
Abi, @427, that's more than fair. I was just trying to provide context for his disenfranchisement comment. I'll cease the third-party comments forthwith.
For those not following the business meeting, from File770:
'Not only did the Resolution to request the 2015 nomination data pass, it passed as a request for the data from both 2015 and 2016. Members of the Sasquan Hugo Administration Committee spoke and said that they had already decided (with some reservations) to release this data if the resolution passed, and that this data will now be available on Sunday morning after the Hugos Saturday night.'
(4/6 is apparently having a hard time.)
4/6 has become __/__ with blanks to be filled in at the Business Meeting.
Looks like ASCII art of a windshield wiper.
I have written an independent Perl implementation of the SDV-LPE method described in the E Pluribus Hugo proposal. The code is available under the Perl license. So far, the results I'm getting with it are identical to what I'm getting when I run the Visual Basic program that Kilo kindly provided after today's business meeting.
Galen Charlton #434:
Thank you. That's good to know.
So who's working on the COBOL implementation?
[I was wondering about doing it in ADABAS/Natural, just because there should be a mainframe implementation.]
436
The version that was used in 1971 and 1972 was in full PL/1.
Brian Z @now unpublished
Nope. Just nope. Not even just as a pointer to elseweb.
These constant, gradual violations of the boundaries I have set stop now. You're here as an honest, honorable participant in the conversation, which means cleaning up the mess you've left, or you're not here at all.
OK, I want to see versions in APL, Forth, and Prolog.
Brian Z @unpublished:
No, every single instance of {*}. Either answering fully, substantively and accurately, with links and evidence that conform to what you're saying, or explaining why you said that thing in a clear and upright fashion. You're far too good with words to convince me you can't parse this request.
You've made a mess of the conversation. Clean it up if you want to post even one more thing on Making Light and have it stay.
Abi #438: Thank you. Pity we still have Brad inking up the Open Thread about write-ins.
Brad is dealing honestly with the conversation; there is no comparison to be made between the two of them.
Don't like the topic, scroll on by. There's a book discussion going on over in the ancestry thread that might be fun; what are you reading these days?
APL, that brings back memories.... At one point (for reasons which seemed excellent at the time) we needed to return, from a string describing a mathematical expression, the start and end of each nested parenthetical expression, and the level of nesting it was on.
The single line of APL that did this was actually quite elegant. Wish I could remember what it was, now.
I'm wondering if it could be done in TECO. Not that I ever reached anywhere close to that level. (I did repeating expressions, but not any more than that. Because typing the same command string n times is a pain.)
Lisp is really the only appropriate language for EPH. In any other language dealing with fractions is messy at best. In Lisp, dividing 1 by 3 gives you 1/3 exactly.
Plus, Lisp is the (ostensible) language of creation (http://xkcd.com/224/).
#444: I'm sure it could be done in TECO. If I recall correctly, TECO is Turing Complete.
I'm thinking about how to do it using standard UNIX text processing techniques and shell scripts...
Given a ballot file which consisted of one ballot per line, each nominated work in a separate tab-deliminated field, you could use awk to split each line into three fields: the work name, "1", and the points for that nomination, then sort to group the same works together, awk to sum to "1" and the points for each work, then sort again on the points column to get the works eligible for elimination at the top, awk to compute which of the top works should be eliminated, and finally sed to modify the original ballot file to get the input ballot file for the next round.
I wonder if it could be done in one gloriously-long single pipeline.
If it can be done like that with sorting, counting, some arithmetic, selecting, and other text manipulations, it can be done in TECO ;-).
#445:
While (some) Lisps natively have a rational number representation, many other languages either do as well, or have standard libraries (like boost::rational for c++, or Math::BigRat for Perl, lib/fractions.py for Python, the Rational class in Ruby, etc) supporting them. Saying Lisp is the "only appropriate" language because of rational numbers is hyperbole.
446
I'd think it would be easier to calculate the points at the start of each round: it's just 1/(number of nominees remaining on this ballot) for each ballot.
Buddha Buck,
"Saying Lisp is the "only appropriate" language because of rational numbers is hyperbole."
Well, there are certainly many other reasons why Lisp is the only appropriate language....
;-)
Lisp users? You don't want to be bracketed with those people....
It seems to me that a version written in ook! really ought to be in the library of EPH implementations available to Worldcons.
450
You might be able to get LASFS to go for it, but it would have to be modified to include slobber and drool.
I'm sitting in the audience waiting for the Hugos. As soon as it's done and they release the ballot data, we'll be crunching the numbers and posting our findings here. We have a room ( in the 15th floor of the Doubletree, vegan room). Gonna be a long night for me.
頑張ってね, Jameson! (がんばって, ganbatte)
So, here I am laboring in the 15th floor. We were not able to get full ballot data but we do have some totals. That is not enough to be sure exactly how EPH would have come out but it's enough to make some well-founded guesses.
My first numbers (these are NOT about EPH, just about how prevalent puppy voting was):
There were 51 items that were on both Sad and Rabid slates. Of those, the lowest 4 vote totals were 150, 156, 158, and 160; while the highest 4 were 769, 487, 387, 372. Lowest and highest were MDSurridge, fan writer, and Guardians of the Galaxy, respectively.
For the 6 items that were on the sad but not rabid slate, the totals were
Jameson @454: Boy, when you said you didn't have complete results, you weren't kidding! :-)
(I expect that a more complete result, without the obvious technical glitch, might even beat this comment to the punch)
Oops. Grr.
So, for things that were sad but not rabid: that is 6 things. 2 of them are episodes on Cartoon Network which got dramatically lower votes than the others. Perhaps most puppies feel themselves to be too dignified to vote for Adventure Time?
Vote totals were:
35 regular show, saving time 0 0 0 0 0 0
41 Adventure Time: The Prince Who Wanted Everything
132 A Single Samurai Steve Diamond 0
132 A Single Samurai Steve Diamond 0
185 Tuesdays with Molakesh the Destroyer Megan Grey
199 Trial By Fire Charles E. Gannon 10.90%
Glitch again. The second sad but not rabid thing with 132 votes was
132 Dave Freer.
Now for rabid but not sad. 18 such things.
Min totals:
66 Daniel Enness
86 Supernatural: Dog Dean Afternoon
100 Coherence James Ward Byrkit
100 Andromeda Spaceways InFlight
Max totals:
196 The Chaplain’s War Brad Torgersen
187 The Revenge of Hump Day edited by Tim Bolgeo
172 The Plural of Helen of Troy John C. Wright
166 Vox Day
The category of "Best Fanwriter" is particularly interesting. This is a category where both the sad and rabid slates had 5 items, and yet did not overlap completely; and also one which seemed to get relatively low voting (both puppy and otherwise) relative to other categories.
Enness, the puppy-only candidate, got 66; Dave Freer, the sad-only one, got 132; and Matthew Surridge, the weakest of the both-slates candidates, got 150.
This is a bit of a head-scratcher: why is the overlap candidate getting significantly less than the sum of the two separate slates? I have some ideas which I'll post in the next comment.
So, why isn't R+S≤R𝕌S?
There could be two reasons:
1. Extra non-slate votes for the candidates. You'd expect all slate candidates to be pulling from the same distribution (ie, rolling the same number of dice) for those extra votes; so at a first approximation, that would mean subtracting the same number from each total to get the true puppy numbers.
2. Incomplete slate voters. This would be people who say something like "I do have one favorite fan writer who is not on the slates, so I'll vote for them; but then I'll fill out the rest of my ballot with candidates from the slate." You would imagine that such people would be more likely to take from the top of the slate list than from the bottom; though it's hard to know how strong this tendency would be.
Further analysis to follow...
When we cross-posted, I had already given you a substantive reply, which you seemed to accept. But I will meet your requests as a show of good faith.
"EPH and utilitarianism."
I think EPH is designed as if we wanted “our” authors to win out of self-interest. My perspective is that if an author I don’t personally like wins I'm still happy to applaud them along with their fans.
"Examples that would support this contention."
Sherri Tepper. Changing of fifth-place winners without clustering of votes, could be more with clustering.
"Reconcile this with the negative statements about "wisdom of the crowds" and "the needs of the many"."
I'd rather consult the majority of WSFS members when deciding how they will select award winners.
"Citations and insults, please."
You addressed me on another blog and invited me to come discuss EPH in this thread. http://file770.com/?p=23227&cpage=21#comment-290754
On arriving here, from #64, you immediately called me a “troll,” "you are wasting enough of my time already," “flouncing," etc, questioned my honesty and threatened "disemvoweling" (#258) if I didn’t do specified things, then said (260) "I'm a firm believer in the redeemability of humanity," yet wrote about Cairnes (281), "The evidence points to the idea that he (unlike, ironically, Brian Z) does not, in fact, want to discuss EPH and its effects. He just wants to have a fight." If so, why question my honesty? In 290, you again declared apropos of nothing that you will disemvowel my past and future comments. You may not consider that insulting, but I felt so.
"Links, comment numbers, clear and detailed explanations of which, please."
I was bothered by what I felt was a selective questioning of motives and threat of "disemvoweling." Other comments here have elements that can be hostile, accusatory, overly personal or demeaning. (To meet your request for comment numbers: I can see I was already feeling put off by #5, 7, 10, 59, 62, 67, and it did not improve farther down). So my subjective experience was that this felt unfair.
"Where did Patrick May say this?"
See links at: http://file770.com/?p=24489&cpage=4#comment-322947
I hope you will understand my point that if I hear from a moderator things like “clean it up if you want to post even one more thing and have it stay,” I won’t wish to continue having a discussion.
The problem with hypothesis 2 for explaining the fan writer category is that Matthew Surridge, the weakest double-puppy, was actually at the top of the sad list and second-to-top on the rabid list. True, he was one of the two double-puppies who was not linked by either list; but on the other hand, the other unlinked double-puppy was Amanda Green, who got the second-to-top votes (175).
So if the answer is something like hypothesis 2, at least 1/3 the puppies would have to have been exercising some "statistically biased" process such as actual aesthetic judgement, when they decided whom to leave off of the slate. (It doesn't have to be judgement; it could be, for instance, that some preferred the woman on the slate lists, while others preferred the people who were linked).
So for hypothesis 1, in order to really do this right from a statistical point of view, I'd have to do as much work as I did on my qualifying exams. (Assume that the slates have R and S respectively, while each candidate also gets a bonus amount of nonslate voters, drawn from a negative binomial distribution with parameters α and β, find the MLE of... then use empirical Bayes...)
However, a quick and dirty estimate is: each type of candidate is getting an extra bonus which averages to a fixed number. So the average double-puppy has 170.5; the sad has 132; and the rabid has 66. 132 + 66 - 170.5 =27.5 would be the estimate for the nonslate votes each pubby candidate was getting. Eyballing things, this would probably have placed them in about 10th-15th places if the slates had not existed. This seems not unreasonable.
So using those numbers, the number of to-the-letter slate voters was probably only about 143; with (in this category, at least) more sads (over 100) than rabids (under 40).
So for hypothesis 1, in order to really do this right from a statistical point of view, I'd have to do as much work as I did on my qualifying exams. (Assume that the slates have R and S respectively, while each candidate also gets a bonus amount of nonslate voters, drawn from a negative binomial distribution with parameters α and β, find the MLE of... then use empirical Bayes...)
However, a quick and dirty estimate is: each type of candidate is getting an extra bonus which averages to a fixed number. So the average double-puppy has 170.5; the sad has 132; and the rabid has 66. 132 + 66 - 170.5 =27.5 would be the estimate for the nonslate votes each pubby candidate was getting. Eyballing things, this would probably have placed them in about 10th-15th places if the slates had not existed. This seems not unreasonable.
So using those numbers, the number of to-the-letter slate voters was probably only about 143; with (in this category, at least) more sads (over 100) than rabids (under 40).
I note that I sometimes have a Resident Statistician, but he left on his trip home at about eleven ack emma.
I know the difference between mean, mode, and median, and why any politician who talks about an average should be regarded with a little caution.
Brian Z @461:
When we cross-posted, I had already given you a substantive reply, which you seemed to accept.
Rereading the exchange (comments 419 and following on this thread), I can't see any grounds for thinking that you had substantively answered the questions I'd asked.
But I will meet your requests as a show of good faith.
Thank you. I will remove you from automatic moderation, but do be aware that I will need you to continue providing links and backup for any assertions you make; you have still not raised your level of credibility in this community to ground level. With a consistent pattern of sensible and reliable commentary, you can achieve this.
"EPH and utilitarianism."
I think EPH is designed as if we wanted “our” authors to win out of self-interest. My perspective is that if an author I don’t personally like wins I'm still happy to applaud them along with their fans.
This is of course your perspective, but it's not remotely true outwith that headspace. That's the point of an algorithmic change as opposed to a judgmental one: it's neutral. It applies to authors we like and authors we don't; it simply makes sure that the candidates on the final ballot better represent the nature of the nominating community.
"Examples that would support this contention."
Sherri Tepper. Changing of fifth-place winners without clustering of votes, could be more with clustering.
I'd love to see some more models of this occurring. You should generate some datasets that illustrate this effect, so that we have more examples to discuss and consider.
Also, not everyone recalls the Sherry S Tepper issue; next time you bring up such a thing, feel free to provide a link.
"Reconcile this with the negative statements about "wisdom of the crowds" and "the needs of the many"."
I'd rather consult the majority of WSFS members when deciding how they will select award winners.
The issue of how the WSFS constitution is amended is of course an interesting and contentious one, and has always been. But I'm not sure how that ties into the wisdom of the crowds and the needs of the many being bad things while democracy is good.
"Citations and insults, please."
You addressed me on another blog and invited me to come discuss EPH in this thread. http://file770.com/?p=23227&cpage=21#comment-290754
You make it sound like you hadn't been here before and I judged you ab initio, but as your commenting history here shows, you have been in the process for some time; my comments were based on your already-established record for misquoting across multiple sites without the courtesy of links.
On arriving here, from #64, you immediately called me a “troll,”
Actually, the relevant comment is 66, and the precise line is:
The best way to refute accusations of being a concern troll is not to link to your rather strained rewrite of Le Guin; it's to stop acting like a concern troll.
Which is not the same as calling you a troll; it's telling you to stop trolling lest people think you are one.
"you are wasting enough of my time already," “flouncing," etc
(comment 73, for those following along at home): these are things you do, not things you are. That's the distinction I consistently try to make, as I mentioned when I said I try very hard to focus on what people do rather than who they are in my comments in comment 419.
questioned my honesty and threatened "disemvoweling" (#258) if I didn’t do specified things, then said (260) "I'm a firm believer in the redeemability of humanity," yet wrote about Cairnes (281), "The evidence points to the idea that he (unlike, ironically, Brian Z) does not, in fact, want to discuss EPH and its effects. He just wants to have a fight." If so, why question my honesty?
Because you're clearly not stupid or ineffective with language, so I'm unable to excuse your repeated failures to accurately quote or engage with the plain text of what people say as linguistic infelicity. So I've reluctantly fallen back on the idea that although you do want to discuss the matter, you are choosing to do it in a dishonest fashion. (I am not alone in this view.)
In 290, you again declared apropos of nothing that you will disemvowel my past and future comments.
No, it was apropos of the fact that you were active in the wider community but had left some threads dangling here, ones I was prepared to snip off if you didn't come back.
You may not consider that insulting, but I felt so.
I'm so sorry to hear that you feel that way, but I really can't control your feelings. I don't see anything in that list of things you dislike that is inconsistent with my values and my approach as a moderator. I've focused on what you do rather than essentializing it into what you are, and I have given you repeated opportunities to behave more productively and guidance on how to do so.
"Links, comment numbers, clear and detailed explanations of which, please."I was bothered by what I felt was a selective questioning of motives and threat of "disemvoweling."
Again with the scare quotes. It really is the actual term for the action. And it was "selective" only in the sense that I deal with everyone according to the ways that they've behaved on-thread. You are where you are in my esteem due to your own actions.
Other comments here have elements that can be hostile, accusatory, overly personal or demeaning. (To meet your request for comment numbers: I can see I was already feeling put off by #5, 7, 10, 59, 62, 67, and it did not improve farther down). So my subjective experience was that this felt unfair.
Again, that is regrettable, but having reviewed the comments in question and the entire thread, I didn't see anything that moved beyond the general to and fro of the discussion we were having. It's certainly been less personal than the discussion on File770; I presume you'll be talking to Mike Glyer about this? (Can I watch?)
I hope you will understand my point that if I hear from a moderator things like “clean it up if you want to post even one more thing and have it stay,” I won’t wish to continue having a discussion.
That is very much your choice. And yet here we are. There are two additional points which you must now take on board to retain posting privileges here:
Vote from the business meeting:
186 for, 62 against.
EPH goes to MidAmericon!
Congrats Kilo, Jameson, and all! The sunset clause (EPH comes up for re-ratification in 2022, if passed in 2016) looked like a good compromise to help push support up and over.
Congratulations and thanks for all your hard work to get this proposed and past the first hurdle.
Congrats to Making Light and thank you for giving us EPH. Your effort is appreciated. It passed.
A couple of procedural questions:
1) I understand that EPH can no longer be amended or procedurally killed. At MidAmericon it gets a straight-up vote, with no debate, no postponed indefinitely, no adjourn sine die, etc. Is this correct?
2) EPH was approved by this WorldCon BM by a factor of 3:1. Has any previous amendment passed by such a margin failed ratification?
EPH passes!
186 for / 62 against.
http://katsudon.net/?p=4333
http://file770.com/?p=24507&cpage=3#comment-324914
Five year sunset clause added.
Thank you to everyone for all their efforts!
Technical question; does the sunset clause mean that it has to be revoted twice in consecutive years? Or just once, five years from approval? And is it an every-five-year thing, or a once-and-done thing?
Thanks for any clarification; I couldn't get the live-stream to work so I couldn't follow this as closely as I'd've liked to, and don't know the wording of the sunset clause.
Cassy, it's apparently to revisit in 2022 to see whether we still need EPH.
My bet is that people will have gotten so used to it that some will think 'it's always been done this way'.
Congratulations from here too. Watching the reports from the business meeting was an agony of suspense; and then, eucatastrophe! They also serve who only stand on the sidelines making vague noises of encouragement ...
They also serve who only stand on the sidelines making vague noises of encouragement ...
Beats showing up at the last minute and complaining about how it's being done and that it's going to be Too Much Work and It Isn't A Proven Algorithm.
Does 4/6 also have a sunset clause?
479
No - and it barely passed, 86-82.
Thank you, thank you all. Especially Keith and Jameson.
I hope you found much Worldcon to enjoy in between the hard work.
4 and 6 doesn't have a sunset clause, but I would welcome an amendment to add one, and might make that amendment myself, just so people feel more comfortable about passing it. (For those who don't know, I'm one of the three co-sponsors, and I may be the only one of us in Kansas City next year.) That would be a one-time requirement for re-ratification, after which it's in the Constitution permanently, or until somebody amends the Constitution to take it out.
1) I understand that EPH can no longer be amended or procedurally killed. At MidAmericon it gets a straight-up vote, with no debate, no postponed indefinitely, no adjourn sine die, etc. Is this correct?
Not entirely. There's no PI. There will be debate, and the motion can be amended. If the motion is amended, it would require re-ratification unless the change were a lesser one. The only lesser change that occurs to me would be to shorten the sunset clause to be earlier than 2022, but there may be others.
2) EPH was approved by this WorldCon BM by a factor of 3:1. Has any previous amendment passed by such a margin failed ratification?
I don't know, but I'd look at the 1986 motions and 1987 minutes. In '87 we defeated something that had passed pretty easily at '86, I believe. I recall the 1986 minutes as being sketchy at best, so it may not be apparent.
Sunset clauses, as typically added to some items in recent years, require the Business Meeting N years in the future to re-ratify the item. It's not two years; it's a single re-ratification. The clauses also require that future Worldcon to put the question of re-ratification on the agenda as if it were a constitutional amendment passed on from the previous year. (This prevents a WSFS BM staff from "burying" a ratification by not scheduling it and hoping that nobody will remember to bring it up.)
It seems likely that adding a sunset clause to an amendment awaiting ratification (such as 4/6) would be a lesser change, and I would so rule, but I'm not presiding next year. In fact, I'm likely to be assistant videographer and not even in the chain of command.
I heard that EPH will be tested against the actual data for 2015. When do you think we'll know the results of that?
Greg@485:
As yet, we haven't received any of the 2015 ballot data, so we really don't know. Once we have the data, we should be able to run the results the same day.
Kilo
Kevin@484:
Thanks for the insight, Kevin. I think in general sunset clauses are a good idea with new systems such as these. It gives us the chance to test them in real-world conditions and then decide if this is really the way we want to go. I'm pleased that the procedural hurdles are greatly reduced, as well.
Kilo
I suspect in 2022, when EPH comes up, many people will be going 'it's always been this way'.
Talking of that, I've been wondering for a while now when the Hugo voting went from first-past-the-post to the current IRV. File 770 a little while ago had a copy of the tallies for Way Station's win in 1964, so it was definitely later than that. Whose idea was it, and what kind of struggles were there to get it adopted? This is surprisingly undocumented in anything I've been able to find.
David @489: it's in the 1971 Constitution. There doesn't seem to have been the current "amend, then ratify" step, so that may have been its first usage. Or not...
489
It was IRV in 1971 - 1972 used the same program, and it's definitely the same method.
Speaking of history... Occasionally someone asks me "Why is there a No Award rule?" I've been telling them "It was originally intended to cover the case where you read every SF novel published in a year and decide none of them are very good. Which was possible in the 1950s."
Is that basically accurate?
The WSFS rules talk about No Award being declared when voters ignore a category, but they don't say anything explicit about the case where voters actively vote for it.
Andrew Plotkin @492:
The WSFS rules talk about No Award being declared when voters ignore a category, but they don't say anything explicit about the case where voters actively vote for it.
Not exactly, but they do say:
3.10.3: “No Award” shall be listed in each category of Hugo Award on the final ballot.
Therefore, since it's always a candidate in every category, it can always win a category. This isn't like the pretend "None of These Candidates" choice we have in elections in Nevada where I live, where the second-place finisher wins if NOTC wins, as it did in the Democratic primary election in 2014.
(Aside: It does seem to me that at least one and probably more of the nominees for this year who lost to No Award either didn't know how the rules worked or assumed that No Award was treated like Nevada's NOTC. They're now complaining about how unfair it is, when in fact the rules are not new and have been in place for nearly my entire lifetime, and I turned 50 during the drive home from Spokane to Fernley NV.)
Kevin:
Ah, I see. But that doesn't give us a sense of what case 3.10.3 is supposed to handle, or what argument (was there an argument?) led to its inclusion.
I am comfortable with my guess but I was hoping somebody could point at a history.
492
'When you think none of them were good enough' is a good reason to vote No Award first. (It's how it was explained to me lo these many years ago.)
IIRC, there were several years when a lot of people grumbled about all the nominees not being very good (especially in DP, which only had one length back then, and really wasn't always good). And folks who felt that way went to the Business Meeting, and decided that if it was useful in one category, it might well be useful in all of them -- it was easier to pass it as a uniform change than to single any one category out, because people felt that was fairer. So it got passed, and left in even though it wasn't used very often -- just often enough that people saw it was useful. Back in those days, there didn't have to be a ratification at the next BM -- passed was passed.
Nobody expected something like what happened with this year's nominations, but it turns out to have dealt with that in an efficient manner. It's not the original intended use, but it is one very effective use.
Kieth@486 After the Sunday business meeting, one of the IT guys for Sasquan told me he'd run his own implementation of EPH on the data and that the results were "not as good as people seem to be expecting." He told me that it was possible that his code had an error, and that he thought they were going to let you guys at the real data very soon.
I did my own estimate based on the data that was released with the Hugo voting results. Just looking at Novellette (and pretending that no one was disqualified or withdrew), I estimated that even with EPH, the puppies would have won 3 out of the 5 slots. Better than 5 out of 5, but still not quite as good as expected. But that's making several assumptions about the missing data, of course.
Have you done that exercise? That is, take the data on the top 15 nominees, assume that puppies only nominated items from their slates (and that a few puppies didn't nominate all five). Then assume that all other works were nominated by people who only nominated that one work. So the puppies' vote is spread as broadly as possible across their five slate choices while the non-puppy vote is as concentrated as possible in the other choices.
I think that creates the best possible case for EPH. And, as I said, puppies still carried 3 out of 5 slots.
Greg@497:
I'll see if I can run some simulations this weekend, but if I understand what you've set up, I think you've set up the worst-case scenario for EPH and not the best case. If there was little to no overlap between non-slate nomination ballots, then that means that a significant number of voters will have all their choices eliminated instead of transferring their support to other works. This means that the slate has effectively a much higher percentage of the total ballots, so will get more slots. This is the "dispersion" issue brought up in the other thread. Basically when there is no real preference among voters, then slates will always do better. In the best case scenario, there is a lot of overlap among ballots, so a lot of support gets concentrated into the non-slate works.
This is just me thinking about it off the top of my head, but I'll see what I can set up and run. This is why just having the total nominations without the actual ballots really doesn't help us very much...
Kilo
Kilo@498 Here's one way to think about it:
When we see the total nominations for the top 15, we really don't know how many individual people that represents. However, we can put bounds on it.
Worst case, there is no overlap--each vote represents a different person--1933 total for best Novelette, of which 1205 or 62% are puppy picks.
Best case, the puppies overlap totally, so there are only 267 of them, but there are still 728 regular fans, so puppies are only 27% of the total.
In the worst-case scenario, the puppies sweep the awards. In the best-case scenario, they still get 3 of the 5 slots--60% of the nominations but with only 27% of the voters.
Or am I figuring it wrong?
499
FWIW: the means from 1984:
1984
category . . . . mean per ballot
novel . . . . . 3.0 noms
novella . . . . 1.2
novelette . . . 1.6
short story . . 1.7
non-fiction . . 0.8
dramatic . . . . 1.7
pro editor . . . 1.6
pro artist . . . 1.9
semiprozine . . 1.0
fanzine . . . . 1.1
fan writer . . . 1.0
fan artist . . . 1.1
Campbell . . . . 1.0
mean per ballot: 18.8 noms
Greg@489:
I think I see what you're saying, but I don't think I agree with your percentages exactly (though I'm getting the same general ballpark). Looking at novelette, the top 15 works were:
267 Championship B’tok
266 Ashes to Ashes, Dust to Dust, Earth to Alluvium
259 The Journeyman: In the Stone House
248 The Triple Sun: A Golden Age Tale
72 The Day the World Turned Upside Down
69 Each to Each
65 The Devil In America
54 The Litany of Earth
54 The Magician and Laplace’s Demon
46 A Year and a Day in Old Therandane
43 The Bonedrake’s Penance
42 Spring Festival: Happiness, Anger, Love, Sorrow, Joy
42 We are the Cloud
38 A Guide to the Fruits of Hawai’i
38 The Husband Stitch
(Note that Wright's work from the Rabid slate was ineligible.)
There was an obvious slate (identical for both Puppies without Wright) of around 267 ballots. We can probably safely assume that the anything below the top 15 would have been eliminated early, so we'll pretend that they don't exist (which may not be a safe assumption, but without data what can you do?). That means that there were a total of 563 non-slate nominations. This means that the slate got a minimum of 31% of the top-15 work ballots. The reality, of course, is that they probably got a significantly higher percentage.
It's not quite that simple to get an upper bound, since we don't know the dispersion of the non-slate ballots. When the dispersion is wide, then you will have lots of ballots that get all their works eliminated. This is one of the main things I'd like to see investigated with EPH once we have data to work with. There's no question it helps, but knowing the break points in dispersion will let us predict how much it will help.
We could look at it another way: Counting total nominations, there were 1040 nominations for the four slate works. Compare that to the 563 total nominations for all non-slate works combined. There's nothing anybody can do -- with any system -- if there isn't a general preference in fandom, or if fandom simply doesn't vote. Basically none of the non-slate works got much support compared to the slate works. The obvious answer is to get more people nominating (something many others have also advocated, though it won't help much alone), and then the percentage of slate ballots drops into something more reasonable. But as long as they have such a huge percentage of the ballots, I don't think there is any fair way counter that. That said, rescuing two slots out of this situation isn't bad...
Disclaimer: All this is just musings -- I may be looking at this entirely the wrong way. If so, hopefully someone will jump in here and do a better analysis. If/when we have the data, then we can take a definitive look, but without the ballots it's really hard to say. Given how few people nominated for novelette, though, I'm not really surprised that the slates would take a large number of slots.
Just my thoughts,
Kilo
Nah, your analysis is fine. If the slate has 2/3rd of the total votes (probably less when you add the smaller ones), I'd expect to see 2 or 3 (or even 4 if it is really 2/3rd)) entries making the ballot. This is the whole point of EPH after all - it is all about the ratios.
I will be interested to see the total nominations (not just the top 15) and what percentage of them are these 1040. Although this category actually is not the best one to look at - because we need to add the 250 or so that went to the then removed story -- so the puppies total is closer to 1300 really.
With that data, it's hard to see Triple Sun getting fewer than 62 points in the last few rounds, and Journeyman getting fewer than 65.66 points. Anything from The Devil in America down could never get more than 65.66 points (since you can never have more points that nominations), so they'd be eliminated by Triple Sun in a showdown. Each to Each might get more than 65.66 points in the final round, if it overlapped with The Day the World Turned Upside Down on at most four ballots, but it would be pretty close. So it looks like the Puppies are pretty much guaranteed three slots under EPH, and would probably get four. But even if they had nominated five eligible works, I don't think they could get a ballot sweep.
I had a thought about dispersion that may be applicable to the novelette data that we're looking at here.
Ultimately, EPH works well because it causes slate works to compete with and eliminate each other. In order to do that, the slate works cannot be in the "fourth place safe zone", because otherwise their nominations will never be compared with each other. What you need is for at least one non-slate work to have enough points to keep the slate works out of that zone.
Of course, as the slate works compete with each other, they will get more points, which may put them in the safe zone, so the number of surviving slate works depends on the dispersion of the non-slate works. If the non-slate works end up combining to give enough points to continue to keep a slate work out of fourth place, then at some point, that slate work will be compared -- almost certainly to another slate work, since they have about the same number of points -- and then another slate work will be eliminated. If the non-slate works are so dispersed (in other words, there are no general fan favorites) that they can't get enough points to force the slate works to be compared, then the slate will do much better.
I think that's a better analysis of what's going on with the novelette category than to look at percentages. In this case, while no non-slate work got enough nominations to keep the slate from sweeping, there are two or three works that potentially have enough points to force some of the slate works to compete with themselves. In this case, then, the slate will lose at least one and possibly two works (leaving 2 or 3 slots for non-slate works).
Again, the answer is to get more voters so that EPH has enough "raw material" to do its job. Under the current system, even if we add more voters, slates will still sweep the ballot because there will be no concentration of support in the non-slate works at all. EPH can amplify that concentration of support so that you don't need as many voters, but it does need something to work with.
Does this seem like a reasonable analysis? All this would be very simply demonstrated if we had the ballot data, but until then, we have to look at it from a theoretical viewpoint. I'd be interested in other perspectives and simulations.
Kilo
With that data, you couldn't possibly eliminate more than one of the slate works: as soon as you drop one, the others all have at least 86 points, which means they couldn't even be one of the bottom two works. So they're guaranteed three slots, and maybe get four.
Basically, if the nth-ranked slate nominee has more than n times as many nominations as any non-slate work, then the slate is guaranteed n slots on the final ballot (for n less than or equal to the number of finalists).
It occurs to me that, while it's generally expressed just as, 'get more voters,' what the Hugos need most (and indeed have for years, long before Puppydoggle) is more voters who read shorter fiction. I wonder if it's worth putting some discussion into how to either encourage Hugo voters to pay more attention to shorter fiction, or to encourage habitual readers of shorter SFF to participate in Hugo voting? I suspect the latter might be more productive; Hugo voters, and/or other components of Worldcon-going fandom, have been handwringing at each other for about as long as I can remember about the shorter categories being neglected, with little effect.
Kilo@501 remember that you don't get to exclude the disqualified work for this experiment. We cannot depend on slate works being disqualified in the future. The pre-disqualification list shows what the slates were capable of doing. Removing the disqualified item artificially makes EPH look better.
If you don't like imagining a list where these were the only nominated works, then imagine that the list of 15 was the result after N rounds of EPH. We know the vote totals; we just need to model the points. So let's maximize the point totals for the non-slate works (that is, each gets a point total equal to the vote total). Then minimize the point total for the five slate works. In descending order, I get point totals of 69.83, 67.75, 60.17, 53.75, 33 (again, including all five slate-nominated works). That's from 267 slate voters who voted for the top work, then 266 who voted for the first two, etc. down to 165 who actually voted for all five. 267 slate voters total and 728 non-slate voters.
I don't think you can make it any better for EPH than that. But with only 27% of the voters, the slates still take 60% of the slots on the final ballot.
SunflowerP@506 the trouble with short fiction is that if I recommend a story to you, how are you going to read it? For example, I think "The Molenstraat Music Festival," by Sean Monaghan
(Asimov's Science Fiction, September 2015, 14,596 words) is a great nominee for Best Novelette. If you aren't a subscriber to Asimov's, how are you going to read it? I can't give you a link to it.
Since this is the current issue, you can always buy it (or even subscribe today) but what if it were in the January issue? Ever tried to get hold of an old issue? For ANY amount of money?
I think Worldcon needs to have a "prenomination" phase in which members submit stories they're thinking of nominating and then authors give the text to Worldcon to cache. Members could then link to those stories, but non-members would reach a page saying "you need to sign in or join to view this page."
I'd volunteer to implement that code, if the admins wanted me to.
I think you're right that the problem is not needing more voters, it's needing more short fiction voters. Looking at novel and even novella, EPH isn't going to have any problem limiting the slates. Short story is going to be borderline (not many voters, but the slate also isn't as strong), and of course we've been discussing novelette. Best related work and maybe the Campbell award may be a problem as well. In the other categories, it looks like EPH would do a good job, though (again, with the disclaimer that we don't have the actual ballots to really know).
So as others have said, how do we increase readership in short fiction? This is not just a Hugo problem, it's an industry problem. All of the paper short fiction magazines are perennially struggling. There is a huge plethora of online magazines, but I suspect many fans don't know they exist or how to find then if they do.
A pre-nomination page might help (though it veers close to being Sad Puppies 4), but it doesn't solve the problem of the fact that there aren't just few nominators, there are few(ish) readers. I think the entire industry would like to know how to fix that. How do we get the online fiction magazines better well known?
As a writer myself, I've had several discussions with my friends about how short fiction is no longer the path to a writing career that it used to be. It's perfect for learning the craft (and occasionally getting paid) because you can go through the process many, many more times than you can with a novel. But you can't live on it. On the flip side, I know several successful novelists who have never published a short story at all. Short fiction requires an entirely different skill set (and I would argue a more challenging skill set) than novels, so building one doesn't necessarily lead to the other. I'm just not sure what the answer is to that.
But I recognize that has very little to do with EPH...
Kilo
I agree with the short fiction problem; frankly, I've been following links to online fiction in order to find worthy works to nominate. Which means excellent works published in dead-tree magazines are going unread and unconsidered by me. I don't have a solution to this; I wish I did.
And I see that - unbeknownst to me when I made post #506, since I can't catch up on everything simultaneously,and six days out of the loop left me with a lot of catching up - the short-fiction discussion was/is already happening over in the Hugo discussion thread that is called 'Hugo discussion thread'.
Cassy B@510: I suspect dead-tree magazines (or at least dead-tree-only ones) will be increasingly at a disadvantage for this purpose - as, indeed, is already the case in other respects for periodicals, and has been for some years. But the 'unread and thus unconsidered' thing isn't anything new when it comes to individuals nominating for the Hugos; even back when 'read everything published in the field this year' was possible in terms of time, not everyone had the budget to purchase every issue of every magazine. It occurs to me that this might be a factor in why the shorter categories have always been 'neglected' relative to novel (libraries being a more reliable source for books, than for genre magazines) - perhaps the ready availability of online 'zines will result in a 'golden age' for short fiction.
Keith@509, Cassy@510, Sunflower@511
Over the past few weeks, my husband and I put together a web site (free and unmonetized) both to recommend short fiction AND to make it easy to get back copies of the professional magazines.
http://www.rocketstackrank.com/
It turns out that there are several different ways to get online copies of back issues, but it varies by magazine. Have a look at it, if you're so disposed, and give it a try. We don't make any money from it, but we'd love to know whether others are able to use the instructions to get back copies.
Greg Hullender @512, thank! My reading time is going to be vastly curtained for a month or so, but I'll bookmark the site and try to get back to it when I have a life again.
Me, above: curtailed, not curtained. And I can't even blame autocorrect...
Cassy B@513 "vastly curtained"
I suppose the term COULD refer to an intention to start a marathon reading of assorted curtainfic...
:-)
Pay no attention to the manual behind the curtain.
Since "E Pluribus Hugo" is on the Business Passed on to MidAmeriCon II site (a.7), is showing up and voting all that needs to be done on that front?
Steve Halter:
Yes. If ratified by a majority of those people voting at this year's WSFS Business Meeting, EPH first affects the 2017 Hugo Awards.
Note, however, that supporters will need to show up and vote at more than one business meeting session. In addition to the ratification vote in the main business meeting, there will almost certainly be procedural shenanigans in the preliminary business meeting designed to undermine or kill the proposal.
Well, SR 1.2 prevents anyone from doing much to an amendment pending ratification.
Ah, SR 1.2 does seem to preclude the most obnoxious procedural twaddle I remember from last year's meeting. That's good. (I've been to several business meetings in the past, but I've developed only a fairly basic understanding of the rules.)
Jameson Quinn and Bruce Schneier have been given an anonymized copy of the 2015 nominating data, and Jameson has posted a summary of their results over at File 770:
Steven desJardins@523:Cool. It is nice to see that the preliminary results with actual data support what we were seeing in the various models that were run.
Also interesting (although not terribly relevant) that they had to do an NDA with someone--presumably the holder of the data.
524
It's also interesting that someone is unhappy about even that much information being released - even though all Jamieson did was confirm the previous tests. Someone seems to be afraid of something.
PJ Evans@525:Someone is being very cautious. The nda implies lawyers and cautiousness is the nature of the profession.
Based on what we saw last year, I would guess someone is worried about 1) the thoroughness of the anonymization and 2) maintaining the appearance of impartiality.
Complete conjecture on my part, of course.
What entity actually owns the Hugo voting data?
Is it WSFS or the particular worldcon or something else?
Steve Halter:
Each year's Worldcon owns the Hugo Award voting data for that year. Per WSFS Constitution section 3.12, Worldcons generally "delegate all authority under this Article to a Subcommittee whose decisions are irrevocable by the Worldcon Committee" and cannot interfere with that subcommittee.
The WSFS Business Meeting cannot override these decisions, nor can they demand that committees release raw voting data. They can ask, but committees are not required to obey such requests. Last year's WSFS Business Meeting made such a request, and an individual member of the Hugo Award Administration Subcommittee indicated that such information would be released; however, that statement was made without consulting with the other members of the committee, and the ultimate decision of the committee was that a general release of the information could not be done without compromising individual voters' privacy.
The WSFS Mark Protection Committee, which is the custodian of the Hugo Award service mark on behalf of all Worldcons and their members, cannot demand this data or compel its release, either. It is entirely in the hands of the individual Worldcon and its Hugo Award Administration Subcommittee.
Each individual Worldcon irrevocably delegates their entire authority over the Hugos to a Hugo Subcommittee. Those subcommittees control access to the voting data, and the Worldcon itself has no authority to override their decisions.
Kevin's answer is, of course, more complete and precise than my own.
Thanks Kevin and Steven. It looks like the intent of 3.12 is to shield the rest of the committee from not being able to vote for the Hugo's.
The data availability could be addressed by an amendment along the lines of:
After X years, all Hugo voting data must be made public.
For testing purposes it seems like that could be useful and avoid some contention although it doesn't seem to be a pressing issue at this point. Exactly what X should be would probably be an interesting discussion.
We could add in anonymized data, but then a suitable definition of anonymized would also have to be provided.
Steve Halter:
It's not to shield the committee from being able to vote for the Hugo Awards. It's to shield them from being eligible for them. For example, suppose $AUTHOR agreed to be on a given Worldcon committee as SFWA Liaison. Without 3.12, $AUTHOR would be ineligible for that year's Hugo Awards because s/he would be on that year's Worldcon committee. (Note that the US definition of "committee" is significantly broader than the UK one.) Under those circumstances, who would be willing to take the job?
This is not academic. The rule is a reaction to a specific case, but it definitely applies to a lot of people.
Thanks Kevin--yes, that's clear when I read the rule again.
P.S. to Kevin's specific case: $AUTHOR offered to withdraw the work in question. Instead, $AUTHOR was removed from the committee, with many jokes and good wishes.
And Brad Templeton has shown up at File770 talking about write-ins on final ballots and the #$%^&*()!!! longlist idea that got stomped all over last year.
There seem to be a number of people leaping to unwarranted conclusions based on Jameson's initial post.
536
There also seem to be people trying to re-argue the stuff they didn't get last year.
Plus the one who wants to not count ballots that have the same items, for whatever (as-yet-unclear) reason. I think that one assumes that 'ballots with the same works' = 'slate', when that's not in fact true.
While totally understanding the impulse to vent and the frustrations that give it rise, can I ask whether this conversation is making us smarter, wiser, or more joyful?
Update on E Pluribus Hugo:
Bruce Schneier and Jameson Quinn have posted an analysis on EPH's effectiveness over on File770 and there's a discussion going on there.
Comments containing more than seven URLs will be held for approval. If you want to comment on a thread that's been closed, please post to the most recent "Open Thread" discussion.
You can subscribe (via RSS) to this particular comment thread. (If this option is baffling, here's a quick introduction.)
HTML Tags:
<strong>Strong</strong> = Strong
<em>Emphasized</em> = Emphasized
<a href="http://www.url.com">Linked text</a> = Linked text
Spelling reference:
Tolkien. Minuscule. Gandhi. Millennium. Delany. Embarrassment. Publishers Weekly. Occurrence. Asimov. Weird. Connoisseur. Accommodate. Hierarchy. Deity. Etiquette. Pharaoh. Teresa. Its. Macdonald. Nielsen Hayden. It's. Fluorosphere. Barack. More here.