Further Hugo Thoughts

[drplokta]

I've been reflecting on my proposal for making the Hugos less gameable. I think the principle is correct, that the number of nominees should automatically increase in categories where an algorithm detects slate-like behaviour. I now favour reducing the number of nominations per nominator to four, and having between five and ten nominees per

Comments

[pauldormer]

I still have all the nomination statistics for 2005 on my computer, or is that too historic?

For instance, I see that 230 novels were nominated in total in 2005 (although not all were eligible) and 93 received just one nomination and 36 just two.

[drplokta]

I think 2005 would be useful, if you can let me have the data (assuming it contains no one's personal information).

Thanks.

[pauldormer]

I used an Access database to count the nominations and I still have that. Should only take a few minutes to write a query that counts how many works got x nominations, export that to a spreadsheet and store the spreadsheets on a shared drive somewhere. Is that the sort of thing you want? (In a previous life, I was a certified Access programmer, but counting the nominations was the last substantial programming I ever did.)

[pauldormer]

I've produced spreadsheets for the four written fiction categories and put them on Googledrive. Is this the sort of thing you're interested in?

https://drive.google.com/folderview?id=0B8jcg91oruibeWhKa1c1SEpKeEk&usp=sharing

[despotliz]

The 2013/2014 nomination data has a table with this information in them at the end, I think? In 2013 it's called people/works, in 2014 I think the column for entries is the equivalent data. I'm not sure how good historical data is any more, but the 2009 nomination stats has the total number of things nominated and they are also unusually detailed because they list all nominees down to 5 nominations, rather than 5%.

[netmouse]

I am not sure why you want to try to 'detect' a slate looking only at the nominations for a single category. All the statistics in the nominations are available. The program can compare nomination patterns between entire ballots and determine whether or not some are more identical than is statistically probable using similar algorithms to how they detect cheating on tests. The problem is comparable-- you want to detect whether someone is just copying someone else's answers or are using their own thoughts as input. Once the system has detected identical ballots it can then figure out what percentage of all nominating ballots in each category is on those identical ballots and increase the number of nominees accordingly.

[drplokta]

I don't believe there is any chance that the business meeting will pass a Hugo process that goes "...and now put all the nominating ballots through this complex algorithm designed to detect collusion...", and so I don't intend to propose any such thing. Just looking for identical ballots is too easy to work round -- tell everyone who is in on the collusion to leave off one item from the slate, chosen at random. I'm thus looking for a simple algorithm that can be expressed in at most two paragraphs of text that can detect slate-like behaviour. The nominating numbers for this year are obviously screwy to human observation, so it should be possible.

[jamesb]

Have you come any closer to something you could propose, or be happy with?