FiveThirtyEight.com - Minnesota
I am told the guy who runs fivethirtyeight.com has an amazing track record predicting baseball player performance. I am also told his track record in the primaries was good, but have never been linked to any demonstrations illustrating this last point. Still, I continue to not have faith in fivethirtyeight.com's methods of analysis. In this post, I use Minnesota polls as an example to illustrate my lack of faith.
Looking at Minnesota, just at polls taken in September (in order):
+14, +2, TIE, +3, +2, +8, +1
The mean for Sept. is +4.3 with a standard deviation (SD) of 5. In addition to being the oldest of the Sept. polls, the +14 CNN poll falls outside the standard deviation (4.3 mean + 5 SD = 9.3). Thus, the +14 is an outlier. Factoring out the +14 CNN poll, we get a mean of +2.7.
RealClearPolitics puts Obama up +2.8.
Pollster.com puts Obama up +3.2.
fivethirtyeight.com projects Obama +7.9.
RealClearPolitics lists Minnesota as "toss up."
Pollster.com lists Minnesota as "toss up."
fivethirtyeight.com has Minnesota just about as blue as blue can be.
http://www.realclearpolitics.com/epolls/2008/president/mn/minnesota_mccain_vs_obama-550.html
http://www.pollster.com/polls/mn/08-mn-pres-ge-mvo.php
http://www.fivethirtyeight.com/
When 5 of 7 polls in the last month have Obama 0 to +3 in Minnesota and fivethirtyeight.com decides to put Obama up +7.9, I have to question the logic.
The Logic: fivethirtyeight.com (1) weights polls based on pollster accuracy, poll size and recentness, (2) factors in state demographics, (3) adjusts results based on trend lines, (4) runs 10,000 simulations based on historic polling data from past elections back to 1952 and based on the assumption that like states move alike, i.e. "future polling movement in states like Michigan and Ohio, or North and South Carolina, is likely to be in the same direction."
It takes a lot of outside weighting to have a projection align more with two polls as opposed to five others. It seems the outside factors are carrying significantly more weight than the polls themselves. I'll be interested to see the predictive accuracy of fivethirtyeight.com on Nov. 5th.
Looking at Minnesota, just at polls taken in September (in order):
+14, +2, TIE, +3, +2, +8, +1
The mean for Sept. is +4.3 with a standard deviation (SD) of 5. In addition to being the oldest of the Sept. polls, the +14 CNN poll falls outside the standard deviation (4.3 mean + 5 SD = 9.3). Thus, the +14 is an outlier. Factoring out the +14 CNN poll, we get a mean of +2.7.
RealClearPolitics puts Obama up +2.8.
Pollster.com puts Obama up +3.2.
fivethirtyeight.com projects Obama +7.9.
RealClearPolitics lists Minnesota as "toss up."
Pollster.com lists Minnesota as "toss up."
fivethirtyeight.com has Minnesota just about as blue as blue can be.
http://www.realclearpolitics.com/epolls/2008/president/mn/minnesota_mccain_vs_obama-550.html
http://www.pollster.com/polls/mn/08-mn-pres-ge-mvo.php
http://www.fivethirtyeight.com/
When 5 of 7 polls in the last month have Obama 0 to +3 in Minnesota and fivethirtyeight.com decides to put Obama up +7.9, I have to question the logic.
The Logic: fivethirtyeight.com (1) weights polls based on pollster accuracy, poll size and recentness, (2) factors in state demographics, (3) adjusts results based on trend lines, (4) runs 10,000 simulations based on historic polling data from past elections back to 1952 and based on the assumption that like states move alike, i.e. "future polling movement in states like Michigan and Ohio, or North and South Carolina, is likely to be in the same direction."
It takes a lot of outside weighting to have a projection align more with two polls as opposed to five others. It seems the outside factors are carrying significantly more weight than the polls themselves. I'll be interested to see the predictive accuracy of fivethirtyeight.com on Nov. 5th.