What do Nate Silver, Darryl Holman, Drew Linzer, and Sam Wang all have in common? They all use statistical methods to forecast elections, especially presidential ones. Their models all tend to say the same thing: the odds are pretty good that Obama is going to win. Yet they often make different predictions about the number of electoral votes that, say, Obama will get, and about the probability that Obama would win if an election were held right now.

For example, as of right now, Silver predicts 294 electoral votes to Obama with 3 to 1 odds of an Obama win. Holman predicts an average 299 electoral votes with 9 to 1 odds of an Obama win. Wang predicts a median 291 electoral votes, also with 9 to 1 odds of an Obama win. Linzer predicts a whopping 332 electoral votes and doesn't report the probability of an Obama win.

I contacted each of those men to request access to their electoral vote probability distributions. So far, Sam Wang and Darryl Holman have accepted. Drew Linzer declined. Nate Silver hasn't answered, likely because his mailbox is chock full of fan and hate mail.

Wang and Holman now both offer their histogram of electoral vote probabilities on their respective web pages. I went and grabbed these discrete probability distributions and did what a good, albeit naive model averager would do: I averaged the probability distributions to come up with a summary probability distribution (which, by the way, still sums to one).

This method makes sense because, basically, these guys are estimating 538 parameters, and I'm simply averaging those 538 parameters across the models to which I currently have access because I currently have no reason to think they are much different in predictive power (although later on the method could be extended to include weights).

From the aggregated electoral vote distribution, I calculated the mean, median, 2.5th percentile, and 97.5th percentile of the number of electoral votes (EV) to Obama. I also calculated the probability that Obama will get 270 EV or more, winning him the election.

Mean EV: 296
Median EV: 294
95% Confidence interval: 261, 337
Probability Obama wins: over 90%

So 9 to 1 odds Obama wins. Something like 294 or 296 electoral votes.

I'd love to see what happens if I put Nate Silver into the equation. Obviously, it will drag the distribution down. I might look into modeling weights at that point, too, because both Holman and Wang predicted the electoral votes better than Silver, and I believe Wang did a slightly better job than Holman, although I forget.

This was originally posted at Malark-O-Meter.

#### Tags

EMAIL TO A FRIEND X
You must add at least one tag to this diary before publishing it.

Add keywords that describe this diary. Separate multiple keywords with commas.
Tagging tips - Search For Tags - Browse For Tags

?

More Tagging tips:

A tag is a way to search for this diary. If someone is searching for "Barack Obama," is this a diary they'd be trying to find?

Use a person's full name, without any title. Senator Obama may become President Obama, and Michelle Obama might run for office.

If your diary covers an election or elected official, use election tags, which are generally the state abbreviation followed by the office. CA-01 is the first district House seat. CA-Sen covers both senate races. NY-GOV covers the New York governor's race.

Tags do not compound: that is, "education reform" is a completely different tag from "education". A tag like "reform" alone is probably not meaningful.

Consider if one or more of these tags fits your diary: Civil Rights, Community, Congress, Culture, Economy, Education, Elections, Energy, Environment, Health Care, International, Labor, Law, Media, Meta, National Security, Science, Transportation, or White House. If your diary is specific to a state, consider adding the state (California, Texas, etc). Keep in mind, though, that there are many wonderful and important diaries that don't fit in any of these tags. Don't worry if yours doesn't.

You can add a private note to this diary when hotlisting it:
Are you sure you want to remove this diary from your hotlist?
Are you sure you want to remove your recommendation? You can only recommend a diary once, so you will not be able to re-recommend it afterwards.
Rescue this diary, and add a note:
Are you sure you want to remove this diary from Rescue?
Choose where to republish this diary. The diary will be added to the queue for that group. Publish it from the queue to make it appear.

You must be a member of a group to use this feature.

Add a quick update to your diary without changing the diary itself:
Are you sure you want to remove this diary?
 Unpublish Diary (The diary will be removed from the site and returned to your drafts for further editing.) Delete Diary (The diary will be removed.)
Are you sure you want to save these changes to the published diary?

#### Comment Preferences

• ##### Tip Jar(8+ / 0-)

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

• ##### you need to correct(1+ / 0-)
Recommended by:
Clem Yeobright

The odds aren't 9-1 if they were he'd be a huge longshot.  Try 1-9 etc :)

• ##### No.(3+ / 0-)

No. The odds are 9 to 1. The way you express odds is by dividing the probability that the event will happen by the probability it won't happen, and then expressing it with an intuitive ratio. For example, the odds against a random day being Sunday are 6:1. The odds for it being Sunday are 1:6. Likewise, the odds of Obama winning are 9:1. The odds he doesn't win, 1:9.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### So when a horse goes off at 50 to 1(0+ / 0-)

Wanna go out to the track with me tomorrow?

Am I right, or am I right? - The Singing Detective

[ Parent ]

• ##### Nope.(1+ / 0-)
Recommended by:
Dream It Real

You misunderstand. That is the probability if an election were held today. Obviously there is a lot more uncertainty in the result of the November 6th election. But, as of now, it looks quite good for Obama. If you don't believe me, then never look at another election poll again, because they are worthless, too.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### No, not really(0+ / 0-)

The odds of an random day being Sunday are not 6 to 1, they are 1 in 7, or 1:6.
The odds against a day being Sunday are not 1 to 6, they are 6 in 7, or 6:1.

A 9 to 1 shot means you have a 1 in 10 chance of winning.
A 1 to 9 shot means you have a 9 in 10 chance of winning.

• ##### I did the first part wrong.(0+ / 0-)

Being Sunday: 1 in 7 or 6 to 1.
Not being Sunday: 6 in 7 or 1 to 6.
Too much wine.

• ##### No.(1+ / 0-)
Recommended by:
Dream It Real

Go back up. Read what I wrote. Realize we're saying the same thing in different format.

The odds of an event happening are the probability of that event occurring divided by the probability of that event not occurring. If Obama has 90% probability of winning, that means he 10% probability of losing (the probability he doesn't win), this 90/10 odds, or 9/1, or 9:1, or 9 to 1.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### My predictions using Excel and state polling (1+ / 0-)
Recommended by:
Brash Equilibrium

are similar and have not changed the past week. Obama winning with 290 to 248 for Romney.

"We must be the change we wish to see in the world" - Gandhi
"The test of our progress is not whether we add more to the abundance of those who have much; it is whether we provide enough for those who have too little" – FDR

• ##### Dag nabbit, I'm trying to upvote! It's not workin!(0+ / 0-)

Way to carry out your own original polling research. Could you describe your process?

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### It's cool that you're trying,(0+ / 0-)

and we data heads will enjoy watching.

I know that a few bad breaks could change everything, and it is entirely possible that all of the meta-analytic models will be fundamentally wrong.

Strange things happen the last 10 days of a presidential campaign, you know?

I try to remember the cautionary words:

"you and your arithmetic, you'll probably go far"

• ##### Do the others do Monte Carlo simulations?(1+ / 0-)
Recommended by:
Xapulin

Nate does publish his histogram every day just under his summaries ...

Am I right, or am I right? - The Singing Detective

• ##### True, but...(0+ / 0-)

...he doesn't publish the exact probabilities in each bin. That's what I need.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### Until Nate answers, try this(0+ / 0-)

Assume any state not listed in his "swing state" listing as 100% for R or D, depending on red/blue color on the map.

Use the probabilities listed for swing states.

Run simulations

See if your histogram looks similar to his.

If it does, this approximation is probably good enough.

If not, go to 99%, 98% etc on the non-swing-states until it looks about right.

I think though, that what I just described is close enough as to make no difference.

• ##### Good idea, but...(0+ / 0-)

I'd rather use the actual model output. That said, I'll do this if push comes to shove.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### I don't see the point in making such effort.(0+ / 0-)

This reminds me of the proverbial snake chasing its own tail - - it doesn't really get you anywhere.

You'll end up with models of perhaps greater precision, but based on incommensurable data - putting apples, oranges, pears, and berries into a kind of fruit salad - perhaps tasty but you'll have to do it again every day, and the results will change from day to day, and what will you really have proved in the end if your meta model is the best predictor?

Will the same equation be most accurate in 2014?  2016?  It's a matter of dynamics, in the often whimsical realm of social & human behavior (non-linear dynamics, stochastics).

• ##### I do.(0+ / 0-)

In the long run, a properly averaged model should perform better than the models of which it is comprised. If I continue this project for many years, updating my model weights accordingly, my averaged model will be very very precise....but what is more important than precision is accuracy

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### As for Monte Carlo(1+ / 0-)
Recommended by:
N in Seattle

Darryl Holman does Monte Carlo simulations, Sam Wang analytically calculates the probabilities like a boss, and Drew Linzer uses a hierarchical Bayesian forecasting method.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### There is a basic flaw in combining(0+ / 0-)

data using different methods - not only different mathematical procedures, but more importantly, how poll data are selected for inclusion.

One way, as you're doing, is to hope that the law of large numbers or law of averages will cause these problems to become inconsequential.  But they are not.

Polling firms introduce systematic bias into the data, and this can result in shifting the entire distribution % pts in either direction (usually to the political right, but depending on whether LV or RV are used).

Nate tries to control for house effects, and he also has complex regression models not used by others.  Not sure he's most accurate, but he's doing something important, more like science & logic combined than pure statistics.  But his house effect controls are less than perfect, as admitted.

BUT WE CAN AVERAGE THE NUMBERS IN OUR HEADS IF WE HAVE THE MEAN FOR EACH METHOD.  And the numbers aren't far apart.  NONETHELESS, THE AVERAGE OR EVEN THE RANGE COULD BE INACCURATE IN PREDICTING FINAL RESULT.

Why?

Because to model an election, one is modeling a dynamic process, not a static one.

So the models of behavior today are not going to predict what happens in a week or two - - as we have seen, even a Senate candidate (Akin, Mourdock) can affect polling, or in 2004 the "Bin Ladin Tape" appears to have had a huge effect (Kerry was ahead in the polls on Friday, but polls shifted immediately after the Bin Ladin Tape was aired nationally.

So, most importantly, it is the dynamics that are missing from the meta-meta-analysis.

One approach to this problem would require comparisons of data models and outcomes over time, across multiple elections, and even then the findings would not generalize perfectly, as no two elections are alike.

• ##### Actually...(0+ / 0-)

...the fact that the different models use different methods and make different assumptions, and yet none of them is probably quite right, is precisely the reason why I am interesting in averaging them. What do they have to say in aggregate? That is a useful question, and the answer is useful.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

I know that anybody can get to Silver's stuff.

I, for one, know exactly where to find Holman's latest work (and his explanatory FAQ), because I blog with him at HorsesAss.org, as well as quaff some brews with him weekly at DrinkingLiberally.

But I don't have much knowledge of what Wang and Linzer do, or where they do it.

Labor is prior to, and independent of, capital. Capital is only the fruit of labor, and could never have existed if labor had not first existed. Labor is the superior of capital, and deserves much the higher consideration. -- K.Marx A.Lincoln

• ##### How?(0+ / 0-)

Can you link me to where I can get the equivalent to Holman's data, which I requested from him, and which he now graciously links under the EV histogram, because he is a beard-wearing beauty of a man?

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### Oh, I see what you're saying now.(0+ / 0-)

Yes, I'll link to them all in the OP.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### if Wang did better than Holman in 2008...(0+ / 0-)

...then he got it exactly right.

Darryl predicted 364-174 against the actual 365-173 ... because he didn't parse out the individual EVs in Nebraska and Maine.

Labor is prior to, and independent of, capital. Capital is only the fruit of labor, and could never have existed if labor had not first existed. Labor is the superior of capital, and deserves much the higher consideration. -- K.Marx A.Lincoln

• ##### YUp(1+ / 0-)
Recommended by:
N in Seattle

And it's differences like these that, in the longrun, would factor into model weights.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### Romney hasn't been able to move the (1+ / 0-)
Recommended by:
Brash Equilibrium

predictions of Obama winning with 290 to 294 electoral college votes for the past month meaning it's going to be a tough climb for Romney to catch up.

"We must be the change we wish to see in the world" - Gandhi
"The test of our progress is not whether we add more to the abundance of those who have much; it is whether we provide enough for those who have too little" – FDR

• ##### Amen!(0+ / 0-)

Yes, and my averaged model corroborates that statement!

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

• ##### Sam Wang in 2008(0+ / 0-)

I believe Sam Wang predicted electoral college results with the most accuracy in 2008, he was only off by 1 electoral vote.  http://election.princeton.edu/...

• ##### And information like that..(0+ / 0-)

..would factor into model weights later on.

Brash Equilibrium /brASH ēkwəˈLIBrēəm/ Noun: a state in which the opposing forces of snark and information are balanced

[ Parent ]

Subscribe or Donate to support Daily Kos.