Obama's Real Problem: Fat Tails

by David Epstein

In the past few days the analyses of which candidate is most likely to win the general election have been, like spring, bustin' out all over. David Ignatius has a good column in the WaPo, Marie Cocco has another in RealClearPolitics, SurveyUSA has put together a state-by-state analysis comparing the two against McCain, and so on.

There are two things to note here. First, the very fact that the discussion has turned to this type of analysis is a big, big win for Clinton. Gone are the days when you heard that the superdelegates should just copy what a majority of the democratic electorate said -- now the onus is on the candidates to prove themselves most worthy of the nomination.

And this is as it should be -- when the Democratic electorate forms a consensus around a single candidate, it's proper for that candidate to be the nominee. But when the electorate is so split among two or more candidates that neither gets a majority via committed delegates alone, it's the superdelegates' job to figure out which has the best chance of winning in the general election. Borrowing from E. E. Schattschneider, parties are not democracies; they are organized teams that try to seize control of government through the means of elections.

Second, the articles coming out now, based on slicing and dicing the electorate by their demographic characteristics, all make the same crucial error: they assume that the candidate expected to receive the most number of votes is the more likely to win the election. This sounds plausible but it's wrong, as any bond trader could tell you.

To see why, instead of asking the question (a) who has the better chance of winning, ask (b) who has the bigger chance of losing. The answer to (b) will clearly be the opposite of the answer to (a), but it emphasizes more clearly the fact that what you care most about is getting over 50% of the electoral college vote, not racking up the largest majority possible.

The situation is illustrated in graphic form below. The Obama curve has a higher average than the Clinton curve, but it also has a larger area to the left of 50%. In financial terms, Obama is the high-mean-high-variance variance candidate; in statistical terms, he has fat tails. (International bankers as a recent conference described Clinton as "having a high ceiling and trading in a narrow range." That's about the highest compliment you can get out of that crowd.) And as every bond trader knows, if the price of an option falls below it's par value, it is "under water" and worthless. It's the same with candidates: you could argue that winning with a bigger majority is better than winning with a smaller majority (mandate talk and all that), but the difference pales to the difference between winning and losing.

And this is exactly the worry about Obama. No matter how wonderful he looks now, there's the lingering suspicion that  something could go wrong with him as a general election candidate. Perhaps he'll run a bad campaign (although he's run a brilliant primary campaign, thus dramatically lowering that risk). Or maybe he has skeletons in his closet that aren't easily explained away. Or maybe something will happen to suddenly make the public worry about national security, in which case all those wonderful contrasts between Obama and McCain won't look so wonderful any more.

Clinton, on the other hand, is much more of a known quantity. She probably has less of a chance of winning an overwhelming victory than does Obama, but one can be pretty sure that the bottom won't fall out of her campaign. Her stance on the war, so derided by the majority of democrats, insulates her to a large extent from any October surprises. She's everything McCain is, and more.

In sailboat racing terms, when you're ahead in the race, tack. Don't give the opponent contrasts to work with, because a sudden shift in the wind could spell disaster. The Democrats have every advantage coming into this election, and despite Obama's obvious strengths, Clinton is the candidate most likely to clinch a victory.

Comments

[Peg Welch]

I think your conclusion Mr. Epstein misses the REAL ISSUSE! What we should be concerned about is the "soon to be noted" SPLIT amongst the Dems who support these two candidates. The poll done that says the Obama people would more readily vote for Hillary is insane. And the Hillary people who support Hillary will not vote for Obama either. The longer this goes on the more apparent it becomes.....we lose in November, regardless of who the nominee is.

[David Epstein]

So once again we agree -- I'm still not sure if this is reassuring or scary. I didn't realize that I was echoing the "roll the dice" phrase used by the Clinton campaign, but I do think it's apt. Let others make the perfect the enemy of the good; as for me, I just want us to win this one, very badly.

[xyz]

So in other words, we shouldn't "roll the dice." :-) Personally, I am willing to. However, I have a pervasive feeling that the Dems could totally screw up this shot (in fact, I almost expect it...), so I definitely understand the desire to go with a known entity.

Of course, I think Obama is the better candidate (as you so astutely pointed out in one of your comments before). There's no denying that he's "high variance," though.