The Nov. 6 presidential election was a big victory for Barack Obama, but it may have been an even bigger one for statistical analysts — especially Sam Wang, who did not have to eat a big bug.
In the days leading up to the election, most major media outlets had been characterizing the presidential election “too close to call” based on national polling reports. Republican Mitt Romney was reportedly buying ads in several states, hoping to upset Obama in those states and tip the Electoral College vote count in his favor.
But Wang, a Princeton University professor who had been watching statewide polls closely for months, made a bold prediction on his blog, the Princeton Election Consortium, on Nov. 3:
“A few days ago, the word was that Team Romney was buying ads in Minnesota and Pennsylvania. If he wins either of those states I will eat a bug. [If he wins] Ohio…a really big bug. And yes, I will post a photo.”
So on election night he sat, eyes glued to the television, facing the prospect of having to eat that bug. But around 9:30 p.m., when the networks called New Hampshire for Obama, Wang closed the browser tab he had opened on his computer for an edible bug website. At that point, he knew he would not lose his bet.
“I was pretty confident about not having to eat a bug all along because the state polls did extremely well,” the Princeton resident said in a post-election interview. “I never expected to have to pay off.”
Wang is among a small group of poll watchers who used sound statistical analysis to predict the likelihood of various outcomes of the election. On Nov. 3, he gave Romney just a 6 percent chance of winning, and it had never been much higher in the preceding months, by Wang’s reckoning—despite what some pollsters, like Gallup, CNN/Opinion Research and Rasmussen were reporting.
Every four years, Wang runs a blog at election.princeton.edu where he crunches the poll numbers to predict the outcome of the election. In the blog, he churns out hard quantatative analysis coupled with a somewhat flamboyant tone, bug-eating pledges and all, the end result of which is an entertaining and informative brew.
An election “thermometer” on the site gives a visual representation of the likely outcome. During the 2012 election, the gauge always stayed in Obama territory.
The reason Wang’s numbers stayed accurate while national polls fluctuated wildly throughout the fall, was that Wang didn’t use national polls to make his predictions. He only used statewide polls, aggregating the results and applying Bayesian statistical analysis — the same techniques he uses in his neuroscience research — to weed out outliers.
The end result was that he predicted 10 of the 10 closest Senate races, and every electoral vote except Florida, which he had declared a virtual tie.
Not only were Wang and the other poll number crunchers right, but their predictions of a solid Obama victory were so accurate, their critics wound up looking foolish. Wang and the other numbers gurus were the heroes of the day. Most famously among them is New York Times contributor Nate Silver, whose blog, FiveThirtyEight, got most of the Times’ traffic on election night.
Wang said he and Silver have a “friendly rivalry.”
The two have appeared on talk shows together and both are on the side of going with statistical analysis of state polls, rather than “gut feelings” or national polls to predict the outcome of elections.
However, Wang and Silver differ in the way they crunch the numbers. Silver incorporates econometrics into his polls — measures of the economy that could influence how voters feel.
Wang rejects all econometrics and only includes actual poll numbers in his calculations.
“The best information about a political race are the polls themselves, because they directly measure opinion,” Wang said. “The kind of econometric measures that Nate uses are useful mainly before the race starts. During the election, the econometrics mainly fuzzed up the picture.”
So, if poll aggregation is the superior method for predicting an electoral winner, why did no one try it before 2004?
“The number of polls only became very large starting eight years ago,” Wang explained. “This year, aggregation became more important because it was a fairly close race.”
Since Obama had a reliably large lead on John McCain four years ago, forecasting election results in 2008 was not quite as difficult as 2012.
“It was close enough that individual polls did not always give a clear picture of where things were going to go. National surveys showed a tie, and many showed Romney ahead, and those turned out to be incorrect. In such a close situation, aggregation became much more important,” he said.
Furthermore, politics has been a holdout against what is seen as geeky statistical analysis, even as spreadsheets and algorithms have become accepted in other fields, including sports. Silver, notably, first became well known for his analysis of baseball statistics.
One example of gut-based journalism was Wall Street Journal columnist Peggy Noonan, who predicted Romney would win Florida based on seeing lawn signs.
“Politics is not considered to be a domain for quantitative analysis,” Wang said. “Most pundits deal in gut feelings and information based on their experiences and connections and opinions. In a situation when things are pretty close, quantitative analysis is better than any pundit.”
Wang hopes the vindication of quantitative analysis will lead to better news coverage in the next election. He wants to see more journalists paying attention to poll averages instead of writing a fresh news story for each poll that came out, as if individual polls had significance.
“Maybe even in my dreams, it will allow us to focus on issues and what really matters and the impact of the race on people’s lives,” he said.