How'd the polling go so, so, so wrong?
LatestIf you looked at the forecasts from across the media (and even the RNC’s own model) when polls closed Tuesday, Hillary Clinton had nothing to worry about. Many poll aggregators, like the Princeton Electoral Forecast and the HuffPollster, showed Clinton win probabilities of 98-99%.
Well. That’s not what happened.This New York Times chart sums up the miss:
The polling data, such as it was, turned out to be as much fiction as fact. All the little charts and interactive gizmos and clever animations and forecasts, which showed Hillary Clinton with a comfortable lead, were wrong. Donald Trump handily won the electoral college vote by taking states (Wisconsin, Michigan) that almost no professional forecaster thought were in play.
At the citizen level, sociologist Nathan Jurgenson made a trenchant point: While the right might crave truthiness, the left loves its “factiness.”
“Perhaps we need to start articulating a left-wing version of truthiness: let’s call it ‘factiness.’ Factiness is the taste for the feel and aesthetic of ‘facts,’ often at the expense of missing the truth,” Jurgenson writes. “From silly self-help-y TED talks to bad NPR-style neuroscience science updates to wrapping ourselves in the misleading scientism of Fivethirtyeight statistics, factiness is obsessing over and covering ourselves in fact after fact while still missing bigger truths.”
And that bigger truth is that Donald Trump’s white nationalist message appealed to many more white people than were willing to acknowledge it in the polls. And Hillary Clinton’s diverse coalition of supporters did not turn out in the same numbers as they did for President Obama in 2012, to say nothing of 2008. (Yes, the Voting Rights Act has been gutted since then, leading to fewer polling places in, particularly, largely black areas.)
Data scientist Sam Wang, who runs the Princeton Electoral Forecast, which pegged a Clinton win at a 99% probability, issued a stark mea culpa. “The entire polling industry – public, campaign-associated, aggregators – ended up with data that missed tonight’s results by a very large margin,” he wrote. “There is now the question of understanding how a mature industry could have gone so wrong.”
But why were the polls so off in one direction? Maybe it’s something systemic about who the pollsters can reach or even who is willing to talk to pollsters, given their (odious) association with the media.
In 2008, many Democrats worried that Barack Obama would be subject to the so-called Bradley effect, “an undetectable tendency in the behavior of some white voters who tell pollsters that they are ‘undecided’ when in fact their true preference is to vote against the black candidate.” He wasn’t.
But Hillary Clinton does appear to have encountered just such an “undetectable tendency.” The white people in the rust belt and south broke hard for Trump, and in so doing, broke all the carefully tabulated polls and meticulously computed forecasts.
Should someone (anyone, really, outside the Trump camp) have seen this coming? Nate Silver, in his last election forecast, diagnosed the proximate problem, actually. The state polls missed, and the way they missed was correlated: Everything came up Trump. He considered exactly this scenario, while assigning it a fairly low probability.
“But if there’s a 3-point error against Clinton?” Silver wrote. “That would still leave her with a narrow lead over Trump in the popular vote — by about the margin by which Gore beat Bush in 2000.”
There were, in hindsight, some signs that Trump might have some hidden supporters. A May 2016 New York Times editorial pointed out that Trump was doing better in online polls than in telephone surveys, especially among more educated voters.
Now, the trend was by no means completely consistent. There were times during the election when Trump performed better in live-caller polls than in online ones.
While dismissing the possibility that these voters would impact the election, the polling firm Morning Consult noted the online/telephone discrepancy. “Trump’s edge over Clinton online instead of in phone polling is especially pronounced among people with a college degree or people who make more than $50,000,” they wrote in a November 3 post. “Just as we found in our December study, more-educated voters were notably less likely to say they were supporting Trump during a phone poll than in an online survey.”
In the end, the poll miss was big, but the final vote spread — nationally and in the key states Clinton had to win—was not. However, where the surprises did come in Wisconsin, Michigan, and Pennsylvania, the key was exactly the demographics that Morning Consult points out. Maybe the warning signs were there. Maybe the media narratives should have been more diverse and open to the possibility of the quiet, Rust Belt white, middle-class Trump voter.
Certainly, now, reading the Facebook-ready headlines about Trump’s online polling numbers that started rolling in at the beginning of the summer, you can see a tantalizingly missed thread in the election coverage: