Trump, Clinton, and the failures of election forecasting

By | November 13, 2016

Election season has come and gone. Donald Trump pulled off what has been repeatedly characterized as a “stunning upset” over Hillary Clinton. The web has reacted with intense postmortem analysis about the failures of election polling and forecasting. But was it really that surprising?

Several polls conducted in the lead up to the election reported on a virtually deadlocked race, all within any reasonable margin of error:

Quinnipiac University reported on the situation in key battleground states (Nov 2)

Democrat Hillary Clinton’s October momentum comes to a halt as she clings to a small lead in Pennsylvania, while Republican Donald Trump moves ahead in Ohio, leaving Florida and North Carolina too close to call.

Probability forecast models on the other hand were systematically off-mark and predicted Clinton far ahead just the night before the election:

  • New York Times Upshot: Clinton 84%, Trump 16%
  • FiveThirtyEight: Clinton 66.9%, Trump 33%
  • PredictWise: Clinton 89%, Trump 11%

During the final rallies held by both candidates the night before election day, each candidate expressed positive sentiment about their chances of winning. Here is a revealing segment from Trump’s penultimate rally in New Hampshire (8pm on Nov 7):

We are going right after this to Michigan, because Michigan is in play… The polls just came out: we are leading in Michigan; we are leading in New Hampshire; we are leading in Ohio; we are leading in Iowa; leading in North Carolina; I think we are doing really, really well in Pennsylvania; and I do believe we are leading in Florida.

In the meantime, according to the New York Times:

Mrs. Clinton’s campaign was so confident in her victory that her aides popped open Champagne on the campaign plane early Tuesday.

Either way, each candidate and their popular base was clearly happy to live in their own reality right down to the wire. Some personal takeaways:

  • The law of total variance for polling. Uncertainty about polling metrics is best understood not by not only assessing variance within a single poll or averaging multiple polls, but studying the variance across multiple polls.

  • Forecasting is a complex empirical problem. Formal statistical methods are unable to accurately quantifying uncertainty around the non-statistical and psychological effects that ultimately influence a “swing” voter’s decision at the ballot box.

  • A common reaction to the election was for people disappointed with the outcome to blame journalists or cancel their newspaper subscriptions. Well-informed voters should not derive their information solely from a few sources on social or national news media. They are better off surveying competing analyses and projections from across the spectrum, independently of their political views.

Leave a Reply

Your email address will not be published. Required fields are marked *