[ANALYSIS] What went wrong with the US election polls?

After Brexit and Trump’s surprise victory the credibility of opinion polls has received a drubbing. Have the polls really got it so wrong and if so why, James Debono asks

Hillary Clinton went into Election Day with a 3.2 point lead but the actual result handed the keys to the White House to her rival Donald Trump
Hillary Clinton went into Election Day with a 3.2 point lead but the actual result handed the keys to the White House to her rival Donald Trump

How wrong were the polls?

The US were faced with two three kind of polls; national polls showing how the candidates stood in the popular vote, polls in each state particularly in swing states which ultimately determine who fins and finally an assessment of different polls showing the probability of a Hillary Clinton or Donald Trump win.  

Not far off the mark

Since the US President is not elected by a majority of the popular vote but by a majority in the electoral collage based on a winner takes all basis - in which a candidate wins all the votes allocated to each one of 50 states - national polls assessing the popular vote are simply an indication on how candidates are performing. Polling site Real Clear Politics’ last survey polling average (based on results of different polling institutes) found Clinton holding a 3.2-point lead (46.8 to 43.6), down 3 points since mid-October. This was a clear indication that the election was close.  While most polls showed Clinton leading by 3 to 6 points, the LA Times poll showed Trump leading by 3 points.  

Clinton is set to win the popular vote by around 1.5%, therefore on this level the polls were not far off the mark, rightly predicting Clinton winning the popular vote. 

The only fluke was a surge in support for Clinton in October as a reaction to recordings exposing Trump’s misogynist character. But this surge had dissipated in the last week of the campaign after the FBI’s controversial decision to resurrect an investigation in Clinton’s use of her private email. In the final instance, US polls were more reliable than British polls predicting a Remain victory in the June Brexit referendum.

But it is clear that polls assessing the national strength of candidates are not a credible indication on who would win. In fact the result simply confirms the increasing discrepancy between the popular vote and the actual outcome of elections, one that makes national overall polls that simulate the popular vote less relevant to predictions over time.

Mixed verdict on state polls

Of more crucial importance were polls conducted on a state level. State polls showed Trump going into Election Day edging out Clinton in some key battleground states, including Nevada and North Carolina, leading by 3 or more points in Iowa, Arizona and Ohio. In the tightly contested swing state of Florida, Trump held a 0.2% advantage.  

He also closed the gap significantly in Pennsylvania, Colorado and Michigan, but remained down by 2, 3 and 3.4 points in the states, respectively.  

On Election Day, Trump managed to win Ohio, Pennsylvania, Michigan, North Carolina and Florida. Polls in the final weeks of the campaign definitely showed Trump making significant gains in swing states, but with the gap being so small in states like Florida, it was hard to imagine Trump to make such a clean sweep.  

For while Clinton had several paths to the White House, Trump’s strategy relied on him winning all the Rust Belt states including Pennsylvania, Ohio and unlikely ones like Michigan and North Carolina. This is what Trump did exactly on election night while also winning Florida.

Flawed predictions

This leads us to the most dangerous of all predictions made by pollsters; that on who was the most likely to win a majority in the Electoral College. 

FiveThirtyEight’s forecast gave Trump much better odds than other polling-based models. Their final forecast, issued early Tuesday evening, had Trump with a 29% chance of winning the Electoral College. By comparison, other models tracked by The New York Times put Trump’s odds at: 15%, 8%, 2% and less than 1%. 

As it turned out, Trump won largely for the same reasons that FiveThirtyEight chief pollster Nate Silver’s model had given him better odds than others i.e. the high number of undecided voters had ended up breaking in Trump’s favour, and Clinton had underperformed in the Midwest.  

FiveThirtyEight’s forecast only gave Trump a 29% chance of winning
FiveThirtyEight’s forecast only gave Trump a 29% chance of winning

Reasons why polls went wrong

1. Polling in smart phone era

The advent of mobile and smart phones has made finding accurate and random samples of voters to poll more difficult. Prior to mobile phones, the universality of landline telephones made finding random and representative samples easy, as pollsters could just pick random names out of phone books, call potential voters, and talk them through interviews. 

This method ensured high response rates among which roughly reflected the class and occupational structure of society. In contrast mobile phone numbers are not usually publicly listed, making it harder and harder to find representative samples.

Moreover demographic differences exist between people still relying on landlines and those who can only be reached on cell phones. Coupled to this are changing working and lifestyle patterns of people who are rarely at home, or only at home at times when they are least likely to be reached by pollsters. 

Various online survey methods have been used to supplement more expensive phone methods, but they often also suffer from bias and are generally considered of lower quality than other polls.

2.  Trump changed the electorate

The main reality check for pollsters is the past voting patterns of respondents who are asked whom they have voted for in previous elections.  Respondents are also asked how the voted in past elections and this gives pollsters the opportunity to assess shifts from one party or candidate to another.  Moreover this also shows how reliable the sample is.  But unlike Malta where more than 90% of the population votes, the US is a country were only half the electorate votes.  Therefore any change in the composition of the electorate like for example an increase in turnout among a particular category of voters like white rural voters in a particular region will render polling based on past electoral behaviour problematic. Trump may well have managed to get the support of voters located in strategic swing states who stayed at home in previous elections. Moreover compared to a relatively homogenous society like Malta, the US is a highly diverse country were one finds pockets of different voters living a short distance from each other. The electoral map emerging from the election was that of urban densely populated democratic enclaves surrounded by a sea of red of more sparsely populated areas. A surge in support in these areas may well explain the Trump phenomenon.

3. White rural voters were under represented

Probably polls understate the fact that Trump picked up unprecedented turnout from rural voters, This is because these voters may not even have voted in previous electoral appointments. Polls based on past votes might be ill-suited to understanding sudden changes in the electorate or the way the electorate votes. 

Trump’s focus on white rural voters, who historically had shown a fairly low propensity to vote, may have motivated them to turn out in greater numbers. Such enthusiasm is hard for pollsters to detect.  

This may also have been a factor in Brexit where the referendum was lost in rural communities and run down industrial communities. Many have described these voters as “white working class”, even if statistics which still showed Clinton prevailing among low income earners, indicate that electoral choices were more determined by the rural/ urban divide than by class.  

4. Trump voters were shy 

The discrepancy between internet polls and phone polls before the Brexit referendum in the UK has been attributed to shy leavers who were reluctant in telling pollsters their intentions over the phone.

It is possible that “shy Trump voters” didn’t want to admit their support to pollsters. However, there was no evidence of such a pattern during the Republican primaries, when pollsters experienced no such problem.  

Moreover given Trump’s strong margin among rural voters, it is hard to imagine that people whose friends and neighbours mainly backed him would be ashamed to say so themselves.  

No discrepancy between internet and phone polling was observed in the US election.  But it could well be the case that moderate Republican voters who disliked Trump but still preferred him to Clinton may have been reluctant to express this preference over the phone. But a likelier cause for underestimating Trump’s popularity in key marginal states was a higher “non-response” rate among working-class whites who backed Trump. Was this category more reluctant or unable to answer the phone? Some suspect that these voters may have deliberately refused to engage with polling associated with what was perceived as a hostile liberal media.  

5. Trump voters were more enthusiastic

But the main reason for the discrepancy between polls in swing states and the actual result may well have been that these voters simply turned out in larger numbers than Clinton voters who were less enthusiastic.  

One major let down for Clinton was that the much hoped for Hispanic wall of voters which would have stopped Trump from winning states like Florida did not materialise as widely anticipated. 

With the margin being so close in most swing states, Trump may have simply won because Clinton underperformed among her likely voters in swing states, while Trump over performed among his support base in a number of key states.

A Maltese parallel-the hunting referendum

One major polling blunder in Malta was the spring hunting referendum in 2015. MaltaToday surveys were spot on predicting the outcomes of the 2008 and 2013 general elections, the MEP elections in 2009 and 2014 and the divorce referendum but failed to do the same with regards to the referendum on spring hunting.  

Other surveys conducted by Xarabank and Misco also foresaw a majority against spring hunting. 

The main reason for the discrepancy between polls and the actual result – in which the pro-hunting camp won by a whisker - was the higher turnout in pro-hunting rural districts, and lower turnout in more urban anti-hunting districts.  

In this way the No majority in anti-hunting districts was lower than that predicted in polls and the Yes majority in pro-hunting districts higher than the No majority predicted in polls.  

As may have been the case with Brexit and the US election, the hunting lobby won the day because their supporters were keener to vote while their opponents were more complacent.

Between 2013 and 2015, eight out of nine opinion surveys held by MaltaToday, but also one by Xarabank and one by Misco International, had shown a majority of people against spring hunting. But the result showed the Yes to spring hunting winning by a small margin. 

Only one MaltaToday survey held in January had shown the Yes leading by 1 point but all other subsequent polls showed the No winning by a 5 to 7 point margin. 

The discrepancy could not even be attributed to the margin of error, as the No majority in the last survey was just beyond the +/- 3-point margin of error. 

The discrepancy between the result and polls attributed to two factors: a much higher turnout in districts supporting the hunting community, and a shift of the undecided voters towards the Yes camp in the final days. 

MaltaToday’s last survey before the referendum had correctly shown the Yes leading in fifth, sixth and seventh districts (where there was a higher turnout in the actual referendum) and no majority in the eight, ninth, tenth, tenth and twelfth district (where there was a lower turnout in the actual referendum).  

As may well have been the case with Trump victory’s in the US it all boiled down to greater voting enthusiasm among a category of voters compared to dampened enthusiasm in the opposing campaign.  

An analysis of the result by district clusters showed the yes winning by over 60% in the Labour-held, hunting country of the fifth, sixth and seventh districts – southeast and west, and in Gozo, as correctly predicted in MaltaToday surveys.

On the other hand the No won by over 60% collectively in the cluster including the eighth, ninth, tenth, eleventh and twelfth districts, which includes the urban areas north of the harbour and in the northeast.

MaltaToday surveys also showed the No leading by 67% in the cluster, which included the eight, ninth, tenth, eleventh and twelfth districts. 

But in an indication that a segment of northern voters stayed at home, in the actual referendum 62% of voters in these districts voted No. 

What MaltaToday surveys clearly got wrong was the vote of the inner and outer harbour districts represented by the first to fourth electoral districts.  

In these districts the MaltaToday survey showed the No and the Yes in a tie. In the actual referendum 55% of voters in these four districts voted ‘No’. This may be an indication that a number of undecided voters ended up voting according to party lines.

The failure by pollsters to predict the referendum was only one of many similarities with Trump's victory and Brexit.  These included a unified mainstream media whose editorial line was defied by the electorate and a regional divide between cosmopolitan urban areas and more insular rural areas.