The post-truth world of unpredictable politics

LOOKING back at the punditry that preceded the US election, one of the few forecasters who distinguished himself was Geda, a mystical monkey who hangs out at the Shiyan Lake Ecological Tourism Park in China's Hunan province.

On Nov 3, five days before the voting, a specially organised ceremony took place at the park. Wearing a yellow T-shirt, the simian "king of prophets" was flanked by two life-size cardboard cut-outs, one of Donald Trump and the other of Hillary Clinton. He had to pick between two bananas placed next to each candidate.

After some "deliberate thought", according to the park's authorities, the prophetic primate picked the banana next to Mr Trump. He then promptly planted a kiss on the Donald's mouth.

Geda aside, just about every forecaster got the result wrong, and they were off by a mile. On the eve of the election, The New York Times' prediction, based on an aggregate of various polls was that Mrs Clinton had an 84 per cent chance of winning. Hours before the first votes were cast, betting markets put Mrs Clinton's chances at 82 per cent. The Huffington Post gave her a 98.2 per cent chance. Wall Street too forecast a Clinton victory, as reflected in the run-up in stock markets just before the vote. Indeed, even the Trump camp was convinced that Mrs Clinton would win and was reported to be despondent on the morning of voting day.

The high priest of American electoral forecasting, Nate Silver, who had predicted the results of the previous two US Presidential elections with uncanny accuracy, was off target on this occasion. However, he did, as he put it after the election, get the result "righter than anyone else", having given Mr Trump a 29 per cent chance in his final forecast. He also hedged his prediction.

On Nov 1, he wrote an article titled "Yes, Donald Trump Has A Path To Victory" pointing out that the tycoon was running very close behind in the critical Midwestern "swing" states of Michigan and Wisconsin. Mr Silver also noted that there were more undecided and third-party voters than in previous elections. As a result, he said, this time his model was not the normally symmetrical bell shaped curve, but a bell curve with "fat tails" (with the lips of the bell more flared), which points to more voters than usual at the extremes, raising the chances of an unexpected result.

The US election is only the most recent example of pollsters getting it wrong. We saw the same thing with the Brexit vote in June. Right up to voting day, pollsters (and markets) had the "Remain" camp slightly ahead. In the event, the "Leave" camp won the vote by four percentage points (52 per cent versus 48 per cent).

Going further back, pollsters overrated the enthusiasm of Scots for independence from the United Kingdom, predicting a photo-finish during the Scottish referendum of September 2014. The "No" camp won the vote by more than 10 percentage points. Pollsters also failed to predict the landslide victory of British prime minister David Cameron in the general elections in May 2015 and a similarly sweeping victory by Indian Prime Minister Narendra Modi's Bharatiya Janata Dal party a year earlier.

One clear trend that seems apparent is a rising populist, nationalist and nativist trend. In many countries, fringe parties are eating into the vote banks of the centrist establishment. Besides the US where significant numbers of voters have evidently been seduced by "Trump-ism", this trend is rampant in Europe, where maverick right-wing politicians are riding high: Marine Le Pen in France, Victor Orban in Hungary, Beppe Grillo in Italy, Geert Wilders in the Netherlands and Frauke Petry in Germany, among others.

On the Left too, we have seen previously fringe parties doing well, such as Syriza in Greece and Podemos in Spain. In the UK, the Labour party leader Jeremy Corbyn is arguably further to the left of the political spectrum than any of his predecessors in recent memory. And there was Nigel Farage who came from nowhere to lead the anti-immigration UK Independence Party and was a key champion of Brexit.

Many explanations are on offer for these developments: the great recession of 2008, from which recovery has been agonisingly slow where it has happened, which is not everywhere; double-digit unemployment - sometimes as high as 30 per cent - in Europe; rising economic inequality; a drug epidemic in the US for which Mexican drug cartels are high on the blame list, and in recent years, a flood of immigration, partly as a result of conflicts in Iraq, Syria and Afghanistan.

The question though, is why voters' reactions to these developments were not accurately captured by pollsters. And why did political preferences previously considered "fringe" or "extreme" garner sufficient support to win elections?

In the post-mortems of elections where pollsters got it wrong, a number of answers to these questions have emerged.

One is the presence of large numbers of undecided voters that Mr Silver detected. "Pollsters simply can't do much about voters who make up their minds only after the survey is completed," he noted.

Or maybe they can. Anthony Wells, research director of yougov.com, a UK-based research and polling organisation, suggested that there need to be models for reallocating "don't know" voters: "You can't vote 'don't know' in an election," he pointed out. "So those responses need to be reallocated to one of the voting options."

In the case of the Brexit vote, the adjustments that were made were in favour of the remain camp - which rendered the polls less accurate. In the US too, it turned out that a lot of the "don't knows" ended up voting for Mr Trump, but were not allocated to him by the pollsters.

There's also the possibility that the "don't know" voters are actually "don't want to tell" voters. Some observers noticed a phenomenon that could be called "Trump shyness" and "Brexit shyness": There were Trump and Brexit supporters who did not want to be identified as such. Ivor Crewe, Master of University College Oxford and a specialist on polling suggested that from his experience, people sometimes say "don't know" because they support an unfashionable view (a pro-racist or bigoted position, for instance) and don't want to admit it openly, even to their family or their friends - and also to pollsters.

In countries where voting is not mandatory, turnout is another critical issue; even if voters accurately reveal their preferences, this is irrelevant if they don't actually vote. The more inaccurate the assumptions about turnout, the more off-target the polls will be. It seems that both in the Brexit vote and the US election, turnout played a key role.

In the former, the turnout among young people (who were largely "Remain" supporters) was lower than expected, while the more pro-Brexit 50-plus crowd came out to vote in droves. In the US election, black voters did not turn out in anywhere near the same numbers as they did when Barack Obama was running for president, and Hispanic voter turnout dropped sharply after early voting, to Mrs Clinton's cost. The turnout numbers could have been affected by the polls themselves. Polling specialists believe that the closer the expected result, the greater the turnout on both sides. But if one candidate is expected to win easily, his or her supporters don't turn out in as large numbers. Thus the 80 per cent-plus probability attached to a Clinton victory in the mainstream media just before the election could have kept many of her supporters from going to the polling booths on voting day on the grounds that "it looks like she's going to win anyway, so why bother".

Some of the mainstream media and its pollsters might also have got their sampling wrong and their questions slanted - sometimes deliberately. A lot of polls are not objective exercises designed to glean honest answers from a true representative sample of the population; they are designed to influence perceptions - to suggest, for instance, that a particular candidate is so far behind that he cannot win, or that one candidate is obviously unqualified.

Some of the tricks include a deliberate oversampling of certain demographics deemed favourable to a candidate (for instance, more Hispanic voters and fewer gun owners for a Democratic candidate in the US). Questions are also slanted. In the race for the Democratic nominee, for instance, a voter in the state of Nevada was asked this apparently innocent, but actually loaded question by a pollster: "if I were to tell you The New York Times expressed serious reservations about Bernie Sanders being able to work with Congress and described him as 'divisive' - would you now be more likely or less likely" to vote for Mr Sanders, or has your opinion remained unchanged?

*****g idea what's going on," he said, to The Hollywood Reporter. "It's a closed circle of information from which Hillary Clinton got all her information - and her confidence. That was our opening[/EXPLETIVE]."

He may have a point; a lot of people live in echo chambers where they interact mainly with people who think like them.

The social media reinforces this. An interesting analysis on the publishing platform Medium by Tobias Rose-Stockwell suggests that the way we consume information may change how we think about politics. Most Americans (and a growing number of Singaporeans) get their news and views from their social media feed. But that feed is curated by us, the users, with the help of Facebook and Twitter. We select certain friends, people and news sources to follow and interact with. We might not agree with them about everything, but they don't make us uncomfortable. We would probably not interact with Neo-Nazis or religious fanatics - we filter them out, with the help of Facebook's algorithm. Likewise, they would have their own groups and news sources, and they would filter us out. So in the self-selecting world of social media, we tend to see fewer views that make us uncomfortable and more views that reinforce our own. Unwittingly, we insulate ourselves in echo chambers.

These echo chambers can be global, and so ideas spread across borders. Trump supporters took a lot of inspiration from Brexiters (whose victory was hailed by Mr Trump). Returning the favour, the pro-Brexit champion Mr Farage, in turn, came to the US to cheer the Donald on. France's potential presidential candidate Ms Le Pen, whose ideas - for example on trade and immigration - are not dissimilar to those of candidate Trump, has taken inspiration from Mr Trump's victory.

Ideologies feed on each other, across borders. Writing in Foreign Affairs, the political scientist Mark Blyth (who correctly predicted the Brexit vote) identified this as the "Galton problem" named after the statistician Francis Galton, who pointed out that many events are treated as being "independent" when they actually may not be. Think Brexit, the rise of right-wing parties in Europe, the surprising success of Bernie Sanders - a self-confessed socialist, a rare breed among US politicians - and then the US election result. Pollsters might have missed the possible contagion effects between these events.

The echo chambers enabled by social media were amplified by fake news. People were informed, for instance, that Pope Francis had endorsed Mr Trump, that Mrs Clinton sold weapons to ISIS and that Ireland was accepting American refugees fleeing from the prospect of a Trump presidency. Some of these (most of them pro-Trump) were more widely shared on social networks than even the most popular election stories in the mainstream media. Brexiters also put out fake news to bolster their case; most infamous was their claim that European Union membership cost the UK £350 million (S$622.2 million) a week, a falsehood that was repeated ad nauseam - even emblazoned on a campaign bus - and widely believed, even after it was debunked.

The late US senator Daniel Patrick Moynihan who served in the cabinets of four US presidents liked to say: "You are entitled to your own opinion, but you are not entitled to your own facts." But in our post-truth world, this dictum is disregarded. Reality is not what things really are, but what people perceive and want them to be - and if necessary, they create their own "facts". This is a fact of which politicians and pollsters - and we too - must be mindful.