AI won’t replace the president. It might govern our decisions

HOW reassuring is it to hear “I asked Gemini” from someone defending their position amid a political furore? If you are the president of South Korea, apparently not very.

President Lee Jae-myung kicked off an outcry after posting on social media platform X that he had checked with Alphabet’s Google artificial intelligence tool to rebut critics who said inviting officials from two investigative bodies to a Cabinet meeting could undermine their independence.

The blowback revealed an almost comedic stage of the AI boom, fuelled in equal measure by suspicion and hype.

Should it be so shocking that the president of a nation betting on AI as a driver of future growth actually uses it? If anything, Lee is behaving like a typical South Korean.

About two-thirds of employees at large companies use the technology at work, based on a Korea Chamber of Commerce and Industry report released this year.

Other surveys put the figure even higher.

South Korea has recorded the largest gains in AI diffusion over the past year, according to Microsoft’s index. Ironically, a 37-country Pew survey found that the country also has some of the highest levels of anxiety about AI taking jobs.

Even before taking office, Lee made headlines for saying he relied on ChatGPT more than his aides for data research and analysis. (Though, as someone who regularly consults several tools, why Gemini this time is a fair question.)

His office said that he was merely verifying facts with AI, not asking it how to govern. That may be true, but it has not quelled the critics.

The ultimate irony is that the critics are now turning to Gemini, too. Shortly after Lee’s post, some politicians fired back with their own Gemini responses that appeared to differ from his.

That is a known weakness of large language models.

While rates of wildly inaccurate responses, or hallucinations, have fallen as models have improved, research shows that outputs can still vary substantially depending on how a prompt is framed.

That is worth keeping in mind before asking a chatbot to mediate a political argument.

Yet, one of the loudest lines of attack in this saga, echoed by many online commentators and a former Reform Party leader, was simple: If the president is going to govern by asking Gemini, why not just replace him with AI?

Lee said he was merely checking facts, not outsourcing decisions. But the jab struck a nerve.

The duelling Gemini screenshots that have consumed Korean social media may look absurd, but they also expose the limits of public trust in a technological revolution racing ahead of the rules governing it.

When the machine does the thinking

And beneath the noise lies a harder question: How much of a leader’s judgment are we willing to outsource to machines?

A common refrain from the tech industry is that as AI takes over tedious work, human judgment will become more valuable. What we might not realise is just how easily that decision-making can be shaped by machines, too.

Even experts can be heavily influenced by AI, new Anthropic research found. In an experiment, professional analysts working on a realistic statistical task went along with an AI agent’s randomly assigned “lean” on key analytical choices 86 per cent of the time.

When asked to explain their preferences, 77 per cent of analysts used the agents’ own arguments, sometimes verbatim, while only 15 per cent acknowledged its influence.

That is harder to catch than an outright factual error, such as a tool getting a number wrong. Judgment calls often have no single correct answer, so the work can look perfectly sound whichever option is chosen, the paper argued.

“Asking experts afterwards may not help either, because they may not notice the agent’s influence,” said the Anthropic report. From the White House to the boardroom, we should worry as much about AI coming for our decision-making as our jobs.

There are several ways to push back. The researchers suggest designing agents to present competing options, asking users for their own views before offering advice and having AI sound less certain.

But none is an easy fix, and some might defeat the point of turning to AI in the first place.

Another recommendation from the world-leading AI lab? Institutions “may want to protect human skills and oversight for reasons of safety or resilience”.

How nice. Anthropic is right, but that problem gets thornier when it comes to training the next generation.

If people learn their professions with machines making more of the tougher judgment calls, they may never develop the discernment we are counting on humans to retain.

As concerns about AI-driven deskilling mount, I am less worried about experts suddenly losing their abilities than about entry-level workers never acquiring them.

Some researchers call this gradual handoff of deciding to a machine “disempowerment”. It is the slow, insidious erosion of human agency.

There are myriad ways that AI can make governing more efficient. Using a chatbot to research how to interpret the law is not the same as handing over the executive branch.

And, at this point, I would be more suspicious of someone in office denying their use of these tools. Lee is not the first world leader to talk about using AI, and he certainly will not be the last.

To his credit, he disclosed what he was doing, which is more than can be said for others. But in the future, disclosure will only go so far if users themselves cannot recognise how much a machine is swaying their judgment.

Ultimately, where assistance ends and influence begins is becoming harder to distinguish. As AI moves deeper into governments far beyond Seoul, the line may disappear before we decide where to draw it. BLOOMBERG