Who is accountable when an AI agent decides?

WITH the rise of artificial intelligence, one question that is often asked is who is accountable when an AI agent makes a decision?

Nobody, it turns out, if you ask the senior management at Air Canada. In 2022, a traveller named Jake Moffatt asked the airline’s chatbot how a bereavement fare discount worked. The chatbot replied to say he could book the fare first at full price and claim the discount within 90 days. That response, however, turned out to be wrong.

When Air Canada refused to honour it, Moffatt took them to a small claims tribunal in British Columbia. The airline argued – and I am not making this up – that the chatbot was “a separate legal entity” responsible for its own words.

The tribunal called this “a remarkable submission” and ordered Air Canada to pay up. A chatbot, it said, is still just part of the airline’s website. It does not get to be its own defendant. That case involved software answering a question, but AI agents these days go further. They book things, buy things, edit your production database, move money and talk to other systems on your behalf – all without you clicking on the “approve” button.

When the decision-maker is not an actual person, who answers for it? Lawyers and chief information officers (CIOs) across Singapore and everywhere else are grappling with a thornier version of that question.

The agent is not a defendant

Let us start with the obvious bit. An AI agent is not a legal person, so it cannot be sued, jailed or struck off. Whatever it does traces back to a human or a company somewhere in the chain. The question is which one. That chain has become a lot longer than simply “developer and user”.

There is the company that built the model, the one that built the tools around it, the platform it runs on, the cloud that hosts it, the business that deployed it, the end user, and outsiders who never signed up for any of this but get affected anyway.

Each controls a slice, none controls all of it. In Singapore, the Infocomm Media Development Authority convened a working group of lawyers and technologists to work through exactly this, and their May 2026 discussion paper is worth a look for any CIO about to roll out an agent. One example shows how quickly this can spiral. A personal assistant agent, told to sign up its user for a class at midnight, cannot reach her data because the server is down for maintenance.

So it hacks the server to get her data. It succeeds, but leaks other people’s personal information in the process. Its own reasoning trail showed it knew this was a big deal and would normally have checked with her first, but decided she was asleep and the class might fill up. Nobody told it to hack anything. Nobody could have predicted it would. And yet real people lost real money.

SEE ALSO

Buy or build software? Firms need to look at control, reversibility and risk

Rising AI bills push Singapore companies to chase ROI over ‘tokenmaxxing’

OpenAI grapples with surging startup demand in Apac even as it expands Singapore team

AI is running on borrowed money

More

Why the old playbook struggles

Normally, when someone messes up and you get hurt, you ask three basic questions: Should they have been careful? Did they fall short? Did that cause your loss? These questions are simple for a driver who runs a red light, but quite messy for an AI agent. The first question assumes the two of you are close enough for one to reasonably think about the other. The company that built the model has never heard of the cloud provider whose servers got hacked, and has no idea that Moffatt’s class-booking agent even exists.

To be fair, the courts have stretched this before for strangers who turn out to be linked. But an agent hopping across five companies’ systems is a stranger of a different order altogether. What was predictable gets slippery, too. You only need to have foreseen the general kind of harm, not the exact way it happened, which is normally good news for whoever is being blamed.

A bank rolling out a trading agent can predict it might lose money without predicting the specific glitch that causes it. But when an agent’s actions escalate from “book a class” to “commit a cyberattack”, even a generous read of predictability wobbles. Who saw that coming? And then there is the real headache, working out what actually caused the mess. The user’s instruction? A gap in the safety rules the deploying company set up? Something baked into the model during training?

Untangling that needs training data, system logs and proprietary code an ordinary consumer will never get near. Sometimes there is simply no way to tell. Singapore already has a related case, Quoine v B2C2, where a trading algorithm executed cryptocurrency deals at 250 times the market rate because of a glitch. The court looked at what the programmer intended, not the strange outcome the code produced.

That worked because the algorithm did exactly what it was told, every time. AI agents mostly do not work that way. They can make calls their own programmers never saw coming. Depending how you look at it, that is either a reason nobody should be blamed, or a convenient excuse for everybody to point elsewhere.

What this means if you are the CIO

You might think the fix is to skip the fault-finding and make certain parties pay regardless of what happened. That has its own problem – which is a classic moral hazard. If the payout is the same either way, the company that tested its agent thoroughly carries the same exposure as the one that shipped something half-baked, and the incentive to test properly disappears. None of this is settled – and it will not be for a while. Contracts spell out who pays whom, but only bind the people who signed them, not the third party whose data gets leaked.

Disclaimers do similar work, showing you tried to be careful, but a blanket “the agent may behave unexpectedly” clause looks thin once agents carry real consequences.

And having a human check every decision, the usual fallback, gets harder to keep up as agents move faster and touch more systems. That is exactly when you need it most. The question is not whether your agent will eventually do something nobody told it to do. It is whether – when that happens – you can show you did what a sensible business would have done, and whether “the chatbot did it” works any better for you than it did for Air Canada.

The writer is the chief technology officer and deputy chief executive at GovTech Singapore