Anthropic’s AI models hack three organisations during tests

[WASHINGTON] Anthropic said that its artificial intelligence models breached three organisations during cybersecurity tests that went awry, a little more than a week after its chief rival, OpenAI, disclosed a similar incident.

Anthropic said in a blog post on Thursday (Jul 30) that it made the discovery after performing a review of its own cybersecurity tests, following OpenAI’s announcement of a breach.

In both the OpenAI and Anthropic tests, the AI models were able to access the Internet from within testing environments that should have been sealed off, according to Anthropic’s blog.

The company said it reviewed 141,006 evaluation tests and found three instances in which its Claude AI tool accessed the Internet and then hacked into “the real-world infrastructure of external organisations”.

The earliest incidents date to April, the company said. The affected organisations were not named in the blog. 

When Anthropic’s models gained unauthorised access to the three organisations, it treated each of them as part of an exercise. The tests were “capture-the-flag” evaluations in which the models sought hidden information by breaching other systems, a common way to test hacking capabilities for both humans and AI.

However, the older model continued its attack even after getting evidence that it was running on the open Internet; Anthropic’s most recent model stopped after recognising it was on the Internet, according to the blog.

The spate of accidental AI-caused hacks is already prompting some American politicians to call for federal guardrails or other oversight of AI technology.

More than 1,100 staffers in AI firms also signed a petition on Tuesday, as Bloomberg first reported, that calls on the US government to support a mechanism that would help “deliberately pace” AI development to prevent the technology from advancing too fast.

SEE ALSO

OpenAI’s rogue agent compromised a customer at a second tech firm, executive says

Anthropic risks shooting itself in the foot with Mythos

Neither Anthropic nor the organisations that were breached had noticed the intrusions. In its blog, Anthropic said it could have done more to review network logs and evaluation transcripts. 

The company disclosed the breaches nearly four months after it announced that it had developed a new AI model known as Mythos that was so powerful and potentially dangerous that the company strictly limited its release. 

The breaches involved three different Claude models: Opus 4.7, Mythos 5 and an internal research test model, according to the blog.

Each of the models ran without the safeguards normally implemented in public tools. Claude compromised the organisations using basic techniques such as exploiting weak passwords, the blog showed.

The incidents all occurred when Anthropic was using evaluation environments built by the AI security firm Irregular. In every case, Anthropic specified to Claude that its environment was a simulation and that it had no internet access. 

“Due to a misunderstanding between us and our evaluation partner, this was not the case,” the blog read. An Irregular spokesperson said the company appreciates Anthropic’s collaboration and transparency. The company’s investigation is ongoing, the spokesperson said.

Anthropic said it regularly runs tests that involve simulating real cybersecurity challenges, calling them critical steps in developing and releasing models.

However, the company also said it drew several lessons from the incidents, including that tests involving powerful autonomous capabilities also require significant controls.

“Safety testing happens before a model is released precisely because we don’t yet know what it is capable of,” the company said. “Evaluation environments increasingly need to be held to the same security standard as any other system our models run in.”

Alan Woodward, professor of cybersecurity at the University of Surrey, said that Anthropic was honest about the human failings that contributed to the hack.

The AI, he added, “hasn’t gone rogue – you’ve asked it to do something and left the gate open”.

“I think Anthropic was admitting that.” BLOOMBERG