Anthropic calls for global AI pause to face immense human labour risks

[NEW YORK] Anthropic called for the creation of an industrywide mechanism to pause the development of artificial intelligence, giving society the opportunity to “deal with its immense implications”.

The technology is advancing to the point where AI will make human labour thousands of times more efficient or even replace it, creating a new set of risks, the company said in a lengthy blog post on Thursday (Jun 5).

AI could begin to improve itself and build its own successors, diminishing the role of humans, the company wrote. “That collision, where recursive intelligence building itself ever faster meets the world of humans, relationships, and governance, is another part of this future we can’t predict,” it noted.

The blog post compared the idea to the international regulation of nuclear weapons. Any pause mechanism would also need to ensure less scrupulous AI labs don’t secretly keep working to advance the technology during a break, likely through checks by peer labs.

Anthropic said that it would organise conversations with policymakers, researchers and other AI companies to discuss a structure for “coordination and deliberation” and will publish the results.

Anthropic’s post isn’t the first time a frontier AI researcher has called for a break in development. In 2023, the Future of Life Institute, a nonprofit focused on mitigating risks from technology, called for a pause of at least six months to put guardrails on AI, warning about potentially catastrophic effects.

Billionaire Elon Musk signed it along with more than 1,000 other researchers and executives.

Anthropic itself has continued to put out advanced models and tools such as its popular assistant Claude, and the new Mythos model that it says can detect and exploit cybersecurity flaws at remarkable speed.

The company is also preparing for an initial public offering.

SEE ALSO

As AI reshapes entry-level work, fresh grads must prepare for the ‘third-year job’: panel

Only 15% of workers in Singapore feel their jobs are safe amid AI-driven anxieties: survey

AI unlikely to lead to ‘jobs apocalypse’, OpenAI CEO Altman says

AI needs to ‘work in support of humans’: Josephine Teo

Meta launches enterprise-focused AI ‘business agent’ to automate daily operations

The returnees: Inside China’s AI talent reversal

JD.com founder vows to protect Chinese jobs from AI and robots

More

Critics at the time of the 2023 letter said that stopping development would unnecessarily crimp innovation, and that US companies agreeing to tap the brakes would hand an advantage to other countries. Former Google chief executive officer Eric Schmidt warned that the pause would give an advantage to competitors in China, which is locked in a technological trade war with the US.

In the blog, Anthropic acknowledged this difficulty, and said “arms control” for AI is more challenging than other technologies.

“Training runs are far easier to conceal than missile silos, their inputs are general-purpose, and the incentive to defect quietly is enormous, because whoever continues while others pause could inherit the lead,” Anthropic wrote. While the world has built similar regimes in the past, they “took decades to build both the infrastructure and the trust. We don’t have that long”. BLOOMBERG