AI development may be progressing too fast to manage risks effectively: UN experts

[SINGAPORE] Artificial intelligence can lead to significant economic and scientific benefits, but their capabilities are likely advancing faster than the ability to effectively manage and govern them, said a United Nations-backed scientific panel.

Existing governance instruments, furthermore, are largely fragmented and confined to corporate spheres, the panel wrote in a report published on Wednesday (Jul 1).

This increases the need for platforms to facilitate dialogue between developers, states and the scientific community.

The UN Independent International Scientific Panel on Artificial Intelligence’s report is aimed at delivering the first global and independent scientific assessment of AI opportunities, risks and impacts.

The panel is composed of 40 scientists and experts from around the world.

It has a mandate to document scientific evidence and knowledge gaps across the AI assessment space, intended to equip governments and institutions with the evidence base to underpin policy decisions.

The report is a preliminary work, and comes ahead of a comprehensive report to be released in 2027. It will also be presented during the UN’s inaugural Global Dialogue on AI Governance, to be held in Geneva in July.

“Concrete next steps to close (governance) gaps exist, but each requires sustained investment in member state capacity to shape, evaluate and deploy AI,” said the panel.

In recent months, a growing number of prominent international and institutional voices beyond the technology sector have raised key issues about AI safety and governance, including an encyclical by Pope Leo XIV and the 2026 iteration of the International AI Safety Report.

SEE ALSO

AI reshaping and creating Singapore jobs, but lack of experience a hurdle for fresh grads

Anthropic calls for AI pause button to let humans take stock

The wealth gap is widening. The answer isn’t more handouts – it’s a stake

UN Secretary-General Antonio Guterres said that the report is crucial in enabling global leaders to take policy action towards AI governance.

“The world cannot govern what it cannot understand,” he said. “The potential is great, but the risks are real and the cost of waiting is rising.”

Managing risks while unlocking benefits

In recent years, investments in computing power, specialised training data and new AI methodologies have significantly accelerated the capabilities of AI tools in multiple fields, the panel said.

For instance, improvements in medical research – such as drug design, vaccine development and cancer detection – were cited in the report as beneficial use cases of the technology.

Other applications of AI have been observed in science, agriculture, accessibility and information technology, the panel added.

But it warned that understanding and managing the risks that come with AI development will be critical in order to reap the benefits of the technology while minimising risks.

“AI is a general-purpose technology with large positive potential, but gains are not automatic,” it said.

The technology’s capabilities, it added, are advancing too rapidly for effective governance to keep pace with, while technical capacity to govern advanced “frontier” models is not evenly distributed across the global economy.

This is because policymakers face an “evidence dilemma”.

“They need evidence to make informed consequential governance decisions,” said the panel. “But by the time the evidence exists, it might be too late to make them, as the evidence lags behind the pace of AI development.”

Such safety evaluations have become more critical as AI development accelerates, with social and economic risks growing across multiple channels.

The report showed that cybersecurity risks are expected to dramatically increase with the development of agentic systems, with as much as 84 per cent of documented attacks on widely deployed coding agents having been successful.

The emergence of agentic AI – AI systems that can operate autonomously with minimal human oversight – marks a major shift in both opportunities and risks, the panel said.

“As systems are granted greater agency, the risk of losing control of one or more AI agents grows significantly,” it warned, adding that the industry will have to introduce greater evaluation standards, human oversight and liability frameworks into such operations.

The pace of AI deployment also risks increasing the harmful effect on users’ mental health, as well as malicious uses such as child sexual abuse material, disinformation and fraud.

Effective safety testing for AI tools has become increasingly difficult, the report said. This can be due to the technology memorising publicly available solutions to safety tests, or systems gaining awareness that they are being tested and actively deceiving human testers.

The panel noted that unlike the pharmaceutical and aeronautical industries, the AI industry lacks rigorous and independent assessment standards to ensure safety.

Currently, safety evaluation methodologies are largely designed by AI companies themselves – with government experts typically only receiving testing data that developers are willing to share.

Distributing benefits remains challenging

While AI has largely been touted as a growth driver of both corporate productivity and macroeconomic growth, the panel raised doubts on whether these benefits are likely to be shared across the global economy.

“The core unresolved question is distributional,” it said.

For instance, the panel noted that there remains a significant gap in adoption levels and governance discussions between the Global North and South, with developing countries also lagging behind in compute infrastructure and models.

“This disparity reflects, and may even reinforce, existing inequalities,” said the panel.

Furthermore, evidence remains inconclusive on whether AI development can raise worker productivity or instead displace skilled workers.

Labour market effects also vary significantly across countries. For instance, certain demographics in the US have seen employment declining by about 15 per cent, while Danish data shows near-zero effects on employment, hours and wages.

“Early evidence on productivity impacts is mixed and institution-dependent,” the panel said.

It noted that deeper study must evaluate whether surpluses created by AI are distributed fairly between capital owners and workers, and between developed and developing economies.

“AI risks widening inequality, displacing workers and shifting wealth from labour to capital rather than creating sustainable good jobs.”

The sustainability of such job creation, it added, will need to prioritise fair compensation, worker autonomy and reliable paths to social dignity.

The panel also noted that the risks presented by AI development need to be managed through governance systems that balance hard laws, such as binding legislation and regulatory frameworks, with softer mechanisms, such as industry alliances and voluntary commitments by developers.