Is creating an AI Stefanie Sun unethical? Experts flag the tech’s pitfalls

COPYRIGHT infringement, legal liability and unequal representation – these are some of the pitfalls to navigate in regulating artificial intelligence (AI), according to a panel of experts at the Asia Tech x Singapore conference on Wednesday (Jun 7).

A prime case study is how recent videos duplicated the voice of Singaporean singer Stefanie Sun, using AI to make covers of other songs. The artiste’s management label reportedly did not consider legal action due to a lack of regulation around AI.

Referencing the incident, panellist Zeng Yi from the Chinese Academy of Sciences (CAS) noted that the rise of generative AI has stirred concerns around intellectual property (IP) protection laws.

“Her voice has been digitally cloned to play other songs that don’t belong to her. And of course, she (could) have copyright problems… Those AI versions are still on the web, with millions and millions of clicks,” he said.

While the legality of AI voice clones is grey, the bigger question is whether we want a society that accepts using AI to replicate someone without consent.

“I think we shouldn’t guide future society only by using IP protection laws. We need to go back to think about the moral and ethical (impact on) society,” added Prof Zeng, who is the director of the International Research Centre for AI Ethics and Governance at CAS.

Copyright infringement was just one of the many thorny issues around AI raised by Prof Zeng and fellow panellists in the wide-ranging discussion at the conference.

Another concern is liability, especially in sensitive sectors, noted Kay Firth-Butterfield, executive director of the Centre for Trustworthy Technology, under the World Economic Forum.

She cited how, at a recent US health conference, worries were raised about emergency room doctors using large language models (LLMs) to help them with diagnoses.

“If an LLM gets it wrong, whose liability is it – the doctor who asked the question, or the people who provided the incorrect information? These are huge issues that we have to deal with,” she said.

A similar question of liability came up recently when a New York lawyer submitted a brief full of errors that he had drafted using ChatGPT. The brief contained six non-existent court decisions, possibly due to a “hallucination”, where AI throws up inaccurate or even nonsensical answers.

And yet, perhaps terms like “hallucination” should not be used at all, since they create the impression that AI is more human than it actually is.

Ansgar Koene, global AI ethics and regulatory leader at EY, said: “The use of this kind of language makes you think that, well, this is only when the system is operating in a false state. So if I just don’t do something that causes it to have a ‘chemical imbalance’, it will generate true outputs. That leads to a misunderstanding about how you’re actually interacting with it.”

Instead, AI throwing out false information is a feature, not a bug. The “hallucinating” AI is actually not in an altered state, but “simply operating the way that it’s supposed to”, said Koene.

He added: “I would see it as a core responsibility for the technical community to think about what language we are actually using here… if it’s a term that’s precise enough to actually describe what is going on.”

Another concern is unequal representation in the data that AI models are trained on. For instance, models that are trained on online data may be more representative of men than women, and of developed countries than developing ones.

“It actually means that we are getting answers drawn from data which has been created mainly in the Global North, and mainly from guys,” said Firth-Butterfield, also noting how an estimated three billion people still lack Internet access.

With such concerns over AI mounting, policymakers will need to act quickly on setting the groundwork for regulation of the technology.

Elham Tabassi, associate director for emerging technology at the US-based National Institute of Standards and Technology, said: “One immediate need is to have guidance on standards (and) norms on how to verify and validate these models, pre-release. And then have the right transparency mechanisms and documentation of what type of testing has been done.”

Ultimately, the discussion around AI regulation is as much philosophical as it is technical. Prof Zeng expressed his distaste for the prospect of AI being omnipresent.

“I don’t like the argument of making AI everywhere… Maybe half of society can be optimised by using AI. But we need to leave the rest of the space to human connection, to the human way of interaction,” he said.