OpenAI’s pivot to porn is problematic – but lucrative

WHEN you are building artificial intelligence (AI) to benefit humanity, you might have to compromise. AI is expensive, so you raise billions of dollars from investors such as Microsoft, Nvidia and the United Arab Emirates.

As you strive to build super-intelligent computers that will cure cancer, you also need to make money for your backers. So, after pitching your powerful chatbot technology to businesses, who struggle to make it useful, your next option may be monetising your enormous user base of 800 million weekly visitors – with a sex bot.

That is the ignoble trajectory of OpenAI under Sam Altman, who has made a career of justifying opportunistic business moves – such as inflating the AI bubble with circular dealmaking or releasing a TikTok clone – with the promise that his tech will eventually, one day, solve intractable human problems.

There is little evidence that OpenAI’s systems will do that; but in the short term, it can make some money – especially with erotic roleplay.

This December, ChatGPT will be imbued with more personality and the ability to engage in “erotica” with verified adult users, Altman brazenly announced on X.

The news was met with some repulsion on X but celebrated on Reddit, where many ChatGPT users have spent the last two years sharing tricks for jailbreaking the bot to sext with it.

OpenAI has batted off criticism that its bot causes mental health problems and dependency, and was recently sued by the parents of a teenage boy who followed instructions on the bot to die by suicide.

Altman said, with no further explanation, that those mental health risks had now been “mitigated” and it would relax ChatGPT’s restrictions “in most cases”.

Chatbot romance was around for years before Altman brought ChatGPT to the world in November 2022, and it is a confirmed money maker. Replika launched in 2017 as a friendship bot – until most of its users started treating it like a virtual girlfriend or boyfriend, and its growth soared.

Elon Musk’s Grok, a chatbot made by his company xAI, counts two hyper-sexualised manga characters as its main draw for users.

Ads for AI girlfriends have swarmed Facebook and Instagram, and a recent study by academics from Oxford and Cambridge Universities found that the 110 most popular AI companion apps were “overwhelmingly designed around heterosexual male fantasies”.

In aggregate, those dating-themed AI chatbots captured 88 million monthly visits globally (more than the 75.8 million visits that social media platform Bluesky gets), a conservative estimate that did not include popular, general-purpose companion apps such as Character.ai.

Character.ai, which had around 20 million monthly users earlier this year, had also been heavily used for sexual role-play up until the company recently put new restrictions in place.

The makers of ChatGPT will have noticed this phenomenon. Roughly 30 per cent of the prompts being typed into general purpose AI assistants including ChatGPT are romantic or sexual in nature, said Lauren Kunze, chief executive officer of San Francisco-based AI chatbot company Iconio.

What is the appeal? Kunze, who has been in the chatbot space for 20 years and recently launched a virtual astrologist called Celeste, suggested there might be an evolutionary drive in humans to evaluate any seemingly sentient being as a potential mate. And historically, porn has driven mainstream tech adoption.

“The consumer demand is overwhelmingly there,” she said. Chatbot companies that do not allow pornographic content often find it harder to stay in business. When Replika banned sexual roleplay in February 2023, its user numbers dropped from the single-digit millions to the hundreds of thousands. Character.ai also saw a usage drop when it did the same earlier this year.

Altman himself seems to have tried to resist all this. This was what he said on video journalist Cleo Abram’s podcast two months ago:

Altman: There’s a lot of short-term stuff we could do that would, like, really like, juice growth or revenue or whatever and be very misaligned with (our) long-term goal… Sometimes we do get tempted.

Abram: Are there specific examples that come to mind?

Altman: Well we haven’t put a sexbot avatar in ChatGPT yet.

Abram: That does seem like it would get time spent.

Altman: Apparently it does.

“Time spent” is a metric used by social media developers to gauge the stickiness of an app. OpenAI’s pivot to erotica points to an uncomfortable truth about the state of AI today – that for all its promise to transform industries and uplift civilisation, the easiest way to commercialise it is still through the basest human instincts. Altman’s vigorous dealmaking has left him with few qualms about chasing them. BLOOMBERG