Rakuten AI boss diverges from big tech in prioritising low cost
RAKUTEN Group is expanding its AI team under the stewardship of a Google veteran and building models with a focus on cost efficiency.
Ting Cai, now three years into his tenure at the head of the e-commerce pioneer’s artificial intelligence team, has the task of creating AI systems that would augment the company’s many businesses and support the handling of commercial transactions at a minimal cost. He oversees a team that’s grown to 1,000 this year and has a battery of “thousands” of Nvidia chips to work with.
Tokyo-based Rakuten is wrestling with a struggling mobile business and constant competition in online shopping, both of which could get a significant boost from effective deployment of new AI tools. But the emphasis on doing that profitably from early on is where the company stands out from other large tech firms.
“Rakuten is very business-focused, applying the latest tech to solve customer problems,” Cai, 53, said in an interview. “In order to do this at large scale, we have to deliver the maximum margin. That’s why reducing the cost is super important for us in deploying generative AI.”
Cai’s team last week unveiled version 3 of the company’s large language model, which it says is 90 per cent cheaper to run than existing comparable LLMs.
The company segments tasks down to simpler jobs and develops smaller models catered to addressing specific needs for each of its services. Version 3 activates about 40 billion parameters for each individual token within its 700 billion parameters, while the remaining parameters stay inactive for increased efficiency.
AI functions contributed 10.5 billion yen (S$86.2 million) to operating income in 2024, and the company aims to double that figure this year.
Intelligent ad targeting and placement have boosted the return-on-investment for sellers using Rakuten’s online storefront, while AI-powered semantic search and personalised recommendations have lifted user engagement and click-through rates.
“We have observed that users come back more often after using the Rakuten AI Ichiba,” Cai said, referring to the full name of Rakuten’s virtual shopping mall. “So what we need to do is further reduce the cost of these conversations. We want each purchase through conversation to be profitable.”
Before Rakuten, Cai worked on Google Maps and local search services at Alphabet, following a long stint as a software engineer at Microsoft Corp. He wasn’t too familiar with Rakuten when an executive at the Japanese company contacted him about a job.
It was a “strange company” to Cai at first, he said, but after learning about founder and chief executive officer Hiroshi Mikitani and holding a few conversations with the man himself, Cai realised the two had a lot in common.
The job has turned out to be much more than what Cai believed he was signing up for, he said. His role has grown into a pivotal one for a company committed to integrating AI across its broad range of services, spanning everything from a mobile assistant to autonomous delivery vehicles.
He is proud of the work his team has accomplished building from the ground up, and Rakuten’s long-term vision includes having business customers tapping its AI tools and expertise.
“A lot of people are already depending on Rakuten. But I think we can do so much more than that,” Cai said. “Rakuten’s goal is to become Japan’s leading AI empowerment company.” BLOOMBERG
Decoding Asia newsletter: your guide to navigating Asia in a new global order. Sign up here to get Decoding Asia newsletter. Delivered to your inbox. Free.
Share with us your feedback on BT's products and services
