Amazon rushes out latest AI chip to take on Nvidia, Google
AWS is the largest seller of rented computing power and data storage
[WASHINGTON] Amazon.com’s cloud unit raced to get the latest version of its artificial intelligence (AI) chip to market, renewing efforts to sell hardware capable of rivalling products from Nvidia and Google.
The accelerator, called Trainium3, was recently installed in a few data centres and will be available for customers beginning on Tuesday (Dec 2), Dave Brown, a vice-president with Amazon Web Services (AWS), said.
“As we get into early next year, we will start to scale out very, very quickly,” he said.
The chip push is a key element of Amazon’s strategy to stand out in AI. AWS is the largest seller of rented computing power and data storage. But it has struggled to replicate that dominance among leading developers of AI tools, as some companies opt instead to rely on Microsoft, which has close ties to ChatGPT maker OpenAI, or Alphabet’s Google.
Amazon shares rose 1.6 per cent to US$237.71 at 11.09 am in New York. Nvidia shares pared gains, while AI chip rival Advanced Micro Devices dropped to a session low.
Amazon hopes to entice companies looking for a bargain. Trainium chips are capable of powering the intensive calculations behind AI models more cheaply and efficiently than Nvidia’s market-leading graphics processing units, according to the company.
“We have been very pleased with our ability to get the right price performance with Trainium,” Brown said.
Amazon is releasing Trainium3 about a year after deploying its predecessor accelerator. That’s a sprint by chip industry standards. “The main thing we are gonna be hoping for here is just that we don’t see any kind of smoke or fire,” an AWS engineer joked when the chip was first fired up in August.
The quick turnaround also matches the pace of Nvidia, which has pledged to release a new chip every year.
SEE ALSO
There’s a catch: Amazon’s chips lack the deep software libraries that help customers get Nvidia’s graphics processing units up and running fast. Bedrock Robotics, a company that uses AI models to enable construction equipment to operate autonomously, runs its infrastructure on AWS servers. But when it comes to building models to help guide an excavator, Bedrock uses Nvidia chips, according to chief technology officer Kevin Peterson.
“We need it to be performant and easy to use,” Peterson said. “That’s Nvidia.”
Many of the Trainium chips in use today are at the disposal of Anthropic, inside data centres in Indiana, Mississippi and Pennsylvania. AWS said earlier this year that it had strung more than 500,000 of them together to help the AI startup train its latest models and aims to dedicate one million of the chips to Anthropic by the end of the year.
Amazon is betting Anthropic’s success, along with its own AI services, can entice other companies. Amazon has announced a few other major customers for the chip, leaving analysts struggling to assess Trainium’s effectiveness.
Anthropic is also using Google’s Tensor Processing Units and cut a deal earlier this year with the Alphabet unit that gives the startup access to tens of billions of US dollars worth of computing power.
In an interview with Bloomberg Television, AWS chief executive officer Matt Garman said the company’s relationship with Anthropic was “incredibly strong”. He said that the startup has an enormous demand for computing power, leading Anthropic to use a variety of providers.
Amazon made the chip announcement at re:Invent, its annual user conference, which in recent years has become a rolling advertisement for AI services where Amazon courts builders of cutting-edge tools and companies that might want to pay for access to them.
On Tuesday, Amazon also announced updates to its main line of AI models, called Nova. New Nova 2 products include a variant called Omni, which can receive text, image, speech or video inputs and respond with both text or images.
As with its chips, Amazon has tried to sell customers on the performance of its models for their price. Prior Nova models generally have not ranked among the industry leaders in benchmarks that track how AI models perform in answering standardised questions.
“The real benchmark is the real world,” Rohit Prasad, who leads much of Amazon’s model development and the company’s Artificial General Intelligence team, said, adding that he expected the new models to be competitive.
The company also plans to let customers bring more data to bear when customising Amazon’s models. Nova Forge, a new product, is designed to let sophisticated users grab versions of Amazon’s Nova models before their training is complete and customise them with their own data.
Reddit is using Nova Forge to build a model capable of assessing whether a post on the digital message board violates the site’s safety policies. Chris Slowe, the company’s chief technology officer, says that some AI customers are tempted to use the most advanced model to solve every problem, rather than seeking one with a particular expertise.
“The fact that we can make it an expert in our specific area is where the value comes from,” he said. BLOOMBERG
Decoding Asia newsletter: your guide to navigating Asia in a new global order. Sign up here to get Decoding Asia newsletter. Delivered to your inbox. Free.
Share with us your feedback on BT's products and services
