China’s AI charge: Short on chips and talent
AS CHINA’S artificial intelligence (AI) industry develops, companies are facing an imminent challenge – a lack of chips.
Yin Qi, the co-founder and chief executive officer of Megvii Technology – one of China’s top AI companies – told Caixin in a late March interview that there are only about 40,000 of chipmaker Nvidia’s data-centre-grade A100 graphics processing units (GPUs) in China. That is the kind used to build large-scale machine learning infrastructure.
Companies connect computers equipped with these chips to create powerful GPU clusters for running advanced programs, including those related to AI and machine learning – with more chips translating into more processing power.
For a company to build a GPT (generative pretrained transformer) large language model, it would need at least 10,000 A100 chips, Yin said.
And due to the chip shortage, companies in China would only be able to create GPU clusters of around 3,000 to 5,000 chips, he added. For an AI-related comparison, it would take more than 30,000 A100 GPUs to run OpenAI’s ChatGPT, estimated market researcher TrendForce.
To avoid falling behind, companies building large AI models have boosted their budgets for these industry-leading Nvidia GPUs. In China, demand for these chips has been driven by companies training generative AIs, such as ChatGPT.
However, Chinese companies have been unable to get their hands on these chips because the US government blocked Nvidia last year from exporting the A100 and its newer data centre chip, the H100 – which are key to developing large language models and generative AI – to Chinese customers.
Chinese companies can only purchase Nvidia’s A800 chip, a scaled-back version of the A100 whose chip-to-chip data transfer rate is two-thirds of the latter’s. This could restrict the overall computer power of any application running on the chip.
However, this bone thrown from Nvidia will not relieve China’s chip shortage on its own. An executive of a Chinese mainland-listed chip design company predicted that it would be a challenge to meet the chip demand from all Chinese companies due to the explosive pace of development.
There are domestic alternatives to GPUs from Nvidia and its main competitors Advanced Micro Devices and Intel.
Chinese GPU startup Shanghai Biren Intelligent Technology – considered a promising contender to Nvidia – has impressive computer power, but it falls short in the speed at which data can be transferred between two devices or systems.
Still, even if Chinese companies could design chips that can compete with or outperform the A100, they would not be able to find a foundry to manufacture them.
In October, the US Department of Commerce issued new regulations that would restrict wafer foundries’ access to US technology if they produced advanced chips for Chinese customers. In addition, mainland GPU-makers have, over the years, come to rely heavily on Taiwanese companies for manufacturing and packaging processes.
They are also dependent on US companies for electronic design automation tools – key software that allows developers to design, model, simulate and test circuit designs prior to production.
Of the three core elements of AI development – data, algorithms and computing power – the US has targeted computing power to restrain the advancement of China’s AI industry.
Developers will not be able to fine-tune algorithms without sufficient computing power, which would render vast amounts of their data useless, said one person familiar with the United States’ semiconductor export control policy.
While Chinese companies have troves of data at their disposal, there is a consensus that the quality is lacking. One US fund investor noted that Chinese companies need to spend a significant amount of time and manpower to clean up the data-sets needed for machine learning.
Hence, to save time, many large Chinese models are trained on US data-sets, which could leave Chinese companies with an AI that has trouble generating responses that require knowledge on local norms and contexts.
For example, with the image-generation feature from Baidu’s Ernie Bot, some people have complained that their prompts were actually translated into English before answers were generated, and so it failed to understand things such as Chinese idioms and the names of certain local foods.
China faces another obstacle on the global AI battlefield – a lack of top talent. Even though Chinese universities have been adding more AI courses to their curriculums, China’s supply of AI specialists cannot keep up with demand.
The country is currently short some 300,000 AI professionals, according to a report by the Chinese Academy of Labour and Social Security. The areas suffering the largest shortages include AI chip design, machine learning, natural language processing, algorithm research and application development.
China held 232 spots on the 2022 World’s Most Influential AI Scholar List, as ranked by Tsinghua University and other institutions. By comparison, US researchers took 1,146 spots on the nearly 1,900-person list.
The United States also produced more technical talent for the AI industry. It supplied 39.4 per cent of the world’s specialised AI technical professionals in 2020, followed by India and the United Kingdom.
China ranked fourth, with less than 5 per cent of the total, according to a report by Jean-Francois Gagne, an AI entrepreneur and founder of Element AI. Technical roles include research, data engineering, and AI and machine learning engineering and productisation.
China knows it needs to produce more AI experts and specialists if it wants to stand a chance.
In its report, the Chinese Academy of Labour and Social Security urged the country to more closely align its AI talent cultivation efforts with the needs of the industry, draw on the experience of the world’s best universities, and establish innovative teaching models to increase its AI talent pool.
Beyond building up its talent pipeline, Chinese enterprises, governments and universities may also need to consider how to improve their institutional mechanisms and come up with innovative ways to create a better working environment to more effectively attract and retain AI talent. CAIXIN GLOBAL
Decoding Asia newsletter: your guide to navigating Asia in a new global order. Sign up here to get Decoding Asia newsletter. Delivered to your inbox. Free.
Share with us your feedback on BT's products and services
