China's Answer to GPT-3 May in fact be Better

Beijing’s plans for artificial intelligence are serious, perhaps more serious than OpenAI or DeepMind’s capabilities. Established in November 2018 in Beijing, China, the Beijing Academy of Artificial Intelligence (BAAI) is a non-profit research institute dedicated to promoting collaboration among academia and industries.

If you enjoy articles like these, sign up to the Last Futurist, my speculative blog where we explore AI, stocks, tech innovation and breaking news. Simply put your Email on the top right side of the home page.

The Beijing Academy of AI is Evolving in the 2020s

The BAAI aims to foster top talent with a focus on long-term research on the fundamentals of AI technology. As a collaborative hub, BAAI’s founding members include leading AI companies, Chinese universities and research institutes.

• BAAI Introduces Superscale Intelligence Model ‘Wu Dao 1.0’ will likely out innovative GPT-3 and OpenAI, now being monetized by Microsoft.
• You can visit their site here. The current leader of the team is Ji Rong Wen.
• Technically speaking, the WuDao 2.0 natural language processing model had 1.75 trillion parameters, topping the 1.6 trillion that Google unveiled in a similar model in January.
• BAAI is essentially updating its model faster than OpenAI.

China has been pouring money into AI to try to close the gap with the US, which maintains an edge because of its dominance in talent and semiconductors, among other things like the top Universities.

China can’t buy its way to AI dominance, but within a generation it could fill the talent gap. Getting talent through immigration doesn’t appear to be China’s way. China's goal to catch up and lead AI by 2030 is very ambitious.

Still the BAAI shows evidence of China’s significant concentration on becoming the world leader in AI by 2030. The Beijing Academy of Artificial Intelligence, styled as BAAI and known in Chinese as 北京智源人工智能研究院, launched the latest version of Wudao 悟道, a pre-trained deep learning model.

It’s not all about quantity but China’s claim is interesting. Wudao has 150 billion more parameters than Google’s Switch Transformers, and is 10 times that of OpenAI’s GPT-3, which is widely regarded as the best model in terms of language generation.

Can China Catch up in Deep Learning?

• Unlike conventional deep learning models that are usually task-specific, Wudao is a multi-modal model trained to tackle both text and image, two dramatically different sets of problems.
• At BAAI’s annual academic conference on Tuesday, the institution demonstrated Wudao performing tasks such as natural language processing, text generation, image recognition, image generation, etc. 

What can Wudao AI do?

The model is capable of writing poems and couplets in the traditional Chinese styles, answer questions, write essays, generate alt text for images, and generate corresponding images from natural language description with a decent level of photorealism. It is even able to power “virtual idols”, with the help of XiaoIce, a Chinese company spun off of Microsoft–so there can be voice support too, in addition to text and image. 

There's some sense this technology could scale to other technologies and digital innovations.

Wudao does suggest China’s organization around artificial intelligence research is far superior now as of 2021.

The latest iteration of the project was led by the non-profit research institute Beijing Academy of Artificial Intelligence (BAAI) and developed with more than 100 scientists from multiple organizations.

While China does not yet have the talent of the U.S. or the University talent pool of international students, it’s starting to have a collaboration network and the funding to move fast in the 2020s and 2030s in the domain.

Can Supercomputers Super-charge Multi-modal AI?

This model with 1.75 trillion parameters is already the 2.0 version of Wudao, whose first version was just launched less than 3 months ago.

One of the main reasons the Chinese researchers may have made progress quickly was that they were able to tap into China’s supercomputing clusters, with the help of a few of its core members who also worked on the national supercomputing projects. 

The BAAI has a scholarship program. BAAI supports researchers to solve the most basic and most important problems of AI development. The “BAAI Scholars Program”, which funds researchers from universities, research institutes, and enterprises, focuses on the original innovations and core technologies of AI.

China's Parameters race is Going Well

In an era of fast-evolving AI models and China’s bet on deeper pockets, BAAI researchers claim to have broken the record set in January by Google’s Switch Transformer, which has 1.6 billion parameters. OpenAI’s GPT-3 model made waves last year when it was released with 175 billion parameters, making it the largest NPL model at the time.

How Good is Wu Dao’s 2.0

WuDao 2.0 covers both Chinese and English with skills acquired by studying 4.9 terabytes of images and texts, including 1.2 terabytes each of Chinese and English texts. It already has 22 partners, including smartphone maker Xiaomi, on-demand delivery service provider Meituan and short-video giant Kuaishou.

According to some academics in the field:

• Such models act as strategic infrastructure for AI development
• Large-scale pre-trained models are one of today’s best shortcuts to artificial general intelligence

How was Wudao Innovated and How does it compare with Google?

• BAAI researchers developed and open-sourced a deep learning system called FastMoE, which allowed Wudao to be trained on both supercomputers and regular GPUs with significantly more parameters.

AI Innovation will Accelerate Due to National Competition between China and America in the 2020s and 2030s

The emphasis on technological innovation in the 2020s could allow China's faster growing economy to dethrone the U.S. sometimes before 2035 in AI and a variety of related fields. For example how AI could impact healthcare, education, biotechnology, quantum computing, super-computing and eventually space technology.

BAAI is a matter of national pride for China. BAAI is funded by the Beijing government, which put 340 million yuan (US$53.3 million) into the academy in 2018 and 2019 alone, pledging to continue its support, a Beijing official said in a 2019 speech. The actual numbers in 2021 could be much higher.

Synced covered this really well, you can read their article here. A less edited version of this article was originally published on the Last Futurist.