Key Insights from the 2023 AI (Artificial Intelligence) Index Report

Artificial Intelligence is advancing rapidly, with recent developments exceeding even the most optimistic predictions from just a few years ago. Technologies such as OpenAI's ChatGPT and Google Bard, which were once considered futuristic concepts, are now a reality. This rapid evolution of AI has left many people and organizations stunned by its capabilities and uncertain about its potential implications. The pace of progress in AI technology has been astonishing, raising both excitement and concern among those who are following the field.

The Artificial Intelligence Index Report is an annual publication from Stanford University that tracks and highlights the progress and trends in Artificial Intelligence (AI) research, development, and adoption. The report is produced by a team of researchers from the AI Index, a project that aims to provide unbiased, comprehensive, and accessible data for policymakers, researchers, and the general public.

The report covers various aspects of AI, including research and development, education and workforce, hardware and software infrastructure, applications and implications, and public perception and attitudes towards AI. It includes data-driven insights, analysis, and visualizations to help readers understand the state of AI and its impact on society.

If you don't have time to read Stanford University's 386-page 2023 AI Index Report cover to cover, don't worry - I've summarized some of the key highlights for you to review.

Industry has more chance to create better AI

AI models are becoming more advanced, with significantly more data being used to train them than a decade ago. According to the AI Index Report, industry has taken the lead in producing significant machine learning models, due to their access to resources such as data, computing power, and funding, which are often lacking in academia and non-profit organizations.

Rising Costs for Rapidly Increasing Size of Large Language Models

The size and cost of large language models, such as GPT-2 and PaLM, have been increasing dramatically in recent years. While GPT-2 had 1.5 billion parameters and cost $50,000 to train, PaLM has 540 billion parameters and cost $8 million, indicating a trend of models becoming much larger and more expensive to develop.

Generative AI: An Overview of the Latest Systems and Their Limitations

In 2022, several generative AI systems such as DALL-E 2, Stable Diffusion, Make-A-Video, and ChatGPT gained public attention due to their impressive abilities to generate images, videos, and conversations. However, these systems can still produce unreliable or nonsensical outputs, making them unsuitable for critical applications.

Rise of AI Misuses

The number of AI-related incidents and controversies has increased 26 times since 2012, according to the AIAAIC database. Notable incidents in 2022 included the use of deepfake videos to create false narratives and the use of call-monitoring technology on prisoners, reflecting both the wider use of AI and a growing awareness of its potential for misuse.

Academy/Education Chapter Highlights

There has been a significant increase in the percentage of new computer science PhD graduates from U.S. universities who specialize in AI over the past decade, with the proportion rising from 10.2% in 2010 to 19.1% in 2021, according to recent data. In AI Index Report there was a point change graph about new Computer Science (CS) PHDS

The number of AI publications worldwide more than doubled from 2010 to 2021, growing from 200,000 to nearly 500,000.

Since 2015, publications in the field of Pattern Recognition and Machine Learning have doubled. Although there is no such increase in other areas, it still continues to increase.

Technology Chapter Highlights

Computer Vision - Image Classification

This image displays the change in accuracy of Image Classification using the ImageNet database between 2022 and 2023, which contains over 14 million images categorized into 20,000 categories and is available for research purposes. The latest data indicates a slight increase of only 0.1 percentage points in classification accuracy.

Computer Vision - Face & Image Generation

Image generation technology is utilized by companies such as NVIDIA in gaming applications, which involves generating images that are indistinguishable from real images through the use of artificial intelligence.

In the past year, text-to-image generation models such as DALL-E 2, Stable Diffusion, Midjourney, Make-AScene, and Imagen gained popularity, allowing users to generate images based on a text prompt. Figure 2.2.18 compares the images generated by DALL-E 2, Stable Diffusion, and Midjourney using the prompt "a panda playing a piano on a warm evening in Paris."

Computer Vision - Deep Fake

Artificial Intelligence technology enables the creation of Deep Fake images that are indistinguishable from real images, and can be used in disinformation campaigns and advertising. This technology involves superimposing one person's face onto another in a realistic manner, with developers making significant improvements in Deep Fake algorithms in recent years.

The Celeb-DF dataset, which includes original celebrity YouTube videos manipulated into deepfakes, is currently one of the toughest benchmarks for deepfake detection, with the Deakin University JDFD model achieving the highest AUC score of 78 in 2022.

In Image Processing, AI can outperform humans in detecting faces, classifying images, and segmenting objects into categories or segments. This succes is not new for 2023.

So How is Artificial Intelligence's Reasoning Success?

Computer Vision - Visual Reasoning

According to the 2022 report, while AI has been less successful in reasoning with visual data than previous technologies, the trend suggests that it may soon surpass human capabilities in this area, as the current report indicates that AI now exceeds human average performance.

Computer Vision - Visual Commonsense Reasoning

VCR A different Technique used for artificial intelligence in making sense of images. With this technique, artificial intelligence is asked to answer the challenging questions in the scenarios presented in the images and to provide the logic behind it.

While there is currently a gap between artificial intelligence and human performance in VCR technology, it is expected that AI will close this gap in the coming years.

Computer Vision - Video - Activitiy Recognation

The field of Activity Recognition has achieved high success rates, and the 2023 report indicates that the accuracy rate is inching closer to 100%.

Computer Vision - Video Generation

In 2023, multiple high-quality text-to-video models were released, including CogVideo, which achieved the then-highest inception score on the UCF-101 benchmark. In September 2022, Meta's Make-A-Video model significantly surpassed CogVideo's score by 63.6% on UCF-101, and in October 2022, Google released a text-to-video system called Phenaki.

Language - Chat GPT

The AI Index tested three large language models from different years on their ability to answer the same question about Theodore Roosevelt's presidency, and found that more recent models were more effective. While AI systems have improved in reasoning tasks, a recent study found that they struggle with planning and reasoning tasks in a Blocksworld problem environment. The study showed that large language models performed much worse than humans, suggesting that while they are capable, they lack human reasoning capabilities.

The Blocksworld problem is a well-known challenge in AI research that involves a simulated world with blocks of various colors and shapes that must be manipulated to achieve certain goals. It is a standard test for planning and reasoning algorithms in the field of artificial intelligence.

THE SUMMARY OF AI TECHNOLOGY IMPROVEMENT

The AI Index report for this year highlights a recurring theme of performance saturation across various technical performance benchmarks. This trend was also observed in last year's report, but the saturation has become even more pronounced in recent times. A graph in the report shows the relative improvement in these benchmarks since their launch and within the last year, presented as percentage changes. It reveals that, except for seven benchmarks, the improvements have been less than 5%. The median improvement within the last year is only 4%, while the median improvement since launch is 42.4%. The report notes that some traditionally popular benchmarks like SQuAD1.1 and SQuAD2.0 were not featured this year due to a lack of new state-of-the-art results. Additionally, the speed at which benchmark saturation is being reached is increasing. To tackle this issue, researchers have introduced newer and more comprehensive benchmarking suites like BIG-bench and HELM.

The AI Index report is eagerly anticipated and examined every year. I research the data that I find important, especially when comparing it to the previous year's report. In this article, I wanted to summarize the important points of the 300-page report from my perspective, and I used data from the 2022 report in the visuals. I hope this article is useful. To download the AI Index report, you can go to this link: https://aiindex.stanford.edu/report/ .