AGI Defined

A new paper outlines a framework to gauge progress toward the creation of artificial general intelligence (AGI), a concept that's been gaining traction in the AI field, particularly among organizations like DeepMind and OpenAI, which are actively pursuing this goal. AGI, a term first introduced by Shane Legg of DeepMind and Ben Goertzel of SingularityNet in a 2007 essay collection, refers to AI capable of performing any cognitive task that a human can.

The framework, developed by Meredith Ringel Morris and her team at Google, presents a taxonomy that classifies AI systems based on their breadth of abilities and cognitive task performance. It differentiates between narrow AI, which excels in specific tasks, and general AI systems, which demonstrate a wide range of capabilities encompassing various real-world activities valuable to humans, such as language, math, logic, spatial reasoning, social interaction, learning, and creativity. Significantly, these general systems have the capacity for meta-learning and know when to seek human input.

The taxonomy divides both narrow and general AI into five performance levels beyond a basic Level 0. Level 1, termed "emerging," is on par or slightly better than unskilled human ability. Levels 2 ("competent"), 3 ("expert"), and 4 ("virtuoso") represent AI that exceeds the 50th, 90th, and 99th percentiles of human skill, respectively. The pinnacle, Level 5, labeled as "superhuman" or "artificial superintelligence," surpasses all levels of human skill.

While most current high-performing systems, like AlphaFold in protein folding, are categorized as narrow (Level 5 in its specific task), large multimodal models such as Bard, ChatGPT, and Lama 2 are considered general systems at Level 1, with potential Level 2 capabilities in certain tasks. The framework also introduces an autonomy scale, ranging from systems that assist humans in tasks to fully independent agents, suggesting that higher performance levels might be necessary for complete autonomy in applications like self-driving vehicles.

The taxonomy, however, does not specify a definitive list of tasks for AGI or a method for choosing them. Instead, it encourages the research community to continuously update and refine the benchmark for generality, including adding new tasks as they become relevant.

Understanding and defining AGI is crucial, not just for technological discourse but also for regulatory and business considerations. For example, OpenAI's partnership with Microsoft includes a clause allowing OpenAI to withhold models achieving AGI status. A concrete framework like this one could provide a more objective basis for such decisions. The definition of AGI is complex and evolving, as exemplified by OpenAI's definition of AGI as a system that excels in most economically valuable work, a description that would have applied to the internal combustion engine in the early 20th century.

Ref: DeepLearning.AI