Generalist AI Systems: Causal World Models: Building Intelligent LLMs
"All alterations take place in conformity with the law of the connection of cause and effect"; "Everything that happens, that is, begins to be, presupposes something upon which it follows according to a rule." — Kant, Critique of Pure Reason follows.
We advance Causal Fundamentalism taking interaction and causation to be absolutely fundamental to the study of the world and building general-purpose disruptive technologies, as generalist AI, understanding the causal behavior of the world and its contents.
Causal/Interactive Fundamentalism
Reality, the world, the universe or Nature is governed by causal interactions; and the ultimate goal of humanity is to "To Know the Causes of Things", finding the causal laws and technological innovations of the general principle in specialized domains.
We demonstrate how to create generalist AI or real intelligent general-purpose foundational LLMs relying on the Causal Fundamentalism.
Introduction
As stated in a short consensus article, Managing extreme AI risks amid rapid progress , describing extreme risks from upcoming, advanced AI systems:
"Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act in the world and pursue goals.
We don’t know for certain how the future of AI will unfold. However, we must take seriously the possibility that highly powerful generalist AI systems—outperforming human abilities across many critical domains—will be developed within the current decade or the next. What happens then?"
On the other side, due to all the overhype and sensationalism, the N3A2M (Nvidia, Apple, Amazon, Alphabet, Microsoft, Meta) $T club, plus TSMC. over-profiteering on selling the fake AI's picks and shovels, hardware and software, and we have "AI apocalypse: 80% of projects crash and burn, billions wasted", as to RAND corporation report .
The Economist complains that Artificial intelligence is losing hype , and the share prices of the N3A2M $T club driving the ai revolution have dropped by 15%.
So, its big question is:
"What happened to the artificial-intelligence revolution? When the world will “get AGI”, or when machines will become more advanced than humans. So far the technology has had almost no economic impact".
To have real impact, social, economic or political, any technological revolution is in need of reality or fundamentality, which is real-world modeling autonomous machine intelligence for AI technology.
As Yann LeCun wisely noted, “arguably, designing architectures and training paradigms for the world model constitute the main obstacles towards real progress in AI over the next decades.” “A Path Towards Autonomous Machine Intelligence .”
Causal/Rational/Real/True/Scientific/Computational World Models (CWMs)
The Root Cause of Real Intelligence, Human or Machine, is World Knowledge, Scientific Knowledge as a Causal World Model or World Schema, a fundamental framework for learning and inference that helping intelligent agents categorize and organize, interpret and understand world's data/information.
In its deep sense, to be causal is to be logical, rational, real, true, scientific and computational.
No causal laws, patterns and schemas, no understanding the world; for identifying and establishing causal relationships is the aim of all studies, including all sciences ranging from?the natural sciences to?social sciences, politics?and?economics, as well as engineering sciences and technologies.
The Root Cause of Failure for Artificial Intelligence Projects is a Lack of Causal/Rational/Real/True/Scientific/Computational World Models (CWMs) processing/computing world's data, information or knowledge to know, understand and interact with the world.
In terms of ontological categories, the highest genera, kinds or classes of things, T, the world model could be expressed as the world tensor/vector representation:
W = <E, S, C, R; D; I; Com> (1)
where, W is the totality of ontological/categorical variables or the universal class of all things, T = E, S, C, R - the general classes of entities (substances, objects), states (quantities/qualities), changes, and relationships, respectively; D - the data universe of data entities, states, changes (events, phenomena, observations) and relationships; I- Intelligence process involving causal/logical algorithms and data structures, which capacities and capabilities are dependent on its power to model and simulate and effectively interact with reality; Com is the computation, as mathematical equation problem solving or executing computing algorithms, or arithmetic calculations, or symbolic manipulations of data structures.
The critical question: "What are the fundamental capabilities and limitations of AI computers?" is replied the scope and scale of its computational or causal world models.
There are lots of confusion and misinterpretations what reality or world as a whole and its model mean, identified in different ways:
in philosophy and metaphysics, with "the sum or aggregate of all that is real or existent within the?universe, as opposed to that which is only?imaginary, nonexistent or nonactual".?
in cosmology and physics, with "a totality of entities extending through space and time",
in psychology, with common sense, a collection of models of the world that can guide on what is likely, what is plausible, and what is impossible;
in predictive analytics or ML, with a neural network architecture for learning through observation and prediction, approximating human observation, learning, reasoning, planning, and acting … thinking;
in LLMs, as auto-regressive generative computational DNNs models of natural language data, text, images, video, etc.; "a general-purpose large-context multimodal autoregressive model, performing language, image, and video understanding and generation"
in human-mimicking intelligence, the predictive world model, the centerpiece of the Joint Embedding Predictive Architecture (JEPA) for autonomous human-like machine intelligence predicting plausible future states of the world.
in computational ontology, computer science and engineering, AI and ML, a causal world hypergraph network architecture for learning, understanding through measurement, simulations, experiment, observation, inferences, prediction and interactions:
CWM = T x T = {(E, S, C, R) x (E, S, C, R)} (2)
Note it embraces a popular interpretation of causality "an influence by which one event, process, state, or object contributes to the production of another event, process, state, or object".
A causal world model is fully encoded with a multi-hypergraph which contains 4 types of nodes (vertices, points, or elements), as its order, 16 types of relationships to other nodes, as its size, with multiple/parallel each hyperedge (hyper-links, lines, arcs, arrows) and causal loops joining any number of elements for modeling multi-way group interactions among nodes.
Each node in the undirected multigraph hypergraph network with all possible causal loops is representing a causal factor or causal variable, as a cause or effect in four major categories:
as entities, substances, objects or agents, E,
states, quantities or qualities, S, as in with a causal state machine that defines states as causes and links each cause to a specific action,
changes, events, actions, activities, or processes, C,
relations, links, connections, or interactions, R.
Furthermore, a node may contain a collection of causes or even another causal graph. That means a causal model represented as a graph may store other causal sub-models in each node of the causal graph, thus representing complex causal structures.
The AI CWMs can reason over a single cause in the graph, a selected sub-graph, or the hypergraph itself, identifying real computational causality, C x C, as the cause-effect interrelationships or interactions of true causal variables, which are changes, events, actions, activities, or processes.
All real-world systems are modelled as causative multigraph hypergraphs, in which a causal link can connect any number of nodes, and a causal node might have any number of causal connections and loops.
In complex systems or interaction networks, one cause-variable has multiple effects-variables, and conversely, one effect may be caused by a multitude of causes, where causal emergence is the feedbacking effect of complex and nonlinear interactions between components.
Complex natural, living, cognitive, and artificial systems consist of diverse and heterogeneous elements/units/agents that interact through complex nonlinear CAUSAL relationships, modelled as causative multigraph hypergraphs with causal paths and feedback loops/circles/circuits, positive or negative, reinforcing or balancing, constructive or destructive.
In reality, the world is the largest environment ever, known as reality or existence or being or the universe, taken as a whole. It is the totality of all entities and interactions, of all matter and energy, of all space and time.
The AI CWM is representing its nature, structures, mechanisms, patterns, laws and principles by the Computational World Modeling Engine for Knowing, Learning, Inference and Interaction.
USECS as the Catalogue of The World, a computational ontology system of categories that provides an encompassing classification of all possible entities in the world. fuels the CWM Engine, and its Generalist AI.
CWMs are generalist AI systems that learn to understand and interact with any complex environments, physical or digital.
computing/acquiring/sensing/measuring real-world data by data-collecting tools, learning about the environment, modeling and simulating its content, identifying causal patterns, explaining the models and simulations, infer/deduce causality, predicting the effects-outcomes, computing rational decisions and performing intelligent interactions with environmental data, machines or humans.
CWMs are what come next after statistical AI/ML/LLMs, driving innovations such as:
Causal Autonomous Systems, as causality-intelligent machines, drones and robots navigating and performing complex tasks in unpredictable environments.
Causal Virtual Reality, creating realistic immersive and interactive virtual worlds for business, education, gaming, etc.
Real AI Governance, simulating political, social and economic scenarios to test and evaluate the impact of policies before implementing in the real world.
Generalist AI Systems that learn to understand and interact with any complex environments, physical or digital. mental or social..., knowing how things in the world can be divided into different categories and classes, kinds and types. items and instances, from the mots general features to the mots specific units.
CWMs will learn about reality and its features and contents, like space and time, deducing specific cause-and-effect relationships form the causal hypergraph network, on top of which scientific, technological, linguistic, social knowledge can be acquired.
CWMs could transform LLMs into Generalist AI Systems allowing it to know grammar and syntax, semantics and conceptual relationships without nuclear energy-hungry training the LLM on a massive corpora of text (in the billions of pages or trillion of tokens).
Generalist AI Systems: RILLMs: Really Intelligent LLMs: = CWM [USECS ] + LLMs [NLU/NLP + GenAI/Foundation Models] + Narrow Superintelligent AI/ML Models
领英推荐
Unlike LLMs and generative AI or causal AI, which are limited to analyzing stochastic patterns and correlations or linear causation across data, CWMs enable deep understanding and explainability, truth and factuality, by leveraging causal world modeling to gain insights in any dataset.
Designed to understand how the world works, to identify and the cause and effect of relationships across data, CWMs are to replace or embrace LLMs , suffering from the principal limitations as annotated in the Supplement.
Real AI: Generalist AI Systems as Causal World Models
The AI CWMs are combining the Reality Modeling Machine (the CWM Multi-Hypergraph Network) + Causality Engines + Data Ontology Engine + AI Models + ML/Deep Neural Networks + Human Intelligence
AI = Real AI = Transdisciplinary AI = Interactive AI =
Reality Machine (World Modeling and Reality Simulation Platform, the World Hypergraph Networks + Scientific World Knowledge + the Internet/Web Data) +
Causality Engines (Physical CE, Chemical CE, Biological CE, Mental CE, Social CE, Economic CE, Political CE, Informational CE, Digital CE, Virtual CE, Technological CE) +
PATS [Predictive Analytics Statistic Techniques, Statistical Models, Narrow AI, ML, ANNs, DL] +
[LLMs, Generative AI] +
Knowledge Graphs + Domain Ontologies + Data Sets +
Causal AI (Explainable AI (XAI), "Understandable AI")
Robotic Hyper-Automation +
the Internet of Things +
Emerging Technologies +
Human Intelligence +
Hyperintelligent Hyper-Automation ...
It is driven by the AI Reality Engine, the World Modeling and Reality Simulating Platform, representing the universe as global causal hypergraphs with complex hyperedges, interrelationships, interactions and interdependencies, among and between all nodes, things, entities, substances, states, changes, or relationships.
Resources
SUPPLEMENT
As someone working in statistics and peripherally machine learning it has been endlessly tiresome to hear LLMs be marketed as "AI" to an unsuspecting audience. LLMs are no closer to AI than Alexa was this time last year.
While the capabilities of Large Language Models are impressive, calling them "AI" remains contentious. Here's why some in the technical community, including Sam Altman, have our doubts:
Limited understanding and reasoning: LLMs excel at pattern recognition and statistical analysis, but they lack true understanding of the data they process. They can't reason logically, draw meaningful conclusions, or grasp the nuances of context and intent. This limits their ability to adapt to new situations and solve complex problems beyond the realm of data driven prediction.
Black box nature: LLMs are trained on massive datasets. This "black box" nature makes it challenging to explain their predictions, debug errors, or ensure unbiased outputs.
Lack of "general intelligence": LLMs currently lack the broad, transferable intelligence that characterizes humans. They excel at specific tasks within their training data, but struggle with novel situations or requiring different skills. An inability to generalize outside their training data restricts their claim to the title of "AI."
Focus on prediction over understanding: LLMs, for all their impressive feats, remain slaves to their training data. They excel at mimicking and recombining existing information, akin to a masterful DJ remixing familiar tracks. They remain powerful tools, like supercharged search engines and spell checkers, but calling them AI risks mistaking virtuosity for originality. LLMs are inherently statistical models, predicting outputs based on past observations, nothing more.
Overestimating progress: The rapid advancements in LLMs can lead to overoptimistic claims about their capabilities. Comparing them to intelligence is misleading, the underlying mechanisms and levels of understanding differ significantly.
LLM is dumb as any tool must be, The problems are, overhype and sensationalism and big lies. LLMs are a category of foundation models trained on immense amounts of data making them capable of understanding and generating natural language and other types of content to perform a wide range of tasks.
Abstract Artificial Intelligence (AI) is progressing rapidly, and companies are shifting their focus to developing generalist AI systems that can autonomously act and pursue goals. Increases in capabilities and autonomy may soon massively amplify AI’s impact, with risks that include large-scale social harms, malicious uses, and an irreversible loss of human control over autonomous AI systems. Although researchers have warned of extreme risks from AI1 , there is a lack of consensus about how exactly such risk arise, and how to manage them. Society’s response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems. In this short consensus paper, we describe extreme risks from upcoming, advanced AI systems. Drawing on lessons learned from other safety-critical technologies, we then outline a comprehensive plan combining technical research and development (R&D) with proactive, adaptive governance mechanisms for a more commensurate preparation.
Claims that large language models lack intelligence are abundant in current AI discourse. To the extent that the claims are supported by arguments, these usually amount to claims that the models (a) lack common sense, (b) know only facts they have been trained with, (c) are merely matrix multiplications, (d) only predict the next word in a chain, (e) lack a world model, (f) have no grounding of symbols, (g) lack creativity, or (h) lack consciousness. Here, each of these arguments is applied, with minor modifications, to demonstrate that humans also lack intelligence. This should make us suspicious of the validity of these arguments.
Quantitative and qualitative intelligence
To evaluate the intelligence performance of LLMs, the problem can be easier accessed by separating the task into two separate metric classes: quantitative intelligence and qualitative intelligence. With quantitative intelligence we refer to the model’s inherent data storage and the model’s ability to navigate, use and remix this information, analogous to the knowledge of a human. In contrast, qualitative intelligence refers to the ability to analyze, judge, and conclude from that data storage and novel information. At first glance, this broad separation may seem obvious, especially in the context of human intelligence. Nevertheless, there is still no standardized paradigm for evaluating intelligence for LLMs and similar AI systems.,,,
Computational growth versus intelligence growth
The exponential growth of used training data with accompanying increase in model size will likely put the models in the foreseeable future to a limit regarding data availability. As a thought experiment, imagine training a model on the complete information on the internet, all text ever written, and all thoughts every human ever said. How smart could such a model potentially become given the current self-supervised training paradigm predicting a word in a sentence? Quantitatively, the model would encompass the entire human knowledge. Qualitatively it potentially could outsmart every person ever lived, being fed with countless human ideas. However, we argue that the hypothetical qualitative improvement over humans would still be on a comparable scale as the whole training basis inherently relies on human thoughts and language from which the model can not escape. In the same way, a perfect model for the language of an animal species, e.g., songbirds, would have difficulty explaining Einstein’s theory of relativity. The basis of large language models relies on their training data, which is based on the mental capacity of the underlying species. While LLMs undoubtebly demonstrate impressive linguistic precision, it remains unclear how that translates into the broader cognitive capabilities often associated with language [14 ]. This raises the question of whether language alone is a sufficient means for acquiring such capabilities [15 ]. Furthermore, the debate persists as to how a simple reorganisation of huge amounts of data might not be sufficient to achieve results that resemble human behaviour[16 ].
Relating these thoughts to the notion of “super-human” AI, we see that current and future state-of-the-art LLMs can exceed human intelligence quantitatively with ease. This is especially the case, when AI tasks are combined with their usually very short inference time. On the contrary, future LLMs that have a qualitative improvement orders of magnitudes higher than humans seem still very unlikely or even impossible with current learning paradigms. On top, truly emergent intelligence properties of such models might even be invisible to current measures. To ultimately prove or disprove emergent intelligence in concrete settings or for specific LLMs, nuanced and combined attention to both quantitative and qualitative intelligence measures will be crucial. The development of frameworks that address these issues should receive dedicated awareness by the research communities.
Undoubtedly, even without singular intelligence growth of LLMs, their societal impact is already tremendous [6 ]. With growing quantitative intelligence, models can be applied to a plethora of domains combining expertise from various fields. Tasks that currently require a team with different backgrounds could be handled by a single LLM in the future. Thus, a simple prompt can be sufficient to launch a new campaign, create a movie, or plan a medical study. These advancements are fundamental for accelerating existing processes. Access to superior technology or new physical concepts, on the other hand, is not on the immediate horizon. Accordingly, the creation of an uncontrollable superintelligence may not be the most imminent threat, but large-scale job losses, misinformation and election interference are.
In summary, current and future developments in LLMs will lead to models that represent a significant part of the entire human knowledge, but which might not quickly surpass the qualitative output of humans. To address this bivalence and to reliably examine emergent intelligence properties or even “super-human” behavior, the intelligence performance of models should be accessed with separated quantitative and qualitative intelligence metrics.
Five leading root causes of the failure of AI projects were identified
Understanding tail events is crucial for understanding the world, natural and human history, to foresee, explain or mitigate the impact of black swan events, such as existential tail risks, black swans or anomalies, catastrophes, disruptions, and natural disasters, unexpected deaths or wars. These events are rare but have a disproportionately disruptive impact or major effects, causing significant losses or gains and uncertainty and unpredictability. The probability of a tail event is low, but the consequences can be devastating or disruptive.
Artificial Intelligence (AI) should be benchmarked, tested or evaluated, by its power to identify, analyze and understand the world, why it behaves the way it is, including the causes and consequences of the tail extreme events, providing insights into the causal factors behind crucial disruptions. A machine rote learning of a range of standard human tasks, including commonsense sense and reasoning, simple coding, scripted conversations, some math problem-solving, question answering, summary generation and machine translation are hardly true benchmarks for Real Intelligent Large language models (RILLMs).
Or, to be real and true AI models, the RILLMs operate a Causal World Model to have a Deep Understanding of the Real World, with all its complexity.
By testing AI models on simulations of extreme real-world scenarios, when historical events have no value, such real-world problem-solving benchmarking reveals their true learning and intelligence, as the strengths and limitations of LLMs & CPT-x, AGI & ASI.
The history of the universe, nature or humanity is driven by extreme outlying or singular events, such as the Big Bang explosion, the Great Oxidation Event (GOE), the rise of eukaryotic organisms, multicellular life forms, the Holocene/Anthropocene or the Human Epoch, the man-machine AI era.
General-purpose artificial intelligence (AI) technologies, such as ChatGPT, are quickly transforming the way AI systems are built and deployed. While these technologies are expected to bring huge benefits in the coming years, spurring innovation in many sectors, their disruptive nature raises policy questions around privacy and intellectual property rights, liability and accountability, and concerns about their potential to spread disinformation and misinformation. EU lawmakers need to strike a delicate balance between fostering the deployment of these technologies while making sure adequate safeguards are in place.
Everything we can imagine is real
3 个月I think we need to avoid AGI at all costs in my opinion, it is a unnecessary risk