Unpacking the Neural Network Hype: Separating Promise from Performance

Neural networks, despite the excitement surrounding them, are expensive, opaque, unreliable, and have an insatiable appetite for data. Their potential is inherently limited and they often fail to meet expectations.?

The increasing overlap between statistics, machine learning, and AI has led to significant confusion, as the distinctions between these fields become less clear and their methodologies more intertwined [1]. Traditional statistics are straightforward, operating within clear parameters and guarantees. However, in the real world, these guarantees often do not hold, resulting in misleading conclusions.?

Statistics require more than mechanical application; they demand ongoing assessment of whether the method suits the context. When misunderstood, statistics can be dangerously misapplied, as seen in the financial crash of 2008 and the ongoing replication crisis in medical research.?

Once a niche within artificial intelligence, machine learning became distinct during the AI Winter of the 1990s and 2000s [2], concentrating on data analysis methods with uncertain theoretical foundations. It separated itself from AI, but now machine learning is considered a subset of AI.?

Machine learning excels with "messy" datasets that traditional methods struggle with, making it appear more powerful, but its success is often unpredictable due to the absence of solid theory supporting its effectiveness in specific cases.?

Unlike traditional statistics, which provide clear methods to assess and interpret results, machine learning depends on empirical tests: does the model accurately predict past data? This approach focuses less on understanding and more on determining if the outcome is appropriate.?

Machine learning depends on historical data, assuming that future data will resemble the past. However, predicting the future is challenging, and the definition of "similarity" can vary depending on the task and model.?

Uncertainty makes machine learning more error-prone than traditional statistics. Using it responsibly requires skepticism and continuous accuracy monitoring.?

From Tool to Focal Point

Today, "artificial intelligence" is often associated with one technique: error backpropagation [11], frequently mislabeled as "neural networks" or "deep learning." This method, unique in its applications, has significant drawbacks.?

Neural networks were initially just another tool in the machine learning toolbox, not the main focus. However, with their wide range of applications, they are now considered more powerful than any other method. Recent improvements in backpropagation have raised hopes that it could realize the dream of advanced artificial intelligence, leading to the perception that AI is a subset of backpropagation's applications.?

Equating AI with backpropagation is misleading. Backpropagation has limitations that humans and other AI methods do not. Many neural network applications don't truly qualify as AI. When I question AI applications, people often point to neural networks, blurring the distinction.?

There are many '“smart, neural network powered” kitchen appliances (https://t.ly/e_f-j) where I believe traditional control theory methods could be as effective as a neural network. They would be simpler and offer mathematical assurances.?

Why did they opt for a neural network? It could be due to marketing hype. Companies frequently promote "AI" to enhance their image, even when it's not involved. Perhaps a neural network was selected for its sophistication, or the engineers found it more thrilling to work with "AI" than conventional methods.?

If there’s a neural network inside, it’s likely a small one, either manually configured or meticulously designed to ensure appropriate behavior.?

The designers of the kitchen appliances likely incorporated safety mechanisms to prevent malfunctions, irrespective of the actions of the neural network. Implementing such precautions should be standard when using neural networks for purposes beyond mere novelty.?

Flaws in Neural Networks

AI is said to transform industries, but a closer examination reveals:?

• Many AI applications do not involve genuine "intelligence."?
• Few AI revolutions are genuinely significant, both technically and economically. Many AI-impacted markets are smaller than anticipated—sometimes even less significant than the rice cooker market.?
• Many of these apps are either unsafe or irresponsible.?

Methods that are reliable and well-understood often outperform neural networks.?Traditional statistical methods or simpler machine learning techniques are usually sufficient in many cases. People often resort to backpropagation because of its popularity, even though other alternatives might be more effective. When backpropagation operates efficiently, it’s typically because the problem is straightforward, which makes the use of this complex method unnecessary.?

Backpropagation can achieve unique results for certain tasks. It requires significant supercomputer resources, struggles with many tasks, and is inherently unreliable, making it unsuitable for most applications.?

From an engineering perspective, neural networks have significant flaws:?

• They require significant computational power.?
• Achieving good results in new real-world scenarios is often impossible, and when successful, it typically demands years of effort from expensive specialists.?
• Backpropagation can be unstable and may not always be reliable, which makes them unsafe for some practical applications.?

Despite its flaws, backpropagation networks can occasionally achieve unique feats. Even if they often fall short, some outputs are astonishing. This makes modern AI intriguing. Take DALL-E, for example— it doesn’t always deliver what you want, and precise results are a challenge. But with effort, it might produce something close, and sometimes the inaccurate results are impressive, even if the people have six fingers. This makes it a fantastic toy, great demo material, and potentially useful for commercial illustrations—provided someone counts the fingers. However, it’s unsuitable for serious engineering applications.?

Due to the unreliability of backpropagation networks, they should only be used when necessary and under specific conditions:?

• When bad outputs are neither costly nor harmful.?
• When every output can be reviewed by a competent person.?
• When the error rate is driven so low that failures are nearly nonexistent, there is justified confidence that this will hold true despite real-world volatility, uncertainty, complexity, and ambiguity.?

Spurious Correlations

Most deployed systems fall into one or both of the first two categories:?

1. Backpropagation networks improve digital photography; errors are either obvious or insignificant.?
2. Image generators such as DALL-E are primarily used for entertainment and aesthetics, making them mostly harmless—at least for now. However, there are growing concerns about the misuse of fake images in scams, propaganda, and the impact on artists’ livelihoods.?
3. Systems like Copilot produce useful but unreliable code, requiring highly skilled software engineers to verify and correct each output. Connecting AI to critical systems, especially in policing, criminal justice, and the military, is reckless and should be condemned. Kapoor and Narayanan highlight real-world examples in their work on AI risk prediction tools. (https://t.ly/dx76d)?

The internet, our most powerful tool, faces significant risks from AI text generators. The primary concern is the inundation of AI-generated spam, which diminishes the usefulness of Google Search. Consequently, many are now turning to ChatGPT as an alternative for web searches.?

Backpropagation networks can often mislead creators due to subtle statistical quirks that are difficult to identify and correct.?

Backpropagation trains a network using known input/output pairs (training data). After training, the network predicts outputs for new inputs. For example, recommender systems predict ad clicks, while models like GPT generate responses.?

Backpropagation can exploit spurious correlations in the training data—patterns that do not hold in real-world use. It is easier for backpropagation to latch onto these accidental patterns than to solve the intended problem.?

When backpropagation exploits spurious correlations, it may perform well during training but fail in real-world applications. Detecting and preventing this deception is challenging.?

If you train a supermarket AI to distinguish apples from oranges using images, it may perform flawlessly in tests. Nevertheless, in the warehouse, it could mistakenly identify a green, bruised apple as an orange.?

If your training data did not include bruised apples, you would never realize the AI was being misled. It associated red with apples and orange with oranges, capitalizing on a mistaken correlation. This type of error is frequent, and a significant part of AI development is dedicated to identifying and addressing these problems.?

Backpropagation networks make unreliable predictions with unfamiliar inputs based on similar training data cases[3].?

Polishing the AI's Image with Alignment

Training involves learning from experience. Training data consist of past experiences—situations (inputs) and outcomes (outputs). Recommender AIs utilize past click data, while text generators use previous texts. Interpolation refers to the assumption that new inputs will yield outputs similar to those of the most similar training inputs.?

Some dismiss backpropagation networks as mere interpolation, but this underestimates their capabilities. Similarity isn’t just numerical closeness; it’s context-dependent. Training creates a “latent space” where numerical proximity indicates similarity for prediction purposes [12]. Networks that are adequately trained can make intelligent predictions by positioning similar inputs near each other in this space. This process abstracts task-relevant categories from features, enabling effective interpolation.?

Good AI performance requires dense sampling in the vast and nonlinear latent space [9]. Similar inputs might map to different locations, resulting in random outputs without similar training data. This dependence on “big data” explains why AI requires more data than humans.?

Occam’s Razor advises us to assume the least amount of abstraction necessary to explain a network’s behavior. Outputs usually resemble training examples or their combinations.?

Text and image generators function effectively because they are trained with vast amounts of data, which populates the latent space. Present models are trained on trillions of words, which can skew our perception of their intelligence.?

Backpropagation systems can fail. Similarity in their latent space doesn’t always align with our expectations. For example, requesting DALL-E to produce a photorealistic image of someone doing a somersault might result in a surreal outcome, such as depicting that person with three legs. This discrepancy underscores why these systems are adept at generating fantastical scenes.?

AI systems often struggle in real-world scenarios that deviate significantly from their training data, resulting in unpredictable behavior and requiring the AI to extrapolate instead of interpolate.?

Interpolation-based methods and conventional statistical techniques are susceptible to these problems. AI systems that employ statistical methods are likely to be unsafe in open-ended real-world scenarios, which are more complex than any training dataset.?

This skepticism extends to “alignment” approaches such as Reinforcement Learning with Human Feedback (RLHF), which aims to prevent undesirable outputs by training on a database of “good” and “bad” interactions. However, this method still relies on the same statistical foundations, making it susceptible to the same pitfalls.?

Reinforcement Learning with Human Feedback (RLHF) was essential [10] for ChatGPT’s success [4], helping to minimize negative public relations from undesirable outputs. However, presenting RLHF as a method for AI safety alignment is not accurate. It tackles minor ethical issues but doesn’t address the vital concern of harm prevention. Although chatbots currently pose a low safety risk, labeling RLHF as an alignment method could result in complacency. Even with a reduction in undesirable outputs, chatbots continue to produce them regularly.?

RLHF, like all machine learning methods, is unreliable and probabilistic. It should not be used where “bad” outputs could cause serious harm. Even with acceptable training probabilities, real-world behavior remains unpredictable due to varying input distributions. It is impossible to foresee all potential AI misbehaviors to train against.?

Brittleness

Engineering strives for graceful degradation—machines should fail gradually, not catastrophically. Text generators frequently fail spectacularly, generating falsehoods, nonsense, or toxic language. Detecting out-of-distribution inputs and refusing to proceed might help, but if it depends on backpropagation, it won’t be reliable. The similarity metric might be flawed, resulting in bizarre judgments.?

Brittleness, the opposite of graceful degradation, is a serious concern. It’s prudent to assume that new AI systems could be hazardous and should not be deployed hastily. However, it becomes problematic when people integrate brittle AI with critical systems.?

Intelligence relies on reasoning when faced with unprecedented situations. We use deductive or causal logic to solve problems, such as scaling a sauropod to 70 meters, which is unlikely due to weight increasing as the cube of length. We might explore various body plans and apply engineering techniques to assess plausibility.?

Many researchers argue that backpropagation networks lack the ability to reason logically, which limits their intelligence. They suggest coupling these networks with logical reasoning systems or training them to reason.?

However, this approach is risky. Networks using backpropagation that struggle in new situations can be dangerous, potentially causing accidental harm. More competent AI systems acting out of distribution could intentionally cause harm, which is a frightening scenario.?

We should not pursue AI rationality unless it is both morally and cognitively sound. Systems based on backpropagation cannot be reliably rational. Training them to approximate rationality and expecting consistent behavior could lead to disaster.?

Two major misconceptions impact AI decision-making:?

1. Neural networks are unfathomable: This myth suggests that it is impossible to understand them, so there’s no point in trying.?
2. The belief that neural networks are the only path to advanced AI discourages the exploration of alternative methods for achieving advanced artificial intelligence.?

These myths portray AI systems as unreliable, which experts and decision-makers accept as inevitable.?

A frequent question is, “If backpropagation is flawed, why use it?” The answer being: it offers unparalleled capabilities. Yet, this isn’t the entire story. Many believe that since natural intelligence originates from biological brains, artificial intelligence should stem from artificial brains.?

Despite the differences between “neural networks” and biological ones, the terminology confuses both the public and professionals [8], fostering a belief that backpropagation is the essential method and discouraging exploration of alternatives.?

The Neural Misnomer

To promote safer AI, we should avoid using the term “neural” to describe backpropagation networks. The use of “neuro” obstructs progress and safety in AI development. Intellectual honesty and clarity are essential for the reliability and safety of AI systems.?

AI hype propaganda suggests that trained networks operate on intuition rather than rationality [5], presenting them as incomprehensible. It asserts that these networks use a holistic approach, unlike the reductionist symbolic AI, with intelligence distributed across billions of adjustable values. This myth depicts neural networks as mysterious black boxes, lacking explicit representations.?

Backpropagation networks are made up of small, specific components that have identifiable functions. They carry out understandable computations in engineering terms, not mystical holistic processes.?

The notion of inscrutability benefits both technologists and decision-makers:?

• Technologists often claim that the inscrutability of their products absolves them from needing to understand how they work, allowing them to focus on creating impressive demos instead of tackling complex real-world tasks.?
• Decision-makers prevent technologists from delving into the application domain, avoiding potential safety concerns or exposing dubious agendas.?

Inscrutability benefits decision-makers by making it difficult to hold them accountable for negative outcomes. Authorities justify deploying questionable AI systems by alternating between rationalist and anti-rational descriptions, as necessary. AI is promoted as a powerful technology that must be utilized, yet its errors are considered inexplicable and must be accepted without question.?

The myth of AI hype obstructs progress and safety in AI development. Intellectual honesty requires us to abandon the misleading idea of neural networks as incomprehensible black boxes and acknowledge their understandable, engineering-based nature.?

The hype around AI often aligns with scenarios of extreme risk and benefit. Networks using backpropagation are portrayed as minds, which can seem frightening. Without analysis, their capabilities might seem limitless, leading to narratives about planetary destruction or utopian solutions.?

Propaganda around AI hype suggests that neural networks can achieve general artificial intelligence, capable of performing any task that human brains can, portraying them as a magical solution for all problems without the need for specific engineering.?

Unassisted backpropagation often fails. Early achievements in the ‘80s were frequently due to self-deception. Contemporary successes employ backpropagation alongside task-specific mechanisms: convolutions for images, tree search for games, sequence alignment for proteins, and transformers for text. These achievements depend on extensive, ad hoc algorithmic adjustments lacking theoretical foundation.?

From Marginal to Dominant

The basic backpropagation algorithm is straightforward, like Newton’s method for identifying function maxima or minima through gradient descent. It reduces the error between the network’s output and the correct output for each training input, optimizing the network’s fit to the data.?

However, this basic algorithm often fails. Various enhancements address its shortcomings, such as regularizing the error function to prevent overfitting. These modifications are improvised and not theoretically derived, with their effects controlled by poorly understood hyperparameters.?

Achieving good results with backpropagation depends on fine-tuning hyperparameters [6]. Since there’s no theoretical foundation, neural networks are often developed through semi-random adjustments or based on intuition rather than through principled design. When no hand-chosen set of hyperparameters performs adequately, researchers undertake hyperparameter searches, experimenting with numerous combinations to discover an effective one. This continuous need for adjustments might suggest a fundamentally flawed approach.?

It is widely recognized that the effectiveness of backpropagation is mysterious, despite extensive adjustments. Although the concept of gradient descent is straightforward in theory, it is challenging to understand why overparameterized function approximation does not lead to overfitting or how error signals effectively propagate through deep, non-linear networks without losing significance. These are scientifically fascinating questions, and exploring them could lead to advancements in AI.?

In practice, making backpropagation effective involves hyperparameters to fine-tune each modification. Each adjustment has a clear abstract rationale, but none is effective on its own, and it is not clear why they sometimes work well together.?

If it seems that the resulting system might have unpredictable properties and fragile performance, that is often the case. The dominance of backpropagation in AI may appear inevitable today, but it was once considered a marginal concept. Recent successes might have validated it, or significant funding steered progress in a less-than-ideal direction.?

Backpropagation’s inefficiency, inscrutability, and unreliability are significant flaws. Its current dominance might be a historical accident. In technology, inferior designs often overshadow better ones if they gain early momentum through funding, hype, ease of use, or random chance.?

The Fork

AI research has traditionally been split between theories focusing on minds and brains [7], both of which likely have flaws that have hindered progress. Symbolic AI, which relies on logical reasoning, faltered in the late 1980s, resulting in an “AI winter.” This downturn was not merely technical but was also based on fundamental principles.?

Neural networks, inspired by an inaccurate 1940s theory of biological neurons, gained renewed interest in the mid-1980s as symbolic AI struggled. They appeared to circumvent the primary problems of symbolic AI: combinatorial explosions, vagueness, and the requirement for manually coded knowledge. This revival sparked a conflict fueled by excessive enthusiasm between proponents and mainstream researchers.?

By the early 1990s, the initial promise of backpropagation had faded. Technical progress in the 1980s often didn’t scale, and the excitement was more about the idea that neural networks mimicked brains than actual results.?

The rise of backpropagation might be more attributable to historical circumstances than its inherent superiority. Both symbolic AI and neural networks have significant limitations, and the search for better AI methods continues.?

During the AI winter, the focus of machine learning was on statistical optimization, while neural networks were viewed as unconventional and inferior. Mainstream research sought to address challenges such as overfitting, local minima, high computational costs, and the requirement for large amounts of data. Advances were frequently achieved through unprincipled algorithmic hacks that enhanced performance without theoretical support.?

Neural networks gained prominence in 2012, achieving significant victories in the ImageNet classification competition. This success can be attributed to two main advances:?

1. Modern backpropagation networks incorporate atheoretical add-ons such as max-pooling, batch normalization, and dropout, addressing issues like overfitting and local minima without biological analogies.?
2. GPUs, originally developed for video games, made neural networks manageable, surpassing the performance of older supercomputers.?

Despite these advances, the success of backpropagation was partly due to exploiting spurious correlations in benchmarks such as ImageNet. This attracted the attention of technology giants, leading to substantial funding for research on neural networks, overshadowing all other AI technologies combined.?

We might benefit if the funding, talent, and computational resources were allocated to a range of research approaches. Investigating different methods could result in a superior mix of capability, efficiency, and safety. Depending solely on backpropagation appears imprudent.?

No other AI method had made significant strides for decades, prompting the question of why invest further? However, during the AI winter, these methods received minimal attention, meaning their apparent lack of progress could be misleading.?

The increase in popularity of neural networks today can be attributed as much to hype as to actual technical achievements. AI labs, directed by experts who participated in the neural-vs.-symbolic debates of the 1980s, are determined to demonstrate that they represent the route to human-like AI because they resemble the brain’s functions—even if that comparison is not accurate. They continue to position themselves in opposition to symbolic AI, perpetuating a debate that has already been resolved.?

But did neural networks truly triumph? Or did random algorithms, which have no biological basis, merely leverage supercomputers effectively??

The past decade has shown that investing billions in advanced software can produce impressive results that are often misunderstood, unreliable, and unsafe, despite lacking a theoretical basis.?

Conclusion

The prevailing emphasis on backpropagation in AI research favors eye-catching demonstrations and mathematical optimization, misleadingly referred to as “learning.” This focus overlooks the scientific exploration of how networks function and the development of dependable, efficient solutions. Consequently, we possess potent methods that deliver remarkable outcomes but fall short in reliability and efficiency for particular issues.?

To address these challenges, we should become more skeptical of spectacles, decrease the focus on mathematics, and treat AI research as a combination of science and engineering.?

A deeper understanding might result in the replacement of backpropagation and GPTs with safer, fundamentally different technologies.?

1. Artificial Intelligence (AI) is a broad field aiming to create intelligent machines. Machine learning, a subset of AI, focuses on algorithms that improve through experience. Statistics, while often used in both, deals with collecting, analyzing, and interpreting data.?
2. The 'AI Winter' refers to a period in the 1990s and 2000s when funding and interest in AI research declined significantly, following a cycle of hype and subsequent disappointment in AI capabilities.?
3. AI systems often work by interpolation - making predictions based on data similar to what they've seen before. However, they struggle with extrapolation - making predictions about situations very different from their training data. For instance, an AI trained on images of cats and dogs might easily distinguish between the two, but fail when shown a picture of a fox.?
4. RLHF is a technique where AI systems are trained using human input to guide their learning process, aiming to align AI behavior with human preferences and values.?
5. The 'black box' nature of neural networks makes it difficult to understand how they arrive at their decisions. This lack of transparency can be problematic, especially in critical applications where we need to trust and verify AI decisions.?
6. Training AI systems involves adjusting many internal settings, called hyperparameters. This process is often more art than science, requiring extensive trial and error to achieve good results.?
7. AI research has historically been divided into two main approaches: symbolic AI, which focuses on logical reasoning and explicit knowledge representation, and neural networks, which attempt to mimic the brain's structure. Symbolic AI dominated until the late 1980s, when neural networks gained prominence. This shift led to the current deep learning revolution.?
8. While neural networks are inspired by the brain, they differ significantly from biological neural systems. Modern AI 'neurons' are vastly simplified mathematical models, and the learning processes are fundamentally different. The term 'neural network' is more metaphorical than literal.?
9. AI systems, particularly deep learning models, require vast amounts of data to function effectively. This 'big data' requirement is both a strength and a limitation of current AI technologies.?
10. AI alignment refers to the challenge of ensuring that artificial intelligence systems behave in ways that are beneficial and aligned with human values and intentions. This is particularly crucial as AI systems become more powerful and autonomous.?
11. Most current AI systems are examples of narrow AI, designed for specific tasks. General AI, which would match or exceed human-level intelligence across a wide range of tasks, remains a distant goal and subject of much debate.?
12. In machine learning, 'latent space' refers to a compressed representation of data that captures its essential features. It's like a map that the AI uses to understand and generate new data.?

?

?