AI - The Perils of Accepting the Unknown

The Perils of Accepting the Unknown: A Case for Curiosity over Complacency

Science has long relied on simplifications and generalisations, focusing on the known while accepting gaps in understanding. This pragmatic approach can foster intellectual apathy toward the unknown with significant consequences. As we rapidly advance AI capabilities, we find ourselves at a similar crossroads, building and deploying systems we only partially comprehend.

Physics provides many examples of accepting partial understanding. The Standard Model precisely describes forces governing subatomic particles. But it overlooks gravity and unseen ‘dark matter’ comprising 95% of the universe—like mapping a city while ignoring the surrounding forest. Atomic models explain chemical bonds through interactions between protons, neutrons and electrons. But these common particles constitute only 5% of an atom’s mass. The rest involves a buzzing world of virtual particles and quantum behaviours—akin to focusing on a few trees while missing the complex ecosystem around them.

Geneticists emphasize the 5% of human DNA encoding proteins, dismissing the remainder as ‘junk’ of unclear function. It’s as if a builder drew detailed blueprints for a house’s rooms while ignoring the architectural elements that physically support the structure. In each case, grasping a fraction of the system enables applications—but questioning the unknown majority seems less urgent, allowing broad resignation toward the unexplained.

When the scientific status quo disregards huge unknowns, it propagates a culture of intellectual apathy and overconfidence beyond academia. Counterintuitively, advances built on simplifications are used to justify complacency about the short-cuts taken. However, history repeatedly shows blind spots and oversights eventually surface.

For instance, ‘junk DNA’ is now known to be crucial gene regulation. Dark matter’s gravitational pull shapes galactic structures. Strange quark interactions produce nearly all visible mass in the universe. Such paradigm-shifting discoveries rarely arrive through begrudging exploration of knowledge gaps. Instead, they disrupt incremental progress, exposing flaws in previous understanding.

Society has suffered the consequences of this mindset many times over. Engineers overlooked complex climate feedbacks while designing fossil fuel infrastructure, enabling immense environmental harm. Economists disregarded speculative financial assets and systemic risks, contributing to devastating economic crises. When key variables are dismissed, no amount of precision elsewhere prevents sudden, catastrophic failure. However, each blind spot we paper over only bolsters faith in existing models, diminishing the drive to ask foundational questions.

AI: Replicating a Perilous Approach

Few fields exemplify this complacent attitude more than artificial intelligence. Developers create highly optimised neural networks using large but still limited datasets and constrained objectives. Corporations rapidly deploy AI systems seeking profit without deeper consideration of societal impacts. The current approach fixates on measurable optimisation and incremental gains, targeting the 5% of intelligence that is easily quantifiable, profitable and comprehensible. The remaining 95% involving complex reasoning, contextual adaptation, generalisation capabilities and emergent behaviours remains opaque.

Of course, grasping advanced AI systems in their entirety is not currently possible. However, the nonchalant attitude toward their unfathomable complexity should deeply concern us. AI research aims to push performance boundaries on narrow benchmarks, rather than gain basic insight. Fully autonomous systems are deployed before we remotely understand their failure modes. Algorithmic black boxes designed by a homogenous tech elite are readily ceded authority over public health, human rights, and democracy itself. Each added layer pushes the technology further past human interpretability.

Soon systems will self-improve through recursive techniques like automated neural architecture search. At that point, unravelling their intricacies becomes impossible. Some researchers have raised urgent concerns, but calls for oversight from figures like Elon Musk draw ridicule. Developers acknowledge limited understanding only in principle, while urging blind trust in their internal governance. Given past disasters linked to such arrogance, this seems dangerously cavalier.

Advanced AI brings immense promise but also carries unpredictable risks. Incentives driving its progress reward short-term capabilities over safety. If misguided objectives emerge or unpredictable behaviours arise, the consequences could exceed anything humanity has faced. By the time any issues are detected, it may be too late for course correction.

The Perils of Oversimplification

Clearly, good intentions and surface-level comprehension alone cannot prevent disasters spawned by complexity. Some argue AI is too important and sophisticated for external oversight. But in fact, complexity demands more inclusive, multidisciplinary scrutiny. Those building advanced systems cannot evade ethical responsibility for considering implications and risks. Governments must urgently implement frameworks supporting transparency and accountability, such as independent auditing bodies. Citizens should have a voice in deliberating about AI applications that could profoundly impact society. Only through collaboration spanning developers, policymakers and the public can we work to close today’s expansive understanding gap.

The ancients warned that hubris and ignorance doomed civilizations. Today, that threat remains as urgent as ever. Powerful technologies feed into a culture disconcertingly comfortable with yawning unknowns and simplistic models. If history is any guide, this arrogance virtually ensures unforeseen consequences will emerge. While we cannot instantly illuminate all of AI’s shadows, we can shed the dangerous assumption that it is safe to blindly create and deploy it while in the dark. There are always more questions to ask, perspectives to integrate, and fundamentals to rethink. True progress demands the courage to challenge assumptions, even when it slows advancement.

To responsibly advance AI, we must shed the hubris that it is safe to proceed in darkness. Citizens should advocate for policies like algorithmic auditing, support research into AI ethics and safety, and engage in community discussions about AI’s societal impacts. Journalists can investigate AI systems as rigorously as political leaders. Educators can teach critical thinking around new technologies. Experts in law, ethics and philosophy must help address AI’s profound challenges. There are always more questions to ask and perspectives to integrate.

With diligent curiosity and humility, we can collectively illuminate shadows and guide innovations like AI to empower our shared future. But it begins with action. Will you join in this essential work? The unknown inspires fear in many. However, with principled questioning and openness to different viewpoints, we can build an AI future defined by wisdom and humanity rather than compounding the errors of the past. Progress takes courage, caution and collective care. If we take up this challenge, perhaps one day the shadows will slowly recede, revealing AI we truly comprehend, control and trust.

First published on Curam-AI