The Quest for "What is Where?"
[From a keynote at the European Rover Challenge last week in Krakow, and celebrating the 150th anniversary of "The Lunar Trilogy" by Jerzy Zulawski. Video at the end]
The conference hall was buzzing. Outside, teams of students around the world were competing with their DIY robots on a Mars landscape, driven by the same age-old question that has fueled explorers for centuries: "What is where?"
This is the question that echoes not only across space but also on our own planet. Mars and the Moon call to us, but so does Earth. While we think we know what's where on our home planet, do we? Yes, we have satellite images of forests, cities, and rivers—but can we really count the trees? Can we track how many we lose? Do we know where the fires or floods are? We have the data, but we haven’t fully unlocked its potential. It’s like hoarding on books in language we barely understand. What if we could answer this question on an unprecedented scale, harnessing the transformative power of AI?
The Gap Between Human Intuition and Machine Perception
The challenge lies in the stark contrast between human and machine perception. It is hard to explain why machines are so bad reading images because our minds are so innately good at it. We humans effortlessly interpret the world around us. Decades of education atop millennia of evolution. A satellite image reveals a lake, a river, a suburban area - we grasp its essence instantly. Machines, however, struggle. They see pixels, not meaning. To them, the flat desert, grassy field and a concrete jungle are all patterns of signal and noise, without explicit labels.
This disconnect stems from the limitations of traditional computer vision, which requires painstaking, step-by-step instructions for every task. It's a far cry from the intuitive understanding our brains possess.
> Earth awareness is stunted due to high cost of using the data we have.
The AI Revolution and Earth Observation
The recent AI boom has revolutionized countless fields. We have AI transcribing conversations, translating languages, even detecting diseases. In fact, in all cases, we see the exact same AI architecture, “Transformers”, the T in ChatGPT. So the question is clear, what about Earth? Why not apply this transformative power to enhance our understanding of our planet?
This is where foundational AI models step in. Trained on massive datasets, these models 'pre-digest' the visual world, recognizing patterns and features across countless images. It's akin to teaching a machine the basic grammar of visual language, enabling it to comprehend increasingly complex sentences.
领英推荐
Clay: Pioneering AI for Earth
At Clay , the non-profit I lead, we're building foundational models specifically for Earth observation. Imagine summarizing a vast satellite image into a few hundred numbers, then reconstructing the image from that summary. It sounds like science fiction, but AI can achieve this, forcing it to distill the most crucial elements of the image. On of the reasons this works is that we can literally give it trillions of? examples to learn. Once they finish, by necessity, the AI has learnt extremely nuisance ways to explain the wonderful variations of our planet.
These summaries, or "embeddings," unlock a treasure trove of insights. Want to locate all the rugby fields in a city? Simply click on one, and the AI will leverage its embeddings to pinpoint similar areas. This same principle can be applied to identify deforestation, track floods, or monitor any feature imaginable. We’ve seen better than 90% accuracy across several key examples, all orders of magnitude faster than traditional methods.
We firmly believe that this iAI wave is neither the solution to our problems nor hype. It is a tool, and a tool we can deploy with astounding results. We simply cannot afford to ignore AI for Earth. In fact, this technology should be accessible to everyone. That's why Clay's models are open and free, empowering researchers, businesses, and anyone with a curiosity about our planet.
A New Era of Discovery
Standing at the cusp of this AI-powered revolution in Earth observation, I'm reminded of the thrill I felt reading Jerzy Zulawski's "The Lunar Trilogy" written more than century ago. The excitement of exploring the unknown, of pushing the boundaries of human knowledge, is palpable. And beyond that, which leaders we follow, and what kind of value we value in society, specially around science, curiosity and critical thinking.
Embeddings may seem like abstract mathematical concepts, but they represent a new lens through which to view our world. They hold the key to answering "What is where?" at an unprecedented scale, paving the way for solutions to some of our most pressing global challenges.
As we celebrate the ingenuity of those building robots outside,? those inventing and exploring with AI, let's use this incredible opportunity to shape the creation and expectations of a tool that might very well change how we rediscover our own planet. The quest for knowledge continues, and the tools at our disposal are more powerful than ever before.
Watch the full keynote presentation below.