The Potemkin Villages of AI

It's an amazing time to build AI-based tech. It's also a scary time to build AI-based tech. To make sense of the swarm of demos and announcements released every week, it greatly helps to understand the concept of a Potemkin Village.

The phrase Potemkin village bears the name of Russian statesman Grigory Potemkin. He wanted to impress the empress. The story goes that he built makeshift facades of fake villages which seemed real to the empress as she toured through her empire. The story is based on rumours and may have never taken place but the concept has endured and is used in courts, politics and elsewhere.

The essential underlying observation is that it's much much easier to build a facade of a complex system that seems fully functional from a given perspective than it is to build the system itself. This is critical to understanding AI demos. In fact, this is what demos are: a demo of a product is a 'Potemkin product'.

Many people, it seems, see a demo and infer that a perfectly functional product is just a matter of time and is just around the corner. This is actually understandable - it can be hard to distinguish a facade from the real thing. This leads to some of the misconceptions that are popular today - that "AI will do all work very soon and that we can all relax" or that "superhuman AI that will replace all engineers and scientists is coming next year". The more you actually read AI research or work on building real-world solutions with AI, the more it becomes clear that this is very far from reality - at least the near-term reality.

In fact, there is a critical perspective to this related to my field of AI research (reliable deep learning). A lack of reliability - non-robustness and miscalibrated overconfidence - on the "long tail" of scenarios that are out-of-distribution relative to typical training data is one of the fundamental unsolved problems in AI. Solving this long tail is a big part of the process of getting a system to work on real-world scenarios.

A recent article I wrote for Cantor's paradise (a popular math publication on medium) is relevant in this context and mathematically deconstructs the phenomenon of Potemkin Villages in the context of a magical 'genius' solution to a challenging math problem.

This parallel between AI demos and Potemkin Villages is nuanced - it requires making the distinction between useful and perfect. Consider for instance, the demo of Durable, the website creation tool. The demo does correspond to an excellent released product which is already useful to create websites. However it is a long arduous route from there to making Durable a product that replaces a website design and development team. Creating actual websites with effective messaging isn't as simple (yet) as typing in a paragraph on Durable, chatting with it to make changes like you would to a human and then clicking on a "publish" button. Replacing human website creation teams will take a lot of problem solving and sophisticated reasoning capabilities. There are AI influencers who will hype up Durable's release as "say goodbye to your website creation team". They are incentivized to generate hype and obfuscate perception by equating Potemkin villages with real villages. Irrespective of all that noise, Durable has the potential to revolutionize website creation. This distinction is the critical nuance that is missing in the popular perception of AI today.

PS: The phrase "Potemkin Village" also has a negative connotation of a fraud. This does not apply to the incredible AI demos we're seeing today. In fact, these come from some of the best teams of problem solvers solving some of the most important problems of the present times.