Adversity begets Leapfrog

I have been ruminating about the deep correlation of adversity and leapfrog for some time. We often say, “necessity is the mother of invention”. But adversity has a more existential urgency. And while leapfrog involves innovation, it also means a complete paradigm shift. There is a deeper significance to the phrase “adversity begets leapfrog” and that is what I want to discuss today.

The recent news about India’s successful Chandrayaan 3 mission is a prime example of this thesis. India landed a space craft successfully at the moon’s south pole and sent out a rover to explore the terrain for water and minerals that could enable future human settlement. This is an immensely complex endeavor involving many diverse engineering subsystems such as materials, propulsion, telemetry, control, guidance, and communications, as well as the core competency of delivering a large-scale project efficiently and on time. What is especially noteworthy is that the mission ran on a shoestring budget of $75 million of so, which is an order of magnitude smaller compared to the budgets of other space faring countries doing similar missions. That really causes a sense of wonder: how can an endeavor so sophisticated and complicated, a veritable leapfrog, be achieved with such a miniscule investment? Indeed, making movies about space exploration have cost Hollywood studios more time and money!

From across the world, we have generally heard voices of admiration and congratulations. Especially engineers and scientists, irrespective of their nationality, share a strong sense of camaraderie and are generous in their appreciation of well-executed advanced engineering undertakings. But there have also been condescending commentators who could not resist bringing out the worn-out trope - “What is a country with millions of poor starving people doing investing in space technology?” I feel there is a fundamental fallacy in their argument.

If you observed the Chandrayaan 3 mission control room, you would be struck by the profile of the scientists and engineers in that chamber. They are demographically very diverse - young and old, male and female, and with diverse educational backgrounds. But they all have one thing in common: they look unassuming, middle class, bright eyed and enthusiastic. They look like regular people used to making a living under constraints, always working with meager resources but still dreaming big. They are a living illustration of the idea that deep innovation is much more likely to happen when there are serious constraints even to the point of adversity.

A drought prone country like Israel makes deep advances in water conservation and irrigation. A country like Japan facing a demographic crunch makes huge advances in robotics to take care of many day-to-day tasks that would otherwise need significant human resources. African and Asian countries like Ghana and Bangladesh that lack strong financial institutions innovate into microfinancing. A country like the Netherlands which has a very low elevation excels in flood control. These are not coincidences. In fact, they are almost inevitable developments - contingent on the fact that the people caught in those circumstances had the courage to persevere and dream big.

Of course, not everyone always wins out against adversity. Indeed, it can often be overwhelming and lead to crushing despair. Make no mistake, there is nothing beautiful or admirable about poverty and adversity. The point is not to make a virtue of poverty, but rather to make the case that people with meager resources should nevertheless continue to dream big. They are much more likely to succeed in achieving leapfrog by developing elegant solutions that deliver a quantum jump, compared to those who live in affluence. That is why a country like India must invest in reaching for the moon. What is true for countries is also true for communities, companies, families, and individuals. Inspired and courageous people facing adversity come up with uniquely innovative solutions that eventually benefit all of humanity.

But can we explain why that happens? I would venture that it boils down to one simple idea – “lateral thinking”. Instead of attacking a problem frontally, adversity encourages (indeed requires!) approaching problems from the side and in increments. In India, there is a very special word for this – it is called “Jugaad”. Lateral thinking is about breaking the norm and thinking of alternatives. It requires courage and obstinacy. It requires the appetite to try many ideas quickly, fail fast, and learn fast. It also requires a deep grounding in fundamentals of science and engineering. Lateral thinking does not mean crazy thinking! Rather it is the ability to look at less probable (but still possible) solutions, instead of succumbing to the echo-chamber of the majority opinion. In computer science lingo, adversity tilts the “explore-exploit” strategy more to the “explore” side leading to rapid discovery and innovation.

Such lateral thinking is precisely why start-ups often succeed in innovation whereas big well-endowed companies fail, as has been eloquently described by Clayton Christensen in his classic book “The innovator’s dilemma”. People facing adversity come endowed with the precious “nothing to lose” mentality. Like the famous line in the movie Gattaca, they “never save anything for the swim back”, because there is nothing to swim back to!

Interestingly the converse statement may also be true: ?If adversity leads to leapfrog, then perhaps leapfrog requires adversity. If you are fortunate to have plentiful resources and are driven to innovate and push frontiers it may still be helpful to set up artificial constraints on yourself. ?Otherwise, notwithstanding your qualifications and commitment, you may not be able to come up with the ground-breaking solution you desire. Ground-breaking solutions come into existence exactly when the “normal” way of doing things does not work.

I often have this conversation with my children. I argue that they have it “too easy” - they just need to ask for something and they will get it, usually within a couple of hours from Amazon. It is not their fault, they were born and brought up in circumstances of plenty, in a bubble with similarly fortunate, educated, and worry-free peers. But I worry that this will make them soft and vulnerable in some way. Granted, when I say that, there is a sense of wistfulness on my part. Our parents loved us unconditionally and made every effort to care for us, but their humble circumstances meant that they could not fulfill our every whim and fancy. We were pretty much left to our devices and forced to find solutions. We rarely replaced anything outright with a new purchase – the unwritten rule was “fix it yourself” or “work around it”.? That is what I feel my kids may miss out on - they do not need to think laterally to solve tough problems because there are no tough problems to solve in their immediate vicinity. No need to conserve water, no need to be frugal with food, no need to make an electronic device last another year, no need to mend clothes or shoes, no need to borrow books or bicycles. Perhaps that atrophies a very important facility of the human brain: the ability to do lateral thinking. Of course, life will certainly throw them their share of challenges, and they will learn to cope “on the job”. But that process would be less painful and traumatic if they learnt to tackle enforced adversity in their childhood, almost like immunizing yourself to a disease. I must admit I usually end up on the losing side of these arguments, but I am not giving up!

Let me give you a few more examples. Those of us driving engineering projects know this very well: simply adding more engineering resources and people to a project does not make it go faster, cheaper or produce better results. The well-known mythical man-month principle prevents that from happening. The proximate explanation for this phenomenon is the difficulty in communication and management of large groups. But perhaps the more fundamental reason is that where there are too many resources, there is little urgency – there is a false sense of comfort that discourages people from hitting for the fences. Conversely, small tight engineering teams working with modest resources, aiming for daunting goals against hard timelines, often achieve great results.

Operations research provides another example. It is well-known that constrained optimization produces good robust solutions. Without adequate constraints, an optimization algorithm produces a shallow lazy solution that is neither efficient nor cost effective, and more importantly, not robust.

Machine learning practitioners are also aware of this principle: keep the model as small as possible and train it with a lot of “dropout”, and then it will actually learn something deep from the data. Otherwise, we end up with a “just-so” over-fitted model that has little ability to generalize on unseen data. In fact, an ML model’s ability to generalize and do transfer learning is very similar to a human’s ability to do “Jugaad”!

Finally, even biology provides us examples. When our body is faced with a moderate amount of starvation, it goes into hyperdrive. Doing intermittent fasting, in moderation and in a safe manner, improves thinking, memory, heart health, tissue health and physical performance. When faced with adversity, the body does better, not worse!

The social and environmental challenges we are facing today are tremendous. Climate change is literally consuming the world in fire and fury. We are running out of precious resources like water, arable land, and cheap energy. Generative AI is questioning the very basis of what it means to be uniquely human and is potentially threatening the security and stability of human society. To make headway with these challenges we need to leapfrog … produce solutions that the dominant voices of pundits refuse to even consider as a possibility. And that leapfrog will likely be achieved by those facing the most adversity. When J. F. Kennedy famously said, “We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard”, he was on to something very deep.