Why diversification maybe a better idea than hyper-specialization (A review of David Epstein's Range)

For years, we have been told that the only way to “succeed” in life is to start early and subject oneself to perpetual, systematic and focused practice – specialization has been sold as the mantra for exceptional achievement and a fulfilling life.

David Epstein gives a convincing argument claiming that this might not be the only route to success and tries to explore the anthropological, psychological, and historical reasons for the same. He goes on to cite numerous anecdotal and research evidence across fields ranging from sports and music to scientific research from throughout history all the way up to the present to convince his reader that diversification might in fact be advantageous over hyperspecializing in most endeavors that human beings undertake.

Why Generalists triumph in a Specialized World?

Epstein sets out by exploring a fundamental question – What is the source of a specialist’s expertise? The answer (which might seem oversimplified at the first go) is “pattern recognition”. Chess Grandmasters, he argues, have a “photographic memory” when it comes to arrangement of pieces on a board, they can perfectly reproduce a game situation, just by looking at it for a couple of seconds; but there’s a catch – they can only do so if the arrangement is a logical outcome of a chess game; if pieces are just assorted randomly in an arrangement that would never occur in an actual game, Grandmasters perform as badly as anyone else. This is because when they look at the board, they don’t remember the position of each piece, instead they just see the “pattern” of game that would have led to the game outcome which they see on the board.

This phenomenon is not particular to chess players, expertise in most domains is an outcome of skilled pattern recognition, people who have developed extraordinary skill through perpetual deliberate practice are called as savants; musician savants excel exceptionally at tonal music, the genre of most pop and classical music, they can play thousands of songs out of their memory seamlessly, as if they have a perfect memory; on the contrary, when they try their hand at atonal music, in which successive notes do not follow a familiar pattern, they fail to recreate the compositions. Similarly, artistic savants reproduce images of real-life objects better as compared to more abstract depiction. Fire fighters rely on the pattern of the progression of a fire to instinctively build up their strategy to extinguish it, there are many examples across domains which lead to the same conclusion, specialization is nothing but advanced pattern recognition.

In 2009, Psychologists Daniel Kahneman and Gary Klein came out with a paper, which explained which domains of human endeavors really require pattern recognition as the primary skill, they called these as kind learning environments (Kahneman & Klein, 2009) – these include sports like chess and golf; poker and firefighting. They are domains where patterns repeat over and over and feedback is usually very accurate and quick; in golf or chess, a ball or piece is moved according to rules and within defined boundaries, the player can directly observe the outcome and attempt to correct the error, but most environments that human beings face aren’t kind, they’re wicked – the rules of the game are often unclear or incomplete, and there may not be any repetitive patterns at all. Examples of these include everything from scientific research, to political analysis and management consulting.

In most wicked environments, a dynamic problem-solving skill is more important than a repetitive pattern recognition skill. Domain experts more than often tend to become slaves of their “familiar tools” and this can lead to a disastrous outcome when dealing with an unusual problem. The world we live in is becoming more wicked by the day – it is changing faster than ever with technological advancements to the extent that every few years we have a substantial number of people engaged in careers which were unheard of, until a couple years back.

Throughout the book, Epstein uses anecdotes and research evidence to show the advantages that generalist outsiders might have over specialized experts in any domain; here are a few -

A sampling period: In various studies undertaken by John Sloboda, one of the most influential researchers in the psychology of music, he found that the single most important factor that differentiated exceptional musicians from others was the breadth of instruments they picked up before finally specializing in one (Sloboda, 1986). People don’t know beforehand what vocations or activities will best match their skill-set, they only learn this by sampling through a range of activities before they finally find out the one that matches their skillset the best. There is no shortcut to this process. Psychologists highlight various paths to excellence, but the most common one is a sampling period.

The Outsider Advantage: Innocentive is an open innovation and crowdsourcing company headquartered in Massachusetts, US which posts highly challenging problems from numerous hyperspecialized domains which include astrophysics and molecular biology. Many of these problems are the ones which domain experts have failed to solve for decades and there have been numerous instances where people from unrelated domains have been able to solve them in months. In one such case, NASA was unable to predict solar particle storms, radioactive materials spewed by the sun that can gravely damage astronauts and the equipment they depend on. Solar physicists were stuck at this problem for three decades before they finally posed it through Innocentive in 2009. Within six months, Bruce Cragin, a retired engineer, solved the challenge using radio waves picked up by telescopes. Epstein gives numerous examples to show that people from unrelated domains have an outside advantage over the domain experts. They are able to approach the problem from a different perspective using analogies from their own domain which escape the deep but narrow vision of domain experts.

Learning to drop your familiar tools: A lot of studies have documented that when a fire gets completely out of control and experienced firefighters are asked by their foreman to drop their tools and run for their lives, they refuse or fail to do so and end up losing their lives in the process. The behavioral reason responsible for this is that in unfamiliar situations, rather than adapting, experienced groups tend to become rigid under pressure and regress to what they know the best. This lack of adaptation and flexibility is an inevitable consequence of experience and expertise. One of the primary reasons for the failure of the NASA’s Challenger mission which led to a loss of seven lives and billions of dollars was that a NASA manager had categorically denied to cancel the launch based on the visual evidence of a possible malfunction presented by one of the engineers unless he could produce data to back his claim. NASA’s culture of relying only on data-based evidence and failing to drop familiar tools is what led to this gross tragedy. The manager failed to drop his familiar tools in an unusual situation.