Fiefdoms of specialization and the next frontier of knowledge

This is part of a series of posts about operational performance, customer success, and decision-making, as well as more general topics about work and life. Some of these are published earlier elsewhere, and the newsletter #zenofbusiness is designed to bring these together for you.

The following article is a fresh take from my writing desk on the nature of knowledge and how decades of teaching and learning practices, as well as management approaches, have led to silos not only in working organizations but also in the minds of individuals. There is also some speculative musing, laced with hope, on the new technical advances in AI and their ability to break these fiefs and barriers of knowledge.

#natureofknowledge #knowledgecreation #AI #specializations

--

Why do we have specializations and super-specializations of knowledge? Why are there experts of specific niches, may it be in medicine, or in law, or in history, or in any field of knowledge for that matter? There are even experts on experts, doing meta-studies and bibliographic indexing of these niches. The answer, of course, is in the vastness of human knowledge and its compounding, additive nature.

From the time we, as a species, discovered a way to codify knowledge in the form of words and numbers, we also realized that this knowledge can be stacked and piled, with layers building onto other layers, a true tower of Babel. In our joy of this discovery, we went about collecting these trinkets and valuable pieces with great alacrity over the last ten thousand years of recorded history.

The obvious fallout of this zeal and collective intelligence was the sheer size of human knowledge built over time. The current (potentially erroneous) estimates of all the human knowledge in the world range between 295 #Exabytes to 300 EB (one EB is 1 billion GB). It is a tidy sum, a rather unwieldy one though, for any one individual to carry in their heads.

Specialization and Generalization

The result of this vast sum of knowledge is that we can no longer be generalists (although some act as the “know-it-all”s to the sheer annoyance of others). How does one make use of this vast sum of knowledge? Simple. We take little George Washington’s hatchet and slice away the cherry tree of knowledge into manageable chunks that people can study and become experts in. As the knowledge grows, so do these branches.

Ancient Egyptians regarded Math and Astronomy as one. Ancient Greeks thought of Philosophy and Science as one. Over time, this approach gave way to splitting Science from Philosophy. The domain of Science further got separated into Natural Sciences and Social Sciences. From the field of Social Sciences emerged Management Studies, which then branched into areas of functional expertise, e.g., Marketing, Operations, Finance, etc., and areas of process expertise, i.e., International Trade and Business, People Management, Quality, Process Control, and so on. The area of managing processes led to waves of theories, like the Systems Theory, and further sub-specializations within these theories.

This #Descartian approach became hugely successful over human history. It allowed people to focus on specific parts of the whole, observe them, build hypotheses around them, and then test them. It led to vast improvements in our understanding of and our influence on the world around us through science and technology. It also created what is called ‘#scientific #temper’ among individuals, a way to study and affect changes where required.

So, where is the problem?

The problem, other than the dogma of “#scientism”, an excessive belief in the power of scientific method and techniques, is the over-reliance on breaking things down. In this pursuit of specialization and sub-specialization, we seem to be too caught in these minutiae and increasingly missing the bigger picture. The approach of specialization, that of breaking the knowledge down into manageable chunks, leaves little room for generalization.

I was speaking with a group of students of Strategy this weekend about how to understand a business process within an organization, and I mentioned the tool called #SIPOC, i.e., Supplier, Input, Process, Output, and Customer, to have a high-level understanding of the process. The moment I said this, one student quipped, as expected, “but isn’t that one from Quality Management?”. It was as though SIPOC belonged to the discussions alongside #sixsigma and #TQM and #Quality circles and so on, and the students felt almost heretical to generalize the concept of SIPOC and use it in the field of Business Strategy.

The problem with our approach of specialization is that it is usually in the generalizations that the human knowledge grows in its character, not its size, and leads to wisdom. It is one thing to break things down to study them in isolation. It is another thing altogether to take a step back, look at the overall status of things, and then come up with what to do next.

Newton broke down the wheel of time and space into parts that could be studied in isolation. This point of view led to immense development in science and technology for humans, until Einstein came along and looked at the space-time continuum “as a whole”, which led to the next level of knowledge and accelerated our advent in the last century.

Not all this needs to be groundbreaking or revolutionary. As it is to the overall field of human knowledge, so it is to specific areas too. Control charts used by the physicists and the army in the wars were generalized and brought into process management by Walter #Shewhart, which led to an entire theory on statistical quality control. Lynn #Shostack took the concepts of design, i.e., complexity and divergence, from manufacturing and generalized them to apply to services operations, which led to a fascinating way of #design #thinking of services.

These are not exceptions; rather they are the norm. It is in the generalization of knowledge that we find breakthroughs and advances. Every time we think that we have reached the frontiers of knowledge in any field, someone takes this step back, and brings in a fresh point of view of synthesizing the knowledge.

Analysis and Synthesis

Thus, the two processes of human knowledge, “#analysis” – which is breaking things down, and “#synthesis” – which is combining and building things up, need to coexist and co-operate. Only then can we hope to advance. However, our overall approach of specializing knowledge, due to the nature of knowledge as it is, leads to a rather disproportionate reliance on analysis and a relatively less focus on synthesis.

#Specialization leads to compartmentalization of knowledge. Analysis leads to further breaking down of the problem area. While all this is good, there needs to be a process to “bring it all back”, which we sometimes miss. This leads to certain theories and concepts being treated as exclusive rights of certain specializations, a “knowledge fief”.

In most of my advisory projects for companies, the value comes mainly from synthesis, from breaking down these knowledge fiefs. Some of these fiefs are created due to the organizational structures. One function holds the data of hiring employees, another holds the data of training these employees, while a third has the information about their performance in their respective jobs. Unless there is a synthesis of this information, we are not going to be able to see the patterns and know what really is going on. Unless there is a synthesis of data over time, we continue to look at the latest instance of failure and not see the overall pattern.

In a recent celebrated #HBR article, there is a mention of characteristics of successful CEOs. Prominent among these was the ability to hold, at the same time in their minds, two levels of thought. These two could be the operational minutia and the strategic intent, or short-term and long-term, or individual market level and overall regional level. Only by holding both levels of thought together were they able to manage successfully, maneuverer difficult times, and affect the change they wanted to make.

The hardest part to teach my students is the ability to see the patterns. I first thought that this is due to nature of what I teach and the field that we deal in, but I soon realized that there is a more fundamental gap. This seems to happen because they are never taught how to take a step back and see the patterns in data or concepts or ideas. Due to the way knowledge is structured now, our educational practices as well as management practices have a disproportionate focus on analysis and hardly any, if at all, on synthesis. Even when they are taught how to synthesize, they are not taught the importance of it in the overall process of knowledge.

Juxtaposition as a method of ideation

How to beat this habit of micro-analysis that is so ingrained in us? To jump-start the process of synthesis, one potential method is to juxtapose different domains and fields of knowledge. I have always found that when two unrelated fields of knowledge are brought together, something interesting, almost magical, comes out of it.

August Kekule saw three snakes in his dream arranged in a circle, with one’s tail in another’s mouth, and so on. He used that to draw the molecular structure of benzene, which then led to a whole new domain of organic chemistry. Several accidental discoveries in science and technology follow the tenet of this story, which is bringing together different fields of thought and finding insights out of this process. ??

Part of the need to bring two domains together is also due to the over-saturation of ideas within single domains. When I was working on my PhD thesis defence almost a decade ago, my then thesis advisor suggested to look at two independent fields of knowledge, because "new ideas are always found in the gaps between these fields of knowledge". True enough; I took the domain of Decision Making #decisionmaking, mapped it to the Theory of Constraints #TOC, and for good measure, added the field of Service Operations #serviceoperations. At the centre of this 3-circle Venn diagram of Domain and Theory and Field, I could safely lodge my work and find new enriching insights.

As it is to big breakthroughs, so it is for smaller ones. I have already written two issues in this newsletter that demonstrate this method. In one, I have tried to juxtapose the concept of a balanced diet with the habit of reading. In another, I have compared the ways to value a diamond from gemmology to the ways to value a business idea. Both thought experiments work, at least to an extent.

Do this activity for yourself. In the next four days, think about two completely different domains or fields of human knowledge. They could be as diverse as sports and quality control. Remember to include one of these from your area of expertise, what you know well, part of your specialization. Now pick one idea or concept from the other domain, the distant one, and see how the concept, at a very broad level, can fit into your domain.

The specifics do not matter. Does the overall framework fit? What interesting parallels can you bring out? If it does not fit, where does it deviate? Does that tell you something interesting too? Do you see some patterns? Do you see some connections? You will find that there is always something interesting that comes up with these kinds of thought experiments. And once you get a hang of it, use this the next time you have a creative block.

AI and the new frontier of knowledge creation

And this is where the recent advents in artificial intelligence (AI) are exciting. With our limited expertise and poor processing power, and with the clutches of specialization built around us, we humans have only a limited capacity to generalize things, see the big picture, look at patterns, see the synaptic connections, and observe the forest for the trees. We can barely hold one train of thought in our mind at a time, (and sometimes even that is difficult), what with the constant interruptions and the bombardment of new information updates by the minute.

In contrast, the AI-powered decision systems can hold a much larger data set in context. The recent news coming from that direction is extremely thrilling. Just two weeks ago, #Microsoft announced that their newly released #LongNet can hold a #billion #tokens (units of text fragments) of context length!

Just so that you appreciate how massive this leap is, consider this: ChatGPT had a context length restriction of 4k tokens (about 8,000 words). With GPT-4, this restriction went up to 32k tokens. So, of course, “a billion tokens” is huge. How they managed to do this is also interesting. The tech apparently uses what is called “#dilated #attention”, which creates sparse representations of larger sequences of data quickly. This is the same process in our brain that allows us sometimes to read words and paragraphs without reading every single letter sequentially. But I am digressing.

A million tokens are equivalent to about 1,500 pages of memory in the current language models, which is equivalent to about 20 books. So, a billion tokens would mean about 20,000 books. An entire #human #life is estimated (again, perhaps erroneously) to be around 860 million tokens. A billion tokens would mean that the AI model can hold an entire human life in context, and some more.

Now, if these 20,000 books belong to different domains and different fields, imagine the connections one can draw between them. All one needs to do is to let these models loose among groups of domains of knowledge, and we will start getting waves of new ideas. And it is in this that we hope to move to the next level of our knowledge. By focusing in the right direction. Not in knowing how computers will start behaving like humans, but how they can further the synthesis of knowledge for us.

I am hoping that at least some AI models and companies that build these models are working in this direction. If they are, it is going to be fascinating to witness the dawn of the next frontier of human knowledge.

Regards,

Dr. Shreekant Vijaykar

09 August 2023