Dancing on the Shoulders of Giants

This article originally appeared on Aneesh's blog.

In Newton's era it was rare to say things like "if I have seen further, it is by standing on the shoulders of giants" and actually mean it. Now it's trivial, with education, training, and experience professionals always stand "on shoulders of giants" (OSOG). Experts readily solve complex problems but the truly difficult ones aren't solved through training. Instead, a combination of muddling through and the dancer style of curiosity is deployed, more on this later. We have industries like semiconductors, solar, and gene sequencing with such high learning rates that the whole field seems to ascend OSOG levels daily.

These fast moving industries follow Wright's Law. Most industries don't follow Wright's law due to friction against discovering and distributing efficiencies. In healthcare regulatory barriers, high upfront research costs, and resistance to change keeps learning rates low. Of course, individuals have expert level proficiencies, many with private hacks to make life easier. Unfortunately, the broader field does not benefit from individual gains and progress is made only when knowledge trickles down to the level of education, training, and regulation.

This makes me rather unhappy, and I wonder if even highly recalcitrant fields like healthcare could be nudged into the Wright's law regime.

No surprise that I view AI being central, but it's a specific cocktail of intelligence that has my attention. Even before silicon, scaling computation has advanced intelligence. However, we will soon run into limits of scaling compute and the next stage of intelligence will need a mixed (or massed, as proposed by Venkatesh Rao). Expertise + AI Agents + Knowledge Graphs will be the composite material that will enable us not just to see further, but to bus entire domains across what I think of as the Giant's Causeway of Intelligence. Lets explore the properties of this composite material a little deeper, starting with expertise and it's effects.

An individual's motivation and drive are touted as being the reason behind high levels of expertise and achievement. At best, motivation is an emergent phenomenon, a layer that people add to understand their own behavior and subjective experience (ref, ref). Meanwhile, curiosity is a fundamental force. Building knowledge networks, compressing them, and then applying them in flexible ways is a core drive. Everyday, all of us (not just the "motivated") cluster similar concepts under an identity then use that identity in highly composable ways ref.

There are a few architectural styles of curiosity that are deployed. 'Architecture' is the network structure of concepts and connections uncovered during exploration. STEM fields have a "hunter" style of curiosity, tight clusters and goal directed. While great for answers, the hunter style has difficulty making novel connections. Echoing Feyerabend's 'anything goes' philosophy, novel connections require what is formally termed as high forward flow. An exploration mode where there is significant distance between previous thoughts and new thoughts ref. Experts don't make random wild connections when at the edge of their field but control risk by picking between options likely to succeed, what has been termed as 'muddling through'.

Stepping back, if you consider that even experts are muddling at the edges then the only difference between low and high expertise is their knowledge network. The book Accelerated Expertise, summarized here, explores methods of rapidly extracting and transmitting expertise in the context of the US military. Through the process of Cognitive Task Analysis expertise can be extracted and used in simulations to induce the same knowledge networks in the minds of trainees. From this exercise we can take away that expertise can be accelerated by giving people with base training access to new networks of knowledge.

Another way to build a great knowledge network is through process repetition, you know... experience. These experience/learning curves predict success in industries that follow Wright's Law. Wright's Law is the observation that every time output doubles the cost of production falls by a certain percentage. This rate of cost reduction is termed as the learning rate. As a reference point, solar energy drops in price by 20% every time the installed solar capacity doubles. While most industries benefit from things like economies of scale they can't compete with these steady efficiency gains. Wright's Law isn't flipped on through some single lever but emerges through the culture right from the factory floor all the way up to strategy.

There are shared cultural phenomena that underlie the experience curve effect:

• Labor efficiency - where workers are more confident, learn shortcuts and design improvements.
• Methods improvement, specialization, and standardization - through repeated use the tools and protocols of work improve.
• Technology-driven learning - better ways of accessing information and automated production increases rates of production.
• Better use of equipment - machinery is used as full capacity as experience grows
• Network effects - a shared culture of work allows people to work across companies with little training
• Shared experience effects - two or more products following a similar design philosophy means little retraining is needed.

Each of these is essentially a creation, compression, and application of knowledge networks. In fields like healthcare efficiency gain is difficult because skill and knowledge diffusion is slow.

Maybe, there could be an app for that...

Knowledge graphs (KGs) are databases but instead of a table they create a graph, capturing relationships between entities where both the entities and the relationship have metadata. Much like the mental knowledge networks built during curious exploration, knowledge graphs don't just capture information like Keanu → Matrix but more like Keanu ?star of→ Matrix. And all three, Keanu, star of, and Matrix have associated properties. In a way KGs are crystalized expertise and have congruent advantages. They don't hallucinate, are easy to audit, fix, and update. Data in KGs can be linked to real world evidence enabling them to serve as a source of truth and even causality, a critical feature for medicine (ref).

Medicine pulls from a wide array of domains to manage diseases. It's impossible for all the information to be present in one mind, but knowledge graphs can visualise relationships across domains and help uncover novel solutions. Recently projects like PrimeKG have combined several knowledge graphs to integrate multimodal clinical knowledge. KGs have already shown great promise in fields like drug discovery and leading hospitals, like Mayo Clinic, think that they are the path to the future. The one drawback is poor interactivity.

LLMs meanwhile are easy to interact with and have wonderful expressivity. Due to their generative structure LLMs have zero explainability and completely lack credibility. LLMs are a powerful, their shortcomings make them risky in applications like disease diagnosis. The right research paper and textbooks trump generativity. Further, the way that AI is built today can't fix these problems. Methods like fine-tuning and retraining exists, but they require massive compute which is difficult to access and quality isn't guaranteed. The current ways of building AI, throwing in mountains of data into hot cauldrons of compute and stirred with the network of choice (mandatory xkcd), ignores the very accessible stores of expertise like KGs.

LLMs (and really LxMs) are the perfect complement to KGs. LLM can access and operate KGs in agentic ways making understanding network relationships easy through natural language. As a major benefit, retrieving an accurate answer from KGs is 50x cheaper than generating one. KGs make AI explainable "by structuring information, extracting features and relations, and performing reasoning" ref. With easy update and audit abilities KGs can easily disseminate know-how. When combined with a formal process like expertise extraction, KGs could serve as a powerful store of knowledge for institutions and even whole domains. We will no longer have to wait a generation to apply breakthroughs.

Experts+LxMs+KGs are the composite material to accelerate innovation and lower costs of building the next generation of intelligence. We have seen how experts are always trying to have a more complete knowledge network with high compression and flexibility allowing better composability. The combination of knowledge graphs and LLMs provide the medium to stimulate dancer like exploration of options. This framework will allow high-proficiency but not-yet-experts to cross the the barrier of experience with ease. Instead of climbing up a giant, one simply walks The Giant’s Causeway. Using a combination of modern tools and updated practices for expertise extraction we can accelerate proficiency even in domains which are resistant to Wright’s Law enabling rapid progress.

Check out the original blog post for a healthcare specific appendix section.

If you're working in at the intersection of healthcare, AI and knowledge graphs, I'd love to chat.