Blog-post on generative AI

I had the pleasure to attend the The Montgomery Summit in Los Angeles last week. Thank you Jamie Montgomery Sumant Mandal for inviting me and letting me participate in all these exciting private sessions. If I had to summarize my 3 main takeaways from the Summit, it would be AI, AI and again AI. The age of AI is here!

As Charles Lamanna from Microsoft put it so well during his closing keynote address, we are quickly transitioning from a world where "Software is eating the world" to one of "AI is eating software". Most advanced AI models are being trained into next generation computing platforms, and when one looks at previous platform transitions over the last 4 decades from Mainframes to Desktop PCs to Internet and finally to Mobile, it is striking to note that the coming AI transition could still be the largest in its socio-economic impact.

As the Enterprise IT Tech stack gets re-wired over the next few years, the first large impact should be on the Applications layer. We currently have stand-alone enterprise applications across CRM, ERP and HCM verticals each with their own databases acting as Systems of record. The AI transition should require next-Gen applications to become Systems of intelligence built around a new Tech stack with: (1) A foundation AI models layer (OpenAI, BERT, ResNet), (2) A data Cloud layer (Snowflake, Databricks) and (3) finally a 3rd party developer/vertical applications layer sitting on top (ASAPP, Uniphore etc) via APIs. The foundation models layer would include a repository of all pre-trained transformer AI models, while the applications/developer platform layer would include specific vertical AI with supervised fine-tuning deliberately automating enterprise workflows. The analogy is the Apple AppStore, and as such, OpenAI for example would monetize its underlying AI engines IP via recurring royalties from 3rd party developers. Initially developed by Google, a transformer is a deep-learning (DL) model-based neural network which also forms the basis of the GPT-3 large language model (LLM) trained to undertake AI conversational services. Contrary to traditional machine learning (ML) which involves pattern recognition, the new DL based generative AI creates completely new content out of a trained model.

AWS, Azure and GCP collectively form the AI supercomputing infrastructure in the Cloud, and this boat has already sailed in our view with these 3 platforms dividing the attractive mid-term AI infrastructure opportunity among themselves in the West. The other large opportunity lies in the picks & shovel Silicon layer. GPU architecture is best suited to perform AI training functions, and 80% of AI training today is based on the Nvidia software/hardware platform. For example, OpenAI's chatGPT conversation service is based on 175bn parameters, which needs 10k nodes of GPUs with a node as a basic building block consisting of a cluster of 8 single GPUs. New LLM with an even higher number of parameters (such as GPT4) will then subsequently need even more connected GPUs for pre-training.?While this is great news for Nvidia and potentially AMD, the costs of such training infrastructure will continue to scale exponentially from currently around $ 200m and could become prohibitively expensive to run.

The incremental opportunity set within Silicon thus lies in new semiconductor design start-ups which will allow the cost of training smaller, niche AI services through lower cost, open source solutions (such as Risk V-based processors). At the Montgomery Summit, I was asked to moderate a fire-side chat with one such company called Tenstorrent and its CEO Jim Keller . Stay tuned for more innovation on the silicon front as Nvidia keeps pushing ahead as the incumbent.