Insights into GenAI Today and Nvidia Tomorrow

In a remarkable hour, Dr. Amr Awadallah , Founder and CEO of Vectara , Founder of Cloudera , and Dr. 贾扬清 , Founder and CEO of Lepton AI and creator of #Caffe provided insights into the current state and future direction of #GenerativeAI. I aim to encapsulate the essence of our conversation below, but I encourage you to watch or listen to it as well: https://www.youtube.com/watch?v=ujly-4dG-yM

1. Nvidia's Momentum is a Double-Edged Sword

Dr. Amr Awadallah provided an intriguing analogy when discussing Nvidia's current market position, stating, "Nvidia is the canary of the Gold Mine," and further noting, "There is a massive signal of massive market coming." He highlighted the irony of Nvidia's success and suggested that the very dominance that has propelled Nvidia to its current heights could also herald its challenges ahead.

Dr. Amr Awadallah, however, made a bold prediction that "Nvidia will get Nvidia'ed."

Awadallah referred to how Nvidia once disrupted SGI in the 1990s by offering a $300 consumer-grade card against SGI's $30,000 offerings. Jia and Awadallah identified companies like Grok, Cerebras, and SambaNova as potential disruptors in the current landscape.

2. Decoding the Complexity of GenAI: From Development to Deployment

Dr. Awadallah elucidated the core challenges Vectara is tackling with GenAI: "hallucination, attribution, and security." It resonates strongly with me, now a GenAI practitioner.

Dr. Yangqing Jia indicated Lepton focuses on "how to reliably run those models at scale, with high performance, and in a cost-effective manner," highlighting innovative strategies such as "batching the request at the same time, speculative decoding, packing multiple CUDA cores into a single CUDA core to enhance efficiency." Jia proudly noted, "We can run some models 10x faster than the vanilla models" and "three to five times faster than the best open source solutions."

Furthermore, Dr. Yangqing Jia remarked on the goal of GenAI applications, saying, "Most GenAI applications are for productivity improvement, not generating new revenue," and used the analogy of different use cases for Lamborghini versus Camry, underscoring the importance of cost efficiency in many scenarios.

Dr. Awadallah also mentioned customers' concerns over lock-in and their desire for autonomy in running their models, prompted by trends in deglobalization.

3. Revolutionizing Open Source: From "Open Core" to "Dual Core" and "Open Perimeter"

The discussion took a critical turn towards the viability of open-source business models in the AI space. Dr. Awadallah learned a hard lesson from his Cloudera experiences where he used to believe in the open-core model but learned there are two main challenges:

1) Some hyper scale cloud companies are "vampires" 
2) Your customers become your competitors        

He advocated for an "Open Perimeter" paradigm where companies may selectively open-source certain components, similar to practices by companies like Datadog. Vectara has open-sourced a popular hallucination benchmark model under this new paradigm.

Dr. Jia believes in the evolution of open core towards what he describes as an "open standard", featuring one open-source base that's "good enough"—yet still better than the industry alternative—and another core that's significantly better. Both "cores" adhere to the same standard and interface, emphasizing the benefits of no lock-in and providing peace of mind to customers. Awadallah enthusiastically embraced Jia's "Dual Core" concept.

As an enterprise software veteran myself, I firmly believe both "Dual Core" and "Open Perimeter" are the future of Open Source Business Models.

4. Perspectives on AGI and jobs for software programmers

We concluded the panel by briefly touching on the intriguing topic of the AGI timeline. Dr. Awadallah posited that AGI could arrive by 2029, a viewpoint not shared by Dr. Jia, who expressed skepticism towards the feasibility of AGI. Nonetheless, both acknowledged the significance of 英伟达 CEO Jensen Huang 's philosophy on "learn how to learn," for younger generations.