AI Field Day 4 kicks off with VMware's Private AI

This week I'm at AI Field Day 4, and I plan to give my quick thoughts of every company's presentation in real time. Here's my overview of the VMware paper.

Introduction to VMware Private AI

Chris Wolf, Global Head of AI & Advanced Services, kicked of the first presentation. The first slide was about app innovation, which was great to see. He mentioned that there isa $4T market for generative AI.

Customers are wanting to bring their AI applications to existing data, which seems to be a sweet spot for existing VMware customers who have already have clusters close to their data.

Wolf said that they can stand up a VMware instance with the customer's model loaded up in seconds.

Private AI from VMware relies on vSphere security from SecureBoot, virtual TPM, etc. You can build it DYI, but it runs on VCF with a Broadcom VCF.

VMware started using code generation with ESXI to test it out. They had a high amount of adoption from engineers, but are still working on results (such as documentation of

VMware has an AI council, which is good because Private AI only describes protection from data privacy, not ethical problems that cause unintended consequences.

VMware Private AI Foundation with NVIDIA Overview

@Justin Murray, Product Marketing Engineer at VMware, got into the details of the solution. The entire name of the solutions is VMware Private AI Foundation with NVIDIA solution.

Justin described what a VMware's self service catalogue in VMware Aria. VMware ships OVAs that have all the deep learning VMs that data scientists would want to use.

Justin gave a nice explanation to RAG (Retrieval Augmented Generation).

VMware has about 60 customers currently. The solution also provides GPU and other performance monitoring. It works with the NVIDIA Inference Server and Management Service. The architecture is a combo of Kubernetes containers and virtual machines.

VMware Private AI Foundation with NVIDIA Demo

The demo was a RAG application with a chat bot in front of it. Justin walked us through setting everything up, as well as monitoring it. Be sure to check out the videos when they are posted.

Running Best-of-Breed AI Services on a Common Platform with VMware Cloud Foundation

Shawn Kelly, Principal Engineer, led the next session about why you should use VCF for AI production. He described how Ray works on the VCF AI solution.

Next, Kelly described the VCF solution with IBM WatsonX. This helps companies run IBM Watson on premises.

This solution requires VCF.

Real-World Use of Private AI at VMware by Broadcom

Ramesh Radhakrishnan, Technologist, Office of the CTO gave some use cases that are being used at VMware. One use case search is for improved documentation search. There are so many vSphere versions, they have improved it by over 500% (5.7x better).

There is lots of engineering that goes into a question and answering service like the one VMware has built. They used the Stanford Col(v)BERT IR model. But the production model is fairly complex:

#AI #VMware #Broadcom #NVIDIA #RAG #VCF #RAY