A special invitation ??: Evaluating LLMs for Your Applications talk with Google GenAI product leader

Having worked in the ML/AI field for over 20 years, including at leading tech companies like Google GenAI, Meta, Microsoft and AWS AI teams, MAHESH YADAV 've witnessed firsthand the transformative power of Large Language Models (LLMs) in product development. With the proliferation of models like Gemini 1.5 Pro, Llama 3, GPT-4-Turbo, and over 100 others, selecting the right LLM for your GenAI application is a critical decision that can make or break your project's success.

Selecting the right cloud provider can be challenging, and the same is true for choosing a large language model (LLM). Prompt techniques vary from one model to another, so if you switch models, you’ll need to re-run all your tests from scratch. Therefore, it's beneficial to invest more effort upfront in choosing the right model to avoid accumulating technical debt in the future.

Mahesh has made this choice across open source (pie vs Llama 3) and Gemini or GPT4/ Claude 3 many times while advising startups. Mahesh will share his expertise via a comprehensive talk on choosing and evaluating LLMs on Maven. This session is designed to equip product managers and AI builders with the knowledge and tools needed to select LLM or SLM (small language models) based on their needs.

In this talk, Mahesh'll cover three essential areas:

1. A framework for model selection, taking into account crucial factors such as budget constraints, latency requirements, privacy and team capabilities.
2. Strategies for establishing clear, actionable evaluation criteria, leveraging industry benchmarks to save model evaluation cost.
3. A practical walkthrough using a contract processing application, demonstrating how to align model selection with specific business requirements and performance benchmarks.

By the end of the session, attendees will walk away with a structured approach to selecting and evaluating GenAI models, enabling them to make informed decisions. This talk will be especially useful for product leaders who need to gain insights into developing effective LLM evaluation strategies, and AI builders who need the right tools and practical insights to apply these principles directly to their projects.

?? RSVP here

Mahesh’s Bio

MAHESH YADAV is a Product Leader at Google GenAI team. Mahesh is one of the world's top AI executives and an award-winning AI Product Educator. His work on AI has been featured in the Nvidia GTC conference, Microsoft Build, and Meta blogs.

Mahesh has 20 years of experience in building products at Meta, Microsoft and AWS AI teams. Mahesh has worked in all layers of the AI stack from AI chips to LLM and has a deep understanding of how GenAI companies ship value to customers.

Currently, he leads an AI agent for Google cloud support team. He uses Gemini latest models from deepmind and multi agent framework with knowledge graph to automate support agent function for Google cloud customers.

?? RSVP here