Databricks Mosaic Research

Part 2 of our investigation into long-context #RAG is out! After the results we published in August, there was tremendous interest in how the latest releases from OpenAI and Google perform against our benchmarks. We evaluate: * OpenAI o1-preview and?o1-mini * Google Gemini 1.5 Pro,?Gemini 1.5 Flash?(May release) After running these additional experiments, we found that: ? * OpenAI o1 models show a consistent improvement over Anthropic and Google models on our long context RAG Benchmark up to 128k tokens. * Despite lower performance than the SOTA OpenAI and Anthropic models, Google Gemini 1.5 models have consistent RAG performance at extreme context lengths of up to 2 million tokens. * Models fail on long context RAG in highly distinct ways Full details in our blog post: https://lnkd.in/gKj--uq3

Join us in Mountain View for the next event in the Compound #AI Systems meetup series! Date/time: October 22, 2024, 6:30-9:00 PM Pacific Location: Databricks Mountain View office (see RSVP link for details) RSVP here: https://lu.ma/hfqn3lj3 We bring together people from research and industry to highlight Composable Data Systems, AI/ML integrations, and Open Source project you can’t miss. Speakers present tools, tips and tricks, and discussions will put a spotlight on challenges and opportunities in the field today. Co-sponsored by LanceDB and Databricks.

Our research and engineering teams did great work making the latest Llama models from?AI at Meta?available to our?Databricks?Mosaic AI customers. These new, smaller models are excellent options for latency-sensitive use cases. Fine-tuning Llama 3.2 on your data in Databricks is just one simple command away; learn more in this blog post by Daniel King, Hanlin Tang and Patrick Wendell.

If you're in San Francisco Bay Area, join us at the inaugural Compound AI Systems meetup at the Databricks SF office tomorrow evening, Tuesday, Sep 24, 5:30-8:30 PM! RSVP: https://lu.ma/roev9rc6 #AI engineers, researchers and practitioners are coming together to discuss the practical realities of composing these systems and services, with a focus on modularity and observability. Co-sponsored by LanceDB and Databricks. Speakers: Chang She, CEO/Cofounder LanceDB Erik Bernhardsson, Founder/CEO Modal Kobie Crawford, Developer Advocate, Databricks RSVP here: https://lu.ma/roev9rc6

?? ?? Mosaic AI Model Training?now supports the full context length of 131K tokens when fine-tuning Meta Llama 3.1 models. Databricks customers can build even higher-quality #RAG systems by using long context length enterprise data to create specialized LLMs: https://lnkd.in/gdum_XJS

?? At Databricks, we want to help customers build more #inference-friendly #llms. Our MixAttention architecture demonstrates that reductions in KV cache size make it possible to maintain model quality while improving inference speed and reducing memory footprint. Read our latest blog post to learn more: https://lnkd.in/gcHxM4DX

Great work by our Mosaic AI researchers + engineers to develop + ship a new judge for our #LLM agent eval tool, used to improve the quality of agentic #GenAI applications. Kudos to Max Marion, Arnav Singvi, Samraj M., Avesh Singh, Michael Carbin and Alkis Polyzotis!

New blog post from one of our Mosaic AI customers: learn how Twelve Labs enables developers to create multimodal embeddings for advanced video understanding use cases by using Databricks Vector Search to index and query high-dimensional vectors.? https://lnkd.in/guZEig9A

#DSPy is an excellent example of the approach that is needed to automate the improvement of modular applications that have generative AI at their core. Great to see the community connecting and discussing these next steps toward reliable, useful, sophisticated AI functionality. Kudos to Stanford University NLP researchers and our colleagues at Databricks!

We are looking forward to hosting this Thursday's South Bay GenAI meetup at our Mountain View office! Omar K. and Arnav Singhvi of Databricks, along with Stanford University colleagues Krista Opsahl-Ong and Dilara Soylu, will be presenting some of the latest innovations in #DSPy. This Python package helps you optimize your #LLM prompts programmatically, borrowing the patterns used in training and hyperparameter tuning to improve application performance without manual intervention. If you're an aficionado of GenAI in the South Bay, this is a session you don't want to miss! RSVP here: https://lnkd.in/gmTNJKwy