Runhouse

Had a great time at Ignite Talks last night presenting about the value of first-party #data to make #AI work and how #LLM fine-tuning is now easy if your infrastructure is in a good place. Thanks to the great audience and speakers alongside us, and special thanks to Brady F. for having us in the beautiful Betaworks Ventures space.

??AI/ML MAKERS UNPLUGGED (WITH GOOGLE CLOUD NYC) – That Was EPIC! ?? Packed house. Game-changing AI talks. Brilliant minds. THIS is what makes the MLOps community special! ?? From fine-tuning models beyond GPT-4 to vector search and AI-driven finance, our speakers dropped next-level insights and hands-on demos that sparked incredible discussions.?? Huge thanks to our phenomenal speakers: ???Bon S. - Unlocking Agentic Era with Gemini ???Donny Greenberg – Beat GPT-4: Mobilizing First-Party Data with Fine-Tuning ???Chuxin Liu, PhD – AI Agents in Data Analysis ???Casey Clements – Vector Search & Langchain ???Arkin Gupta – The Landscape of Risk Models in Equities ???Charles Antoine Malenfant –Agents in Investment Management: Aladdin Copilot?? Massive appreciation to Google Cloud- Kathryn Petrini and Kira Frank ?? MLOps NYC - Adam Boaz Becker (Habibi), Charles Antoine Malenfant The GenAI Collective - Prithvi R. Patrick Ward See you at the next one! ???Check out the event highlights below! #MLOps #MachineLearning #AI #DataScience #Networking #GenerativeAI #LLMs #NYCTech #TechCommunity

Had a great time presenting about the importance of training and tuning #LLMs and #embedding models for #genai use cases at MLOps Community / The GenAI Collective NYC meetup. Mobilizing first party data into continuous learning loops has always been part of the magic of ML and it’s also true for AI agents and systems. Great talks by Chuxin Liu, PhD, Arkin Gupta, Casey Clements, and Charles Antoine Malenfant as well and thanks Google for hosting us!

2025 will be the year of AI training. After all, who even convinced us that LLMs are magic AI systems that do not benefit from training? Whether it’s recommender systems at TikTok or Meta, or a fraud model at a bank, value-driving ML has always relied on improvement loops based on proprietary data. Waymo could never succeed using off-the-shelf vision models - its moat is the millions of real-world driving it has used to feed its proprietary vision models. Language AI is no different. Today, teams have invested time in launching production-ready MVPs with prompt tuning or iterating on RAG but have hesitated to start training models to fit their use cases. However, this leaves two extremely valuable datasets unused. First, your company's domain-relevant proprietary information has never been accessible to pre-trained AI models. Second, production usage means actual success and failure traces from users that represent a direct target for improving your system. Meanwhile, the open-source ecosystem has caught up, making training code simpler than ever. So we think 2025 is the year of training. Read more: https://lnkd.in/eUGCct6d #LLM #GenAI #Experimentation #ML #AI

Thanks for the shoutout! Among great company in a great neighborhood

Thanks for having us FinTech Innovation Lab, Accenture, and Partnership for New York City! In case you missed us at the event, shoot us a note and we'll show you how Runhouse can create self-improving AI systems with your first party data.

Thanks Work-Bench and Jacob Portes for co-hosting AI bagels with us yesterday and everyone who came by for great conversations about #AI, #ML, and #LLMOps!

A blog post collab with our friends at marimo! Run every DS/ML pipeline perfectly reproducibly without sacrificing the ability to iterate quickly or see results interactively. In this blog post, Paul Yang worked with Akshay Agrawal to show how to fine-tune a Llama LLM by defining the code in the marimo notebook and then using Runhouse to launch ephemeral GPU-accelerated compute from your own cloud account to execute. Code is updated locally, but can be iteratively dispatched to run on that remote GPU in <2s. As you move to production, check your marimo notebooks into version control, since these notebooks are simple `.py` files. In the future, when you bring in new data or want to reproduce this checkpoint, simply run the notebook as-is and it will identically use Runhouse to launch compute and execute. https://lnkd.in/ewma6MVw #ML #Python #jupyter #LLM #AI #datascience

Launch #DeepSeek R1 on elastic compute in your own cloud infra so you can provide proprietary data and questions to have DeepSeek reason over it securely. In our example, it's as low as $1 / hour to run experiments on AWS with spot instances and the Qwen 32B distillation (though, beware, minimum cost machines provide awful $ / token at scale). With Runhouse, you can easily swap out the model_id, number of nodes, number of GPUs, and GPU type to try the various models or scale up inference to many nodes. We chose to launch the largest distillations (unquantized) since, for a tolerable loss in quality, both the 70B and 32B distills fit neatly on a single node of readily available L4s. if you have quota for H100s, you can change three lines and launch the full model. Blog Post: https://lnkd.in/eGybnEsf Github (Distill Llama 70B): https://lnkd.in/eikkv3-e And ping us if you hit any snags, we'd love your feedback! #GenAI #LLM #ML #vLLM

Fine-tuning isn't magic, but you can easily get a high quality, small model that solves your use case with much lower latency and cost. Runhouse makes it easy to run this training loop by taking your regular fine-tuning and distributing it "magically" to arbitrary scale multi-node compute.