DVC Community December Updates

???? Hi friend! Before we head into the end of the year and all the festivities that brings, I give you all that's new and cool in the DVC Community. ???

?The Latest

?? New Videos

• Data Versioning in Generative AI: A Pathway to Cost Effective ML - Dmitry Petrov 's presentation at MLOps World and Generative AI World Summit is now available. This video addresses the special challenges of GenAI’s unstructured data and how that problem can be addressed through metadata.
• How to Turn Your IDE into a Machine Learning Experimentation Platform with the DVC Extension for VS Code - Tapa Dipti Sitaula shares a new tutorial on getting started with the extension!
• DVC Extension for VS Code Product Update: How to Filter Experiments by Git tags - Matt Seddon shows off this new feature in the extension and there’s a surprise at the end with the back story in the description. Curious? ??
• Applying GitOps Principles at Every Step of an E2E MLOps Project - A workshop - The recording of Tibor Mach ’s workshop at MLOps World is now available!
• Great Practices for Retrieval Augmented Generation (RAG) in Production - Noé Achache presents at the GenAI Days organized by Aleios in which he uses DVC for iterating on prompts, chunks, models, and more.

??? New Blog post

• Turn Your IDE into a Machine Learning Experimentation Platform with the DVC Extension for VS Code - Tapa Dipti Sitaula shows off the extension and all the improvements we’ve made to it in the last year?since we released it.

?? What we're looking at

The Busy Person’s Intro to LLMs - Andrej Karpathy recently presented this talk at a conference that wasn’t recorded so by popular demand he recorded it for the world. He discusses what they are, and where they are going with comparisons and analogies to help your understanding. Importantly he also addresses security issues you should be aware of.

Inside Ghostbuster: Berkeley University’s New Method for Detecting AI-generated Content - Jesus Rodriguez writes on Berkeley’s new model agnostic method to help us be able to tell the difference between human and AI-generated text content. You can find a link to the paper here as well .

Open source AI Definition - Open source AI initiative updates - v. 0.0.3 is available for review and comment and the next meeting is in San Jose on December 12th at the Linux Foundation AI Day conference .

The Copyright Case Against AI Art Generators Just Got Stronger with More Artists and Evidence - Carl Franzen ’s article following the milestones of this case and the latest development that more plaintiffs have joined and the court is acknowledging artwork as copyrighted if it has the artist’s mark on it even if not registered. And of course the Open AI/Sam Altman saga and ALL the opinions and theories surrounding it all, but I know you already know that.? I can't wait for the movie to come out because we are still missing the real reason the board fired him in the first place! ??This was my favorite of the clean (??) memes and Tweets:

?? Meme of the Month

??What's Coming!

Generative AI Data Chain at Scale - Join Tibor Mach and me on Wednesday, December 6th, for a webinar where he will provide a sneak peek into the new tool we are building for Generative AI unstructured data management. Bring your questions and use cases!

Zilliz Advent of Code - Zilliz created an Advent of Code program with 23 other open-source projects including DVC! We are on Day 10 this coming weekend. Come join the fun and hone your coding skills!

?? Community-Generated Content

Complete End-to-End Deep Learning Project with MLFlow, DVC and Deployment - Another awesome, in-depth four-hour video tutorial from Krish Naik on DVC!

Amazing Langchain Series with End To End Projects - Pre-requisites to Start With - Krish Naik also is starting an entire LangChain series starting with the pre-requisite video in which he recommends learning DVC in preparation for the LangChain series.

How Veriff decreased deployment time by 80% using Amazon SageMaker multi-model endpoints | AWS ? - Ricard Borràs Navarra , Jo?o Moura , Miguel Ferreira found this solution to decrease deployment time in projects requiring GPU instances.? Part of the workflow includes DVC which was used to sync large binary files (model weights) with Git in a versioned manner.

MLOps with Yolov8: Object Detection - Nicolò Campagnoli presents this university project with YOLOv8, Weights and Biases, Docker PyTorch, FatAPI, GCP, and of course DVC for data version control.

YOLOv8 DVCLive integration - Speaking of YOLOv8, Abirami Vina just released a very nice guide to the Ultralytics docs?for DVCLive's integration with the popular computer vision model.

Building an MLOps Pipeline in 3 weeks - a paint-by-numbers approach - Patrice Matz of Netlight writes about an AWS-based MLOps pipeline solution, proposing a dual lifecycle approach for data and model changes. He includes?the use of GitLab CI as an orchestration server, uses?DVC for data version without the need for further infrastructure as it's Git-based.

Empower Your Machine Learning Projects with Data Version Control - Amit Kulkarni describes DVC as a crucial tool for managing data and machine learning experiments. This tutorial walks you through the setup to the experimentation process, highlighting DVC's contribution to making work more efficient, less error-prone, and enhancing ML outcomes.

"Scaling Machine Learning with Spark" Sophia Yang, Ph.D. 's interview with Author Adi Polak - They go through the topics of the book including, distributed machine learning systems, Spark/PySpark basics, MLflow, PyTorch, TensorFlow. It also delves into topics such as distributed computing models, ensemble methods, and more.? DVC is mentioned in the book for data version control best practices.?

MLOps: Benefits, Applications, and Popular Platforms - Open Source for You - Aptly titled, Dr. Kumar Guarav shares some great used cases in the MLOps space to orient readers on the possibilities across industries and tooling.?

Building a Data Architecture for Generative AI Using Open Source Software - Duy Nguyen writes a short piece packed with some good open-source tool suggestions to tackle your Generative AI data infrastructure.?DVC is mentioned to?"elegantly track[s]?data changes, making data versioning and experimentation transparent and simple."

Despite the year's definite challenges, I think we can be grateful for a year of a lot of learning! ?? Our team is also grateful to all of you who are contributing and building amazing solutions across a spectrum of industries and use cases with our tools.? We wish you all a joyous and safe holiday season.? Let's rest up and get ready for what 2024 has in store!?

???? See you in January!?? ?

To your continued success,

Jeny De Figueiredo

Community Manager

DVC.org