AI Stories

From GPT2 to Llama 3.1, from text-based to multimodal LLMs. I released last week a technical conversation around LLMs with Sebastian Raschka, PhD on the AI Stories Podcast. Want to check it out? It's available on your favourite platforms ??: Youtube: https://lnkd.in/e4h9vWGS Spotify: https://lnkd.in/exCbPMis Apple Podcast: https://lnkd.in/eXpDrdxg Ps: We also dive into his new book: “Build a LLM from Scratch” that I cannot recommend enough! Have you read it? Keen to get your thoughts in the comments!

Comments like this make my days! Thanks for describing so well what I'm trying to do with AI Stories: ?? A mix of technical and personal / career questions ?? When I ask technical questions, I first try to give enough context to allow less technical listeners or people who aren't familiar with the topics to still understand what we discuss and follow along. ?? And if you're more technical, a short reminder of the basics never hurts before diving deeper :) Always open to feedback! Have you listened to the show? Let me know what you think and what I can improve in the comments.

How do you build a LLM from Scratch? This is the key question that I discussed with Sebastian Raschka, PhD on the AI Stories Podcast! For those who still don’t him, Sebastian is a Senior Staff Research Engineer at Lightning AI and a bestselling book author. He recently released a new book: "Build A Large Language Model From Scratch". This is a technical conversation around LLMs, we cover various topics including:? ?? His new book: "How to Build A LLM From Scratch" (link in the comments) ?? Differences in architecture between GPT2 and more recent LLMs like Llama 3.1 ?? How to build high performing LLMs ?? How to train multimodal LLMs ?? Lightning AI, PyTorch Lightning and litgpt libraries to train, finetune & deploy LLMs ?? Long context windows vs RAG And much more ... What are you waiting for? Go and listen to our conversation on your favourite platforms ??: Youtube: https://lnkd.in/e4h9vWGS Spotify: https://lnkd.in/exCbPMis Apple Podcast: https://lnkd.in/eXpDrdxg Have you read Sebastian's book? Keen to get your thoughts in the comments! What other LLM books do you recommend? Please share them in the comments as well :)

This week's agenda: ? Open Source of the Week - the dlt project ? Learning resources - Hugging Face code generation with LLM, building RAG application with JS, insurance premium prediction ? Book of the week - Data Science at the Command Line by Jeroen Janssens #datascience #machinelearning #llm #dataengineering

Future LLMs will be trained on synthetic data! Here are the pros and cons of using synthetic data to train them: Pros:? ? Synthetic data can be information dense and of high quality which could make it better suited to train LLMs than random content taken from the web ? Some niche topics don’t have much content on the web and so generating synthetic data can help improve LLMs in those particular areas Cons:? ? Generating synthetic data is expensive, especially if you want to train LLMs on a fully synthetic?dataset ? Since the datasets are generated by LLMs, hallucinations can be present ?? Future LMMs will probably be trained on a mix of real and synthetic datasets. The better the LLMs, the better the synthetic datasets that they will generate. And as the synthetic datasets get better, we will be able to build better LLMs! This will be a huge virtuous cycle! Recently released my conversation with Loubna Ben Allal, ML engineer at Hugging Face on the AI Stories Podcast. Want to learn how she built Cosmopedia, a dataset 100% synthetic and how she managed to build robust LLMs trained on it? Go and listen to our conversation on your favourite platform ??: Youtube: https://lnkd.in/eZu2aUmM Spotify: https://lnkd.in/e3KTc9ef Apple Podcast: https://lnkd.in/exdG_bzA What do you think? Will future LLMs be trained on synthetic data? Let me know in the comments!

“What makes an LLM better than another one? A lot has to do with the data!?If you look at recent papers, there are lots of details on LLMs architectures but not so much on the datasets used.” How can you leverage data to build robust LLMs? What are the data collection and filtering techniques that can be used? And what about Synthetic Data? How do you generate Synthetic Datasets? How useful can they be? What are the pros and cons? New episode of the AI Stories podcast is live ?? Our guest is Loubna Ben Allal, Machine Learning Engineer at Hugging Face. Loubna has done some amazing work for the ML community building code generation models like StarCoder and StarCoder2, Synthetic Datasets like Cosmopedia and more recently lead the efforts to train SmolLM: a family of state-of-the-art small models with 135M, 360M, and 1.7B parameters. Here are the key topics that we discussed: ?? How to built code generation models like StarCoder and StarCoder2? ?? Data techniques to build robust LLMs ?? How to build Synthetic Datasets? ?? Pros and Cons of using Synthetic Datasets to train LLMs? ?? Will GenAI soon replace developers?? And much more?... What are you waiting for? Go and listen to this episode on your favourite platforms ??: Youtube: https://lnkd.in/eZu2aUmM Spotify: https://lnkd.in/e3KTc9ef Apple Podcast: https://lnkd.in/exdG_bzA What do you think? Is synthetic data the future to building robust LLMs? Let me know your thoughts in the comments. #llms #genai #machinelearning

Interested in sponsoring the show? Send us a DM ??

As a Data Scientist, you should work on a side project! Whether at work or outside of work, having a side project is important. It helps you disconnect from your main task when you’re tired or out of ideas and it allows you to build something different and cool. It’s like taking a productive break! You’ll feel more relaxed when you get back to your main project. For Petar Veli?kovi?, Senior Staff Research Scientist at Google DeepMind, side projects ended up working quite well as he ended up building Google Map’s ETA algorithm along with an AI football coach assistant for Liverpool FC! Want to learn more about his story? Go and listen to our conversation on the AI Stories Podcast: Youtube: https://lnkd.in/e3UiF2jR Spotify: https://lnkd.in/eZYy9Z-H Apple Podcast: https://lnkd.in/e6bBg2Un What do you think? Should DS work on side projects? On what side project are you currently working on? Let me know in the comments!

AI Stories is Back! For the first episode of season 4, we’ve got Petar Veli?kovi?, Staff Research Scientist at Google DeepMind and Affiliated Lecturer at University of Cambridge. Episode full of learnings where we discuss:? ?? How to get into Deepmind ?? Petar’s most cited paper Graph Attention Networks? ?? How Petar Built Google Maps ETA algorithm? ?? How he develop and AI coach assistant for Liverpool FC? ?? Research Scientists vs Research Engineers And much more! What are you waiting for? Got and listen to this episode on your favourite platforms ?? Youtube: https://lnkd.in/e3UiF2jR Spotify: https://lnkd.in/eZYy9Z-H Apple Podcast: https://lnkd.in/e6bBg2Un Which guest do you want to see on season 4? Let me know in the comments :)

15 episodes later, and it’s the end of AI Stories season 3! This has been an amazing season with a couple of highlights: - AI stories has now over 100k downloads across all platforms??? - The number of YouTube subscribers increased by 280% from 800 to over 3k ?? - New record episode with Dr. Christof Henkel - Number 1 in the world on Kaggle and over 15k downloads ?? - Amazing guests from all around the world with different roles, different backgrounds and different visions of the field. A big thanks to Davis Blalock Marianne Ducournau Noah Gift Demetrios Brinkmann Christoph Molnar Ryan Shannon Harpreet Sahota ?? Maxime Labonne Franziska Kirschner Andras Palffy Erwin H. Reah Miyara Maria Vechtomova Lewis Tunstall & Dr. Christof Henkel Time to take a break now and prepare for season 4, AI stories will be back in October 2024!