AI Stories

How do you build a LLM from Scratch? This is the key question that I discussed with Sebastian Raschka, PhD on the AI Stories Podcast! For those who still don’t him, Sebastian is a Senior Staff Research Engineer at Lightning AI and a bestselling book author. He recently released a new book: "Build A Large Language Model From Scratch". This is a technical conversation around LLMs, we cover various topics including:? ?? His new book: "How to Build A LLM From Scratch" (link in the comments) ?? Differences in architecture between GPT2 and more recent LLMs like Llama 3.1 ?? How to build high performing LLMs ?? How to train multimodal LLMs ?? Lightning AI, PyTorch Lightning and litgpt libraries to train, finetune & deploy LLMs ?? Long context windows vs RAG And much more ... What are you waiting for? Go and listen to our conversation on your favourite platforms ??: Youtube: https://lnkd.in/e4h9vWGS Spotify: https://lnkd.in/exCbPMis Apple Podcast: https://lnkd.in/eXpDrdxg Have you read Sebastian's book? Keen to get your thoughts in the comments! What other LLM books do you recommend? Please share them in the comments as well :)

This week's agenda: ? Open Source of the Week - the dlt project ? Learning resources - Hugging Face code generation with LLM, building RAG application with JS, insurance premium prediction ? Book of the week - Data Science at the Command Line by Jeroen Janssens #datascience #machinelearning #llm #dataengineering

Future LLMs will be trained on synthetic data! Here are the pros and cons of using synthetic data to train them: Pros:? ? Synthetic data can be information dense and of high quality which could make it better suited to train LLMs than random content taken from the web ? Some niche topics don’t have much content on the web and so generating synthetic data can help improve LLMs in those particular areas Cons:? ? Generating synthetic data is expensive, especially if you want to train LLMs on a fully synthetic?dataset ? Since the datasets are generated by LLMs, hallucinations can be present ?? Future LMMs will probably be trained on a mix of real and synthetic datasets. The better the LLMs, the better the synthetic datasets that they will generate. And as the synthetic datasets get better, we will be able to build better LLMs! This will be a huge virtuous cycle! Recently released my conversation with Loubna Ben Allal, ML engineer at Hugging Face on the AI Stories Podcast. Want to learn how she built Cosmopedia, a dataset 100% synthetic and how she managed to build robust LLMs trained on it? Go and listen to our conversation on your favourite platform ??: Youtube: https://lnkd.in/eZu2aUmM Spotify: https://lnkd.in/e3KTc9ef Apple Podcast: https://lnkd.in/exdG_bzA What do you think? Will future LLMs be trained on synthetic data? Let me know in the comments!

“What makes an LLM better than another one? A lot has to do with the data!?If you look at recent papers, there are lots of details on LLMs architectures but not so much on the datasets used.” How can you leverage data to build robust LLMs? What are the data collection and filtering techniques that can be used? And what about Synthetic Data? How do you generate Synthetic Datasets? How useful can they be? What are the pros and cons? New episode of the AI Stories podcast is live ?? Our guest is Loubna Ben Allal, Machine Learning Engineer at Hugging Face. Loubna has done some amazing work for the ML community building code generation models like StarCoder and StarCoder2, Synthetic Datasets like Cosmopedia and more recently lead the efforts to train SmolLM: a family of state-of-the-art small models with 135M, 360M, and 1.7B parameters. Here are the key topics that we discussed: ?? How to built code generation models like StarCoder and StarCoder2? ?? Data techniques to build robust LLMs ?? How to build Synthetic Datasets? ?? Pros and Cons of using Synthetic Datasets to train LLMs? ?? Will GenAI soon replace developers?? And much more?... What are you waiting for? Go and listen to this episode on your favourite platforms ??: Youtube: https://lnkd.in/eZu2aUmM Spotify: https://lnkd.in/e3KTc9ef Apple Podcast: https://lnkd.in/exdG_bzA What do you think? Is synthetic data the future to building robust LLMs? Let me know your thoughts in the comments. #llms #genai #machinelearning

Interested in sponsoring the show? Send us a DM ??

As a Data Scientist, you should work on a side project! Whether at work or outside of work, having a side project is important. It helps you disconnect from your main task when you’re tired or out of ideas and it allows you to build something different and cool. It’s like taking a productive break! You’ll feel more relaxed when you get back to your main project. For Petar Veli?kovi?, Senior Staff Research Scientist at Google DeepMind, side projects ended up working quite well as he ended up building Google Map’s ETA algorithm along with an AI football coach assistant for Liverpool FC! Want to learn more about his story? Go and listen to our conversation on the AI Stories Podcast: Youtube: https://lnkd.in/e3UiF2jR Spotify: https://lnkd.in/eZYy9Z-H Apple Podcast: https://lnkd.in/e6bBg2Un What do you think? Should DS work on side projects? On what side project are you currently working on? Let me know in the comments!

AI Stories is Back! For the first episode of season 4, we’ve got Petar Veli?kovi?, Staff Research Scientist at Google DeepMind and Affiliated Lecturer at University of Cambridge. Episode full of learnings where we discuss:? ?? How to get into Deepmind ?? Petar’s most cited paper Graph Attention Networks? ?? How Petar Built Google Maps ETA algorithm? ?? How he develop and AI coach assistant for Liverpool FC? ?? Research Scientists vs Research Engineers And much more! What are you waiting for? Got and listen to this episode on your favourite platforms ?? Youtube: https://lnkd.in/e3UiF2jR Spotify: https://lnkd.in/eZYy9Z-H Apple Podcast: https://lnkd.in/e6bBg2Un Which guest do you want to see on season 4? Let me know in the comments :)

15 episodes later, and it’s the end of AI Stories season 3! This has been an amazing season with a couple of highlights: - AI stories has now over 100k downloads across all platforms??? - The number of YouTube subscribers increased by 280% from 800 to over 3k ?? - New record episode with Dr. Christof Henkel - Number 1 in the world on Kaggle and over 15k downloads ?? - Amazing guests from all around the world with different roles, different backgrounds and different visions of the field. A big thanks to Davis Blalock Marianne Ducournau Noah Gift Demetrios Brinkmann Christoph Molnar Ryan Shannon Harpreet Sahota ?? Maxime Labonne Franziska Kirschner Andras Palffy Erwin H. Reah Miyara Maria Vechtomova Lewis Tunstall & Dr. Christof Henkel Time to take a break now and prepare for season 4, AI stories will be back in October 2024!

Just listened to the latest episode of the AI Stories podcast featuring Lewis Tunstall, LLM Engineer and Researcher at ?? Why I recommend it: ?? Lewis shares his relatable career journey from academic research to data engineering, data science, and ML engineering, showing how AI careers can have many "twists and bends" ?? The practical perspective: For instance, the importance of finding the simplest solution to?business problems as a data scientist ?? As a mathematician, I was inspired to explore more about Topological Machine Learning and its applications ?? Interesting insights on Open Source and Closed Source LLMs ?? Learning about Lewis's work at Hugging Face and his book on Transformers A big thanks to Neil Leiser for consistently delivering high-quality AI stories! ?? #AI #MachineLearning #PodcastRecommendation

"Being an effective Data Scientist is actually not about training the best model... It's about finding the simplest solution to the actual business problem!" I released last week my conversation with Lewis Tunstall, LLM Engineer at Hugging Face, on the AI Stories podcast. Lewis also wrote together with Leandro von Werra and Thomas Wolf the well known book "Natural Language Processing with Transformers". If you want to learn more about fine-tuning LLMs, transformers architectures, topological machine learning and how Lewis broke prod on a Friday evening then go and listen to the episode on your favourite platforms??: Youtube: https://lnkd.in/ecCPJnpH Spotify: https://lnkd.in/etZhBM2C Apple Podcast: https://lnkd.in/erN5-qZk Do you agree? What makes an effective Data Scientist according to you? Let me know in the comments! #llms #datascientist #machinelearning