neptune.ai

Training large foundation models? We built Neptune Scale to let you monitor such training and debug any issues quickly. Now available in beta: https://buff.ly/4eCFUpz Coming soon for everyone. #generativeai #genai #llm

Choosing the right compute strategy starts with understanding the task's complexity. In this clip from our State of LLM Training series, Ram Singh explains how different models require different compute resources—some can run efficiently on CPUs, while others demand GPUs even at inference. Balancing model size and deployment costs is key to scalability, and this is just one of the many considerations explored in our upcoming report. — Follow along for more insights! (Link to the full snippet playlist in the comments) #generativeai #genai #llm

During last year’s NeurIPS, we put Lukas Klein, Graduate Researcher at ETH Zürich and DKFZ German Cancer Research Center, in the hot seat. He accepted the challenge of answering ‘impossible’ questions asked by other AI researchers. Watch to see the result! — (Link to the full playlist in the comments) #neurips #generativeai #genai #llm

We're launching a new video series featuring the most trending AI discussions straight from Reddit—technical insights served with a touch of Reddit snark. First up: How did DeepSeek R1 achieve massive training cost reductions? Let us know what you think! — (Link to the full thread in the comments) #generativeai #genai #llm #foundationmodels

“Our logs balloon in size if we try to log all gradients, so we just skip it—and then we’re missing crucial clues when something goes wrong.” Ever happened to you? But if browsing and visualization are fast and efficient, you don’t have to choose what to track—you just log it all. This way, you can debug every training issue. #generativeai #genai #llm #foundationmodels

How predictably does the performance of a language model scale? During last year’s NeurIPS, Yangjun Ruan, PhD student at the University of Toronto, presented his work titled “Observational Scaling Laws and the Predictability of Language Model Performance”. — Read the paper: https://buff.ly/pve35l3 Watch the full presentation: https://buff.ly/djgkvH6 #generativeai #genai #llm #neurips

[New on our blog] Bayesian Deep Learning is Needed in the Age of Large-Scale AI [Paper Reflection] Author: Vincent Fortuin Reading time: 6 min — (link to the full article in the comments) #generativeai #genai #llm

For large foundation models, subtle issues in just a few layers can cause silent degradation in the training process. The problem? Aggregate metrics often mask these instabilities. Without tracking layer-wise activations, gradients, and losses, you often don’t see the issues. How granular is your logging—do you monitor individual layers or only global loss? #generativeai #genai #llm

Some AI questions seem impossible—until someone dares to answer. At NeurIPS 2024, we challenged Amaury Gouverneur, PhD student at Kungliga Tekniska h?gskolan, with some of the toughest ones, like: “What combination of existing tech plus new developments will it take for us to run billion-parameter architectures on edge devices?” Watch to hear his perspective. — (Link to the full playlist in the comments) #neurips #generativeai #genai #llm

Maintaining AI infrastructure requires constant work—one that many ML/AI teams are forced to handle on their own. Keunwoo Choi shares the challenges AI teams face when training foundation models from scratch without dedicated infra support: → Role conflict: researchers take on infrastructure maintenance, often diverting focus from model development. → Choosing between GPU utilization vs. delivery: maximizing GPU efficiency is tempting (given the cost), but sometimes, the speed of iteration matters more. → Debugging nightmares: as GPU clusters scale, failures increase, and error messages rarely provide useful diagnostics. — Our upcoming report dives deeper into these challenges. Follow along for more insights! #generativeai #genai #llm #foundationmodels