Co-founder @ CambioML (YC | HUB71) | ex-DeepMind | ex-AWS | ex-Microsoft | Stanford AI | Learn Everything | Hiring
?? Recap from Day 1: FlashMLA On Day 1, DeepSeek introduced FlashMLA, an innovative solution that optimizes both memory and computation on a single GPU using Multi-Head Latent Attention (MLA). Benchmark tests on the H800 SXM5 GPU with CUDA 12.6 have shown impressive performance—up to 3000 GB/s in memory-bound configurations (about 1.8× improvement) and 580 TFLOPS in computation-bound scenarios (roughly 10× faster). For more insights into the challenges they’re tackling, check out this Databricks post: https://lnkd.in/gNQzCRaR. ?? Day 2: DeepEP On Day 2, DeepSeek extends its innovation to the network layer with DeepEP, a communication library designed to optimize data exchange in Mixture-of-Experts (MoE) and Expert Parallelism (EP) architectures across multiple GPUs. DeepEP efficiently manages both intra-node communication (using NVLink for GPU-to-GPU transfers within a single node) and inter-node communication (using InfiniBand/RDMA for data exchange between separate nodes). It achieves near-peak performance on the H800—leveraging approximately 160 GB/s over NVLink and 50 GB/s over InfiniBand/RDMA. ?? Takeaway: With optimizations across compute, memory, and network, DeepSeek is offering a holistic solution for both inference and training. I’m excited to see what they will open source next! - open-infra-index: https://lnkd.in/drYtxerk - DeepEP: https://lnkd.in/gQsv_j6M #AI #DeepLearning #OpenSource #Networking #MachineLearning #DemocratizingAI #DeepSeek