NVIDIA: Cosmos World Foundation Model Platform for Physical AI
Overview
Introducing NVIDIA's Cosmos World Foundation Model (WFM) platform for Physical AI. Cosmos uses a pre-training and post-training paradigm, employing both diffusion and autoregressive models trained on a massive, curated video dataset (20M hours) to create generalist WFMs.
These are then fine-tuned for specialized Physical AI tasks like robotic manipulation and autonomous driving. The platform includes a novel video tokenizer for efficient processing and a guardrail system for safety.
Results demonstrate state-of-the-art performance across various benchmarks and applications.
Podcast
Full Report: