Issue 25: NVIDIA

Introduction?

Posting after a long hiatus due to travel.

As the leading maker of chips for artificial intelligence with more than 80% of the GPU market share Nvidia is currently one of the most watched companies in the AI world. They unveiled the Blackwell platform at their annual GPU technology conference (GTC) on March 18th. We are focusing on understanding the Blackwell platform and ecosystem today.??

GPUs (Graphics Processing Units), once only designed for gaming acceleration, are especially well suited for AI tasks because their massively parallel architecture accelerates the immense number of matrix multiplication tasks necessary to run today's neural networks.??

Market overview?

Nvidia has a dominant position in the AI chips market, which is estimated to reach?$140B by 2027 according to Gartner.??

Nvidia’s stock has soared over 435% since the beginning of 2023 riding on the high GPU demand. Its top and bottom lines have exploded, rising at triple-digit rates quarter after quarter. Last year Nvidia’s value soared past $1 trillion, an all-time high. In the company's recently concluded fiscal year 2024 (which ended on Jan. 28), the data center business produced a record $47.5 billion in revenue, accounting for 79% of its top line. That was a massive increase of 217% from the year-ago period. The data center business recorded a much stronger year-over-year increase of 409% in revenue to $18.4 billion in fiscal Q4, significantly outpacing the segment's annual growth which suggests Nvidia's data center business is still gaining momentum. Nvidia expects revenue of $24 billion in the first quarter of fiscal 2025, which would be a 233% increase from the year-ago period.?

Nvidia's R&D expenses have also climbed over the years, reaching $8.7 billion in fiscal 2024. That's up nearly 500% in just seven years, indicating Nvidia has been focused on driving innovation over that period.?

NVIDIA Business

Nvidia's inception aimed to revolutionize 3D computer graphics for gaming and multimedia sectors. Until fiscal 2022 (ended Jan. 30, 2022), gaming was routinely Nvidia's largest revenue driver. Then there was rapid change. The Data centers, where companies stored valuable information, evolved to power cloud computing. Today, data centers are home to powerful chips designed by Nvidia to process AI workloads. With hyperscalers like Microsoft and Amazon ordering hundreds of thousands of them to give cloud customers the computing power they need to develop AI, Nvidia's data center revenue soared and now accounts for 80% of Nvidia's total revenue, leaving the gaming segment far behind.

NVIDIA GPU Architectures

NVIDIA has developed several GPU architectures over the years, each improving upon the last in terms of performance, efficiency, and capabilities across gaming, AI, data center, and professional visualization. Each series of GPU's was purpose built to address a problem.

• Volta: Set the groundwork for AI-focused Tensor Cores.
• Turing: Brought real-time ray tracing and AI upscaling to consumer graphics.
• Ampere: Improved upon Turing’s capabilities for both AI and gaming.
• Hopper: Specialized for large-scale AI and HPC applications.
• Ada Lovelace: Optimized for gaming, real-time ray tracing, and professional visualization.
• Blackwell (anticipated): Expected to push efficiency and performance in AI and HPC further.

The NVIDIA Blackwell architecture is expected to serve as a foundation for multiple GPU lines, similar to how previous architectures like Ampere and Ada Lovelace powered various NVIDIA product series, including GeForce, Quadro, and TITAN.

1. GeForce Series (Gaming and Consumer Graphics)

If Blackwell is extended to GeForce GPUs, it would likely bring significant improvements to gaming and consumer GPUs, especially around enhanced ray tracing, AI-accelerated features, and performance-per-watt efficiency. Blackwell-based GeForce GPUs would potentially be branded as the next “RTX 50” series, focusing on delivering a high-performance experience for gamers and content creators.

2. Quadro Series (Professional Visualization)

For the Quadro series, Blackwell could bring new advancements in computational graphics, especially in rendering, simulation, and CAD applications for professionals in engineering, media, and design. Blackwell’s efficiency and AI capabilities could also enable faster 3D modeling, higher-quality visualizations, and optimized productivity in workstation environments.

3. TITAN Series (Prosumer and AI Enthusiast GPUs)

If NVIDIA continues with a TITAN product line under Blackwell, this GPU would likely sit between the GeForce and Quadro lines, offering powerful performance suited to both prosumer gaming and AI research. The TITAN Blackwell variant would likely be an excellent fit for researchers, developers, and AI enthusiasts who need a powerful GPU without moving to the more specialized (and more costly) data center GPUs.

Blackwell Platform?

Fun fact - Nvidia named the Blackwell architecture after?David Harold Blackwell, a mathematician who specialized in game theory and statistics and was the first Black scholar inducted into the National Academy of Sciences.??

?

Blackwell platform is the latest version of Nvidia’s AI-powering technology. According to Nvidia, Blackwell architecture will enable up to 30 times greater inference performance and consume 25 times less energy for massive AI models and scaling up to 10 trillion-parameter AI models that will make today's generative AI models look rudimentary in comparison. For reference, OpenAI's GPT-3, launched in 2020, included 175 billion parameters and GPT4 is rumored to have around 1.5 trillion parameters (Parameter count is a rough indicator of AI model complexity).??

?

Blackwell Innovations to Fuel Accelerated Computing and Generative AI?

Blackwell GPU architecture features six transformative technologies for accelerated computing, which will help unlock breakthroughs in data processing, engineering simulation, electronic design automation, computer-aided drug design, quantum computing and generative AI. These include:?

• World’s Most Powerful Chip?— Packed with 208 billion transistors, Blackwell-architecture GPUs are manufactured using a custom-built 4NP TSMC process with two-reticle limit GPU dies connected by 10 TB/second chip-to-chip link into a single, unified GPU.?
• Second-Generation Transformer Engine?— Fueled by new micro-tensor scaling support and NVIDIA’s advanced dynamic range management algorithms integrated into NVIDIA TensorRT?-LLM and NeMo Megatron frameworks, Blackwell will support double the compute and model sizes with new 4-bit floating point AI inference capabilities.?
• Fifth Generation NVLink?— To accelerate performance for multitrillion-parameter and mixture-of-experts AI models, the latest iteration of NVIDIA NVLink? delivers groundbreaking 1.8TB/s bidirectional throughput per GPU, ensuring seamless high-speed communication among up to 576 GPUs for the most complex LLMs.?
• RAS Engine?— Blackwell-powered GPUs include a dedicated engine for reliability, availability, and serviceability. Additionally, the Blackwell architecture adds capabilities at the chip level to utilize AI-based preventative maintenance to run diagnostics and forecast reliability issues. This maximizes system uptime and improves resiliency for massive-scale AI deployments to run uninterrupted for weeks or even months at a time and to reduce operating costs.?
• Secure AI?— Advanced confidential computing capabilities protect AI models and customer data without compromising performance, with support for new native interface encryption protocols, which are critical for privacy-sensitive industries like healthcare and financial services.?
• Decompression Engine?— A dedicated decompression engine supports the latest formats, accelerating database queries to deliver the highest performance in data analytics and data science. In the coming years, data processing, on which companies spend tens of billions of dollars annually, will be increasingly GPU-accelerated.?

Blackwell GPU Chips B100, B200 And GB200 Chips?

Nvidia unveiled the first GPU designs to use the Blackwell architecture, the B100 and the B200 GPUs and GB200 Superchip. The B200 is the company’s most powerful single-chip GPU with 208 billion transistors (160% increase against their previous model H100), which Nvidia claims can reduce AI inference operating costs (such as running ChatGPT) and energy consumption by up to 25 times compared to their previous H100 chips. expected to include a greater high-bandwidth memory capacity than the B100.??

The GB200 Grace Blackwell Superchip, which, on a single package, connects a B200 GPU with the company’ s Arm-based, 72-core Grace CPU significantly increases the performance scaling and is expected to handle demanding AI workloads with trillion parameters. This is made possible through its 2nd Gen Transformer engine.??

Blackwell GPUs can perform up to 20 petaflops, which amounts to 20 quadrillion calculations per second and delivers. They are able to do this using a new numerical format called four-bit floating point, FP4 that is more granular in precision than the previous format. This allows Blackwell to double the size of an AI model that can fit on a single GPU which increases performance.??

The Blackwell chips are also available on?NVIDIA DGX Cloud, a cloud-based AI supercomputing platform co-engineered with leading cloud service providers that gives enterprise developers dedicated access to the infrastructure and software needed to train and deploy advanced generative AI models.?

?

Big tech already using Blackwell?

Nvidia had already sold tens of thousands of Blackwell GPUs before the platform was even announced.?AWS, Microsoft Azure, Google Cloud, Dell Technologies, Meta, OpenAI, Oracle Cloud, Tesla, xAI are among the partners set to use Blackwell. The CEOs for all these companies released statements supporting Blackwell timed for this launch at GTC.??

Cisco,?Dell,?Hewlett Packard Enterprise,?Lenovo?and?Supermicro?are expected to deliver a wide range of servers based on Blackwell products, as are Aivres,?ASRock Rack,?ASUS, Eviden,?Foxconn,?GIGABYTE,?Inventec,?Pegatron,?QCT, Wistron,?Wiwynn?and ZT Systems.?

Several companies in the NVIDIA Cloud Partner program, and a growing network of software makers in the engineering simulation space (Ansys, Cadence and Synopsys) announced that they will use Blackwell-based processors to accelerate their software for designing and simulating electrical, mechanical and manufacturing systems and parts.??

?

The Future?

This is just the beginning of the AI transformation and NVIDIA as the market leader for GPU chips is poised to take advantage of that for several years to come. Their competitive advantage is their GPU combined with software, frameworks and tools required by developers to work with their GPUs. These strengths, combined with the high costs of switching AI platforms give NVIDIA a strong moat.??

That said, there is competition ahead. All the leading hyperscale vendors like Amazon, Microsoft, Google and Meta are seeking to reduce their reliance on Nvidia and are seeking to diversify their semiconductor and software supplier partners. Amazon announced an investment of $4B in Anthropic (will be covered in depth later in this newsletter) in September 2023 with Anthropic agreeing to build its AI solutions using AI chips designed by Amazon. Meta announced plans to deploy in-house custom chips to power AI. Microsoft and Google are also developing in-house custom AI ships to train their models. OpenAI is pushing to raise billions for a network of AI chip factories.??

Rival chip makers like AMD and Intel are working on providing alternatives to Nvidia GPU chips. Meta, OpenAI and Microsoft all said that they will use AMD’s new AI chip, the Instinct MI300X as companies look for alternatives to the expensive Nvidia GPUs. If the MI300X proves to be good enough, this can lower the costs for developing AI models and take market share from Nvidia. Intel also announced new chips including Gaudi3 which compete with Nvidia’s H100.??

This is an exciting space to watch, and we can expect more news coming from all the players in this space which will hopefully lead to more efficient and lowered costs for training and running AI models.??

?

?

References?

NVIDIA Blackwell platform arrivers to power a new era of computing?

Nvidia’s AI chip dominance?

Nvidia facts and statistics?

Nvidia unveils Blackwell B200?

Nvidia reveals next-gen Blackwell GPUs?

Challenge to Nvidia’s dominance?

Meta plans to in-house custom AI chips?