REVO.tech - B2B Global Electronics Marketplace的动态

The Secret to Faster #AI: The Revolution Driving Host CPUs [Part-II] L3 Cache and CXL 2.0 Support The increased L3 cache, which can be as large as 504 MB, significantly boosts performance by storing frequently accessed data close to the CPU. This reduces the need for the CPU to fetch data from slower memory sources, speeding up processing times, especially for repetitive tasks common in AI workloads. Additionally, support for CXL 2.0 and MRDIMM technology are game-changers. CXL 2.0 enables memory coherency between the CPU and attached devices, such as GPUs, allowing for seamless resource sharing and improving overall system performance. MRDIMM further enhances memory bandwidth by using multiplexing techniques to increase data throughput, which optimizes access to high-speed memory and reduces latency. Together, CXL 2.0 and MRDIMM contribute to a more efficient and scalable system architecture, reducing complexity in software management and ensuring fast, reliable performance in large-scale AI workloads. Energy Efficiency and Scalability As businesses scale up their AI operations, energy efficiency becomes a priority. New chips, such as the Intel? Xeon? 6 with P-cores, offer up 5.5x higher AI inferencing performance compared to other processors and up to 1.9x better performance per watt compared to earlier generations. This is crucial for businesses looking to balance high performance with energy costs. Another example of enhanced performance provided by these new processors is the support for FP16 models on Intel Xeon 690-series with P-cores, available only as a P-core feature. Learn more about how processor advancements are leveling up CPU power and driving companies toward greater technological performance by reading the entire article: https://bit.ly/4fiV1or by Ronald van Loon| #IntelAmbassador Intel Corporation #ArtificialIntelligence #MachineLearning #DataScience #Technology #Innovation Cc: Pascal BORNET | Yves Mulkers | Franco Ronconi | Terence Leung |

AMD Unveils AI-Focused Processor Lineup AMD has launched its latest lineup of AI-optimized processors at Advancing AI 2024, targeting the booming data center and AI chip market. Key Features: ?? ?Ryzen AI PRO 300: 40% better performance than Intel's Core Ultra chips for enterprise AI PCs ?? ?Instinct MI325X AI accelerator: 1.8x higher memory capacity, 1.3x more bandwidth than Nvidia's H200 GPU ?? ?EPYC 5th Gen CPUs: "World's best for enterprise, AI, and cloud" (CEO Lisa Su) ?? ?Annual AI chip releases planned: MI350X (2025), MI400 (2026) Impact: ?? ?AMD aggressively competes with Nvidia and Intel in the AI chip market ?? ?Expands possibilities for AI adoption in enterprise, cloud, and data centers ?? ?Addresses growing demand for powerful AI processors Article: https://lnkd.in/gy2bacSp How will AMD's new AI-focused processors impact your business or projects? Will you leverage these advancements? #AMD #AIProcessors #DataCenter #ArtificialIntelligence #TechInnovation #ChipWar

?? The Future of Accelerators is Here! ?? ? Server accelerators are revolutionizing AI, HPC, and edge computing by reshaping data center architectures and enhancing compute efficiency. ? Supermicro is at the forefront of this evolution, offering innovative GPU servers tailored to meet the most diverse application needs. ??????? ?? Accelerator Architectures for Cutting-Edge Applications: 1?? Integrated Accelerators: - High-speed GPU-to-GPU communication without CPU bottlenecks. - Perfect for large AI models requiring massive memory and data-sharing capabilities. - Enables scaling across servers, allowing accelerators to share data seamlessly. 2?? PCIe Interconnect Accelerators: - Scalable from 1 to 10 GPUs per server, ideal for independent parallel tasks. - Excels in VDIs, visualization, and content delivery, where workloads are independent. - Supports flexible configurations like Direct, Single Root, and Dual Root modes. 3?? Integrated GPUs and CPUs: - 1TB/s bandwidth between CPU and GPU for unmatched speed and shared memory. - Simplifies programming and enables new applications requiring real-time computation. - Ideal for AI inferencing and other high-speed, tightly coupled workloads. ?? Supermicro’s Role in Accelerator Innovation: Supermicro’s GPU servers, designed for AI training, inferencing, HPC, and edge deployments, lead the way with: - Scalable and energy-efficient architectures. - Advanced liquid cooling to optimize performance and reduce costs. - Support for cutting-edge GPU technologies from the world leaders NVIDIA, AMD and Intel Corporation, enabling the most demanding workloads. ?? Discover More: Explore how Supermicro GPU servers can elevate your AI and HPC performance: ?? Learn More About Supermicro GPU Solutions: https://lnkd.in/dbztp45H https://lnkd.in/e2ZthFjh ? ?? Let’s shape the future of AI and HPC and drive innovation together! ?? ? Contact your local Italian Supermicro Team to find YOUR tailored solution. ????? ? ? #Supermicro?#AMD?#Nvidia?#Intel?#ARM #ampere #AI?#ML?#LLM #LMM #Liquidcooling?#Solutions?#Server?#GPU?#storage #vSAN #Intelligence #Evolution?#technology?#performace?#greenIT #cluster #generative #hpc #greencomputing?#TCO?#innovation #media #entertainment #HPC #cloud #inference #scale #5G #Telco?#DataAnalytics #enterprise?#edge #Virtualization #cloud #data #healthcare?#medical #BOT #chatbot #DataCenterEfficiency #TechInnovation #AIsupercomputer #AIRevolution #GenerativeAI #SuperCluster #AIInnovation #SustainableAI #TechRevolution #ArtificialIntelligence #FutureOfAI #AIInfrastructure #Healthcare #Finance #Telecommunications #SupermicroItalia #EdgeComputing #RetailTech #SustainableTech

I've been researching Building the Next-Gen Data Center with NVIDIA Hardware Are you planning to construct a high-performance data center tailored for AI, machine learning, or HPC? Here's your blueprint using NVIDIA's cutting-edge technology: ? Hardware Components: GPUs: Opt for NVIDIA's A100 or the latest H100 from the Hopper architecture. For even more advanced AI computing, consider the new Blackwell GPUs. ?? Servers: NVIDIA DGX Systems like DGX A100 or H100 are ideal for pre-configured AI solutions. Alternatively, customize with servers from Dell, HPE, or Lenovo, ensuring MGX compatibility for flexibility. ?? Networking: NVIDIA Spectrum, particularly the Spectrum-4, offers up to 400Gbps for seamless data center networking. ?? CPU: Integrate NVIDIA Grace CPU with Hopper or Blackwell GPUs for the ultimate in performance and efficiency. ? Infrastructure: Cooling: Liquid cooling is crucial for managing heat from high-performance GPUs. Look into solutions from NVIDIA's cooling partners like AVC. ? Power: Plan for robust power supply systems to support high wattage GPUs. ? Storage: High-speed SSDs to complement your processing power. ?? Software & Management: Software: Leverage NVIDIA AI Enterprise for a complete AI solution stack. ?? Management: Use Kubernetes or SLURM for workload management, paired with NVIDIA's monitoring tools for optimal performance. ? Deployment Strategy: Colocation vs. In-house: Decide if you'll build from scratch or utilize NVIDIA DGX-Ready colocation centers. ? Scalability: Design with modularity in mind for future growth. #DataCenter #NVIDIA #AI #MachineLearning #HPC #Technology

NVIDIA Blackwell's High Power Consumption Drives Cooling Demands; Liquid Cooling Penetration Expected to Reach 10% by Late 2024, Says TrendForce Corporation ?? With the growing demand for high-speed #computing, more effective cooling solutions for #AI #servers are gaining significant attention. TrendForce Corporation's latest report on AI servers reveals that #NVIDIA is set to launch its next-generation Blackwell platform by the end of 2024. Major CSPs are expected to start building AI server #datacenters based on this new platform, potentially driving the penetration rate of liquid cooling solutions to 10%. TrendForce reports that the NVIDIA #Blackwell platform will officially launch in 2025, replacing the current Hopper platform and becoming the dominant solution for NVIDIA's high-end #GPUs, accounting for nearly 83% of all high-end products. High-performance AI server models like the B200 and GB200 are designed for maximum efficiency, with individual GPUs consuming over 1,000W. HGX models will house 8 GPUs each, while NVL models will support 36 or 72 GPUs per rack, significantly boosting the growth of the liquid cooling supply chain for AI servers. TrendForce highlights the increasing TDP of #server #chips, with the B200 chip's TDP reaching 1,000W, making traditional air cooling solutions inadequate. The TDP of the GB200 NVL36 and NVL72 complete rack systems is projected to reach 70kW and nearly 140kW, respectively, necessitating advanced liquid cooling solutions for effective heat management. Thanks again to TrendForce Corporation for the full article with more background and insights via the link below ?????? https://lnkd.in/eVkkM7Av #semiconductorindustry #semiconductors #semiconductormanufacturing #technology #chip #chips #artificialintelligence #tsmc #icdesign #usa #it #taiwan #advancedpackaging #computer #computing #innovation #cpu

Micron launches “world’s fastest” data center SSD Micron has unveiled the 9550 SSD, claimed to be the world’s fastest data center solid-state drive (SSD). This integrated solution combines Micron’s own controller, NAND, DRAM, and firmware, designed for critical workloads like AI, performance databases, caching, online transaction processing (OLTP), and high-frequency trading. The SSD offers 14.0 GBps sequential reads and 10.0 GBps sequential writes, boasting up to 67 percent better performance than competitor SSDs from Kioxia and Samsung. It also shows a 35 percent improvement in random reads (3,300 KIOPS) and a 33 percent improvement in random writes (400 KIOPS). Micron's 9550 SSD excels in AI workloads, with up to 33 percent faster completion times and 60 percent faster feature aggregation in GNN training using Nvidia's Big Accelerator Memory (BaM). According to MLPerf tests, it uses up to 35 percent less SSD energy and 13 percent less system energy. Available in capacities from 3.2TB to 30.72TB and in U.2, E1.S, and E3.S form factors, the SSD is tailored for OEMs and hyperscalers. In April, Micron received $6.14 billion under the CHIPS and Science Act for new memory chip fabs in New York and Idaho. However, construction in New York is delayed due to the discovery of endangered bat species on the land. #lureit #wannabelured #hardware #software #network #base #data #it #project #development #company #digital #power #launch #solution #chip #microsoft #micron #ssd #power #construction #datacenter

?? AMD approached to make world's fastest AI supercomputer powered by 1.2 million?GPU ?? ?? Nvidia has been the leading GPU supplier for data centers, but AMD is emerging as a potential competitor. ?? AMD was approached to create an AI training cluster with 1.2 million GPUs. ?? This would make it 30 times more powerful than the current fastest supercomputer, Frontier. ?? In 2023, AMD supplied less than 2% of data center GPUs. ?? Forrest Norrod, AMD's GM of Datacenter Solutions, confirmed the inquiries in an interview with The Next Platform. ?? Typical AI training clusters use a few thousand GPUs, making the scale of this project unprecedented. ?? Challenges include ensuring low latency, managing power consumption, and handling hardware failures. ?? AI training requires significant computing power and synchronous processing across all nodes. Follow us for more latest Semiconductor Updates! KeenHeads #vlsi #semicondcutor #keenheads #innovation #technology #future #india #ai #gpu #amd #News #electronics

What are the best figures required to manage high-level AI traffic? Here are some general guidelines for preferred hardware specifications: ???????GPUs: High-performance GPUs are crucial for efficient training and running AI models. NVIDIA's A100 or H100 Tensor Core GPUs are popular choices for demanding AI workloads due to their high processing power and specialized capabilities for AI tasks. ???????CPUs: A powerful CPU is still essential for general tasks and data handling, like multi-core processors from Intel (e.g., Xeon Scalable) or AMD (e.g., EPYC). ???????Memory: AI workloads require substantial RAM for high-performance applications, 512 GB or 1 TB of RAM, depending on the complexity ???????Storage: Fast storage solutions, such as NVMe SSDs, are preferred for quick data access and high throughput ???????Networking: High bandwidth networking capabilities, such as 100 GbE or higher, are essential for efficiently handling large data transfers and distributed computing tasks. ???????Cooling and Power: Given the intensity of AI workloads, effective cooling, and a reliable power supply are essential to maintaining hardware performance and longevity. ???????Scalability: For extensive AI projects, consider hardware that supports scalability, allowing you to expand resources as needed. ? When choosing a hardware supplier for AI traffic management, several factors include performance, scalability, and specialized AI capabilities. Here are some top contenders: 1. NVIDIA: Strengths: NVIDIA is a leader in AI hardware, particularly its GPUs, widely used in AI and deep learning tasks. NVIDIA offers specialized AI tools and libraries (e.g., TensorRT, CUDA) optimized for its hardware, providing robust support for traffic analysis, simulation, and real-time processing. 2. Intel provides a range of AI-focused hardware, including CPUs, FPGAs, and specialized AI accelerators (e.g., Intel Movidius). 3. AMD offers competitive GPUs and CPUs that are increasingly being used in AI applications. Their EPYC processors and Radeon Instinct GPUs perform well and are cost-effective for AI workloads. 4. Lenovo offers a range of servers, including the ThinkSystem line, optimized for AI workloads. Its collaboration with NVIDIA enhances its AI capabilities, making its hardware suitable for complex AI tasks, including traffic management.

Over the past few weeks, there has been a plethora of negative reports on GPUs/AI Hardware pointing to a mismatch between investment and usage/applications... But our boots-on-the-ground research indicates the opposite—we are about to experience a big step up in demand. Is it time for the first 1M GPU Cluster? When the H100 was introduced in’23, a large customer order would have been for 10-50K GPUs, with companies like Meta and Microsoft standing out at 150K GPUs. Our discussions with industry players (from power gen suppliers to chip manufacturers) indicate that the B100/B200 orders are coming in at 300K to potentially 1M GPU clusters. We will be hosting a Data Center Value Chain Part II Webinar to share our findings and we will cover the following: - What does a 1M GPU cluster imply for power demand? - Who will be the ultimate beneficiaries? - What are the use cases and where are the constraints today? Register here: https://lnkd.in/e_3-Mn5w With interest rates trickling up, investors have shortened their investment horizons and are only bidding up companies that are delivering #AI earnings today, creating very attractive entry points for people who can take a longer-term view. Loved sharing our thoughts on AI hardware, hyberscalers capex, AMD vs. Nvidia, and the broader opportunity set including #cybersecurity, with Charles Payne from Fox Business Network. Download our primer here: https://lnkd.in/eBk8Qv5v #technology #ai #datacenter

Under the heavy workload conditions, as seen during genAI training in AI clusters, GPUs typically have a mean time between failures (MTBF) of 20,000 to 30,000 hours. It means that, in a cluster of 24,000 GPUs, one GPU may fail every hour and 15 minutes. This figure does not even include the failures of network switches and interconnects. ? In addition, high-energy particles from cosmic rays and environmental neutrons interacting with silicon atoms could lead to bit flips (soft errors) in the Asics/HBM. Error-correcting code (ECC) protection can correct single-bit flips in the SRAMs. Most networking and GPU vendors these days provide close to 100% ECC protection for their SRAMs (as in Juniper's #Express5). ECC not only corrects single-bit flops caused by soft errors but also extends the device's lifespan by correcting single-bit errors in read data due to degraded SRAM cells. HBM vendors also offer ECC protection for memory contents. However, providing similar protection for flops inside logic is impractical - this can result in silent failures with no indication. ? How does training cope with these hardware errors? ?? The training state is typically checkpointed at regular intervals, usually at the boundaries of training iterations. The checkpoint frequency depends on the scale of the training cluster and the system's MTBF. In the 24K GPU scenario, check-pointing must happen at least once an hour. It involves copying most of the gigabytes of memory contents from all GPUs in the training cluster to the CPU and, from there, to the storage system, which slows down training and adds a heavy burden on the storage fabric. How do you reduce the overhead of checkpointing? ?? Some algorithms monitor training progress from the last checkpoint and use heuristic methods to decide between the cost of a new checkpoint versus retaining the previous checkpoint. Others use AI algorithms to monitor the health of GPUs, network elements, and links, triggering checkpoints if imminent failure is predicted. Asynchronous checkpointing allows training to continue while the state transfers from GPU to CPU. ? Regardless of checkpoint frequency, each hardware failure increases the effective training time due to lost progress and the additional time required to detect failures, replace hardware, reinitialize the GPUs with checkpoint state, and restart all the software containers. The more frequent the errors, the longer the training duration. ? Fault-tolerant implementations with error correction mechanisms such as ECC for memories, FEC and retries for interconnects, periodic scrubbing of memories, and, on a memory failure, restarting only affected training jobs instead of resetting the entire hardware are some of the techniques that allow systems to degrade performance gradually rather than fail suddenly... As cluster sizes continue to grow, designing resilient networking/GPU hardware is more critical than ever for sustainable training! #LLMs #GPUs #GenAI