How HBM-PIM is Making Hyperscale AIs More Efficient

Hyperscale AIs like ChatGPT have taken the world by storm with their human-like ability to answer questions, hold conversations, and even compose music and computer programs. But behind that miraculous facade, a huge amount of memory-intensive data calculations are needed.

??

To solve the exponentially increasing demands that Hyperscale AIs make on traditional memory solutions, Samsung Electronics has integrated an AI-dedicated semiconductor into its high-bandwidth memory (HBM). Called Processing-in-Memory (PIM), our technology allows us to implement processors right into the DRAM, reducing data movement and improving the energy and data efficiency of AI accelerator systems by offloading some of the data calculation work from the processor onto the memory itself.

??

One graphics processing unit (GPU) boasting Samsung HBM-PIM memory is?a?testing?variant of?the?Instinct?MI100 card made by @AMD. By applying our HBM-PIM technology to large-scale AI and HPC applications,?Samsung has seen the potential for doubling?the performance of the GPU accelerator while reducing energy consumption.

??

AMD talked about this in the ISSCC 2023 conference in February where they said, “From a systems standpoint, we want to be as efficient as possible and with our partner Samsung and their PIM technology, we have seen up to an 85% savings in data movement energy within memory processing, and this is a promising technology for optimization in future systems.”

??

Because we know the importance of Hyperscale AIs like ChatGPT will only grow through the 21st century, #SamsungSemiconductor is dedicated to continuing to develop next-gen technologies for them, including HBM-PIM, as well as releasing the necessary software and simulators to support them.