AI-Driven HBM Uptake Is Power-Sensitive

Find the Latest SoC Solutions
for... Automotive... IoT... Security... Audio... Video

High-bandwidth memory (HBM) has become the artificial-intelligence memory of choice, and as HBM3 goes into volume production, more attention is being paid to power consumption in the wake of 2023's generative AI boom.

www.eetimes.com/, Apr. 09, 2024 – 

Lou Ternullo, senior director of product marketing for silicon IP at Rambus, told EE Times in an exclusive interview that the increasing demand for memory bandwidth from AI is directly correlated to increasing HBM bandwidth. "Across the market, we have seen datasets and training models getting larger and larger, and 2023's GenAI boom only accelerated this," he said.

Performance needs, memory bandwidth and memory sizes are growing exponentially, Ternullo said, putting higher expectations and pressure on the next generation of HBM.

While bandwidth per watt as it relates to HBM is not particularly new, he said, energy consumption by data centers has been on the rise. "The huge investments and deployments into generative AI in 2023 have some predicting that data center electricity use will double by 2026."

click here to read more...

 Back