HBM is the Ugly Baby: What Now?
Innovations in memory architecture will save the day.
At the Raise Summit held in early July 2026, Pat Gelsinger called HBM a lousy memory and (“not to call it”) SK Hynix’s ugly baby, in the presence of Hoshik Kim (SVP and Fellow at SK Hynix) who was also on the panel. Kim then agreed with Gelsinger that perhaps HBM is not the end-game in memory technology, to which one of the panel members asked: “SK Hynix is actually admitting that HBM is a bad memory?” Awkwardness ensued.
Since the video’s release on Youtube late last week, shock clips have appeared on social media that take the whole conversation out of context. What actually follows in the whole 35 minute discussion is an insightful discussion into the future of memory, how it is an exciting time for memory innovation, where memory is going in the next few years, and what the industry is looking at today.
This article discusses these points in greater detail, providing necessary color where helpful so that the whole discussion can be viewed in a more helpful and positive context for the future of memory.
By reading this post, you agree to the terms and conditions. Also see the full ethics statement.
If you’re new, check out the About page. A lot of readers expense the subscription to this newsletter as it helps their professional work. Group subscriptions (3+) are 20% off. If you have any questions, reply to this email and let me know!
If you are not a paid subscriber, you can purchase just this article using the button below. You can find the whole catalog of articles for purchase at this link.
Also check out the Semi Doped podcast with and myself, and our daily free newsletter with latest semi news.
If you would like to engage independent research services for your project or need expert calls, check out my boutique consulting practice at SemiExponent.
The Three Fold Problem
Gelsinger points out that there are actually three major problems with HBM today: (1) insatiable demand, (2) larger capacity demand, and (3) greater memory bandwidth requirements.
When these three elements are put together, only one technology really emerges to satisfy the requirements of the industry: high bandwidth memory, or HBM. Stacking DRAM chips to make HBM ticks the higher capacity with high bandwidth boxes, while creating a massive appetite for memory. Today, AI is a memory-limited problem.
The consequence of this is that memory makers enjoy a near 90% margin, which Gelsinger points out is something that should never happen. Memory companies used to have one good year out of four as they usually built out capacity too quickly, and were left holding the bags in a downturn. Today, the memory cycle is looking strong several years into the run up, with no end in sight.
Unfortunately, the memory crunch is here for at least another 2-3 years, as Gelsinger points out, because capacity buildouts take time. He also points out that in that time frame, Apple may increase prices yet again. The consumer market will continue to be choked by AI demand in the near future. The hardest question to answer today is: When capacity does come online, will we still need all this memory?
Vijay Shilpiekandula from Dilation Capital makes an interesting point: token growth is exponential, while capacity buildout is linear. He mentions that memory intensity per token is on the rise - a framing that I particularly like. As context lengths continue to grow, and the user base (human and agent) increases, the amount of memory that the system has to support only increases, and from what we can tell today, there is demand inflection on the horizon.
HBM: The Ugly Baby
The said ugly baby was actually born out of a collaboration between AMD and SK Hynix as early as 2013, when they were trying to solve the memory bandwidth problem for supercomputing applications. HBM1 stacked four 1 GB DRAM chips for a total capacity of 4 GB, with a memory bandwidth of 128 GB/s. Today’s HBM4 stacks sixteen 4 GB dies for 64 GB capacity and a total bandwidth of 2TB/s.
The baby is all grown up now, but has become increasingly difficult to handle. Stacking to increase capacity causes thermal, yield, and capacity issues. DRAM chips generate heat, and it is becoming difficult to ensure that it is effectively removed. Yield is also becoming difficult as stack heights increase. Capacity-wise, HBM is incredibly inefficient because the bit-density of a DRAM die for HBM is 1/3rd of a standalone DRAM chip - a consequence of putting through-silicon-vias (TSVs) through the DRAM die. Making HBM now pulls all the capacity out of the DRAM supply chain because it takes 3x as many wafers to make the same HBM capacity, as you would with DRAM.
The problem gets worse with HBM5. JEDEC, the standard body that defines how high these stacks can physically be, has been relaxing the specs because keeping the physical dimension constrained would involve a shift to hybrid bonding - a technology still new to the memory industry at scale. Concept demonstrations for HBM5 from the big memory makers now include integrated cooling solutions co-packaged with HBM to extract heat. In addition, the base die that sits under the DRAM stack is going to a custom design that makes memory no longer a commodity, off-the-shelf product that is fungible between different designs.
The memory game has fundamentally changed, and Gelsinger’s main point is that the next few years will see the rise of new architectures and technologies if we were to fulfill our AI dreams, giving new life to a technology that has stopped scaling for the good part of a decade. It is the greatest time in history to be in the memory business.
After the paywall:
The Era of Memory Innovations
Bandwidth solutions (SRAM, 3D-DRAM)
Capacity solutions (SOCAMM/LPDDR, Flash)
Nobody knows how this memory cycle ends
