The Memory Bottleneck Reshaping The AI Race

Welcome to Memorandum Deep Dives. In this series, we go beyond the headlines to examine the decisions shaping our digital future. 🗞️

This week, we look at a quiet shift that is rewiring the economics of the AI boom. For years, the entire industry fixated on a single prize, and the companies that locked it in early gained a measurable head start. But that prize is no longer the constraint it once was.

Something stranger is now happening behind the scenes. The world's largest technology firms, sitting on hundreds of billions in capital, are lining up to make an unusual offer to a small group of suppliers, and those suppliers are in no hurry to say yes.

The reasons why reveal a great deal about where the real leverage in AI now sits, and why the next phase of the boom may be decided less by who builds the smartest models and more by who controls something far less glamorous.

The bottleneck hiding in plain sight

Whenever energy or information must continuously flow through a system, its overall performance depends on how efficiently that flow is maintained. An internal combustion engine, for instance, is ultimately limited not just by its power, but by how effectively fuel reaches the combustion chamber. If fuel injectors become scarce and difficult to manufacture, they quickly turn into a bottleneck for the entire industry.

Something similar is now happening in AI. For most of the AI boom, the race centered on securing NVIDIA GPUs, with companies spending billions to lock in computing power. But by late 2025, a quieter shift began reshaping the semiconductor industry, one that is now pushing the world’s largest technology firms to do something unprecedented: invest directly in memory chip production lines.

According to Reuters, SK Hynix, the South Korean company that controls roughly 57% of the global market for high-bandwidth memory (HBM, the specialized stacked memory chips that feed data to AI accelerators), has been flooded with proposals from major technology firms.

The offers go far beyond ordinary supply agreements. Hyperscalers are reportedly now offering to finance dedicated production lines and even help pay for ASML’s extreme ultraviolet lithography systems.

Collectively, three companies, Alphabet, Meta, and Microsoft, have raised their 2026 capital expenditure guidance to nearly $725B, with Microsoft alone saying that roughly $25B of its AI infrastructure spending is being absorbed by rising memory and chip costs.

The financial impact of that scramble is already becoming visible. SK Hynix is now approaching a $1T valuation, only weeks after Samsung crossed that milestone, as investors increasingly view memory suppliers as among the most strategically important companies in the AI economy.

From GPU shortage to memory bottleneck

However, beyond the market valuations and the massive investment opportunities lies a bigger problem: one that could take years to resolve.

High-bandwidth memory, or HBM, has become one of the most critical bottlenecks in AI infrastructure because it is essential for training and running advanced AI models. And as AI adoption continues to grow, demand for HBM will grow with it. According to TrendForce, the demand for memory chips is projected to increase by another 70% year over year in 2026.

The reason for this growth is that AI systems are becoming increasingly memory-intensive, as modern LLMs must rapidly move enormous amounts of data between processors during both training and inference. Every increase in model size, context window, multimodal capability, or real-time reasoning raises the amount of data that must remain instantly accessible to AI accelerators. HBM addresses this by physically sitting closer to GPUs and delivering data at extremely high speeds, keeping processors fully utilized rather than waiting for data to arrive.

The problem is that AI workloads are increasing memory requirements faster than manufacturing can realistically keep up with. Training frontier models now involves trillions of parameters distributed across thousands of accelerators, while inference workloads are also becoming more memory-intensive as companies deploy AI systems to millions of users in real time. Features such as persistent memory, longer conversational context, AI agents, video generation, and multimodal reasoning all require larger memory footprints, increasing pressure not only on processors but also on the memory systems that feed them.

Why the supply chain cannot scale fast enough

But while the demand for memory chips is increasing, manufacturing is not keeping pace. The gap between this demand and supply stems from the fact that manufacturing memory chips is extraordinarily resource-intensive. Producing a single bit of HBM requires roughly three times the wafer capacity needed for standard DDR5 memory, meaning supply cannot expand quickly even when companies are willing to spend aggressively.

At the same time, the industry’s three dominant HBM suppliers, SK Hynix, Samsung, and Micron, have reportedly already committed nearly all of their 2026 production capacity to existing customers. The problem is that expanding supply in advanced semiconductor manufacturing is an extremely slow, infrastructure-intensive process. Building new packaging plants takes years; installing and calibrating fabrication equipment takes additional time; and some of the most critical tools, including ASML’s EUV lithography machines, already face long delivery backlogs due to high global demand.

SK Hynix’s $12.9B advanced packaging facility in Cheongju, for instance, is not expected to begin operations until late 2027, while its large EUV equipment order will only be fully delivered by the end of that year.

This explains why hyperscalers continue facing shortages despite spending close to $190B annually on AI infrastructure.

AI demand is expanding far faster than the global chip supply chain can realistically scale. Which brings us to the second half of the equation: HBM manufacturers’ response to Big Tech’s offers.

Why chipmakers hold the leverage

SK Hynix, according to sources cited by the Korea Times, is cautious about accepting Big Tech’s offer, and that caution is rational. The company posted record quarterly revenue of 52.6T KRW ($35.5B) in Q1 2026, with operating margins near 47%. According to Reuters, available capacity is essentially zero right now, with nothing that can be designated for a specific customer.

For memory chip manufacturers, accepting customer capital would mean tying production to specific buyers, potentially at negotiated prices below what the open market would bear. This makes little sense from the perspective of a company that has already sold its product at record margins and does not want to be encumbered by capital worth less than flexibility.

Additionally, the memory chip industry carries deep institutional memory of its own boom-and-bust cycles, in which peak-to-trough price declines have historically reached 50 to 70%. SK Hynix posted multiple quarters of losses in 2022 and 2023.

With that in mind, locking into long-term commitments with floor prices and dedicated capacity can be stabilizing during a shortage, but it becomes a constraint when the cycle turns. At the same time, the broader contract landscape is also shifting in favor of suppliers. Samsung and SK Hynix have begun moving away from fixed-price annual contracts toward shorter deals with post-settlement pricing, where final payments adjust to reflect market conditions at delivery.

According to TrendForce, chipmakers are experimenting with three-to five-year agreements featuring minimum price floors and upfront payments worth 10–30% of total deal value. The structure allows suppliers to preserve the upside of the current seller’s market while protecting themselves if the cycle turns.

Meanwhile, whether SK Hynix accepts these proposals or not, the scarcity is already reshaping markets beyond AI. IDC describes the shift as a strategic reallocation of wafer capacity, in which each HBM wafer reduces the supply of consumer DRAM.

Companies like Apple have warned that memory shortages could compress iPhone margins, while Elon Musk said Tesla had hit a chip wall. With DRAM prices expected to rise more than 70% in 2026, the broader electronics industry is increasingly absorbing the cost of a supply chain redirected toward AI infrastructure.

Memory as the next chokepoint

Add to this already complicated mix the knowledge that many large technology companies are trying to reduce their reliance on NVIDIA by building their own AI chips, and the dependency becomes even deeper.

Currently, Amazon has Trainium3, Google uses its TPUs (tensor processing units), and Meta has developed its MTIA inference chips. But all of these processors still depend on high-bandwidth memory, or HBM. This means that Goldman Sachs forecasts demand for HBM from custom AI chips will grow 82% in 2026 and could eventually make up a third of the total HBM market. Companies moving away from NVIDIA are not actually reducing their dependence on the broader AI supply chain; in some cases, they may be increasing it, since every new chip design requires its own testing, certification, and supply agreements with memory manufacturers.

Then there is the geopolitical layer that makes this more than a procurement story. South Korea sources nearly half of its rare earth inputs from China, and U.S. export controls now require SK Hynix and Samsung to obtain individual licenses for American equipment used in their facilities in China.

The investment offers from American technology firms can be read, in part, as a way to ensure memory production stays within allied nations as the global chip supply chain splits along geopolitical lines. SK Hynix’s $3.87B packaging plant in Indiana and its massive Yongin complex in South Korea both sit firmly within that allied perimeter.

For technology executives watching this story, the pattern should feel familiar.

A few years ago, the companies that secured early access to NVIDIA's H100 GPUs gained a measurable head start in building AI services. UBS now projects SK Hynix will capture approximately 70% of the HBM4 market for NVIDIA’s next-generation Rubin platform, and SK Group’s chairman has suggested that AI-driven memory demand pressure may persist toward 2030.

Which brings us back to the dilemma of the engine and the fuel injector. As AI labs continue to push toward more powerful models and demand for AI services grows, the underlying infrastructure struggles to keep pace. The constraint is no longer the machine’s power, but the system that feeds it data.