Memory has grown to nearly two-thirds of AI chip component costs

A new data analysis of the cost structure of AI hardware has highlighted a trend drawing attention in the sector. According to the analysis published by Epoch AI and widely noticed via Hacker News, memory components now make up nearly two-thirds of the component cost of AI chips.

This share shows how central memory has become in the economics of AI chips. Whereas the compute unit (the processor cores) was once thought to be the dominant part of the cost, the analysis points to memory's share rising markedly.

The core factor behind this shift is the intense need of modern AI workloads for high-bandwidth memory. Training and running large models requires processing vast amounts of data quickly, which makes advanced memory technologies a critical component.

Technologies such as HBM, known as high-bandwidth memory, are especially important in this equation. Because such memory is complex and costly to manufacture, it accounts for an ever-larger share of chips' total component spending.

The trend that Epoch AI's data points to is also meaningful for the AI hardware supply chain. Memory's rising weight in cost strengthens the strategic position of memory makers in the sector and makes the supply-demand balance even more critical.

This picture also aligns with earlier public discussions about cost pressures in AI hardware. The increase in memory demand is regarded as a dynamic that can put pressure on both prices and production capacity.

The view the data analysis offers focuses not on a single company but on the overall cost structure of AI hardware. For this reason, the findings give a sense of a structural trend across the sector rather than of any particular product.

Memory's rising share also directly affects the scaling costs of AI systems. As larger models and more intensive workloads increase the need for memory, the total cost of ownership of the hardware is also affected by this trend.

Data insights of this kind, published by organisations such as Epoch AI, offer useful references for tracking the rapidly changing economics of AI hardware. Making numerical trends visible makes it easier to understand the direction of transformation in the sector.

This article is a sector analysis and should not be regarded as investment advice. Decisions about AI hardware and the semiconductor market are advised to be approached with an independent and careful assessment.