Memory Accounts for Over 63% of AI Accelerator Costs

The share of memory in AI accelerator costs rose from 52% in early 2024 to 63% by the end of 2025. Samsung Electronics disclosed this fact at Hot Chips 2026, citing data from Epoch AI, noting that the cost of remaining components, including the processor within the package, is currently about one-third of the total.

While compute speed is increasing by approximately 3x every two years, the supporting memory bandwidth has less than doubled over the same period. This has led to longer wait times for chips to receive data, and with memory accounting for two-thirds of costs, the industry is overturning its 50-year-old priority: reducing data movement is now considered more efficient than shrinking logic with the latest process nodes.

Memory manufacturers have begun moving from standard products to custom designs, and the spot price per GB of DRAM has risen approximately 7x. Wafer production capacity has stagnated for over 10 years, and new fab construction takes more than 2 years, so the top three companies are posting record revenues without an increase in supply volume. However, since memory prices themselves surged several-fold while the memory cost share rose by 11 percentage points, this implies an overall increase in total costs, with the costs of remaining components within the package also rising.

Samsung Electronics has applied technology that places multiplication operation units inside memory chips to prevent data from moving to the processor. In actual silicon tests, token generation speed for the Llama 3.1 8B model improved by approximately 3x. Additionally, Oxmiq proposed stacked flash memory for storing model weights, offering a method that increases capacity by up to approximately 14x while reducing bandwidth to half.