Samsung Electronics Unveils LPDDR5X-PIM at Hot Chips 2026, Boosting Llama 3.1 Inference Speed by 3x

Samsung Electronics unveiled a 16GB memory package featuring LPDDR5X-PIM (Processing-in-Memory) technology at Hot Chips 2026. This technology places compute units inside DRAM banks to resolve external interface bottlenecks and maximize the bandwidth required for LLM inference.

Core Technology and Performance

• Internal Bandwidth Innovation: While the external pin bandwidth of existing LPDDR5X is only 76.8 GB/s, PIM technology utilizes internal banks to provide 614 GB/s of internal bandwidth. This level is comparable to the total unified memory bandwidth of the Apple M5 Max.
• Inference Speed Improvement: Measured on the Llama 3.1 8B model (320-token context), execution time was reduced from 12.3 seconds to 5.4 seconds compared to standard LPDDR5X, and throughput improved by approximately 3x from 27 tok/s to 81.3 tok/s.
• Compute Precision: The package achieves 2.4 TOPS when using SINT4 weights and approximately 1.2 TFLOPS when using FP8.

Compatibility and Applicability

Samsung Electronics introduced Address Align Mode to ensure compatibility with existing memory controllers. This allows existing DRAM controllers to utilize PIM features, enabling switching between single-bank mode (standard DRAM) and multi-bank mode (PIM computation). This lays the groundwork for adopting PIM technology in servers, mobile devices, and client devices without developing new memory controllers.

Limitations and Implications

The benchmark is based on a single model and short context lengths, and does not consider bandwidth contention issues caused by KV cache growth. However, it demonstrates that memory bandwidth bottlenecks can be effectively resolved in the GEMV (matrix-vector multiplication)-centric LLM decoding phase, marking a significant milestone for AI infrastructure and on-device inference optimization.