AI BriefingBringing HBM3 to RNGD: Challenges and Benefits
Key point
FuriosaAI adopted HBM3 in its next-generation AI chip RNGD to maximize inference performance for LLM and multimodal models.
Details
HBM3 is a technology that is extremely difficult to implement due to 2.5D packaging, specialized interposers, and complex thermal management challenges. However, for RNGD, designed for LLM and multimodal model inference, HBM3, which provides massive memory bandwidth, was an essential choice.
The FuriosaAI engineering team performed 3D electromagnetic (EM) simulation to precisely model the interposer for successful HBM3 integration. They also verified signal integrity through S-parameter and eye diagram simulations, and thoroughly checked power integrity through simulations across various load profiles.
Through the adoption of HBM3, RNGD secured the following key performance:
- Provides 1.5 TB/s of memory bandwidth
- Secures 48GB of memory capacity
- Achieves industry-leading bandwidth-per-power and capacity efficiency within 180W TDP
In particular, by utilizing 12-high HBM3, large models can now be run with fewer chips, and KV cache storage capacity has been increased, significantly improving throughput.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Qwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance Metrics