AI BriefingFuriosaAI Unveils AI Accelerator RNGD at Hot Chips
Key point
FuriosaAI unveiled RNGD, an AI accelerator for high-performance LLM inference, at Hot Chips 2024.
Details
FuriosaAI showcased its AI accelerator RNGD (Renegade), optimized for high-performance LLM and multimodal model inference, at Hot Chips 2024. FuriosaAI, founded by engineers from AMD, Qualcomm, and Samsung, successfully completed full bring-up of RNGD using silicon samples received from TSMC.
Initial test results demonstrated strong performance, recording throughput of 2,000-3,000 tokens per second on models such as Llama 3.1 and GPT-J at around 10B parameters.
The key innovations of RNGD are as follows:
- Tensor Contraction Processor (TCP) architecture: provides a perfect balance of efficiency, programmability, and performance.
- Outstanding power efficiency: achieves a TDP of 150W, showing very high energy efficiency compared to conventional GPUs that exceed 1,000W.
- High-performance memory: equipped with 48GB of HBM3 memory, enabling efficient operation of large-scale models on a single card.
Samples are currently being provided to early customers, with full-scale supply expected to begin from early 2025.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Underdog Releases 'Husky', a Model-Specific Inference Engine Up to 4.5x Faster than MLX