Why FuriosaAI's RNGD Deserves Your Attention

Running LLMs and multimodal models in production today comes with enormous costs and technical challenges. High-performance GPUs not only carry high upfront costs but also consume over 1,000W of power per card, driving up electricity bills and requiring complex liquid cooling systems, all while raising compatibility issues with existing server infrastructure.

FuriosaAI's RNGD (pronounced: Renegade) is an AI accelerator designed to solve these problems. It reliably runs models such as Llama 3.1 70B, offering the following key strengths:

• Efficiency: Achieves very low power consumption with a 180W TDP.
• Performance: Handles high-performance LLM workloads smoothly.
• Programmability: The compiler processes the entire model as a single set of fused operations, eliminating the need for tedious manual kernel optimization.

RNGD is built on a new chip architecture called the Tensor Contraction Processor (TCP) to balance efficiency, performance, and programmability. It also incorporates HBM3 and a 5nm process to enhance technical completeness.

This allows enterprises to lower their total cost of ownership (TCO) by reducing energy and infrastructure costs. It can also be deployed on-premises or in the cloud as easily as a standard CPU server, maximizing operational efficiency while securing sustainability through a reduced carbon footprint.