Why We're Joining UEC: Multi-Chip LLM Inference

As LLMs and multimodal models rapidly grow in size, there are increasing cases where they exceed the memory capacity of a single chip. Accordingly, multi-chip solutions are essential for scaling AI inference.

Multi-chip deployment uses various strategies such as data parallelism, tensor parallelism, and expert parallelism for MoE models. FuriosaAI's second-generation chip, RNGD (Renegade), efficiently leverages these various forms of parallelism through its TCP (Tensor Contraction Processor) architecture, which eliminates the need to decompose tensors into matrices.

The Ultra Ethernet Consortium (UEC) develops open standards for high-bandwidth, low-latency inter-chip communication. UEC offers higher bandwidth than existing PCIe, and its vendor-neutral approach provides the advantage of easily mixing different accelerators or scaling out to large multi-rack configurations.