Taalas Demonstrates 17,000 tok/s Inference for 8B Llama Model on Single Custom Chip

·2026.10.21 21:30

Key point

The experimental proof of concept burns the model into silicon, achieving speeds 213x faster than an RTX 5070.

Taalas developed an experimental chip that runs an 8B Llama model at 17,000 tokens per second.
The speed is 213x faster than the same model running on an RTX 5070, which averages 80 tokens per second.
The technology works by burning the model directly into silicon, making the LLM the chip itself rather than loading it onto hardware.
The approach is currently a proof of concept, not a commercial product, with a key limitation being that the model cannot be swapped once burned onto the chip.