Strata: Run a 125B Parameter Model at 60 Tokens/s on a Gaming PC

The Qwen3.8-Flash-Next model, which has 125B parameters and typically requires server-grade specifications, runs in a standard desktop environment. With an NVIDIA RTX 20 series or higher graphics card and 64GB of RAM, you can achieve fast inference speeds of 60–95 tokens per second without a server.

It bypasses memory limitations by intelligently distributing model weights between RAM and VRAM. It supports a 128K context window and maintains processing speeds of over 2,000 tokens per second even with long prompt inputs, making it suitable for document analysis and code review tasks.

The Coder version, a coding-specialized model, maintains 91% of the SWE-bench Verified score while reducing RAM requirements to around 32GB. In multi-GPU environments, distributing layers across cards can improve prompt processing speed by up to 20%.