Strata Engine Released to Run Qwen3.8-Flash-Next 125B on 12GB VRAM

Developer Niko1221 released Strata, a dedicated inference engine for the Qwen3.8-Flash-Next (125B MoE) model. Instead of a general-purpose engine, it utilizes GPU, CPU, RAM, and SSD resources simultaneously, allowing execution in environments with 12GB VRAM and 32GB RAM.

Key Technologies and Performance

• Resource Distribution: Common parts and frequently used experts are placed in VRAM, while remaining experts are placed in RAM for parallel computation by the CPU. N-gram embedding tables are stored on the SSD.
• Speed Improvement: Generation speed is approximately 4.6x faster than llama.cpp v0.6.0 (based on RTX 3060), with significantly reduced time-to-first-token for 32K prompts.
• Optimization Techniques: Speculative decoding utilizing the model's built-in MTP layers increases generation speed by 1.6–1.8x, and Tail Skip (omitting lower experts) achieves additional acceleration.

Limitations and License

• Single Model Only: Supports only Qwen3.8-Flash-Next and its derivative versions; other models cannot be run.
• Lack of Concurrency: Default settings are optimized for processing 1 request, so performance degradation may occur with concurrent requests.
• License: The engine is under the MIT License, but the model files follow the Qwen Community License 1.0, requiring separate license verification for commercial service provision.