FreeToken: Interactive inference of 290B MoE models on gaming PCs

An edge-native MoE serving engine that brings datacenter-grade model serving to personal hardware. It enables running frontier open-weight models of 290B parameters or more at interactive speeds on gaming PCs or workstations. It natively supports NVIDIA RTX 30, 40, and 50 series GPUs and operates in various environments, from consumer laptops to desktops.

The core is integrating the GPU, CPU, host memory, and interconnect into a single elastic inference platform. Efficiency is improved through bandwidth-adaptive CPU-GPU co-execution and global LRU expert caching. It dynamically reallocates VRAM between expert cache and KV memory at runtime, performing memory management without engine restarts or weight reloading.

It supports various frontier MoE models such as DeepSeek-V4-Flash, Qwen3.6-35B-A3B, and GLM-5.2. It handles multiple quantization formats including MXFP4, NVFP4, FP8, and BF16, and provides Anthropic and OpenAI compatible APIs. This allows for seamless integration with real coding and tool-calling agents such as Codex and Claude Code.

It features semantic anchor checkpoints for iterative states and KV caches, avoiding unnecessary recomputation during agentic context editing. It provides desktop apps and CLIs for Windows and Linux and is released under the Apache-2.0 license. It is suitable for developers who want to run large models quickly in local environments.