FreeToken Enables High-Speed Execution of 290B MoE Models on Local PCs

FreeToken is an edge-native serving engine designed to run frontier MoE (Mixture-of-Experts) models of 290B+ scale locally on personal hardware such as gaming PCs. The tool treats heterogeneous edge resources—including GPUs, CPUs, and host memory—as a unified, elastic inference platform.

Key features include:

• High-speed edge-native runtime: Supports bandwidth-adaptive CPU-GPU co-execution, double-buffered prefill streaming, and global LRU expert caching.
• Semantic-aware caching: Provides semantic anchor checkpoints to prevent unnecessary recomputation during agentic context editing.
• Elastic memory management: Allows dynamic VRAM reallocation without engine restarts.
• Broad model support: Supports open-weight MoE models such as DeepSeek-V4-Flash and Qwen3.6-35B-A3B, along with various quantization formats (MXFP4, NVFP4, etc.), and offers Anthropic/OpenAI-compatible APIs.

It is designed to run on a wide range of consumer hardware, including NVIDIA RTX 30, 40, and 50 series GPUs.