FreeToken: Efficient Edge-Native MoE Serving via Bandwidth-Adaptive Execution

While state-of-the-art open-weight models are increasingly being released, their serving methods still assume data center infrastructure. FreeToken is an edge-native MoE serving system that redefines personal devices not as simple small GPUs, but as flexible, integrated inference platforms.

This system is designed to address two realities of local AI: the continuously changing execution patterns of agent workloads and the varying balance of hardware resources across devices. Instead of sticking to fixed offloading strategies, it adopts a bandwidth-adaptive execution approach that continuously maps computation and model state to the resources actually available.

FreeToken co-designs the entire serving stack, including model layout and loading, expert residency, CPU-GPU execution, agent state reuse, and runtime memory management. This enables support for over 20 MoE models and real coding and tool-use agents on diverse hardware ranging from 8GB laptop GPUs to single workstation GPUs.

Key achievements include:

• Enabling the execution of 35B models on laptops, 284B models on gaming desktops, and 753B GLM-5.2 on a single workstation GPU.
• Transforming devices already owned by users into practical platforms for frontier-scale intelligence.

The system has been released at flashml.ai.