Moonshot AI Unveils Kimi K3 with 2.8T Parameters

Moonshot AI has released the weights for its most powerful model, Kimi K3. The model is designed as the world's first open 3T-class model, with a scale of 2.8T (2.8 trillion) parameters.

Key Features and Technical Specifications:

• New Architecture: Built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), the model maximizes efficiency through a Stable LatentMoE framework that activates 16 out of 896 experts.
• Multimodal and Long Context: It understands text, images, and video within the same model, and supports a 1 million token context window.
• Agentic Capabilities: It offers agentic features optimized for coding, knowledge work, complex reasoning, and visual tool use (CAD, game development, chip design, etc.).
• Model Scale: Of the 2.8T total parameters, active parameters number 104B, with 93 layers in total.

Kimi K3 is provided as open weights under the Kimi K3 License for research and deployment.