llama.cpp Adds MTP Support for Mellum2.1 MoE Model

llama.cpp has merged support for Mellum2.1, the latest iteration of the Mellum2 Thinking model, specifically adding Multi-Token Prediction (MTP) capabilities. The model retains the previous architecture as a 12B mixture-of-experts (MoE) system with 2.5B active parameters, but shifts focus heavily toward post-training via reinforcement learning.

Model Architecture and Specs

Mellum2.1 Thinking is defined by the following technical specifications:

• Parameters: 12B total, 2.5B active
• Structure: 28 layers, 64 experts with 8 activated per token
• Context: 131,072 tokens with a 1,024 sliding window
• Precision: bfloat16
• License: Apache 2.0

Training and Capabilities

The primary development effort for this version focused on scaling reinforcement learning (RL) from a final stage to the main component of training. This involved:

• Enhanced Data Filtering: New RL tasks in math, competitive programming, science, and tool use, combining open datasets with custom tasks.
• Agentic Skill Development: Training occurred in real software engineering environments where the model used shell and file-editing tools, receiving rewards for passing tests.
• Improved Autonomy: As a result of millions of sandboxed training runs, Mellum2.1 demonstrates superior performance in agentic tasks, including codebase exploration and self-verification of edits.