ik_llama.cpp Merges Qwen3.8 MTP Support

Qwen3.8-Flash-Next model support for MTP (Multi-Token Prediction) has been officially merged into the ik_llama.cpp main branch. This introduces functionality that previously existed only in unofficial forks, making it usable without separate patches or forks.

Performance Improvements

Loading the MTP head (2.6B parameters) allows the model to generate the next token as a draft from the hidden state and then verify it. This significantly improves inference speed while keeping the output identical to the original model.

• RTX 5090 + 128GB DDR5: Token generation speed for coding tasks increased from 45 to 90 tokens per second, approximately 2x faster.
• RTX Pro 6000: Speed improved from 85 to 113 for coding tasks and from 82 to 109 for extraction tasks.
• RTX 4070 (12GB): Improved from 9.5 to 12.5 for coding tasks, but gains may be negligible or negative for story generation tasks.

Caveats and Limitations

• Variation by Task Type: Draft acceptance rates are high (93-99%) for coding tasks but lower (60-65%) for prose generation, limiting speed improvements.
• Configuration Options: Currently supported only in single-slot mode (-np 1). Using the --jinja template enables reasoning mode, which treats output like prose and may lower acceptance rates.
• Hardware Compatibility: Performance may degrade on some low-spec cards or with specific fine-tuned model combinations, requiring testing based on hardware and model combinations.