llama.cpp Proposes Dynamic MTP Optimization Feature

A Pull Request proposing the addition of an adaptive MTP (Multi-Token Prediction) mode to llama.cpp has been submitted. This feature utilizes a counting-based state machine to dynamically determine the appropriate MTP depth according to the model's output characteristics.

This allows the server to find the optimal value in real-time and optimize performance, eliminating the need for users to manually set the optimal MTP depth.

Key Performance and Features:

• Coding Tasks: Approximately 10~15% performance improvement during code generation, with speeds increasing by more than 50% when recalling code from the thinking phase.
• File Rewriting: When the model rewrites entire files in memory, it demonstrates speeds up to 100% faster compared to the existing MTP=3.
• Text Recall: Approximately 20~30% speed improvement during general sentence recall.
• General Sentence Generation: Performance may decrease by about 3% compared to the existing method when generating sentences that are difficult to predict, but the benefits in specific tasks such as coding are much greater.

Recommended Settings: MTP depth can be variably operated between 3 and 12 using the --spec-type draft-mtp-adaptive --spec-draft-n-max 12 options.