Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling

Existing generation length modeling has remained a coarse-grained approach targeting the entire sequence, making fine-grained control difficult. To address this, LenVM (Length Value Model) was introduced as a token-level framework that predicts the remaining generation length at each decoding step.

LenVM defines length modeling as a value estimation problem. By assigning a constant negative reward to each generated token, it is trained to predict the discounted return over the remaining generation span. This approach provides a dense, unbiased training signal without the need for separate annotation, and is easy to scale.

Experimental results demonstrate LenVM's outstanding performance on LLMs and VLMs:

• LIFEBench: When applied to a 7B model, accuracy dramatically improved from 30.9 to 64.8, surpassing closed-source models.
• GSM8K: Under a 200-token limit condition, it maintained 63% accuracy compared to the existing method's 6%, adjusting the balance between performance and efficiency.
• Interpretability: Token-level value values make it possible to grasp the dynamics of whether a specific token shortens or lengthens the reasoning process.