vLLM rope_scaling Override Causes 36% Model Configuration Mismatch

When running vLLM 0.30 with rope_scaling overridden, it was discovered that 64 models (36%) out of the 180 models with rope_scaling applied among the top 300 Hugging Face text generation models run with a different RoPE base without warning. This occurs because, following the structural change to rope_parameters in Transformers v5 which removed rope_theta, vLLM defaults to 10000 when no default value is present in the model file.

Performance Degradation and Verification

Actual performance degradation was confirmed in an RTX 4070 Ti environment, with Llama-3.2-3B-Instruct's GSM8K score dropping from 379 to 273 (approximately 28% decrease), and Qwen3-4B-Instruct-2507 dropping from 183 to 175 when YaRN was applied. Numerous similar configuration mismatches exist, such as the missing soft-capping in Gemma 2, but they are difficult to detect with existing evaluation metrics.

Release of the entail Library

To prevent such configuration mismatches, the open-source library entail was released to compare the declarations in model files with the actual execution values in the engine and correct them if necessary. Installable via pip install entail-ai, it achieved 100% output consistency, a loading cost increase of 0.7~0.9%, and 0 false positives across tests involving 38 models and 102 runs. Related issues have currently been reported upstream to vLLM and SGLang.