Qwen3.5-4B Prompt Processing Speed Improved Up to 3.7x via Model Grafting

The Model Grafting technique, which retroactively applies the causal encoder-decoder architecture originally trained from scratch in DeepSeek-V4.1-Flash to existing models, has been released. This method involves cutting the model at a specific depth so that lower layers read the prompt and upper layers use the encoder's residual stream as prefix KV, connecting them via an identity-init adapter, and then recovering the model through self-distillation from the unmodified parent model. The decoding part retains its original structure.

Two variant models applying this technique to Qwen3.5-4B have been released on Hugging Face.

• graft8 variant: Achieved approximately 3.7x speed improvement at a 128K prompt length, but incurred some accuracy loss.
• graft16 variant: Improved prompt processing speed by 2.0x while maintaining performance similar to the parent model, with accuracy loss minimized.