Qwen3.6 Crushes the Competition

A positional recall benchmark was run using a large service.js file of 336K size, roughly 108,000 tokens.

The test fed the entire file content at once, then asked the model to restore the 20 lines following the opening brace of a specific function. The target file contained 8,000+ lines and 1,300 functions, with the KV cache set to 8-bit.

Results varied significantly by model.

• Gemma 4 27B: only 6 out of 16 prompts passed the quality bar
• Other Gemma variants were worse, with some passing only 2
• Qwen 3.5 35B produced cleaner output than Gemma
• Qwen 3.6 35B was rated as delivering the best results on this large-scale JS reverse-engineering task

The core issue was Gemma 4's sliding window attention and its 1K look-back window limitation, which caused it to miss parts of function bodies and return statements—or hallucinate—over long context. In contrast, the Qwen models restored function content more reliably under the same conditions.