Cloudflare fixes cross-request KV cache exposure in GLM-5.3 Workers AI model

Cross-request KV cache exposure in the GLM-5.3 Workers AI model affected 155 responses between August 27 and September 9, 2026, where responses to prompts exceeding 681,280 tokens contained data from other customers' sessions.
The issue resulted from a prefill KV cache pool limit of 681,280 tokens being smaller than the 1,048,576-token context window, a mismatch Cloudflare was aware of but did not validate, combined with an SGLang bug that allowed rejected requests to proceed with stale cache pages.
Cloudflare remediated the issue by expanding the prefill KV pool to approximately 2.04 million tokens and patching SGLang to discard rejected requests in the admission queue before KV transfer.
The fix was released in SGLang v0.5.20, with follow-ups in v0.5.21 and v0.5.22; operators are advised to upgrade to v0.5.22 or later and verify KV pool size against context length.

Root Cause and Fix

SGLang's disaggregated prefill/decode mode failed to reject requests exceeding the KV pool; instead, it substituted a 1-token placeholder and proceeded with prefill.
Decode nodes incorrectly assumed previous request KV cache was overwritten, reading stale data from 7 of 8 pages while returning HTTP 200.
The fix ensures rejected requests are discarded with their original error in the admission queue, preventing bootstrap, allocation, or KV transfer.

Operators must compare KV pool size and context length in engine startup logs and lower context length if the pool is smaller.
Upgrade to SGLang v0.5.22 or later to ensure requests exceeding pool size are rejected before handoff.
Enable deferred KV release on both prefill and decode roles and set limits on Mooncake transfer batches to prevent unbounded delayed writes.