Qwen3.5 DFlash Running on RTX 2080 SUPER

This is a case of running llama.cpp's DFlash speculative decoding on an RTX 2080 SUPER 8GB with the Qwen3.5-35B-A3B combination. The target model was Q5_K_M and the draft model was Q4_K_M, tested with the CUDA backend and a DFlash PR-based build (67cb0d507 (8942)).

The main model GGUF was about 24.44 GiB, so it couldn't be loaded directly into 8GB VRAM, and MoE expert CPU offload was used together with DFlash. The top speed in a normal non-DFlash setup was about 26.8 tok/s, with the optimal value being -ncmoe 32 at that time.

With DFlash turned on, since the draft model also uses VRAM, the optimal -ncmoe shifted to 34. The draft model is about 267.8 MiB, with roughly 474M parameters.

The reproduction test was performed with the following key settings.

• --dflash
• --draft-max 6
• -cd 512 -c 4096
• --temp 0 --top-k 1 --seed 42
• -ngl 999 -ngld 99 -ncmoe 34