tandem.cpp v0.3.0 releases with 2-6x faster prompt processing and DeepSeek-V4.1 GGUF support
·2026.11.16 19:48
Key point
The update enables 4096-token prompt batches that borrow VRAM dynamically, significantly outperforming llama.cpp and Strata on mixed hardware.
Details
- tandem.cpp v0.3.0 is the first tagged release, featuring 2-6x faster prompt processing by running in 4096-token batches with dynamic VRAM borrowing.
- On an RTX 5070 Ti 16 GB with 64 GB RAM, prompt processing for Qwen3.8 IQ3_S (5.5k tokens) improved from 12.4s (444 tok/s) in previous tandem.cpp to 4.1-7.0s (790-1,340 tok/s), compared to 28.9s (191 tok/s) for llama.cpp.
- DeepSeek-V4.1 IQ2_XS (15.7k tokens) prompt processing dropped from 885s (18 tok/s) to 57s (276 tok/s) on the same hardware, though Strata remains faster for models exceeding RAM capacity.
- Project Maya's DeepSeek-V4.1 GGUFs are supported via a one-time conversion script, with tandem.cpp v0.3.0 achieving 60.3 tok/s decode speed versus Maya's 39.0 tok/s on an RTX 5090 32 GB setup.
- The software is currently Linux-only with AMD support lacking VRAM borrowing, while Windows prebuilt binaries are still planned.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.