AI BriefingLFM2.5 Inference Accelerated 3.2x with DSpark
·2026.08.21 01:52
Key point
Applying DSpark speculative decoding to LFM2.5 models improves GPU and on-device inference speed by up to 3.2x.
Details
DSpark draft model checkpoints have been released for the LFM2.5 model family (1.2B, 2.6B, 8B-A1B), significantly improving inference speed.
DSpark Technical Features
- Speculative decoding: A lightweight draft model generates candidate tokens while the target model verifies them, reducing memory load.
- Components: DFlash-style parallel backbone, Markov chain-based sequential head, and confidence-based verifier.
- Quality Preservation: Output results remain identical to previous versions during greedy decoding, with no change in benchmark accuracy.
Performance Improvement Metrics
- GPU (H100): Up to 3.18x throughput improvement.
- On-device (M4 Max): Up to 2.87x throughput improvement.
- Latency Reduction: For LFM2.5-2.6B, function-calling latency decreased by an average of 57%.
Compatibility and Deployment
- Day-one support provided for llama.cpp and SGLang.
- DSpark integration code is open-source and available for immediate use.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
KV Cache Research: Limitations of LRU Alternatives and Failure in Capacity-bound Environments
3.85x Inference Speedup via Diffusion Draft Generation and AR Verification Based on GLM-OCR