AI Briefing
KOSign in

Lafa Achieves 2.54x Speedup with Flow-Based Generation

·2026.10.23 10:58

Key point

Combining block-level Flow generation with AR verification increased throughput by 2.54x.

1 / 3

Details

  • Researchers from UC Berkeley, Amazon, KAIST, and CMU released Lafa, a Flow-based language model
  • Achieved 2.54x higher throughput compared to AR SFT 7B with a block size of 16 on a single H100 (100.2 tokens/s vs 39.5 tokens/s)
  • Uses Flow Matching to generate tokens within a block simultaneously and Self-speculative Decoding to adopt only the parts matching AR predictions
  • Official checkpoints based on Qwen2.5 0.5B/7B Instruct and Qwen3 8B Base have been released in the official repository
  • Currently supports only Greedy decoding; Sampling and Beam search are not supported
qwenspeculative-decodinginference-optimizationarxivflow-matchinglafa
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditSpecter Achieves 1.50x Speedup in MoE Inference via Quantization-Enabled Speculative OffloadingRedditResearch Shows Frozen 16 MB Vocabulary Outperforms Learned Embeddings in Small Language ModelsHacker NewsHybrid Reasoning Agents achieve 35-55% token reduction and up to 2.6x faster inferencePyTorchKRJeff-Code Reduces Qwen3.8-27B Coding Time by 32%NVIDIA BlogShopify Cuts Inference Costs by 90% with NVIDIA Open Models as European Retailers Adopt Physical AIHuggingFace BlogTorchNeuron Profiling on AWS Trainium: System-Level Analysis and Fusion Benefits