AI BriefingHummingbird+ FPGA Inference
Key point
Hummingbird+ ran Qwen3-30B-A3B Q4 at 18 t/s on a board with 24GB and a mass-production price of about $150.
Details
Hummingbird+ is research that brings LLM inference to the edge-product stage on a custom FPGA board based on the Zynq UltraScale XCZU2CG/3EG.
The board is equipped with 24GB of memory and targets a BOM under $150 at mass-production scale.
The key result is that loading GPTQ 4-bit Qwen3-30B-A3B without additional compression achieved decode 18+ tok/s and prefill 50+ tok/s.
The implementation was achieved through the following optimizations.
- Miniaturized the GEMV engine and scalar engine to reduce resource usage.
- Improved computational efficiency through dual-precision operand packing and chain-tree mixing.
- Demonstrated the potential to deploy large MoE LLMs by combining 24GB of memory with low-cost FPGAs.
The key point is that, on a tokens-per-dollar basis, FPGAs could become a practical option for edge LLMs.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
Qwen3.5-based Small Decision Model 'Kev' Released with TypeSafe API Compatibility and Performance MetricsJohn Hennessy et al. Publish Paper on Intelligence per Watt (IPW) Efficiency of Local LLMs