AI Briefing
KOSign in

Edge0-145B: 145B-Parameter Open-Source Model Runs on Laptops at 80+ Tokens/s

·2026.11.13 03:00

Key point

The model achieves 80+ tokens/s decoding and 29GB peak memory usage on compatible laptops, outperforming several cloud models on coding and reasoning benchmarks.

1 / 6

Details

  • Edge0-145B is a 145B-parameter open-source model designed for on-device inference, achieving 80+ tokens/s decoding and 1,100+ tokens/s prefill with 29GB peak memory on laptops.
  • The model uses a sparse expert architecture where only ~10B parameters are active per inference step, enabling efficient execution on consumer hardware like MacBook Pro and NVIDIA RTX 4090/5090.
  • Benchmarks show Edge0-145B scoring 64.6 on DeepSWE 1.1 and 90.4 on GPQA-Diamond, outperforming or matching models like Qwen3.8-Flash-Next and DeepSeek-V4-Flash-0731.
  • The release includes the Edge0-AI inference framework on GitHub and model weights on Hugging Face, supporting macOS, Windows, and Linux with one-click deployment tools.
  • The Edge0 ecosystem includes four model families (Edge0, Audio8, Decision0, Embedding0) with over 200 models released in the past year, covering agent tasks, audio, decision-making, and embeddings.
  • Hardware optimization targets NPU architectures and numeric formats (INT4, INT8, FP4), currently covering 95% of mainstream chip platforms including Qualcomm, NVIDIA, and Rockchip.
local-inferenceopen-source-llmon-device-aiinference-optimizationedge0-145bsparse-experts
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

RedditXHToken releases Spark-X2.6-4B with improved agentic and coding capabilitiesPyTorchKRFlashStates Improves GLM-5.3-Flash Prefill Throughput by 3.14xHacker NewsNew Arm NEON kernel speeds up llama.cpp prompt processing by up to 6.5xPyTorchKRArete-27B reduces Chinese government narrative adoption rate from 22% to 0.9% using SFT onlyRedditKittenML releases inferencekit to optimize LLM inference on local hardware via coding agentsTLDR AILiquid AI releases LiquidCodec, a 74M parameter audio codec optimized for on-device speed