AI BriefingUltra-Low-Bit AI Strikes Back
Key point
With 1.58-bit ternary weights, an 8B model scored 75.5 points.
Details
PrismML unveiled Ternary Bonsai. It's a language model applying 1.58-bit ternary weights { -1, 0, +1 } across the entire network, released in three sizes: 8B / 4B / 1.7B.
Memory usage is about 9x smaller than regular 16-bit models, with the 8B model running at 1.75GB. They emphasized that embeddings, attention, MLP, and even the LM head all use low-precision representation.
On performance, Ternary Bonsai 8B recorded an average of 75.5 points, 5 points higher than the same-line 1-bit Bonsai 8B (70.5 points). They stated that improvements were consistent across multiple benchmarks including MMLU Redux, MuSR, GSM8K, HumanEval+, IFEval, and BFCLv3.
Speed and power efficiency were also presented.
- M4 Pro: 82 toks/sec, 0.105 mWh/tok
- iPhone 17 Pro Max: 27 toks/sec, 0.132 mWh/tok
- About 5x faster than 16-bit 8B, with 3-4x improved energy efficiency
The model runs natively on Mac / iPhone / iPad via MLX, and the weights are released under an Apache 2.0 license. Detailed training and evaluation methods are provided in a whitepaper.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.
Related
2026 AI Inference Hardware Innovations: New Architectures for Prefill/Decode Disaggregation and Solving Memory Bottlenecks
Pinterest Reduces Costs by Up to 30% with SSD Serving and Quantization in Manas Embedding Search Platform