Introducing Ternary Bonsai: State-of-the-Art Intelligence at 1.58 Bits

Ternary Bonsai is a family of 1.58-bit language models designed to meet both memory constraints and accuracy requirements at the same time. PrismML stated that this model expands one step beyond the existing 1-bit Bonsai, meaningfully improving performance in exchange for accepting greater memory usage.

The model is offered in three sizes—8B, 4B, and 1.7B—and uses ternary weights {-1, 0, +1}. This reduces memory usage by roughly 9x compared to a typical 16-bit model, while aiming to outperform most competing models in each parameter class.

The key point is that the entire network consistently uses 1.58-bit representation. The embeddings, attention layers, MLPs, and even the LM head all use the same representation, with no higher-precision escape hatch provided.

Weights are processed using group-wise quantization, and each weight is constrained to one of {-s, 0, +s}. These three states are encoded as (-1, 0, +1), together with an FP16 scale factor (s) shared across every 128 weights.

In terms of performance, Ternary Bonsai 8B scores an average of 5 points higher than 1-bit Bonsai 8B, while using only 600MB more memory. Its average score is 75.5, placing it right after Qwen3 8B (16.38 GB) among the compared models while surpassing the rest, and it showed consistently competitive results across benchmarks including MMLU Redux, MuSR, GSM8K, HumanEval+, IFEval, and BFCLv3.

PrismML explains that this model is not meant to replace the existing 1-bit Bonsai, but rather to offer a different trade-off for environments that can accommodate greater memory usage. The 1.7B, 4B, and 8B versions allow memory, throughput, and quality to be flexibly adjusted according to deployment tiers.

Real-world performance was also emphasized. On M4 Pro, the 8B model achieved 82 toks/sec, and on iPhone 17 Pro Max, 27 toks/sec, which was presented as roughly 5x faster than an equivalent 16-bit model. Energy efficiency was reported at around 0.105 mWh/tok (M4 Pro) and 0.132 mWh/tok (iPhone 17 Pro Max), a 3-4x improvement over full-precision.

The model runs natively on Mac, iPhone, and iPad via MLX, and the weights have been released under the Apache 2.0 License. Detailed training, evaluation, and benchmarking methodology are documented in the whitepaper.