PrismML Releases 'Ternary Bonsai 2 27B', a Ternary Quantized Model Maintaining 98.2% of FP16 Performance

PrismML has released Ternary Bonsai 2 27B, which converts existing FP16 weights into ternary values {-1, 0, +1}. The model maintains an average performance of 98.2% across 20 benchmarks compared to the original Qwen3.8-27B FP16 model and is distributed under the Apache License 2.0.

Key Technologies and Performance

• Compression Efficiency: Applied Hadamard rotation basis to minimize outlier variance, achieving 1.76 bits per weight via PTQ1_0 packing. According to the white paper, file size was reduced by approximately 9.1x from 53.80GB to 5.93GB.
• Performance Retention: Performance loss is minimized compared to the original model at xhigh inference intensity. Losses are low in math and coding domains, while instruction-following ability actually improved by 1.41 points. However, retention drops to approximately 75% for long-horizon agent tasks (such as Terminal-Bench).
• Hardware Optimization: Recorded a decoding speed of 27.7 tokens per second on Apple Silicon (M5 Pro), with GPU power consumption at approximately 27.0W.

Deployment and Constraints

• Execution Environment: Upstream llama.cpp does not yet support it, requiring a PrismML fork. MLX and WebGPU demos are also provided.
• Data Discrepancy: There are differences in file size and benchmark scores between the white paper and the Hugging Face model card. The white paper presents 5.93GB/83.9 points, while the model card presents 5.95GB/84.78 points. The actually uploaded file matches the model card figures.
• Inference Intensity Impact: The 98.2% retention rate is the value at xhigh inference intensity; at medium intensity, the retention rate drops to approximately 96.0%.