Bonsai 27B 1-bit Quantization Runs on iPhone at 3.9GB

Bonsai 27B, developed by PrismML, is a 2.7 billion parameter model based on Qwen 3.6-27B that has been extremely compressed using 1-bit quantization. It has been reduced from the original 54GB to 3.9GB, enabling local execution on an iPhone 15 Pro Max (8GB RAM).

Quantization Technique: True 1-bit quantization (Binary g128) was applied, encoding all weights as a single sign bit while sharing an FP16 scale for every group of 128. A notable feature is that embeddings, attention/MLP projections, and the LM head are all binarized.

Performance Retention: Across 15 benchmarks, the average score was 76.1 (89.5% retention compared to the original FP16's 85.1). The math benchmark score was excellent at 91.7, but knowledge/reasoning ability dropped to 73.4 (from the original 83.2). Memory usage is 5.2GB at 4K context, and 6.8GB at 100K context when using a 4-bit KV cache.

It has been released on Hugging Face and can be run on iPhone via Atomic Chat.