PrismML says Bonsai 27B runs on a phone

PrismML introduced Bonsai 27B, a multimodal flagship based on Qwen3.6 27B that it says is the first model in its capability class to run on a phone. The release targets multi-step reasoning, structured tool calls, vision tasks, and computer-use agentic loops, with low-bit weights applied across the language network and a compact 4-bit vision tower.

The model ships in two variants: Ternary Bonsai 27B uses ternary {−1, 0, +1} weights with FP16 group-wise scaling for 1.71 effective bits per weight and a 5.9 GB size, while 1-bit Bonsai 27B uses binary {−1, +1} weights for 1.125 effective bits per weight and a 3.9 GB footprint. PrismML says the 1-bit version fits within the memory budget of an iPhone 17 Pro, while the ternary version is aimed at everyday laptops.

Across a 15-benchmark suite, Ternary Bonsai 27B retains 95% of the full-precision baseline, and 1-bit Bonsai 27B retains 90%. Bonsai 27B also supports a full 262K-token context and speculative-decoding, with performance reaching up to 163 tok/s in 1-bit and 134 tok/s in Ternary on an NVIDIA GeForce RTX 5090, and up to 87 tok/s in 1-bit and 58 tok/s in Ternary on an M5 Max.

Bonsai 27B runs natively on Apple devices through MLX and on NVIDIA GPUs through CUDA, using custom low-bit kernels built for its hybrid-attention architecture. Model weights are available under the Apache 2.0 License, and PrismML is offering a free, limited-time developer preview API.