A 27 billion parameter model is now running on an iPhone

A 27 billion parameter model, in its full version, weighs about 54 GB. Impossible to load on a phone, even difficult on a decent Mac. PrismML has just released Bonsai 27B, a compressed version of the same model that fits into 3.9 GB and runs directly on an iPhone. Not a makeshift lab demo: a model that can be downloaded starting today, for free, under a license that even allows commercial use (Apache 2.0).

PrismML starts from Qwen3.6 27B, a model from the Chinese Alibaba, and compresses it into two lighter versions. The ternary version (each weight of the model, one of the numbers that define what it "knows", is now only -1, 0, or +1 instead of a floating-point number) weighs 5.9 GB and targets laptops. The 1-bit version, even more radical (the weights are now only -1 or +1), weighs 3.9 GB and targets phones. Here’s what it looks like on paper.

The technique: no need to retrain everything

What makes this kind of announcement interesting is rarely the compression figure, it’s the method used to achieve it. BitNet, Microsoft's most well-known project on ternary weights, retrains a model from scratch with this constraint imposed from the start. This is very costly in computation, and it means starting from scratch with each new model base.

Bonsai does it differently: a quantization (the technical term for "reducing the precision of the numbers that make up the model") applied afterwards, on a model already trained, with a calibration setting shared by small groups of 128 weights. In other words, we take the already ready Qwen3.6 27B, and compress it in small batches without going through a complete retraining. On paper, such brutal compression without retraining should yield a model that is half-baked. Personally, this figure is what caught my attention: the ternary version retains 94.6% of the original model's score on the 15 tests that PrismML uses to compare its models, and even the 1-bit version retains 89.5%. In detail, it degrades more on certain tasks: in code, the 1-bit drops to 81.9 compared to 88.7 for the full model; on tasks where the model must use tools on its own, the gap widens further, 66 versus 80. Nothing unreadable, but that’s where it stings the most if you were counting on Bonsai to run an agent autonomously.

What it looks like in real use

On an iPhone 17 Pro Max, PrismML reports about 11 tokens per second in generation with the 1-bit version (a token is the small piece of word that the model produces at each step, roughly three-quarters of a word). On a Mac M5 Max, the ternary version climbs to around 87 tokens per second according to their own figures. On an RTX 5090 graphics card, we reach 163 tokens per second. The context window, meaning the amount of text that the model can read and keep in mind at once, goes up to 262,000 tokens: plenty to read an entire code file without forgetting the beginning.

A small note of caution, because it’s PrismML measuring their own figures: independent tests published by MarkTechPost give much more modest throughput on Mac, around 26 tokens per second in ternary. As often with a launch, the figure from the official presentation is the best case, not the average. Before building a project on it, test it yourself on your hardware.

Why this matters for a developer

The interest is not in having one more chatbot in your pocket. It's in having a model that runs directly on the device, without an internet connection, without an API bill at the end of the month, with enough memory for real agent tasks: reading a project, calling tools, reasoning in multiple steps. Everything that, until now, only ran in the cloud because no model of this size could fit on a normal device.

The model can be retrieved from Hugging Face (the platform where most open AI models are hosted) and on GitHub, or directly via the iOS app Locally AI for those who just want to try it without setting up a whole home inference system. Apache License 2.0, so no restrictions for commercial use.

I remain cautious about the 90% retention until I have run it myself on a real code project, but the idea of having a model of this class that responds directly on the device, without network latency and without your prompts going to anyone else, is clearly worth it.