Small models on your own device
A 2GB model file: how do you choose local AI for your device?
Bonsai 2 adds three small models and a phone-app route. Compare files, vision components, compatible software and vendor speed examples to understand what local use actually requires.
PrismML expanded Bonsai 2 on October 15 with Qwen3.5-based 2B, 4B and 9B models. We checked original model documentation and the current app entry, without running the weights on a phone or computer.
Which model produces 120 tokens per second on a phone?
PrismML reports up to 120 generated tokens/s for 2B on an iPhone 18 Pro Max, versus 34 for 9B. These offer different local choices. The smaller model's speed does not describe 9B, and token generation is only part of the time spent reading an image, reasoning and answering.
Ternary language weights use three values and scaling factors to reduce stored and transferred data. Smaller files offer another route to local chat, vision and tool integration. Whether a model explains your screenshot usefully remains a task question, beyond bit counts and a throughput chart.
Sources and further reading
- PrismML · Bonsai 2 small-model releaseOfficial release · October 15, 2026New 2B/4B/9B models, phone throughput examples and the source chart; vendor tests differ from device experience.
What needs space beyond the 2GB file?
The 9B card lists a minimum 2.00GB language file and a default PQ2_0 pack of 2.42GB. Image input adds a recommended 0.62GB Q8_0 vision component. Context, image processing and the application also need memory. Two free gigabytes therefore do not establish that a device can run 9B.
The table describes files, not peak-memory tests. Its MLX column includes vision, while the GGUF columns do not. The models' listed 262K context ceiling is also not a recommendation to allocate the largest possible window on a phone.
| Model | Minimum GGUF PTQ1_0 | Default GGUF PQ2_0 | MLX including vision |
|---|---|---|---|
| Bonsai 2 · 2B | 0.43 | 0.52 | 1.34 |
| Bonsai 2 · 4B | 0.95 | 1.15 | 2.05 |
| Bonsai 2 · 9B | 2.00 | 2.42 | 3.46 |
Sources and further reading
- Bonsai · Model lineup and file contentsOfficial documentation · Checked October 16Separates language-only GGUF from MLX including vision; file size is not peak runtime memory.
- PrismML · Original 9B model cardAuthors' model documentation · Formats and components2.00/2.42GB language files, the 0.62GB vision component and alternative packings of the same weights.
The software needs to understand the file
The GGUF packings can represent the same weights while requiring different software. Current official documentation requires PrismML's llama.cpp fork for PTQ1_0 and PQ2_0; upstream Q2_0 support starts at b11555. Older builds may load Q2_0 without the required transform and produce gibberish. A successful load alone is not a usable result.
Apple Silicon also has an MLX route, documented for mlx-vlm 0.7.8 or later rather than the unsupported released mlx-lm. Choose compatible software and files together, instead of sorting downloads only by size.
Sources and further reading
- PrismML · Original 9B model cardAuthors' model documentation · Formats and components2.00/2.42GB language files, the 0.62GB vision component and alternative packings of the same weights.
- Bonsai · Formats and runtime supportOfficial compatibility documentation · Current versionThe b11555+ requirement for Q2_0, the PrismML fork for the other formats and the MLX route; an old README is not current compatibility evidence.
An app route and a small first task
Apple's October 16 public response lists Locally AI by LM Studio 1.66.0 in the U.S. store, with new Ternary Bonsai 2 models, a free download and iOS 18.1 minimum. The China query returned no listing. This establishes an app route without compiling a runtime, not that every compatible iPhone can load 9B. We did not install or run it.
An editorial starting point is one screenshot or passage you can verify: check the answer, full waiting time and comfort during sustained use. This is a selection suggestion, not our test result. The earlier Gemma scam-detection project integrates a small model into a specific message task. Bonsai offers another set of local choices; neither inherits the other's application results.
- See the U.S. Locally app and version
- Choose an official runtime and format
- Continue to a small model checking suspicious chats
Sources and further reading
- Apple · Locally in the U.S. App StorePublished app entry · Version 1.66.0The release notes add the new Ternary Bonsai 2 models; store availability is not a BitShovel device test.
- Apple · Current version responseOfficial public endpoint · Read October 16Version 1.66.0 released October 15 at 20:06:43 UTC, free in the U.S. store with iOS 18.1 minimum; the China query returned no listing.