Same size, quarter the bytes: Qwen 3.5 vs Ternary Bonsai at 2B, 4B and 9B

The server now carries two model families at the same parameter counts: the Qwen 3.5 line it launched with, and the Ternary-Bonsai-2 small line — the same Qwen 3.5 trunks, post-trained by prism-ml into ternary {−1, 0, +1} weights at ~2.2 bits each. Same architecture, a quarter of the bytes. So: what do you actually give up, and what do you gain? We measured both families in the same session, on the same machine, with the page methodology (single-stream greedy, 3×256 tokens, Apple M4-class, nothing else running).

Size	Qwen 3.5	Ternary Bonsai	Advantage	Bonsai file
2B	42.2 tok/s	62.5 tok/s	+48%	0.52 GB
4B	23.4 tok/s	36.3 tok/s	+55%	1.15 GB
9B	13.8 tok/s	22.4 tok/s	+62%	2.42 GB

Single-stream decode is memory-bandwidth-bound: every generated token streams the whole weight set through the memory bus. Ternary weights are roughly a quarter of the bytes of the int8-packed Qwen weights, so the advantage should grow as models get bigger and more bandwidth-starved — and it does: +48% at 2B, +62% at 9B. The 9B bonsai decodes faster than the 4B Qwen while fitting an 8 GB host.

Neither prism-ml nor we publish a full eval suite for these yet, so we ran an honest spot check: 16 exact-answer questions — arithmetic, multi-step word problems, facts, extraction, logic — both families at their shipped defaults, temperature 0, answers graded by exact match. Sixteen questions is a smoke signal, not a benchmark; here is the smoke:

Size	Qwen 3.5	Ternary Bonsai (thinking)
2B	10/16	14/16
4B	15/16	15/16
9B	15/16	15/16

The 4B and 9B pairs tie. The interesting row is the 2B: the bonsai’s thinking mode walks through the multi-step arithmetic that the plain q35-2b fumbles (unit conversions, discount problems, digit-sums). At small scale, reasoning tokens substitute for parameters — and at 62.5 tok/s the thinking is cheap.

• Short token budgets read as quality loss. Our first pass capped responses at 400 tokens and the bonsais appeared to fail several questions — the responses were actually empty, because the budget ran out inside the reasoning block before any answer was emitted. At 800 tokens the real scores appeared. If you cap max_tokens below ~512 on a thinking-default model, you are measuring truncation, not intelligence.
• One prompt loops in thought at any budget. A terse extraction task ("answer with just the invoice number") sent all three bonsais into unbounded rumination — and the same request with "enable_thinking": false answers instantly and correctly. For extraction and other single-token-ish outputs, turn thinking off; for actual reasoning, leave the default on.

Full tables on the performance page; both families install from the catalog (bonsai-small fetch group, ~6.1 GB for all three with vision projectors). Same OpenAI- compatible API, served on CPU, no data leaves the box.