A calibrated decision model, 100% local: strands-decider on /v1/systemone

Every automated workflow eventually hits the same wall: the model gave an answer — is it sure enough to act on? Ordinary LLM output doesn't answer that. Even probability-style endpoints (including our own /v1/score) report relative preferences: great for picking the best option, misleading if you treat 0.9 as "right 90% of the time".

v1.5.0 adds a model built specifically for that wall. strands-decider-2b answers typed questions — true/false, pick-one, ordered score — over text, JSON records, and screenshots, and its probabilities are calibrated: shipped with per-question-type calibration so the number is honest enough to drive an act / confirm / escalate policy. It serves on a new endpoint, POST /v1/systemone, runs on CPU in 8 GB of RAM (2B parameters), and — like everything on this server — your tickets, records, and screenshots never leave the box.

Instead of free text, you send a state (the thing to judge) and one or more questions, each with a declared type. Here's the smallest one — a true/false ("noul") question (response is from a live run of this exact request):

curl -s http://127.0.0.1:8081/v1/systemone \
  -H "Authorization: Bearer $SEARCHAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"strands-decider-2b",
       "state":"Help! My payouts have been failing for 3 days!",
       "questions":{"is_urgent":{"type":"noul",
                                 "instructions":"Does this convey urgency?"}}}'

# → {"model":"strands-decider-2b",
#    "answers":{"is_urgent":{"type":"noul","noul":0.835}},
#    "usage":{"input_tokens":86,"output_tokens":0}}

Three answer types cover most gates:

• noul — a calibrated P(true): 0.835 above.
• choice — pick one of your named options, with a full probability distribution and a confidence score. Options take optional descriptions that steer the decision ("billing: payments, invoices, payouts").
• score — an expectation over ordered levels (lowest first): rate frustration over ["calm","frustrated","desperate"] and get 1.31 — between frustrated and desperate — plus the per-level distribution.

Note output_tokens: 0: there is no text generation at all. The model reads the state once and the answers are read directly from its internal representation — which also means the same request returns the same answer every time.

A model is calibrated when its confidence matches its accuracy: across many answers where it reports 0.9, about 90% are actually right. The standard metric is expected calibration error (ECE) — the average gap between reported confidence and observed accuracy. strands-decider's published figures are ECE 0.052 on text and 0.014 on vision benchmarks; running the public JevBench set against our own serving stack we measured 0.054 — i.e. the reported probabilities track reality to within about five points.

That's the property that makes a threshold policy meaningful, and it's the pattern we'd suggest as the default (the convention from the model's own documentation):

P ≥ 0.9      → act automatically
0.5 ≤ P < 0.9 → act, but queue for human confirmation
P < 0.5      → hand to a human

The payout message above lands at 0.835 — fast-track it, but confirm. A "no rush at all" email scores 0.18 urgency — don't page anyone.

JevBench is the public 231-task decision benchmark the model family is evaluated on. Because our /v1/systemone implements the same wire API, we ran the official benchmark harness, unchanged, pointed at our own CPU server:

• 167/231 = 72.29% — identical to the published reference result for this checkpoint on its own GPU serving stack (also 167/231). A paired per-task comparison is a statistical tie (8 vs 8 discordant tasks, p = 1.0).
• Schema validity 1.000, 0 failed requests, 231/231 attempted.
• ECE 0.054 vs the reference run's 0.052.
• By tier: easy 48/48, standard 65/72, hard 54/111.

The point isn't the 72% (hard-tier questions are genuinely hard — see the caveats). The point is parity: moving the model from a GPU reference stack to a local CPU binary cost nothing measurable in accuracy or calibration.

A request can carry several questions about one state; the state is read once and the extra questions are nearly free (three answers in ~4.5 s on an M4 CPU, vs ~1.8 s for one):

curl -s http://127.0.0.1:8081/v1/systemone \
  -H "Authorization: Bearer $SEARCHAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"strands-decider-2b",
       "state":"Hi team, just checking in on the invoice from March — no rush at all, whenever you get a chance.",
       "questions":{
         "is_urgent":{"type":"noul","instructions":"Does this convey urgency?"},
         "team":{"type":"choice","instructions":"Which team should handle this?",
                 "criteria":{"billing":"payments, invoices, payouts",
                             "technical":"bugs, outages, API errors",
                             "sales":"pricing and upgrades"}},
         "mood":{"type":"score","instructions":"How frustrated is the writer?",
                 "criteria":["calm","frustrated","desperate"]}}}'

# → is_urgent 0.1785 · team "billing" (0.9524, confidence 0.93) · mood 0.51 (calm-leaning)

The state doesn't have to be prose — send a JSON object (a ticket record, an order) and it's judged as-is, deterministically. And with the bundled vision projector, the state can include images (≤4, up to 4096×4096). From a live run against a checkout screenshot showing a full-page error:

IMG=$(base64 -w0 checkout-result.png)   # macOS: base64 -i checkout-result.png
curl -s http://127.0.0.1:8081/v1/systemone \
  -H "Authorization: Bearer $SEARCHAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"strands-decider-2b\",
       \"state\":\"The screenshot shows the final screen of a payment attempt.\",
       \"images\":[\"$IMG\"],
       \"questions\":{\"payment_ok\":{\"type\":\"noul\",
                       \"instructions\":\"Did the payment succeed?\"}}}"

# → {"answers":{"payment_ok":{"type":"noul","noul":0.2124}}}   (~8 s on an M4 CPU)

0.21 — do not mark that order paid. The same shape handles form-completeness checks, document-kind gates, and UI-state verification for automation; the console ships a new Decisions (System One) prompt section with all of these as copyable, validated requests.

curl -fsSL https://inference-server.searchblox.com/install | sudo bash
sudo ./fetch-models.sh strands    # f16 + Q8 + vision projector, ~6.2 GB
# then POST /v1/systemone with "model": "strands-decider-2b"

RAM: ~6 GB for the f16 model (the benchmarked reference), ~4 GB for strands-decider-2b-q8 (validated within the f16 tolerance envelope on our parity fixtures; serves text-only — the vision projector pairs with the f16 entry). Works on any CPU install; no GPU involved anywhere in this post.

• It is not a chat model. The checkpoint's text-generation head is deliberately untrained; asking it for prose yields babble by design. Serve it for /v1/systemone only, and pair it with a chat model for drafting — the decide-then-generate pattern.
• Calibrated ≠ always right. 72.29% overall, and under 50% on JevBench's hard tier (multi-hop, long-policy reasoning). Calibration means the model knows when it's unsure — which is exactly what the <0.5-goes-to-a-human band in the routing policy is for.
• Latency scales with state length. Short messages decide in ~2 s on a laptop CPU; 3,000-token policy documents took up to ~90 s in our JevBench run (median 3 s). Budget for the long tail if your states are documents.
• /v1/score is still there — it works with the chat models you already have, needs no extra download, and is fine for relative picks. Just don't read its numbers as calibrated confidence; that's this model's job.

Model: StrandsAgents strands-decider-2B (Apache 2.0), Qwen3.5-2B torso + pointer head, served as strands-decider-2b. Numbers: JevBench = official harness at its pinned commit, public 231-task set, run against our server's /v1/systemone (f16); ECE/accuracy as reported by the harness. Published ECE 0.052/0.014 from the model card. Timings measured on Apple M4 (release build), 2026-10-05.