Enterprise AI agents, running on your own hardware
Nine use cases you can reproduce end to end today — agents, retrieval, and calibrated decisions on a private OpenAI-compatible endpoint, with nothing leaving your network. Every walkthrough below was run as shown on a CPU-only server; every timing is measured, and the caveats are printed next to the wins.
all on CPU · 4B modelmeasured timings copy-paste walkthroughshonest caveats
structured JSON outself-verifying33 s/doc measured
#1 — Document & contract extraction pipelines
Contracts, policies, surveys, resumes — turned into typed, verified JSON that downstream systems can consume, without a page leaving your network.
Measured: 33 s per contract · 43 s for a 3-doc batch · 4B–9B models, CPU is enough. Add the vision models for scanned documents.
retrieve + answer + actreranker built in58 s measured
#2 — RAG with actions over internal systems
The agent retrieves from your knowledge base, answers with a citation, and then acts — files the ticket, updates the record — in one loop on one box.
Measured: 58 s: retrieve → cited answer → ticket filed → verified · 4B–9B models, CPU is enough. /v1/embeddings + /v1/rerank run on the same endpoint.
draft-for-reviewclause-level citations64 s measured
#3 — Compliance & incident report drafting
The agent reads the policy and the incident record and drafts the compliance report — clause citations, violations, remediation table, open questions — for a human to review. The sensitive material never leaves your infrastructure.
Measured: 64 s: policy + incident → structured report draft · 4B works for structured drafts; 9B–27B for longer reports. Async work — CPU is fine.
grep-verifiedseverity + actions51 s measured
#4 — Log & incident triage
Point the agent at a raw application log; it separates the distinct incidents, verifies counts with grep before writing, and produces a triage report with severity and a recommended next action per incident.
Measured: 51 s: raw log → per-incident summary with recommended actions · 4B–9B, CPU. Logs often contain customer identifiers — a private endpoint keeps them home.
writes AND executesiterates to green66 s measured
#5 — Write-and-run code & back-office batch jobs
The agent writes a script against your data, runs it, fixes what breaks, and shows the output — useful for one-off transformations and recurring back-office glue, with honest limits for bigger coding work.
Measured: 66 s: script written, run, fixed, output produced · 4B for single-shot scripts and batch glue; long autonomous coding chains want 27B + GPU or tight human review.
one request fieldcross-session recall2.6 s measured
#6 — Assistants that remember: sessions & long-term memory
Support desks, account teams, personal assistants — the server keeps the conversation and remembers durable facts about a user across sessions, so every app you build gets memory without building a memory stack.
Measured: 2.6 s cross-session recall; 2.1 s with a format preference applied from memory · Works with any model; the memory layer itself uses the small 0.8B + embed models in the background. CPU is plenty.
one prefill · no decodedeterministic~355 ms/decision
#7 — Fast local decisioning: classify & route with /v1/score
Route tickets, gate actions, and label inputs in a single prefill with no decode — a deterministic decision plus a probability distribution, entirely on-box.
Measured: ~355 ms per decision (q35-4b) · ~84 ms on an exact repeat · Qwen 3.5 or Spark X2.5 models, CPU is enough. No extra model to install.
calibrated probabilitiesnoul · choice · score~1.8 s text · ~8 s vision
#8 — Calibrated decision gates with vision
Gate automated actions on probabilities you can actually trust: strands-decider-2b answers typed questions — true/false, pick-one, ordered score — over text, JSON records, and screenshots, with calibrated confidence numbers, entirely on-box via POST /v1/systemone.
Measured: ~1.8 s per text decision · ~4.5 s for three questions on one state · ~8 s with a screenshot attached (Apple M4 CPU, strands-decider-2b f16) · strands-decider-2b (strands fetch group) — a dedicated decision model, not a chat model. 8 GB RAM for f16, 4 GB for the Q8 variant (text-only); vision uses the bundled mmproj. CPU is enough.
gibberish · language · intentcorrect + expand~5.7 s/query measured
#9 — Search query understanding
Classic search-side query processing — gibberish gating, language ID, intent classification, spelling correction, synonym expansion, filter extraction — as a decide-then-generate chain: calibrated decisions answer the classification questions, small generative models do the rewriting, all on one private box.
Measured: ~5.7–6.4 s per query for the full chain (3-question screen ~5.1 s + correction 0.1–0.5 s + expansion 0.5–0.9 s, warm) · single gate ~1.9 s · misspelling gate via /v1/score 60–230 ms · strands-decider-2b for the calibrated gates + q35-2b for correction/expansion (spark-x25-1.7b for JSON filter extraction, q35-0.8b for keyword rewrites). CPU is enough — this is an async enrichment workload.
The pattern behind all of them
One clear task → a few tool calls → the agent verifies its own output → a human checkpoint before anything irreversible. That design is why 4B-class models on CPU are enough for these workloads — and where they are not (long autonomous coding chains), the pages say so and point to the 27B + GPU tier.
Setup for all of them is the same 10 minutes: the pi agents guide. Give agents durable memory with the built-in memory API.