Enterprise AI agents, running on your own hardware
Six use cases you can reproduce end to end today — an open-source agent, a private OpenAI-compatible endpoint, and nothing leaving your network. Every walkthrough below was run as shown on a CPU-only production server; every timing is measured, and the caveats are printed next to the wins.
all on CPU · 4B modelmeasured timings copy-paste walkthroughshonest caveats
structured JSON outself-verifying33 s/doc measured
#1 — Document & contract extraction pipelines
Contracts, policies, surveys, resumes — turned into typed, verified JSON that downstream systems can consume, without a page leaving your network.
Measured: 33 s per contract · 43 s for a 3-doc batch · 4B–9B models, CPU is enough. Add the vision models for scanned documents.
retrieve + answer + actreranker built in58 s measured
#2 — RAG with actions over internal systems
The agent retrieves from your knowledge base, answers with a citation, and then acts — files the ticket, updates the record — in one loop on one box.
Measured: 58 s: retrieve → cited answer → ticket filed → verified · 4B–9B models, CPU is enough. /v1/embeddings + /v1/rerank run on the same endpoint.
draft-for-reviewclause-level citations64 s measured
#3 — Compliance & incident report drafting
The agent reads the policy and the incident record and drafts the compliance report — clause citations, violations, remediation table, open questions — for a human to review. The sensitive material never leaves your infrastructure.
Measured: 64 s: policy + incident → structured report draft · 4B works for structured drafts; 9B–27B for longer reports. Async work — CPU is fine.
grep-verifiedseverity + actions51 s measured
#4 — Log & incident triage
Point the agent at a raw application log; it separates the distinct incidents, verifies counts with grep before writing, and produces a triage report with severity and a recommended next action per incident.
Measured: 51 s: raw log → per-incident summary with recommended actions · 4B–9B, CPU. Logs often contain customer identifiers — a private endpoint keeps them home.
writes AND executesiterates to green66 s measured
#5 — Write-and-run code & back-office batch jobs
The agent writes a script against your data, runs it, fixes what breaks, and shows the output — useful for one-off transformations and recurring back-office glue, with honest limits for bigger coding work.
Measured: 66 s: script written, run, fixed, output produced · 4B for single-shot scripts and batch glue; long autonomous coding chains want 27B + GPU or tight human review.
one request fieldcross-session recall2.6 s measured
#6 — Assistants that remember: sessions & long-term memory
Support desks, account teams, personal assistants — the server keeps the conversation and remembers durable facts about a user across sessions, so every app you build gets memory without building a memory stack.
Measured: 2.6 s cross-session recall; 2.1 s with a format preference applied from memory · Works with any model; the memory layer itself uses the small 0.8B + embed models in the background. CPU is plenty.
one prefill · no decodedeterministic~355 ms/decision
#7 — Fast local decisioning: classify & route with /v1/score
Route tickets, gate actions, and label inputs in a single prefill with no decode — a deterministic decision plus a probability distribution, entirely on-box.
Measured: ~355 ms per decision (q35-4b) · ~84 ms on an exact repeat · Qwen 3.5 or Spark X2.5 models, CPU is enough. No extra model to install.
The pattern behind all of them
One clear task → a few tool calls → the agent verifies its own output → a human checkpoint before anything irreversible. That design is why 4B-class models on CPU are enough for these workloads — and where they are not (long autonomous coding chains), the pages say so and point to the 27B + GPU tier.
Setup for all of them is the same 10 minutes: the pi agents guide. Give agents durable memory with the built-in memory API.