Build private AI agents with pi + the SearchAI Inference Server

An AI agent — a model that can read files, run commands, write outputs, and check its own work — is only as private as the endpoint it talks to. Point an agent at a metered cloud API and every file it reads ships off-network. Point it at your own inference server and the whole loop stays on your hardware.

This guide connects pi — a deliberately minimal open-source coding agent (four tools: read, write, edit, bash; its system prompt and tool definitions fit in under 1,000 tokens, a fraction of the big-vendor harnesses) — to the SearchAI Inference Server. Every command and timing below was run as written on a CPU-only server (32-vCPU Graviton) with the 4B default model: real agent tasks completed in 26–66 seconds each, no GPU involved.

Any install works — Linux one-liner, macOS, Docker, or the GPU variant. The agent just needs the endpoint URL and API key:

curl -fsSL https://inference-server.searchblox.com/install | sudo bash

The agent can run on the same box as the server (what we did here) or on your laptop pointing at a shared server — baseUrl in step 3 is the only thing that changes. See Pick your install.

# Node 22+ if you don't have it (Linux; nodejs.org for other platforms)
curl -fsSL https://deb.nodesource.com/setup_22.x | sudo bash -
sudo apt-get install -y nodejs

# the pi coding agent
npm install -g --ignore-scripts @earendil-works/pi-coding-agent
pi --version   # 0.84.4 at time of writing

pi treats custom OpenAI-compatible endpoints as a first-class provider. Create ~/.pi/agent/extensions/searchai.ts:

import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";

export default function (pi: ExtensionAPI) {
  pi.registerProvider("searchai", {
    baseUrl: "http://127.0.0.1:8081/v1",   // your server
    apiKey: "$SEARCHAI_API_KEY",           // reads the env var
    api: "openai-completions",
    models: [
      { id: "q35-4b", name: "SearchAI q35-4b", reasoning: false,
        input: ["text"],
        cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
        contextWindow: 32768, maxTokens: 4096 },
      { id: "q35-9b", name: "SearchAI q35-9b", reasoning: false,
        input: ["text"],
        cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
        contextWindow: 32768, maxTokens: 4096 }
    ]
  });
}

Export your key and you're done:

export SEARCHAI_API_KEY='your-api-key'   # from your install output

In our testing the server worked with pi stock — no compatibility flags needed: streaming, tool-call round-tripping, and multi-step loops all just worked against /v1/chat/completions. Add any model from the catalog to the list the same way.

Interactive (a full TUI session in your project directory):

pi --provider searchai --model q35-4b

Or one-shot — the mode all the use cases below use:

pi --provider searchai --model q35-4b -p \
  "Create a file greeting.txt containing exactly the text hello from searchai, \
   then run wc -c greeting.txt with bash and report the byte count."

Measured: the agent called its write tool, then bash, then answered — correct file, correct count, 26 seconds end to end on the CPU box.

The strongest enterprise fit: documents that must not leave the network, turned into structured data. With a contract in the working directory:

pi --provider searchai --model q35-4b -p \
  "Read agreement1.txt and extract the parties, effective date, term length, \
   monthly fee, termination notice period, governing law, and contact email \
   into a JSON file named agreement1.json with sensible snake_case keys. \
   Then verify it parses with python3 -m json.tool."

Measured result, verbatim from the run:

{
    "parties": { "provider": "Acme Logistics LLC",
                 "client": "Bluewater Foods Inc" },
    "effective_date": "2026-03-15",
    "term_length_months": 24,
    "monthly_fee_usd": 4750.0,
    "termination_notice_period_days": 60,
    "governing_law": "State of Delaware",
    "contact_email": "sarah.chen@bluewaterfoods.example"
}

Every field correct, the date normalized to ISO, numbers typed as numbers — and the agent verified its own output with a tool call. 33 seconds.

pi --provider searchai --model q35-4b -p \
  "Extract the same fields from EVERY .txt file in this directory into one \
   summary.jsonl file — one JSON object per line per contract, with a \
   source_file key. Verify each line parses."

Measured: three contracts → three valid JSONL lines in 43 seconds. One instructive miss: the agent dropped the requested source_file key. That's the honest shape of 4B-class agents — near-perfect on the extraction itself, occasionally lossy on secondary instructions — which is why the pattern below (verify + checkpoint) is part of the design, not an afterthought.

pi --provider searchai --model q35-4b -p \
  "Triage app.log: identify the distinct incidents, their severity, affected \
   orders/customers, and write an incident summary to triage.md with a \
   recommended next action for each. Use grep to verify counts before writing."

Measured: the agent grep-verified the log, correctly separated the three real incidents (repeated payment-gateway timeouts with the affected order/customer IDs, a 502 burst on checkout, a slow token refresh) and produced per-incident recommended actions in 51 seconds. Tip from this run: name the output file explicitly and check it exists — small models sometimes print the report instead of writing it.

Yes, the agent writes and executes code against your private endpoint — that's precisely pi's write + bash loop:

pi --provider searchai --model q35-4b -p \
  "Write a python script csv_report.py that reads ../contracts/summary.jsonl \
   and produces report.csv with columns source_file, client_or_buyer, fee, \
   governing_law — then RUN it and show the output. Fix any errors you hit \
   until it runs clean."

Measured: the agent wrote the script, ran it, iterated to a clean run, produced the CSV, and — notably — flagged on its own that one requested column wasn't derivable from the input data rather than inventing values. 66 seconds.

• Single-shot and short-loop tasks are the sweet spot. On public function-calling benchmarks, 4B–9B open models are at near-frontier accuracy for single-turn structured tool calls — but all small generic models fall off steeply on long autonomous multi-turn chains. Design agents as: one clear task, a few tool calls, verify, hand to a human.
• Verify outputs. As use case 2 shows, secondary instructions can get dropped. Make the agent self-verify (it happily runs json.tool/grep) and keep a human checkpoint before anything irreversible.
• CPU is genuinely enough for these patterns — every number above is from a CPU-only box. For heavier agents (9B–27B models, longer chains, many users), the same install takes the one-flag GPU add-on (3–4× decode).
• Start at 4B. Sub-1B models are below the practical floor for tool use; 4B is where agent behavior becomes reliable.

A 20-line provider file connects an open-source agent to your private endpoint, and real work — contracts to JSON, logs to incident reports, scripts written and executed — happens in under a minute per task on hardware you already own. The documents the agent reads, the commands it runs, and everything it writes stay inside your network, at flat cost.

npm install -g --ignore-scripts @earendil-works/pi-coding-agent
# + the 20-line extension above
pi --provider searchai --model q35-4b

All timings measured 2026-09-01 on a CPU-only 32-vCPU Graviton server with q35-4b, tasks run as shown. pi is an independent open-source project (pi.dev); any agent that speaks the OpenAI completions protocol works the same way — see function calling on a 2B model for the API-level view.