RAG with actions over internal systems

The agent retrieves from your knowledge base, answers with a citation, and then acts — files the ticket, updates the record — in one loop on one box.

Why run it locally: Plain RAG answers questions; agents close the loop. Because the server exposes embeddings and a cross-encoder reranker on the same OpenAI-compatible endpoint as chat, the whole retrieve-answer-act loop runs against one private URL — no vector database service, no second vendor.

Model & hardware fit: 4B–9B models, CPU is enough. /v1/embeddings + /v1/rerank run on the same endpoint.

1. The server — any install works; one line on Linux:

curl -fsSL https://inference-server.searchblox.com/install | sudo bash

2. The agent — pi, a minimal open-source coding agent (four tools: read, write, edit, bash), needs Node 22+:

npm install -g --ignore-scripts @earendil-works/pi-coding-agent

3. Connect them — create ~/.pi/agent/extensions/searchai.ts:

import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";

export default function (pi: ExtensionAPI) {
  pi.registerProvider("searchai", {
    baseUrl: "http://127.0.0.1:8081/v1",
    apiKey: "$SEARCHAI_API_KEY",
    api: "openai-completions",
    models: [{ id: "q35-4b", name: "SearchAI q35-4b", reasoning: false,
      input: ["text"], cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
      contextWindow: 32768, maxTokens: 4096 }]
  });
}

export SEARCHAI_API_KEY='your-api-key'   # printed by the installer

Full setup detail (and four more worked tasks) in the pi agents article. The server works with pi stock — no compatibility flags.

Stage a small IT knowledge base (five markdown articles) and a retrieval script the agent can call as a tool — it asks the server's own reranker to rank the KB against the query:

# retrieve.sh <query> — rank kb/*.md with the built-in reranker, print the top passages
curl -s http://127.0.0.1:8081/v1/rerank \
  -H "Authorization: Bearer $SEARCHAI_API_KEY" -H "Content-Type: application/json" \
  -d "{\"model\":\"qwen3vl-rerank-2b\",\"query\":\"$1\",\"documents\":[...kb texts...],\"top_n\":2}"

Then one prompt drives the whole loop:

pi --provider searchai --model q35-4b -p \
  "Answer this employee question using our IT knowledge base: A contractor \
   needs VPN access for six months, what do we do? First run: bash retrieve.sh \
   'contractor VPN access duration'. Then (1) write answer.md: a concise answer \
   citing the KB article id, and (2) because the request exceeds the policy \
   limit, file an exception ticket as tickets/TICKET-001.json with keys: \
   summary, kb_source, priority, requested_exception, next_step. \
   Verify the ticket parses with python3 -m json.tool."

Measured run (58 seconds, CPU 4B): the reranker put the right article first (relevance 0.861 vs 0.361 for the runner-up), and the agent — unprompted about the specifics — spotted that six months exceeds the 90-day policy maximum, answered citing KB-101, and filed a valid ticket:

{
  "summary": "Contractor VPN access request exceeds policy limit (6 months vs. 90-day maximum)",
  "kb_source": "KB-101: VPN access for contractors",
  "priority": "P2",
  "requested_exception": "Extend contractor VPN access beyond the standard 90-day maximum...",
  "next_step": "Review and approve exception request; upon approval, update IT portal..."
}

The action here writes a ticket file — in production, point the same pattern at your ticketing API and keep the approval step human. For corpora beyond a few hundred documents, do a first pass with /v1/embeddings and rerank the top-50 (the two-stage pattern from the embeddings article).

Measured: 58 s: retrieve → cited answer → ticket filed → verified.

Next: all the use cases · give the agent memory across sessions · 3–4× the speed with one GPU flag