Assistants that remember: sessions & long-term memory

Support desks, account teams, personal assistants — the server keeps the conversation and remembers durable facts about a user across sessions, so every app you build gets memory without building a memory stack.

Why run it locally: Conversation state is the workload most teams quietly ship to a cloud vendor's servers. Here sessions, summaries, and long-term memories live on your disk, 0700, under your service user — and the embeddings powering recall run on the same box. No second database, no third party.

Model & hardware fit: Works with any model; the memory layer itself uses the small 0.8B + embed models in the background. CPU is plenty.

1. The server — any install works; one line on Linux:

curl -fsSL https://inference-server.searchblox.com/install | sudo bash

2. The agent — pi, a minimal open-source coding agent (four tools: read, write, edit, bash), needs Node 22+:

npm install -g --ignore-scripts @earendil-works/pi-coding-agent

3. Connect them — create ~/.pi/agent/extensions/searchai.ts:

import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";

export default function (pi: ExtensionAPI) {
  pi.registerProvider("searchai", {
    baseUrl: "http://127.0.0.1:8081/v1",
    apiKey: "$SEARCHAI_API_KEY",
    api: "openai-completions",
    models: [{ id: "q35-4b", name: "SearchAI q35-4b", reasoning: false,
      input: ["text"], cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
      contextWindow: 32768, maxTokens: 4096 }]
  });
}

export SEARCHAI_API_KEY='your-api-key'   # printed by the installer

Full setup detail (and four more worked tasks) in the pi agents article. The server works with pi stock — no compatibility flags.

Sessions: add one field, send only the NEW message each turn — the server keeps the history (no agent needed; this is plain /v1/chat/completions):

curl -s http://127.0.0.1:8081/v1/chat/completions \
  -H "Authorization: Bearer $SEARCHAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"q35-4b", "session":"support-desk", "user":"lisa",
       "messages":[{"role":"user","content":"Account note: Meridian Foods is migrating to the enterprise plan; brand rule: all UI mockups must use teal."}]}'

# next turn — just the question, the server supplies the context:
#   "What color should the Meridian mockups use?"  →  "…must use **teal**"  (2.6 s)

Long-term memory: store durable facts against the user scope — they follow that user into every FUTURE session:

curl -s -X POST http://127.0.0.1:8081/v1/memories \
  -H "Authorization: Bearer $SEARCHAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"scope":"lisa","text":"Customer Meridian Foods is migrating to the enterprise plan; brand rule: all UI mockups must use teal."}'
curl -s -X POST http://127.0.0.1:8081/v1/memories ... \
  -d '{"scope":"lisa","text":"Lisa always wants dates in ISO format (YYYY-MM-DD) in reports."}'

Measured, in a brand-new session days-later style (CPU 4B):

Q: "Which plan is Meridian Foods migrating to, and what color must their mockups use?"
A: "Based on the long-term memory records:
    - Plan: Meridian Foods is migrating to the **enterprise plan**.
    - Color Requirement: ...all UI mockups must use **teal**."      (2.6 s)

Q: "One line for my report: the Meridian kickoff is March 3 next year.
    Use my preferred date format."
A: "The Meridian migration kickoff is scheduled for 2026-03-03."   (2.1 s)

Both memories fired — the account fact answered the question, and the format preference silently shaped the output. Recall is hybrid (vector + keyword + optional reranker), so exact identifiers are found even when embeddings would miss them.

Automatic fact extraction ("dreaming") runs in the background when long conversations compact — for facts that must be remembered immediately, write them explicitly with POST /v1/memories as shown; that is the production pattern. Session and memory files hold conversation content: treat the directories with log-level sensitivity (the installer creates them 0700). Full API detail in the memory & sessions article.

Measured: 2.6 s cross-session recall; 2.1 s with a format preference applied from memory.

Next: all the use cases · give the agent memory across sessions · 3–4× the speed with one GPU flag