Blog
Engineering notes and field reports from the team building the SearchAI Inference Server — private AI on the hardware you already own, with every claim tied to a published measurement.
prompt engineeringsmall models console workbenchbenchmarks
Optimize the prompt, shrink the model: a practical workflow for efficient AI at scale
August 2026 · 8 min read
When outputs disappoint, most teams reach for a bigger model — the most expensive possible fix. On our 380-prompt, 13-industry evaluation the quality gap between 4B-class and 27B-class models is a few points; the cost gap is an order of magnitude per served user. This article shows the compare → optimize → re-compare workflow using the console's built-in workbench: side-by-side model comparison (local or against a frontier API), a one-click Prompt Optimizer, and the prompt patterns that consistently let small models match big ones on your task.
RAGAWS EC2 cost analysisbenchmarks
Run private RAG on one EC2 box — and stop paying by the token
August 2026 · 10 min read
Retrieval-augmented generation re-sends the same tokens over and over — and per-token APIs bill you every time. We put the model inside the VPC on a fixed-price Graviton instance instead, benchmarked it with a 380-prompt, 13-industry test pack, and published the math: measured latency, a side-by-side monthly cost comparison with the assumptions on the table, and the one-config-line clustering story for scale-out.