Search query understanding

Classic search-side query processing — gibberish gating, language ID, intent classification, spelling correction, synonym expansion, filter extraction — as a decide-then-generate chain: calibrated decisions answer the classification questions, small generative models do the rewriting, all on one private box.

Why run it locally: Search queries are short, messy, and full of intent — and the query log is one of the most sensitive datasets a company holds. This pipeline follows the multi-step query-understanding pattern from search practice: classification steps (is it junk? what language? what intent?) are calibrated zero-decode decisions you can threshold, and rewriting steps (correct, expand, extract) are tightly-constrained generations from the smallest model that passes each task — measured, per task, below.

Model & hardware fit: strands-decider-2b for the calibrated gates + q35-2b for correction/expansion (spark-x25-1.7b for JSON filter extraction, q35-0.8b for keyword rewrites). CPU is enough — this is an async enrichment workload.

1. The server — any install works; one line on Linux:

curl -fsSL https://inference-server.searchblox.com/install | sudo bash

2. The models — the strands decision group plus a small chat model:

sudo ./fetch-models.sh strands 2b

Everything below is also runnable from the built-in console: the Query Understanding prompt category (P388–P397) and the Pipeline tab's "Search Query Understanding" example chain these exact requests.

Screen the raw query — three questions, ONE request (the state is prefilled once and forked per question; ~5.1 s measured for all three):

curl -s http://127.0.0.1:8081/v1/systemone \
  -H "Authorization: Bearer $SEARCHAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"strands-decider-2b",
       "state":"recipies with chiken breast",
       "questions":{
         "is_gibberish":{"type":"noul","instructions":"The text is random keyboard noise or gibberish, not a meaningful search query."},
         "language":{"type":"choice","instructions":"What language is this search query written in?",
                     "criteria":{"en":"English","es":"Spanish","fr":"French","de":"German",
                                 "pt":"Portuguese","other":"any other language"}},
         "intent":{"type":"choice","instructions":"What is the search intent of this query?",
                   "criteria":{"informational":"the user wants to learn something or find information",
                               "navigational":"the user wants to reach a specific website or page",
                               "transactional":"the user wants to buy something or complete an action"}}}}'
# → is_gibberish 0.2441 · language "en" 0.8559 · intent "informational" 0.8285

The same screen on "asdkjh qwelkj zzz" measures is_gibberish 0.8663 — and language "other" at 0.50: the model correctly refuses to assign a language to noise. Gate junk before spending any generation. The typo-ridden query above is not junk, so correct it (q35-2b, temp 0, 0.1–0.5 s warm — 5/5 exact on our typo set where the 0.8B scored 3/5):

curl -s http://127.0.0.1:8081/v1/chat/completions \
  -H "Authorization: Bearer $SEARCHAI_API_KEY" -H "Content-Type: application/json" \
  -d '{"model":"q35-2b","temperature":0,"max_tokens":24,"enable_thinking":false,
       "messages":[{"role":"system","content":"You correct spelling in search queries. Return only the corrected query — no explanation, no quotes, no extra words. If the query is already spelled correctly, return it unchanged."},
                   {"role":"user","content":"recipies with chiken breast"}]}'
# → "recipes with chicken breast"

Then expand the corrected query for recall (0.5–0.9 s; the format instruction keeps it machine-parseable):

# same endpoint, system prompt: "Return exactly 5 synonyms or closely related
# search terms for the query, as one comma-separated line. No numbering, no explanation."
# → chicken breast recipes, chicken breast dishes, chicken breast meals,
#   chicken breast cooking methods, chicken breast preparation

It follows the query's language unprompted — "zapatos de cuero para hombre" screens as es 0.9717, survives correction unchanged, and expands to calzado de cuero para hombre, botas de cuero masculinas, zapatos de piel hombre… And for shopping-style queries, extract typed filters (spark-x25-1.7b with a one-shot example in the prompt — all named brands caught in our runs where the 2B missed one):

# q: "nike running shoes under $100 size 10"  (+ response_format json_object)
# → {"product": "running shoes", "brand": "nike", "max_price": 100, "attributes": ["size 10"]}

The assembled result per query — exactly what the console's Pipeline tab produces (whole chain ~5.7 s warm, measured):

{
  "original": "recipies with chiken breast",
  "corrected": "recipes with chicken breast",
  "correction_applied": true,
  "language": "en",          "language_confidence": 0.8559,
  "intent": "informational", "intent_confidence": 0.8285,
  "gibberish_p": 0.2441,
  "expansion_terms": ["chicken breast recipes", "chicken breast dishes",
                      "chicken breast meals", "chicken breast cooking methods",
                      "chicken breast preparation"]
}

Be honest about where this runs: ~6 s per query is async-enrichment speed, not live-search speed — mine the query log nightly (correct + expand + tag the top and zero-result queries into synonym dictionaries and intent rules your search engine applies at full speed), or use it as a zero-results fallback; a live search box budgets ~100 ms. Two detectors failed validation and are deliberately absent: asking strands "does this need spelling correction?" is flat (~0.45 either way — subword tokens hide typos; always correct, then diff), and query ambiguity detection failed on every model we tried. Language ID is validated on en/es/fr/de/pt only — strands' typed-decision training is English-heavy, and the "other" escape option is the honest handler for everything else (a Japanese query scored "other" 0.78 rather than being forced into a wrong language). The fast /v1/score misspelling gate (60–230 ms on spark-x25-1.7b) works but false-positives on lowercase proper nouns ("cheap flights to london" → flagged); its probabilities are preferences, not calibrated — for numbers you can threshold, stay with /v1/systemone (use case #8).

Measured: ~5.7–6.4 s per query for the full chain (3-question screen ~5.1 s + correction 0.1–0.5 s + expansion 0.5–0.9 s, warm) · single gate ~1.9 s · misspelling gate via /v1/score 60–230 ms.

Next: all the use cases · give the agent memory across sessions · 3–4× the speed with one GPU flag