13 models. Three families. One endpoint.
Every model below serves the same OpenAI-compatible API — pick by RAM tier
and capability, not by integration effort. Load any of these from the
console's Models tab with one click — the server downloads and brings it
online. Or pre-fetch from the shell with
fetch-models.sh (installed next to the models; also a
direct download).
The name on each card is the model id — the exact string for the API model field. The fetch-models.sh argument (its install group) can be shorter; use the card's model id in requests.
text + tools on every model vision · audio · video 8 GB to 64 GB tiers one-click console load
Qwen 3.5 family
The workhorse family — from the 0.8B speed demon to the 35B mixture-of-experts, all sharing one prompt format and one API.
q35-0.8b
Qwen 3.5 · 0.8B · needs 8 GB RAM
The fastest model in the catalog — text and tool calling only. Fits 8 GB edge hosts and doubles as a speculative-decoding draft model.
q35-2b
Qwen 3.5 · 2B · needs 16–32 GB RAM · ~1.9 GB download
Light multimodal — full vision, audio, and video understanding on a 16 GB host.
q35-4b
Qwen 3.5 · 4B · needs 32 GB RAM · ~3.4 GB download
The flagship default the installer ships — best speed/quality balance, zero-error record on the 380-prompt pack, identical results on Arm, Intel, and AMD.
q35-9b
Qwen 3.5 · 9B · needs 32 GB RAM · ~6.6 GB download
The strongest text + vision quality in the Qwen 3.5 family.
q35moe-35b
Qwen 3.5 · 35B MoE · needs 64 GB RAM
General-purpose mixture-of-experts: 35B-class answers at active-parameter speed. Audio input via the ASR sidecar.
coder-next
Qwen 3.5 · 35B MoE · needs 64 GB RAM
Agentic coding specialist — built for long tool-calling sessions and repository-scale code work.
Qwen 3.8 family
The deep-reasoning generation: hybrid attention for long context, thinking mode, and per-request reasoning_effort control.
qwen38-27b
Qwen 3.8 · 27B · needs 64 GB RAM · ~17.2 GB download
Dense hybrid reasoner with thinking mode and reasoning_effort levels (low / medium / high) for dial-a-depth answers. Audio via the ASR sidecar; plan for a 64 GB node.
Gemma 4 family
Native multimodal end to end — vision, audio, and video decoded in-model, no projector round-trips, on hosts as small as 8 GB.
gemma-4-E2B-it
Gemma 4 · E2B · needs 8 GB RAM · ~2.7 GB download
The smallest full-multimodal model — native text, vision, audio, and video on an 8 GB host.
gemma-4-E4B-it
Gemma 4 · E4B · needs 16 GB RAM · ~4.5 GB download
Full multimodal on a 16 GB host — 320/380 on the prompt pack, within five answers of the 4B flagship.
gemma-4-12B-it
Gemma 4 · 12B · needs 32 GB RAM · ~6.7 GB download
The media flagship: image encode in 19 ms and audio encode in ~1 ms, with native video too. The SAI_G4_W4=only profile serves it in ~7 GB RSS with instant load.
gemma-4-26B-A4B-it
Gemma 4 · 26B A4B MoE · needs 32 GB RAM · ~13.9 GB download
Highest pack quality in the catalog (329/380) at ~10 s typical answers — MoE active-parameter economics on a 32 GB host. Audio via the ASR sidecar.
Retrieval family
The search building blocks — multimodal embeddings and reranking that complete a single-binary RAG stack: embed, recall, rerank, and generate on one CPU server.
qwen3vl-embed-2b
Retrieval · 2B · needs 8 GB RAM · ~2.5 GB download
Multimodal embeddings for search and RAG — text and images share one 2048-dim vector space (OpenAI-compatible /v1/embeddings). Matryoshka dimensions 64–2048 let you shrink vectors without re-embedding; 73.2 MMEB-V2. Remote image URLs and data URLs both accepted.
qwen3vl-rerank-2b
Retrieval · 2B · needs 8 GB RAM · ~2.5 GB download
Second-stage reranker for precision — scores each document (text or image) against the query, 0–1, via /v1/rerank (Jina/Cohere-compatible). Pair it with the embedder: vector recall first, rerank the top candidates.
Pick by RAM
| Host RAM | Models that fit comfortably |
|---|---|
| 8 GB | q35-0.8b, gemma-4-E2B-it, qwen3vl-embed-2b, qwen3vl-rerank-2b |
| 16 GB | q35-2b, gemma-4-E4B-it — and gemma-4-12B-it under SAI_G4_W4=only (~7 GB RSS) |
| 32 GB | q35-4b (default), q35-9b, gemma-4-12B-it, gemma-4-26B-A4B-it |
| 64 GB | q35moe-35b, coder-next, qwen38-27b |
Every model in the catalog does function calling.
audio* = audio input served via the ASR sidecar: install the
asr group and set asr-model — speech is transcribed
by the dedicated ASR model and handed to the text model on the same request.
The low-RAM Gemma tier (SAI_G4_W4=only) serves Gemma 12B in
~7 GB RSS with instant weight load. Sizing details in the
Getting Started guide; measured
speeds on the Performance page.