top-5 reranked, cited passages
Enterprise RAG that cites
Embed, retrieve, rerank, answer — with cached prefixes and audit trails.
Bring your own retrieval: chunk to roughly 512 tokens, embed and rerank with your stack (the /v1/embeddings and /v1/rerank endpoints are rolling out), pull the top 20 by cosine similarity, then rerank to the top 5. Rerank quality is where most RAG wins hide. The retrieval step stays yours; runaii meters only the model tokens.
Answer with a Flash-class model over a shared system prompt. Qwen3.8-27B Flash (`qwen3.8-flash`) is the balanced option at 33K context and $0.40/1M input, $0.04/1M cached input, and $1.60/1M output; DeepSeek V4.1 Flash (`deepseek-v4-flash`) is the long-context option at 1.05M and $0.22/1M input.
Repeated questions bill at cached rates, so a shared corpus prefix is the main cost lever: $0.022/1M on DeepSeek V4.1 Flash against $0.22/1M uncached. Retrieval context can outgrow Qwen's 33K window, which is why the long-context route names DeepSeek rather than Qwen at 1.05M tokens.
Every retrieval is logged, so regulated teams can show exactly what the model saw; batch backfills run through the Batch API on the same ledger. Follow the RAG tutorial for the embed-rerank call path, then choose the model from its context window and cached rate. If the corpus prefix is stable across questions, the cached rate applies automatically.
- ✓Embeddings + rerank — rolling out
- ✓Cached corpus prefixes up to 90% off
- ✓Request logs for compliance review
- ✓Batch backfills via Batch API (beta)
Keep exploring
Production inference
Serverless tokens or warm dedicated GPUs — one gateway, OpenAI-compatible.
Fine-tuning that ships
SFT → DPO → RL, then deploy the checkpoint in seconds.
Agents that stay up
Low-latency tool loops, streaming, and spend guardrails for autonomous workloads.
Batch API (beta)
Evals, backfills, and bulk scoring — async, without holding a connection.
Long-context production
Text models behind one key — vision is not served yet.