Intermediate12 min
Ship RAG with Qwen3 embeddings + rerank
Multilingual embeddings, cross-encoder reranking, and a Flash model to answer — the whole pipeline on one gateway.
1. Embed your docs
Chunk to ~512 tokens, embed with qwen3-embedding-8b ($0.10/1M). Store vectors in Postgres + pgvector.
emb = client.embeddings.create(
model="runaii/qwen3-embedding-8b",
input=chunks)2. Retrieve, then rerank
Pull top-20 by cosine, rerank to top-5 with qwen3-reranker-8b. Rerank quality is where most RAG wins hide.
# POST /v1/rerank
{"model": "runaii/qwen3-reranker-8b",
"query": q, "documents": docs, "top_n": 5}3. Answer cheap
Feed the 5 passages to glm-5.3-flash. Cache the system prompt + corpus prefix — repeat questions bill at cached rates.
resp = client.chat.completions.create(
model="runaii/glm-5.3-flash",
messages=[{"role": "system", "content": SYS},
{"role": "user", "content": q + ctx}])