Beginner8 min
Run evals at 50% off with Batch
Nightly SWE-bench-style evals without the daytime bill. Submit async, download results.
1. Package your prompts
JSONL: one object per case — input, expected, rubric. Up to 100k requests per job.
{"input": "…", "expected": "…", "id": "case-001"}2. Submit from console
Console → Batch API → pick model + count. Same gateway, half the per-token rate.
# billed at 0.5× serverless — shows in the same ledger
3. Score on download
Join results with expected, track pass-rate per model version in /console/usage. Promote winners to serverless.
# usage API groups cost by model — compare candidates