Your prompts are cached — here is what that saves you
Prompt caching on GLM-5.3, Kimi K3, and DeepSeek V4 bills cache hits at up to 10x less. How the split works, and how to structure prompts to hit it.
The pricing model
For models with prompt caching, input tokens are billed in two buckets:
| Bucket | Rate (glm-5.3-flash) |
|---|---|
| Cache miss (first time a prefix is seen) | $0.15 / 1M tokens |
| Cache hit (same prefix seen again) | $0.03 / 1M tokens — 5x cheaper |
The split is per-request and automatic: whatever portion of your prompt the provider
serves from cache bills at the cached rate, the rest at full rate. Nothing to
configure. usage.prompt_tokens_details.cached_tokens in the response tells you
exactly how many tokens hit.
How to actually hit the cache
The cache works on identical prefixes. The winning pattern is boring and effective:
- Static system prompt — put your instructions, schemas, and few-shot examples at the start of the messages array and never change a byte of them.
- Dynamic user content last — the volatile part of your request goes at the end.
- Reuse deployments — cache lives per routing path; consistent model ids help.
RAG pipelines benefit most: embed your corpus prefix once, then every question bills the corpus at the cached rate. At 50k tokens of context, that is $1.50 → $0.15 per question in prompt costs before the model even starts answering.
Verifying it on your own bill
resp = client.chat.completions.create(
base_url="https://api.runaii.cloud/v1",
model="runaii/glm-5.3-flash",
messages=[{"role": "system", "content": BIG_STATIC_PREFIX},
{"role": "user", "content": question}])
print(resp.usage.prompt_tokens_details.cached_tokens) # tokens billed at $0.03
print(resp.usage.cost) # what your wallet was charged
Second call, same prefix: cached_tokens jumps to your prefix length and cost
drops accordingly. If it does not, your prefix is not byte-identical — check for
timestamps or counters injected into the system prompt.