Caching

The gateway remembers the answer to a deterministic request and returns it again when the same request comes back. Repeats are faster, and are billed at the cached-input rate instead of running the model twice.

When a request is cached#

All of the following must hold. Otherwise the request goes to the model as usual.

It is not streamed.
temperature is 0, or the request carries X-Ahura-Cache: aggressive.
The model is served by a partner backend. Hosted models are not cached.
Neither X-Ahura-Cache: off nor Cache-Control: no-cache is set.

The cache key is a hash of model, messages, tools, tool_choice, response_format, max_tokens, temperature, top_p and seed. Anything else in the body, including user, is not part of it. Entries are scoped to your organization; another customer’s identical request never sees your answer.

Controlling it#

Header	Effect
`X-Ahura-Cache: off`	Skip the cache for this request, reading and writing.
`Cache-Control: no-cache`	Same as above, using the standard header.
`X-Ahura-Cache: aggressive`	Cache even when temperature is not 0. Use only where a repeated creative answer is acceptable.
`X-Ahura-Cache-TTL: 900`	How long, in seconds, the answer stays valid. Default 300; clamped to 60 to 3600.

a cacheable request with a 15-minute TTL

curl https://api.ahurasense.com/v1/chat/completions \
  -H "Authorization: Bearer $AHURA_API_KEY" \
  -H "X-Ahura-Cache-TTL: 900" \
  -H "Content-Type: application/json" \
  -d '{"model":"openai/gpt-5.4-mini","temperature":0,"messages":[{"role":"user","content":"Classify: \"card declined twice\""}]}'

Reading the result#

Every non-streamed response says what the cache did:

X-Ahura-Cache	Meaning
`hit`	Served from the cache. `X-Ahura-Cache-Age` is the age in seconds.
`miss`	Eligible, not found; the answer was generated and stored.
`non-deterministic`	Not cached because temperature is above 0.
`bypass`	You asked to skip it.
`streaming-skipped`	Streamed responses are never cached.

Billing#

A hit is recorded as a request whose prompt tokens were all cached, so the prompt is billed at the model’s cached_cents_per_mtok rate rather than the input rate; the completion is billed at the output rate as usual. The usage row carries cache_kind: l1 so you can see the effect per key on the usage page.

Privacy#

A cached entry is the response body, its content type and its token counts, stored for at most the TTL and never longer than an hour. Keys with zero data retention never read or write the cache. See Privacy & data retention.

Something missing or wrong on this page? Tell us, and quote the page title.