Prompt caching
Reusing computation for the repeated prefix of prompts — system instructions, tool schemas, standing context — so consecutive calls pay only for what changed. For agent loops sharing a large stable prefix, cost and latency can drop dramatically.
Why agent loops are the ideal case
An agent turn repeats an enormous stable prefix — system prompt, tool schemas, skill text, conversation so far — and appends a little new material. Caching that prefix means consecutive calls pay full price only for what changed. Few workloads are shaped as favourably for it as an agent loop.
Prefix stability is the whole game
Caching works on exact prefixes, so anything that varies must live at the end. A timestamp injected into the system prompt, tools listed in a different order, or a memory block that reshuffles will all invalidate the cache on every call while looking harmless in the code. Order your context deliberately: most stable first, most volatile last.
Measure before you believe
Cache behaviour differs sharply between providers in minimum cacheable size, retention window and whether writes cost extra. The savings are real and sometimes dramatic, but they are an empirical property of your prompt shape against a specific provider — not a number you can assume.
In AGNT
AGNT’s cache-efficiency research measured exactly this across agent harnesses — see The Cache Wars whitepaper.
Prompt caching — common questions
How much can prompt caching save?
On long-prefix agent loops the reduction in prefix cost is often substantial, and latency usually improves alongside it. The exact figure depends on prefix size, call frequency and the provider’s pricing.
Does caching change the model’s output?
No. It reuses computation for identical input; the result is the same as an uncached call.
Has AGNT measured this?
Yes — cache efficiency across agent harnesses is the subject of the Cache Wars research published in the AGNT whitepapers.