How much does prompt caching save on LLM API costs?

As of September 2026 cached input tokens cost 10% of the uncached rate on OpenAI's GPT-6 models ($0.20 vs $2 on GPT-6 Sol, $1 vs $10 on GPT-6 Astra, $0.01 vs $0.10 on GPT-6 Luna), 5% on Claude Opus 5.5 ($0.20 vs $4), 2.5% on Claude Fable 5.1 ($0.25 vs $10) and 10% on Claude Sonnet 5 and Haiku 4.5. On agentic workloads where 90% or more of each request is repeated context, that cuts input cost by 80-90%. Anthropic says cache reads make up the majority of agentic and coding costs, and GitHub reports caching improvements halved the share of Copilot prompt tokens needing fresh processing.

Why is my prompt cache missing?

The common causes are a changed prefix: a timestamp or user name at the top of the system prompt, tool definitions added, removed or reordered between turns, a model or settings change mid-conversation, edits to earlier messages, dynamic content placed before static content, prompts below the minimum cacheable length (512 tokens on Claude Opus 5.5), or exceeding the time-to-live (30 minutes on GPT-6 by default; 5 minutes or 1 hour on Claude). OpenAI's prompt caching diagnostics tool returns a reason such as tools_changed with the number of tokens affected; Anthropic's usage fields report cache_read_input_tokens per request.

Does changing reasoning effort or tools break the prompt cache?

On GPT-6 models, no longer: since September 22, 2026 you can change reasoning effort between responses by appending a configuration_update, and toggle tools with allowed_tools or tool_choice none, without invalidating earlier cached context, as long as tool definitions, schemas and ordering stay in place. On Claude, adding or removing tool definitions changes the prefix and breaks the cache; use mid-conversation system messages to add instructions, keep tool definitions stable, and change effort via the per-message effort beta rather than editing history. Editing earlier turns also violates Claude's thinking-block binding on newer accounts.

What is cache prewarming?

Prewarming sends the static part of a prompt (system instructions, tool definitions, reference documents) to the API before the first user request so the prefix is already cached when the real request arrives. OpenAI documented it for GPT-6 in September 2026 as a way to move prefix processing out of the user's wait time, for example at application startup. On Claude the equivalent is a cheap request that writes the cache breakpoint ahead of traffic; the 1-hour TTL at 2x write cost is designed for this.

Quick Answer

How to Maximize Prompt Cache Hit Rate for AI Agents (2026)

Published: September 23, 2026

The short answer

Prompt caching is now the largest single lever on LLM agent costs: cached input costs 3-10% of list price across OpenAI and Anthropic, and agents re-send 90%+ of their context every turn. Hit rate is engineering, not luck: keep the prefix byte-stable, put static content first and dynamic content last, never mutate tool definitions or earlier turns, choose TTLs deliberately, and read the diagnostics both vendors now expose. This guide is provider-neutral with the September 2026 specifics for GPT-6 and Claude called out.

Step 1: Know your provider’s cache rules

	OpenAI GPT-6 (Astra, Sol, Luna)	Anthropic Claude (Opus 5.5, Fable 5.1, Sonnet 5)
Cached input price	10% of uncached	5% (Opus 5.5), 2.5% (Fable 5.1), 10% (Sonnet 5, Haiku 4.5)
Cache write price	1.25x uncached (e.g. $2.50 on Sol)	1.25x for 5-minute TTL, 2x for 1-hour TTL
TTL	30-minute reuse window (default)	5 minutes or 1 hour, refreshed on each hit
Minimum cacheable prefix	Provider-managed	512 tokens on Opus 5.5 (varies by model)
Breakpoints	Explicit breakpoints supported (new)	Explicit `cache_control` breakpoints
Effort change without miss	Yes, via `configuration_update`	Per-message effort (beta)
Tool toggling without miss	`allowed_tools` / `tool_choice: none`	Keep definitions stable; use `tool_choice`
Diagnostics	Prompt Caching Dashboard + diagnostics tool	`cache_read_input_tokens`, `cache_creation_input_tokens` in usage
Long-prompt catch	>272K input reprices whole request 2x	None

Verify these against the vendor pricing page before budgeting; both changed in September 2026.

Done when: you can state, for each model you use, the cache read price, write premium, TTL and minimum prefix.

Step 2: Order the prompt for stability

Caches match on a prefix. Anything that changes invalidates everything after it.

System instructions (static)
Tool definitions (static, fixed order)
Reference material: docs, schemas, few-shot examples (static)
Conversation history (append-only)
The current user turn and any per-request data (dynamic, last)

Remove timestamps, request IDs, user names and A/B flags from the top of the prompt; inject them in the final user message instead. Serialise tool schemas deterministically (sorted keys, no random ordering from a map). Done when: two consecutive requests for the same conversation produce byte-identical prefixes up to the new turn.

Step 3: Make history append-only

Both vendors now treat edits to earlier context as cache-breaking, and Anthropic additionally enforces it: on accounts created on or after August 31, 2026, replaying a Claude Opus 5.5 or Fable 5.1 thinking block after the system prompt, tools or an earlier message changed returns a 400 error. Practical rules:

Add instructions with mid-conversation system messages (Claude) or new developer messages appended near the end (OpenAI) instead of rewriting the system prompt.
Never delete or reorder tool definitions mid-conversation. On GPT-6 use allowed_tools to expose a subset or tool_choice: "none" when no tool is needed; on Claude leave the list intact and steer with the prompt.
Compact or summarise history only at defined checkpoints, then treat the summary as the new stable prefix; expect one cache write at each compaction.

Done when: your agent loop never mutates prior turns, tools or system prompt in place, and compaction events are logged.

Step 4: Place breakpoints deliberately

Explicit breakpoints tell the provider where a reusable prefix ends. Put one after the static block (instructions + tools + reference) and, on long conversations, one that advances with the history so recent turns are also cached. On Claude, a breakpoint below the 512-token minimum is ignored silently. On GPT-6, explicit breakpoints are new; use them when the dashboard shows less reuse than your prompt structure should allow. Done when: cache_read_input_tokens (Claude) or the dashboard’s cached share (OpenAI) exceeds 80% on steady-state turns.

Step 5: Choose TTLs by traffic shape

Interactive agents with turns every few seconds or minutes: the default 5-minute Claude TTL or the 30-minute GPT-6 window is enough; each hit refreshes it.
Slow human-in-the-loop workflows (approvals, reviews) where gaps exceed five minutes: use Claude’s 1-hour TTL. It costs 2x on the write but avoids a full re-write on every resume.
Batch or scheduled jobs: group requests that share a prefix into the same window, or use the Batch API (50% off on both vendors) where latency does not matter.

Done when: you have measured the median gap between turns per workflow and set TTLs accordingly.

Step 6: Prewarm before the first request

At startup, or when a session is created, send the static prefix once so the first real user request hits the cache. OpenAI documents this for GPT-6 as moving processing out of the user’s wait time; on Claude a minimal request that includes the breakpoint does the same. Prewarm again after deploys that change the system prompt or tools, since those invalidate every cached prefix at once. Done when: first-turn latency for a new session matches steady-state latency.

Step 7: Instrument and diagnose

OpenAI: watch the Prompt Caching Dashboard for hit rate over time and the input-composition chart for cached vs uncached tokens. When a miss surprises you, run the diagnostics tool against the request and its predecessor; it returns a reason (model_changed, tools_changed, settings or input changed) and cache_missed_tokens so you can size the leak.
Anthropic: log cache_creation_input_tokens and cache_read_input_tokens from every response. A healthy agent shows large reads and near-zero creation after turn one. Repeated creation on every turn means a moving prefix.
Both: alert on hit-rate drops after deploys; a stray timestamp in a template is the classic regression.

Done when: cache hit rate is a dashboard metric with an alert, not a quarterly discovery.

Step 8: Recompute the real cost

With an 80-90% hit rate, list prices stop describing your bill. Example: a 30K-token agent turn with 90% cached on Claude Opus 5.5 costs 27K × $0.20 + 3K × $4 = $0.0174 in input, versus $0.12 uncached, an 85% reduction before output. The same turn on GPT-6 Sol costs 27K × $0.20 + 3K × $2 = $0.0114. This is why identical $0.20 cache-read prices make Opus 5.5 and GPT-6 Sol closer on long sessions than their $4 vs $2 list prices imply; see Opus 5.5 vs GPT-6 Sol. For how caching combines with batching and compaction, see Prompt cache vs Batch API vs compaction.

Common mistakes

Mistake	Symptom	Fix
Timestamp in system prompt	0% hit rate	Move to last user message
Tools serialised from an unordered map	Intermittent misses	Sort keys; freeze order
Rewriting the system prompt to add rules	Miss every time rules change	Mid-conversation system / developer messages
Prefix under minimum length	Breakpoint ignored	Pad static reference above 512 tokens on Claude
Compacting history every turn	Constant cache writes	Compact at checkpoints only
Switching models mid-session	Full miss plus dropped thinking blocks on Claude	Pin the model per session
>272K prompts on GPT-6	Cache read priced at 2x	Chunk or move to a no-threshold model

Last verified: September 23, 2026.