Bedrock Prompt Caching: the Cost and Latency Math

The pricing math: writes cost more, reads cost 90 percent less

Send a 10,000-token contract to the model alongside every one of 50 reader questions and you have bought the same 10,000 tokens fifty times over: half a million input tokens for material the model rereads on each call. Prompt caching changes the billing units to stop that bleed. Standard input and output tokens are joined by two new categories, and the whole business case lives in the spread between them. AWS first laid out the scenarios and the pricing structure in a September 2026 post on the AWS Machine Learning Blog; the figures below are cross-checked against the Bedrock documentation and the Bedrock pricing page.

Token category	Where it appears	Cost vs standard input
cacheWriteInputTokens	First request that populates an entry	25 percent higher (1.25x)
cacheReadInputTokens	Subsequent prefix-matched requests	90 percent lower on Claude and GPT models, 75 percent lower on Nova
cacheWriteInputTokens, 1-hour TTL	First request with ttl set to 1h	100 percent higher (2x)

Now the worked example, using a 10,000-token document asked 10 different questions, all inside the TTL window. Without caching you pay for 10 full passes: 100,000 token-units of input. With caching you pay one write at 1.25x plus nine reads at 0.1x: 12,500 plus 9,000 equals 21,500 token-units. That is a 78.5 percent reduction; the AWS post quotes approximately 75 percent for the same shape. The saving grows with the question count and shrinks whenever an expired cache forces a fresh write.

Latency moves in the same direction as cost. Because the cached prefix skips prefill computation, time to first token drops, and the effect strengthens as the prefix grows. The AWS post’s measurements show the gain becoming pronounced beyond roughly 10,000 cached tokens, while documents in the 2,000 to 5,000 token range can be too small for the difference to clear measurement noise.

What prompt caching actually does

Every long prompt makes the model redo the same prefill work each time it arrives, and you pay full input price for every repetition. Prompt caching is Bedrock’s answer: the service keeps the computed state of a prompt prefix warm, and the next request that opens with those exact tokens continues from that state instead of recomputing it.

Bedrock offers the feature in two flavors. Implicit caching needs no markers: the service tries to reuse eligible prefixes automatically, best effort, with no guarantee that repeating a prompt produces a hit. Explicit caching puts you in control. The marker itself sits where fixed material ends and varying material begins; Bedrock then treats everything ahead of that cachePoint as cacheable. Explicit is what the rest of this tutorial uses, because production cost math needs predictable behavior.

The feature works through the Converse and ConverseStream APIs, through InvokeModel, and alongside cross-region inference. The syntax is model-agnostic: the same cachePoint block works across Anthropic Claude and Amazon Nova families that support explicit caching. One boundary worth flagging: cached prefixes are an on-demand endpoint behavior, and batch jobs have no way to consume them.

Support is not a blank check. A cache checkpoint placed before your prompt prefix reaches the model’s minimum token count is silently ignored: inference succeeds, nothing is cached, and you never see an error. The minimums matter enough to deserve their own table later in this post.

Break-even: when caching wins and when it quietly loses

The write premium is the whole risk profile of prompt caching. Pay 25 percent more once, then 90 percent less on every later request, and the arithmetic runs heavily in your favor: two uses of a prefix cost 1.35 token-units against 2.0 uncached, a 32.5 percent saving on the second request alone. Pay 25 percent more on every request because nothing ever hits, and you have built a machine that makes your prompts more expensive.

The deciding variable in prompt caching is cadence against TTL. The default TTL is 5 minutes, and it resets on every successful hit at no additional charge. A workload that reuses the prefix more often than every 5 minutes writes the cache once and then keeps it alive indefinitely on free refreshes. A workload that asks once every 20 minutes never hits: each request pays the 25 percent write premium on a cache that dies before the next question arrives.

The 1-hour TTL exists for exactly that gap, at double the write cost. Three uses within the hour cost 2.0 plus 0.1 plus 0.1, or 2.2 token-units per prefix-token, against 3.0 uncached: a 27 percent saving. Two uses cost 2.1 against 2.0, a marginal loss. So the rule of thumb reads: two or more reuses inside 5 minutes, use the default; three or more inside an hour, use the 1-hour TTL; fewer, restructure the prompt or skip caching for that content.

Two secondary effects can tip a borderline case. Cache hits do not draw down the per-minute input token allowance, so caching also buys throughput headroom on bursty agentic loops. And cross-region inference can quietly add writes: each call goes to whichever Region the router prefers inside your geography, so under heavy demand consecutive requests can land in different Regions and each rebuilds the prefix on its own. Pin the region or accept the writes when the prefix is expensive enough.

How does the TTL sliding window actually behave?

The TTL behaves like an idle timeout, not a hard lifetime. Each successful cache hit both reads the entry and restarts its clock, and that refresh is free: no write tokens are billed for keeping an entry warm. An entry only expires when a full TTL passes with no matching request.

This is why the Bedrock documentation recommends staying on the 5-minute TTL for anything used more often than every 5 minutes: the entry never dies, and you never pay the premium twice. The 1-hour TTL is for prompts used less often than every 5 minutes but more often than hourly, for long-running agentic sessions, and for follow-ups that might land past the 5-minute mark while latency still matters. Claude Fable 5.1, the Opus line from 4.5 up, Sonnet from 4.5 up, and Haiku 4.5 support the 1-hour option; GPT-5.6 models on Bedrock use a fixed 30-minute TTL with no short option.

System prompts and tool definitions cache too

The most stubborn repeat offenders in production are not documents but the two surfaces that ship with every request: the system prompt and the tool list, and both are prime prompt caching targets. Persona-heavy system prompts, the ones with role definitions, guardrails, and embedded policy, run to thousands of tokens and never change between turns. Tool definitions in agentic applications, dozens of JSON schemas strong, are equally static. Each takes the marker in a different field than messages do.

For the system prompt, the system parameter accepts an array of blocks, and the marker goes after the text:

response = bedrock.converse(
    modelId=MODEL_ID,
    system=[
        {"text": SYSTEM_PROMPT},
        {"cachePoint": {"type": "default"}},
    ],
    messages=[{"role": "user", "content": [{"text": question}]}],
)

For tools, the marker is appended as the final element of the tools array inside toolConfig, after every toolSpec. The order is the entire trick: system prompt, then tools, then messages, each static layer sealed off with its own checkpoint where needed. A typical agent request caches all three, so the persona, the toolbox, and the working document are each reused across turns while only the latest user input is processed fresh.

Caching a document: the first cachePoint

The document-plus-questions pattern is the canonical prompt caching case for RAG: one long retrieved chunk, many queries against it. Structure the content array so the static document comes first, the marker sits between it and the question, and only the question changes between calls.

import boto3, json, time

bedrock = boto3.client("bedrock-runtime", region_name="us-west-2")
MODEL_ID = "anthropic.claude-sonnet-4-5-20250929-v1:0"

def ask(document, question):
    content = [
        {"text": document},
        {"cachePoint": {"type": "default"}},
        {"text": question},
    ]
    response = bedrock.converse(
        modelId=MODEL_ID,
        messages=[{"role": "user", "content": content}],
        inferenceConfig={"maxTokens": 512},
    )
    return response["usage"]

usage = ask(DOCUMENT, "What are the key points?")
time.sleep(1)
usage = ask(DOCUMENT, "Summarize the conclusions.")

The first call writes the cache. The second, with a different question but the same document prefix, reads it. The usage object tells you which happened, through two fields that exist precisely for this audit: cacheWriteInputTokens and cacheReadInputTokens. In the AWS team’s published test against Claude Sonnet 4.5 with a document above the 1,024-token minimum, the first response reported 1,898 written and zero read; the second reported 1,898 read, zero written, and just 28 tokens of standard input for the question itself. Those two numbers are how you verify the feature is earning its keep in production, because support never guarantees a hit.

Claude models add a convenience called simplified cache management. Mark the end of your static material with one cachePoint; Bedrock scans backward from it, testing prefix boundaries across roughly the last 20 content blocks, and keeps the longest match it finds. That 20-block lookback is a hard ceiling. Static material that runs longer needs either extra checkpoints to segment it, or a rewrite that puts the blocks you reuse most near the front, where the backward scan still reaches them. For finer control you can place up to four checkpoints in one request, one per logical section, and get partial hits when only the leading sections match.

How do you read cache metrics in the usage object?

On the synchronous Converse API the two fields arrive directly in response["usage"]. On converse_stream they arrive later, in the metadata event that closes the stream, so accumulate events until you see that key before computing hit rates.

The OpenAI models on Bedrock report prompt caching through the Responses API with different names: cached_tokens for reads and cache_write_tokens for writes, the latter billed at 1.25x. A fully matched request shows reads above zero and writes at zero, which is the signature of maximum savings. A request that shows writes on every call is telling you the prefix changed or the TTL expired between calls, and that diagnosis is the first place to look when the bill does not match expectations.

Mixed TTLs and tenant isolation

Real prompts are layered, and the layers age at different speeds. A product catalog might be stable for a day, session context for minutes. Prompt caching lets you tier them inside one request by giving each checkpoint its own TTL, under a single ordering rule that Bedrock enforces: stack the checkpoints from the longest-lived tier down to the shortest-lived.

content = [
    {"text": CORE_REFERENCE},
    {"cachePoint": {"type": "default", "ttl": "1h"}},
    {"text": SESSION_CONTEXT},
    {"cachePoint": {"type": "default", "ttl": "5m"}},
    {"text": question},
]

The other production wrinkle is isolation. Bedrock ties every cache entry to a single account and Region; inside that boundary, requests that have nothing to do with each other can still share one entry. If two tenants of a multi-tenant application send identical instruction prefixes, they can legitimately hit the same entry, which is harmless for cost and wrong for compliance expectations. The pattern AWS documents is a prefix hash: prepend the hex digest of a SHA-256 over the tenant identifier to the cached content. The hash changes the prefix, so each tenant gets a separate cache entry, at a cost of about 16 tokens.

import hashlib

def tenant_prefix(tenant_id):
    return hashlib.sha256(tenant_id.encode()).hexdigest() + ":"

instructions = tenant_prefix(tenant_id) + INSTRUCTIONS

Teams standardizing on LangChain get the same mechanics without raw API handling: ChatBedrockConverse.create_cache_point() in langchain_aws emits the well-formed cachePoint block wherever you would have written it by hand, inside message content arrays or prompt templates.

Where prompt caching goes wrong

Most prompt caching disappointments trace back to four mistakes. Each is avoidable with a check you can automate against the usage fields.

• Prefix drift. A timestamp or a reordered document in front of the checkpoint changes the prefix and turns every request into a miss. Audit anything that varies before the marker.
• Cadence slower than TTL. Requests spaced beyond the expiry pay the write premium every time and never read. Match the TTL tier to the inter-arrival time or stop caching that content.
• Below the minimum. A checkpoint on a prefix under the model’s floor, 1,024 tokens on current Sonnet models, is silently skipped. Size the prefix or move the marker earlier in the stack where system and tools count toward the same cumulative minimum.
• Expecting a guarantee. Both implicit and explicit caching are best effort. The usage fields are the only truth; alert on write-heavy patterns instead of assuming hits.

Pick your targets first

Not every prompt deserves a checkpoint. Profile before you instrument: pull the requests with the largest repeated prefixes, which in practice means RAG documents queried in bursts, agentic loops that resend system prompts and tool schemas every turn, and support assistants with policy-laden personas. Those three shapes cover most of the recoverable spend. Our companion piece on cutting RAG costs with query-aware compression tackles the same bill from the opposite end, shrinking what you send at all, and the two compose: compress the context, then cache what remains.

If even this instrumentation is more than a prototype needs, implicit prompt caching gives you a zero-code baseline on the models that support it: front load the parts of a prompt that never change, leave per request material for the end, and the service attempts prefix reuse with no markers. Hits are best effort and unmeasured, which is exactly why the explicit path earns its place the moment real money is involved.

The minimum token counts vary by model generation, so check the floor before you size a cache. Current values from the Bedrock documentation:

Model	Minimum tokens per checkpoint	Max checkpoints	TTL options
Claude Opus 5	512	4	5 min, 1 hour
Claude Sonnet 4.5, 4.6, 5	1,024	4	5 min, 1 hour
Claude Opus 4.8	1,024	4	5 min, 1 hour
Claude Opus 4.5 to 4.7	4,096	4	5 min, 1 hour
Claude Haiku 4.5	4,096	4	5 min, 1 hour
GPT-5.6 Sol, Terra, Luna	1,024	4	30 minutes

Each minimum is measured over the whole accumulated prefix, system, tools, and messages together, that sits in front of its checkpoint, and consecutive checkpoints may sit as close together as the content allows. The complete working notebooks for every pattern in this post, including the LangChain variants, live in the amazon-bedrock-samples repository on GitHub.

Does prompt caching help with rate limits and batch jobs?

Rate limits, yes: a cache hit on Bedrock draws down none of the per-minute input token allowance, which turns caching into a throughput tool as much as a cost tool when a bursty agent run fires many calls inside a single minute.

Batch jobs, no: cache hits work only on on-demand endpoints, and batch inference runs cannot draw on them. A batch workload that reuses prefixes needs a different lever, such as the batch discount itself or reducing context before submission. Cross-region inference sits in between: caching composes with it, but under high demand the routing optimization can land consecutive calls on different regions, each of which maintains its own cache, so expect elevated write volumes and pin regions when the prefix is expensive.

Prompt caching rewards you for knowing your own traffic. Measure the reuse, place checkpoints after the stable layers, pick the TTL tier that matches your cadence, and let the usage fields confirm the hits. Done right, the 25 percent write premium becomes the cheapest line item in the whole bill.