Same system prefix, zero cache hits on the branching path: debugging GPT-5.6 prompt caching

也有中文版。

TL;DR

The problem. In one agent, the main conversation hits prompt cache 54.9% of the time on the first call of a turn. Another path that shares the exact same system prefix hits 0 out of 24 times. The system prefix both paths send is byte-identical — 56,454 characters of it.

The cause. OpenAI caches an entry: a prefix with a start and an end. Without a breakpoint, the API decides where the entry ends, and that boundary falls before the last message of the request. So requests that extend the previous one match; requests that branch off the same shared prefix don’t — and a miss is the whole request at full price, not a partial saving.

The fix. Mark an explicit breakpoint at the end of the shared prefix.

// attached to message[0] — the system prefix
responses.ResponseInputTextParam{
	Text:                  systemPrefix,
	PromptCacheBreakpoint: responses.NewResponseInputTextPromptCacheBreakpointParam(),
}
params.PromptCacheOptions = responses.ResponseNewParamsPromptCacheOptions{Ttl: "30m"}
params.PromptCacheKey     = openai.String(hashOf(systemPrefix, tools))

Three things go together:

1. prompt_cache_breakpoint at the end of the shared prefix — an A/B that isolated the two fields showed this alone takes the branch from 0% to 96.7%
2. prompt_cache_options.ttl replaces prompt_cache_retention — the latter is deprecated on GPT-5.6+; sending it neither errors nor does anything
3. prompt_cache_key set to a hash of the prefix — the docs say GPT-5.6+ requires it, though at close range I couldn’t measure any effect

Leave mode at the default implicit. Implicit keeps the API’s own breakpoint (which serves extensions) and honors yours (which serves branches). Setting explicit removes the former — you’d be trading extensions for branches.

Two other GPT-5.6-only things. Cache writes now cost 1.25× uncached input instead of being free (reads are still 0.1×, so 0.28 reuses breaks even). And prompt_cache_retention only accepts 30m.

The starting point had nothing to do with caching

I run a personal-assistant agent on my own machine. The question that day was whether a sentence fired in from a phone shortcut would pollute the context of the conversation I had going.

It doesn’t. That was quick to establish. But while reading the log I noticed something that shouldn’t be there: on the path the phone uses (call it capture), the first model call of every turn had cached_tokens at 0.

Twenty-four times. No exceptions.

Background: why there are several paths

The agent has four paths, each solving a different problem.

Fast path is the cautionary tale here. Most things you hand an assistant are small — “add a meeting at 10 tomorrow” — and running those through the main model with the full toolset and full history is slow and expensive. So there’s a cheap classifier in front of main that decides whether a sentence is self-contained or needs the history. Small ones go to a throwaway session: cheap model, a handful of hand-picked tools, blank conversation, discarded after use.

It was measured at launch: on a short-command-dominated workload, cost drops close to half — but the saving is the model-tier gap (small things run on a cheap model), not fewer tokens. A same-model comparison across the two paths came out roughly flat: fast path chops small work into disposable contexts, never touches main’s deep cache, and ends up sending 2.5× more full-price tokens. That fact comes back later.

Capture exists for a different reason. If phone input landed in main, you’d be discussing a technical problem and a “sleep report: six hours last night” would drop into the middle of it — messy on screen, and dirty in the model’s context. So capture also opens a disposable blank conversation, but with the full toolset and the main model, because filing something properly needs full capability.

Here’s the part that matters: capture and main share the same system prefix. Same builder object, same string. That was deliberate, specifically so it would hit cache.

So a 0% hit rate is strange.

First pass at the numbers

The log is one JSON object per line, one record per model call, carrying prompt_tokens, cached_tokens, origin (which path), and iter (which call within the turn).

Overall first:

import json, collections
tot = collections.defaultdict(lambda: [0,0,0])  # calls, prompt, cached
for line in open('calls.jsonl'):
    r = json.loads(line)
    o = r.get('origin') or 'main'
    t = tot[o]
    t[0] += 1; t[1] += r['prompt_tokens']; t[2] += r.get('cached_tokens', 0)
for o, t in sorted(tot.items(), key=lambda x: -x[1][1]):
    print(f'{o:12} {t[0]:5d} calls  hit {t[2]/t[1]*100:5.1f}%')

main          1715 calls  hit  76.6%
capture         65 calls  hit  61.4%
fastpath       579 calls  hit  54.3%
task           147 calls  hit  81.9%
wake           209 calls  hit  12.9%
gmail          129 calls  hit   1.3%

61% isn’t low. But split capture by turn and the shape shows up:

2026-07-16T18:04  calls=3  cached: [0, 22073, 22112]
2026-07-17T08:22  calls=3  cached: [0, 20252, 20291]
2026-07-18T11:29  calls=5  cached: [0, 0, 20608, 23004, 23287]
2026-07-19T20:30  calls=6  cached: [0, 22703, 24431, 24470, 24546, 24706]
...

estimated turns: 24
turns whose first call missed: 24 / 24
later calls in the same turn: 41   of which missed: 1

Full price on the first call of every turn, near-perfect after. That 61% is entirely carried by the later calls.

My first thought was expiry. The times things arrive from the phone — 7am, 1am, the daily 10pm sleep report — are times when main has often been idle for hours. Reasonable.

Then these two showed up:

2026-07-24T10:00  cached=0   previous call was main, 47 seconds earlier
2026-07-24T13:25  cached=0   previous call was main, 84 seconds earlier

Forty-seven seconds. Expiry doesn’t explain that.

Getting “turn” right, or the numbers mean nothing

I got this wrong first.

To ask “did the first call of each turn hit,” you need to know which records belong to the same turn. One turn is: the user says something, the model says it wants a tool, the tool runs, the model is called again — usually three to six records per user sentence.

The first script guessed by time: a gap over 120 seconds means a new turn.

time →
09:00:00  09:00:00  09:00:00  │  09:01:20  09:01:20
 call1     call2     call3    │   call1     call2
  └───── real turn A ─────┘   │  └─ real turn B ─┘
                       80s gap → script says "still turn A"

Reading the code that writes the records, it turns out main stamps every record at the end of the turn — all three calls in a turn share one timestamp. Within a turn the gap is always 0 seconds, which groups correctly. But two adjacent turns get glued together whenever you type a second sentence within 120 seconds, which is common — and the second turn’s first call gets counted as “a later call of the previous turn.”

Later calls almost always hit. Every glue event removes one genuine miss from the bucket I was measuring.

The records already carry iter. 0 is the first call. No guessing needed:

m = [r for r in rows if r.get('origin') == 'main']
first = [r for r in m if r.get('iter') == 0]
hit = sum(1 for r in first if r.get('cached_tokens', 0) > 0)
print(f'first-of-turn {hit}/{len(first)} = {hit/len(first)*100:.1f}%')

first call of a turn   190/346 = 54.9%
later calls in turn    474/491 = 96.5%

Population: origin=main records from 2026-07-11 to 07-24. This also surfaced a problem — 700-odd older records predate the origin field and had been counted as main all along. Dropping them is what produces the number above.

Bucketing the first calls by “how long since the previous main turn”:

112+54+95+84 = 345, i.e. the 346 first calls minus the very first one, which has no predecessor. Hits total 190, matching above.

It’s a decay curve, fine. Except the code sets PromptCacheRetention: 24h — a full day of retention — while the observed half-life is on the order of ten minutes.

One caveat: iter is always 0 on the side paths; only main increments it. So the side-path “first of turn” numbers below have to be grouped by time. That’s safe here because side-path turns are far apart and can’t glue together.

Pulling every case of “first call of a non-main turn, with main having run the same model within ten minutes”:

task    3s after main   cached=0
task    4s after main   cached=0
wake    2s after main   cached=0
wake    5s after main   cached=0
capture 47s after main  cached=0
...
total: 0 hits / 44 misses

Zero out of forty-four, including cases two seconds apart. Main-to-main at the same moment is 95%.

At that point I concluded cross-path cache sharing was structurally impossible. That conclusion was wrong.

Seven hypotheses, eliminated one by one

Every way the two paths could differ in what they send, killed one at a time. Each one was handed to Claude Code: write the probe, run it, paste the raw output back.

1: the prefix isn’t byte-identical

Swap the provider for a fake one that just records what it receives, run one main turn and one capture turn, and print the actual message arrays (content replaced with placeholders):

MAIN    first call: 6 messages
   [0] system  56454 chars  <SYSTEM_PREFIX>
   [1] system     78 chars  <reminder: day_state>
   [2] system    211 chars  <reminder: toolbox>
   [3] system    624 chars  <reminder: tree_index>
   [4] system     60 chars  <reminder: desk>
   [5] user       28 chars  "Reply with just the word OK."

CAPTURE first call: 6 messages
   [0] system  56454 chars  <SYSTEM_PREFIX>
   [1] system   2692 chars  <reminder: capture>
   [2] system     78 chars  <reminder: day_state>
   ...

message[0] IDENTICAL = true

56,454 characters, exact. Divergence starts at message 1. Dead.

2: the tool array differs

Tool definitions count as part of the cached prefix, and tool visibility varies per session. Hash both lists:

advertised specs  main=63 tools  capture=63 tools
  main    50b39fd4fd356606f0faa7a9
  capture 50b39fd4fd356606f0faa7a9

Identical. Dead.

3: the model doesn’t serve partial prefixes at all

The most plausible guess at the time. Main stores “prefix plus a long history”; capture only wants the leading segment. Maybe only full extensions are served.

A small script that bypasses the app and hits the API directly, four calls:

1 LONG  (cold, writes the entry)      prompt= 18493 cached=     0 (  0.0%)
2 SHORT (shares only the prefix head) prompt= 13036 cached= 13022 ( 99.9%)
3 LONG  again (control)               prompt= 18493 cached= 18490 (100.0%)
4 SHORT again (control)               prompt= 13036 cached= 13033 (100.0%)

99.9% on call 2. Partial prefixes are served fully, effective within three seconds. Dead — and now I was more confused.

4 to 6: tools, streaming, reasoning effort

If a raw API call hits and the real program doesn’t, the difference must be in what the real program does on top. Add them to the script one at a time:

All held. Three dead at once.

7: the system prefix happens to be rebuilt when capture arrives

The system prefix rebuilds on two events: clearing the conversation, and the daily midnight rollover. The log’s history_start field jumps on a clear, so check it on the main calls surrounding each capture:

prefix unchanged around capture: 20   rebuilt in between: 4

Twenty of twenty-four unchanged, still 0%. Effectively dead.

Then the thing I should have done first

After all that, it occurred to me to reproduce it cleanly in the real program. Full app in a temp environment, run one main turn, wait three seconds, then fire a capture through the actual phone-shortcut injection path:

origin      prompt  cached   rate
main         23897       0   0.0%   ← cold, expected
main         23916   23894  99.9%   ← control: second main turn extends, hits
capture      24474       0   0.0%   ← branches three seconds later, still 0
capture      24707   24471  99.0%   ← only hits its own previous call

Reproduced. Same process, three seconds apart, byte-identical prefix, identical tools. The same shape hitting the API directly gives 99.9%; the real program gives 0%.

Every variable I could think of was aligned and the gap was still that large. One road left: have Claude Code read the entire official documentation.

Three sentences in the docs

All three are in the official docs, and I’d missed all three in two years of use.

One: this field is required on newer models

On GPT-5.6 models and later model families, you must set prompt_cache_key to use the more reliable matching for both implicit and explicit caching.

Set prompt_cache_key on requests that share long, common prompt prefixes. Reuse the same key for those requests to help route them to the same cache and improve cache hit rates.

My mental model of prompt caching had always been “it only looks at the prefix.” That’s true, but a layer short.

The cache lives in one machine’s memory, not in a shared database everyone can query. So a hit needs two things at once: the prefix content matches, and this request happens to land on the machine that has it.

                    ┌─ machine A ───────────┐
                    │ remembers [prefix 13k] │
your request ─▶ LB ─┼─ machine B ───────────┤
                    │ has nothing            │
                    └─ machine C ───────────┘
                        has nothing

prefix decides: if you land on A, is there something to match
routing decides: whether you land on A at all

prompt_cache_key is a string you make up whose only job is “route requests with the same string to the same machine.” It isn’t the cache’s name — you can’t use it to fetch a specific cache, and a made-up key won’t make different content match. The docs say help route, not identify cache.

That’s sensible, and at the time it looked like the answer. A later A/B disproved it: routing was not the cause here.

Two: the one that actually fixed it

prompt_cache_breakpoint — Marks the exact end of a reusable prompt prefix. The breakpoint inherits its TTL from the request’s prompt_cache_options.ttl; the boundary is not rounded to a token block.

I didn’t know this existed, and the variable-isolating A/B afterwards confirmed it’s what fixed the problem.

Terminology first. What’s cached isn’t “a pile of tokens you can truncate anywhere” — it’s an entry: a prefix with a start and an end. To hit, some entry’s entire content must equal the leading bytes of the new request. The breakpoint is what decides where the entry ends.

By default (the docs call it implicit mode) the API places exactly one breakpoint. Where, the docs don’t say. Measured, the matchable boundary is “before the last message.”

Careful with the wording. What the measurement supports is “later requests can only pick up as far as this point,” not “only that much was stored” — cache_write_tokens in the same output shows the write covers nearly the whole prompt (15,196 sent, 15,193 written). So the whole thing is probably written; only the part up to the point that definitely can’t change any more is matchable. “The entry ends at” below is shorthand for that matchable boundary.

Call the system prefix Sp and each subsequent block A, B, C:

send Sp + A + B + C           (C is the last message)
                ↑ implicit breakpoint here
stored entry: one, content = Sp+A+B   (no C)

later: Sp + A + B + C + E     (extension)
       └─ leading Sp+A+B matches ─┘         → hit

later: Sp + D                 (branches at Sp)
         no entry ends at Sp                → full price

later: Sp + A + E             (branches at Sp+A)
         no entry ends at Sp+A either       → also full price

later: Sp + A + B + E         (branches at Sp+A+B)
         exactly the entry boundary         → hits Sp+A+B

That rule is measured, not documented. One probe, three cases: write the entry with [Sp][A][B][user C], then query each divergence point from its own cold prefix (none of them sending prompt_cache_key, so this measures default behavior):

The writing call was 15,196 tokens total; the hit is 15,182 — the missing 14 are that final user message.

To be honest about it: each of the three cases ran once. On its own, the 0% for Q1/Q2 could be “this call happened to land on another machine.” Two other datasets support the boundary reading: the A/B below ran the same divergence point twice, both 0, and showed that sending the routing key changes nothing.

Why that position is so damaging

Because the entry’s boundary isn’t yours to choose — it’s determined by the shape of whatever request happened to write it. The same Sp gets cut at different points depending on the request around it:

The richer the main conversation’s shape gets (another environment card, another reminder), the further back the boundary moves and the less the side paths can reach. And the main conversation’s shape grows as the product grows.

Extension versus branch

Which is why in one system some requests are permanently healthy and others permanently 0:

A system with a single conversation stream will never discover this, because every request it makes is an extension. The moment a second path branches from the same head, it’s hit across the board.

And a miss is the whole request

Not “you save in proportion to what you share.” Sp+A+E already shares Sp+A with the entry — over fifteen thousand tokens — but the boundary doesn’t line up, so the call is recomputed from token zero. Off by one costs the same as off by everything.

After marking an explicit breakpoint

[tool defs + Sp]▮[whatever each path adds]
                ↑ explicit breakpoint: entry ends here
stored entries: two — one ending at the breakpoint, one ending before the last message

Sp + D      → recovers Sp
Sp + A + E  → recovers Sp (A is still full price unless you mark after it too)

Mode stays at the default implicit. From the docs:

With implicit, OpenAI creates one implicit breakpoint and writes up to the latest three explicit breakpoints in the request. With explicit, OpenAI does not create an implicit breakpoint and writes up to the latest four explicit breakpoints.

Three: a field I’d been sending that does nothing

For GPT-5.6 models and later model families, the only supported value is 30m, which is also the default.

The hardcoded PromptCacheRetention: 24h is an older-model feature. The new family only takes 30m, and the field as a whole is deprecated in favour of prompt_cache_options.ttl.

Production requests never failed because of it, so it’s either ignored or tolerated. Either way it wasn’t extending anything. Worse, I’d been reasoning from it — assuming the cache lived a full day, which is exactly where the earlier “decay curve doesn’t match the setting” confusion came from.

For what it’s worth, someone on the community forum reported the identical shape: in a multi-agent system, an orchestrator branching off a static prefix gets 0%, while other agents in the same setup that extend get 71–89%. They’d SHA256’d the prefix and set the cache key. No official reply on that thread. Another thread is titled, straightforwardly, “Prompt Caching Is a Core GPT-5.6 Feature. Why Are Customers Still Reverse-Engineering It?”

After the fix

First, an SDK bump: the version in use had none of the three fields; they appear seven minor versions later (v3.39.0 → v3.46.0). Zero compile errors, two test files needing new parameters.

Three changes:

// routing key: hash of message[0] plus the tool names.
// Paths sharing a prefix naturally compute the same key; paths with their own
// prefix naturally compute a different one.
params.PromptCacheKey = openai.String(promptCacheKey(msgs, tools))

// cache policy, replacing the deprecated retention field
params.PromptCacheOptions = responses.ResponseNewParamsPromptCacheOptions{Ttl: "30m"}

// explicit breakpoint attached to message[0]
toResponsesInput(msgs, cacheBreakpointIndex(msgs))

The breakpoint takes one extra step. Messages were built with the string-taking constructor, which can’t carry attributes, so it has to become the content-list form:

func breakpointMessage(text string, role responses.EasyInputMessageRole) responses.ResponseInputItemUnionParam {
	content := responses.ResponseInputMessageContentListParam{{
		OfInputText: &responses.ResponseInputTextParam{
			Text:                  text,
			PromptCacheBreakpoint: responses.NewResponseInputTextPromptCacheBreakpointParam(),
		},
	}}
	return responses.ResponseInputItemParamOfMessage(content, role)
}

Verified with the reproduction test above, plus one critical detail: write a line stamped with the current nanosecond first, forcing a cold prefix that no earlier run could have warmed. Without that step the test reports a false pass.

But which one fixed it?

Two things shipped together and the number went 0% → 96.5%. That sentence doesn’t say who deserves the credit. So: copy the production request builder, turn the two cache mechanisms into parameters, pin everything else.

func abParams(p *OpenAIProvider, msgs []Message, tools []ToolSpec,
	useKey, useBreakpoint bool) responses.ResponseNewParams {

	bp := -1
	if useBreakpoint {
		bp = cacheBreakpointIndex(msgs) // breakpoint on message[0]
	}
	params := responses.ResponseNewParams{
		Input: responses.ResponseNewParamsInputUnion{
			OfInputItemList: toResponsesInput(msgs, bp),
		},
		// model / store / verbosity / reasoning / ttl all identical to production
	}
	if useKey {
		params.PromptCacheKey = openai.String(promptCacheKey(msgs, tools))
	}
	return params
}

Four configurations, three calls each:

MAIN    = [Sp][short day_state][user]        ← cold, writes the entry
   ↓ wait 3s
CAPTURE = [Sp][2,704-char card][user]        ← branches at Sp (production's shape)
   ↓
LATE    = [Sp][same day_state][other user]   ← branches at Sp+A (extra control)

LATE isn’t a production shape; it’s there to test whether sharing one more block changes anything.

Two anti-contamination measures: each cell uses its own prefix (a marker carrying a nanosecond runID appended to the tail), so the four configurations can’t read each other’s entries; and the whole matrix runs twice, because without a key routing is best-effort and a single run is noise.

The two rounds agree exactly. The breakpoint is what fixed it; in this rig prompt_cache_key contributed zero tokens. And the hit tops out at 15,134 against MAIN’s 15,171 for the whole call — the 37-token difference is MAIN’s own day_state plus user message, which is not over-fetched.

This also kills the machine-affinity story from earlier: main is healthy not because it stays pinned to one machine, but because every one of its calls is an extension that happens to cover the implicit entry.

prompt_cache_key stays, with an honest caveat: this rig is far too kind to it — same process, three seconds apart, low ambient traffic, so two calls would tend to land together anyway. What it’s meant to solve is routing divergence across hours, across paths, with other traffic interleaved. Only production numbers will show that. So the correct claim is “no measurable effect in this setting,” not “it does nothing.”

The “contradiction” wasn’t one

Hypothesis 3’s small script had no breakpoint and its branch hit 99.9%. The A/B’s “neither” cell also had no breakpoint, and its branch was 0%.

The difference isn’t the breakpoint. It’s the message count:

The small script had two messages, so its entry ended at Sp itself, and its branch was exactly at Sp — hit. Real capture’s MAIN has three or more, so the entry ends at Sp+A while capture branches at Sp: one block early, 0%.

I had been comparing two shapes that differed in message count the entire time.

All of this was traceable because every call is accounted for

Every number above came from one calls.jsonl. That machinery has been there for over a year. Four layers.

Collect: the provider hands up four numbers

When a call finishes, the provider layer passes usage up untouched: prompt / cached / written / output. This layer does no arithmetic, no guessing, no rounding. Anything above it that wants to know the cost has to derive it from those four numbers.

This investigation incidentally found that written was never wired: the SDK provides InputTokensDetails.CacheWriteTokens; our Usage struct didn’t read it.

Store: one record per call, append-only JSON Lines

One model call, one record — not one turn, one record. That’s the most consequential decision in the whole design. A turn can call the model six times (a few tool round-trips); if you only record the turn total, the question “what’s the first-of-turn hit rate” cannot be asked at all — and that question is the entire entry point of this post.

Fields on each record:

Labels have to be stamped at write time; they can’t be backfilled. This investigation hit both sides of that rule.

The positive side is history_start. Hypothesis 7 — was the system prefix rebuilt right when capture arrived — was answerable straight from the data, because that field was recorded at the time.

The negative side, twice. origin was added later, so 700-odd earlier records don’t have it and that stretch of data now has to be discarded wholesale. And iter only increments for main — every side-path record writes 0 — so “which call of the turn is this” still has to be guessed from timestamps on the side paths. This time the guess was safe (side-path turns are far apart). Next time it might not be.

Compute: cost isn’t stored, it’s derived at read time

Each record stores tokens only, never a dollar amount. Cost is computed when you read from a model → rates map:

type rates struct{ inputPerM, cachedPerM, outputPerM, writeMult float64 }

func cost(input, cached, written, output int, r rates) float64 {
	return float64(input-cached)/1e6*r.inputPerM +
		float64(cached)/1e6*r.cachedPerM +
		float64(written)/1e6*r.inputPerM*(r.writeMult-1) +
		float64(output)/1e6*r.outputPerM
}

The benefit is that a price change touches no data: adding a model is one line in the map, and past records automatically read at the new rates.

But there’s one thing it can’t save here, worth being explicit about: written was only wired up now, so older records have no such number and read back as 0. Two weeks of history don’t retroactively get more expensive just because writeMult exists — the recomputed monthly figure below is an estimate using uncached input as a stand-in for write volume, not a measurement. An accurate number has to wait for new records.

Which is the flip side of the previous section: a field you didn’t collect can only ever be estimated afterwards.

Query: one scan answers it

Because it’s append-only JSON Lines with complete labels, every question is under ten lines of Python:

rows = [json.loads(l) for l in open('calls.jsonl')]

# 1. per-path overall hit rate  → group by origin
# 2. first-of-turn vs later     → split on iter == 0
# 3. bucket by gap since last   → diff timestamps of adjacent iter==0

Without iter you’re back to guessing turn boundaries from timestamps, which is exactly where this went wrong earlier. What you can ask is decided at write time.

After GPT-5.6, there’s one more term in the cost

Before 5.5: cache writes were free. You paid for uncached input + cached input + output.

From 5.6:

For GPT-5.6 models and later model families, cache writes cost 1.25× the uncached input token rate.

An example. A 10,000-token prefix, uncached input priced at $1 per million tokens:

Only the first send costs more, by 25%; reads are still 0.1×. So a cache pays for itself after 0.28 reuses — in other words, the first reuse already puts you ahead.

Three shapes, measured:

The middle row answers a worry: does adding an explicit breakpoint mean writing two copies every time? No — the branching call wrote 7 tokens.

One trap in the pricing function: written tokens were already paid for at full rate inside the “uncached input” term, so you may only add the 0.25 differential (r.writeMult-1 in the code). Charging the full 1.25 double-bills. Unit tests in both directions pin that down.

With the term added, two weeks recomputed: $23.18/month on the books → about $26.65/month actual, $3.47/month that was never recorded. As noted above, old records carry no write volume, so this is an upper-bound estimate from uncached input.

The final numbers

Breaking down two weeks of first-call misses shows what the change can actually recover:

156 calls, $4.83 over two weeks, about $10.35/month, 39% of the bill. That’s a ceiling — only the third category is certain.

Conclusion

The mechanism. A cache hit needs three things at once: the prefix matches byte for byte, the request reaches the machine holding that cache, and the segment you want ends on a matchable boundary. Without a breakpoint the third is the API’s decision, and that boundary falls before the last message.

Who gets hit. Systems with a single conversation stream never do — every request they make is an extension. Systems with a second path branching from the same prefix get hit across the board: first turn after a clear, side paths, another agent sharing the prefix — and each miss is the whole request at full price.

The fix. Mark an explicit breakpoint at the end of the shared prefix, leave mode at implicit, set prompt_cache_key to a hash of the prefix, and stop using prompt_cache_retention.

Four things about the debugging:

1. Change one variable at a time. Two fields shipped together and 0% → 96.5% says nothing about which one did it; isolating them showed the routing key contributed zero tokens in this setting.
2. A simplified test will lie to you. The two-message script passed in all four variants, and its one difference from a real request — the message count — was the deciding dimension.
3. Confirm how you’re slicing before you conclude. Guessing turn boundaries from timestamps skewed the entire baseline; iter was in the data all along.
4. A cache rig needs a fresh cold prefix per run. A fixed marker reads entries left by the previous run and the whole table falsely reports 100%. Acceptance condition: the writing call must show cached=0 and written>0.