Avoid breaking Grok API prompt cache

Prompt cache hits require an unchanged message prefix. Only append new messages at the end. Editing, deleting, or reordering earlier messages forces a miss. Sticky routing (x-grok-conv-id or prompt_cache_key) still matters — see Maximize Grok API prompt cache hits — but a mutated prefix will miss even with the same id.

Keep the prefix byte-stable

from openai import OpenAI

client = OpenAI(api_key="YOUR_XAI_API_KEY", base_url="https://api.x.ai/v1")
conv_id = "conv_abc123"
messages = [
    {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},
    {"role": "user", "content": "What is prompt caching?"},
]

r1 = client.chat.completions.create(
    model="grok-4.6",
    messages=messages,
    extra_headers={"x-grok-conv-id": conv_id},
)
messages.append({"role": "assistant", "content": r1.choices[0].message.content})
messages.append({"role": "user", "content": "Show me a code example."})

r2 = client.chat.completions.create(
    model="grok-4.6",
    messages=messages,
    extra_headers={"x-grok-conv-id": conv_id},
)
print("Turn 2 cached:", r2.usage.prompt_tokens_details.cached_tokens)

Turn 2 should show cached_tokens covering the earlier prefix. Chat Completions: usage.prompt_tokens_details.cached_tokens. Responses: usage.input_tokens_details.cached_tokens.

What causes a miss

Change	Result
Edit an earlier user or assistant string	Miss
Remove any prior message	Miss
Swap system and user order	Miss
Drop reasoning_content on a reasoning model when replaying history	Miss (top cause per docs)

For reasoning models, either send encrypted reasoning content back or continue with previous_response_id so the server keeps the chain.

Read the meter

cached_tokens	Meaning
0	Full miss (first turn or broken prefix / routing)
> 0	Partial or full hit — that many prompt tokens reused
Equal to prompt/input tokens	Entire prompt served from cache

Cached tokens still count toward TPM rate limits; they bill at the reduced cached-prompt rate. See pricing for the live per-model numbers.

Pitfalls

• Rewriting the system prompt mid-thread "for clarity" and wondering why every turn is cold.
• Omitting reasoning content on grok-4.6 / grok-4.5 history replays.
• Checking intermediate stream chunks for usage without reading the final usage object.