Maximize Grok API prompt cache hits

The xAI API caches matching message prefixes automatically. Sticky routing with x-grok-conv-id (Chat Completions) or prompt_cache_key (Responses) keeps follow-ups on the same server so more of that prefix hits cache. Cached tokens bill at a reduced rate and skip recomputation for faster first tokens.

Chat Completions: x-grok-conv-id

Send the same conversation id on every turn of a thread.

curl https://api.x.ai/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -H "x-grok-conv-id: conv_abc123" \
  -d '{
    "model": "grok-4.6",
    "messages": [
      {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},
      {"role": "user", "content": "What is prompt caching?"}
    ]
  }'

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_XAI_API_KEY",
    base_url="https://api.x.ai/v1",
)

response = client.chat.completions.create(
    model="grok-4.6",
    messages=[
        {"role": "system", "content": "You are Grok, a helpful and truthful AI assistant built by xAI."},
        {"role": "user", "content": "What is prompt caching?"},
    ],
    extra_headers={"x-grok-conv-id": "conv_abc123"},
)

print(response.choices[0].message.content)
print(f"Cached tokens: {response.usage.prompt_tokens_details.cached_tokens}")

Responses API: prompt_cache_key

Same sticky routing, as a body field instead of a header.

curl https://api.x.ai/v1/responses \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -d '{
    "model": "grok-4.6",
    "input": "What is prompt caching?",
    "prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366"
  }'

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_XAI_API_KEY",
    base_url="https://api.x.ai/v1",
)

response = client.responses.create(
    model="grok-4.6",
    input="What is prompt caching?",
    extra_body={"prompt_cache_key": "b79ad29b-b3f9-463c-bca6-041d5058d366"},
)

print(response.output_text)
print(f"Cached tokens: {response.usage.input_tokens_details.cached_tokens}")

xAI SDK (gRPC metadata)

from xai_sdk import Client
from xai_sdk.chat import system, user

client = Client(
    api_key="YOUR_API_KEY",
    metadata=(("x-grok-conv-id", "conv_abc123"),),
)

chat = client.chat.create(model="grok-4.6")
chat.append(system("You are Grok, a helpful and truthful AI assistant built by xAI."))
chat.append(user("What is prompt caching?"))

response = chat.sample()
print(response.content)
print(f"Cached tokens: {response.usage.cached_prompt_text_tokens}")

How the prefix works

Cache matches from the start of the messages array. Keep the system prompt and earlier turns byte-identical across requests. Append only the new user turn; rewriting an earlier message breaks the prefix and forces a full recompute.

Pitfalls

• Caching is automatic but not guaranteed. Memory pressure and server routing can still miss; sticky ids maximize hits, they do not promise 100%.
• Changing anything in the shared prefix (system text, tool defs order, earlier turns) invalidates that prefix.
• Use one stable id per conversation. Reusing the same id across unrelated threads fights the sticky routing goal.
• Console API credits are separate from SuperGrok's weekly pool on grok.com.