Analyze an image with the Grok API

Send user content as a list that mixes input_image (public URL or base64 data URL) with input_text. Call the Responses API with model grok-4.6. Related curl + xAI SDK walkthrough: Understand an image with the Grok API. Get a key at console.x.ai.

OpenAI SDK (Responses)

import os
from openai import OpenAI

client = OpenAI(
    api_key=os.getenv("XAI_API_KEY"),
    base_url="https://api.x.ai/v1",
)

response = client.responses.create(
    model="grok-4.6",
    input=[
        {
            "role": "user",
            "content": [
                {
                    "type": "input_image",
                    "image_url": "https://science.nasa.gov/wp-content/uploads/2023/09/web-first-images-release.png",
                    "detail": "high",
                },
                {
                    "type": "input_text",
                    "text": "What is in this image? Describe the main subjects.",
                },
            ],
        }
    ],
)

print(response.output_text)
print("response.id:", response.id)

Keep response.id and pass it on later turns to continue the same conversation. For a local file, encode it as a data URL (data:image/jpeg;base64,... or data:image/png;base64,...) and put that string in image_url.

Pitfalls

• Storing request/response history on the server when the request includes images — docs warn the call may fail; avoid server-side history for image turns.
• Using formats other than jpg/jpeg or png, or files over 20 MiB.
• Inventing a separate vision-only model id — use grok-4.6 as shown in the image understanding docs.