Pin keyframes on Imagine video 1.5

Storyboard a clip by pinning stills at exact moments inside the video so the model animates between your anchors instead of inventing the whole shot from one frame. Official Reference-to-Video documents keyframes on grok-imagine-video-1.5 for POST https://api.x.ai/v1/videos/generations. Each entry pairs an image with a timestamp_s strictly inside the clip. You may send at most four keyframes, and anchors snap to a one-third-second grid, so keep them at least one-third of a second apart. Pin the opening with image and the closing with last_frame when the endpoints matter. Classic grok-imagine-video rejects keyframes. Create a key and load credits at console.x.ai before you bill a storyboarded run.

What you need

An XAI_API_KEY with Console balance, two to four stills that share a scene (same subject, horizon, and camera angle work best), a duration long enough for the interior timestamps, and optional endpoint pins. Use this job when mid-clip beats matter; for endpoints alone, start with Pin first and last frames on Imagine video 1.5. Each keyframe image accepts a public HTTPS URL, a data URI, or a Files API file_id.

Storyboard with interior pins

1. Export the key locally and keep it out of chat logs and public repos:

export XAI_API_KEY="your_api_key"

2. Send REST with keyframes (Vercel AI SDK does not expose this field yet — put it on the REST body):

REQUEST_ID=$(curl -s -X POST https://api.x.ai/v1/videos/generations \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $XAI_API_KEY" \
  -d '{
    "model": "grok-imagine-video-1.5",
    "prompt": "The oak moves through the seasons as light and leaves shift, camera locked on the trunk.",
    "image": {"url": "<WINTER_FRAME_URL>"},
    "last_frame": {"url": "<AUTUMN_FRAME_URL>"},
    "keyframes": [
      {"image": {"url": "<SPRING_FRAME_URL>"}, "timestamp_s": 3.0},
      {"image": {"url": "<SUMMER_FRAME_URL>"}, "timestamp_s": 6.0}
    ],
    "duration": 9,
    "aspect_ratio": "16:9",
    "resolution": "720p"
  }' | jq -r '.request_id')

echo "Request ID: $REQUEST_ID"

3. In the Python SDK, flatten each keyframe to image_url (or image_file_id) plus timestamp, and pin endpoints with image_url / last_frame_url as documented on the same page.
4. Poll GET https://api.x.ai/v1/videos/$REQUEST_ID until status is done, then download the temporary URL promptly.
5. Combine keyframes with reference_images or reference_audios when you also need style or voice guidance; prompt stays optional whenever any frame is pinned.

After the generate

Watch the clip at each anchor time and confirm the stills appear literally at those moments before you spend another run. If only the start and end are wrong, fix endpoints with Pin first and last frames on Imagine video 1.5 instead of adding interior pins. For character or product continuity without locking frames, use Generate a video from reference images. Keep resolution at 720p or below on this reference-style path.

Pitfalls

Timestamps at 0 or equal to duration are rejected — use image / last_frame for the endpoints. Two keyframes that round to the same one-third-second slot fail validation. Sending more than four keyframes returns an invalid-argument failure. Classic grok-imagine-video rejects keyframes entirely. Pins that do not share a scene force the model to invent large jumps and look less coherent. Pasting keys into tickets or committing them to git forces a rotate.