Stream speech to text over WebSocket

For live captions and assistants, open wss://api.x.ai/v1/stt with query params, wait for transcript.created, send raw audio as binary frames, then finish with {"type":"audio.done"}. Proxy the socket through your backend.

Connect and stream

wss://api.x.ai/v1/stt?sample_rate=16000&encoding=pcm&interim_results=true&language=en

import asyncio, json, os, websockets

WS = "wss://api.x.ai/v1/stt?sample_rate=16000&encoding=pcm&interim_results=true&language=en"

async def transcribe(path: str):
    async with websockets.connect(
        WS, additional_headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"}
    ) as ws:
        assert json.loads(await ws.recv())["type"] == "transcript.created"
        with open(path, "rb") as f:
            f.read(44)  # skip WAV header for PCM16 demo
            chunk = 16000 * 2 // 10  # 100 ms
            while data := f.read(chunk):
                await ws.send(data)
                await asyncio.sleep(0.1)
        await ws.send(json.dumps({"type": "audio.done"}))
        async for message in ws:
            event = json.loads(message)
            if event["type"] == "transcript.partial":
                tag = "FINAL" if event["is_final"] else "partial"
                print(f"[{tag}] {event['text']}")
            elif event["type"] == "transcript.done":
                print(event["text"], event["duration"])
                break

asyncio.run(transcribe("audio.wav"))

Partial vs final

is_final	speech_final	Meaning
false	false	Interim (needs interim_results=true)
true	false	Chunk locked (~3s of speech)
true	true	Utterance done

Force an utterance end for push-to-talk with {"type":"Finalize"} (session stays open). Optional channel scopes finalize when multichannel=true.

Useful knobs

• Prefer 16 kHz PCM (sample_rate=16000&encoding=pcm) — native rate, no server resample.
• encoding=opus: one Opus packet per binary frame; mono only; ~4 KB/s at 24 kHz vs 48 KB/s PCM.
• smart_turn=0.7 plus smart_turn_timeout=3000 cuts false ends on mid-sentence pauses.
• diarize=true adds speaker on each word. multichannel=true&channels=2 for agent/customer trunks (not with Opus).
• keyterm= biases product names (repeatable, max 100 × 50 chars).

Pitfalls

• Wait for transcript.created before sending audio.
• Send real-time-paced chunks (~100 ms). Dumping the whole file at once breaks timing.
• Undecodable frames usually close the session.
• Never put the API key in the browser.
• Console API credits are separate from SuperGrok's weekly pool on grok.com.