Use Grok on Google Cloud Vertex AI

Grok on Vertex AI / Gemini Enterprise Agent Platform runs as a partner model through Google’s OpenAI-compatible API (Responses and Chat Completions). Enable models in Model Garden, then call them with the openai SDK under Application Default Credentials (ADC).

Prerequisites

• GCP project with billing
• Permission to enable APIs and Model Garden (Vertex AI User or Project Editor)
• aiplatform.googleapis.com (or equivalent Agent Platform API) enabled
• gcloud installed

gcloud auth application-default login
gcloud config set project YOUR_PROJECT_ID
gcloud services enable aiplatform.googleapis.com
pip install -U openai google-cloud-aiplatform

Enable Grok in Model Garden

1. Open Model Garden in the Google Cloud Console (search “Model Garden”).
2. Search “Grok” or browse publisher xAI.
3. Open the model card (for example Grok 4.2 or Grok 4.3).
4. Review capabilities, quotas, pricing, and regions.
5. Click Enable or Deploy / request access if prompted.

Use the model ID from the card. Names often carry a publisher prefix, for example xai/grok-4.6. Availability follows the xAI API subject to Google regional quotas.

Point the OpenAI client at Vertex

Set the OpenAI-compatible base URL from the model card or Google documentation for the Agent Platform (do not invent hostnames):

export OPENAI_BASE_URL="https://YOUR_VERTEX_ENDPOINT"

The openai client picks up ADC / env vars automatically.

Responses API

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="xai/grok-4.6",
    input="Explain the advantages of using Grok for agentic workflows with parallel tool calling.",
    max_output_tokens=800,
)

print(response.output_text)

Chat Completions

from openai import OpenAI

client = OpenAI()

response = client.chat.completions.create(
    model="xai/grok-4.6",
    messages=[
        {
            "role": "user",
            "content": "Which city has a higher temperature right now, Boston or New Delhi, and by how much in Fahrenheit?",
        }
    ],
    tools=[
        {
            "type": "function",
            "function": {
                "name": "get_current_weather",
                "description": "Get the current weather in a given location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description": "The city and state, e.g., San Francisco, CA",
                        },
                        "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
                    },
                    "required": ["location"],
                },
            },
        }
    ],
    tool_choice="auto",
)

print(response.choices[0].message.content)

Streaming works on both interfaces. Supported capabilities per the guide include function calling (including parallel), reasoning / extended thinking, structured outputs / JSON mode, and fixed quotas / committed-use discounts through Google Cloud. Context windows vary by model card.

Data retention and compliance

Retention and processing follow Google Cloud Vertex AI policies. Many deployments offer Zero Data Retention (ZDR) options. Review the model card and your org’s Google Cloud data governance settings. Vertex AI request-response logging can capture activity for audit and debugging.

Endpoints and ops

Choice	Notes
Global endpoints	Dynamic routing; recommended for most workloads
Regional endpoints	Pin a region for compliance
Auth	Prefer ADC and IAM / service accounts over long-lived keys
Billing	Monitor Google Cloud Billing and Quotas; request increases as needed

When migrating from the direct xAI API, update base URL, client config, and model prefix. Most prompts and tool schemas transfer with small edits.

Troubleshooting

Issue	What to check
Authentication errors	gcloud auth application-default login; project permissions
Model not found	Enabled in Model Garden; exact xai/... ID from the card
Quota exceeded	Google Cloud quotas console; request increase
Endpoint / base URL issues	Exact endpoint or env var from the model card or Google docs

Pitfalls

• Hard-coding a guessed Vertex hostname — use the endpoint from the model card or Google documentation.
• Calling with an xAI Console inference key against Vertex — Vertex uses Google ADC / IAM.
• Skipping Model Garden enable before the first API call.