Skip to main content

Vertex AI - Anthropic, DeepSeek, Model Garden, xAI

Supported Partner Providers​

ProviderLiteLLM RouteVertex Documentation
Anthropic (Claude)vertex_ai/claude-*Vertex AI - Anthropic Models
DeepSeekvertex_ai/deepseek-ai/{MODEL}Vertex AI - DeepSeek Models
ZAI (GLM)vertex_ai/zai-org/{MODEL}Vertex AI - GLM Models
Meta/Llamavertex_ai/meta/{MODEL}Vertex AI - Meta Models
Mistralvertex_ai/mistral-*Vertex AI - Mistral Models
AI21 (Jamba)vertex_ai/jamba-*Vertex AI - AI21 Models
Qwenvertex_ai/qwen/*Vertex AI - Qwen Models
OpenAI (GPT-OSS)vertex_ai/openai/gpt-oss-*Vertex AI - GPT-OSS Models
xAI (Grok)vertex_ai/xai/{MODEL}xAI models (incl. Vertex), Vertex AI Model Garden

Latest model aliases​

Since v1.109.0

vertex_ai/<family>-latest names follow a model family as LiteLLM moves them to newer models, so your application keeps one model string. Use them like any other Vertex AI model name. LiteLLM sends Vertex AI the current target model and bills at that model's price. See Latest model aliases for how aliases work and the full list across providers.

AliasModel family
vertex_ai/claude-opus-latestClaude Opus
vertex_ai/claude-sonnet-latestClaude Sonnet
vertex_ai/claude-haiku-latestClaude Haiku
vertex_ai/claude-fable-latestClaude Fable
vertex_ai/grok-latestGrok

Vertex AI - Anthropic (Claude)​

Model NameFunction Call
claude-sonnet-4-5@20250929completion('vertex_ai/claude-sonnet-4-5@20250929', messages)
claude-3-opus@20240229completion('vertex_ai/claude-3-opus@20240229', messages)
claude-3-5-sonnet@20240620completion('vertex_ai/claude-3-5-sonnet@20240620', messages)
claude-3-sonnet@20240229completion('vertex_ai/claude-3-sonnet@20240229', messages)
claude-3-haiku@20240307completion('vertex_ai/claude-3-haiku@20240307', messages)
claude-3-7-sonnet@20250219completion('vertex_ai/claude-3-7-sonnet@20250219', messages)

Usage​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "claude-sonnet-5"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    temperature=0.7,
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

Usage - thinking / reasoning_content​

from litellm import completion

resp = completion(
    model="vertex_ai/claude-sonnet-5",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    thinking={"type": "enabled", "budget_tokens": 1024},
)

Expected Response

ModelResponse(
    id='chatcmpl-c542d76d-f675-4e87-8e5f-05855f5d0f5e',
    created=1740470510,
    model='claude-sonnet-5',
    object='chat.completion',
    system_fingerprint=None,
    choices=[
        Choices(
            finish_reason='stop',
            index=0,
            message=Message(
                content="The capital of France is Paris.",
                role='assistant',
                tool_calls=None,
                function_call=None,
                provider_specific_fields={
                    'citations': None,
                    'thinking_blocks': [
                        {
                            'type': 'thinking',
                            'thinking': 'The capital of France is Paris. This is a very straightforward factual question.',
                            'signature': 'EuYBCkQYAiJAy6...'
                        }
                    ]
                }
            ),
            thinking_blocks=[
                {
                    'type': 'thinking',
                    'thinking': 'The capital of France is Paris. This is a very straightforward factual question.',
                    'signature': 'EuYBCkQYAiJAy6AGB...'
                }
            ],
            reasoning_content='The capital of France is Paris. This is a very straightforward factual question.'
        )
    ],
    usage=Usage(
        completion_tokens=68,
        prompt_tokens=42,
        total_tokens=110,
        completion_tokens_details=None,
        prompt_tokens_details=PromptTokensDetailsWrapper(
            audio_tokens=None,
            cached_tokens=0,
            text_tokens=None,
            image_tokens=None
        ),
        cache_creation_input_tokens=0,
        cache_read_input_tokens=0
    )
)

VertexAI DeepSeek​

PropertyDetails
Provider Routevertex_ai/deepseek-ai/{MODEL}
Vertex DocumentationVertex AI - DeepSeek Models

Usage​

LiteLLM Supports all Vertex AI DeepSeek Models. Ensure you use the vertex_ai/deepseek-ai/ prefix for all Vertex AI DeepSeek models.

Model NameUsage
vertex_ai/deepseek-ai/deepseek-r1-0528-maascompletion('vertex_ai/deepseek-ai/deepseek-r1-0528-maas', messages)

VertexAI ZAI (GLM)​

PropertyDetails
Provider Routevertex_ai/zai-org/{MODEL}
Vertex DocumentationVertex AI - GLM Models

LiteLLM Supports all Vertex AI GLM Models. Ensure you use the vertex_ai/zai-org/ prefix for all Vertex AI GLM models.

Model NameUsage
vertex_ai/zai-org/glm-4.7-maascompletion('vertex_ai/zai-org/glm-4.7-maas', messages)

Usage​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

response = completion(
    model="vertex_ai/zai-org/glm-4.7-maas",
    messages=[{"role": "user", "content": "hi"}],
    vertex_project="your-vertex-project",
    # vertex_location routes to "global"
)
print("\nModel Response", response)

VertexAI Meta/Llama API​

Model NameFunction Call
meta/llama-3.2-90b-vision-instruct-maascompletion('vertex_ai/meta/llama-3.2-90b-vision-instruct-maas', messages)
meta/llama3-8b-instruct-maascompletion('vertex_ai/meta/llama3-8b-instruct-maas', messages)
meta/llama3-70b-instruct-maascompletion('vertex_ai/meta/llama3-70b-instruct-maas', messages)
meta/llama3-405b-instruct-maascompletion('vertex_ai/meta/llama3-405b-instruct-maas', messages)
meta/llama-4-scout-17b-16e-instruct-maascompletion('vertex_ai/meta/llama-4-scout-17b-16e-instruct-maas', messages)
meta/llama-4-scout-17-128e-instruct-maascompletion('vertex_ai/meta/llama-4-scout-128b-16e-instruct-maas', messages)
meta/llama-4-maverick-17b-128e-instruct-maascompletion('vertex_ai/meta/llama-4-maverick-17b-128e-instruct-maas',messages)
meta/llama-4-maverick-17b-16e-instruct-maascompletion('vertex_ai/meta/llama-4-maverick-17b-16e-instruct-maas',messages)

Usage​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "meta/llama3-405b-instruct-maas"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI Mistral API​

Supported OpenAI Params

LiteLLM Supports all Vertex AI Mistral Models. Ensure you use the vertex_ai/mistral- prefix for all Vertex AI Mistral models.

Overview

PropertyDetails
Provider Routevertex_ai/mistral-{MODEL}
Vertex DocumentationVertex AI - Mistral Models
Model NameFunction Call
mistral-large@latestcompletion('vertex_ai/mistral-large@latest', messages)
mistral-large@2407completion('vertex_ai/mistral-large@2407', messages)
mistral-small-2503completion('vertex_ai/mistral-small-2503', messages)
mistral-large-2411completion('vertex_ai/mistral-large-2411', messages)
mistral-nemo@latestcompletion('vertex_ai/mistral-nemo@latest', messages)
codestral@latestcompletion('vertex_ai/codestral@latest', messages)
codestral@@2405completion('vertex_ai/codestral@2405', messages)

Usage​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "mistral-large@2407"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

Usage - Codestral FIM​

Call Codestral on VertexAI via the OpenAI /v1/completion endpoint for FIM tasks.

Note: You can also call Codestral via /chat/completion.

from litellm import text_completion
import os

# os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""
# OR run `!gcloud auth print-access-token` in your terminal

model = "codestral@2405"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = text_completion(
    model="vertex_ai/" + model,
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
    prompt="def is_odd(n): \n return n % 2 == 1 \ndef test_is_odd():", 
    suffix="return True",                                              # optional
    temperature=0,                                                     # optional
    top_p=1,                                                           # optional
    max_tokens=10,                                                     # optional
    min_tokens=10,                                                     # optional
    seed=10,                                                           # optional
    stop=["return"],                                                   # optional
)

print("\nModel Response", response)

VertexAI AI21 Models​

Model NameFunction Call
jamba-1.5-mini@001completion(model='vertex_ai/jamba-1.5-mini@001', messages)
jamba-1.5-large@001completion(model='vertex_ai/jamba-1.5-large@001', messages)

Usage​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "jamba-1.5-mini@001"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI Qwen API​

PropertyDetails
Provider Routevertex_ai/qwen/{MODEL}
Vertex DocumentationVertex AI - Qwen Models

LiteLLM Supports all Vertex AI Qwen Models. Ensure you use the vertex_ai/qwen/ prefix for all Vertex AI Qwen models.

Model NameUsage
vertex_ai/qwen/qwen3-coder-480b-a35b-instruct-maascompletion('vertex_ai/qwen/qwen3-coder-480b-a35b-instruct-maas', messages)
vertex_ai/qwen/qwen3-235b-a22b-instruct-2507-maascompletion('vertex_ai/qwen/qwen3-235b-a22b-instruct-2507-maas', messages)

Usage​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "qwen/qwen3-coder-480b-a35b-instruct-maas"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

VertexAI GPT-OSS Models​

PropertyDetails
Provider Routevertex_ai/openai/{MODEL}
Vertex DocumentationVertex AI - GPT-OSS Models

LiteLLM Supports all Vertex AI GPT-OSS Models. Ensure you use the vertex_ai/openai/ prefix for all Vertex AI GPT-OSS models.

Model NameUsage
vertex_ai/openai/gpt-oss-20b-maascompletion('vertex_ai/openai/gpt-oss-20b-maas', messages)

Usage​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

model = "openai/gpt-oss-20b-maas"

vertex_ai_project = "your-vertex-project" # can also set this as os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location" # can also set this as os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/" + model,
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)

Usage - reasoning_effort​

GPT-OSS models support the reasoning_effort parameter for enhanced reasoning capabilities.

from litellm import completion

response = completion(
    model="vertex_ai/openai/gpt-oss-20b-maas",
    messages=[{"role": "user", "content": "Solve this complex problem step by step"}],
    reasoning_effort="low",  # Options: "minimal", "low", "medium", "high"
    vertex_ai_project="your-vertex-project",
    vertex_ai_location="us-central1",
)

VertexAI xAI (Grok)​

xAI Grok models available in Vertex AI Model Garden use the same OpenAI-compatible chat-completions path as other Model Garden publisher models. Use the vertex_ai/xai/ prefix (not xai/, which is the direct xAI API with XAI_API_KEY). See also xAI provider.

PropertyDetails
Provider Routevertex_ai/xai/{MODEL}
Vertex / xAI docsxAI models, Model Garden
Model NameUsage
vertex_ai/xai/grok-4.1-fast-non-reasoningcompletion('vertex_ai/xai/grok-4.1-fast-non-reasoning', messages)
vertex_ai/xai/grok-4.1-fast-reasoningcompletion('vertex_ai/xai/grok-4.1-fast-reasoning', messages)
vertex_ai/xai/grok-4.20-non-reasoningcompletion('vertex_ai/xai/grok-4.20-non-reasoning', messages)
vertex_ai/xai/grok-4.20-reasoningcompletion('vertex_ai/xai/grok-4.20-reasoning', messages)

Usage​

from litellm import completion
import os

os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = ""

vertex_ai_project = "your-vertex-project"  # or os.environ["VERTEXAI_PROJECT"]
vertex_ai_location = "your-vertex-location"  # or os.environ["VERTEXAI_LOCATION"]

response = completion(
    model="vertex_ai/xai/grok-4.1-fast-non-reasoning",
    messages=[{"role": "user", "content": "hi"}],
    vertex_ai_project=vertex_ai_project,
    vertex_ai_location=vertex_ai_location,
)
print("\nModel Response", response)
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails, built for production.
Learn more →