---
title: "Client-Side Fallbacks"
url: "/docs/proxy/client_side_fallbacks"
canonical_url: "https://docs.litellm.ai/docs/proxy/client_side_fallbacks"
type: "docs"
last_updated: "2026-10-14"
summary: "Set fallback models per request from the client or the Python SDK, and control the prompt sent to each fallback model."
related:
  - "/docs/proxy/reliability"
  - "/docs/proxy/timeout"
---
# Client-Side Fallbacks

> Index of all LiteLLM docs: https://docs.litellm.ai/llms.txt


Set fallbacks in the `.completion()` call for SDK and client-side for proxy. 

In this request the following will occur:
1. The request to `model="zephyr-beta"` will fail
2. litellm proxy will loop through all the model_groups specified in `fallbacks=["gpt-5.6-luna"]`
3. The request to `model="gpt-5.6-luna"` will succeed and the client making the request will get a response from gpt-5.6-luna 

👉 Key Change: `"fallbacks": ["gpt-5.6-luna"]`

**SDK**

```python
from litellm import Router

router = Router(model_list=[...]) # defined in Step 1.

resp = router.completion(
    model="gpt-5.6-luna",
    messages=[{"role": "user", "content": "Hey, how's it going?"}],
    mock_testing_fallbacks=True, # 👈 trigger fallbacks
    fallbacks=[
        {
            "model": "claude-sonnet-5",
            "messages": [{"role": "user", "content": "What is LiteLLM?"}],
        }
    ],
)

print(resp)
```

**PROXY**

**OpenAI Python v1.0.0+**

```python
import openai
client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="zephyr-beta",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={
        "fallbacks": ["gpt-5.6-luna"]
    }
)

print(response)
```

**Curl Request**

```shell
curl --location 'http://0.0.0.0:4000/chat/completions' \
    --header 'Content-Type: application/json' \
    --data '{
    "model": "zephyr-beta",
    "messages": [
        {
        "role": "user",
        "content": "what llm are you"
        }
    ],
    "fallbacks": ["gpt-5.6-luna"]
}'
```
**Langchain**

```python
from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
)
from langchain.schema import HumanMessage, SystemMessage
import os 

os.environ["OPENAI_API_KEY"] = "anything"

chat = ChatOpenAI(
    openai_api_base="http://0.0.0.0:4000",
    model="zephyr-beta",
    extra_body={
        "fallbacks": ["gpt-5.6-luna"]
    }
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that im using to make a test request to."
    ),
    HumanMessage(
        content="test from litellm. tell me why it's amazing in 1 sentence"
    ),
]
response = chat(messages)

print(response)
```

## Control Fallback Prompts

Pass in messages/temperature/etc. per model in fallback (works for embedding/image generation/etc. as well).

Key Change:

```
fallbacks = [
  {
    "model": <model_name>,
    "messages": <model-specific-messages>
    ... # any other model-specific parameters
  }
]
```

**SDK**

```python
from litellm import Router

router = Router(model_list=[...]) # defined in Step 1.

resp = router.completion(
    model="gpt-5.6-luna",
    messages=[{"role": "user", "content": "Hey, how's it going?"}],
    mock_testing_fallbacks=True, # 👈 trigger fallbacks
    fallbacks=[
        {
            "model": "claude-sonnet-5",
            "messages": [{"role": "user", "content": "What is LiteLLM?"}],
        }
    ],
)

print(resp)
```

**PROXY**

**OpenAI Python v1.0.0+**

```python
import openai
client = openai.OpenAI(
    api_key="anything",
    base_url="http://0.0.0.0:4000"
)

response = client.chat.completions.create(
    model="zephyr-beta",
    messages = [
        {
            "role": "user",
            "content": "this is a test request, write a short poem"
        }
    ],
    extra_body={
      "fallbacks": [{
          "model": "claude-sonnet-5",
          "messages": [{"role": "user", "content": "What is LiteLLM?"}]
      }]
    }
)

print(response)
```

**Curl Request**

```bash
curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-d '{
    "model": "gpt-5.6-luna",
    "messages": [
      {
        "role": "user",
        "content": [
          {
            "type": "text",
            "text": "Hi, how are you ?"
          }
        ]
      }
    ],
    "fallbacks": [{
        "model": "claude-sonnet-5",
        "messages": [{"role": "user", "content": "What is LiteLLM?"}]
    }]
}'
```

**Langchain**

```python
from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
    ChatPromptTemplate,
    HumanMessagePromptTemplate,
    SystemMessagePromptTemplate,
)
from langchain.schema import HumanMessage, SystemMessage
import os 

os.environ["OPENAI_API_KEY"] = "anything"

chat = ChatOpenAI(
    openai_api_base="http://0.0.0.0:4000",
    model="zephyr-beta",
    extra_body={
      "fallbacks": [{
          "model": "claude-sonnet-5",
          "messages": [{"role": "user", "content": "What is LiteLLM?"}]
      }]
    }
)

messages = [
    SystemMessage(
        content="You are a helpful assistant that im using to make a test request to."
    ),
    HumanMessage(
        content="test from litellm. tell me why it's amazing in 1 sentence"
    ),
]
response = chat(messages)

print(response)
```

To set fallbacks for every request in the gateway config instead, see [Fallbacks](./reliability.md).

## Related pages

- [Fallbacks](https://docs.litellm.ai/docs/proxy/reliability.md)
- [Timeouts](https://docs.litellm.ai/docs/proxy/timeout.md)