---
title: "Semantic Auto Router (deprecated)"
url: "/docs/proxy/auto_routing_semantic"
canonical_url: "https://docs.litellm.ai/docs/proxy/auto_routing_semantic"
type: "docs"
last_updated: "2026-10-01"
summary: "The semantic Auto Router is superseded by Auto Routing, which folds semantic keyword matching, complexity scoring, and adaptive routing into a single autorouter/complexity_router. New deployments should start there; the semantic router..."
related:
  - "/docs/proxy/auto_routing"
  - "/docs/proxy/auto_routing_benchmark"
---
# Semantic Auto Router (deprecated)

> Index of all LiteLLM docs: https://docs.litellm.ai/llms.txt


:::warning[Deprecated]

The semantic Auto Router is superseded by [Auto Routing](./auto_routing.md), which folds semantic keyword matching, complexity scoring, and adaptive routing into a single `auto_router/complexity_router`. New deployments should start there; the semantic router page is preserved for existing configs.

:::

LiteLLM can auto select the best model for a request based on rules you define.

[Image: Auto Routing]

## LiteLLM Python SDK

Auto routing allows you to define routing rules that automatically select the best model for a request based on the input content. This is useful for directing different types of queries to specialized models.

### Setup

1. **Create a router configuration file** (e.g., `router.json`):

```json
{
    "encoder_type": "openai",
    "encoder_name": "text-embedding-3-large",
    "routes": [
        {
            "name": "litellm-gpt-4.1",
            "utterances": [
                "litellm is great"
            ],
            "description": "positive affirmation",
            "function_schemas": null,
            "llm": null,
            "score_threshold": 0.5,
            "metadata": {}
        },
        {
            "name": "litellm-claude-35",
            "utterances": [
                "how to code a program in [language]"
            ],
            "description": "coding assistant",
            "function_schemas": null,
            "llm": null,
            "score_threshold": 0.5,
            "metadata": {}
        }
    ]
}
```

2. **Configure the Router with auto routing models**:

```python
from litellm import Router
import os

router = Router(
    model_list=[
        # Embedding models for routing
        {
            "model_name": "custom-text-embedding-model",
            "litellm_params": {
                "model": "text-embedding-3-large",
                "api_key": os.getenv("OPENAI_API_KEY"),
            },
        },
        # Your target models
        {
            "model_name": "litellm-gpt-4.1",
            "litellm_params": {
                "model": "gpt-5.6-terra",
            },
            "model_info": {"id": "openai-id"},
        },
        {
            "model_name": "litellm-claude-35",
            "litellm_params": {
                "model": "claude-sonnet-5",
            },
            "model_info": {"id": "claude-id"},
        },
        # Auto router configuration
        {
            "model_name": "auto_router1",
            "litellm_params": {
                "model": "auto_router/auto_router_1",
                "auto_router_config_path": "router.json",
                "auto_router_default_model": "gpt-5.6-luna",
                "auto_router_embedding_model": "custom-text-embedding-model",
            },
        },
    ],
)
```

### Usage

Once configured, use the auto router by calling it with your auto router model name:

```python
# This request will be routed to gpt-5.6-terra based on the utterance match
response = await router.acompletion(
    model="auto_router1",
    messages=[{"role": "user", "content": "litellm is great"}],
)

# This request will be routed to claude-sonnet-5 for coding queries
response = await router.acompletion(
    model="auto_router1",
    messages=[{"role": "user", "content": "how to code a program in python"}],
)
```

### Configuration Parameters

- **auto_router_config_path**: Path to your router.json configuration file
- **auto_router_default_model**: Fallback model when no route matches
- **auto_router_embedding_model**: Model used for generating embeddings to match against utterances

### Router Configuration Schema

The `router.json` file supports the following structure:

- **encoder_type**: Type of encoder (e.g., "openai")
- **encoder_name**: Name of the embedding model
- **routes**: Array of routing rules with:
  - **name**: Target model name (must match a model in your model_list)
  - **utterances**: Example phrases/patterns to match against
  - **description**: Human-readable description of the route
  - **score_threshold**: Minimum similarity score to trigger this route (0.0-1.0)
  - **metadata**: Additional metadata for the route

## LiteLLM Proxy Server

### Setup

Navigate to the LiteLLM UI and go to **Models+Endpoints** > **Add Model** > **Auto Router Tab**.

Configure the following required fields:

- **Auto Router Name** - The model name that developers will use when making LLM API requests to LiteLLM
- **Default Model** - The fallback model used when no route is matched (e.g., if set to "gpt-5.6-luna", unmatched requests will be routed to gpt-5.6-luna)
- **Embedding Model** - The model used to generate embeddings for input messages. These embeddings are used to semantically match input against the utterances defined in your routes

#### Route Configuration

[Image: Auto Router Setup]

<br />

<br />

Click **Add Route** to create a new routing rule. Each route consists of utterances that are matched against input messages to determine the target model.

Configure each route with:

- **Utterances** - Example phrases that will trigger this route. Use placeholders in brackets for variables:

```json
[
"how to code a program in [language]",
"can you explain this [language] code",
"can you explain this [language] script",
"can you convert this [language] code to [target_language]"
]
```

- **Description** - A human-readable description of what this route handles
- **Score Threshold** - The minimum similarity score (0.0-1.0) required to trigger this route

### Usage

Once added developers need to select the model=`auto_router1` in the `model` field of the LLM API request.

**OpenAI Python v1.0.0+**

```python
import openai
client = openai.OpenAI(
    api_key="sk-<your-litellm-api-key>", # replace with your LiteLLM API key
    base_url="http://localhost:4000"
)

# This request will be auto-routed based on the content
response = client.chat.completions.create(
    model="auto_router1",
    messages=[
        {
            "role": "user",
            "content": "how to code a program in python"
        }
    ]
)

print(response)
```

**Curl Request**

```shell
curl -X POST http://localhost:4000/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $LITELLM_API_KEY" \
-d '{
    "model": "auto_router1",
    "messages": [{"role": "user", "content": "how to code a program in python"}]
}'
```

## How It Works

1. When a request comes in, LiteLLM generates embeddings for the input message
2. It compares these embeddings against **all** utterances defined across **all** your routes simultaneously
3. It identifies the route with the **highest similarity score**. If that score exceeds the route's defined threshold, the request is routed to that model. *(Because the router selects the global maximum score rather than stopping at the first match, the order of routes in your configuration does not affect which route is selected.)*
4. If no route's maximum score meets its threshold, the request goes to the default model

## Related pages

- [[Beta] Auto Routing](https://docs.litellm.ai/docs/proxy/auto_routing.md)
- [Auto Routing Benchmark: Cost Ladders](https://docs.litellm.ai/docs/proxy/auto_routing_benchmark.md)