---
title: "Core, config and host interfaces"
url: "/docs/router/api_reference/core_and_host"
canonical_url: "https://docs.litellm.ai/docs/router/api_reference/core_and_host"
type: "docs"
last_updated: "2026-11-05"
summary: "The LiteLLM Router step protocol, the Rust Router and Host trait, RouterRequest, Attempt, Output, RouterEvent and FailureKind."
related:
  - "/docs/router/api_reference/state_and_cache_keys"
  - "/docs/router/api_reference/code_map"
---
# Core, config and host interfaces

> Index of all LiteLLM docs: https://docs.litellm.ai/llms.txt


Available in LiteLLM v1.110.0 and later.

The interfaces between the Python facade, the Rust core and the host it runs
under. [The core and the host](../learn/core_and_hosts.md) explains why
they are shaped this way.

[Image: The Router facade uses the RouterCore protocol; RustCore implements it and
holds a handle to the Rust RustRouter, which calls the Host trait that
PythonHost implements]

## Python side

| Name | File | What it is |
| --- | --- | --- |
| `Router` | `litellm/router.py` | the facade. `Router(**arguments)` and `Router.from_config(config, host=python_host)` |
| `RouterCore` | `litellm/router_core.py` | the protocol the facade delegates to |
| `make_core(config, host)` | `litellm/router_core.py` | builds the core once, when the Router is built |
| `RustCore` | `litellm/router_rust/core.py` | holds one `RouterCoreHandle` and answers its steps |
| `RustBridgeDeclined` | `litellm.rust_bridge._native` | raised when the handle cannot read a config |
| `RouterConfig` | `litellm/router_config.py` | the constructor arguments, frozen, with plain containers copied (`build_router_config`) |
| `RouterDynamicConfig` | `litellm/router_config.py` | a versioned snapshot of the process settings the router reads per call (`current_dynamic_config()`) |
| `PythonHost` | `litellm/router_python_host.py` | `invoke(op)` returns the `litellm` endpoint function, plus `sleep`, `dynamic()`, the callback lists, the global cache and `routing_cache(config)` |
| `router_attempt` | `litellm/router_attempt.py` | builds one attempt's keyword arguments for the deployment the core picked |
| `RouterHost` | `litellm/router_host.py` | what only the application can answer (vertex pass-through credentials, member auto-router authorization) |
| `DeprecatedRouterMembers` | `litellm/router_deprecated.py` | the old Router member names, served with a `DeprecationWarning` |

`RouterDynamicConfig` fields are `num_retries`, `max_fallbacks`,
`router_max_fallbacks`, `allowed_fails`, `request_timeout`, `fallbacks`,
`default_fallbacks`, `context_window_fallbacks`, `content_policy_fallbacks`,
`drop_params`, `provider_list`, `expose_router_debug_in_errors`,
`redact_user_api_key_info`, `model_alias_map`, `model_cost`,
`known_custom_logger_compatible_callbacks`, `exception_types`, `callbacks`,
`success_callback`, `failure_callback`, `input_callback`, `callback_manager` and
`cache`, plus its `version`.

## The step protocol

`RouterCoreHandle(config, dynamic)` in `litellm.rust_bridge._native` wraps one
Rust `Router`. `start(op, request)`, `select(model, request)` and
`healthy(model, request)` return a `RouterCall`, and so do the report and model
info queries. The driver calls `RouterCall.next(answer)` until it gets `done` or
`failed`. Every step carries the routing events since the last step.

| Step | The core needs | The driver answers with |
| --- | --- | --- |
| `attempt` | one attempt run on the named deployment | the response and its total tokens, or the typed failure |
| `sleep` | a backoff of `seconds` | nothing, after sleeping through the host |
| `allow_fallback` | whether a cross-group fallback `target` is allowed | a bool from the fallback access and budget checks |
| `pre_route` | the sub-routers, session binding or a custom strategy before a pick resolves `model` | a model group, a deployment the custom strategy picked, or `None` |
| `select` | one host stage run on the candidates, named by `stage` | see the stage table below |
| `cache` | a routing cache read or write (`get`, `set`, `batch_get`, `increment` and the rest) | the values read, or an acknowledgement |
| `secrets`, `validate`, `provider`, `added` | one deployment's build, while a config or change set is applied | the resolved secrets, the validated entry, the provider, the added entry |
| `model_info` | a cost map or capability lookup during a model info query | the lookup's documents |
| `done` | nothing more | the result goes back to the caller, with routing headers |
| `failed` | nothing more | the error goes back to the caller, by its `error_kind` |

| `select` stage | The host returns |
| --- | --- |
| `routing_plugins` | the candidate models the routing plugins left, after running them when `run` is set |
| `callback_filters` | the deployment ids the `CustomLogger` filters left, or a failure |
| `plugins` | the deployments `Router(plugins=[...])` objects left |
| `pre_call` | the facts the pre-call checks decide by, such as token counts |
| `tags` | the deployments tag routing left |
| `text_only` | the candidates that declare no vision support |
| `route_incapable` | the candidates whose provider cannot serve the call the `handler` names |
| `media_refusing` | the candidates that refuse the audio or videos a Decisions call carries |
| `embedding_batch` | the candidates that admit the embedding input |
| `override_check` | the candidates after a per-request strategy's pre-call check, or its refusal |
| `provider_request` | the `provider/model` form of a bare name, for wildcard matching |
| `strategy` | the candidates with the pick first, for a strategy the core does not implement |

`apply(changes)` returns the new registry version, `registry_version()` reads
it, and `retire()` retires the router.

## Rust side

All of these live in `litellm-rust/crates/router-core/src/`.

| Type | Module | What it is |
| --- | --- | --- |
| `Router`, `RouterBuilder` | `lib.rs` | `Router::new(config, dynamic, host)`, or the builder with `randomness` and `clock` for tests |
| `RouterConfig`, `RouterDynamicConfig`, `Op`, `RoutingStrategy`, `RetryPolicy`, `AllowedFailsPolicy` | `config.rs` | static and dynamic config, the operation name, the strategies (`is_native` is true for `simple-shuffle`) |
| `Registry`, `Deployment`, `ChangeSet`, `RegistryVersion`, `deployment_id` | `registry.rs` | the immutable, indexed deployments |
| `Settings` | `settings.rs` | the frozen settings view |
| `Health`, `Outcome`, `Clock`, `cooldown_key`, `usage_key` | `health.rs` | cooldowns, allowed fails, usage and the outcome counters |
| `RouterCache`, `InMemoryRouterCache` | `cache.rs` | the routing state store the host provides |
| `Randomness`, `SeededRandomness`, `simple_shuffle` | `select.rs` | the filter chain and `simple-shuffle` |
| `resolve_request`, `all_deployments` | `resolve.rs` | name resolution |
| `routing_context`, `request_override` | `strategy.rs` | which strategy a pick uses |
| `RouterWeights`, `validate` | `weights.rs` | per-request weights |
| `RouterResponse` | `call.rs` | the call loop's result with the deployment, group, retries and fallbacks used |
| `StreamPolicy`, `StreamOrigin` | `call/stream.rs` | what may take over a stream that broke |
| `Fallbacks`, `FallbackTarget` | `fallback.rs` | the parsed fallback lists |
| `Host`, `Attempt`, `RouterRequest`, `Output`, `RouterEvent`, `RoutingContext`, `SelectStage` | `host.rs` | the host seam |
| `Error`, `ConfigError`, `Unsupported`, `AttemptError`, `FailureKind` | `error.rs` | errors at build time, at call time, and per attempt |

The other modules hold one stage or query each: `access.rs` (access groups and
windows), `ptu.rs` (PTU shares), `health_routing.rs`, `precall.rs`,
`prerouting.rs`, `routing_groups.rs`, `routing_plugins.rs`, `retry.rs`,
`mock.rs` (`mock_testing_*` failures), `build.rs` (the deployment build),
`lookup.rs` and `host_index.rs` (the indexes the Python surface reads),
`model_info.rs` and `group_info.rs` (model info merges).

`Op` is the operation's name as a string, such as `acompletion`, `aembedding`,
`aresponses` or `aanthropic_messages`. The core passes it to the host and never
interprets it.

### `Host`

| Method | Called for |
| --- | --- |
| `invoke(attempt)` | every attempt. There is no native attempt path |
| `select(stage, ctx, candidates)` | the `Plugins`, `Strategy`, `Tags`, `TextOnly`, `RouteIncapable`, `MediaRefusing`, `EmbeddingBatch` and `OverrideCheck` stages |
| `pre_route(ask)` | pre-routing, and the whole pick for a custom strategy |
| `routing_plugin_models`, `callback_filters`, `pre_call_facts`, `provider_request` | the stages that need live Python objects |
| `allow_fallback(request, target)` | the access and budget checks before a cross-group fallback |
| `refuses_fallbacks(level, targets)` | whether a chain is refused for one of its dict targets, such as a server-owned federation field or an `oidc/` api key. Defaults to false |
| `sleep(duration)`, `observe(event)`, `cache()` | backoff, routing events, the routing state store |
| `secrets`, `validate`, `provider`, `added` | the deployment build |
| `model_info(lookup)` | cost map and capability lookups |
| `supports(op, provider)`, `can_select()` | what the host can serve. Both default to true, and `PythonHost` serves everything |

### `RouterRequest`

| Field | Meaning |
| --- | --- |
| `model` | the requested name, a model group, alias, routing group, deployment id or wildcard match |
| `body` | the request params the core's own checks read |
| `team_id`, `listing_team_id`, `proxy_admin`, `allowed_models`, `admin_model` | the caller facts the team and access filters read. `listing_team_id` is the team a failed level lists the group's deployments for |
| `specific_deployment` | the caller named a deployment's `litellm_params.model` |
| `pinned_deployment_id` | a deployment the session or affinity binding pinned |
| `num_retries`, `uncounted_retries`, `model_group_retry_policy`, `retry_policy_raises` | per-request retry settings, which win over the router's. `uncounted_retries` holds a `num_retries` that is not a count |
| `disable_fallbacks`, `fallbacks`, `context_window_fallbacks`, `content_policy_fallbacks`, `max_fallbacks` | per-request fallback settings |
| `same_group_fallbacks_only` | the request names a resource only one provider holds, such as an uploaded file, so it falls back inside its own group |
| `cooldown_time` | a per-request cooldown |
| `router_weights` | per-team weights, model group to deployment id to weight |
| `routing_strategy` | a per-request strategy override |
| `filter_tags`, `has_image` | inputs to the tag and text-only stages |
| `endpoint`, `web_search`, `route_handler`, `decisions_media`, `batch_create`, `embedding_batch` | what the call needs, which removes deployments that declare they cannot serve it |
| `wildcard_request` | the `provider/model` form of the name, for a lookup that cannot ask the host |
| `pass_through` | a pass-through route, which serves only pass-through deployments |
| `target_order`, `excluded_ids`, `retry_skipped_ids` | the order an order fallback targets, and ids the pick must skip |
| `pre_routes`, `unrouted`, `skip_weighted_failover` | whether and how pre-routing and weighted failover run |
| `passthrough_on_no_deployment` | a failed pick hands the attempt to the host under the requested name |
| `outcome_callbacks` | which of the router's outcome callbacks run for this request |
| `mock_failure`, `health_check_probe` | test failures and health check probes |
| `payload` | what the host needs to run the call itself, such as the Python kwargs |

### `Attempt` and `Output`

`Attempt` carries `op`, `request`, `deployment`, `name` (the requested name or
the fallback target), `model_group` (the group served, after fallbacks), `routed`
(the group a pre-routing hook selected, when it differs), `target_order`, `candidate_ids`, `retry` (0 for the first attempt in this
group), `fallback_depth`, `fallback_level` (0 for the requested name),
`timeout`, `params`, `scope`, `max_retries` and `model_group_size`.

`Output` carries the `value` the host hands back unchanged, `total_tokens`,
`blocked` (`ContentFilter` or `Refusal`, which the fallback rules read) and
`deferred`, which is set for a stream or realtime session whose outcome the host
reports when it ends, and `tracks_usage`, which says whether the attempt counts
against the deployment's per-minute usage when the host knows better than the
deployment's own limits.

### `RouterEvent`

| Event | Fields |
| --- | --- |
| `PickStarted` | `model` |
| `Selected` | `model_group`, `deployment_id`, `pick` |
| `Succeeded` | `deployment_id` |
| `AttemptFailed` | `deployment_id`, `kind`, `status` |
| `CooldownSet` | `deployment_id`, `cooldown` |
| `Retry` | `model_group`, `retry`, `delay` |
| `Fallback` | `from`, `to`, `order`, `depth`, `level`, `scope`, `parent`, `max_fallbacks`, `unroutable` |
| `FallbackDone` | `from`, `to`, `level`, `scope`, `parent`, `succeeded` |
| `Blocked` | `level`, `model`, `blocked` |
| `RetryLogged` | `scope`, `level`, `model`, `attempted_retries`, `cause` |
| `RetriesOpened` | `scope`, `max_retries`, `model_group_size` |
| `RetryBudget` | `scope`, `max_retries` |
| `PickFailed` | `level`, `model`, `error` |
| `LevelFailed` | `level`, `model`, `error`, `handling` |
| `MockFailed` | `mock`, `level`, `model`, `num_retries` |
| `RetryPolicyRaised` | `level`, `model`, `cause` |
| `RetryCountRaised` | `level`, `model`, `cause` |
| `FallingBack` | `level`, `model` |
| `OrderChainFailed` | `level`, `last_hop` |
| `FallbacksFailed` | `level`, `chain`, `last_hop` |
| `RetriesExhausted` | `model_group`, `attempted`, `max`, `level`, `picked` |

The driver uses the later events to raise the exception a caller expects at that
point, with the retry and fallback attributes set.

### `FailureKind`

The kinds are `RateLimit`, `ContextWindowExceeded`, `ContentPolicyViolation`,
`Authentication`, `PermissionDenied`, `NotFound`, `BadRequest`, `Timeout`,
`ApiConnection`, `InternalServer`, `ServiceUnavailable`, `BadGateway`,
`GuardrailIntervention` (never retried or fallen back from), `Declined` (litellm
rejected the request before calling, and it never cools a deployment down),
`CallerCredentialAuthentication` and `CallerCredentialRateLimit` (the caller's own
credential failed) and `Other`.
`AttemptError` adds `status`, `retry_after`, `num_retries`, the provider's error
text for the cooldown record, and `agentic_followup`, which marks an agentic
follow-up call that failed after the first call was served and is never retried.

### Build-time errors

`Router::new` returns `ConfigError` when the registry cannot be built, for
example from a malformed deployment or a duplicate id, or when the host cannot
serve the config. `Unsupported` names what is missing: a strategy or plugin on a
host that cannot select, or an operation and provider pair the host does not
support. Under `PythonHost` neither happens, since it serves every operation and
strategy.

## Related pages

- [State and cache keys](https://docs.litellm.ai/docs/router/api_reference/state_and_cache_keys.md)
- [Code map](https://docs.litellm.ai/docs/router/api_reference/code_map.md)