Skip to main content

v1.94.0rc1 - Router Plugins, MCP Client-Held Credentials & Shared DataTable UI

Deploy this version​

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.94.0-rc.1

Key Highlights​

  • Router plugin pipeline and Auto-Router v2 - a new Router(plugins=[...]) extension point, resolvable from proxy YAML config, plus soft-floor adaptive mode, opt-in (now default) session affinity, multi-model tier random-pick, and user-triggered escalation keywords for the complexity router.
  • MCP client-held credentials mature - interactive SSO sign-in for dcr_bridge oauth_delegate DCR clients, client-held refresh envelopes, gateway-bound envelopes minted at the token endpoint, issuer-anchored OAuth discovery (RFC 8414 §3.3) to close the authorization-server mix-up, and ID-JAG support for MCP egress.
  • Shared DataTable dashboard migration - Virtual Keys, Teams, Guardrails, Tags, Vector Stores, Prompts, Skills, AI Hub, MCP Toolsets, and Policy Attachments all move onto the shared composable DataTable.
  • Python 3.14 support - the requires-python cap moves to <3.15, pyo3 rises to 0.29 so the native Rust bridge compiles, and redisvl / pypdf / openapi-core are unblocked on 3.14.
  • Per-model prompt cache minimums - prompt_cache_min_tokens is now recorded across the Anthropic and Bedrock Claude cost map entries, and the router resolves the real per-model minimum instead of a flat 1024.

New Models / Updated Models​

New Model Support (5 new pricing entries)​

ProviderModelContext WindowInput ($/1M tokens)Output ($/1M tokens)Features
Google AI Studiogemini/gemini-omni-flash-preview1.05M$1.50$9.00 (video output $17.50)Reasoning, vision, audio input, video input/output
Google Vertex AIgemini-omni-flash-preview1.05M$1.50$9.00 (video output $17.50)Reasoning, vision, audio input, video input/output
Amazon Bedrock (Mantle)bedrock_mantle/openai.gpt-5.6-sol272K$5.50$33.00Reasoning, vision, function calling, prompt caching, Responses API
Amazon Bedrock (Mantle)bedrock_mantle/openai.gpt-5.6-terra272K$2.75$16.50Reasoning, vision, function calling, prompt caching, Responses API
Amazon Bedrock (Mantle)bedrock_mantle/openai.gpt-5.6-luna272K$1.10$6.60Reasoning, vision, function calling, prompt caching, Responses API

Beyond the new entries, this release records prompt_cache_min_tokens on the Anthropic and Bedrock Claude families (512-4096 depending on the model), marks the gpt-realtime family as mode: realtime rather than chat, flags Gemini image-generation models as non-reasoning, and corrects the Fireworks glm-5p2 prompt-cache read price.

Features​

Bug Fixes​

  • Anthropic
    • Emit message_start exactly once in the Responses stream adapter - PR #32667
    • Translate raw adaptive thinking for pre-4.6 models on Chat Completions and Bedrock Converse - PR #32944
    • Drop the incompatible temperature param when downgrading adaptive thinking for pre-4.6 models on pass-through - PR #33244
    • Stop a 500 on a combined thinking-plus-signature streaming chunk - PR #33505
    • Self-heal on missing thinking-signature errors from Bedrock and Vertex - PR #33719
    • Drop empty content_block_delta events in the Anthropic adapter - PR #33315
    • Honor the messages request timeout - PR #33418
    • Unblock lite autoroute proxy deps, adaptive thinking, and thinking-plus-signature streaming in the CLI - PR #33507
  • Google Vertex AI
    • Surface Gemini grounding toolUsePromptTokenCount in Usage - PR #33533
    • Exclude Google Search grounding tokens from input token billing - PR #33742
    • Mark Gemini image generation models with supports_reasoning: false - PR #32836
  • Amazon Bedrock
    • Forward AWS credential kwargs into litellm_params so the responses bridge keeps WIF auth - PR #32956
  • Fireworks AI
    • Bill prompt-cache hits at the cache_read rate - PR #33714
    • Correct the glm-5p2 prompt-cache read price to $0.14/1M - PR #33796
  • OpenAI
    • Mark realtime-only gpt-realtime models as mode: realtime - PR #33728

LLM API Endpoints​

Features​

  • General
    • Add an x-litellm-model-name response header carrying the deployment model string - PR #33698

Bugs​

  • Responses API
    • Continue MCP gateway tool turns from the final response and surface failures - PR #33025
    • Clamp max_output_tokens below the API minimum - PR #33098
    • Intercept web search on the Responses API - PR #33129
  • Pass-through
    • Stop classifying plain predict / search paths as Vertex - PR #33658
    • Stop treating an upstream model body field as a LiteLLM model on auth-enforced pass-through routes - PR #33710
  • General
    • Surface upstream connection resets instead of an empty 200 stream - PR #33222
    • Source /v1/models token limits from the cost map instead of Router.get_model_group_info - PR #33721
    • Treat malformed configured token limits as absent on /v1/models - PR #33864

Management Endpoints / UI​

Features​

  • UI (shared DataTable migration)
    • Rebuild the Virtual Keys and Teams tables on the shared DataTable - PR #32991, PR #33128
    • Migrate the guardrails, tags, and policy attachments tables onto the shared DataTable - PR #33303, PR #33314, PR #33827
    • Migrate the vector stores, prompts, and skills tables, plus five more simple tables - PR #33343, PR #33548
    • Migrate the AI Hub, public hub, and MCP Toolsets tables - PR #33629
  • UI
    • Convert the endpoint usage charts to shadcn/recharts - PR #32723
    • Adopt openapi-react-query ($api) and convert useCustomers - PR #32949
    • Working Test Connection for the complexity auto router, and adaptive routing settings in Auto-Router v2 - PR #32950, PR #33146
    • Require an embedding model for the semantic auto router - PR #33313
    • Configure Anthropic automatic prompt caching from the Admin UI - PR #33581
    • Show the exact license expiration date in the usage cards - PR #33478
    • Move Caching out of Experimental into Developer Tools - PR #33432
    • Add a reusable BetaBadge and use it for the Projects sidebar item - PR #33449
    • Add a personal Logs view scoped to the current user in the chat UI - PR #33829
    • Left-anchor the Create Key and Create Team CTAs - PR #33248
    • Consolidate the Add/Edit credential modals into one CredentialModal - PR #32572
    • Colocate the mcp-servers view while keeping the shared mcp_tools surface - PR #32968
    • Standardize debounce waits behind a shared DEBOUNCE_WAIT_MS constant and migrate value and callback debounces to react-pacer - PR #33040, PR #33042, PR #33043, PR #33041
    • Remove the unmounted UsageIndicator and the Hide Usage Indicator flag - PR #33482
  • Auth & Management
    • Ingest and round-trip SCIM entitlements and roles user attributes - PR #33587
    • Add a disable_auto_add_proxy_admin_to_teams flag - PR #33563
    • Add lite up / lite down to ambiently route Claude Code through the proxy - PR #33231

Bugs​

  • UI
    • Address Virtual Keys redesign review nits - PR #33112
    • Drop w-full from page-content wrappers to remove 32px horizontal overflow - PR #33118
    • Render the sidebar scrollbar with the shadcn ScrollArea - PR #33124
    • Show and allow editing team model aliases after team creation - PR #33047
    • Respect litellm_key_header_name in BYOK credential save and workflow-run fetches - PR #33103
    • Stop credential edit from persisting the masked API key - PR #33797
    • Resolve chat routes at render time so navigation works under server_root_path - PR #33446
    • Navigate to /ui/login/ with a trailing slash via hard navigation - PR #33561
    • Show all teams in the policy attachment form for admins - PR #33628
    • Remove the Chat item from the dashboard left nav - PR #33647
    • Migrate tag deletion to the shared DeleteResourceModal - PR #33795
    • Stop sending the complexity-router pseudo-model to /health/test_connection - PR #33498
    • Filter embedding models out of the complexity tab dropdowns, require all tiers, and validate inline - PR #32978
  • Auth & Management
    • Route the master key to team-scoped models - PR #32926
    • Stop unrecognized model namespaces slipping through provider wildcard keys - PR #32979
    • Persist key_type so the UI shows the correct key scope instead of "All Proxy Models" - PR #33115
    • Enforce a minimum custom key length and mask short keys in key_name - PR #33462
    • Paginate through all pages when fetching SSO service principal group assignments - PR #33149
    • Scope the JWT enterprise gate to actual JWTs - PR #33296
    • Stop enforcing the UI session budget on CLI login tokens - PR #33312
    • Surface actionable CLI SSO errors when the CLI and proxy versions skew - PR #33309
    • Resolve team wildcard credentials for vector store files - PR #33649
    • Make CLI output ASCII-only so it does not crash legacy Windows consoles - PR #33465

AI Integrations​

Logging​

  • OpenTelemetry
    • Push-based OTLP billable-request metering for enterprise deployments - PR #31592
    • Restore proxy-level error.* attributes on v2 failure spans - PR #33664
  • Langfuse
    • Build a per-request OTLP exporter from key- and team-level dynamic Langfuse credentials - PR #32437
  • Prometheus
    • Expose video duration and image count consumption metrics - PR #33138
    • Read v3 rate limiter remaining values for the per-key model gauges - PR #33119
  • S3
    • Sanitize slashes in the response-id-derived object key file name - PR #33271
  • General
    • Add user- and team-level spend and budget to StandardLoggingPayload metadata - PR #33459
    • Add structured budget fields to budget rejection failure logs - PR #33460
    • Preserve callback order in get_combined_callback_list - PR #33005
    • Classify async anthropic_messages and generate_content as async - PR #33589
    • Redact the async complete streaming response for custom callbacks - PR #33106
    • Redact assistant tool call arguments in spend logs - PR #33111
    • Never log raw virtual keys in key insertion debug output - PR #33268

Guardrails​

  • Straiker
    • Add the Straiker guardrail integration - PR #33781
  • Compresr
    • Add the Compresr guardrail for query-aware context compression - PR #33295
  • Bedrock Guardrails
    • Add a resource-less InvokeGuardrailChecks detect-only mode - PR #33299
  • Model Armor
    • Restore reference attachments via skip_unscannable_attachments and remove the attachment count cap - PR #33554
  • Lasso
    • Send source.type for Used By attribution - PR #33090
  • LLM Guard
    • Apply the sanitized prompt returned by the moderation API to the request - PR #33331
  • General
    • Streaming text transformation in generic_guardrail_api - PR #33110
    • Forward optional metadata on POST /guardrails/apply_guardrail - PR #33067
    • Run apply_guardrail-style model-level pre_call guardrails at the deployment hook - PR #33136
    • Show YAML-defined guardrails in the Guardrail Monitor - PR #32853
    • Walk custom_tool_call_output items in the shared content helpers - PR #32969
    • Use StandardLoggingGuardrailInformation in the xecguard logging hook, and sanitize the scan result before recording it - PR #32911, PR #32935
    • Remove the stray docstring from the singulr module for consistency - PR #33800

Spend Tracking, Budgets and Rate Limiting​

  • Budgets
    • Enforce user budget on team keys at read time and on reservation, with a UI opt-out - PR #32005
    • Coerce default_internal_user_params.max_budget to a float on config load - PR #32434
  • Rate Limiting
    • Enforce max_parallel_requests as a per-slot concurrency gauge - PR #32441
  • Cost Tracking
    • Use the provider-reported usage cost for OpenRouter streams - PR #32255
    • Track cost for unmanaged Bedrock batches and generalize the flag - PR #32315
    • Track unauthenticated pass-through requests in spend logs - PR #32410
    • Track LLM completion usage and spend for /v1/rag/query - PR #32438
    • Bill partial streamed spend when the client disconnects mid-stream - PR #33736
    • Remove the dead user-cache lookup with a None key in the spend-update path - PR #33555

MCP Gateway​

  • Client-Forwarded Credentials (dcr_bridge / oauth_delegate)
    • Interactive SSO sign-in for dcr_bridge oauth_delegate DCR clients - PR #32946
    • Mint a gateway-bound envelope at the token endpoint, and add a client-held refresh envelope for the flow - PR #32828, PR #32980
    • Extract the dcr_bridge token flow into bridge_token_flow.py - PR #33141
    • Persist config.yaml DCR clients in a server-scoped store so refresh survives token expiry - PR #33768
    • Surface rejected delegate-auth upstream tokens as a connect-time 401, and relay upstream OAuth token and DCR rejections instead of a generic 500 - PR #32741, PR #33113
    • Make the preemptive-401 OAuth challenge decision mode-aware - PR #33586
  • OAuth & Identity
    • Issuer-anchored OAuth discovery (RFC 8414 §3.3) to close the authorization-server mix-up - PR #33450
    • Add ID-JAG (identity assertion authorization grant) support for MCP egress - PR #31516
    • Persist discovered OAuth endpoints and keep the last known good set on a failed re-discovery - PR #33286
    • Discover missing OAuth scopes and token_url when authorization_url is set manually - PR #33317
    • Cap the per-user OAuth token cache TTL at the token's own lifetime - PR #33346
  • Tools & Permissions
    • Per-server outcomes for aggregate tools/list and truthful single-server REST statuses - PR #33153
    • Expand toolset grants in the shared permission primitives so tools/call honors them - PR #33612
    • Index authed request-time tools missing from the semantic filter startup index - PR #33318
    • Keep the MCP reference intact when the semantic filter narrows tools - PR #33584

Performance / Loadbalancing / Reliability improvements​

  • Routing
    • Add a Router(plugins=[...]) routing-plugin pipeline, resolve auto-router routing plugins from proxy YAML config, and resolve router_settings.plugins dotted paths from installed packages - PR #32972, PR #33251, PR #33644
    • Soft-floor adaptive mode and random-pick multi-model tiers for the complexity router - PR #32947, PR #32967
    • Opt-in session affinity for the complexity router, then enabled by default, with the session id derived from Anthropic metadata.user_id - PR #33126, PR #33500, PR #33723
    • User-triggered escalation keywords, and per-tier semantic keyword prompts in the configure wizard - PR #33656, PR #33508
    • Honor a per-request routing_strategy from key/team router_settings, and apply team/key enable_tag_filtering to tag routing - PR #33429, PR #33436
    • Tag-aware pre-routing strategy selection for a shared model_name - PR #33691
    • Resolve the prompt cache minimum per model instead of a flat 1024 - PR #33637
    • Enforce context-window pre-call checks for Responses API input - PR #33706
    • Correct the Responses API tool_choice shape and propagate alias litellm_params in the auto router - PR #32974
    • Return an empty dict from _classifier_call_metadata when metadata is absent - PR #33452
    • Cast model_info cost values to float in _set_model_group_info - PR #33556
  • Reliability
    • Route Azure Anthropic /messages through Rust behind rust:true - PR #33616
    • Stop pinning large request payloads past request end - PR #33455
    • Stop a stale auth cache re-publish to Redis so key updates propagate across replicas - PR #33565
    • Honor the ssl value instead of key presence when building the async Redis connection pool - PR #32590
    • Reap orphaned Prisma query-engine processes when a worker dies - PR #33424
  • Deployment
    • Add per-component PodDisruptionBudget and topologySpreadConstraints to the componentized Helm chart - PR #33430
    • Bake the Prisma CLI and engines at a fixed path so fresh-DB migrations work for any uid offline - PR #33853
    • Restore the litellm-proxy-extras source dir in the runtime images - PR #33592

Documentation Updates​

  • Add a QA runbook section to the PR template - PR #32965
  • Add a router plugin reference catalog - PR #33746
  • Align the e2e skip-vs-fail docs with the hard-fail contract and scope the no-unit-tests rule - PR #33755
  • Add provider coding standards and provider abstraction standards for litellm-rust, and require the official Rust Style Guide in the agent rules - PR #33833, PR #33865, PR #33867

PR roll-up by ownership area​

PRs by ownership area (total: 236)

  • Other (CI / chore / tests / build / version bumps): 69
  • UI: 39
  • Performance / Routing / Reliability: 26
  • Models & Providers: 21
  • MCP: 17
  • Guardrails: 14
  • Logging: 13
  • Auth & Management: 13
  • Spend / Budgets / Rate Limits: 9
  • LLM API Endpoints: 9
  • Docs: 6

New Contributors​

  • @Napuh made their first contribution in PR #32667

Full Changelog​

https://github.com/BerriAI/litellm/compare/v1.93.0-rc.1...v1.94.0-rc.1

🚅
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.
Learn more →