Skip to main content

v1.95.0rc1 - Claude Opus 5, MCP Gateway DCR & Rust /v1/messages

Deploy this version​

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.95.0-rc.1

Key Highlights​

  • Claude Opus 5 on day zero, everywhere - the new 1M-context Opus lands simultaneously on Anthropic, Amazon Bedrock (including the us, eu, au, jp, and global inference profiles), Google Vertex AI, and Azure AI Foundry, with adaptive thinking, xhigh reasoning effort, computer use, PDF input, and prompt caching all recorded in the cost map.
  • Gemini 3.6 Flash and Gemini 3.5 Flash Lite - day-0 pricing on both Google AI Studio and Vertex AI, at $1.50/$7.50 and $0.30/$2.50 per 1M tokens respectively.
  • MCP gateway grows a real front door - an always-on aggregate DCR discovery endpoint, identity-only session tokens, RFC 8707 resource indicators on upstream OAuth legs, MCP server support on the Anthropic /v1/messages API, and a standalone /connect route that no longer depends on the Chat UI flag.
  • The Rust gateway takes over /v1/messages - native Anthropic Messages now routes through the axum gateway behind LITELLM_RUST, joined by a 1:1 port of the Responses API WebSockets surface, BaseAWSLLM credential resolution and SigV4 in litellm-core, and Bedrock audio transcription over the Python-to-Rust bridge.
  • SAML 2.0 SSO for the admin UI - a second enterprise SSO path alongside the existing OIDC flow.
  • The dashboard finishes its shadcn and DataTable migration - roughly twenty routes move onto shadcn and the shared composable DataTable, plus new react-hook-form and zod form infrastructure behind Organization Settings and Create Organization.
  • Budget resets get correct - a configurable budget_reset_time of day, word-form budget_duration values that no longer silently collapse to daily, and repair for users and teams whose budget_reset_at was left NULL.

New Models / Updated Models​

New Model Support (16 new pricing entries)​

ProviderModelContext WindowInput ($/1M tokens)Output ($/1M tokens)Features
Anthropicclaude-opus-51M$5.00$25.00Reasoning (adaptive + xhigh), vision, computer use, function calling, PDF input, prompt caching, native structured output
Amazon Bedrockanthropic.claude-opus-51M$5.00$25.00Reasoning (adaptive + xhigh), vision, computer use, function calling, PDF input, prompt caching, parallel tool use config
Amazon Bedrockglobal.anthropic.claude-opus-51M$5.00$25.00Same as above
Amazon Bedrockus.anthropic.claude-opus-51M$5.50$27.50Same as above
Amazon Bedrockeu.anthropic.claude-opus-51M$5.50$27.50Same as above
Amazon Bedrockau.anthropic.claude-opus-51M$5.50$27.50Same as above
Amazon Bedrockjp.anthropic.claude-opus-51M$5.50$27.50Same as above
Google Vertex AIvertex_ai/claude-opus-51M$5.00$25.00Reasoning (adaptive + xhigh), vision, computer use, function calling, PDF input, prompt caching
Google Vertex AIvertex_ai/claude-opus-5@default1M$5.00$25.00Same as above
Azure AI Foundryazure_ai/claude-opus-51M$5.00$25.00Reasoning (adaptive + xhigh), vision, computer use, function calling, PDF input, prompt caching
Google AI Studiogemini/gemini-3.6-flash1.05M$1.50$7.50Reasoning, vision, audio input, video input, PDF input, web search, URL context, prompt caching
Google AI Studiogemini/gemini-3.5-flash-lite1.05M$0.30$2.50Reasoning, vision, audio input, video input, PDF input, web search, URL context, prompt caching
Google Vertex AIgemini-3.6-flash1.05M$1.50$7.50Same as above
Google Vertex AIvertex_ai/gemini-3.6-flash1.05M$1.50$7.50Same as above
Google Vertex AIgemini-3.5-flash-lite1.05M$0.30$2.50Same as above
Google Vertex AIvertex_ai/gemini-3.5-flash-lite1.05M$0.30$2.50Same as above

Beyond the new entries, this release raises the advertised context window on azure_ai/claude-opus-4-6, claude-opus-4-7, and claude-opus-4-8 from 200K to 1M, marks the DeepSeek V4 flash and pro families as reasoning models, and records supports_mid_conversation_system across the Claude Fable 5, Sonnet 5, and Opus 4.8 entries on Vertex AI and Azure AI. No pricing entries were removed.

Features​

Bug Fixes​

  • Anthropic
    • Only inject cache_control when the request does not already carry one - PR #33886
    • Strip uniqueItems and other unsupported array and object constraints from the output_format schema - PR #34313
    • Strip the remaining output_format schema keywords Anthropic rejects - PR #34319
  • Amazon Bedrock
    • Hoist Codex additional_tools input items to top-level tools on Bedrock Mantle - PR #33228
  • Google Vertex AI
    • Incrementally parse accumulated Gemini stream JSON so a multi-value payload no longer wedges the stream - PR #34320
    • Handle an explicit outputInfo: null in the Vertex AI batch response - PR #34473
  • Azure AI Foundry
    • Advertise the 1M context window for Claude Opus 4.6 and newer on Foundry - PR #34556
  • Fireworks AI
    • Restore the application/json Content-Type header, fixing a 415 response - PR #33929

LLM API Endpoints​

Features​

Bugs​

  • Responses API
    • Preserve reasoning content through prompt hooks - PR #33422
    • Keep one chat completion id per stream and always stream completed responses - PR #34539
    • Strip include_usage from stream_options instead of dropping the whole param - PR #34549
  • Anthropic /v1/messages
    • Model-aware mid-conversation system handling for Claude on Vertex AI and Azure - PR #33807
    • Route agentic-hook /messages requests back to Python for all stream modes - PR #34126
    • Backfill usage on non-streaming Bedrock Mantle /v1/messages responses - PR #34446
  • Batches
    • Paginate the managed batch list by unified_object_id cursor - PR #34192
    • Resolve a managed unified input_file_id to a storage URL with an ownership check before dispatch - PR #34474
    • Make managed-file resolution additive, restoring the fall-back for missing-row and lookup errors - PR #34584
  • Realtime
    • Emit the Nova Sonic session.created event on connect and session.updated on session.update - PR #34133
    • Install the bedrock-realtime extra for Nova Sonic realtime - PR #34426
  • A2A
    • Accept semver protocolVersion values such as 0.3.0 in agent cards - PR #34154
    • Allow optional securityScheme fields so /public/agent_hub no longer 500s - PR #33897
    • Route /a2a through the gateway component - PR #34958
  • General
    • Return 400 instead of 500 for chat completions sent without messages - PR #34547
    • Add queued to the Interaction status enum - PR #34135

Management Endpoints / UI​

Features​

  • Auth & SSO
    • SAML 2.0 SSO for the admin UI - PR #31429
    • Stamp the outgoing user param with the key hash via overwrite_user_with_key_hash - PR #34417
  • Organizations
    • RESTful PATCH /v2/organization/{organization_id} - PR #32350
    • Rebuild Organization Settings on react-hook-form and zod with a dirty-field PATCH - PR #34324
    • Migrate the Create Organization form to shadcn and react-hook-form - PR #34552
  • Virtual Keys
    • Block and unblock a key directly from the key info page - PR #34116
    • Deep-link the virtual key detail view via a ?key= query param - PR #34591
    • Surface a key's budget_reset_at in key info and the keys table - PR #34113
  • Models + Endpoints
    • Give each Models + Endpoints tab its own path - PR #34327
    • Extract shared tab-routing helpers and adopt them in Models + Endpoints - PR #34435
    • Make the DB config-reload interval configurable from config.yaml and the Admin UI - PR #34130
  • Shared DataTable migration
  • shadcn migration
  • Typed management API
    • Type the PATCH /team/{team_id} request body - PR #34195
    • Derive the dashboard object_permission type from the generated schema - PR #34454

Bugs​

  • SCIM
    • Use members_with_roles as the source of truth for group membership - PR #34162
    • Prune a deleted user from teams' members_with_roles - PR #34180
    • Parse the membership id from the filtered PATCH path when value is omitted - PR #34181
    • Sync the team roster and dedup teams on existing-user email upsert - PR #34183
  • Teams & Users
    • Make team member add atomic so a concurrent add cannot lose a member - PR #34185
    • Restore the atomic user upsert when adding team members - PR #34457
    • Route the JWT default-team into memberships instead of the create payload - PR #33082
    • Populate user_email on UserAPIKeyAuth for JWT auth - PR #34174
    • Validate default team values in Default User Settings - PR #34815
  • Config & credentials
    • Prevent provider key exposure through URL-valued model destinations and fallbacks - PR #34189
    • Stop save_config from snapshotting environment_variables into the DB - PR #34119
    • Hash a caller-supplied key in the key update audit log object_id - PR #34632
    • Surface SSO and SMTP settings supplied as process env vars - PR #33576
    • Reflect REDIS_* env cache config and stop the UI overwriting the stored password - PR #34160
  • CLI
    • Stable port and persisted master key for lite autoroute up - PR #34026
    • Discover models via /v1/models so an AI-API-only key works for autoroute - PR #34259
  • Dashboard
    • Find logs by request id across pages and dates - PR #31743
    • Scope and bound the End User filter on the logs page - PR #34579
    • Bind the key duration input to a single Form.Item so a pre-filled expiry submits - PR #34521
    • Stop cloning body-carrying requests into stream uploads in the fetchClient middleware - PR #34122
    • Distinguish the response cache from provider prompt caching - PR #34138
    • Keep entity usage tabs aligned with their panels - PR #34573
    • Add a tooltip to the Active key status badge - PR #34109
    • Restore the Add MCP Server dialog size and header spacing, and match MCP Servers tabs to the dashboard line tab pattern - PR #34679, PR #34685
    • Center vertical toolbar dividers and truncate long team names in the models table team dropdown - PR #34684, PR #34689
    • Serve /ui/assets from the nginx image instead of the SPA fallback, and bundle provider logos as static imports - PR #34066, PR #34125, PR #34163
    • Return Models + Endpoints tabs to in-memory routing while keeping the ?model drill-in - PR #34629

AI Integrations​

Logging​

  • OpenTelemetry
    • Keep an MCP tool call in one trace, anchored to its own request - PR #34537
    • Stamp an MCP tool failure on the request that carried it - PR #34551
  • Langfuse
    • Send the v4 ingestion header for the otel callback - PR #33907
  • Prometheus
    • Populate cache write token metrics for OpenAI-style usage - PR #34803
  • General
    • Record cost and usage reported by the upstream target on passthrough routes - PR #34590
    • Move the logs end-user filter onto /management/v1 - PR #34691
    • Sanitize per-key callback config out of logged metadata - PR #32583
    • Match the exact class in callback dedup so a custom subclass no longer blocks a built-in logger - PR #34804
    • Stop scheduling the sync failure_handler concurrently with async_failure_handler - PR #34306
    • Surface env-var-sourced theme and logging-callback settings, and drop the misleading os.environ tooltip - PR #34156, PR #34305

Guardrails​

  • General
    • Add only_scan_new_messages for per-session incremental scanning - PR #33278
    • Add a run_in_parallel opt-in for concurrent pre_call and post_call guardrails - PR #33770
    • Add DeepKeep as a custom guardrail - PR #33844
    • Merge model-level guardrails before pre_call_hook - PR #29654
    • Classify all 4xx HTTPException guardrail blocks as intervened - PR #33821
    • Stop reporting a no-op guardrail as applied on passthrough - PR #34411
    • Keep guardrail information in spend logs when the caller sends its own metadata - PR #34458
    • Resolve judge_model credentials via a lazy Router lookup in llm_as_a_judge - PR #34509
  • Model Armor
    • Sanitize error details by default - PR #33908
    • Handle None metadata in the post_call response processor - PR #34405
  • Compresr / Headroom
    • Compress content-parts messages in the headroom guardrail for Anthropic traffic - PR #34586
    • Derive tokens_saved when the compression service omits it - PR #34578
    • Preserve cache_control breakpoints in the compresr write-back - PR #34660
  • Straiker
  • Bedrock Guardrails
    • Stop replaying expired Google OIDC tokens to STS on guardrail auth - PR #34637
  • Dashboard
    • Hide guardrail group headers when only one group has entries - PR #33885

Spend Tracking, Budgets and Rate Limiting​

  • Budgets
    • Configurable budget_reset_time of day - PR #31007
    • Reset users and teams whose budget_reset_at is NULL - PR #33623
    • Resolve word-form budget_duration so it no longer silently resets daily - PR #34250
    • Enforce a global max_budget against the resettable proxy budget row so budget_duration is honored - PR #33732
    • Set budget_reset_at when a JWT upsert seeds a budget_duration - PR #34050
    • Reject failed atomic budget reservations under fail_closed_budget_enforcement - PR #34429
    • Handle a tz-aware temp_budget_expiry, apply temp_budget_increase for cache-hit keys, and derive the increase without mutating the token - PR #33840, PR #33841, PR #34121
    • Raise the dashboard session budget default to $1 and make it configurable in config and the Admin UI - PR #34146
  • Cost tracking
    • Map OpenAI cache_write_tokens for prompt cache creation billing - PR #34046
    • Track prompt compression saved tokens in daily spend aggregates - PR #33810
    • Attribute org spend for team-linked credentials minted without an org_id - PR #34577
    • Gate an unsupported service_tier on drop_params for the Bedrock Mantle Responses API - PR #34058
    • Raise the /spend/logs/v2 page_size cap to 1000 - PR #33994
  • Cost Optimization page
    • Add spend-by-tool and cache leakage views - PR #33978
    • Add configuration tabs - PR #33899
    • Mark Cost Optimization as beta in the left nav - PR #34984
    • Anchor the savings line at a $0 range start and swap the methodology Collapse for a shadcn HoverCard - PR #34453, PR #34598
    • Keep the cache leakage time range picker inline at narrow widths and the date picker on the right - PR #34439, PR #34885
  • Tool spend
    • Roll up tool spend daily instead of scanning SpendLogs - PR #34675
    • Cap the /v1/tool/spend window at 30 days and bound every SpendLogs read - PR #34582
  • Dashboard
    • Stop the key-edit form 403ing on non-budget saves - PR #34112

MCP Gateway​

  • Dynamic Client Registration
    • Always-on aggregate gateway DCR discovery front door - PR #33174
    • Identity-only session tokens for the gateway DCR front door - PR #33182
    • Admit gateway DCR session bearers at the aggregate /mcp scope - PR #33190
    • Return the DCR client's own redirect_uris to stop the /callback self-redirect loop - PR #33756
    • Fall through to an ephemeral DCR mint when passthrough authorize has no client_id - PR #33884
  • OAuth
    • Send RFC 8707 resource indicators on upstream OAuth legs - PR #34265
    • Migrate client_credentials (M2M) onto the v2 resolver arm - PR #32259
    • Delete the unreachable v1 OBO handler and gate REST OAuth on the v2 resolver - PR #34407
    • Let an admin-pinned issuer drive OAuth discovery for url-less servers - PR #34065
    • Log actionable OAuth discovery failures for misconfigured server urls - PR #34225
    • Store the enterprise IdP identity assertion at SSO login for EMA egress - PR #34072
    • Standalone /connect route for MCP OAuth, decoupled from the Chat UI flag - PR #34334
  • Servers & tools
    • Support MCP servers on the Anthropic /v1/messages API - PR #33631
    • Add Google Sheets, Drive, Calendar, and Docs to the OpenAPI registry, and move Drive to the official streamable HTTP MCP server - PR #34059, PR #34322
    • Attach resolved OAuth credentials to OpenAPI spec_path tool calls - PR #34063
    • Stop leaking upstream server credentials in a tool-call 403 - PR #34340
    • Use a toolset row's stored tool name as written - PR #34559
    • Keep a key's MCP toolsets when saving an edit - PR #34452
    • Consolidate exception-tree walkers into one shared faults traversal - PR #33183

Performance / Loadbalancing / Reliability improvements​

  • Rust core
    • Port BaseAWSLLM auth, credential resolution, and SigV4 to litellm-core as a base provider - PR #33888
    • Bedrock audio transcription via the Rust core over the Python-to-Rust bridge - PR #33990
    • Honor pre-computed Entra ID auth for Azure /messages - PR #34107
  • Streaming & core
    • Build the per-chunk Delta directly instead of setattr/delattr churn - PR #33992
    • Fast-path SafeAttributeModel.__delattr__ for declared fields - PR #33993
    • Forward SageMaker stream events as they arrive to cut TTFT - PR #34338
  • Router
    • Edit fallback chains from router settings - PR #32841
    • return_raw_model_name toggle for the complexity router's response model field - PR #33875
    • Show in the log drawer and session sidebar when an auto-router served a request - PR #34434
    • Don't cool down the parent deployment on an advisor sub-call failure - PR #33792
    • Stop custom model_info leaking onto the shared backend cost map key, and propagate capability flags to it - PR #34041, PR #34047
    • Honor request-level num_retries over litellm_settings.num_retries, and stop per-deployment num_retries double-counting as provider max_retries - PR #34124, PR #34129
    • Release the pre-routing strategy slot when a deployment is replaced or deleted - PR #34564
    • Treat malformed cost-map token limits as absent on /v1/models - PR #33903
  • Proxy reliability
    • Make in-memory and disk cache increments atomic - PR #34013
    • Share CLI SSO login sessions across workers without enable_redis_auth_cache - PR #33261
    • Avoid a DB outage during planned RDS IAM rotation - PR #34749
    • Stop litellm/proxy from shadowing installed packages on sys.path - PR #34656
    • Bake non-root prisma engines at /opt/prisma so migrations run offline for any uid - PR #34325
    • Pass an explicit Python version request to uv tool install - PR #34750
  • Dependencies & build

Documentation Updates​

  • Add a TLDR section to the PR template - PR #34203
  • Ask for a numbered list of reproduction steps in the issue template - PR #34207

PR roll-up by ownership area​

PRs by ownership area (total: 292)

  • UI: 54
  • Other (CI / chore / tests / build / version bumps): 46
  • Spend / Budgets / Rate Limits: 33
  • LLM API Endpoints: 31
  • MCP: 29
  • Auth & Management: 28
  • Performance: 22
  • Guardrails: 19
  • Logging: 14
  • Models & Providers: 14
  • Docs: 2

End-to-End Testing​

We are investing heavily in end-to-end testing to cut regressions and make LiteLLM more stable release over release. Every version is exercised by a live suite that runs against a real deployed proxy and hits real provider endpoints, not mocks, so the behavior we validate is the behavior you get in production.

This window added 62 test PRs, the largest single-release investment in the suite so far. New coverage lands on live A2A agents, /v1/images/edits, /openai and /vllm chat passthrough cost logging, credential-backed /v1/messages, Azure AI Foundry and Anthropic /v1/messages through the Rust bridge, MCP access-group tool selection at key creation, a real Linear OAuth MCP driven through chat completions, budget and rate-limit resets across personal, team, and team-member keys, and a weekly session-anomaly load test against real providers. The Admin UI Playwright suite moved under tests/e2e/ui, and a run of hardening PRs removed cross-suite races on control-plane writes and data-plane sync lag.

New Contributors​

  • @lyb0307 made their first contribution in PR #33228
  • @jyeung-r7 made their first contribution in PR #33623
  • @vineetpuranik made their first contribution in PR #33940

Full Changelog​

https://github.com/BerriAI/litellm/compare/v1.94.0-rc.1...v1.95.0-rc.1

🚅
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.
Learn more →