Skip to main content

1.103.0rc1 - Config File Ownership, Fuse Routing & Gateway Hardening

Deploy this version​

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.103.0-rc.1

The published GitHub tag is v1.103.0-rc.1. These notes compare it with v1.102.0-rc.1, the previous release candidate cut from main. Changes backported onto rc/1.102.0 and already shipped in v1.102.0 are omitted

Breaking Changes

These callouts cover changes to behavior available in v1.102.0, the latest stable release

The config file now owns every setting it declares, and the database no longer overrides it. One rule replaces the per-key mix of DB-wins, config-wins and merged precedence: if config.yaml declares a key, the file owns it and a runtime write to that key is refused with a 400 naming the file to edit, instead of being stored and silently ignored. This covers POST /config/field/update, POST /config/field/delete, POST /config/update, and the allowed_ips routes. Keys the file leaves out still come from the database and stay editable. GET /config/field/info and GET /config/list now resolve through the same store and report source and editable, and a config-owned field reports the value the file declares, so an os.environ/... reference is returned as written rather than resolved. In the Admin UI a config-owned field renders read-only. Startup warns once per key whose stored database value is being ignored, and the refusal carries the same sentence plus stored_database_value_ignored: true. Move any setting you edit at runtime out of the config file, or edit the file and restart. See PR #41779, PR #41862, PR #41868, PR #41931, PR #41985, PR #42009

Budgets are re-checked on every router fallback target. A request that starts on a free model and falls back to a paid one is now gated at the fallback, and targets the caller cannot pay for are skipped rather than served. The primary attempt is unchanged. Workflows that relied on a free primary carrying an over-budget key onto a paid fallback will now receive the next affordable target or a budget error. See PR #41379

An organization or project max_budget of 0 now means zero allowance, not unlimited. This matches the existing key, team and user semantics. Set max_budget to null for unlimited. See PR #41271, PR #41997

A key's own per-model rpm/tpm override now beats the team's per-model limit. Documented precedence (key metadata over team metadata) is now what the code does, on chat, batches and /cost/predict-cache. A key that overrides only RPM still has the team TPM pool enforced, and vice versa. Review teams that relied on the per-model team cap clamping keys that declare their own. See PR #41302

Delegated MCP OAuth requires admission. Legacy delegated MCP integrations that were not admitted to LiteLLM now receive 401 until they admit or migrate to oauth_delegate. See PR #40923

A CustomLogger moderation callback now rejects the request. llm_api_check moderation is dispatched through during_call_hook, so a flagged request returns 400 where it previously returned 200. See PR #41685

Bedrock Realtime requires aws-sdk-bedrock-runtime 0.10 or 0.11. The pin moved to aws-sdk-bedrock-runtime[awscrt]>=0.10.0,<0.12.0; 0.7.x is no longer supported and the init error now distinguishes an absent SDK from an unsupported version. Nova Sonic sessions fail until operators install a supported SDK with the awscrt extra. See PR #41542

/v1/rag/ingest resolves the registered store and refuses providers without ingestion. A request that names a registered store by id now uses that store's provider and litellm_params instead of defaulting to OpenAI Files, only per-upload options survive from the caller, and a provider with no ingestion implementation returns 400 naming the supported ones instead of a 200 with status: failed or a 500. See PR #41940

Capability and Fuse v2 routers are limited to one each without a license. Registering a second router of either classifier returns 403 naming the auto_router entitlement. Existing routers are unaffected. See PR #41326

Pricing copied into model_info by pre-v1.102 Admin UI edits is ignored. A stored model_info blob carrying a key field next to prices is treated as a copied /model/info response, so the row follows the cost map again and heals on its next save. Custom pricing declared in litellm_params, or typed without key, is untouched. /model/info now reports model_info.pricing_overrides. See PR #41843

The litellm-proxy entrypoint is deprecated in favour of lite. It still runs, printing one deprecation line on stderr. lite autoroute up and down are renamed to start and stop, with the old names kept as deprecated aliases. See PR #41673, PR #41672

Repeated failed Admin UI sign-in attempts are throttled, and lite login session tokens are re-checked against the live user and team rows, so a removed member gets 403, a deleted user 401, and a demoted admin loses admin routes on the next request after the cache window. See PR #40982, PR #40657

Key Highlights​

  • Config file ownership: one precedence rule across every settings surface, a source and editable flag on both read endpoints, read-only fields in the Admin UI, and a startup warning naming each stored value the file is ignoring
  • Fuse and Capability routing: a capability classifier, Fuse V2 after capability forecasting, per-model Fast mode, maintained Fuse model and harness presets, TypeSafe Jev as a complexity classifier, and heuristic v2 score estimates in routing details
  • Gateway hardening: MCP client allowlisting, live session visibility with admin force-close, delegated OAuth admission, RFC 8693 token exchange for IdP JWTs, and per-issuer JWT key scoping
  • Spend and budget correctness: per-member organization spend, project budgets enforced additively, team-level model_max_budget with key overrides, temporary budget increases, lifetime total_spend on keys, and budgets re-checked on fallback targets
  • 408 new model catalog entries: additions across OpenRouter, AIHubMix, Azure, Together AI, Vertex AI, Deepgram and others, with 147 pricing corrections

New Providers and Endpoints​

Expanded provider endpoint support​

ProviderEndpointWhat you can do
NVIDIA NIM/nvidia_nim/*Reach NIM object detection and OCR /v1/infer through a pass-through route
Amazon Transcribe/transcribe/*Submit transcription jobs through a pass-through route with completion-time pricing
Deepgram/v1/listenStream audio over a WebSocket pass-through with duration-based cost tracking
Azure AI Speech/azure/speech/*Reach Azure AI Speech through a pass-through route
xAI/v1/audio/transcriptionsTranscribe audio with Grok Voice Transcribe
Vertex AI/v1/realtimeStream Chirp speech-to-text over the realtime API
Hosted vLLM/v1/batchesRun batches inside LiteLLM against hosted vLLM deployments
Mistral/v1/files, /v1/batchesSubmit Mistral files and batches, with per-page OCR batch cost tracking
AWS Textract/v1/ocrRun OCR through Textract on the Rust OCR path
Microsoft Foundry/a2a/*Reach Foundry agents with Entra auth and versioned card discovery
TypeSafe AI/typesafe/*Reach the Jev evaluate endpoint with registry-priced spend tracking

New Models / Updated Models​

New Model Support (408 new models)​

Counts represent new catalog identifiers, including aliases and regional variants. Prices below are the values bundled in this release, in USD; runtime pricing-map reloads can update them. Input and output columns show base token rates; long-context, cache, image-token, and other specialized rates depend on the model

ProviderModelContext WindowInput ($/1M tokens)Output ($/1M tokens)Features / special pricing
AIHubMixaihubmix/agnes-2.5-flash512,000$0.03$0.15Chat; Reasoning; Vision
AIHubMixaihubmix/agnes-2.5-pro1,000,000$0.45$0.9Chat; Reasoning; Vision; Prompt caching
AIHubMixaihubmix/cc-glm-5.1200,000$0.06$0.22Chat; Reasoning; Tool calling; Structured output
AIHubMixaihubmix/claude-fable-51,000,000$11$55Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/claude-haiku-4-5200,000$1.1$5.5Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/claude-opus-4-8-think1,000,000$5$25Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/claude-opus-51,000,000$5$25Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/claude-sonnet-51,000,000$2$10Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/coding-glm-5.31,048,576$0.06$0.22Chat; Reasoning; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/coding-kimi-k31,048,576$0.44$1.61333Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/coding-xiaomi-mimo-v2-omni-$0.08$0.4Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/coding-xiaomi-mimo-v2.51,048,576$0.08$0.16Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/coding-xiaomi-mimo-v2.5-pro1,048,576$0.2$0.4Chat; Reasoning; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/command-a-plus-05-2026128,000$2.5$10Chat; Reasoning; Vision; Tool calling; Structured output
AIHubMixaihubmix/deepseek-v4-flash1,000,000$0.142$0.284Chat; Reasoning; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/deepseek-v4-pro1,000,000$1.69$3.38Chat; Reasoning; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/doubao-seed-2-0-code-preview256,000$0.4822$2.411Chat; Reasoning; Vision; Tool calling; Prompt caching; Web search
AIHubMixaihubmix/doubao-seed-2-0-lite-260428256,000$0.09041$0.54246Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/doubao-seed-2-0-mini256,000$0.030136$0.30136Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/doubao-seed-2-0-pro256,000$0.4822$2.411Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/doubao-seed-2-1-turbo256,000$0.46475$2.32375Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/ernie-5.1119,000$0.5634$2.5353Chat; Reasoning; Prompt caching
AIHubMixaihubmix/gemini-3-flash-preview1,048,576$0.5$3Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gemini-3-flash-preview-search1,048,576$0.5$3Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gemini-3.1-pro-preview1,048,576$2$12Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gemini-3.1-pro-preview-customtools1,048,576$2$12Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gemini-3.5-flash-lite1,048,576$0.3$2.499999Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gemini-3.7-flash1,048,576$0.75$3.75Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gemma-4-26b-a4b-it262,144$0.14$0.39998Chat; Reasoning; Vision
AIHubMixaihubmix/gemma-4-31b-it262,144$0.14$0.39998Chat; Reasoning; Vision
AIHubMixaihubmix/glm-5.2-fast-preview1,000,000$2.254$7.889Chat; Reasoning; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/glm-5.31,048,576$1.1268$3.9438Chat; Reasoning; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/glm-5.3-flash1,048,576$0.11268$0.39438Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/glm-5v-turbo200,000$0.7042$3.09848Chat; Reasoning; Vision; Prompt caching
AIHubMixaihubmix/gpt-5.3-codex400,000$1.75$14Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/gpt-5.4-high1,050,000$2.5$15Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gpt-5.4-low1,050,000$2.5$15Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gpt-5.4-mini400,000$0.75$4.5Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gpt-5.4-nano400,000$0.2$1.25Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gpt-5.51,050,000$5$30Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/gpt-5.5-pro1,050,000$30$180Chat; Reasoning; Vision; Tool calling; Structured output; Web search
AIHubMixaihubmix/gpt-5.6-luna1,050,000$0.2$1.2Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/gpt-5.6-sol-disc1,050,000$4$20Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/gpt-5.6-terra1,050,000$2$12Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/gpt-chat-latest400,000$5$30Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/grok-4-20-non-reasoning1,000,000$2$6Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/grok-4-20-reasoning1,000,000$2$6Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/grok-4.6500,000$2$6Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/grok-build-0.1256,000$1$2Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/hy3256,000$0.1562$0.6248Chat; Reasoning; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/hy4-preview1,048,576$0.845$2.535Chat; Reasoning; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/kimi-k2.6262,144$0.95$3.9995Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/kimi-k2.7-code-highspeed262,144$1.9$7.999Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/kimi-k31,048,576$3$15Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/longcat-2.01,000,000$0.7746$3.0984Chat; Reasoning; Tool calling; Prompt caching
AIHubMixaihubmix/mai-thinking-1256,000$2$8Chat; Reasoning; Structured output
AIHubMixaihubmix/mimo-v2-omni256,000$0.44$2.2Chat; Vision; Prompt caching; Web search
AIHubMixaihubmix/mimo-v2-pro1,000,000$1.1$3.3Chat; Prompt caching; Web search
AIHubMixaihubmix/minimax-m2.7204,800$0.2958$1.1832Chat; Reasoning; Tool calling; Prompt caching; Structured output
AIHubMixaihubmix/minimax-m31,000,000$0.288$1.152Chat; Reasoning; Vision; Tool calling; Structured output
AIHubMixaihubmix/muse-spark-1.21,048,576$1.375$4.675Chat; Reasoning; Vision; Tool calling
AIHubMixaihubmix/qwen3-coder-next262,144$0.137$0.548Chat; Tool calling; Structured output
AIHubMixaihubmix/qwen3.5-122b-a10b262,144$0.1126$0.9008Chat; Reasoning; Vision; Tool calling; Structured output; Web search
AIHubMixaihubmix/qwen3.5-397b-a17b262,144$0.1644$0.9864Chat; Reasoning; Vision; Tool calling; Structured output; Web search
AIHubMixaihubmix/qwen3.6-27b262,144$0.422$2.532Chat; Reasoning; Vision; Tool calling; Structured output; Web search
AIHubMixaihubmix/qwen3.6-35b-a3b262,144$0.254$1.524Chat; Reasoning; Vision; Tool calling; Structured output; Web search
AIHubMixaihubmix/qwen3.6-max-preview262,144$1.268$7.608Chat; Reasoning; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/qwen3.7-plus1,000,000$0.282$1.128Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/qwen3.8-2.4t-a95b1,000,000$2$6Chat; Reasoning; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/qwen3.8-flash1,000,000$0.1126$0.380025Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/qwen3.8-max1,000,000$1.69$5.07Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; Web search
AIHubMixaihubmix/step-3.7-flash256,000$0.22$1.32Chat; Reasoning; Vision; Prompt caching
Amazon Bedrockwriter.palmyra-vision-7b4,096$0.15$0.6Chat; Vision
Amazon Transcribetranscribe/StartTranscriptionJob---Transcription; input_cost_per_second: $0.0001; output_cost_per_second: $0
Azure AIazure_ai/FLUX.2-flex32,000--Image generation
Azure AIazure_ai/gpt-5.5-2026-04-241,050,000$5$30Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools
Azure OpenAIazure/chat-latest272,000$5$30Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools
Azure OpenAIazure/eu/codex-mini-$1.65$6.6Chat
Azure OpenAIazure/eu/computer-use-preview-$3.3$13.2Chat
Azure OpenAIazure/eu/gpt-4.1-$2.2$8.8Chat; input_cost_per_token_batches: $1.1e-06; output_cost_per_token_batches: $4.4e-06
Azure OpenAIazure/eu/gpt-4.1-mini-$0.44$1.76Chat; input_cost_per_token_batches: $2.2e-07; output_cost_per_token_batches: $8.8e-07
Azure OpenAIazure/eu/gpt-4.1-nano-$0.11$0.44Chat; input_cost_per_token_batches: $5.5e-08; output_cost_per_token_batches: $2.2e-07
Azure OpenAIazure/eu/gpt-4o-2024-05-13-$5.5$16.5Chat; input_cost_per_token_batches: $2.75e-06; output_cost_per_token_batches: $8.25e-06
Azure OpenAIazure/eu/gpt-5-$1.375$11Chat; input_cost_per_token_batches: $6.875e-07; output_cost_per_token_batches: $5.5e-06
Azure OpenAIazure/eu/gpt-5-codex-$1.375$11Chat
Azure OpenAIazure/eu/gpt-5-mini-$0.275$2.2Chat; input_cost_per_token_batches: $1.375e-07; output_cost_per_token_batches: $1.1e-06
Azure OpenAIazure/eu/gpt-5-nano-$0.055$0.44Chat; input_cost_per_token_batches: $2.75e-08; output_cost_per_token_batches: $2.2e-07
Azure OpenAIazure/eu/gpt-5-pro-$16.5$132Chat; input_cost_per_token_batches: $8.25e-06; output_cost_per_token_batches: $6.6e-05
Azure OpenAIazure/eu/gpt-5.1-codex-max-$1.375$11Chat
Azure OpenAIazure/eu/gpt-5.2-$1.925$15.4Chat; input_cost_per_token_batches: $9.625e-07; output_cost_per_token_batches: $7.7e-06
Azure OpenAIazure/eu/gpt-5.2-chat-$1.925$15.4Chat
Azure OpenAIazure/eu/gpt-5.2-codex-$1.925$15.4Chat
Azure OpenAIazure/eu/gpt-5.2-pro-$23.1$184.8Chat; input_cost_per_token_batches: $1.155e-05; output_cost_per_token_batches: $9.24e-05
Azure OpenAIazure/eu/gpt-5.3-chat-$1.925$15.4Chat
Azure OpenAIazure/eu/gpt-5.3-codex-$1.925$15.4Chat
Azure OpenAIazure/eu/gpt-5.4-mini-$0.825$4.95Chat; input_cost_per_token_batches: $4.125e-07; output_cost_per_token_batches: $2.475e-06
Azure OpenAIazure/eu/gpt-5.4-nano-$0.22$1.375Chat; input_cost_per_token_batches: $1.1e-07; output_cost_per_token_batches: $6.875e-07
Azure OpenAIazure/eu/gpt-5.4-pro-$33$198Chat; input_cost_per_token_batches: $1.65e-05; output_cost_per_token_batches: $9.9e-05
Azure OpenAIazure/eu/gpt-5.5-2026-04-241,050,000$5.5$33Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools; input_cost_per_token_batches: $2.75e-06; output_cost_per_token_batches: $1.65e-05
Azure OpenAIazure/eu/gpt-6-astra-$11$55Chat
Azure OpenAIazure/eu/o1-mini-$1.21$4.84Chat; input_cost_per_token_batches: $6.05e-07; output_cost_per_token_batches: $2.42e-06
Azure OpenAIazure/eu/o1-preview-$16.5$66Chat
Azure OpenAIazure/eu/o3-2025-04-16-$2.2$8.8Chat; input_cost_per_token_batches: $1.1e-06; output_cost_per_token_batches: $4.4e-06
Azure OpenAIazure/eu/o3-deep-research-$11$44Chat
Azure OpenAIazure/eu/o4-mini-2025-04-16-$1.21$4.84Chat; input_cost_per_token_batches: $6.05e-07; output_cost_per_token_batches: $2.42e-06
Azure OpenAIazure/eu/text-embedding-3-large-$0.143-Embeddings
Azure OpenAIazure/eu/text-embedding-3-small-$0.022-Embeddings
Azure OpenAIazure/eu/text-embedding-ada-002-$0.11-Embeddings
Azure OpenAIazure/gpt-5.5-2026-04-241,050,000$5$30Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools; input_cost_per_token_batches: $2.5e-06; output_cost_per_token_batches: $1.5e-05
Azure OpenAIazure/gpt-5.6-luna-2026-07-09922,000$0.2$1.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools
Azure OpenAIazure/gpt-5.6-sol-2026-07-09922,000$4$20Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools
Azure OpenAIazure/gpt-5.6-terra-2026-07-09922,000$2$12Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools
Azure OpenAIazure/gpt-6-astra-2026-09-03922,000$10$50Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Computer use; Web search; System messages; Parallel tools
Azure OpenAIazure/gpt-chat-latest272,000$5$30Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools
Azure OpenAIazure/gpt-image-2.5-flare-$5-Image generation; Vision; PDF input
Azure OpenAIazure/gpt-image-2.5-sunburst-$5-Image generation; Vision; PDF input
Azure OpenAIazure/us/codex-mini-$1.65$6.6Chat
Azure OpenAIazure/us/computer-use-preview-$3.3$13.2Chat
Azure OpenAIazure/us/gpt-4.1-$2.2$8.8Chat; input_cost_per_token_batches: $1.1e-06; output_cost_per_token_batches: $4.4e-06
Azure OpenAIazure/us/gpt-4.1-mini-$0.44$1.76Chat; input_cost_per_token_batches: $2.2e-07; output_cost_per_token_batches: $8.8e-07
Azure OpenAIazure/us/gpt-4.1-nano-$0.11$0.44Chat; input_cost_per_token_batches: $5.5e-08; output_cost_per_token_batches: $2.2e-07
Azure OpenAIazure/us/gpt-4o-2024-05-13-$5.5$16.5Chat; input_cost_per_token_batches: $2.75e-06; output_cost_per_token_batches: $8.25e-06
Azure OpenAIazure/us/gpt-5-$1.375$11Chat; input_cost_per_token_batches: $6.875e-07; output_cost_per_token_batches: $5.5e-06
Azure OpenAIazure/us/gpt-5-codex-$1.375$11Chat
Azure OpenAIazure/us/gpt-5-mini-$0.275$2.2Chat; input_cost_per_token_batches: $1.375e-07; output_cost_per_token_batches: $1.1e-06
Azure OpenAIazure/us/gpt-5-nano-$0.055$0.44Chat; input_cost_per_token_batches: $2.75e-08; output_cost_per_token_batches: $2.2e-07
Azure OpenAIazure/us/gpt-5-pro-$16.5$132Chat; input_cost_per_token_batches: $8.25e-06; output_cost_per_token_batches: $6.6e-05
Azure OpenAIazure/us/gpt-5.1-codex-max-$1.375$11Chat
Azure OpenAIazure/us/gpt-5.2-$1.925$15.4Chat; input_cost_per_token_batches: $9.625e-07; output_cost_per_token_batches: $7.7e-06
Azure OpenAIazure/us/gpt-5.2-chat-$1.925$15.4Chat
Azure OpenAIazure/us/gpt-5.2-codex-$1.925$15.4Chat
Azure OpenAIazure/us/gpt-5.2-pro-$23.1$184.8Chat; input_cost_per_token_batches: $1.155e-05; output_cost_per_token_batches: $9.24e-05
Azure OpenAIazure/us/gpt-5.3-chat-$1.925$15.4Chat
Azure OpenAIazure/us/gpt-5.3-codex-$1.925$15.4Chat
Azure OpenAIazure/us/gpt-5.4-mini-$0.825$4.95Chat; input_cost_per_token_batches: $4.125e-07; output_cost_per_token_batches: $2.475e-06
Azure OpenAIazure/us/gpt-5.4-nano-$0.22$1.375Chat; input_cost_per_token_batches: $1.1e-07; output_cost_per_token_batches: $6.875e-07
Azure OpenAIazure/us/gpt-5.4-pro-$33$198Chat; input_cost_per_token_batches: $1.65e-05; output_cost_per_token_batches: $9.9e-05
Azure OpenAIazure/us/gpt-5.5-2026-04-241,050,000$5.5$33Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools; input_cost_per_token_batches: $2.75e-06; output_cost_per_token_batches: $1.65e-05
Azure OpenAIazure/us/gpt-chat-latest272,000$5.5$33Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; System messages; Parallel tools
Azure OpenAIazure/us/o1-mini-$1.21$4.84Chat; input_cost_per_token_batches: $6.05e-07; output_cost_per_token_batches: $2.42e-06
Azure OpenAIazure/us/o1-preview-$16.5$66Chat
Azure OpenAIazure/us/o3-deep-research-$11$44Chat
Azure OpenAIazure/us/text-embedding-3-large-$0.143-Embeddings
Azure OpenAIazure/us/text-embedding-3-small-$0.022-Embeddings
Azure OpenAIazure/us/text-embedding-ada-002-$0.11-Embeddings
Coherecommand-a-plus-05-2026128,000$0$0Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
DashScopedashscope/qwen3.8-flash991,808$0.15$0.47Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; Web search; Video input
DashScopedashscope/qwen3.8-omni-flash991,808$0.15$0.47Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; Web search; Audio input; Video input
Deepgramdeepgram/streaming/detect_entities---Transcription; input_cost_per_second: $2.833e-05; output_cost_per_second: $0
Deepgramdeepgram/streaming/diarize---Transcription; input_cost_per_second: $3.333e-05; output_cost_per_second: $0
Deepgramdeepgram/streaming/keyterm---Transcription; input_cost_per_second: $2.167e-05; output_cost_per_second: $0
Deepgramdeepgram/streaming/nova-3---Transcription; input_cost_per_second: $8e-05; output_cost_per_second: $0
Deepgramdeepgram/streaming/nova-3-multilingual---Transcription; input_cost_per_second: $9.667e-05; output_cost_per_second: $0
Deepgramdeepgram/streaming/redact---Transcription; input_cost_per_second: $3.333e-05; output_cost_per_second: $0
Fireworks AIfireworks_ai/accounts/fireworks/routers/glm-5p3-fast1,048,576$2.1$6.6Chat; Reasoning; Tool calling; Tool choice; Structured output
Fireworks AIfireworks_ai/glm-5p3-fast1,048,576$2.1$6.6Chat; Reasoning; Tool calling; Tool choice; Structured output
FriendliAIfriendliai/LGAI-EXAONE/K-EXAONE-2.0-750B-A37B262,144$0.6$2.4Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output; System messages; Parallel tools
FriendliAIfriendliai/MiniMaxAI/MiniMax-M2.5196,608$0.3$1.2Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output; System messages; Parallel tools
FriendliAIfriendliai/deepseek-ai/DeepSeek-V3.2163,840$0.5$1.5Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output; System messages; Parallel tools
FriendliAIfriendliai/google/gemma-4-31B-it262,144$0.14$0.4Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output; System messages; Parallel tools
FriendliAIfriendliai/zai-org/GLM-5.1202,752$1.4$4.4Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output; System messages; Parallel tools
FriendliAIfriendliai/zai-org/GLM-5.21,048,576$1.4$4.4Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output; System messages; Parallel tools
Geminigemini-3.8-live131,072$0.75$4.5Realtime; Vision; Tool calling; Web search; Audio input; Audio output; input_cost_per_video_per_second: $3.333333333e-05; input_cost_per_audio_token: $3e-06; output_cost_per_audio_token: $1.2e-05
Geminigemini-3.8-live-extended-thinking131,072$0.75$4.5Realtime; Reasoning; Vision; Tool calling; Web search; Audio input; Audio output; input_cost_per_video_per_second: $3.333333333e-05; input_cost_per_audio_token: $3e-06; output_cost_per_audio_token: $1.2e-05
Geminigemini/gemini-3.8-live131,072$0.75$4.5Realtime; Vision; Tool calling; Web search; Audio input; input_cost_per_audio_token: $3e-06; output_cost_per_audio_token: $1.2e-05
Geminigemini/gemini-3.8-live-extended-thinking131,072$0.75$4.5Realtime; Vision; Tool calling; Web search; Audio input; input_cost_per_audio_token: $3e-06; output_cost_per_audio_token: $1.2e-05
Mistralmistral/zai-glm-51,048,576$1.4$4.4Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
Mistralmistral/zai-glm-5-31,048,576$1.4$4.4Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
Mistralmistral/zai-glm-latest1,048,576$1.4$4.4Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
Nebiusnebius/deepseek-ai/DeepSeek-V4-Pro-0813-$1.32$3.96Chat; Reasoning; Tool calling
Nebiusnebius/zai-org/GLM-5.31,048,576$1.4$4.4Chat; Reasoning; Tool calling
OpenAIgpt-5.5-cyber-$12.5$75Chat; Reasoning
OpenAIgpt-rosalind-research-$5$25Chat
OpenRouteropenrouter/aion-labs/aion-2.0131,072$0.8$1.6Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/aion-labs/aion-3.0131,072$3$6Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/aion-labs/aion-3.0-mini131,072$0.7$1.4Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/aion-labs/aion-rp-llama-3.1-8b32,768$0.8$1.6Chat
OpenRouteropenrouter/amazon/nova-2-lite-v11,000,000$0.3$2.5Chat; Reasoning; Vision; Tool calling; Tool choice; PDF input
OpenRouteropenrouter/amazon/nova-lite-v1300,000$0.06$0.24Chat; Vision; Tool calling
OpenRouteropenrouter/amazon/nova-micro-v1128,000$0.035$0.14Chat; Tool calling
OpenRouteropenrouter/amazon/nova-premier-v11,000,000$2.5$12.5Chat; Vision; Tool calling; Prompt caching
OpenRouteropenrouter/amazon/nova-pro-v1300,000$0.8$3.2Chat; Vision; Tool calling
OpenRouteropenrouter/anthracite-org/magnum-v4-72b32,768$2.5$5Chat; Structured output
OpenRouteropenrouter/anthropic/claude-fable-5.1:batch1,000,000$5$25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-fable-5:batch1,000,000$5$25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-haiku-4.5:batch200,000$0.5$2.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-opus-4.1:batch200,000$7.5$37.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-opus-4.5:batch200,000$2.5$12.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-opus-4.6:batch1,000,000$2.5$12.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-opus-4.7:batch1,000,000$2.5$12.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-opus-4.8:batch1,000,000$2.5$12.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-opus-5:batch1,000,000$2.5$12.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-sonnet-4.5:batch1,000,000$1.5$7.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-sonnet-4.6:batch1,000,000$1.5$7.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/anthropic/claude-sonnet-5:batch1,000,000$1$5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/arcee-ai/trinity-large-thinking262,144$0.25$0.8Chat; Reasoning; Tool calling; Tool choice; Prompt caching
OpenRouteropenrouter/baidu/ernie-4.5-vl-424b-a47b123,000$0.42$1.25Chat; Reasoning; Vision
OpenRouteropenrouter/bytedance-seed/seed-1.6262,144$0.25$2Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/bytedance-seed/seed-1.6-flash262,144$0.075$0.3Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/bytedance-seed/seed-2-1-turbo262,144$0.5$2.5Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/bytedance-seed/seed-2.0-code262,144$0.5$3Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/bytedance-seed/seed-2.0-lite262,144$0.25$2Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/bytedance-seed/seed-2.0-mini262,144$0.1$0.4Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/cognitivecomputations/dolphin-mistral-24b-venice-edition128,000$0.2$0.9Chat; Structured output
OpenRouteropenrouter/cohere/command-a256,000$2.5$10Chat; Structured output
OpenRouteropenrouter/cohere/command-r-08-2024128,000$0.15$0.6Chat; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/cohere/command-r-plus-08-2024128,000$2.5$10Chat; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/cohere/command-r7b-12-2024128,000$0.0375$0.15Chat; Structured output
OpenRouteropenrouter/cohere/north-mini-code:free256,000$0$0Chat; Reasoning; Tool calling; Tool choice
OpenRouteropenrouter/deepseek/deepseek-v4-flash-0731:batch1,048,576$0.11$0.33Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/deepseek/deepseek-v4-flash-0731:free1,048,576$0$0Chat; Reasoning; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/deepseek/deepseek-v4-flash-vision-exp:batch1,048,576$0.11$0.33Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/deepseek/deepseek-v4-pro-0813:batch1,048,576$0.66$1.98Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/dots-studio/dots-3-note-preview:free512,000$0$0Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/google/gemini-2.5-flash-lite:batch1,048,576$0.05$0.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $1.5e-07
OpenRouteropenrouter/google/gemini-2.5-flash:batch1,048,576$0.15$1.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $5e-07
OpenRouteropenrouter/google/gemini-2.5-pro:batch1,048,576$0.625$5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $6.25e-07
OpenRouteropenrouter/google/gemini-3-flash-preview:batch1,048,576$0.25$1.5Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $5e-07
OpenRouteropenrouter/google/gemini-3.1-flash-lite:batch1,048,576$0.125$0.75Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $2.5e-07
OpenRouteropenrouter/google/gemini-3.1-pro-preview:batch1,048,576$1$6Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $1e-06
OpenRouteropenrouter/google/gemini-3.5-flash-lite:batch1,048,576$0.15$1.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $1.5e-07
OpenRouteropenrouter/google/gemini-3.5-flash:batch1,048,576$0.75$4.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $1.5e-06
OpenRouteropenrouter/google/gemini-3.6-flash:batch1,048,576$0.375$1.875Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $3.75e-07
OpenRouteropenrouter/google/gemini-3.7-flash:batch1,048,576$0.375$1.875Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $3.75e-07
OpenRouteropenrouter/google/gemini-3.8-flash:batch1,048,576$0.375$1.875Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; Video input; input_cost_per_audio_token: $3.75e-07
OpenRouteropenrouter/ibm-granite/granite-4.0-h-micro131,000$0.017$0.112Chat; Structured output
OpenRouteropenrouter/ibm-granite/granite-4.2-8b131,072$0.06$0.25Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/inception/mercury-2128,000$0.25$0.75Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/inception/mercury-2.5260,000$0.04$0.15Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/inclusionai/ling-3.0-flash262,144$0.021$0.063Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/inclusionai/ling-3.0-flash-fin262,144$0.06$0.18Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/inclusionai/ling-3.0-flash-fin:free262,144$0$0Chat; Reasoning; Tool calling; Tool choice
OpenRouteropenrouter/inclusionai/ling-3.0-flash-sante:free262,144$0$0Chat; Reasoning; Tool calling; Tool choice
OpenRouteropenrouter/inclusionai/ling-3.0-flash-vl131,072$0.06$0.18Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/inclusionai/ling-3.0-flash-vl:free262,144$0$0Chat; Reasoning; Vision; Tool calling; Tool choice
OpenRouteropenrouter/inference-net/schematron-v2-small128,000$0.05$0.23Chat; Prompt caching; Structured output
OpenRouteropenrouter/inference-net/schematron-v2-turbo128,000$0.03$0.15Chat; Prompt caching; Structured output
OpenRouteropenrouter/kwaipilot/kat-coder-pro-v2262,144$0.3$1.2Chat; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/kwaipilot/kat-coder-pro-v2.5262,144$0.74$2.96Chat; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/liquid/lfm-2.5-2.6b:free65,536$0$0Chat; Reasoning; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/meituan/longcat-2.01,048,756$0.3$1.2Chat; Reasoning; Tool calling; Tool choice; Prompt caching
OpenRouteropenrouter/meta/muse-glimmer-30b131,072$0.35$1.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/meta/muse-glimmer-30b:batch131,072$0.175$0.75Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/meta/muse-spark-1.11,048,576$1.25$4.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input
OpenRouteropenrouter/meta/muse-spark-1.21,048,576$1.25$4.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input
OpenRouteropenrouter/meta/muse-spark-1.2-contributor1,048,576$0.1$0.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input
OpenRouteropenrouter/meta/muse-spark-1.31,048,576$1.25$4.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input
OpenRouteropenrouter/meta/muse-spark-1.3-contributor1,048,576$0.1$0.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input
OpenRouteropenrouter/microsoft/phi-416,384$0.07$0.14Chat; Structured output
OpenRouteropenrouter/microsoft/wizardlm-2-8x22b65,535$0.62$0.62Chat; Structured output
OpenRouteropenrouter/minimax/minimax-m3:batch524,288$0.3$1.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/mistralai/codestral-2508:batch256,000$0.15$0.45Chat; Tool calling; Tool choice; Prompt caching; Structured output; PDF input
OpenRouteropenrouter/mistralai/ministral-8b-2512:batch262,144$0.075$0.075Chat; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/mistralai/mistral-large-2512:batch262,144$0.25$0.75Chat; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input
OpenRouteropenrouter/mistralai/mistral-medium-3-5:batch262,144$0.75$3.75Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output; PDF input
OpenRouteropenrouter/mistralai/mistral-medium-3.1:batch131,072$0.2$1Chat; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input
OpenRouteropenrouter/mistralai/mistral-small-2603:batch262,144$0.075$0.3Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/moonshotai/kimi-k3:batch1,048,576$3$15Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/morph/morph-v3-fast81,920$0.8$1.2Chat
OpenRouteropenrouter/morph/morph-v3-large262,144$0.9$1.9Chat; Structured output
OpenRouteropenrouter/nex-agi/nex-n2.5-mini:free262,144$0$0Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/nex-agi/nex-n2.5-pro:free262,144$0$0Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/nousresearch/hermes-3-llama-3.1-405b131,072$1$1Chat; Structured output
OpenRouteropenrouter/nousresearch/hermes-3-llama-3.1-70b131,072$0.7$0.7Chat; Structured output
OpenRouteropenrouter/nousresearch/hermes-4-405b131,072$1$3Chat; Reasoning; Structured output
OpenRouteropenrouter/openai/gpt-3.5-turbo-06134,095$1$2Chat; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/openai/gpt-3.5-turbo:batch16,385$0.25$0.75Chat; Tool calling; Tool choice; Structured output; Web search
OpenRouteropenrouter/openai/gpt-4-turbo:batch128,000$5$15Chat; Vision; Tool calling; Tool choice; Structured output; Web search
OpenRouteropenrouter/openai/gpt-4.1-mini:batch1,047,576$0.2$0.8Chat; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-4.1-nano:batch1,047,576$0.05$0.2Chat; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-4.1:batch1,047,576$1$4Chat; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-4o-mini:batch128,000$0.075$0.3Chat; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-4o:batch128,000$1.25$5Chat; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5-image400,000$10$10Chat; Reasoning; Vision; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5-image-mini400,000$2.5$2Chat; Reasoning; Vision; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5-mini:batch400,000$0.125$1Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5-nano:batch400,000$0.025$0.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5-pro:batch400,000$7.5$60Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.1:batch400,000$0.625$5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.2-pro:batch400,000$10.5$84Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.2:batch400,000$0.875$7Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.4-image-2272,000$8$15Chat; Reasoning; Vision; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.4-mini:batch400,000$0.375$2.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.4-nano:batch400,000$0.1$0.625Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.4-pro:batch1,050,000$15$90Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.4:batch1,050,000$1.25$7.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.5-pro:batch1,050,000$15$90Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.5:batch1,050,000$2.5$15Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.6-luna-pro:batch1,050,000$0.1$0.6Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.6-luna:batch1,050,000$0.1$0.6Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.6-sol-pro:batch1,050,000$1$5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.6-sol:batch1,050,000$1$5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.6-terra-pro:batch1,050,000$1$6Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5.6-terra:batch1,050,000$1$6Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-5:batch400,000$0.625$5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-6-astra-pro:batch1,050,000$5$25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-6-astra:batch1,050,000$5$25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/gpt-oss-120b:batch131,072$0.15$0.6Chat; Reasoning; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/openai/o3-mini:batch200,000$0.55$2.2Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/o3:batch200,000$1$4Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/openai/o4-mini:batch200,000$0.55$2.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/perceptron/perceptron-mk132,768$0.15$1.5Chat; Reasoning; Vision; Structured output
OpenRouteropenrouter/perplexity/sonar127,072$1$1Chat; Vision; Web search
OpenRouteropenrouter/perplexity/sonar-deep-research128,000$2$8Chat; Reasoning; Web search
OpenRouteropenrouter/perplexity/sonar-pro200,000$3$15Chat; Vision; Web search
OpenRouteropenrouter/perplexity/sonar-pro-search200,000$3$15Chat; Reasoning; Vision; Structured output; Web search
OpenRouteropenrouter/perplexity/sonar-reasoning-pro128,000$2$8Chat; Reasoning; Vision; Web search
OpenRouteropenrouter/prism-ml/ternary-bonsai-2-27b262,144$0.075$0.5Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/qwen/qwen3.5-9b:batch262,144$0.17$0.25Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/qwen/qwen3.8-2.4t-a95b:batch1,010,000$2$6Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/qwen/qwen3.8-27b:free262,144$0$0Chat; Reasoning; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/rekaai/reka-edge16,384$0.1$0.1Chat; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/rekaai/reka-flash-365,536$0.1$0.2Chat; Reasoning; Structured output
OpenRouteropenrouter/relace/relace-apply-3256,000$0.85$1.25Chat
OpenRouteropenrouter/relace/relace-search256,000$1$3Chat; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/sakana/fugu-max1,000,000$2$6Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/sakana/fugu-ultra1,000,000$5$30Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; Web search
OpenRouteropenrouter/sakana/fugu-ultra-v21,000,000$5$30Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/sakana/sakana-namazu262,144$0.95$4Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/sao10k/l3-lunaris-8b8,192$0.04$0.05Chat; Structured output
OpenRouteropenrouter/sao10k/l3.1-euryale-70b131,072$0.85$0.85Chat; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/sao10k/l3.3-euryale-70b131,072$0.65$0.75Chat; Structured output
OpenRouteropenrouter/stealth/union-alpha262,144$0$0Chat; Vision; Tool calling; Tool choice; Structured output
OpenRouteropenrouter/stepfun/step-3.5-flash262,144$0.1$0.3Chat; Reasoning; Tool calling; Tool choice
OpenRouteropenrouter/stepfun/step-3.7-flash262,144$0.2$1.15Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/tencent/hunyuan-a13b-instruct131,072$0.14$0.57Chat; Reasoning; Structured output
OpenRouteropenrouter/tencent/hy-mt2-1.8b8,192$0.044$0.177Chat
OpenRouteropenrouter/tencent/hy-mt2-30b-a3b8,192$0.074$0.295Chat; Structured output
OpenRouteropenrouter/tencent/hy-mt2-7b8,192$0.074$0.295Chat; Structured output
OpenRouteropenrouter/tencent/hy3262,144$0.132$0.528Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/tencent/hy3-preview262,144$0.18$0.6Chat; Reasoning; Tool calling; Tool choice; Prompt caching
OpenRouteropenrouter/tencent/hy4-preview1,048,576$0.834$2.501Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/thedrummer/cydonia-24b-v4.1131,072$0.3$0.5Chat; Prompt caching; Structured output
OpenRouteropenrouter/thedrummer/skyfall-36b-v232,768$0.55$0.8Chat; Prompt caching; Structured output
OpenRouteropenrouter/thedrummer/unslopnemo-12b1,024,000$0.4$0.4Chat; Structured output
OpenRouteropenrouter/thinkingmachines/inkling1,048,576$1$4.05Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Audio input
OpenRouteropenrouter/thinkingmachines/inkling-small1,048,576$0.45$1.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Audio input
OpenRouteropenrouter/thinkingmachines/inkling-small:free1,048,576$0$0Chat; Reasoning; Vision; Tool calling; Audio input
OpenRouteropenrouter/thinkingmachines/inkling:batch524,288$1$4.05Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Audio input
OpenRouteropenrouter/thinkingmachines/inkling:free1,048,576$0$0Chat; Reasoning; Vision; Tool calling; Audio input
OpenRouteropenrouter/unbiased/pareto262,144$2.5$7.5Chat; Vision; Tool calling; Tool choice; Prompt caching
OpenRouteropenrouter/upstage/solar-pro-3131,072$0.15$0.6Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/upstage/solar-pro4524,288$0.09$0.36Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/writer/palmyra-x51,040,000$0.6$6Chat
OpenRouteropenrouter/x-ai/grok-4.3:batch1,000,000$1$2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/z-ai/glm-5.2:batch1,048,576$0.7$2.2Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/z-ai/glm-5.3-flash:batch1,048,576$0.075$0.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/z-ai/glm-5.3-flashx1,048,576$0.37$1.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/z-ai/glm-5.3:batch1,048,576$0.7$2.2Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/~anthropic/claude-fable-latest1,000,000$10$50Chat; Reasoning; Vision; Tool calling; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~anthropic/claude-haiku-latest200,000$1$5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~anthropic/claude-opus-latest1,000,000$5$25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~anthropic/claude-sonnet-latest1,000,000$2$10Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~deepseek/deepseek-flash-latest1,048,576$0.13$0.52Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/~deepseek/deepseek-pro-latest1,048,576$0.57816$1.73448Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/~deepseek/deepseek-v4-flash-latest1,310,720$0.04$0.08Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/~google/gemini-flash-latest1,048,576$0.75$3.75Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; input_cost_per_audio_token: $7.5e-07
OpenRouteropenrouter/~google/gemini-pro-latest1,048,576$2$12Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search; Audio input; input_cost_per_audio_token: $2e-06
OpenRouteropenrouter/~moonshotai/kimi-latest1,048,576$1.7$8.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/~openai/gpt-astra-latest1,050,000$10$50Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~openai/gpt-luna-latest1,050,000$0.2$1.2Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~openai/gpt-mini-latest400,000$0.75$4.5Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~openai/gpt-sol-latest1,050,000$2$10Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~openai/gpt-terra-latest1,050,000$2$12Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~x-ai/grok-latest500,000$2$6Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; PDF input; Web search
OpenRouteropenrouter/~z-ai/glm-flash-latest1,310,720$0.075$0.25Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output
OpenRouteropenrouter/~z-ai/glm-latest1,310,720$0.8442$2.6532Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
Qwenqwen_ai_platform/qwen3.8-flash991,808$0.15$0.47Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; Web search; Video input
Qwenqwen_ai_platform/qwen3.8-omni-flash991,808$0.15$0.47Chat; Reasoning; Vision; Tool calling; Tool choice; Prompt caching; Structured output; Web search; Audio input; Video input
Together AItogether_ai/NousResearch/Nous-Hermes-2-Mixtral-8x7B-DPO-$0.6$0.6Chat
Together AItogether_ai/Qwen/Qwen2-1.5B-Instruct-$0.02$0.02Chat
Together AItogether_ai/Qwen/Qwen2-72B-Instruct-$0.9$0.9Chat
Together AItogether_ai/Qwen/Qwen2-VL-72B-Instruct-$1.2$1.2Chat
Together AItogether_ai/Qwen/Qwen2.5-14B-Instruct-$0.8$0.8Chat
Together AItogether_ai/Qwen/Qwen2.5-72B-Instruct-$1.2$1.2Chat
Together AItogether_ai/Qwen/Qwen2.5-Coder-32B-Instruct-$0.8$0.8Chat
Together AItogether_ai/Qwen/Qwen2.5-VL-72B-Instruct-$1.95$8Chat
Together AItogether_ai/arcee-ai/trinity-mini-$0.045$0.15Chat
Together AItogether_ai/deepseek-ai/DeepSeek-R1-Distill-Llama-70B-$2$2Chat
Together AItogether_ai/deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B-$0.18$0.18Chat
Together AItogether_ai/deepseek-ai/DeepSeek-R1-Distill-Qwen-14B-$1.6$1.6Chat
Together AItogether_ai/deepseek-ai/DeepSeek-V4.1-Flash1,048,576$0.3$1.2Chat; Tool calling; Tool choice; Prompt caching; Structured output
Together AItogether_ai/deepseek-ai/deepseek-coder-33b-instruct-$0.8$0.8Chat
Together AItogether_ai/google/gemma-2-27b-it-$0.8$0.8Chat
Together AItogether_ai/meta-llama/Llama-3-8b-chat-hf-$0.2$0.2Chat
Together AItogether_ai/meta-llama/Llama-3.1-405B-Instruct-$3.5$3.5Chat
Together AItogether_ai/meta-llama/Llama-3.2-1B-Instruct-$0.06$0.06Chat
Together AItogether_ai/meta-llama/Llama-3.2-3B-Instruct-$0.06$0.06Chat
Together AItogether_ai/meta-llama/Meta-Llama-3-70B-Instruct-Turbo-$0.88$0.88Chat
Together AItogether_ai/meta-llama/Meta-Llama-3-8B-Instruct-$0.2$0.2Chat
Together AItogether_ai/nvidia/Llama-3.1-Nemotron-70B-Instruct-HF-$0.88$0.88Chat
TypeSafetypesafe/jev-1.13.0-$0.042$0-
TypeSafetypesafe/jev-latest-$0.042$0-
TypeSafetypesafe/jev-preview-$0.042$0-
Vertex AIvertex_ai/gemini-2.5-flash-native-audio-$0.5$2Realtime; input_cost_per_audio_token: $3e-06; output_cost_per_audio_token: $1.2e-05
Vertex AIvertex_ai/gemini-2.5-flash-preview-tts-$0.5$10Speech; output_cost_per_audio_token: $1e-05; input_cost_per_token_batches: $2.5e-07
Vertex AIvertex_ai/gemini-3.1-flash-live-preview-$0.75$4.5Realtime; input_cost_per_second: $8.333333333e-05; input_cost_per_audio_token: $3e-06; output_cost_per_audio_token: $1.2e-05
Vertex AIvertex_ai/gemini-3.1-flash-tts-preview-$1$20Speech; output_cost_per_audio_token: $2e-05; input_cost_per_token_batches: $5e-07
Vertex AIvertex_ai/gemini-3.5-transcribe--$12Transcription; input_cost_per_second: $5e-05; input_cost_per_audio_token: $2e-06
Vertex AIvertex_ai/gemini-3.5-transcribe-live--$21Transcription; input_cost_per_second: $8.333333333e-05; input_cost_per_audio_token: $3.5e-06
Vertex AIvertex_ai/gemini-omni-1.1-flash-$1.5$9Chat
Vertex AIvertex_ai/gemini-robotics-er-2-$1$5Chat; input_cost_per_token_batches: $5e-07; output_cost_per_token_batches: $2.5e-06
Vertex AIvertex_ai/gemma-4-26b-a4b-it-$0.15$0.6Chat
Volcenginevolcengine/doubao-seed-2-1-pro-260628256,000$0.8625$4.3125Chat; Reasoning; Vision; Tool calling; Prompt caching
Volcenginevolcengine/doubao-seed-2-1-turbo-260628256,000$0.43125$2.15625Chat; Reasoning; Vision; Tool calling; Prompt caching
Weights & Biaseswandb/zai-org/GLM-5.3-Flash1,049,000$0.15$0.5Chat; Reasoning; Tool calling; Tool choice; Prompt caching; Structured output
xAIxai/grok-voice-transcribe-1.0---Transcription; input_cost_per_second: $2.778e-05; output_cost_per_second: $0
xAIxai/grok-voice-transcribe-2.0---Transcription; input_cost_per_second: $2.778e-05; output_cost_per_second: $0

Updated pricing (147 models)​

Provider / modelChanged token prices (USD per 1M tokens)
amazon.nova-lite-v1:0Cache read: not set to $0.015
amazon.nova-micro-v1:0Cache read: not set to $0.00875
amazon.nova-pro-v1:0Cache read: not set to $0.2
apac.amazon.nova-lite-v1:0Cache read: not set to $0.01575
apac.amazon.nova-micro-v1:0Cache read: not set to $0.00925
apac.amazon.nova-pro-v1:0Cache read: not set to $0.21
azure/eu/gpt-4o-2024-11-20Cache read: not set to $1.375
azure/eu/gpt-5.1Input: $1.38 to $1.375; Cache read: $0.14 to $0.1375
azure/eu/gpt-5.1-chatInput: $1.38 to $1.375; Cache read: $0.14 to $0.1375
azure/eu/gpt-5.1-codexInput: $1.38 to $1.375; Cache read: $0.14 to $0.1375
azure/eu/gpt-5.1-codex-miniCache read: $0.028 to $0.0275
azure/eu/gpt-5.4Cache read: $0.28 to $0.275
azure/eu/gpt-5.4-2026-03-05Cache read: $0.28 to $0.275
azure/eu/gpt-5.6-solInput: $5.5 to $4.4; Output: $33 to $22; Cache read: $0.55 to $0.44; Cache write: $6.875 to $5.5
azure/gpt-4o-2024-11-20Input: $2.75 to $2.5; Output: $11 to $10
azure/gpt-4o-mini-2024-07-18Input: $0.165 to $0.15; Output: $0.66 to $0.6
azure/gpt-5.6-solInput: $5 to $4; Output: $30 to $20; Cache read: $0.5 to $0.4; Cache write: $6.25 to $5
azure/gpt-realtime-1.5-2026-02-23Cache read: $4 to $0.4
azure/gpt-realtime-2025-08-28Cache read: $4 to $0.4
azure/o1-miniInput: $1.21 to $1.1; Output: $4.84 to $4.4; Cache read: $0.605 to $0.55
azure/us/gpt-4.1-nano-2025-04-14Cache read: $0.025 to $0.028
azure/us/gpt-4o-2024-11-20Cache read: not set to $1.375
azure/us/gpt-5.1Input: $1.38 to $1.375; Cache read: $0.14 to $0.1375
azure/us/gpt-5.1-chatInput: $1.38 to $1.375; Cache read: $0.14 to $0.1375
azure/us/gpt-5.1-codexInput: $1.38 to $1.375; Cache read: $0.14 to $0.1375
azure/us/gpt-5.1-codex-miniCache read: $0.028 to $0.0275
azure/us/gpt-5.4Cache read: $0.28 to $0.275
azure/us/gpt-5.4-2026-03-05Cache read: $0.28 to $0.275
azure/us/gpt-5.6-solInput: $5.5 to $4.4; Output: $33 to $22; Cache read: $0.55 to $0.44; Cache write: $6.875 to $5.5
azure/us/o4-mini-2025-04-16Cache read: $0.31 to $0.303
azure_ai/FW-GLM-5.2-FastInput: $2.1 to $2.31; Output: $6.6 to $7.26; Cache read: $0.21 to $0.231
azure_ai/FW-InklingInput: $1 to $1.1; Output: $4.05 to $4.46; Cache read: $0.17 to $0.19
azure_ai/FW-Kimi-K3Input: $3.3 to $3; Output: $16.5 to $15; Cache read: $0.33 to $0.3
azure_ai/FW-Nemotron-3-Ultra-NVFP4Input: $0.6 to $0.66; Output: $2.4 to $2.64; Cache read: $0.119 to $0.13
azure_ai/Llama-4-Maverick-17B-128E-Instruct-FP8Input: $1.41 to $0.25; Output: $0.35 to $1
azure_ai/Phi-4-mini-reasoningInput: $0.08 to $0.075; Output: $0.32 to $0.3
bedrock/us-gov-east-1/amazon.nova-pro-v1:0Cache read: not set to $0.24
bedrock/us-gov-west-1/amazon.nova-lite-v1:0Cache read: not set to $0.018
bedrock/us-gov-west-1/amazon.nova-micro-v1:0Cache read: not set to $0.0105
bedrock/us-gov-west-1/amazon.nova-pro-v1:0Cache read: not set to $0.24
bedrock_mantle/openai.gpt-5.6-solInput: $5.5 to $4.4; Output: $33 to $22; Cache read: $0.55 to $0.44; Cache write: $6.875 to $5.5
chatgpt-image-latestOutput: not set to $10
deep-research-pro-preview-12-2025Cache read: not set to $0.2
eu.amazon.nova-lite-v1:0Cache read: not set to $0.0195
eu.amazon.nova-micro-v1:0Cache read: not set to $0.0115
eu.amazon.nova-pro-v1:0Cache read: not set to $0.2625
fireworks_ai/accounts/fireworks/models/deepseek-v4-proInput: $1.74 to $1.2; Output: $3.48 to $1.2; Cache read: $0.145 to $0.6
fireworks_ai/accounts/fireworks/models/qwen3-reranker-8bInput: $0 to $0.2
fireworks_ai/deepseek-v4-proInput: $1.74 to $1.2; Output: $3.48 to $1.2; Cache read: $0.145 to $0.6
gemini-2.0-flashInput: $0.1 to $0.15; Output: $0.4 to $0.6
gemini-3-pro-imageCache read: not set to $0.2
gemini-3.1-flash-imageCache read: not set to $0.05
gemini-flash-latestInput: $0.3 to $0.75; Output: $2.5 to $3.75; Cache read: $0.03 to $0.075
gemini-flash-lite-latestInput: $0.1 to $0.3; Output: $0.4 to $2.5; Cache read: $0.01 to $0.03
gemini-pro-latestInput: $1.25 to $2; Output: $10 to $12; Cache read: $0.125 to $0.2
gemini/gemini-flash-latestInput: $0.3 to $0.75; Output: $2.5 to $3.75; Cache read: $0.03 to $0.075
gemini/gemini-flash-lite-latestInput: $0.1 to $0.3; Output: $0.4 to $2.5; Cache read: $0.01 to $0.03
gemini/gemini-pro-latestInput: $1.25 to $2; Output: $10 to $12; Cache read: $0.125 to $0.2
gemini/gemini-robotics-er-2-previewInput: $2 to $1; Output: $10 to $5; Cache read: $0.2 to $0.1
gpt-4o-mini-ttsInput: $2.5 to $0.6
gpt-4o-mini-tts-2025-03-20Input: $2.5 to $0.6
gpt-4o-mini-tts-2025-12-15Input: $2.5 to $0.6
gpt-realtime-miniCache read: not set to $0.06
inception/mercury-2.5Cache read: not set to $0.02
mistral/codestral-mamba-latestCache read: not set to $0.025
mistral/devstral-latestCache read: not set to $0.04
mistral/devstral-medium-latestCache read: not set to $0.04
mistral/devstral-small-latestCache read: not set to $0.01
mistral/mistral-code-agent-latestCache read: not set to $0.04
mistral/mistral-medium-3Cache read: not set to $0.15
mistral/mistral-smallCache read: not set to $0.01
mistral/mistral-tinyCache read: not set to $0.025
mistral/open-mistral-nemoCache read: not set to $0.03
mistral/pixtral-large-latestCache read: not set to $0.2
mistral/voxtral-small-2507Cache read: not set to $0.01
mistral/voxtral-small-latestCache read: not set to $0.01
openrouter/anthropic/claude-3-haikuCache read: not set to $0.03; Cache write: not set to $0.3
openrouter/bytedance/ui-tars-1.5-7bCache read: not set to $0.1
openrouter/deepseek/deepseek-r1-0528Cache read: not set to $0.35
openrouter/deepseek/deepseek-v3.2Cache read: not set to $0.1345
openrouter/deepseek/deepseek-v4-flashInput: $0.0854 to $0.03724; Output: $0.1708 to $0.07448; Cache read: $0.01708 to $0.007448
openrouter/deepseek/deepseek-v4-flash-0731Input: $0.065 to $0.04; Output: $0.18 to $0.08
openrouter/deepseek/deepseek-v4-flash-vision-expInput: $0.22 to $0.2156; Output: $0.66 to $0.6468; Cache read: $0.007 to $0.00686
openrouter/deepseek/deepseek-v4-proInput: $0.859908 to $0.422298; Output: $1.719816 to $0.844596; Cache read: $0.071659 to $0.0351915
openrouter/deepseek/deepseek-v4-pro-0813Input: $0.57948 to $0.57816; Output: $1.73844 to $1.73448; Cache read: $0.019316 to $0.018396
openrouter/deepseek/deepseek-v4.1-flashInput: $0.15 to $0.3; Output: $0.6 to $1.2; Cache read: $0.003 to $0.006
openrouter/google/gemini-2.5-flashCache write: not set to $0.0833333333
openrouter/google/gemini-2.5-flash-liteCache write: not set to $0.0833333333
openrouter/google/gemini-2.5-proCache write: not set to $0.375
openrouter/google/gemini-3-flash-previewCache write: not set to $0.0833333333
openrouter/google/gemini-3.1-flash-liteCache write: not set to $0.0833333333
openrouter/google/gemini-3.1-flash-lite-previewCache write: not set to $0.0833333333
openrouter/google/gemini-3.1-pro-previewCache write: not set to $0.375
openrouter/google/gemini-3.5-flashCache write: not set to $0.0833333333
openrouter/google/gemini-3.5-flash-liteCache write: not set to $0.0833333333
openrouter/google/gemini-3.6-flashCache write: not set to $0.0416666667
openrouter/google/gemini-3.7-flashCache write: not set to $0.0416666667
openrouter/google/gemini-3.8-flashCache write: not set to $0.0416666667
openrouter/google/gemma-4-26b-a4b-itInput: $0.042 to $0.09; Output: $0.22 to $0.3; Cache read: not set to $0.05
openrouter/gryphe/mythomax-l2-13bInput: $0.06 to $0.08; Output: $0.06 to $0.11
openrouter/meta-llama/llama-4-maverickInput: $0.2 to $0.1875; Output: $0.696 to $0.6525
openrouter/minimax/minimax-m1Input: $0.55 to $0.4
openrouter/mistralai/devstral-2512Cache read: not set to $0.04
openrouter/mistralai/ministral-14b-2512Cache read: not set to $0.02
openrouter/mistralai/ministral-3b-2512Cache read: not set to $0.01
openrouter/mistralai/ministral-8b-2512Cache read: not set to $0.015
openrouter/mistralai/mistral-largeCache read: not set to $0.2
openrouter/mistralai/mistral-large-2512Input: $0.5 to $0.55; Output: $1.5 to $1.65; Cache read: not set to $0.055
openrouter/mistralai/mistral-small-3.2-24b-instructInput: $0.075 to $0.09375; Output: $0.2 to $0.25
openrouter/mistralai/mixtral-8x22b-instructCache read: not set to $0.2
openrouter/moonshotai/kimi-k2.7-codeInput: $0.71 to $0.7062; Output: $3.5 to $3.21; Cache read: $0.15 to $0.18
openrouter/moonshotai/kimi-k3Input: $2.1 to $1.7; Output: $10.53 to $8.5; Cache read: $0.235 to $0.17
openrouter/nvidia/nemotron-3-nano-30b-a3bInput: $0.05 to $0.06; Output: $0.2 to $0.24
openrouter/nvidia/nemotron-3-super-120b-a12bInput: $0.085 to $0.08; Output: $0.4 to $0.45
openrouter/nvidia/nemotron-3-ultra-550b-a55bInput: $0.625 to $0.6; Output: $3.125 to $2.4; Cache read: $0.1875 to $0.12
openrouter/nvidia/nemotron-3.5-lightningInput: $0.08 to $0.07; Cache read: not set to $0.04
openrouter/openai/gpt-5.6-lunaCache write: not set to $0.25
openrouter/openai/gpt-5.6-terraCache write: not set to $2.5
openrouter/openai/gpt-oss-120bInput: $0.037 to $0.15; Output: $0.17 to $0.6; Cache read: not set to $0.075
openrouter/openai/gpt-oss-20bCache read: not set to $0.03
openrouter/qwen/qwen-plus-2025-07-28Cache read: not set to $0.052; Cache write: not set to $0.325
openrouter/qwen/qwen3-14bInput: $0.2275 to $0.12; Output: $0.91 to $0.24
openrouter/qwen/qwen3-235b-a22b-2507Input: $0.22 to $0.0875; Output: $0.88 to $0.35; Cache read: not set to $0.0175
openrouter/qwen/qwen3-30b-a3b-instruct-2507Input: $0.09 to $0.04815; Output: $0.3 to $0.19305
openrouter/qwen/qwen3-coderCache read: not set to $0.1
openrouter/qwen/qwen3-coder-plusCache read: not set to $0.13; Cache write: not set to $0.8125
openrouter/qwen/qwen3-vl-30b-a3b-instructInput: $0.15 to $0.13; Output: $0.6 to $0.52
openrouter/qwen/qwen3.5-397b-a17bCache read: not set to $0.225
openrouter/qwen/qwen3.6-plusCache write: not set to $0.40625
openrouter/undi95/remm-slerp-l2-13bInput: $0.45 to $0.35
openrouter/z-ai/glm-4.7-flashInput: $0.06 to $0.0605
openrouter/z-ai/glm-5Cache read: not set to $0.12
openrouter/z-ai/glm-5.2Input: $0.6 to $0.5544; Output: $2 to $1.7424; Cache read: $0.15 to $0.10296
openrouter/z-ai/glm-5.3Input: $1.4 to $0.896; Output: $4.4 to $2.816; Cache read: $0.14 to $0.1664
openrouter/z-ai/glm-5.3-flashInput: $0.15 to $0.09; Output: $0.5 to $0.3; Cache read: $0.03 to $0.018
replicate/google/gemini-2.5-flashInput: $2.5 to $0.3
us.amazon.nova-lite-v1:0Cache read: not set to $0.015
us.amazon.nova-micro-v1:0Cache read: not set to $0.00875
us.amazon.nova-premier-v1:0Cache read: not set to $0.625
us.amazon.nova-pro-v1:0Cache read: not set to $0.2
vercel_ai_gateway/google/gemini-2.5-flashCache read: not set to $0.03
vercel_ai_gateway/google/gemini-2.5-proInput: $2.5 to $1.25; Cache read: not set to $0.125
vertex_ai/deep-research-pro-preview-12-2025Cache read: not set to $0.2
vertex_ai/gemini-3-pro-imageCache read: not set to $0.2
vertex_ai/gemini-3.1-flash-imageCache read: not set to $0.05
wandb/nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55BInput: $0.75 to $0.5; Output: $2.75 to $2.15; Cache read: $0.15 to $0.1
wandb/nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3BInput: $0.1 to $0.07; Output: $0.25 to $0.2; Cache read: $0.05 to $0.04

The registry also updates capability flags, context/output limits, non-token rates, and deprecation dates. Five entries were removed: friendliai/meta-llama-3.1-70b-instruct, friendliai/meta-llama-3.1-8b-instruct, github_copilot/gemini-2.5-pro, github_copilot/gemini-3-pro-preview and gmi/google/gemini-3-pro-preview

Amazon Bedrock​

  • Send aws_session_tags on every STS call via one typed auth struct - PR #40500
  • Sanitize client tool_call ids to Bedrock toolUseId constraints - PR #40872
  • Carry s3_endpoint_url and s3_region_name into file content downloads - PR #41138
  • Grant rerank, retrieve, agent, and agentcore actions in the web identity session policy - PR #41168
  • Make prompt caching work on the Nova InvokeModel route - PR #41343
  • Never emit Converse cachePoint for OpenAI-family models - PR #41419
  • Forward userContext in Knowledge Base Retrieve requests - PR #41475
  • Neutralize orphaned tool blocks instead of raising or injecting a dummy tool - PR #41513
  • Support aws-sdk-bedrock-runtime 0.10/0.11 in Bedrock Realtime - PR #41542
  • Gate Invoke tool search on the model map for Opus 4.8 and gen 5 Claude - PR #41702
  • Bill Bedrock Titan embedding batch lines from inputTextTokenCount - PR #41767
  • Clamp maxTokens to the 16-token minimum for OpenAI GPT and xAI Grok models on Converse - PR #41870
  • Honor eager_input_streaming on Bedrock and Anthropic Claude tools - PR #41871
  • Sign batch retrieve and cancel with deployment credentials when AWS_BEARER_TOKEN_BEDROCK is set - PR #41904

Anthropic​

  • Add the per-turn-control beta when a message carries output_config - PR #41189
  • Register thinking-binding-controls-2026-08-01 in beta headers config - PR #41203
  • Tolerate message_delta events without usage when streaming - PR #41336
  • Carry the served model from message_start onto stream chunks - PR #41446
  • Estimate interrupted Anthropic stream usage from reasoning_content - PR #41503
  • Keep cache_control for Gemini targets on /v1/messages and normalize Anthropic ttl units - PR #41938

Azure​

  • Support FLUX.2 flex images - PR #39424
  • Send the resolved Entra ID token on image generation requests - PR #40147
  • Strip litellm format field from file and image content parts - PR #41275
  • Keep api-version query after vector store search path - PR #41384
  • Classify Azure Speech short audio behind a prefixed api base - PR #41882
  • Drop tool_choice when the request has no tools - PR #42031

Bedrock Mantle​

  • Price GovCloud regions from the regional cost row and accept region-prefixed model names - PR #39846
  • Accept and forward verbosity on gpt-5.x chat completions - PR #41509

DashScope​

  • Forward reasoning_effort to the provider - PR #37506

Fireworks AI​

  • Resolve short model names to long cost map keys - PR #40929
  • Flatten dict-form reasoning_effort to its effort string - PR #41335
  • Bill cache-write, reasoning and audio tokens via the shared cost calculator - PR #41339
  • Restore supports_vision on minimax-m3 in the cost map - PR #41699
  • Default fireworks cached input to the documented 50% discount when the map has no cache-read rate - PR #41917

Gemini and Vertex AI​

  • Bill Gemini Live sessions end to end - PR #40915
  • Map minimal thinking to low for Gemini 3.7 and 3.8 Flash - PR #41201
  • Bill Gemini Omni Interactions usage and Veo sampleCount on passthrough - PR #41322
  • Propagate the provider's modelVersion to the response model - PR #41338
  • Stream GCS batch output files from /v1/files/{id}/content - PR #41506
  • Set vertex gemma-4-26b-a4b-it-maas context window to 262144 - PR #41887
  • Preserve candidates with finishReason and no content - PR #41892

Mistral​

  • Accept reasoning_effort on all models and drop client_metadata for Codex compatibility - PR #41062
  • Add cache-read pricing to Mistral chat models missing it - PR #41736

OpenAI​

  • Add openai_system_messages_first to put system messages first for prompt caching - PR #41304
  • Resolve dated openai/azure snapshots to their undated cost map entry - PR #41423
  • Drop top_p for gpt-5 reasoning models when drop_params is set - PR #41469

xAI​

  • Keep 'instructions' on the xAI Responses API so system messages survive web search - PR #38254
  • Honor nested web_search filters on the xAI Responses API - PR #38268
  • Stop sending web_search_options to xAI's retired Live Search path - PR #38278

Model catalog and pricing​

  • Auto-sync Friendli model metadata into price registry - PR #35918
  • Sync Vertex AI prices: 14 models - PR #40955
  • Add azure gpt-chat-latest rates and drop retired friendliai llama-3.1 entries - PR #40976
  • Provider-scoped fill_missing_for_providers backfill from fallback generalization rules - PR #41093
  • Rolling registry audit: Gemini latest aliases, Nova cache pricing, OpenRouter/Together sync, Mistral GLM 5.3, Azure snapshots, Grok caching - PR #41112
  • Sync Azure, Azure AI, Gemini, OpenAI, Bedrock, Together AI, Fireworks and Vertex prices: 278 models, 59 new, 30 deprecated - PR #41154
  • Add aihubmix provider pricing entries - PR #41179
  • Add provider-neutral Gemini 2.5+ chat baseline fallback generalization - PR #41320
  • Sync Google Gemini prices: 22 models - PR #41457
  • Dedupe Nova cache_read_input_token_cost keys left by a text merge - PR #41496
  • Sync Together AI prices: 6 models, 6 deprecated [sync failed: Google Gemini] - PR #41570
  • Add stealth/union-alpha to the model cost map - PR #41576
  • Rolling registry audit: Azure retirement dates, Bedrock Mantle Grok 4.3 context window - PR #41597
  • Sync OpenRouter prices: 443 models, 191 new, 4 deprecated - PR #41727
  • Add qwen3.8 flash rows, fix Cohere embed v3 context, Bedrock Mantle and OpenRouter pricing - PR #41754
  • Sync OpenRouter prices: 2 models, 1 deprecated - PR #41770
  • Sync OpenRouter prices: 15 models, 6 deprecated - PR #41772
  • Sync OpenRouter prices: 172 models, 2 new - PR #41833
  • Classify off_peak_pricing as a structured object in the model prices schema generator - PR #41847
  • Backfill reseller Gemini entries from provider catalogs and prune retired ids - PR #41902
  • Drop anthropic deprecation floors and correct azure gpt-4.1-nano retirement date - PR #41964
  • Sync Azure prices: 5 models, 5 deprecated - PR #41966
  • Sync OpenRouter prices: 2 models - PR #41996
  • Sync OpenRouter prices: 2 models - PR #42006
  • Sync OpenRouter prices: 5 models - PR #42058
  • Sync OpenRouter prices: 2 models - PR #42063

General​

  • Keep a handler alive while a response it issued is still reading - PR #34829
  • Scan each log record once and collapse base64 payloads before the secret regex - PR #40934
  • Keep body and proxy headers on BadRequestError mapped from a litellm_proxy 400 - PR #40994
  • Keep litellm params out of provider request bodies - PR #41018
  • Keep extra_headers out of the chat request body on the httpx handler path - PR #41141
  • Reject an untranslatable tool_choice with a 400 instead of a 500 - PR #41234
  • Opt-in outbound HTTP/2 for httpx clients - PR #41268
  • Accept custom_provider_map providers before the first completion call - PR #41300
  • Keep the resolved provider so router custom pricing resolves for azure_ai deployments - PR #41623
  • Keep internal_server_error as the public type of an upstream 500 - PR #41930
  • Memoize shared nodes and fail closed past the depth cap - PR #41952

LLM API Endpoints​

Responses API​

  • Stop managed Responses WebSocket from leaking litellm_params into provider request body - PR #33101
  • Route Responses API to native /openai/v1/responses for Foundry Models - PR #33856
  • Guard empty-choices chunks in the Responses API streaming bridge - PR #34455
  • Translate the reasoning object into a chat-completion reasoning effort - PR #36363
  • Emit typed streaming failure events - PR #40243
  • Honor nested additional_drop_params paths - PR #40730
  • Hoist Codex additional_tools input items into the chat bridge tools - PR #40989
  • Filter bridged kwargs like the native Responses path - PR #41144
  • Recount tokens when a streamed response completes without usage - PR #41337
  • Announce message item before text events in the chat completions bridge - PR #41564
  • Keep the addressed response id off bridged provider requests - PR #41689
  • Merge deployment litellm_params into native websocket response.create frames - PR #41881
  • Restore encrypted_content and apply affinity on the native WebSocket relay - PR #41893
  • Return 400 instead of 500 for /v1/responses without input - PR #41939
  • Drop tool_search and local_shell in the chat completions bridge - PR #41953

Anthropic Messages​

  • Log the provider usage on deferred /v1/messages calls and price cache writes without a creation rate - PR #41172
  • Convert mid-conversation system turns to user turns on /v1/messages to chat completions - PR #41493
  • Forward the deployment api_base to agentic follow-up calls on /v1/messages - PR #41918

Batches and Files​

  • Support file delete and list for S3-backed managed files - PR #39836
  • Add general_settings.allowed_file_extensions for /v1/files uploads - PR #41106
  • Support Mistral files/batches and per-page OCR batch cost tracking - PR #41934
  • Run hosted_vllm batches inside LiteLLM - PR #41942

OCR​

  • Keep a downloaded document inlined when callbacks intercept the request - PR #41719
  • Add Rust-only Textract and sign provider requests after host hooks - PR #41977
  • Set DeepSeek OCR sampling defaults - PR #41992

Realtime and Audio​

  • Propagate deferred Nova Sonic stream failures to the router - PR #41064
  • Release max_parallel_requests slot when a realtime session ends without LLM callbacks - PR #41113
  • Resolve litellm_credential_name in realtime health checks - PR #41173
  • Add Azure AI Speech pass-through route - PR #41557
  • Stream Chirp speech-to-text over /v1/realtime - PR #41721
  • Add speech-to-text (Grok Voice Transcribe) via /v1/audio/transcriptions - PR #41914
  • Forward retrieval_filter from retrieval_config to vector store search - PR #34427
  • Resolve model_group_alias to its target for /v1/models metadata - PR #41483
  • Resolve registry stores on /v1/rag/ingest and reject providers without ingestion - PR #41940

Image Generation and Edits​

  • Stop forwarding the raw image[] and mask[] form keys - PR #39512

Agent-to-Agent​

  • Reach Microsoft Foundry agents with Entra auth and versioned card discovery - PR #41511

Rerank​

  • Bill Vertex search_units from input records and give every rerank response a unique id - PR #35180

Pass-through endpoints​

  • Serve the Claude Code gateway protocol under /claude_code_gateway - PR #34267
  • Attribute Vertex passthrough successes to the resolved router deployment - PR #41307
  • Add /nvidia_nim passthrough route for NIM object detection and OCR /v1/infer - PR #41316
  • Keep target URL query when client sends no query params - PR #41448
  • Stop forwarding LiteLLM credential headers on Bedrock agent-runtime passthrough - PR #41504
  • Add Amazon Transcribe pass-through with completion-time job pricing - PR #41515
  • Deepgram streaming /v1/listen WebSocket passthrough with duration-based cost tracking - PR #41554
  • Add TypeSafe AI Jev evaluate passthrough with registry-priced spend tracking - PR #41607
  • Forward every method on the typesafe pass-through route - PR #41723

General​

  • Forward provider request id headers on mapped error responses - PR #40925
  • Resolve x-litellm-call-id from response metadata when routes omit call_id - PR #41056
  • Include litellm_call_id in LLM API exception logs - PR #41205
  • Return 400 instead of 500 for lone surrogate escapes in request body - PR #41297
  • Carry litellm_call_id through endpoint specific error logs and failure responses - PR #41356

Management Endpoints / UI​

Admin UI​

  • Sum multi-round session duration in logs UI - PR #35388
  • Accept ssh clone urls when registering a skill - PR #35418
  • Hide admin write-form tabs on the models page from view-only admins - PR #38867
  • Show per-second pricing for video models instead of $0.00 token costs - PR #39308
  • Let team admins grant a team all proxy models - PR #40196
  • Persist disabling cache control injection points on model update - PR #40632
  • Let admins change a model's team from the model edit page - PR #40700
  • Show user attribution in Top Virtual Keys usage tables - PR #40729
  • Show internal user email in logs table and log detail drawer - PR #40737
  • List every provider in the cache leakage by-model table - PR #40875
  • Show the team alias on the model info page and in its raw JSON - PR #40992
  • Move tags typed into key metadata JSON into the Tags field - PR #41023
  • Block usage export and flag the range when a spend page fails - PR #41294
  • Persist Models table search, filters, sort and page in the URL - PR #41296
  • Add custom request headers to the API Playground - PR #41309
  • Show average response time per model in usage model activity - PR #41313
  • Configure capability and Fuse v2 classifiers - PR #41315
  • Shared URL-state layer for tables and tabs - PR #41331
  • Simplify Capability and Fuse advanced routing options - PR #41371
  • Keep untimed guardrail entries on the request lifecycle - PR #41374
  • Persist organizations and projects list, detail tab and key table state in the URL - PR #41445
  • Link MCP Servers page to the user's connected MCP servers - PR #41888
  • Show heuristic v2 score estimates in routing details - PR #42001
  • Configure web search interception from the Admin UI - PR #42007
  • Report whether the serving proxy has applied web search interception - PR #42042

Keys, teams and organizations​

  • Let proxy admins choose which team fields team admins may edit - PR #39996
  • Let team service account keys use key management endpoints for their own team - PR #40807
  • Bind JWT claims to registered agents via agent_id_jwt_field - PR #40904
  • Answer 409 on a credential name collision, make Terraform adoption opt-in - PR #40917
  • Unified custom_key_policy hook for key generate, update and regenerate - PR #40921
  • Allow virtual_key_claim_field per issuer - PR #40927
  • Add POST /management/v1/users/bulk for batched user and team membership creation - PR #41028
  • Add POST /management/v1/users/bulk_delete and POST /management/v1/teams/{team_id}/members/bulk_delete - PR #41039
  • Keep org admins' own team memberships in other orgs visible on team list - PR #41086
  • Show all model groups to proxy admins in /model_group/info - PR #41094
  • Allow opted-in team members to manage their routers - PR #41175
  • Track per-member organization spend - PR #41255
  • List directly assigned team models in model access errors - PR #41256
  • Enforce organization budgets when max_budget is 0 - PR #41271
  • Filter /key/list by active, expired, revoked or deleted status and serve deleted keys from /key/info - PR #41311
  • Team-level model_max_budget with key-level overrides - PR #41330
  • Apply team_member_budget updates to members still on the team default - PR #41347
  • Track team member spend when the member has no budget - PR #41349
  • Track project spend and enforce project budgets additively - PR #41354
  • Expose lifetime total_spend on virtual keys - PR #41403
  • Let team admins edit rpm_limit and max_budget when enabled - PR #41525
  • Temporary budget increase for team members - PR #41620
  • Bulk update team member budgets - PR #41632
  • Per-key default budget for dynamically created customers - PR #41636
  • Keep a forked member budget's reset window and audit bulk member budget writes - PR #41686
  • Propagate db model renames to key, team, org, project and user model allowlists - PR #41694
  • Emit audit events for member_delete and role changes and carry the final roster on team create - PR #41840
  • Let team admins manage projects via team_admin_editable_team_fields - PR #41916
  • Parse role_permissions where it is read - PR #41924
  • Register transcribe as a known provider for model grants - PR #41926
  • /key/bulk_update writes only the fields each item carries - PR #41949
  • Block project requests when max_budget is 0 - PR #41997

Authentication​

  • Refresh lite login session token grants from the live user and team rows - PR #40657
  • Gate the webhook test alert on proxy admins - PR #40814
  • Limit repeated failed Admin UI sign-in attempts - PR #40982
  • Hide default credentials login hint when UI_PASSWORD is set - PR #41107
  • Scope JWT key mappings by issuer to prevent cross-issuer collisions - PR #41281
  • Keep yaml pass-through endpoints visible to auth after db overlay - PR #41303
  • Never forward the LiteLLM virtual key to Anthropic on the /anthropic passthrough - PR #41340
  • Align pagination count validation with RFC 7644 - PR #41444
  • Add RFC 8693 token exchange for IdP JWTs on the gateway token endpoint - PR #41485
  • Inherit org alias, budget and rate limits for JWT and team-linked keys - PR #41681
  • Evict jwt key mapping cache on user, team, org, and bulk key deletion - PR #41707
  • Accept entitlements and roles entries without a value on SCIM user PUT - PR #41830

Proxy configuration​

  • Honor LITELLM_DISABLE_ACCESS_LOG_PATHS to drop noisy uvicorn access log lines - PR #41096
  • Skip background health check DB writes when the latest-row read fails - PR #41145
  • Honor LITELLM_LOG for uvicorn and proxy extras loggers - PR #41306
  • Hide model allowlist from client-facing model access denied errors - PR #41310
  • Explicit priority for policy attachment execution order - PR #41571
  • Persist only the keys a caller changed in save_config - PR #41748
  • Make the config file win over the database, with source and editable on both read endpoints - PR #41779
  • Make SettingsStore.clear() terminate when the config file owns a key - PR #41862
  • Refuse config-owned keys on POST /config/update - PR #41868
  • Refuse runtime writes to config-owned settings - PR #41931
  • Say when a stored setting is ignored because the config file owns it - PR #41985
  • Close the config-ownership gaps QA found in the settings store - PR #42009

CLI and coding agents​

  • Sync Codex /model picker from proxy /v1/models in lite codex - PR #40476
  • Drop enum.StrEnum so the CLI imports on Python 3.10 - PR #41046
  • Show routed models and session stats for LLM API keys - PR #41116
  • Label router costs and simplify the routed-model header - PR #41186
  • Rename lite autoroute up/down to start/stop, keeping the old names as deprecated aliases - PR #41672
  • Deprecate the litellm-proxy entrypoint in favour of lite - PR #41673
  • Add a VS Code extension that registers LiteLLM as a language model provider - PR #41865

Terraform​

  • Add tpm_limit, rpm_limit, budget_duration, allowed_models to litellm_team_member_add - PR #38682
  • Unlink the Terraform registry docs entries that 404 on click - PR #42003

AI Integrations​

Guardrails​

  • Add new upstream presidio pii entities including german set - PR #36775
  • Add Microsoft Agent 365 MCP tool-call guardrail - PR #38241
  • Record not_run evaluation when scoping leaves nothing to scan - PR #39050
  • Log blocked streaming guardrail responses as failures, not success - PR #40191
  • Protect cache_control-marked rows anywhere in history - PR #40315
  • Don't add post_call output scan for MCP-only Presidio modes - PR #40571
  • logging_only mode scans completed streams after delivery - PR #40702
  • Enforce tag budgets for tags added by guardrails - PR #40842
  • Write per-message guardrail rewrites back onto Responses input items - PR #40939
  • Scan the Anthropic top-level system prompt and tool_use arguments - PR #40984
  • Resolve caller identity from metadata buckets in custom code guardrail - PR #41126
  • Support pre_call and during_call modes for llm_as_a_judge - PR #41128
  • Keep polling file sanitization through non-terminal statuses - PR #41131
  • Derive contextual grounding source and query from plain messages - PR #41132
  • Protect the cached prefix through the last cache_control breakpoint - PR #41161
  • Give post-call scans the scoped request conversation and tools - PR #41220
  • Singulr v2 API contract with logging_only, pre_mcp_call and post_mcp_call - PR #41329
  • Scan a bounded window per streamed chunk - PR #41407
  • Release buffered stream chunks after each passing scan - PR #41425
  • Run prompt injection heuristics off the event loop - PR #41541
  • Stream Prompt Security post_call redactions in incremental_diff mode - PR #41558
  • Name the blocking guardrail in x-litellm-applied-guardrails - PR #41583
  • Preserve request-selected guardrails during tool execution - PR #41619
  • Dispatch llm_api_check moderation through during_call_hook - PR #41685
  • Add TypeSafe Jev relevance-based compaction guardrail - PR #41757
  • Keep requested model guardrails and key disable_fallbacks on rate-limit fallback - PR #41783
  • Resolve openai_moderations model at call time and default to omni-moderation-latest - PR #41895
  • Deliver guardrail text rewrites on multi-choice, unfinished, and envelope-less streams - PR #41933
  • Stop the Javelin api_version default leaking into Azure Content Safety - PR #41941
  • Drop the scoped request conversation and tools from post-call scans - PR #41986
  • Forward stream response attributes through the hook boundary and merge logged applied_guardrails - PR #42027

Logging and observability​

  • Drop None metric and event attributes before OTLP export - PR #36815
  • Clarify budget threshold messages - PR #39102
  • Run the remaining inline token counts off the event loop - PR #40262
  • Let the model emit objective + multi-query search shapes - PR #40399
  • Cap per-index OpenInference message attributes span-wide - PR #40562
  • Propagate W3C trace context on HTTP and WebSocket passthrough - PR #40669
  • Label pre-call rate limit failures with the resolved api_provider - PR #41059
  • Send llm_exceptions Slack alert for 5xx HTTPException and ProxyException - PR #41125
  • Map the caller's Langfuse user, session and tags onto the root and generation spans - PR #41140
  • Count 401 auth failures in litellm_proxy_failed_requests_metric - PR #41170
  • Keep events appended during an in-flight flush instead of clearing them - PR #41288
  • Add s3_log_prompts_only option to log prompts without responses - PR #41327
  • Default litellm_trace_id to the OTel server span trace id - PR #41386
  • Promote nested request metadata keys to litellm.metadata.* span attributes - PR #41462
  • Add customer (end_user) budget gauges - PR #41472
  • Fit per-index OpenInference messages to the span's remaining attribute budget - PR #41498
  • Add all-metrics dashboard and fix stale dashboard_v2 gauges - PR #41578
  • Opt-in llm_only span scope for Langfuse destinations and the operator Langfuse exporter - PR #41740
  • Keep caller traceparent and tracestate on pass-through relays - PR #41786
  • Anchor response duration and overhead at proxy receive time - PR #41891
  • Surface a failed search as a web_search_tool_result_error block and end the turn - PR #41905
  • Scope automatic breakpoints to supported Claude transports - PR #41920
  • Keep request metadata out of the cost tracking failure alert - PR #41950
  • Summarize embedding vectors as Langfuse observation output - PR #41982
  • Map Responses API output onto the Langfuse generation output - PR #41991

Secret Managers​

  • Sync AWS Secrets Manager on body-less key regenerate and key alias changes - PR #41458
  • Rename AWS Secrets Manager secret when key alias changes - PR #41468
  • Add separate login and secret namespaces for HashiCorp Vault - PR #41539

Spend Tracking, Budgets and Rate Limiting​

Cost tracking​

  • Remove unsupported soft_budget param from user docstrings - PR #36585
  • Index LiteLLM_SpendLogs by (api_key, startTime) - PR #37983
  • Store litellm_call_id and match it in request_id lookups - PR #39068
  • Bill cached realtime audio tokens at the audio cache-read rate - PR #40627
  • Predict prompt-cache costs across deployments - PR #40877
  • Report null cost for unpriced deployments instead of 0 - PR #40878
  • Serialize /model/info listing once with orjson - PR #41114
  • Bill gemini-embedding-2 per token and stop double charging audio - PR #41157
  • Track spend for streams a deployment hook converted to non-streaming - PR #41171
  • Estimate auto-router baseline costs from durable cache history - PR #41177
  • Carry image and video input tokens through the Responses usage bridge - PR #41237
  • Keep client User-Agent on auth failure spend logs - PR #41291
  • Split aggregated usage query into key-free rollups and bounded top-N keys - PR #41293
  • Price native Responses WebSocket turns at their returned service_tier - PR #41318
  • Add LiteLLM_DailyGlobalSpend key-free rollup for the usage dashboard - PR #41324
  • Preserve Anthropic pricing modifiers in router savings - PR #41341
  • Remove duplicate user budget hook that 429'd zero-cost models - PR #41345
  • Attribute router-rejected requests to the model group provider - PR #41507
  • Price Azure PTU spillover requests at standard token rates - PR #41569
  • Reject non-string model with 400 and log its spend as unknown-model - PR #41633
  • Bill cache-read tokens at the input rate when the map has no cache-read rate - PR #41832
  • Unpin cost-map pricing copied into model_info and report pricing overrides - PR #41843
  • Requeue daily spend rows when the commit fails without the Redis buffer - PR #41878
  • Keep the raw client model out of spend logs for rejections outside the router - PR #41943
  • Bill DeepSeek V4.1 Flash and V4 Pro at off-peak rates outside peak hours - PR #41960

Budgets​

  • Reconcile budget reservation before enqueuing spend to the DB - PR #40310
  • Reset budgets by decrementing pre-reset spend instead of zeroing rows - PR #41279
  • Key model rpm/tpm override takes precedence over team model limit - PR #41302
  • Re-check budget on router fallback targets - PR #41379
  • Count TPM/RPM usage before building rate-limit headers - PR #41474
  • Page end-user cache invalidation after a budget reset - PR #41488
  • Reject with 429 when a deployment's max_parallel_requests slots are all in use - PR #41555
  • Reset sibling tpm/rpm counters when the shared rate limit window rolls over - PR #41838
  • Render the 429 reset time in UTC as labelled - PR #41911
  • Enforce model tpm limits against shared redis usage across replicas - PR #41915

Rate limiting​

  • Add tpd_limit (tokens per day) for batch submissions - PR #40997

MCP Gateway​

MCP Gateway​

  • Require admission for delegated OAuth - PR #40923
  • Authorize JWT OAuth credential persistence - PR #41314
  • Fail closed on missing upstream credentials - PR #41364
  • Count admin static headers as api_key credential slots - PR #41514
  • Restrict health discovery to virtual key grants - PR #41609
  • Allowlist MCP client applications at the gateway - PR #41667
  • Show live gateway sessions by AI client and user - PR #41692
  • Upgrade SDK2 while preserving legacy gateway behavior - PR #41718
  • Let proxy admins force-close live MCP sessions and revoke stored user credentials - PR #41725

Performance / Loadbalancing / Reliability improvements​

Auto Router and model routing​

  • Preserve provider affinity - PR #40228
  • Record flat retry attempts and cap retries from attempted_retries - PR #40930
  • Route mid-stream error events through exception_type so content_policy_fallbacks fire - PR #40988
  • Cool down team deployments on 429 when a sibling serves the same public model - PR #40991
  • Name the all-deployments-in-cooldown error on 429 responses - PR #40995
  • Honor team and key provider weights - PR #41072
  • Keep weighted routing when a deployment id equals a model_name - PR #41156
  • Preserve session model choice within each complexity tier - PR #41174
  • Bind per-request routing_strategy override selectors to the request's callbacks - PR #41178
  • Count num_retries_per_request across fallback hops - PR #41191
  • Stop counting caller-set timeout 408s toward deployment cooldown - PR #41230
  • Add capability classifier as Fuse foundation - PR #41270
  • Add Fuse V2 classifier after capability forecasting - PR #41272
  • Add per-model Fast mode toggle - PR #41282
  • Stop registering a caller-supplied credential as a router deployment - PR #41289
  • Resolve router_settings.model_group_alias before key/team model auth - PR #41308
  • Limit unlicensed Capability and Fuse v2 routers to one each - PR #41326
  • Validate routing_groups at save time and keep invalid DB groups from blocking SSO load - PR #41351
  • Stream shadow traffic and fan out silent_model to multiple targets - PR #41368
  • Discover token limits for hosted OpenAI-compatible models - PR #41508
  • Add TypeSafe Jev as a complexity router classifier - PR #41615
  • Add maintained Fuse model and harness presets - PR #41617
  • Let a wildcard allowed_features license grant the auto_router feature - PR #41684
  • Honor stream_timeout on the SDK-native passthrough route (/v1/messages, /converse) - PR #41875

Caching, database and runtime​

  • Retry rate-limit fallbacks from a pristine request snapshot - PR #40596
  • Log a timeout streak once per interval instead of one line per cache call - PR #40817
  • Release completed max-parallel slots promptly - PR #40843
  • Log one bounded summary for a burst of timed-out LoggingWorker callbacks - PR #40912
  • Skip correlation contextvar stamping when request_correlation_in_logs is off - PR #41054
  • Load team membership once per request and skip prisma on an L1 hit - PR #41102
  • Cache custom HuggingFace tokenizers across /utils/token_counter requests - PR #41216
  • Keep access-group raw SQL writes on the writer while writer_unavailable is stale - PR #41283
  • Defer fastapi and tiktoken BPE imports out of import litellm - PR #41585
  • Add litellm-http client pool and inject it into the OCR route - PR #41897
  • Refuse native routes in processes forked after the runtime started - PR #41987

PR roll-up by ownership area​

Customer-visible PRs: 366

  • Management Endpoints / UI: 90
  • Models & Providers: 86
  • AI Integrations: 59
  • LLM API Endpoints: 51
  • Spend / Budgets / Rate Limits: 36
  • Performance / Reliability: 35
  • MCP: 9

New Contributors​

Full Changelog​

Compare release contents on GitHub

🚅
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.
Learn more →