LiteLLM v1.99.0 - Dark Mode, CLI OAuth Login & Batch Billing
Released
Deploy this version​
- Docker
- Pip
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.99.0
pip install litellm==1.99.0
Audit logging now defaults to on for proxies running an Enterprise license. store_audit_logs becomes tri-state: unset plus a premium license resolves to enabled, replacing roughly fifteen scattered gate checks with one resolver, and LITELLM_STORE_AUDIT_LOGS is now honoured by the key hooks that previously ignored it. An enterprise proxy that never set the flag starts writing audit rows on upgrade, which will grow the audit table. Set store_audit_logs: false explicitly to keep the prior behavior. See PR #37518.
Shadow eval jobs are budgeted in dollars instead of turns. POST /auto_router/shadow_eval/start now takes max_budget in USD per key; a request that still sends max_turns gets a 422 naming the replacement. The sampler, sweep, stop guard, and derived status all gate on the billed response_cost of the shadow arm and the judge, and admission re-checks the proxy's cross-pod spend counter. Jobs created before the migration keep their configured turn budget, since max_budget stays NULL on those rows. See PR #37555.
A shadow eval job now covers a list of keys rather than one. api_key_ids replaces api_key_id on the start payload, results come back pooled and per key, and sibling rows are tied together by a new group_id column. A caller still sending the singular api_key_id must move to the list form. See PR #37251.
Key Highlights​
- The Admin UI's migration off antd and Tremor is complete -
@tremor/react,antd, and@ant-design/iconsare all removed from the dashboard's dependencies, with the last components, the shared primitives, and every form ported onto shadcn (base-vega) and react-hook-form. 130 UI pull requests land in this window, and the dashboard is now on React 19 - Dark mode ships - a light/dark/system toggle in the top bar, semantic status tokens for success, warning and info, a dark variant of the LiteLLM logo, and an admin-supplied dark variant of a custom logo. Hardcoded Tailwind palette classes across the dashboard are mapped onto theme tokens so surfaces, form controls, inline styles and code blocks all follow the theme
lite loginis a real OAuth flow - the CLI now authenticates with an authorization code plus PKCE against the proxy, stores the credential and the refresh token in the OS keychain rather than atoken.jsonon disk, andlite login --config-claudewires Claude Code up at login- Batch spend is accounted for end to end - enqueued-token rate limiting admits batches against a token budget and refunds on completion or cancellation, cost rows are claimed atomically so multi-pod polling cannot double-bill, cancelled and failed batches that still produced output are billed, a single undecodable output line no longer zeroes a batch's spend, and Bedrock batches can be cancelled through
POST /v1/batches/{id}/cancel - Provisioned throughput can be declared in
config.yaml- a PTU reservation no longer has to be created through the API; the rollup accrues flat cost for config-declared deployments, refuses an incomplete reservation the way the endpoints do, requires an operator-declared id, and warns when a config declares PTU while attribution is switched off - The complexity router is operator-configurable - operator-defined tier sets for the LLM classifier, custom classifier plugins via
classifier_type: custom, a plan-mode tier floor for coding-agent clients, a business classification rubric preset, and per-model reasoning effort in the tier editor
Included from v1.99.0-rc.2​
This stable also carries everything that landed on the release line after the rc.1 cut, shipped in v1.99.0-rc.2 and included here:
- Anthropic
- Translate
tool_resultdocument blocks in the/v1/messagesbridge - PR #38251
- Translate
- Dashboard (dark mode follow-ups)
- Make provider logos readable in dark mode - PR #38588
- One-click theme toggle and matching Docs/Blog styling in the top bar - PR #38601
- Make code blocks and the logs JSON viewer follow the theme in dark mode - PR #38771, PR #38778
- Make playground chat bubbles theme-aware, and theme the created-key box - PR #37978, PR #37985
- Dashboard (bug fixes)
- Open select popups below the trigger instead of over it - PR #38554
- Stop server-searched comboboxes from clobbering picks and queries, and let the paginated search select keep what the user types - PR #38574, PR #38475
- Keep a deleted-from search query instead of blanking the box - PR #38830
- Keep focus in the add model public name input while typing, and restore its tooltip layout - PR #38366, PR #37986
- Restore the reopen control for the log drawer's trace sidebar - PR #38782
- Stack the policy flow builder below the popup layer so guardrail options render - PR #38273
- Drop stray text next to Close in the model connection test dialog - PR #38852
- Docker
- End-to-End Testing
New Providers and Endpoints​
New Providers (5 new providers)​
| Provider | Supported LiteLLM Endpoints | Description |
|---|---|---|
| SCX.ai | /chat/completions | JSON-configured OpenAI-compatible provider, registered in the provider enum, config map, pricing map and dashboard, shipping GLM-5.2 and Qwen3.8-Max |
| Cognition | /chat/completions | Cognition gets its own provider identity rather than riding an OpenAI-compatible alias, with swe-1.6, swe-1.7 and swe-1.7-lightning priced |
| AWS Bedrock AgentCore | /search | Bedrock AgentCore registered as a search provider |
| Amazon Comprehend Medical | /* passthrough | Passthrough provider for Comprehend Medical, routed and credentialed through the proxy |
| Valkey | Vector stores | Valkey as a managed vector store provider alongside the existing backends |
New LLM API Endpoints (4 new endpoints)​
| Endpoint | Method | Description | Documentation |
|---|---|---|---|
/model/deprecations | GET | Lists deployments whose models carry a provider-announced deprecation_date, backing proactive deprecation alerts | Proxy Config |
/team/daily/activity/aggregated | GET | Pre-aggregated team activity for the Usage tab, replacing the per-row scan it used to do client-side | Cost Tracking |
/team/{team_id}/callback/{callback_name} | DELETE | Removes a single team-scoped logging callback without rewriting the whole callback set | Team Logging |
/auto_router/validate_complexity_router_config | POST | Dry-runs a complexity-router config against the write gate so an invalid tier set is rejected before it is saved | Auto Router |
New Models / Updated Models​
New Model Support (136 new models)​
| Provider | Model | Context Window | Input ($/1M tokens) | Output ($/1M tokens) | Features |
|---|---|---|---|---|---|
| OpenAI | gpt-5.6-cyber | 400K | $12.50 | $75.00 | Reasoning, vision, PDF input, function calling, parallel function calling, tool choice, prompt caching, response schema, web search, computer use, native streaming |
| OpenAI | daybreak-red-latest | 400K | $12.50 | $75.00 | Reasoning, vision, PDF input, function calling, parallel function calling, tool choice, prompt caching, response schema, web search, computer use, native streaming |
| OpenAI | daybreak-blue-latest | 1M | $4.00 | $20.00 | Reasoning, vision, PDF input, function calling, parallel function calling, tool choice, prompt caching, response schema, web search, computer use, native streaming |
| OpenAI | chat-latest | 400K | $5.00 | $30.00 | Vision, PDF input, function calling, parallel function calling, tool choice, prompt caching, response schema, web search, native streaming |
| Azure | azure/gpt-audio-mini | 128K | $0.60 | $2.40 | Function calling, parallel function calling, tool choice, native streaming |
| Azure | azure/gpt-realtime-mini | 32K | $0.60 | $2.40 | Realtime, audio input and output, function calling, parallel function calling, tool choice, $0.0000008 per image |
| Amazon Bedrock | us.openai.gpt-5.6-sol | 1M | $5.50 | $33.00 | Vision, function calling, tool choice |
| Amazon Bedrock | global.openai.gpt-5.6-sol | 1M | $5.00 | $30.00 | Vision, function calling, tool choice |
| Amazon Bedrock | us.openai.gpt-5.6-terra | 1M | $2.20 | $13.20 | Vision, function calling, tool choice |
| Amazon Bedrock | global.openai.gpt-5.6-terra | 1M | $2.00 | $12.00 | Vision, function calling, tool choice |
| Amazon Bedrock | us.openai.gpt-5.6-luna | 1M | $0.22 | $1.32 | Vision, function calling, tool choice |
| Amazon Bedrock | global.openai.gpt-5.6-luna | 1M | $0.20 | $1.20 | Vision, function calling, tool choice |
| Amazon Bedrock | us.xai.grok-4.6 | 500K | $2.20 | $6.60 | Reasoning, vision, function calling, tool choice, prompt caching |
| Amazon Bedrock | global.xai.grok-4.6 | 500K | $2.00 | $6.00 | Reasoning, vision, function calling, tool choice, prompt caching |
| Amazon Bedrock | bedrock_mantle/xai.grok-4.6 | 500K | $2.20 | $6.60 | Reasoning, vision, function calling, tool choice, prompt caching, response schema |
| Amazon Bedrock | bedrock/guardrails | - | - | - | Guardrail usage and cost accounting |
| AWS Bedrock AgentCore | agentcore/search | - | - | - | Search |
| Databricks | databricks/databricks-claude-opus-4-6 | 1M | $5.00 | $25.00 | Reasoning, function calling, tool choice |
| Databricks | databricks/databricks-claude-sonnet-4-6 | 1M | $3.00 | $15.00 | Reasoning, function calling, tool choice |
| Databricks | databricks/databricks-gemini-3-pro | 1M | $2.50 | $15.00 | Function calling, tool choice |
| Databricks | databricks/databricks-gemini-3-1-pro | 1M | $2.50 | $15.00 | Function calling, tool choice |
| Databricks | databricks/databricks-gemini-3-flash | 1M | $0.625 | $3.75 | Function calling, tool choice |
| Databricks | databricks/databricks-gemini-3-1-flash-lite | 1M | $0.3125 | $1.88 | Function calling, tool choice |
| Databricks | databricks/databricks-gpt-5-4 | 272K | $2.50 | $15.00 | Chat |
| Databricks | databricks/databricks-gpt-5-4-mini | 272K | $0.75 | $4.50 | Chat |
| Databricks | databricks/databricks-gpt-5-4-nano | 272K | $0.20 | $1.25 | Chat |
| Databricks | databricks/databricks-gpt-5-2 | 272K | $1.75 | $14.00 | Chat |
| Databricks | databricks/databricks-gpt-5-2-codex | 272K | $1.75 | $14.00 | Chat |
| Databricks | databricks/databricks-gpt-5-3-codex | 272K | $1.75 | $14.00 | Chat |
| Databricks | databricks/databricks-gpt-5-1-codex-max | 272K | $1.25 | $10.00 | Chat |
| Databricks | databricks/databricks-gpt-5-1-codex-mini | 272K | $0.25 | $2.00 | Chat |
| Google Gemini | gemini/gemini-3.1-flash-lite-image | 65K | $0.25 | $1.50 | Image generation, vision, function calling, $0.00028 per image |
| Google Gemini | gemini/gemini-3.5-live-translate-preview | - | $3.50 | $21.00 | Audio input and output |
| Google Vertex AI | vertex_ai/gemini-3.1-flash-lite-image | 65K | $0.25 | $1.50 | Image generation, vision, PDF input, video input, prompt caching, $0.00028 per image |
| Google Vertex AI | gemini-3.1-flash-lite-image | 65K | $0.25 | $1.50 | Image generation, vision, PDF input, video input, prompt caching, $0.00028 per image |
| Fireworks AI | fireworks_ai/kimi-k3 | 1M | $3.00 | $15.00 | Reasoning, vision, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/kimi-k3-fast | 1M | $4.50 | $22.50 | Reasoning, vision, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/kimi-k3-us | 1M | $3.30 | $16.50 | Reasoning, vision, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/glm-5p2-fast | 1M | $2.10 | $6.60 | Reasoning, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/glm-5p2-fast-us | 1M | $2.10 | $6.60 | Reasoning, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/deepseek-v4-flash-0731 | 1M | $0.14 | $0.28 | Reasoning, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/qwen3p8-max | 262K | $2.00 | $6.00 | Reasoning, vision, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/muse-glimmer-30b | 131K | $0.35 | $1.50 | Reasoning, vision, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/nemotron-3-ultra-nvfp4 | 262K | $0.60 | $2.40 | Reasoning, function calling, tool choice, response schema |
| Fireworks AI | fireworks_ai/nemotron-lightning-3p5-30b-a3b | 262K | $0.05 | $0.20 | Reasoning, function calling, tool choice, response schema |
| Mistral | mistral/glm-5-2 | 1M | $1.40 | $4.40 | Reasoning, function calling, tool choice, prompt caching, response schema |
| Mistral | mistral/zai-glm-5-2 | 1M | $1.40 | $4.40 | Reasoning, function calling, tool choice, prompt caching, response schema |
| Mistral | mistral/mistral-ocr-4-1 | - | - | - | OCR |
| Moonshot | moonshot/kimi-k3 | 1M | $3.00 | $15.00 | Reasoning, vision, video input, function calling, tool choice, response schema |
| OpenRouter | openrouter/anthropic/claude-opus-5 | 1M | $5.00 | $25.00 | Reasoning, vision, PDF input, function calling, tool choice, prompt caching, response schema, computer use |
| OpenRouter | openrouter/deepseek/deepseek-v4-pro | 1M | $1.32 | $3.96 | Reasoning, function calling, tool choice, prompt caching, response schema |
| OpenRouter | openrouter/deepseek/deepseek-v4-pro-0813 | 1M | $1.32 | $3.96 | Reasoning, function calling, tool choice, prompt caching, response schema |
| Perplexity | perplexity/perplexity/kimi-k3 | - | $3.00 | $15.00 | Responses, reasoning, function calling, web search |
| Perplexity | perplexity/perplexity/glm-5.2 | - | $1.40 | $4.40 | Responses, reasoning, function calling, web search |
| Perplexity | perplexity/perplexity/kimi-k2.7-code | - | $0.95 | $4.00 | Responses, function calling, web search |
| Perplexity | perplexity/perplexity/deepseek-v4-flash-0731 | - | $0.13 | $0.26 | Responses, reasoning, function calling, web search |
| Perplexity | perplexity/pplx-embed-context-v1-4b | 32K | $0.05 | - | Embedding |
| Perplexity | perplexity/pplx-embed-context-v1-0.6b | 32K | $0.008 | - | Embedding |
| Voyage | voyage/voyage-4-large | 32K | $0.12 | - | Embedding |
| Voyage | voyage/voyage-4 | 32K | $0.06 | - | Embedding |
| Voyage | voyage/voyage-4-lite | 32K | $0.02 | - | Embedding |
| Voyage | voyage/voyage-code-4 | 32K | $0.12 | - | Embedding |
| Voyage | voyage/voyage-context-4 | 120K | $0.12 | - | Embedding |
| Voyage | voyage/voyage-multimodal-3.5 | 32K | $0.12 | - | Embedding, image embedding input |
| SCX.ai | scx-ai/GLM-5.2 | 1M | $0.61 | $1.98 | Reasoning, function calling, tool choice, prompt caching, response schema |
| SCX.ai | scx-ai/Qwen3.8-Max | 1M | $1.65 | $4.99 | Reasoning, vision, function calling, tool choice, prompt caching, response schema |
| Cognition | cognition/swe-1.6 | - | $0.50 | $2.50 | Function calling, prompt caching |
| Cognition | cognition/swe-1.7 | - | $0.50 | $2.50 | Function calling, prompt caching |
| Cognition | cognition/swe-1.7-lightning | - | $2.50 | $12.50 | Function calling, prompt caching |
| fal.ai | fal_ai/gpt-image-2, fal_ai/openai/gpt-image-2 | - | - | - | Image generation and edit, priced per size and quality from request params |
The fal.ai entry expands to 57 registry keys: the two base slugs plus every low/medium/high quality crossed with six output resolutions, for both generation and /edit. The Fireworks entries each carry an accounts/fireworks/models/ alias, and the two glm-5p2-fast and two kimi-k3 fast/US variants also carry an accounts/fireworks/routers/ alias, so the slug you already use keeps resolving.
Beyond the new entries, this release is another large cost-map maintenance pass over 288 existing entries. 219 entries gain or correct a provider-announced deprecation_date, 25 gain Batch API input and output rates alongside a regional_endpoint_uplift_multiplier, and 15 gain a prompt_cache_min_tokens floor, including 4096 on Gemini 3.5, 3.6 and 3.7 Flash and 3.1 Pro Preview. Pricing moves in both directions: Gemini 3.6 Flash halves to $0.75 / $3.75 per 1M on both Gemini and Vertex, gpt-5.6 and gpt-5.6-sol drop to $4.00 / $20.00 on a promotional cut, mistral/codestral-latest falls to $0.30 / $0.90 and Vertex DeepSeek V3.1 MaaS to $0.60 / $1.70, while gemini-3.1-flash-image doubles to $0.50 / $3.00 and mistral/mistral-small-latest rises to $0.15 / $0.60. The GPT-5.6 family's max input tokens is corrected from 1.05M to 922K across the Azure, US and EU entries. Flex and priority service-tier rates land on 8 and 5 entries respectively, thinking_always_on on 11, and supports_prompt_cache_breakpoint on 4. No pricing entries were removed.
Features​
- Amazon Bedrock
- Cognition
- SCX.ai
- Add SCX.ai as a JSON-configured OpenAI-compatible provider - PR #34752
- Mistral
- Add
zai-glm-5-2andglm-5-2pricing - PR #37110
- Add
- Perplexity
- Add the Agent API third-party models - PR #37112
- Databricks
- Add cost map entries for 14 newer Databricks models - PR #28501
- Moonshot
- Anthropic
- General
Bug Fixes​
- Amazon Bedrock
- Preserve cache token usage when
invocationMetricsreplaces the usage block, and report provider thinking tokens instead of classifying them as text - PR #36878, PR #35998 - Forward provider response headers on chat completions, degrade gracefully on malformed tool-call arguments, and validate file-content retrieval against the configured output bucket - PR #37003, PR #33842, PR #31435
- Read batch usage by payload shape rather than provider name, and report uploaded size in the
FileObjectreturned by managed batch uploads - PR #37078, PR #36392
- Preserve cache token usage when
- Anthropic
- Emit
tool_usecontent_block_startwithout awaiting the next chunk, and resolve the provider exactly once on/v1/messages- PR #37310, PR #37757 - Fold guardrail-modified leading system rows into the top-level
systemparam - PR #37231 - Stop emitting empty thinking blocks on the Responses adapter, and preserve optional Responses tool properties - PR #36033, PR #36979
- Log partial stream spend when a
/v1/messagesclient disconnects mid-stream - PR #37558 - Gate sampling params on
/v1/messagesthe way/chat/completionsdoes, omitthinking.type=disabledfor always-on thinking models, and accept a boolthinkingparam instead of raisingAttributeError- PR #37868, PR #37510, PR #37423
- Emit
- Google Vertex AI
- Apply the regional endpoint uplift to cost tracking, resolving the served location from the passthrough URL,
optional_params, and the deployment for native/v1/messagescalls - PR #37543 - Only fall back to a placeholder thought signature on the first parallel function call - PR #37541
- Convert messages to contents in Gemini
count_tokens- PR #36981
- Apply the regional endpoint uplift to cost tracking, resolving the served location from the passthrough URL,
- Google Gemini
- Azure
- Fireworks AI
- Skip the
accounts/rewrite forFW-*Foundry deployment ids - PR #37242
- Skip the
- SageMaker
- Send the inference component header and honor
hf_model_name- PR #37766
- Send the inference component header and honor
- General
- Stop forwarding a client Anthropic OAuth token to Bedrock and Vertex - PR #37905
- Stop the Rust flag and the
client_side_timeoutmarker from leaking into upstream provider request bodies - PR #37218, PR #37346 - Resolve the advisor sub-call through the proxy router - PR #36246
- Surface provider errors from the container file content endpoint - PR #37737