Skip to main content

[PRE-RELEASE] v1.74.3-stable

Krrish Dholakia
Krrish Dholakia
CEO, LiteLLM
Ishaan Jaffer
Ishaan Jaffer
CTO, LiteLLM

Deploy this version​

docker run litellm
docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:v1.74.3.rc.2

New Models / Updated Models​

Pricing / Context Window Updates​

ProviderModelContext WindowInput ($/1M tokens)Output ($/1M tokens)Type
Xaixai/grok-4256k$3.00$15.00New
Xaixai/grok-4-0709256k$3.00$15.00New
Xaixai/grok-4-latest256k$3.00$15.00New
Mistralmistral/devstral-small-2507128k$0.1$0.3New
Mistralmistral/devstral-medium-2507128k$0.4$2New
Azure OpenAIazure/o3-deep-research200k$10$40New

Features​

Bugs​

  • Github Copilot
    • Fix API base url for Github Copilot - PR
  • Bedrock
    • Ensure supported bedrock/converse/ params = bedrock/ params - PR
    • Fix cache token cost calculation - PR
  • XAI
    • ensure finish_reason includes tool calls when xai responses with tool calls - PR

LLM API Endpoints​

Features​

  • /completions
    • Return ‘reasoning_content’ on streaming - PR
  • /chat/completions
    • Add 'thinking blocks' to stream chunk builder - PR
  • /v1/messages
    • Fallbacks support - PR
    • tool call handling for non-anthropic models (/v1/messages to /chat/completion bridge) - PR

MCP Gateway​

Features​

  • Cost Tracking
    • Add Cost Tracking - PR
    • Add usage tracking - PR
    • Add custom cost configuration for each MCP tool - PR
    • Add support for editing MCP cost per tool - PR
    • Allow using custom post call MCP hook for cost tracking - PR
  • Auth
    • Allow customizing what client side auth header to use - PR
    • Raises error when MCP server header is malformed in the request - PR
  • MCP Server
    • Allow using stdio MCPs with LiteLLM (enables using Circle CI MCP w/ LiteLLM) - PR, Get Started

Bugs​

  • General
  • MCP Server
    • Fix mcp tool separator to work with Claude code - PR, Get Started
    • Add validation to mcp server name to not allow "-" (enables namespaces to work) - PR

Management Endpoints / UI​

Features​

  • Model Hub
    • new model hub table view - PR
    • new /public/model_hub endpoint - PR
    • Make Model Hub OSS - PR
    • New ‘make public’ modal flow for showing proxy models on public model hub - PR
  • MCP
    • support for internal users to use and manage MCP servers - PR
    • Adds UI support to add MCP access groups (similar to namespaces) - PR
    • MCP Tool Testing Playground - PR
    • Show cost config on root of MCP settings - PR
  • Test Key
    • Stick sessions - PR
    • MCP Access Groups - allow mcp access groups - PR
  • Usage
    • Truncate long labels and improve tooltip in Top API Keys chart - PR
    • Improve Chart Readability for Tag Usage - PR
  • Teams
    • Prevent navigation reset after team member operations - PR
    • Team Members - reset budget, if duration set - PR
    • Use central team member budget when max_budget_in_team set on UI - PR
  • SSO
    • Allow users to run a custom sso login handler - PR
  • Navbar
    • improve user dropdown UI with premium badge and cleaner layout - PR
  • General
    • Consistent layout for Create and Back buttons on all the pages - PR
    • Align Show Password with Checkbox - PR
    • Prevent writing default user setting updates to yaml (causes error in non-root env) - PR

Bugs​

  • Model Hub
    • fix duplicates in /model_group/info - PR
  • MCP
    • Fix UI not syncing MCP access groups properly with object permissions - PR

Logging / Guardrail Integrations​

Features​

Bugs​

Performance / Loadbalancing / Reliability improvements​

Bugs​

  • Responses API Bridge
    • add image support for Responses API when falling back on Chat Completions - PR s/o @ryan-castner
  • aiohttp
    • Properly close aiohttp client sessions to prevent resource leaks - PR
  • Router
    • don't add invalid deployment to router pattern match - PR

General Proxy Improvements​

Bugs​

  • S3
    • s3 config.yaml file - ensure yaml safe load is used - PR
  • Audit Logs
    • Add audit logs for model updates - PR
  • Startup
    • Multiple API Keys Created on Startup when max_budget is enabled - PR
  • Auth
    • Resolve model group alias on Auth (if user has access to underlying model, allow alias request to work) - PR
  • config.yaml
    • fix parsing environment_variables from config.yaml - PR
  • Security
    • Log hashed jwt w/ prefix instead of actual value - PR

Features​

  • MCP
    • Bump mcp version on docker img - PR
  • Request Headers
    • Forward ‘anthropic-beta’ header when forward_client_headers_to_llm_api is true - PR

New Contributors​

Git Diff​