Skip to main content

[Preview] v1.83.3.rc.1 - Introducing MCP Skills Marketplace

Deploy this version​

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:main-v1.83.3.rc.1

Key Highlights​

Skills Marketplace​

The Skills Marketplace gives teams a self-hosted catalog for discovering, installing, and publishing Claude Code skills. Skills are portable across Anthropic, Vertex AI, Azure, and Bedrock — so a skill published once works everywhere your gateway routes to.

Skills Marketplace

Get Started

Guardrail Fallbacks​

Guardrail pipelines now support an optional on_error behavior. When a guardrail check fails or errors out, you can configure the pipeline to fall back gracefully — logging the failure and continuing the request — instead of returning a hard 500 to the caller. This is especially useful for non-critical guardrails where availability matters more than enforcement.

Team Bring Your Own Guardrails​

Teams can now attach guardrails directly from the team management UI. Admins configure available guardrails at the project or proxy level, and individual teams select which ones apply to their traffic — no config file changes or proxy restarts needed. This also ships with project-level guardrail support in the project create/edit flows.

MCP Toolsets​

MCP Toolsets let AI platform admins create curated subsets of tools from one or more MCP servers and assign them to teams and keys with scoped permissions. Instead of granting access to an entire MCP server, you can now bundle specific tools into a named toolset — controlling exactly which tools each team or API key can invoke. Toolsets are fully managed through the UI (new Toolsets tab) and API, and work seamlessly with the Responses API and Playground.

MCP Toolsets

Get Started​

New Models / Updated Models​

New Model Support​

ProviderModelContext WindowInput ($/1M tokens)Output ($/1M tokens)Features
Brave Searchbrave/search---Search tool integration metadata in cost map (PR #25042)
AWS Bedrocknvidia.nemotron-super-3-120b256KAddedAddedChat completions, function calling, system messages (PR #24588)
OCI GenAIMultiple new chat + embedding entriesVariesUpdatedUpdatedExpanded chat + embedding model catalog

Features​

Bug Fixes​

LLM API Endpoints​

Features​

Bugs​

Management Endpoints / UI​

Features​

  • Virtual Keys

    • Add substring search for user_id and key_alias on /key/list - PR #24751, PR #24746
    • Wire team_id filter to key alias dropdown on Virtual Keys tab - PR #25119, PR #25114
    • Allow hashed token_id in /key/update endpoint - PR #24969

  • Teams + Organizations

    • Resolve access-group models/MCP servers/agents in team endpoints and UI - PR #25119, PR #25027
    • Allow changing team organization from team settings - PR #25095
    • Add per-model rate limits to team edit/info views - PR #25156, PR #25144

  • Usage + Analytics

    • Add paginated team search on usage page filters - PR #25107
    • Use entity key for usage export display correctness - PR #25153

  • Models + Providers

    • Include access-group models in UI model listing - PR #24743
    • Expose Azure Entra ID credential fields in provider forms - PR #25137
    • Do not inject vector_store_ids: [] when editing a model - PR #25133

  • Guardrails UI

    • Add project-level guardrails support in project create/edit flows - PR #25100
    • Allow adding team guardrails from the UI - PR #25038

  • UI Cleanup

    • Migrate Tremor Text/Badge to antd Tag and native spans - PR #24750

Bugs​

  • Fix logs page showing unfiltered results when backend filter returns zero rows - PR #24745
  • Enforce upperbound key params on /key/update and bulk update hook paths - PR #25110, PR #25103
  • Fix team model update 500 due to unsupported Prisma JSON path filter - PR #25152

AI Integrations​

Logging​

  • General
    • Eliminate race condition in streaming guardrail_information logging - PR #24592
    • Use actual start_time in failed request spend logs - PR #24906
    • Harden credential redaction + stop logging raw sensitive auth values - PR #25151

Guardrails​

  • Add optional on_error for guardrail pipeline failures - PR #25150, PR #24831
  • Return HTTP 400 (vs 500) for Model Armor streaming blocks - PR #24693

Prompt Management​

  • Add environment + user tracking for prompts (development/staging/production) in CRUD + UI flows - PR #25110, PR #24855

Secret Managers​

  • No major new secret manager provider additions in this RC.

Spend Tracking, Budgets and Rate Limiting​

  • Enforce budget for models not directly present in the cost map - PR #24949
  • Add per-model rate limits in team settings/info UI - PR #25144
  • Fix unversioned Vertex Claude Haiku pricing entry to avoid $0.00 accounting - PR #25151

MCP Gateway​

  • Introduce MCP Toolsets with DB types, CRUD APIs, scoped permissions, and UI management tab - PR #25155
  • Resolve toolset names and enforce toolset access correctly in Responses API and streamable MCP paths - PR #25155
  • Switch toolset permission caching to shared cache path and improve cache invalidation behavior - PR #25155
  • Allow JWT auth for /v1/mcp/server/* sub-paths - PR #25113, PR #24698
  • Add STS AssumeRole support for MCP SigV4 auth - PR #25151
  • Add tag query fix + MCP metadata support cherry-pick - PR #25145

Performance / Loadbalancing / Reliability improvements​

  • Integrate router health-check failures with cooldown behavior and transient 429/408 handling - PR #25150, PR #24988
  • Add distributed lock for key rotation job execution - PR #25150, PR #23364, PR #23834
  • Improve team routing reliability with deterministic grouping, isolation fixes, stale alias controls, and order-based fallback - PR #25154, PR #25148
  • Regenerate GCP IAM token per async Redis cluster connection (fix token TTL failures) - PR #25155, PR #24426
  • Restore MCP server fields dropped by schema sync migration - PR #24078
  • Proxy server reliability hardening with bounded queue usage - PR #25155

Documentation Updates​

Infrastructure / Security Notes​

New Contributors​

Full Changelog: https://github.com/BerriAI/litellm/compare/v1.83.0-nightly...v1.83.3.rc.1

🚅
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.
Learn more →