--- title: "MCP Deployment Guide" url: "/docs/mcp_deployment" canonical_url: "https://docs.litellm.ai/docs/mcp_deployment" type: "docs" last_updated: "2026-10-02" summary: "How to deploy LiteLLM as a central gateway for LLMs, MCP servers, and agents." related: - "/docs/mcp_client_allowlist" - "/docs/mcp_semantic_filter" --- # MCP Deployment Guide > Index of all LiteLLM docs: https://docs.litellm.ai/llms.txt How to deploy LiteLLM as a central gateway for LLMs, MCP servers, and agents. --- ## The core idea LiteLLM is a single control plane for three resource types: | Resource | Registered as | |----------|--------------| | **LLM** | `model_list` in config or via API | | **MCP Server** | `mcp_servers` in config or via UI | | **Agent** | A2A routes | All three share the same auth (LiteLLM API key), rate limiting, and usage dashboard, giving you a central catalog without separate registries. --- ## Deployment topologies ### Option A: Single gateway (recommended) One LiteLLM instance handles LLM routing, MCP tool calls, and A2A agent invocations. ``` Agents / AI clients │ ▼ ┌───────────────────────────────────┐ │ LiteLLM Gateway │ │ /v1/chat/completions (LLMs) │ │ /mcp (tools) │ │ /a2a (agents) │ └───────┬───────┬──────────┬────────┘ │ │ │ OpenAI MCP servers Downstream Bedrock (internal) agents Azure (public) ``` One service, one config, one set of API keys. Use the [public internet filter](./mcp_public_internet.md) to control which MCP servers are visible to external callers (Claude Desktop, ChatGPT) vs. internal-only. ```yaml title="config.yaml" showLineNumbers general_settings: master_key: os.environ/LITELLM_MASTER_KEY store_model_in_db: true mcp_internal_ip_ranges: - "10.0.0.0/8" - "172.16.0.0/12" - "192.168.0.0/16" - "100.64.0.0/10" # VPN/Tailscale range model_list: - model_name: gpt-5.6-terra litellm_params: model: openai/gpt-5.6-terra api_key: os.environ/OPENAI_API_KEY mcp_servers: internal_db: url: http://db-mcp.internal:8000/mcp transport: http available_on_public_internet: false # internal callers only web_search: url: https://mcp.exa.ai/mcp transport: http available_on_public_internet: true # visible to ChatGPT / Claude Desktop (the default) ``` Server names may not contain a hyphen; the proxy rejects `mcp_servers` keys like `internal-db` at startup. Use underscores instead. --- ### Option B: Separate LLM gateway and MCP gateway Split into two LiteLLM deployments: one for LLM routing (no internet exposure), one for MCP serving (optionally internet-facing). ``` Internal AI clients External AI clients │ (ChatGPT, Claude Desktop) │ │ ▼ ▼ ┌────────────────────┐ ┌────────────────────────┐ │ LLM Gateway │ │ MCP Gateway │ │ (no public port) │ │ (port 443 / public) │ │ /v1/chat/... │ │ /mcp │ └────────┬───────────┘ └──────────┬─────────────┘ │ │ LLM providers MCP servers (OpenAI, Bedrock, …) (internal + public) ``` LLM API keys stay behind the firewall. A compromise of the MCP gateway does not expose them. Use this when external MCP access is needed but LLM credentials must stay fully private. --- ## Central catalog LiteLLM exposes all resource types through standard endpoints: | Endpoint | Returns | |----------|---------| | `GET /v1/models` | All registered LLMs | | `GET /v1/mcp/server` | All MCP servers | | `GET /mcp` | All MCP tools (across all servers) | | `GET /a2a/{agent_id}/.well-known/agent-card.json` | A2A agent card | **MCP registry** (opt-in): expose a discovery endpoint for Claude Desktop / Cursor: ```yaml title="config.yaml" general_settings: enable_mcp_registry: true ``` ```json title="Claude Desktop config" { "mcpServers": { "litellm": { "url": "https://your-litellm.example.com/mcp", "headers": { "Authorization": "Bearer sk-..." } } } } ``` --- ## Security considerations ### The open-port problem If you expose LiteLLM's port to the internet (for Claude Desktop / ChatGPT), `/v1/chat/completions` is also reachable externally. LLM credentials stay protected by key auth, but be deliberate about this. **Mitigations:** 1. **Separate deployments** (Option B): the LLM gateway never gets a public port 2. **Firewall** — block `/v1/chat/completions` from public IPs at the network layer 3. **Short-lived scoped keys** — limit blast radius if a key leaks ### MCP servers can reach the public internet When you register an external MCP URL (e.g. `https://mcp.exa.ai/mcp`), LiteLLM makes outbound requests to it on every tool call. Check that your network policy allows it and that your security team is comfortable with data leaving the perimeter. For air-gapped networks: only register MCP servers inside your perimeter and set `available_on_public_internet: false` on each of them. The default is `true`, so a server is visible to external callers unless you explicitly set it to `false`. ### Access controls By default all authenticated callers can call all MCP tools. Use these to restrict: | Control | Where | |---------|-------| | Per-key tool access | [Key-level MCP permissions](./mcp_control.md) | | Per-team tool access | [Team-level MCP permissions](./mcp_control.md) | | Hide internal servers from external callers | [available_on_public_internet](./mcp_public_internet.md) | | Verify requests came through LiteLLM | [MCP Zero Trust (JWT)](./mcp_zero_trust.md) | | Block sensitive data in responses | [MCP Guardrails](./mcp_guardrail.md) | --- ## Related - [MCP Overview](./mcp.md) - [Public Internet Filter](./mcp_public_internet.md) - [MCP Access Control](./mcp_control.md) - [MCP Zero Trust](./mcp_zero_trust.md) - [MCP Guardrails](./mcp_guardrail.md) ## Related pages - [Allowlisting MCP Client Applications](https://docs.litellm.ai/docs/mcp_client_allowlist.md) - [MCP Semantic Tool Filter](https://docs.litellm.ai/docs/mcp_semantic_filter.md)