Skip to main content

Enterprise Quickstart

This guide gives the five steps to roll out LiteLLM Enterprise for all the teams in your company. Do the steps in sequence. Each step uses the deployment, the team, and the virtual key from the steps before it.

  1. Deploy LiteLLM with the critical deployment settings.
  2. Give access to providers, models, MCP tools, and agents.
  3. Configure the client endpoints.
  4. Implement secure authentication.
  5. Set up chargeback and allocate costs across projects and business units.
info

Step 1. Deploy LiteLLM​

Critical deployment settings​

Make these decisions before the first deployment. Each one is difficult to change after teams start to use the gateway.

  • Postgres. The Admin UI, virtual keys, the MCP and agent registries, and budget tracking keep their data in Postgres. Refer to Database sizing.
  • Redis. Run Redis 7.0 or newer when you run more than one instance or more than one worker. Without Redis, each instance enforces rate limits and budgets independently. Refer to What needs Redis.
  • Salt key. Set LITELLM_SALT_KEY before you add the first model. The salt key encrypts the stored credentials, and you must not change it after you add a model. Refer to Set the salt key.
  • Master key. Keep the master key in your secret manager. Applications must use virtual keys, not the master key.
  • Version. Pin an exact version or image digest, and verify the image signature. Stay on a supported version.
  • Topology. For more than one region, read Multi-region deployment before you deploy.

The full list is in Production best practices and Security best practices.

Prerequisites​

  • An API key for an LLM provider (OpenAI, Azure, Anthropic, or a different provider)
  • A Postgres database, or the Postgres database that the deployment method creates
  • Your Enterprise license key
  • A deployment target: Docker Compose, Kubernetes (kubectl), or Helm

Deploy the gateway​

Follow the Quickstart. Condensed steps:

curl -O https://raw.githubusercontent.com/BerriAI/litellm/main/docker-compose.yml

Create .env:

LITELLM_MASTER_KEY="sk-<paste-a-long-random-key>"
LITELLM_SALT_KEY="sk-salt-change-me"
LITELLM_LICENSE="eyJ..."
OPENAI_API_KEY="your-api-key"

Create config.yaml:

config.yaml
model_list:
  - model_name: gpt-5.6-terra
    litellm_params:
      model: openai/gpt-5.6-terra
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  callbacks: ["prometheus"]

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: "postgresql://llmproxy:dbpassword9090@db:5432/litellm"
  store_model_in_db: true
docker compose up

Verify Enterprise Edition​

Open http://localhost:4000/. The API docs page must show Enterprise Edition in the description. If it does not, refer to Activate your license.

Open the Admin UI at http://localhost:4000/ui and sign in with your master key. In Step 4, you replace this login with SSO.

Step 2. Give access to models, MCP tools, and agents​

In this step, you add providers, models, MCP servers, and agents to the gateway. Then you give one team access to them.

2a. Add providers and models​

The deployment in Step 1 adds gpt-5.6-terra in config.yaml. To add more models, use the Models page of the Admin UI or model_list in config.yaml. Use one source of truth for models. Refer to Model management and all the providers.

2b. Create an organization, a team, and a virtual key​

Do these steps in the Admin UI.

StepActionWhy
1Create an Organization and a TeamAn organization is the top-level entity, for example a business unit. An organization contains teams, for example a frontend team.
2Invite Internal Users to the teamEach user gets access to the team models, and the gateway tracks the spend of each user.
3Set the team max_budget (for example $10 for 30d)The budget gives a hard spend limit. In Step 5, you use it to examine budget enforcement.
4Create a team virtual key with access to the modelApplications use this key in Step 3. The gateway records the spend of the key and the team.

Refer to Multi-tenant architecture and Virtual keys.

2c. Add MCP tools​

  1. In the Admin UI, go to MCP Servers and select Add New MCP Server:

    • Name: deepwiki
    • URL: https://mcp.deepwiki.com/mcp
    • Transport: HTTP

    Or add the server to config.yaml:

mcp_servers:
  deepwiki:
    url: https://mcp.deepwiki.com/mcp
    transport: http
    available_on_public_internet: true
  1. In the MCP settings of the team or the virtual key, allow the deepwiki server. Refer to MCP permission management.
  2. Make sure that the tools show in the Admin UI under MCP Servers > MCP Tools.

2d. Add agents​

  1. Deploy a sample A2A agent, for example Multi-agent collaboration using A2A. This agent supports streaming.
  2. In the Admin UI, go to Agents and select Add Agent. Enter the name and the URL of the agent.
  3. In the agent settings of the virtual key, allow the agent. Refer to Agent permission management.

→ MCP overview · Agent Gateway overview

Step 3. Configure the client endpoints​

Applications and tools send requests to the gateway URL with the team virtual key. In this step, you send one LLM request, one MCP tool call, and one agent call. Then you connect your client tools.

3a. LLM requests​

  1. Send a request with the team virtual key:
curl -X POST 'http://localhost:4000/chat/completions' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer sk-team-key' \
  -d '{
    "model": "gpt-5.6-terra",
    "messages": [{"role": "user", "content": "Hello from LiteLLM Enterprise Gateway"}]
  }'
  1. Make sure that the response is 200 OK. The assistant text is in choices[0].message.content.
  2. Open the Logs page. Make sure that the log shows the key, the team, the model, the latency, and the spend.
  3. Open the Teams page and select your team. Make sure that the spend increased.

3b. MCP tool calls​

  1. Send a request that uses the deepwiki tools:
curl -X POST 'http://localhost:4000/v1/chat/completions' \
  -H 'Authorization: Bearer sk-team-key' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5.6-terra",
    "messages": [{"role": "user", "content": "TLDR of BerriAI/litellm repo"}],
    "tools": [{
      "type": "mcp",
      "server_url": "litellm_proxy/deepwiki",
      "server_label": "deepwiki",
      "require_approval": "never"
    }]
  }'
  1. Make sure that the response contains the tool output and a summary from the assistant.
  2. Open the Logs page. Make sure that the log shows the MCP tool call with the tool name and the cost.

Refer to Using your MCP.

3c. Agent calls​

  1. List the agents that the key can use:
curl -H 'Authorization: Bearer sk-team-key' \
  'http://localhost:4000/v1/agents'
  1. Call the agent with the A2A SDK:
invoke_a2a_agent.py
import httpx, asyncio
from uuid import uuid4
from a2a.client import A2ACardResolver, A2AClient
from a2a.types import MessageSendParams, SendMessageRequest

LITELLM_BASE_URL = "http://localhost:4000"
LITELLM_VIRTUAL_KEY = "sk-team-key"

async def main():
    headers = {"Authorization": f"Bearer {LITELLM_VIRTUAL_KEY}"}
    async with httpx.AsyncClient(headers=headers) as client:
        agents = (await client.get(f"{LITELLM_BASE_URL}/v1/agents")).json()
        agent_id = agents[0]["agent_id"]
        base_url = f"{LITELLM_BASE_URL}/a2a/{agent_id}"
        resolver = A2ACardResolver(httpx_client=client, base_url=base_url)
        a2a_client = A2AClient(
            httpx_client=client,
            agent_card=await resolver.get_agent_card(),
        )
        response = await a2a_client.send_message(
            SendMessageRequest(
                id=str(uuid4()),
                params=MessageSendParams(
                    message={
                        "role": "user",
                        "parts": [{"kind": "text", "text": "Hello, what can you do?"}],
                        "messageId": uuid4().hex,
                    }
                ),
            )
        )
        print(response.model_dump(mode="json", exclude_none=True, indent=2))

asyncio.run(main())
  1. Open the Logs page. Make sure that the log shows the key, the team, the latency, and the cost of the agent call.

Refer to Invoking A2A agents and Agent cost tracking.

3d. Connect your client tools​

Code tools and chat apps use the gateway URL and a virtual key or a sign-in token. Each client has a setup page, for example Claude Code, Codex, and Claude Desktop. Refer to Client setup.

Step 4. Implement secure authentication​

In this step, you replace the shared master key login with your identity provider. Then you limit what each person and each workload can do.

SSO for the Admin UI​

SSO controls the login to the Admin UI. API requests use a different method: virtual keys or JWT. Register this redirect URI in your identity provider:

https://<your-proxy-base-url>/sso/callback
GOOGLE_CLIENT_ID="<your-client-id>"
GOOGLE_CLIENT_SECRET="<your-client-secret>"
PROXY_BASE_URL="https://<your-proxy-base-url>"

Sign in to the Admin UI through your identity provider. Then disable the login with the master key and UI_PASSWORD. Refer to Disable environment credential login.

To create users and teams from your identity provider, use SCIM provisioning. For more options, refer to SSO for the Admin UI, SSO event hooks, and CLI SSO.

JWT authentication for API traffic​

Workloads can send JWTs from your OIDC provider in place of long-lived virtual keys. The JWT claims can map each request to a LiteLLM user, team, and model access. Refer to JWT authentication and JWT to virtual key mapping.

Least-privilege access​

Give each person the minimum RBAC role, and keep the number of proxy admins small. Give each production workload its own service account key, so that you can revoke one workload without an effect on the others. To limit the network sources, use IP address filtering.

Secret manager​

Keep the provider keys in your secret manager, not in configuration files. Refer to Secret managers.

Audit logs​

Audit logs are on by default when the gateway has a license. To turn them off, set store_audit_logs: false in litellm_settings. To do a test, delete a virtual key in the API or the Admin UI. Then open the Audit Logs page. Refer to Audit logs.

Step 5. Set up chargeback​

Chargeback assigns the cost of each request to the business unit or project that sent it. LiteLLM attributes spend at each level of the tenant hierarchy: organization, team, user, and virtual key. Projects and tags add more dimensions. Use organizations for business units and teams for the groups in them. Use projects for applications, and tags for costs that go across teams.

The same virtual key and the same budgets apply to LLM, MCP, and agent requests.

5a. Team budget​

You set the team max_budget in Step 2. After Step 3:

  1. Open the Teams page and select your team.
  2. Make sure that the spend includes the LLM, MCP, and agent calls.
  3. Optional: set the team max_budget to a very low value, for example $0.0001. Send one LLM request, and make sure that the gateway returns a budget error.

To send an email to a team before it reaches its budget, use soft budget alerts.

5b. Projects​

A project groups the keys of one application or use case. A project has a budget, owners, rate limits, a model allowlist, and its own spend view. Refer to Projects.

5c. Key budget and rate limits​

  1. Create a key with a small budget and an RPM limit:
curl -X POST 'http://localhost:4000/key/generate' \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{
    "max_budget": 0.01,
    "rpm_limit": 1,
    "team_id": "<your-team-id>"
  }'
  1. Send a request with the new key. Make sure that the response is 200 OK.
  2. Send a second request in the same minute. Make sure that the gateway returns a rate limit error.
  3. Open Virtual Keys in the Admin UI and find the spend of the key.

Refer to Virtual keys.

5d. Tag budget​

  1. Add tag_budget_config to config.yaml and restart the gateway:
litellm_settings:
  tag_budget_config:
    poc:chat-app:
      max_budget: 0.000000000001
      budget_duration: 1d
  1. Send a request with the tag:
curl -X POST 'http://localhost:4000/chat/completions' \
  -H 'Authorization: Bearer sk-team-key' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "gpt-5.6-terra",
    "messages": [{"role": "user", "content": "Hello"}],
    "metadata": {"tags": ["poc:chat-app"]}
  }'

  1. Make sure that the first request is successful. Send a second request with the same tag. Make sure that the gateway returns a budget error.

  2. Get the spend for each tag:

curl -X GET 'http://localhost:4000/spend/tags' \
  -H "Authorization: Bearer $LITELLM_API_KEY"

Make sure that the response shows poc:chat-app with total_spend and log_count.

Refer to Request tags and Tag budgets.

5e. Spend reports​

To send costs to finance, get spend reports by team, key, tag, or model with the API. Refer to Generate spend reports.

For temporary limits, use a temporary budget increase. For tiers of limits, use budget and rate limit tiers.

After the rollout​

  • Guardrails. Create a guardrail, attach it to a team or a key with a policy, and send a request that the guardrail must block. Refer to Guardrails quick start and Guardrail policies.
  • Team logging. Send the logs of each team to its own observability project. Refer to Team and key logging.
  • AI Hub. Show the models, MCP servers, and agents that your users can use on one page. Refer to AI Hub.
  • Security review. Collect the SOC 2 Type II report and the answers for your security team. Refer to Compliance and SOC 2 Type II.

Get help​

Each Enterprise license includes a dedicated Slack or Teams channel with the LiteLLM engineers. You can also send an email to support@berri.ai. Refer to Support and SLA.

LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails, built for production.
Learn more →