---
title: "Getting Started"
url: "/docs/"
canonical_url: "https://docs.litellm.ai/docs/"
type: "docs"
last_updated: "2026-10-08"
summary: "LiteLLM is an open-source library that gives you a single, unified interface to call 100+ LLMs (OpenAI, Anthropic, Vertex AI, Bedrock, and more) using the OpenAI format."
related:
  - "/docs/agent_resources"
---
# Getting Started

> Index of all LiteLLM docs: https://docs.litellm.ai/llms.txt

**Quick Start: start the LiteLLM Gateway**

macOS and Linux:

```bash
curl -fsSL https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.sh | sh
```

Windows (PowerShell):

```powershell
irm https://raw.githubusercontent.com/BerriAI/litellm/main/scripts/quickstart.ps1 | iex
```

Python SDK:

```bash
uv add litellm
```

**LiteLLM** is an open-source library that gives you a single, unified interface to call 100+ LLMs (OpenAI, Anthropic, Vertex AI, Bedrock, and more) using the OpenAI format.

- Call any provider using the same `completion()` interface, with no API to re-learn for each one
- Consistent output format regardless of which provider or model you use
- Built-in retry / fallback logic across multiple deployments via the [Router](./routing.md)
- Self-hosted [LLM Gateway (Proxy)](/docs/simple_proxy) with virtual keys, cost tracking, and an admin UI

[![PyPI](https://img.shields.io/pypi/v/litellm.svg)](https://pypi.org/project/litellm/)
[![GitHub Stars](https://img.shields.io/github/stars/BerriAI/litellm?style=social)](https://github.com/BerriAI/litellm)

---

## Installation

```shell
uv add litellm
```

### A leaner SDK installation

For applications that use the Python SDK directly, `litellm-core` provides the shared SDK with fewer mandatory dependencies and without the bundled dashboard or gateway CLI entry points. Python imports remain unchanged: continue using `litellm`

Install `litellm-core` in a fresh environment instead of installing `litellm`. Add AWS and Python Hugging Face tokenizer packages when your application needs them. The two distributions cannot be installed together because they share the same Python files

See [LiteLLM Core](https://docs.litellm.ai/docs/litellm_core) for installation, package selection, and optional dependencies. Existing `litellm` installations retain their current dependency defaults

To deploy the full AI Gateway (Proxy) with the Admin UI, follow the [Quickstart](./proxy/docker_quick_start.md); it runs as a container and needs no Python setup. To run it from the CLI instead, see the [Gateway Quickstart](./learn/gateway_quickstart.md).

---

## Quick Start

Make your first LLM call using the provider of your choice:

**OpenAI**

```python
from litellm import completion
import os

os.environ["OPENAI_API_KEY"] = "your-api-key"

response = completion(
  model="openai/gpt-5.6-terra",
  messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)
```

**Anthropic**

```python
from litellm import completion
import os

os.environ["ANTHROPIC_API_KEY"] = "your-api-key"

response = completion(
  model="anthropic/claude-sonnet-5",
  messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)
```

**Vertex AI**

```python
from litellm import completion
import os

# auth: run 'gcloud auth application-default login'
os.environ["VERTEXAI_PROJECT"] = "your-project-id"
os.environ["VERTEXAI_LOCATION"] = "us-central1"

response = completion(
  model="vertex_ai/gemini-3.1-pro-preview",
  messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)
```

**Bedrock**

```python
from litellm import completion
import os

os.environ["AWS_ACCESS_KEY_ID"] = "your-key"
os.environ["AWS_SECRET_ACCESS_KEY"] = "your-secret"
os.environ["AWS_REGION_NAME"] = "us-east-1"

response = completion(
  model="bedrock/us.anthropic.claude-sonnet-5",
  messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)
```

**Ollama**

```python
from litellm import completion

response = completion(
  model="ollama/llama3",
  messages=[{"role": "user", "content": "Hello, how are you?"}],
  api_base="http://localhost:11434"
)
print(response.choices[0].message.content)
```

**Azure OpenAI**

```python
from litellm import completion
import os

os.environ["AZURE_API_KEY"] = "your-key"
os.environ["AZURE_API_BASE"] = "https://your-resource.openai.azure.com"
os.environ["AZURE_API_VERSION"] = "2024-02-01"

response = completion(
  model="azure/your-deployment-name",
  messages=[{"role": "user", "content": "Hello, how are you?"}]
)
print(response.choices[0].message.content)
```

Every response follows the OpenAI Chat Completions format, regardless of provider. ✅

### Response Format

Non-streaming responses return a `ModelResponse` object:

```json
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1677858242,
  "model": "gpt-5.6-terra",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello! I'm doing well, thanks for asking."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 13,
    "completion_tokens": 12,
    "total_tokens": 25
  }
}
```

Streaming responses (`stream=True`) yield `ModelResponseStream` chunks:

```json
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion.chunk",
  "created": 1677858242,
  "model": "gpt-5.6-terra",
  "choices": [
    {
      "index": 0,
      "delta": {
        "role": "assistant",
        "content": "Hello"
      },
      "finish_reason": null
    }
  ]
}
```

📖 [Full output format reference →](/docs/completion/output)

:::tip[Open in Colab]
<a target="_blank" href="https://colab.research.google.com/github/BerriAI/litellm/blob/main/cookbook/liteLLM_Getting_Started.ipynb">
[Image: Open In Colab]
</a>
:::

---

## New to LiteLLM?

**Want to get started fast?** Head to [Tutorials](/docs/tutorials) for step-by-step walkthroughs of AI coding tools, agent SDKs, proxy setup, and more.

**Need to understand a specific feature?** Check [Guides](/docs/guides) for streaming, function calling, prompt caching, and other how-tos.

---

## Choose Your Path

- [Python SDK](#litellm-python-sdk): Integrate LiteLLM directly into your Python application. Drop-in replacement for the OpenAI client.
- [Proxy Server (LLM Gateway)](#litellm-proxy-server-llm-gateway): Self-hosted gateway for platform teams managing LLM access across an organization.

---

## LiteLLM Python SDK

### Streaming

Add `stream=True` to receive chunks as they are generated:

```python
from litellm import completion
import os

os.environ["OPENAI_API_KEY"] = "your-api-key"

for chunk in completion(
  model="openai/gpt-5.6-terra",
  messages=[{"role": "user", "content": "Write a short poem"}],
  stream=True,
):
    print(chunk.choices[0].delta.content or "", end="")
```

### Exception Handling

LiteLLM maps every provider's errors to the OpenAI exception types, so your existing error handling keeps working:

```python
import litellm

try:
    litellm.completion(
      model="anthropic/claude-sonnet-5",
      messages=[{"role": "user", "content": "Hey!"}]
    )
except litellm.AuthenticationError as e:
    print(f"Bad API key: {e}")
except litellm.RateLimitError as e:
    print(f"Rate limited: {e}")
except litellm.APIError as e:
    print(f"API error: {e}")
```

### Logging & Observability

Send input/output to Langfuse, MLflow, Helicone, Lunary, and more with a single line:

```python
import litellm

litellm.success_callback = ["langfuse", "mlflow", "helicone"]

response = litellm.completion(
  model="gpt-5.6-terra",
  messages=[{"role": "user", "content": "Hi!"}]
)
```

📖 [See all observability integrations →](/docs/observability/opentelemetry_v2)

### Track Costs & Usage

Use a callback to capture cost per response:

```python
import litellm

def track_cost(kwargs, completion_response, start_time, end_time):
    print("Cost:", kwargs.get("response_cost", 0))

litellm.success_callback = [track_cost]

litellm.completion(
  model="gpt-5.6-terra",
  messages=[{"role": "user", "content": "Hello!"}],
  stream=True
)
```

📖 [Custom callback docs →](/docs/observability/custom_callback)

---

## LiteLLM Proxy Server (LLM Gateway)

The proxy is a self-hosted OpenAI-compatible gateway. Any client that works with OpenAI works with the proxy, with no code changes.

![LiteLLM Proxy Dashboard](https://github.com/BerriAI/litellm/assets/29436595/47c97d5e-b9be-4839-b28c-43d7f4f10033)

#### Step 1: Start the proxy

**LiteLLM CLI**

```shell
litellm --model huggingface/bigcode/starcoder
# Proxy running on http://0.0.0.0:4000
```

**Docker**

```yaml title="litellm_config.yaml"
model_list:
  - model_name: gpt-5.6-luna
    litellm_params:
      model: azure/your-deployment
      api_base: os.environ/AZURE_API_BASE
      api_key: os.environ/AZURE_API_KEY
      api_version: "2023-07-01-preview"
```

```shell
docker run \
  -v $(pwd)/litellm_config.yaml:/app/config.yaml \
  -e AZURE_API_KEY=your-key \
  -e AZURE_API_BASE=https://your-resource.openai.azure.com/ \
  -p 4000:4000 \
  docker.litellm.ai/berriai/litellm:latest \
  --config /app/config.yaml --detailed_debug
```

#### Step 2: Call it with the OpenAI client

```python
import openai

client = openai.OpenAI(api_key="anything", base_url="http://0.0.0.0:4000")

response = client.chat.completions.create(
  model="gpt-5.6-luna",
  messages=[{"role": "user", "content": "Write a short poem"}]
)
print(response.choices[0].message.content)
```

👉 [Full proxy quickstart →](/docs/proxy/docker_quick_start)

:::tip[Debugging tool]
Use **`/utils/transform_request`** to inspect exactly what LiteLLM sends to any provider. It helps when debugging prompt formatting, header issues, and provider-specific parameters.
:::

🔗 [Interactive API explorer (Swagger) →](https://docs.litellm.ai/api-reference/)

---

## Agent & MCP Gateway

LiteLLM is a unified gateway for **LLMs, agents, and MCP**, so you don't need a separate agent or MCP gateway. One endpoint for 100+ models, A2A agents, and MCP tools.

- [A2A Agents](https://docs.litellm.ai/docs/a2a): Add and invoke A2A agents via the LiteLLM gateway.
- [MCP Gateway](https://docs.litellm.ai/docs/mcp): Central MCP endpoint with per-key access control.
- [Enterprise Quickstart](https://docs.litellm.ai/docs/learn/enterprise_quickstart): Quickstart guide for trial customers: LLM, MCP, and Agent gateway.

---

## What to Explore Next

- [Routing & Load Balancing](https://docs.litellm.ai/docs/routing-load-balancing): Load balance across deployments and set automatic fallbacks.
- [Virtual Keys](https://docs.litellm.ai/docs/proxy/virtual_keys): Manage access, budgets, and rate limits per team or user.
- [Spend Tracking](https://docs.litellm.ai/docs/proxy/cost_tracking): Track costs per key, team, and user across all providers.
- [Guardrails](https://docs.litellm.ai/docs/proxy/guardrails/quick_start): Add content filtering, PII masking, and safety checks.
- [Observability](https://docs.litellm.ai/docs/observability/opentelemetry_v2): Integrate with Langfuse, MLflow, Helicone, and more.
- [Enterprise](https://docs.litellm.ai/docs/enterprise): SSO/SAML, audit logs, and advanced security for production.

## Related pages

- [Agent resources](https://docs.litellm.ai/docs/agent_resources.md)