Token Counting

Overview

LiteLLM provides exact token counting by calling provider-specific token counting APIs. This gives you accurate token counts before sending requests, helping with cost estimation and context window management.

Feature	Details
SDK Method	litellm.acount_tokens()
Proxy Endpoints	/v1/messages/count_tokens (Anthropic format), /v1/responses/input_tokens (OpenAI format)
Fallback	Local tiktoken-based counting for unsupported providers

Supported Providers

Provider	Token Counting API	Format
OpenAI	Responses API /input_tokens	OpenAI Responses
Anthropic	Messages /count_tokens	Anthropic Messages
Vertex AI (Claude)	Vertex AI Partner Models Token Counter	Anthropic Messages
Bedrock (Claude)	AWS Bedrock CountTokens API	Anthropic Messages
Gemini	Google AI Studio countTokens API	Anthropic Messages
Vertex AI (Gemini)	Vertex AI countTokens API	Anthropic Messages
Other providers	Local tiktoken fallback	N/A

SDK Usage

Basic Usage

import asyncio
import litellm

async def main():
    # OpenAI
    result = await litellm.acount_tokens(
        model="openai/gpt-5.6-terra",
        messages=[{"role": "user", "content": "Hello, how are you?"}],
    )
    print(f"Token count: {result.total_tokens}")
    print(f"Tokenizer: {result.tokenizer_type}")  # "openai_api"

    # Anthropic
    result = await litellm.acount_tokens(
        model="anthropic/claude-sonnet-5",
        messages=[{"role": "user", "content": "Hello, how are you?"}],
    )
    print(f"Token count: {result.total_tokens}")
    print(f"Tokenizer: {result.tokenizer_type}")  # "anthropic_api"

asyncio.run(main())

With Tools and System Message

import asyncio
import litellm

async def main():
    result = await litellm.acount_tokens(
        model="openai/gpt-5.6-terra",
        messages=[{"role": "user", "content": "What's the weather in Paris?"}],
        tools=[{
            "type": "function",
            "function": {
                "name": "get_weather",
                "description": "Get weather for a city",
                "parameters": {
                    "type": "object",
                    "properties": {"city": {"type": "string"}},
                },
            },
        }],
        system="You are a helpful weather assistant.",
    )
    print(f"Token count (with tools): {result.total_tokens}")

asyncio.run(main())

Response Format

litellm.acount_tokens() returns a TokenCountResponse:

TokenCountResponse(
    total_tokens=15,           # Token count
    request_model="openai/gpt-5.6-terra",  # Model requested
    model_used="gpt-5.6-terra",      # Model used for counting
    tokenizer_type="openai_api",    # "openai_api", "anthropic_api", "local_tokenizer"
    original_response={"input_tokens": 15},  # Raw API response
    error=False,               # True if counting failed
    error_message=None,        # Error details if failed
)

Fallback Behavior

If a provider doesn't support a token counting API, or if the API key is missing, acount_tokens() automatically falls back to local tiktoken-based counting:

# Unsupported provider → automatic fallback
result = await litellm.acount_tokens(
    model="together_ai/meta-llama/Llama-3-8b-chat-hf",
    messages=[{"role": "user", "content": "Hello"}],
)
print(result.tokenizer_type)  # "local_tokenizer"

On the proxy, local counting runs in a worker thread, so a large payload does not hold up other requests. Each worker process counts at most TOKEN_COUNTER_MAX_CONCURRENT_COUNTS payloads at a time (default 4) and queues the rest, which bounds the memory a burst of large counts can take. Strings longer than TOKEN_COUNTER_MAX_EXACT_CHARS characters (default 4,000,000, roughly a million tokens) are estimated by tokenizing 16 evenly spaced samples that together total that many characters and scaling the result by the string's length, which keeps the cost of the largest payloads bounded.

Proxy Usage

OpenAI Format: /v1/responses/input_tokens

curl

curl -X POST "http://localhost:4000/v1/responses/input_tokens" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "gpt-5.6-terra",
    "input": "Hello, how are you?"
  }'

Python (httpx)

import httpx

response = httpx.post(
    "http://localhost:4000/v1/responses/input_tokens",
    headers={
        "Content-Type": "application/json",
        "Authorization": "Bearer sk-<your-litellm-api-key>"
    },
    json={
        "model": "gpt-5.6-terra",
        "input": "Hello, how are you?"
    }
)

print(response.json())
# {"object": "response.input_tokens", "input_tokens": 13}

Response:

{"object": "response.input_tokens", "input_tokens": 13}

Anthropic Format: /v1/messages/count_tokens

See Anthropic Token Counting for full documentation.

curl -X POST "http://localhost:4000/v1/messages/count_tokens" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $LITELLM_API_KEY" \
  -d '{
    "model": "claude-sonnet-5",
    "messages": [
      {"role": "user", "content": "Hello, how are you?"}
    ]
  }'

Proxy Configuration

model_list:
  - model_name: gpt-5.6-terra
    litellm_params:
      model: openai/gpt-5.6-terra
      api_key: os.environ/OPENAI_API_KEY

  - model_name: claude-sonnet-5
    litellm_params:
      model: anthropic/claude-sonnet-5
      api_key: os.environ/ANTHROPIC_API_KEY