Skip to main content

/v1/messages

Use LiteLLM to call all your LLM APIs in the Anthropic v1/messages format.

Overview​

FeatureSupportedNotes
Cost Tracking✅Works with all supported models
Logging✅Works across all integrations
End-user Tracking✅
Streaming✅
Fallbacks✅Works between supported models
Loadbalancing✅Works between supported models
Guardrails✅Applies to input and output text. On streamed output a guardrail can block the response, but text rewrites such as masking only apply to non-streaming responses
Supported ProvidersAll LiteLLM supported providersopenai, anthropic, bedrock, vertex_ai, gemini, azure, azure_ai, etc.

Usage​

LiteLLM Python SDK​

Non-streaming example​

Anthropic Example using LiteLLM Python SDK
import litellm
response = await litellm.anthropic.messages.acreate(
    messages=[{"role": "user", "content": "Hello, can you tell me a short joke?"}],
    api_key=api_key,
    model="anthropic/claude-sonnet-5",
    max_tokens=100,
)

Streaming example​

Anthropic Streaming Example using LiteLLM Python SDK
import litellm
response = await litellm.anthropic.messages.acreate(
    messages=[{"role": "user", "content": "Hello, can you tell me a short joke?"}],
    api_key=api_key,
    model="anthropic/claude-sonnet-5",
    max_tokens=100,
    stream=True,
)
async for chunk in response:
    print(chunk)

Example response:

{
  "content": [
    {
      "text": "Hi! this is a very short joke",
      "type": "text"
    }
  ],
  "id": "msg_013Zva2CMHLNnXjNJJKqJ2EF",
  "model": "claude-sonnet-5",
  "role": "assistant",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "type": "message",
  "usage": {
    "input_tokens": 2095,
    "output_tokens": 503,
    "cache_creation_input_tokens": 2095,
    "cache_read_input_tokens": 0
  }
}

LiteLLM Proxy Server​

  1. Setup config.yaml
model_list:
    - model_name: anthropic-claude
      litellm_params:
        model: claude-sonnet-5
        api_key: os.environ/ANTHROPIC_API_KEY
  1. Start proxy
litellm --config /path/to/config.yaml
  1. Test it!
Anthropic Example using LiteLLM Proxy Server
import anthropic

# point anthropic sdk to litellm proxy 
client = anthropic.Anthropic(
    base_url="http://0.0.0.0:4000",
    api_key="sk-<your-litellm-api-key>",
)

response = client.messages.create(
    messages=[{"role": "user", "content": "Hello, can you tell me a short joke?"}],
    model="anthropic-claude",
    max_tokens=100,
)

Message threads (beta)​

Anthropic message threads keep the conversation on Anthropic's servers between turns. The first request creates the thread with the full prompt, and each later request sends only the new messages plus the id of the previous response, so the model starts answering sooner and you stop paying to resend history. Since v1.111.0

LiteLLM forwards the thread body field on native Anthropic deployments (anthropic/, Microsoft Foundry azure_ai/ Claude deployments, and Claude Platform on AWS bedrock/claude_platform/ deployments), adds the message-threads-2026-08-12 beta header for you, and returns the response id unchanged, so the id a client gets from the proxy is the one it continues from. Streaming works the same way.

Create the thread with the full prompt:

curl -X POST http://0.0.0.0:4000/v1/messages \
  -H "x-api-key: $LITELLM_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5-5",
    "max_tokens": 1024,
    "system": "You are a terse assistant.",
    "messages": [{"role": "user", "content": "My name is Ada."}],
    "thread": {"type": "create"}
  }'

Continue it with only the new messages and the id of the response you just got. A continue request must not carry system or tools, since the thread already stores them:

curl -X POST http://0.0.0.0:4000/v1/messages \
  -H "x-api-key: $LITELLM_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "model": "anthropic/claude-sonnet-5-5",
    "max_tokens": 1024,
    "messages": [{"role": "user", "content": "What is my name?"}],
    "thread": {"type": "continue", "previous_message_id": "msg_01XFDUDYJgAACzvnptvVoYEL"}
  }'

The Anthropic SDK thread helpers (client.beta.messages.threads in the Python and TypeScript SDKs) work through the proxy as they do against Anthropic directly. When Anthropic cannot continue a thread it answers a 404 with error.details.error_code set to thread_not_found, and the proxy relays that status and code, so the helper's replay of the full conversation keeps working.

On a deployment without native thread support, which is every translated provider such as openai/, Bedrock and Vertex AI, and any deployment whose model_info.supported_endpoints forwards /v1/messages to an OpenAI-compatible server, a create request gets a 400 with error.details.error_code set to thread_unsupported_request before any upstream call, the same answer Anthropic gives a request it cannot thread. The SDK thread helpers created with unsupported_request_behavior="continue_unthreaded" then send the conversation without a thread. With drop_params on, LiteLLM instead serves the create without the thread and logs a warning:

litellm_settings:
  drop_params: true

A continue request on such a deployment always gets the 404 thread_not_found before any upstream call, so the SDK thread helpers replay the full conversation instead of getting an answer that ignores the earlier turns.

Every key on the proxy shares the deployment's Anthropic credential, so LiteLLM binds each thread response id to the virtual key that received it, or to the signed-in user, team, end user, org and agent when the request has no key. A continue from another key gets the 404 thread_not_found on every route, and the SDK helper starts a new thread with the full conversation. On /v1/messages, /v1/chat/completions and /v1/responses, a continue naming an id this proxy never returned in a thread gets the same 404, so a thread started outside the proxy or on a pass-through route cannot be continued there. Only threads started on /v1/messages are bound, so pass-through routes such as /anthropic/v1/messages, which forward requests with the deployment's credential, forward an id the proxy never bound as it is. The first key to receive an id keeps it, so a response cache hit that returns the same id to another key does not hand the thread over. A binding lasts one hour, longer than Anthropic keeps an idle thread.

With several proxy workers or pods, configure Redis for the proxy so every worker sees the bindings. Without it, a continue that reaches another worker gets the 404 and the SDK helper replays the conversation. Setting the Redis environment variables is enough:

export REDIS_HOST=redis.internal
export REDIS_PORT=6379
export REDIS_PASSWORD=<your-redis-password>
litellm --config config.yaml --num_workers 4

A model group with several deployments can send a continue to a deployment whose Anthropic credential cannot see the thread, which answers the 404 thread_not_found and makes the SDK helper replay the conversation. Turn on deployment affinity so each key keeps reaching the deployment that created its threads:

router_settings:
  optional_pre_call_checks: ["deployment_affinity"]

The set of anthropic-beta values is one of the fixed fields of a thread, and Anthropic rejects a turn whose set differs from the create request with thread_fingerprint_mismatch. A create request still gets the beta headers LiteLLM adds based on its tools and messages, such as the advisor tool's. The proxy stores the full set the create sent with the thread binding and adds it to every continue on /v1/messages, /v1/chat/completions and /v1/responses, so a continue matches its create without you sending those headers. When you call the LiteLLM Python SDK directly, with no proxy, nothing stores that set, so send the create's beta headers yourself on every continue.

Request Format​

Request body will be in the Anthropic messages API format. litellm follows the Anthropic messages specification for this endpoint.

Example request body​

{
  "model": "claude-sonnet-5",
  "max_tokens": 1024,
  "messages": [
    {
      "role": "user",
      "content": "Hello, world"
    }
  ]
}

Required Fields​

  • model (string):
    The model identifier (e.g., "claude-sonnet-5").
  • max_tokens (integer):
    The maximum number of tokens to generate before stopping.
    Note: The model may stop before reaching this limit; value must be greater than 1.
  • messages (array of objects):
    An ordered list of conversational turns.
    Each message object must include:
    • role (enum: "user" or "assistant"):
      Specifies the speaker of the message.
    • content (string or array of content blocks):
      The text or content blocks (e.g., an array containing objects with a type such as "text") that form the message.
      Example equivalence: 
      {"role": "user", "content": "Hello, Claude"}
      is equivalent to: 
      {"role": "user", "content": [{"type": "text", "text": "Hello, Claude"}]}

Optional Fields​

  • metadata (object):
    Contains additional metadata about the request (e.g., user_id as an opaque identifier).
  • stop_sequences (array of strings):
    Custom sequences that, when encountered in the generated text, cause the model to stop.
  • stream (boolean):
    Indicates whether to stream the response using server-sent events.
  • system (string or array):
    A system prompt providing context or specific instructions to the model.
  • temperature (number):
    Controls randomness in the model's responses. Valid range: 0 < temperature < 1.
  • thinking (object): Configuration for enabling extended thinking. If enabled, it includes:
    • budget_tokens (integer): Minimum of 1024 tokens (and less than max_tokens).
    • type (enum): E.g., "enabled".
    • summary (string, optional): Enables the summary style for thinking blocks. Possible values: "auto", "concise", "detailed", "disabled". When routing to non-Anthropic providers (e.g., openai/gpt-6.1-sol), the summary value is preserved and forwarded to the downstream API.
  • tool_choice (object):
    Instructs how the model should use any provided tools.
  • tools (array of objects):
    Definitions for tools available to the model. Each tool includes:
    • name (string):
      The tool's name.
    • description (string):
      A detailed description of the tool.
    • input_schema (object):
      A JSON schema describing the expected input format for the tool.
  • top_k (integer):
    Limits sampling to the top K options.
  • top_p (number):
    Enables nucleus sampling with a cumulative probability cutoff. Valid range: 0 < top_p < 1.

Response Format​

Responses will be in the Anthropic messages API format.

Example Response​

{
  "content": [
    {
      "text": "Hi! My name is Claude.",
      "type": "text"
    }
  ],
  "id": "msg_013Zva2CMHLNnXjNJJKqJ2EF",
  "model": "claude-sonnet-5",
  "role": "assistant",
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "type": "message",
  "usage": {
    "input_tokens": 2095,
    "output_tokens": 503,
    "cache_creation_input_tokens": 2095,
    "cache_read_input_tokens": 0
  }
}

Response fields​

  • content (array of objects):
    Contains the generated content blocks from the model. Each block includes:

    • type (string):
      Indicates the type of content (e.g., "text", "tool_use", "thinking", or "redacted_thinking").
    • text (string):
      The generated text from the model.
      Note: Maximum length is 5,000,000 characters.
    • citations (array of objects or null):
      Optional field providing citation details. Each citation includes:
      • cited_text (string):
        The excerpt being cited.
      • document_index (integer):
        An index referencing the cited document.
      • document_title (string or null):
        The title of the cited document.
      • start_char_index (integer):
        The starting character index for the citation.
      • end_char_index (integer):
        The ending character index for the citation.
      • type (string):
        Typically "char_location".

  • id (string):
    A unique identifier for the response message.
    Note: The format and length of IDs may change over time.

  • model (string):
    Specifies the model that generated the response.

  • role (string):
    Indicates the role of the generated message. For responses, this is always "assistant".

  • stop_reason (string):
    Explains why the model stopped generating text. Possible values include:

    • "end_turn": The model reached a natural stopping point.
    • "max_tokens": The generation stopped because the maximum token limit was reached.
    • "stop_sequence": A custom stop sequence was encountered.
    • "tool_use": The model invoked one or more tools.

  • stop_sequence (string or null):
    Contains the specific stop sequence that caused the generation to halt, if applicable; otherwise, it is null.

  • type (string):
    Denotes the type of response object, which is always "message".

  • usage (object):
    Provides details on token usage for billing and rate limiting. This includes:

    • input_tokens (integer):
      Total number of input tokens processed.
    • output_tokens (integer):
      Total number of output tokens generated.
    • cache_creation_input_tokens (integer or null):
      Number of tokens used to create a cache entry.
    • cache_read_input_tokens (integer or null):
      Number of tokens read from the cache.
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails, built for production.
Learn more →