Arize AX

AI observability and evaluation for LLM applications, at arize.com.

Arize AX is the hosted platform. If you self-host the open-source tracer, or run it on Phoenix Cloud, see Arize Phoenix instead; the two use different credentials and endpoints.

info

We want to learn how we can make the callbacks better! Meet the LiteLLM founders or join our discord

Pre-Requisites

uv add litellm

Quick Start

SDK

import litellm
import os

os.environ["LITELLM_OTEL_V2"] = "true"
os.environ["ARIZE_SPACE_ID"] = ""
os.environ["ARIZE_API_KEY"] = ""
os.environ["ARIZE_PROJECT_NAME"] = ""   # recommended: names the project traces land in
# LLM API Keys
os.environ["OPENAI_API_KEY"] = ""

# set arize as a callback, litellm will send the data to arize
litellm.callbacks = ["arize"]

# openai call
response = litellm.completion(
  model="gpt-4o",
  messages=[
    {"role": "user", "content": "Hi 👋 - i'm openai"}
  ]
)

LiteLLM Proxy

1. Setup config.yaml

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  callbacks: ["arize"]

2. Set your credentials

LITELLM_OTEL_V2=true
ARIZE_SPACE_ID="your-space-id"
ARIZE_API_KEY="your-api-key"
ARIZE_PROJECT_NAME="your-project-name"   # recommended: names the project traces land in

3. Start LiteLLM Proxy

litellm --config /path/to/config.yaml

4. Test it!

curl -L -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer sk-1234' \
-d '{
  "model": "gpt-4o",
  "messages": [
    {
      "role": "user",
      "content": "Hey, how are you?"
    }
  ]
}'

What Arize renders

Open your Arize project; the trace appears under the project named by ARIZE_PROJECT_NAME. Each request shows up as a chat <model> span under the request root.

The openinference mapper stamps the OpenInference vocabulary onto the LLM-call span alongside the canonical gen_ai.* keys, so Arize reads its native schema without dropping the canonical ones. That covers llm.model_name and llm.provider, the llm.token_count.* usage split, llm.invocation_parameters, the llm.input_messages.* and llm.output_messages.* message arrays when content capture is on, and llm.tools.* for tool definitions. See the full attribute table or the OpenInference spec for the definitive vocabulary.

Configuration

Variable	Required	Notes
ARIZE_SPACE_ID	Yes	ARIZE_SPACE_KEY is the deprecated name and is still read for backward compatibility; prefer ARIZE_SPACE_ID in new configs
ARIZE_API_KEY	Yes
ARIZE_PROJECT_NAME	No	Names the project traces land in. Nothing enforces it and spans are not rejected without it, but set it so your traces are grouped where you expect
ARIZE_ENDPOINT	No	gRPC endpoint, defaults to https://otlp.arize.com/v1
ARIZE_HTTP_ENDPOINT	No	Use instead of ARIZE_ENDPOINT to export over HTTP

Setting ARIZE_ENDPOINT selects gRPC and ARIZE_HTTP_ENDPOINT selects HTTP; if neither is set, litellm uses the gRPC default.

Advanced

Send to Arize and another backend at once

Presets compose, so listing more than one sends the same traces to each destination in that tool's native format:

litellm_settings:
  callbacks: ["arize", "langfuse_otel"]

Per-team and per-key credentials

Arize supports per-request credentials, so different teams or keys can log to different Arize spaces without running separate proxies. Configure this from the Admin UI or the API as described in per-key / per-team destinations.

Full OpenTelemetry reference

This page covers the Arize-specific setup. For span attributes, prompt and response capture, metrics, distributed tracing, and which routes are traced, see the OpenTelemetry v2 guide.

Support & Talk to Founders

• Schedule Demo 👋
• Community Discord 💭
• Our emails ✉️ ishaan@berri.ai / krrish@berri.ai