Features

Features are what LiteLLM does with your requests across providers: load balancing and fallbacks, streaming, tool calling, media input, prompt caching, and more. Most pages show the code for the Python SDK and the setup for the AI Gateway. A feature that applies to one product only is in the section for that product.

New to LiteLLM? Start with the AI Gateway Quickstart or the Python SDK Quickstart.

Routing & Reliability

Load balancing, fallbacks, client-side fallbacks, timeouts, and routing rules.

Requests & Responses

Streaming, batching, structured outputs, and reasoning.

Tool Calling

Function calling, web search and fetch, computer use, code interpreter, and knowledge bases.

Multimodal

Vision, audio, PDF input, and image generation in chat.

Prompts & Context

Prompt caching, assistant prefill, and predicted outputs.

Compatibility

Provider-specific params, unsupported params, and message sanitization.

Other pages that were here

Retries and fallbacks, mock responses, message trimming, model aliases, and prompt formatting are in the Python SDK section. SSL and HTTP proxy settings are in the AI Gateway configuration pages, and custom formats are with the custom plugins. Fine-tuned models are with the providers, and Veo video generation is with the video endpoints.