Tool Search
Tool search enables Claude to dynamically discover and load tools on-demand from large tool catalogs (10,000+ tools). Instead of loading all tool definitions into the context window upfront, Claude searches your tool catalog and loads only the tools it needs.
Supported Providers
| Provider | Chat Completions API | Messages API | Responses API |
|---|---|---|---|
| Anthropic API | ✅ | ✅ | ✅ |
| Azure Anthropic (Microsoft Foundry) | ✅ | ✅ | ✅ |
| Google Cloud Vertex AI | ✅ | ✅ | ✅ |
| Amazon Bedrock | ✅ (Invoke API only, Claude 4.5 and newer) | ✅ (Invoke API only, Claude 4.5 and newer) | ✅ (Invoke API only, Claude 4.5 and newer) |
Benefits
- Context efficiency: Avoid consuming massive portions of your context window with tool definitions
- Better tool selection: Claude's tool selection accuracy degrades with more than 30-50 tools. Tool search maintains accuracy even with thousands of tools
- On-demand loading: Tools are only loaded when Claude needs them
Tool Search Variants
LiteLLM supports both tool search variants:
1. Regex Tool Search (tool_search_tool_regex_20251119)
Claude constructs regex patterns to search for tools. Best for exact pattern matching (faster).
2. BM25 Tool Search (tool_search_tool_bm25_20251119)
Claude uses natural language queries to search for tools using the BM25 algorithm. Best for natural language semantic search.
On Amazon Bedrock, both variants work through the Invoke API (bedrock/invoke/ models), not Converse. Before PR #52959, LiteLLM dropped the BM25 tool from Bedrock Chat Completions requests, so upgrade if Claude on Bedrock can't find your deferred tools with BM25.
Chat Completions API
SDK Usage
Basic Example with Regex Tool Search
import litellm
response = litellm.completion(
model="anthropic/claude-sonnet-5",
messages=[
{"role": "user", "content": "What is the weather in San Francisco?"}
],
tools=[
# Tool search tool (regex variant)
{
"type": "tool_search_tool_regex_20251119",
"name": "tool_search_tool_regex"
},
# Deferred tool - will be loaded on-demand
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the weather at a specific location",
"parameters": {
"type": "object",
"properties": {
"location": {"type": "string"},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
},
"defer_loading": True # Mark for deferred loading
}
]
)
print(response.choices[0].message.content)