Skip to main content

/vector_stores/search - Search Vector Store

Search a vector store for relevant chunks based on a query and file attributes filter. This is useful for retrieval-augmented generation (RAG) use cases.

Overview​

FeatureSupportedNotes
Cost Tracking✅Tracked per search operation
Logging✅Works across all integrations
End-user Tracking✅
Support LLM ProvidersOpenAI, Azure OpenAI, Bedrock, Vertex RAG Engine, Azure AI, Milvus, Valkey, Amazon OpenSearch, GeminiFull vector stores API support across providers

For retrieve, list, update, and delete over HTTP (including custom_llm_provider / model routing), see Create vector store.

Usage​

LiteLLM Python SDK​

Non-streaming example​

Search Vector Store - Basic
import litellm

response = await litellm.vector_stores.asearch(
vector_store_id="vs_abc123",
query="What is the capital of France?"
)
print(response)

Synchronous example​

Search Vector Store - Sync
import litellm

response = litellm.vector_stores.search(
vector_store_id="vs_abc123",
query="What is the capital of France?"
)
print(response)

LiteLLM Proxy Server​

  1. Setup config.yaml
model_list:
- model_name: gpt-6.1-sol
litellm_params:
model: openai/gpt-6.1-sol
api_key: os.environ/OPENAI_API_KEY

general_settings:
# Vector store settings can be added here if needed
  1. Start proxy
litellm --config /path/to/config.yaml
  1. Test it with OpenAI SDK!
OpenAI SDK via LiteLLM Proxy
from openai import OpenAI

# Point OpenAI SDK to LiteLLM proxy
client = OpenAI(
base_url="http://0.0.0.0:4000",
api_key="sk-<your-litellm-api-key>", # Your LiteLLM API key
)

search_results = client.beta.vector_stores.search(
vector_store_id="vs_abc123",
query="What is the capital of France?",
max_num_results=5
)
print(search_results)

Fallbacks​

The proxy supports YAML-configured vector store fallbacks for POST /v1/vector_stores/{id}/search, with cooldown and serving-store response headers. The primary and backups must all be registered in config; API/UI-created database stores do not support fallback routing yet.

Setting Up Vector Stores​

To search a store that already exists on a provider, register it with LiteLLM first via config.yaml, POST /vector_store/new, or the Admin UI; see Managed Vector Stores. For provider-specific configuration, see the Vector Store Configuration Guide:

  • Provider-specific configuration (Bedrock, OpenAI, Azure, Vertex AI, PG Vector)
  • Python SDK and Proxy setup examples
  • Authentication and credential management

Using Vector Stores with Chat Completions​

Pass vector_store_ids in chat completion requests to automatically retrieve relevant context. See Using Vector Stores with Chat Completions for implementation details.

LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails, built for production.
Learn more →