DAY 0 Support: Gemini 3.5 Flash on LiteLLM

LiteLLM now supports gemini-3.5-flash with full day 0 support!

note

If you only want cost tracking, you need no change in your current LiteLLM version. But if you want support for new features introduced with this release, namely thinking levels, strict function-call IDs, and thought signatures, use v1.87.0-dev.1 or above.

Deploy this version

Docker

docker run litellm

docker run \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
ghcr.io/berriai/litellm:v1.87.0-dev.1

Pip

pip install litellm

pip install litellm==1.87.0.dev1

What's New

1. Minimal thinking level

Gemini 3.5 Flash supports the new "Minimal" level. LiteLLM maps OpenAI reasoning_effort to Gemini's thinkingLevel, so use reasoning_effort="minimal".

SDK

from litellm import completion

response = completion(
    model="gemini/gemini-3.5-flash",
    messages=[{"role": "user", "content": "What's 2+2?"}],
    reasoning_effort="minimal",
)

print(response.choices[0].message.content)

PROXY

curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR-LITELLM-KEY>" \
  -d '{
    "model": "gemini-3.5-flash",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "reasoning_effort": "minimal"
  }'

reasoning_effort	thinkingLevel
minimal	minimal

2. Strict function calling

Gemini 3.5+ requires every functionResponse to include the same id as the originating functionCall, plus the matching function name. LiteLLM round-trips this through standard OpenAI fields: tool_calls[].id on the assistant message, and the same value as tool_call_id on the tool result.

How the tool-call loop works

Step 1 : User submits a query that would trigger a tool call

Send the user message and your tool definitions. The model responds with tool_calls; save the id from the first tool call (it may look like 5x450f94__thought__<signature>; pass it back unchanged on the next request).

curl -sS http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR-LITELLM-KEY>" \
  -d '{
    "model": "gemini-3.5-flash",
    "messages": [
      {
        "role": "user",
        "content": "What is the weather in Tokyo right now?"
      }
    ],
    "tools": [
      {
        "type": "function",
        "function": {
          "name": "get_weather",
          "description": "Get current weather for a city",
          "parameters": {
            "type": "object",
            "properties": {
              "city": { "type": "string" }
            },
            "required": ["city"]
          }
        }
      }
    ]
  }' | tee /tmp/gemini_tool_step1.json | jq .

Copy the tool call id from the response:

TOOL_CALL_ID=$(jq -r '.choices[0].message.tool_calls[0].id' /tmp/gemini_tool_step1.json)
echo "$TOOL_CALL_ID"
# e.g. 5x450f94__thought__EvACCu0CAQw51sdR...

Step 2 : Run your tool, then send the result with the same tool_call_id

Run get_weather locally, then call the proxy again with the full message history. Set tool_call_id to the exact id from Step 1; LiteLLM uses it as the Gemini functionResponse.id.

# Result from your local get_weather("Tokyo") call
WEATHER_RESULT='{"temp_c": 18, "condition": "clear"}'

curl -sS http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR-LITELLM-KEY>" \
  -d "$(jq -n \
    --arg id "$TOOL_CALL_ID" \
    --arg content "$WEATHER_RESULT" \
    '{
      model: "gemini-3.5-flash",
      messages: [
        {role: "user", content: "What is the weather in Tokyo right now?"},
        {
          role: "assistant",
          content: null,
          tool_calls: [{
            id: $id,
            type: "function",
            function: {name: "get_weather", arguments: "{\"city\": \"Tokyo\"}"}
          }]
        },
        {role: "tool", tool_call_id: $id, content: $content}
      ],
      tools: [{
        type: "function",
        function: {
          name: "get_weather",
          description: "Get current weather for a city",
          parameters: {
            type: "object",
            properties: {city: {type: "string"}},
            required: ["city"]
          }
        }
      }]
    }')" | jq .

The id on the assistant tool_calls entry and the tool_call_id on the role: tool message must match. The function name must match the tool definition (get_weather).

Step 3 : Model produces the final answer

LiteLLM sends the matching id and name on the Gemini functionResponse part. The model then returns a normal assistant message with the weather summary.

3. Sampling parameters (temperature, top_p, top_k)

Google has advised moving away from temperature, top_p, and top_k for Gemini 3.5+ and steering sampling behavior through system instructions instead. These parameters still work today, but may be removed in a future API release.

LiteLLM follows the same guidance: when you pass temperature, top_p, or top_k on Gemini 3+ models, you will see a deprecation warning in the logs recommending system-instruction-based sampling instead.

Quick Start

SDK

from litellm import completion

response = completion(
    model="gemini/gemini-3.5-flash",
    messages=[{"role": "user", "content": "Summarize this article in 3 bullet points."}],
)

print(response.choices[0].message.content)

PROXY

1. Setup config.yaml

model_list:
  - model_name: gemini-3.5-flash
    litellm_params:
      model: gemini/gemini-3.5-flash
      api_key: os.environ/GEMINI_API_KEY

  # Or use Vertex AI
  - model_name: vertex-gemini-3.5-flash
    litellm_params:
      model: vertex_ai/gemini-3.5-flash
      vertex_project: your-project-id
      vertex_location: us-central1

2. Start proxy

litellm --config /path/to/config.yaml

3. Make requests

curl -X POST http://localhost:4000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <YOUR-LITELLM-KEY>" \
  -d '{
    "model": "gemini-3.5-flash",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Supported Endpoints

LiteLLM provides full end-to-end support for Gemini 3.5 Flash on:

• ✅ /v1/chat/completions - OpenAI-compatible chat completions endpoint
• ✅ /v1/responses - OpenAI Responses API endpoint (streaming and non-streaming)
• ✅ /v1/messages - Anthropic-compatible messages endpoint
• ✅ /v1/generateContent – Google Gemini API compatible endpoint

All endpoints support:

• Streaming and non-streaming responses
• Function calling with thought signatures
• Multi-turn conversations
• All Gemini 3-specific features (thinking levels, thought signatures)
• Full multimodal support (text, image, audio, video)