Skip to main content

Google GenAI SDK with LiteLLM

Use Google's official GenAI SDK (JavaScript/TypeScript and Python) with any LLM provider through LiteLLM Proxy.

The Google GenAI SDK (@google/genai for JS, google-genai for Python) provides a native interface for calling Gemini models. By pointing it to LiteLLM, you can use the same SDK with OpenAI, Anthropic, Bedrock, Azure, Vertex AI, or any other provider while keeping the native Gemini request/response format.

Why Use LiteLLM with Google GenAI SDK?​

Developer Benefits:

  • Universal Model Access: Use any LiteLLM-supported model (Anthropic, OpenAI, Vertex AI, Bedrock, etc.) through the Google GenAI SDK interface
  • Higher Rate Limits & Reliability: Load balance across multiple models and providers to avoid hitting individual provider limits, with fallbacks to ensure you get responses even if one provider fails

Proxy Admin Benefits:

  • Centralized Management: Control access to all models through a single LiteLLM proxy instance without giving developers API keys to each provider
  • Budget Controls: Set spending limits and track costs across all SDK usage
  • Logging & Observability: Track all requests with cost tracking, logging, and analytics
FeatureSupportedNotes
Cost Tracking✅All models on /generateContent endpoint
Logging✅Works across all integrations
Streaming✅streamGenerateContent supported
Virtual Keys✅Use LiteLLM keys instead of Google keys
Load Balancing✅Via native router endpoints
Fallbacks✅Via native router endpoints

Quick Start​

1. Install the SDK​

npm install @google/genai

2. Start LiteLLM Proxy​

config.yaml
model_list:
  - model_name: gemini-2.5-flash
    litellm_params:
      model: gemini/gemini-2.5-flash
      api_key: os.environ/GEMINI_API_KEY
litellm --config config.yaml

3. Call the SDK through LiteLLM​

index.js
const { GoogleGenAI } = require("@google/genai");

const ai = new GoogleGenAI({
  apiKey: "sk-1234",  // LiteLLM virtual key (not a Google key)
  httpOptions: {
    baseUrl: "http://localhost:4000/gemini",  // LiteLLM proxy URL
  },
});

async function main() {
  const response = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: "Explain how AI works",
  });
  console.log(response.text);
}

main();

Streaming​

streaming.js
const { GoogleGenAI } = require("@google/genai");

const ai = new GoogleGenAI({
  apiKey: "sk-1234",
  httpOptions: {
    baseUrl: "http://localhost:4000/gemini",
  },
});

async function main() {
  const response = await ai.models.generateContentStream({
    model: "gemini-2.5-flash",
    contents: "Write a short poem about the ocean",
  });

  for await (const chunk of response) {
    process.stdout.write(chunk.text);
  }
}

main();

Multi-turn Chat​

chat.js
const { GoogleGenAI } = require("@google/genai");

const ai = new GoogleGenAI({
  apiKey: "sk-1234",
  httpOptions: {
    baseUrl: "http://localhost:4000/gemini",
  },
});

async function main() {
  const chat = ai.chats.create({
    model: "gemini-2.5-flash",
  });

  const response1 = await chat.sendMessage({ message: "I have 2 dogs and 3 cats." });
  console.log(response1.text);

  const response2 = await chat.sendMessage({ message: "How many pets is that in total?" });
  console.log(response2.text);
}

main();

Advanced: Use Any Model with the GenAI SDK​

By default, the GenAI SDK talks to Gemini models. But with LiteLLM's router, you can route GenAI SDK requests to any provider: Anthropic, OpenAI, Bedrock, etc.

This works by using model_group_alias to map Gemini model names to your desired provider models. LiteLLM handles the format translation internally.

info

For this to work, point the SDK baseUrl to http://localhost:4000 (without /gemini). This routes requests through LiteLLM's native Google endpoints, which go through the router and support model aliasing.

Route gemini-2.5-flash requests to Claude Sonnet:

config.yaml
model_list:
  - model_name: claude-sonnet
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY

router_settings:
  model_group_alias: {"gemini-2.5-flash": "claude-sonnet"}

Then use the SDK with baseUrl pointing to LiteLLM (without /gemini):

any_model.js
const { GoogleGenAI } = require("@google/genai");

const ai = new GoogleGenAI({
  apiKey: "sk-1234",
  httpOptions: {
    baseUrl: "http://localhost:4000",  // No /gemini — goes through the router
  },
});

async function main() {
  // This calls Claude/GPT-4o/Bedrock under the hood via model_group_alias
  const response = await ai.models.generateContent({
    model: "gemini-2.5-flash",
    contents: "Hello from any model!",
  });
  console.log(response.text);
}

main();

Pass-through vs Native Router Endpoints​

LiteLLM offers two ways to handle GenAI SDK requests:

Pass-through (/gemini)Native Router (/)
baseUrlhttp://localhost:4000/geminihttp://localhost:4000
ModelsGemini onlyAny provider via model_group_alias
TranslationNone — proxies directly to GoogleTranslates internally
Cost Tracking✅✅
Virtual Keys✅✅
Load Balancing❌✅
Fallbacks❌✅
Best forSimple Gemini proxyMulti-provider routing

Environment Variable Configuration​

You can also configure the SDK via environment variables instead of code:

# For JavaScript SDK (@google/genai)
export GOOGLE_GEMINI_BASE_URL="http://localhost:4000/gemini"
export GEMINI_API_KEY="sk-1234"

# For Python SDK (google-genai)
# Note: The Python SDK does not support a base URL env var.
# Configure it in code with http_options={"base_url": "..."} instead.
export GEMINI_API_KEY="sk-1234"

This is especially useful for tools built on top of the GenAI SDK (like Gemini CLI).

🚅
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.
Learn more →