Skip to main content

Scalekit with LiteLLM

Add authenticated tool calls to your LiteLLM-powered agents. Scalekit manages OAuth flows, token storage, and API execution for 100+ third-party apps (Gmail, GitHub, Slack, Salesforce, etc.). Your agent picks tools at runtime and LiteLLM routes the model calls to any provider.

Overview​

  • Fetch user-scoped tool definitions from Scalekit and pass them as function schemas to litellm.completion()
  • Switch models freely, since the same tool definitions work across OpenAI, Anthropic, Bedrock, Vertex AI, and every other provider LiteLLM supports
  • Execute tool calls through Scalekit, with no API keys, endpoints, or auth headers to manage per third-party app

Prerequisites​

  • Python 3.9+
  • A Scalekit account with a connection configured (this tutorial uses Gmail)
  • API keys for at least one LLM provider, or a running LiteLLM proxy
  • Scalekit API credentials (SCALEKIT_CLIENT_ID, SCALEKIT_CLIENT_SECRET, SCALEKIT_ENV_URL) from Dashboard → Developers → API Credentials

1. Install Dependencies​

pip install litellm scalekit-sdk-python

2. Initialize Clients​

setup.py
import os
import json
import litellm
import scalekit.client
from google.protobuf.json_format import MessageToDict  # installed with scalekit-sdk-python

scalekit_client = scalekit.client.ScalekitClient(
    client_id=os.getenv("SCALEKIT_CLIENT_ID"),
    client_secret=os.getenv("SCALEKIT_CLIENT_SECRET"),
    env_url=os.getenv("SCALEKIT_ENV_URL"),
)
actions = scalekit_client.actions

3. Authorize a User​

Create a connected account and complete the OAuth flow. Once the account status is ACTIVE, Scalekit can execute tools on behalf of the user.

authorize.py
connection_name = os.getenv("GMAIL_CONNECTION_NAME", "gmail")

response = actions.get_or_create_connected_account(
    connection_name=connection_name,
    identifier="user_123",  # your app's user ID
)
connected_account = response.connected_account

if connected_account.status != "ACTIVE":
    link = actions.get_authorization_link(
        connection_name=connection_name,
        identifier="user_123",
    )
    print("Authorize Gmail:", link.link)
    input("Press Enter after completing authorization...")

4. Fetch Scoped Tools​

list_scoped_tools returns only the tools this specific user is authorized to call. Convert them to OpenAI's function-calling format, the same format LiteLLM normalizes to across all providers.

fetch_tools.py
scoped_response, _ = actions.tools.list_scoped_tools(
    identifier="user_123",
    filter={"connection_names": [connection_name]},
    page_size=100,
)

# Convert to OpenAI function-calling format (used by litellm for all providers)
llm_tools = [
    {
        "type": "function",
        "function": {
            "name": MessageToDict(t.tool).get("definition", {}).get("name"),
            "description": MessageToDict(t.tool).get("definition", {}).get("description", ""),
            "parameters": MessageToDict(t.tool).get("definition", {}).get("input_schema", {}),
        },
    }
    for t in scoped_response.tools
]

5. Run the Agent Loop​

Call litellm.completion() with the tool definitions. When the model returns tool calls, execute them through Scalekit and feed the results back. Change the model parameter to switch providers; no other code changes needed.

agent_loop.py
messages = [{"role": "user", "content": "Fetch my last 5 unread emails and summarize them"}]

while True:
    response = litellm.completion(
        model="anthropic/claude-sonnet-4-20250514",  # swap to any litellm-supported model
        tools=llm_tools,
        messages=messages,
    )
    message = response.choices[0].message

    if not message.tool_calls:
        print(message.content)
        break

    # Append assistant message with tool calls
    messages.append(message)

    # Execute each tool call through Scalekit
    for tc in message.tool_calls:
        result = actions.execute_tool(
            tool_name=tc.function.name,
            identifier="user_123",
            tool_input=json.loads(tc.function.arguments),
        )
        messages.append({
            "role": "tool",
            "tool_call_id": tc.id,
            "content": str(result.data),
        })

6. Complete Working Example​

Full end-to-end script, copy and run:

scalekit_agent.py
import os
import json
import litellm
import scalekit.client
from google.protobuf.json_format import MessageToDict

# --- Configuration ---
MODEL = os.getenv("MODEL", "anthropic/claude-sonnet-4-20250514")
CONNECTION_NAME = os.getenv("GMAIL_CONNECTION_NAME", "gmail")
USER_ID = "user_123"

# --- Initialize ---
scalekit_client = scalekit.client.ScalekitClient(
    client_id=os.getenv("SCALEKIT_CLIENT_ID"),
    client_secret=os.getenv("SCALEKIT_CLIENT_SECRET"),
    env_url=os.getenv("SCALEKIT_ENV_URL"),
)
actions = scalekit_client.actions

# --- Authorize user ---
response = actions.get_or_create_connected_account(
    connection_name=CONNECTION_NAME,
    identifier=USER_ID,
)
if response.connected_account.status != "ACTIVE":
    link = actions.get_authorization_link(
        connection_name=CONNECTION_NAME,
        identifier=USER_ID,
    )
    print("Authorize Gmail:", link.link)
    input("Press Enter after completing authorization...")

# --- Fetch tools ---
scoped_response, _ = actions.tools.list_scoped_tools(
    identifier=USER_ID,
    filter={"connection_names": [CONNECTION_NAME]},
    page_size=100,
)
llm_tools = [
    {
        "type": "function",
        "function": {
            "name": MessageToDict(t.tool).get("definition", {}).get("name"),
            "description": MessageToDict(t.tool).get("definition", {}).get("description", ""),
            "parameters": MessageToDict(t.tool).get("definition", {}).get("input_schema", {}),
        },
    }
    for t in scoped_response.tools
]
print(f"Loaded {len(llm_tools)} tools for {CONNECTION_NAME}")

# --- Agent loop ---
messages = [{"role": "user", "content": "Fetch my last 5 unread emails and summarize them"}]

while True:
    response = litellm.completion(model=MODEL, tools=llm_tools, messages=messages)
    message = response.choices[0].message

    if not message.tool_calls:
        print(message.content)
        break

    messages.append(message)
    for tc in message.tool_calls:
        print(f"  Calling tool: {tc.function.name}")
        result = actions.execute_tool(
            tool_name=tc.function.name,
            identifier=USER_ID,
            tool_input=json.loads(tc.function.arguments),
        )
        messages.append({
            "role": "tool",
            "tool_call_id": tc.id,
            "content": str(result.data),
        })

Switch models by changing the MODEL environment variable:

# OpenAI
MODEL=gpt-4o python scalekit_agent.py

# Anthropic
MODEL=anthropic/claude-sonnet-4-20250514 python scalekit_agent.py

# AWS Bedrock
MODEL=bedrock/us.anthropic.claude-sonnet-4-20250514-v1:0 python scalekit_agent.py

# Via LiteLLM Proxy
OPENAI_API_BASE=http://localhost:4000 OPENAI_API_KEY=sk-1234 MODEL=claude-sonnet-4 python scalekit_agent.py

Route Through LiteLLM Proxy for Cost Tracking and Rate Limits​

If you're running a LiteLLM proxy, point your agent at it for centralized model management, cost tracking, and rate limiting. The agent code stays the same; set the proxy URL:

proxy_agent.py
import litellm

# Point litellm at your proxy
response = litellm.completion(
    model="claude-sonnet-4",                          # model name from your proxy config
    api_base="http://localhost:4000",                  # proxy URL
    api_key="sk-1234",                                 # proxy virtual key
    tools=llm_tools,
    messages=messages,
)

Or use environment variables so no code changes are needed:

export OPENAI_API_BASE="http://localhost:4000"
export OPENAI_API_KEY="sk-1234"
python scalekit_agent.py

End-to-End Example: Inbox Triage Agent​

For a production-style example that combines Scalekit tool execution with per-stage model routing through LiteLLM, see litellm-agentkit-inbox-triage. It demonstrates:

  • Polling Gmail and classifying threads with different models per pipeline stage
  • Routing to GitHub repos using keyword rules and LLM tie-breaking
  • Searching related GitHub issues through a Scalekit tool-calling loop
  • Notifying Slack and waiting for human approval before creating issues or sending replies

Troubleshooting​

IssueSolution
execute_tool returns "connection not found"The connection_name must match the exact label in Dashboard → AgentKit → Connections (including case). Use an env var instead of hardcoding.
Connected account stays in PENDINGThe user hasn't completed the OAuth flow. Regenerate the authorization link and have them open it in a browser.
Model returns text instead of tool callsNot all models support function calling. Use a model that does (GPT-4o, Claude Sonnet/Opus, Gemini Pro). Check supported providers.
litellm.completion() raises an auth errorVerify your LLM provider API key is set (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.) or that your proxy URL and key are correct.
🚅
LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.
Learn more →