Skip to main content

4 posts tagged with "claude-code"

View All Tags

Claude Code server-side auto mode through LiteLLM

Mateo Wang
AI Engineer, LiteLLM

Last Updated: October 9, 2026

Anthropic is moving the Claude Code auto mode safety classifier from the client to the Claude API. From v2.1.278 onwards (released 19 September), sessions on Enterprise plans and Claude API accounts will have the safety classifier run on the server as part of the model request. There is no cost to Anthropic for the classifier to run on the server. The rollout began on 18 September and is gradual, starting with the Claude Code CLI and VS Code extension, followed by the desktop app and Claude Code in the browser over the following week. On 25 September auto mode will become the default permission mode in Claude Code. Currently the default is auto on Pro, Max and Team plans and manual on Enterprise plans and Claude API keys (the accounts that are typically behind a gateway) as per Anthropic's permission modes reference.

Server side auto mode is predicated on a contract between Claude Code and the API which some gateways (LiteLLM included) did not honour. The fix first appeared in the dev release cut on Tuesday 22nd September and is now in v1.104.0 stable as well as in the patch releases of the earlier lines. This post covers what Claude Code requires from an AI Gateway, what LiteLLM was doing wrong, what has changed, which release has it, and how you can verify your deployment is ready.

Incident Report: Prompt Cache Invalidation for Claude Code on Bedrock Invoke

Mateo Wang
AI Engineer, LiteLLM
Krrish Dholakia
CEO, LiteLLM
Ishaan Jaffer
CTO, LiteLLM

Date: July 4 to July 10, 2026
Affected versions: v1.91.0 and v1.91.1
Severity: Medium (silent cost regression; no correctness impact)
Status: Resolved in v1.91.2

Note: If you run Claude Code against Amazon Bedrock through LiteLLM on either v1.91.0 or v1.91.1, upgrade to v1.91.2 or higher.

Summary​

Between July 4 and July 10, proxies running v1.91.0 or v1.91.1 silently broke Anthropic prompt caching for Claude Code sessions routed through Amazon Bedrock's Invoke API. For the customers who reported it, warm-session cache hit rates dropped from roughly 90% to 25-45% and team daily spend rose 2-3x for the same usage. Requests kept returning 200s with correct completions; the only symptoms were the cache miss rate and the bill.

The cause: PR #31364 moved every role: "system" entry in messages into the top-level system field on the Invoke path, which invalidates every cache breakpoint past the tool definitions and system prompt. The fix shipped July 10 in v1.91.2 (#32578, #32831, #32882), with regression tests that fail on pre-fix code.

We own this outcome entirely. The trigger was a poorly documented change in how new Claude models and Claude Code use system messages, but customers run a gateway precisely so they do not have to track provider quirks. Translating requests faithfully, including their caching semantics, is our core job and here we fell short. This post explains exactly what happened, why our testing and review failed to catch it, and what we have changed so this class of regression does not ship again.

5 ways to cut Claude Code costs with LiteLLM

Krrish Dholakia
CEO, LiteLLM

5 ways to save Claude Code cost with LiteLLM

Claude Code is one of the heaviest consumers of input tokens in a modern engineering org. Long tool loops, large file reads, and MCP catalogs with hundreds of tools push every request toward the top of the context window, and the bill scales with it.

If Claude Code already points at a LiteLLM proxy (via ANTHROPIC_BASE_URL), there are five levers the platform admin can pull to bring that cost down. None of them require a client-side change.