Skip to main content
LiteLLM is moving to Rust Read the latest updates.

LiteLLM v1.104.5 counts Claude tokens on Bedrock Mantle when Bedrock Runtime cannot

Deploy this version​

docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.104.5

This release is published as ghcr.io/berriai/litellm:v1.104.5. See the GitHub release and the full releases page

v1.104.5 is a patch release on top of v1.104.4. It backports the Bedrock Mantle fallback for token counting onto stable/1.104.x. There are no new database migrations. The v1.104.5 tag points at 5275320

What's Changed​

  • Bedrock Runtime CountTokens rejects newer Claude models, so /v1/messages/count_tokens fell back to a local estimate that undercounts. On a 400 or 404 for a Claude model the count now comes from the Bedrock Mantle Anthropic endpoint, and only falls back to the local estimate if Mantle also fails. Non-Claude models never call Mantle - PR #45317, PR #46430

The fix and the version bump are backported in PR #47634

Full Changelog​

https://github.com/BerriAI/litellm/compare/v1.104.4...v1.104.5

LiteLLM Enterprise
SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails, built for production.
Learn more →