LiteLLM is moving to Rust 🦀Read the latest updates.›

v1.105.1 - Batched model usage writes and single spend log on the Responses bridge

Deploy this version

Docker

docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.105.1

Pip

pip install litellm==1.105.1

This release is published as ghcr.io/berriai/litellm:v1.105.1. See the GitHub release and the full releases page

v1.105.1 is a patch release on top of v1.105.0. It backports two spend tracking fixes from main onto stable/1.105.x. There are no new database migrations and no breaking changes. The v1.105.1 tag points at d665dc1

Key Highlights

• Batched model usage writes: the model leaderboard rollup no longer upserts LiteLLM_DailyModelUsage on every request, which removes the row lock contention that dropped SpendLogs rows when heavy traffic went to one model
• Single spend log on the Responses bridge: non-streaming requests of 256 KiB or more sent through the chat to Responses bridge log their spend once instead of twice

Spend Tracking, Budgets and Rate Limiting

• The daily model usage rollup added with the model leaderboard in v1.105.0 wrote one LiteLLM_DailyModelUsage upsert per request. At high request rates for a single model those upserts fought over the same row and held Prisma connections, so some SpendLogs rows were never written. Rollup rows are now queued in memory and written in batches by the existing spend log flush job. See PR #44243
• Non-streaming requests on the chat to Responses bridge whose messages were 256 KiB or larger ran the async success logging twice, which wrote two SpendLogs rows and counted key, team and daily spend twice. The two handlers now run one after the other on the same logging object, and the second one skips if the first already logged. See PR #44508

Both fixes and the version bump are backported together in PR #46313

PR roll-up by ownership area

• Spend / Budgets / Rate Limits: 2

Full Changelog

https://github.com/BerriAI/litellm/compare/v1.105.0...v1.105.1