LiteLLM is moving to Rust Read the latest updates.
LiteLLM v1.109.2 adds Gemini batches and fixes OOM on 429 retries
Released
Deploy this version​
- Docker
- Pip
docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.109.2
pip install litellm==1.109.2
This release is published as ghcr.io/berriai/litellm:v1.109.2. See the GitHub release and the full releases page.
v1.109.2 is a patch release on top of v1.109.1. There are no new database migrations. The v1.109.2 tag points at fa6d80e.
Breaking Changes
- Deployment: Switch exec probes on the UI image from
curltowget, since that image no longer shipscurl- PR #52298
What's Changed​
- OpenAI: Fixed GPT-6.2 prompt caching for system prompts sent to
/v1/chat/completionsor/v1/messageswithstore: false- PR #52599 and PR #52694 - Vertex AI and Gemini: Added Gemini batches on
/v1/batches, with output download through/v1/filesand spend tracking - PR #51032 - Vertex AI and Gemini: Fixed Gemini batches created through
/geminilogging $0 spend with modelunknown- PR #51279 - Vertex AI and Gemini: Fixed
reasoning_effort: "xhigh"returning 400 ongemini-4-hydrogen, and added its pricing - PR #52060 - Other providers: Fixed Perplexity billing cached prompt tokens at the full input rate when the response has no
usage.cost- PR #51669 - Other providers: Fixed the
/fal_aipass-through price check missing submits sent with/requests/..or an encoded?in the path - PR #51671 - Embeddings, images and audio: Fixed custom image handlers not receiving
n,size,quality,response_format,userormask- PR #52051 - Batches and files: Fixed file ownership checks not matching other spellings of the same provider file id, like
files%2F<id>- PR #51498 - Routing and fallbacks: Fixed streamed
/v1/responsescalls that fail before any output skipping the same model group retry - PR #51953 and PR #52461 - Keys, teams and auth: Fixed out-of-memory restarts when blocked or over-limit keys retry large requests, now refused before the body is read - PR #51428, PR #51525, PR #51675 and PR #52169
- Keys, teams and auth: Fixed budgets with long unit spellings like
1month,7daysor2weeksresetting every midnight - PR #51672 - Keys, teams and auth: Fixed team member spend never resetting for users auto-joined to a default team or given
default_team_member_models, broken since v1.95.0 - PR #51728 and PR #51960 - Admin UI: Fixed usage pages returning 500 when a database lookup for key, user or team details fails - PR #50253
- Admin UI: Fixed the Spend Per User card on User Usage showing no users - PR #49464
- Admin UI: Fixed Login with SSO redirecting to
http://behind TLS termination, broken since v1.108.0 - PR #52213 - Deployment: Changed the UI image to slim nginx, which drops
curlbut keepswgetandnc- PR #52298 - Deployment: Changed filelock, oauthlib, opentelemetry, dnspython, tornado, next, soupsieve and msal-node to newer releases - PR #51712, PR #52322 and PR #52420
- Other: Changed
/healtherrors to drop Python stack traces, which stay in the proxy log with--detailed_debug- PR #52005
The changes are backported in PR #51044, PR #52243, PR #52310, PR #52418, PR #52561, PR #52664, PR #52684 and PR #52707.
Full Changelog​
https://github.com/BerriAI/litellm/compare/v1.109.1...v1.109.2