v1.101.2 - Bedrock Invoke Streaming & End-User Budget Resets
Deploy this version​
- Docker
- Pip
docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.101.2
pip install litellm==1.101.2
This release is published as ghcr.io/berriai/litellm:v1.101.2. See the GitHub release and the full releases page
v1.101.2 is a patch release on top of v1.101.1. It stops /v1/messages streams on Bedrock Invoke from stalling and fixes end-user budget resets for large shared budgets. There are no configuration changes. Both the Docker image and the PyPI package were built from ccb327f
Bedrock Invoke streams on /v1/messages no longer stall​
/v1/messages on Bedrock Invoke held upstream bytes in a 1024-byte buffer, so complete events sat in it whenever Bedrock paused mid-stream. Claude Code showed a silent stream while the model wrote a large file. Bedrock bytes now pass straight through, the same way the chat completions Bedrock paths already worked
Large shared end-user budgets reset again​
A shared end-user budget with more than about 32,700 customers never reset, because the reset job listed every customer id in one statement and PostgreSQL refused it, so those customers stayed blocked. End users are now reset by their budget link, so the statement size tracks the number of budgets instead of the number of customers
What's Changed​
- fix(bedrock): stream /v1/messages Invoke bytes through instead of holding them in a 1024-byte chunker - PR #42607
- fix(reset_budget_job): reset end users by budget link, not by user id - PR #40639
Full Changelog​
https://github.com/BerriAI/litellm/compare/v1.101.1...v1.101.2