LiteLLM v1.106.6 and v1.105.7 fix the gzip performance regression
Deploy this version
- Docker
- Pip
docker run \
-e LITELLM_MASTER_KEY=sk-<paste-a-long-random-key> \
-e DATABASE_URL=postgresql://<user>:<password>@<host>:5432/<dbname> \
-e STORE_MODEL_IN_DB=True \
-p 4000:4000 \
docker.litellm.ai/berriai/litellm:1.106.6
pip install litellm==1.106.6
This release is published as ghcr.io/berriai/litellm:v1.106.6. See the GitHub release and the full releases page.
Inference and other endpoints outside the dashboard analytics routes no longer return gzip responses. The proxy ignores Accept-Encoding: gzip on those routes, so their responses are larger on the wire. LiteLLM has no setting to restore the old behavior. If you need compressed responses, turn on compression in a reverse proxy or load balancer in front of the proxy. See PR #46712.
v1.106.6 is a patch release on top of v1.106.5. It fixes the gzip performance regression on stable/1.106.x that the v1.106.5 notes warn about. There are no new database migrations. The v1.106.6 tag points at 61c6ab7.
What's Changed
- The proxy only gzips responses from the dashboard analytics routes, such as the Usage and Cost Optimization pages, and compresses them on a worker thread instead of the event loop. Requests sent with
Accept-Encoding: gzip, which most HTTP clients and the OpenAI SDKs send by default, no longer slow down other requests on the same worker - PR #46712
The change and the version bump are backported in PR #48680.
Full Changelog
https://github.com/BerriAI/litellm/compare/v1.106.5...v1.106.6
