How Pfizer Improved LiteLLM Gateway Performance and Resiliency at Scale
A joint debugging story with LiteLLM, and the release testing that comes next.
A LiteLLM version bump exposed a long-standing Redis configuration bug that cut Pfizer's gateway throughput by ~48%, with zero HTTP errors in the application logs. Here's how the team isolated it through configuration bisection, and the testing infrastructure both teams are building to catch this class of regression before it ships.