Routing in Rust
As of today, v1.110.0, all routing in LiteLLM flows through a Rust core! The previous python code that implemented it is fully deleted, and we have one source of truth in the codebase. That includes all logic around retries, fallbacks, name resolution, deployment selection, etc. It was over 50k LOC deleted in the final swap.
Right now, LiteLLM users won't see a huge change, but the value will manifest when we have incredibly fast messages in December. Additionally, this refactor enabled us to create the , which is great for exploring routing configurations.
This post covers why we chose the Router to operate, and how we managed to migrate it safely without causing regressions.
























