Auto Router v1.97: usage benchmarks and better quality for lower cost
🚀 Help shape the Auto-Router
Get early access, work directly with the LiteLLM team, and influence the roadmap with your production traffic.
Apply to Become a Design PartnerAlready testing it? Share your results in discussion #32168.
v1.97 makes three changes to the auto router.
- The LLM classifier now receives a window of prior conversation turns, defaulting to three. This improves accuracy of follow-up classifications from 14% to 78%, costs at most $0.61 per 1,000 requests, and no additional latency.
- A new Benchmarks view prices routed traffic against an all-frontier baseline and reports the difference, and those savings now also appear in the Cost Optimization totals.
- Session affinity is now off by default, following our previous post showing this was leading to worse quality without cost improvements.
Two defaults changed
classifier_context_window_size now defaults to 3 (LLM classifier only), and session_affinity now defaults to false (all routers). Config files are not modified, but the new defaults apply to any key left unset, so a config that never mentioned session_affinity will reclassify every turn after upgrading. Configs that set either key explicitly are unaffected.
