Skip to main content

Rust Migration

LiteLLM is moving to Rust

See what already runs on Rust and what is next.

Migration tracker

Where we are today

5% of migrated to Rust, shown as a .

Rust migration progress over time5 percent of the work for Major APIs is done in v1.104.0-rc.1. The goal targets 100 percent by 2026-12-31.0%50%100%1.97 rc1Aug 81.99 rc1Aug 231.100 rc1Aug 301.102 rc1Sep 135%1.104 rc1Sep 26 · mainMajor APIs: 100% by 2026-12-31100% goalGoalDec 31

Engineering updates

How we are getting there

  1. · Blog post

    OCR uses Rust by default starting with v1.102.0-rc.1

  2. · Blog post

    Benchmarking the LiteLLM Rust AI Gateway: Overhead, Memory, and Cost

  3. · Blog post

    Migrating LiteLLM to Rust - Building the Fastest and Litest AI Gateway

FAQ

Frequently asked questions

  • How does a feature move to Rust?

    Each feature starts on Python and moves through these stages one release at a time. For the two middle stages, the LITELLM_RUST environment variable flips the default for the whole process.

    • Opt-in

      Python runs by default. Turn Rust on to try it, with Python as the fallback.

      LITELLM_RUST=1Run on Rust

    • Rust default

      Rust runs by default, with Python as the fallback. Turn Rust off to stay on Python.

      LITELLM_RUST=0Stay on Python

    • Rust only

      Always runs on Rust. The Python path is gone and the switch has no effect.

    • Done

      Shared work like cloud auth has no rollout of its own. It is done once every Rust path can use it.