Skip to main content

Multi-Region Deployment

Run LiteLLM in more than one region when users need a nearby gateway or your recovery plan covers a regional outage. Regional gateways use one logical PostgreSQL database and a Redis instance per region.

Start with High Availability for replicas and zone failures within one region. Use Routing to choose a region for each request. Use Failover to plan for regional and shared dependency failures.

Architecture​

PostgreSQL holds the shared keys, teams, users, configuration, and spend records. Use a highly available database service. All regional gateways depend on its writable primary.

ModeNormal request routingCapacity requirement
Active-activeBoth regions serve requests through a latency, location, or weighted policy.The surviving region needs spare capacity for the traffic it takes over.
Active-passiveRegion A serves requests. Region B waits as a standby.Keep standby capacity and provider access ready for the required recovery time.

Test the standby regularly in an active-passive deployment for stale credentials, insufficient quota, or missing configuration.

Authentication caches, configuration reloads, read-replica lag, and batched spend writes can delay updates across regions. See authentication caching and Database Read Replica.

Licensing across regions​

A single LiteLLM Enterprise license covers a multi-region deployment when all regions share one database. Each instance receives and validates the same LITELLM_LICENSE. LiteLLM counts users and teams against the shared database's license limits.

TopologyDatabasesEnterprise licenses
Shared-database multi-region deploymentOne logical databaseOne for the deployment
Independent deployment per regionOne per deploymentOne per deployment

This shared-database design works with the open source proxy. The Global Control Plane adds central management and reporting for independent deployments. Each data plane keeps its own keys, teams, budgets, database, Redis, and master key. The control-plane service requires an Enterprise license.

Requirements​

SettingRequirementPurpose
DATABASE_URLThe same logical writable databaseShared keys, configuration, spend records, and license counts.
LITELLM_MASTER_KEYThe same secretConsistent administrative authentication.
LITELLM_SALT_KEYThe same stable secretDecrypt credentials stored in the shared database. Do not replace it during a rollout.
LITELLM_LICENSEThe same license, when using Enterprise featuresActivate the shared deployment's Enterprise features.
DISABLE_SCHEMA_UPDATEtrue on serving instancesA single coordinated migration job updates the shared schema.

Use the same release and configuration source across regions. Set network endpoints and provider access for each region. Store shared secrets in your secret manager and provide them to each regional deployment.

Configure each region's Redis for its local replicas, including router_settings and any Redis-backed cache settings. See the production Redis configuration.

Regional limits

Separate Redis instances do not share rate-limit counters. A Redis-backed allowance of 100 RPM in each of two regions can permit 200 RPM across them. Shared PostgreSQL does not combine those counters into a global rate limit. For a global quota, design and test enforcement across regions. A shared Redis deployment adds cross-region latency and a shared dependency. Cache and flush intervals also affect how quickly budgets and spend records update.

Regional Redis also creates separate ownership locks for shared background jobs. Coordinate database-wide cleanup, budget resets, and exports across regions. Test which region owns each job during failover. See background job ownership.

Setup​

1. Provision the shared database​

Provision PostgreSQL with zone redundancy, backups, and a tested writer failover procedure. Run schema migrations once per upgrade through the Helm migration job or a shared deployment pipeline. Do not launch a separate migration from every region.

Add a database replica in another region if your recovery plan requires it. Define how to promote it and redirect writes in your database recovery procedure.

2. Connect the regions' networks​

Provide private network routes from every region to the shared database. Allow only the required database connections. Depending on your cloud, use AWS VPC peering, a Google Cloud VPC, or Azure VNet peering. Check that the managed database supports your routes and DNS configuration.

Use TLS and verify the database server certificate. Mount the required CA bundle in serving instances and migration jobs. See database TLS. Database calls across regions add network latency. Measure authentication, management operations, and connection recovery from each region.

3. Deploy proxy instances in each region​

Deploy the regional load balancer and LiteLLM replicas with Helm or Terraform. Disable migration jobs in secondary releases. In the Helm chart, set migrationJob.enabled: false. Keep DISABLE_SCHEMA_UPDATE=true on all serving instances.

Inject secrets through your deployment's secret manager and mount the CA at /certs/db-ca.pem.

DATABASE_URL="postgresql://litellm:<password>@db-writer.internal:5432/litellm?sslmode=verify-full&sslrootcert=/certs/db-ca.pem"
LITELLM_MASTER_KEY="sk-<shared-master-key>"
LITELLM_SALT_KEY="<shared-stable-encryption-key>"
LITELLM_LICENSE="<shared-enterprise-license>"
DISABLE_SCHEMA_UPDATE="true"
REDIS_HOST="redis.region-a.internal"
REDIS_PORT="6379"
REDIS_PASSWORD="<region-a-redis-secret>"

Omit LITELLM_LICENSE for an open source deployment. Apply the Redis settings to your proxy configuration and enable TLS for Redis where required by the service.

4. Optional: add regional read replicas​

Set DATABASE_URL_READ_REPLICA to a regional reader endpoint if you need to reduce database read latency. Writes still go to DATABASE_URL. Configure TLS for the reader as well.

DATABASE_URL_READ_REPLICA="postgresql://litellm:<password>@db-reader.region-b.internal:5432/litellm?sslmode=verify-full&sslrootcert=/certs/db-ca.pem"

Read replicas can lag behind the writer. Test newly created keys, revocations, and budget changes before relying on a replica. The read-replica guide describes query routing and behavior when the reader fails. A read replica does not automatically become the writer when the primary fails.

5. Route clients​

Give clients one gateway hostname. Use a DNS policy, global HTTP load balancer, or highly available proxy to select a regional endpoint. See Routing for examples with AWS Route 53, Global Accelerator, Google Cloud load balancing, Azure Traffic Manager, Front Door, and HAProxy.

Set health checks to match the service you plan to offer during a failure. Process liveness alone does not prove that a region can authenticate and serve requests. Read Failover before enabling automatic traffic changes.

6. Verify​

Create a virtual key through one region. Use it for a model request through the other region's endpoint. Confirm the response, then check the spend record in the Admin UI. Allow time for configured caches, spend writes, and any read-replica lag. The Quickstart covers key creation.

Test routing with each region unavailable in turn. Test shared database loss separately. It can affect both regions at once. Record error rates, stream completion, recovery time, and the spare capacity the surviving region needs.

Optional: dedicated admin instance​

You can keep management access on one private hostname while regional gateways serve model requests.

Enterprise feature

This feature requires a LiteLLM Enterprise license. Start a free 30-day trial or book a demo. See what Enterprise includes.

DISABLE_ADMIN_ENDPOINTS and DISABLE_LLM_API_ENDPOINTS are Enterprise features.

Set these environment values:

DISABLE_LLM_API_ENDPOINTS="true"

DISABLE_ADMIN_UI disables /ui. DISABLE_ADMIN_ENDPOINTS disables management routes while model endpoints, health checks, and metrics remain available. DISABLE_LLM_API_ENDPOINTS disables model-serving routes while management remains available. The componentized Helm deployment also separates gateway, backend, and UI services.

FAQ​

Do I need multi-region? Replicas and redundant data services within one region can cover instance and zone failures. Add regions to meet a specific latency or disaster-recovery requirement. Test the additional dependencies.

What if the primary region's database fails? Both regions can lose normal database-backed operations. Gateway failover alone does not fix this. Use the shared-database recovery procedure. Decide in advance whether to allow restricted service during the outage.

Can regions run different versions? Only during a tested upgrade where both versions support the shared schema and configuration. Run migrations once. Return all regions to the same release after the upgrade. See Blue/Green and Canary.