Overview
Run the control plane in a management network and connect it to each data plane over private HTTPS. Corporate operators use the control-plane endpoint. Applications use their regional inference endpoint.
The control plane uses dedicated PostgreSQL and Redis for runtime configuration . Each data plane keeps its own stores.
See Configuration for service settings. Use the cloud deployment guides for each data plane's replicas, availability zones, and model-provider connections.
For OIDC token exchange , allow outbound HTTPS to the IdP from the control plane and each data plane. The control plane exchanges tokens. Each data plane fetches the IdP's signing keys and validates the exchanged token with its own JWT configuration.
AWS
AWS: Global Control Plane Expand diagram
AWS: Global Control Plane Corporate administrators reach one control plane through private ingress in us-east-1. The control plane sends management requests over Inter-Region VPC peering to independent LiteLLM dataplanes in us-east-2 and eu-west-1. Metadata and usage responses return through the same path. Each dataplane has separate private frontends for inference and management. Applications call the inference frontend. Only the control plane can reach the management frontend. The paired ingress icons represent these two frontends. The control plane uses dedicated PostgreSQL for runtime configuration and Redis for its configuration stream. These stores connect privately and are separate from each dataplane’s PostgreSQL and Redis.The control plane and both dataplanes share one AWS account.The control plane uses outbound HTTPS for audit export and optional OIDC discovery and token exchange. When a dataplane uses OIDC token exchange authentication, it fetches signing keys from the identity provider over HTTPS. Dataplanes initiate the JWKS requests. Control-plane OIDC login can also use service tokens to authenticate to dataplanes, without this JWKS path. Corporate network · VPN / Direct Connect Admins / automation Applications: region A Applications: region B AWS account us-east-1 Management VPC Private subnets Control-plane data subnets Outbound HTTPS us-east-2 Dataplane A VPC Private EKS subnet Private ingress Private data subnets eu-west-1 Dataplane B VPC Private EKS subnet Private ingress Private data subnets Management HTTPS 443 Inter-Region VPC peering PostgreSQL Runtime config Redis Config stream LiteLLM dataplane EKS Internal ALBs LLM / mgmt Aurora PostgreSQL ElastiCache Redis LiteLLM dataplane EKS Internal ALBs LLM / mgmt Aurora PostgreSQL ElastiCache Redis Control plane EKS · replicas Internal ALB HTTPS 443 JWKS · OIDC token exchange Audit sink IdP (optional) AWS: Global Control Plane Close diagram
AWS: Global Control Plane Corporate administrators reach one control plane through private ingress in us-east-1. The control plane sends management requests over Inter-Region VPC peering to independent LiteLLM dataplanes in us-east-2 and eu-west-1. Metadata and usage responses return through the same path. Each dataplane has separate private frontends for inference and management. Applications call the inference frontend. Only the control plane can reach the management frontend. The paired ingress icons represent these two frontends. The control plane uses dedicated PostgreSQL for runtime configuration and Redis for its configuration stream. These stores connect privately and are separate from each dataplane’s PostgreSQL and Redis.The control plane and both dataplanes share one AWS account.The control plane uses outbound HTTPS for audit export and optional OIDC discovery and token exchange. When a dataplane uses OIDC token exchange authentication, it fetches signing keys from the identity provider over HTTPS. Dataplanes initiate the JWKS requests. Control-plane OIDC login can also use service tokens to authenticate to dataplanes, without this JWKS path. Corporate network · VPN / Direct Connect Admins / automation Applications: region A Applications: region B AWS account us-east-1 Management VPC Private subnets Control-plane data subnets Outbound HTTPS us-east-2 Dataplane A VPC Private EKS subnet Private ingress Private data subnets eu-west-1 Dataplane B VPC Private EKS subnet Private ingress Private data subnets Management HTTPS 443 Inter-Region VPC peering PostgreSQL Runtime config Redis Config stream LiteLLM dataplane EKS Internal ALBs LLM / mgmt Aurora PostgreSQL ElastiCache Redis LiteLLM dataplane EKS Internal ALBs LLM / mgmt Aurora PostgreSQL ElastiCache Redis Control plane EKS · replicas Internal ALB HTTPS 443 JWKS · OIDC token exchange Audit sink IdP (optional) Use inter-Region VPC peering for management traffic. Add both sides' routes and security-group rules. Peering is not transitive, so provide corporate connectivity to each required network separately. See peering routes and limits .
Azure
Azure: Global Control Plane Expand diagram
Azure: Global Control Plane Corporate administrators reach one control plane through private ingress in East US. The control plane sends management requests over Global VNet peering to independent LiteLLM dataplanes in Central US and West Europe. Metadata and usage responses return through the same path. Each dataplane has separate private frontends for inference and management. Applications call the inference frontend. Only the control plane can reach the management frontend. The paired ingress icons represent these two frontends. The control plane uses dedicated PostgreSQL for runtime configuration and Redis for its configuration stream. These stores connect privately and are separate from each dataplane’s PostgreSQL and Redis.The control plane and both dataplanes share one Azure subscription.Managed PostgreSQL and Redis sit outside the deployment VNet and connect through private endpoints.The control plane uses outbound HTTPS for audit export and optional OIDC discovery and token exchange. When a dataplane uses OIDC token exchange authentication, it fetches signing keys from the identity provider over HTTPS. Dataplanes initiate the JWKS requests. Control-plane OIDC login can also use service tokens to authenticate to dataplanes, without this JWKS path. Corporate network · VPN / ExpressRoute Admins / automation Applications: region A Applications: region B Azure subscription East US Management VNet Private subnets Control-plane stores Outbound HTTPS Central US Dataplane A VNet Private AKS subnet Gateway subnet West Europe Dataplane B VNet Private AKS subnet Gateway subnet Management HTTPS 443 Global VNet peering Private endpoints PostgreSQL Runtime config Redis Config stream LiteLLM dataplane AKS App Gateways LLM / mgmt Private endpoints PostgreSQL Flexible Server Managed Redis LiteLLM dataplane AKS App Gateways LLM / mgmt Private endpoints PostgreSQL Flexible Server Managed Redis Control plane AKS · replicas App Gateway HTTPS 443 JWKS · OIDC token exchange Audit sink IdP (optional) Azure: Global Control Plane Close diagram
Azure: Global Control Plane Corporate administrators reach one control plane through private ingress in East US. The control plane sends management requests over Global VNet peering to independent LiteLLM dataplanes in Central US and West Europe. Metadata and usage responses return through the same path. Each dataplane has separate private frontends for inference and management. Applications call the inference frontend. Only the control plane can reach the management frontend. The paired ingress icons represent these two frontends. The control plane uses dedicated PostgreSQL for runtime configuration and Redis for its configuration stream. These stores connect privately and are separate from each dataplane’s PostgreSQL and Redis.The control plane and both dataplanes share one Azure subscription.Managed PostgreSQL and Redis sit outside the deployment VNet and connect through private endpoints.The control plane uses outbound HTTPS for audit export and optional OIDC discovery and token exchange. When a dataplane uses OIDC token exchange authentication, it fetches signing keys from the identity provider over HTTPS. Dataplanes initiate the JWKS requests. Control-plane OIDC login can also use service tokens to authenticate to dataplanes, without this JWKS path. Corporate network · VPN / ExpressRoute Admins / automation Applications: region A Applications: region B Azure subscription East US Management VNet Private subnets Control-plane stores Outbound HTTPS Central US Dataplane A VNet Private AKS subnet Gateway subnet West Europe Dataplane B VNet Private AKS subnet Gateway subnet Management HTTPS 443 Global VNet peering Private endpoints PostgreSQL Runtime config Redis Config stream LiteLLM dataplane AKS App Gateways LLM / mgmt Private endpoints PostgreSQL Flexible Server Managed Redis LiteLLM dataplane AKS App Gateways LLM / mgmt Private endpoints PostgreSQL Flexible Server Managed Redis Control plane AKS · replicas App Gateway HTTPS 443 JWKS · OIDC token exchange Audit sink IdP (optional) Use an Application Gateway v2 private deployment with private listeners. Global VNet peering connects the regional networks. Configure private DNS resolution across the peered networks.
GCP
Google Cloud: Global Control Plane Expand diagram
Google Cloud: Global Control Plane Corporate administrators reach one control plane through private ingress in us-central1. The control plane sends management requests over VPC Network Peering to independent LiteLLM dataplanes in us-east4 and europe-west1. Metadata and usage responses return through the same path. Each dataplane has separate private frontends for inference and management. Applications call the inference frontend. Only the control plane can reach the management frontend. The paired ingress icons represent these two frontends. The control plane uses dedicated PostgreSQL for runtime configuration and Redis for its configuration stream. These stores connect privately and are separate from each dataplane’s PostgreSQL and Redis.The control plane and both dataplanes share one Google Cloud project.The project contains separate global VPCs with regional private, frontend, and proxy-only subnets. Data services use private service connections outside the deployment VPC.The control plane uses outbound HTTPS for audit export and optional OIDC discovery and token exchange. When a dataplane uses OIDC token exchange authentication, it fetches signing keys from the identity provider over HTTPS. Dataplanes initiate the JWKS requests. Control-plane OIDC login can also use service tokens to authenticate to dataplanes, without this JWKS path. Corporate network · Cloud VPN / Interconnect Admins / automation Applications: region A Applications: region B Google Cloud project Management VPC (global) us-central1 Private + proxy-only subnets CP stores · us-central1 Outbound HTTPS Dataplane A VPC (global) us-east4 Private GKE subnet Proxy-only subnet Frontend subnet Managed services · us-east4 Dataplane B VPC (global) europe-west1 Private GKE subnet Proxy-only subnet Frontend subnet Managed services · europe-west1 Management HTTPS 443 VPC Network Peering PostgreSQL Runtime config Redis Config stream Private Services Access LiteLLM dataplane GKE Internal ALBs LLM / mgmt LB proxies Cloud SQL PostgreSQL Memorystore Redis Private Services Access LiteLLM dataplane GKE Internal ALBs LLM / mgmt LB proxies Cloud SQL PostgreSQL Memorystore Redis Private Services Access Control plane GKE · replicas Internal ALB HTTPS 443 JWKS · OIDC token exchange Audit sink IdP (optional) Google Cloud: Global Control Plane Close diagram
Google Cloud: Global Control Plane Corporate administrators reach one control plane through private ingress in us-central1. The control plane sends management requests over VPC Network Peering to independent LiteLLM dataplanes in us-east4 and europe-west1. Metadata and usage responses return through the same path. Each dataplane has separate private frontends for inference and management. Applications call the inference frontend. Only the control plane can reach the management frontend. The paired ingress icons represent these two frontends. The control plane uses dedicated PostgreSQL for runtime configuration and Redis for its configuration stream. These stores connect privately and are separate from each dataplane’s PostgreSQL and Redis.The control plane and both dataplanes share one Google Cloud project.The project contains separate global VPCs with regional private, frontend, and proxy-only subnets. Data services use private service connections outside the deployment VPC.The control plane uses outbound HTTPS for audit export and optional OIDC discovery and token exchange. When a dataplane uses OIDC token exchange authentication, it fetches signing keys from the identity provider over HTTPS. Dataplanes initiate the JWKS requests. Control-plane OIDC login can also use service tokens to authenticate to dataplanes, without this JWKS path. Corporate network · Cloud VPN / Interconnect Admins / automation Applications: region A Applications: region B Google Cloud project Management VPC (global) us-central1 Private + proxy-only subnets CP stores · us-central1 Outbound HTTPS Dataplane A VPC (global) us-east4 Private GKE subnet Proxy-only subnet Frontend subnet Managed services · us-east4 Dataplane B VPC (global) europe-west1 Private GKE subnet Proxy-only subnet Frontend subnet Managed services · europe-west1 Management HTTPS 443 VPC Network Peering PostgreSQL Runtime config Redis Config stream Private Services Access LiteLLM dataplane GKE Internal ALBs LLM / mgmt LB proxies Cloud SQL PostgreSQL Memorystore Redis Private Services Access LiteLLM dataplane GKE Internal ALBs LLM / mgmt LB proxies Cloud SQL PostgreSQL Memorystore Redis Private Services Access Control plane GKE · replicas Internal ALB HTTPS 443 JWKS · OIDC token exchange Audit sink IdP (optional) GCP VPC networks are global and subnets are regional . VPC Network Peering connects the separate networks. Configure global access for regional internal load balancers for cross-region management traffic.
Hybrid cloud
Manage an AWS data plane and an on-premises data plane through one control plane in AWS. Applications use the data plane in their own region or data center.
Hybrid cloud: Global Control Plane Expand diagram
Hybrid cloud: Global Control Plane Corporate administrators reach one control plane through private ingress in us-east-1. The control plane sends management requests over VPC peering and Site-to-Site VPN to independent LiteLLM dataplanes in us-east-2 and on-premises data center. Metadata and usage responses return through the same path. Each dataplane has separate private frontends for inference and management. Applications call the inference frontend. Only the control plane can reach the management frontend. The paired ingress icons represent these two frontends. The control plane uses dedicated PostgreSQL for runtime configuration and Redis for its configuration stream. These stores connect privately and are separate from each dataplane’s PostgreSQL and Redis.The control plane and AWS dataplane share one AWS account. The on-premises dataplane runs Kubernetes in the corporate data center. AWS Site-to-Site VPN carries private management HTTPS over an encrypted IPsec tunnel.The control plane uses outbound HTTPS for audit export and optional OIDC discovery and token exchange. When a dataplane uses OIDC token exchange authentication, it fetches signing keys from the identity provider over HTTPS. Dataplanes initiate the JWKS requests. Control-plane OIDC login can also use service tokens to authenticate to dataplanes, without this JWKS path. Corporate network · VPN / Direct Connect Admins / automation Applications: region A Applications: on-premises AWS account Corporate data center us-east-1 Management VPC Private subnets Control-plane data subnets Outbound HTTPS us-east-2 Dataplane A VPC Private EKS subnet Private ingress Private data subnets Production network Dataplane B Kubernetes subnet Private ingress Private data subnets VPC peering · HTTPS 443 AWS Site-to-Site VPN · IPsec Management HTTPS 443 PostgreSQL Runtime config Redis Config stream LiteLLM dataplane EKS Internal ALBs LLM / mgmt Aurora PostgreSQL ElastiCache Redis LiteLLM dataplane Kubernetes Private ingresses LLM / mgmt PostgreSQL Redis VPN router Control plane EKS · replicas Internal ALB HTTPS 443 JWKS · OIDC token exchange Audit sink IdP (optional) Hybrid cloud: Global Control Plane Close diagram
Hybrid cloud: Global Control Plane Corporate administrators reach one control plane through private ingress in us-east-1. The control plane sends management requests over VPC peering and Site-to-Site VPN to independent LiteLLM dataplanes in us-east-2 and on-premises data center. Metadata and usage responses return through the same path. Each dataplane has separate private frontends for inference and management. Applications call the inference frontend. Only the control plane can reach the management frontend. The paired ingress icons represent these two frontends. The control plane uses dedicated PostgreSQL for runtime configuration and Redis for its configuration stream. These stores connect privately and are separate from each dataplane’s PostgreSQL and Redis.The control plane and AWS dataplane share one AWS account. The on-premises dataplane runs Kubernetes in the corporate data center. AWS Site-to-Site VPN carries private management HTTPS over an encrypted IPsec tunnel.The control plane uses outbound HTTPS for audit export and optional OIDC discovery and token exchange. When a dataplane uses OIDC token exchange authentication, it fetches signing keys from the identity provider over HTTPS. Dataplanes initiate the JWKS requests. Control-plane OIDC login can also use service tokens to authenticate to dataplanes, without this JWKS path. Corporate network · VPN / Direct Connect Admins / automation Applications: region A Applications: on-premises AWS account Corporate data center us-east-1 Management VPC Private subnets Control-plane data subnets Outbound HTTPS us-east-2 Dataplane A VPC Private EKS subnet Private ingress Private data subnets Production network Dataplane B Kubernetes subnet Private ingress Private data subnets VPC peering · HTTPS 443 AWS Site-to-Site VPN · IPsec Management HTTPS 443 PostgreSQL Runtime config Redis Config stream LiteLLM dataplane EKS Internal ALBs LLM / mgmt Aurora PostgreSQL ElastiCache Redis LiteLLM dataplane Kubernetes Private ingresses LLM / mgmt PostgreSQL Redis VPN router Control plane EKS · replicas Internal ALB HTTPS 443 JWKS · OIDC token exchange Audit sink IdP (optional) The management VPC's VPN cannot provide corporate access through VPC peering to the AWS data plane. See peering limitations . You can use Direct Connect with a private virtual interface instead of VPN.
