Incident Report: Invalid model cost map on main
Date: January 27, 2026 Duration: ~20 minutes Severity: Low Status: Resolved
Summary​
A malformed JSON entry in model_prices_and_context_window.json was merged to main (562f0a0). This caused LiteLLM to silently fall back to a stale local copy of the model cost map. Users on older package versions lost cost tracking for newer models only (e.g. azure/gpt-5.2). No LLM calls were blocked.
- LLM calls and proxy routing: No impact.
- Cost tracking: Impacted for newer models not present in the local backup. Older models were unaffected. The incident lasted ~20 minutes until the commit was reverted.
Background​
The model cost map is not in the request path. It is used after the LLM response comes back, inside a try/catch, to calculate spend. A missing entry never blocks a call.
Both paths return a response to the caller. When the cost map lookup fails, the only difference is cost=0 on that request.
Root cause​
LiteLLM fetches the model cost map from GitHub main at import time. If the fetch fails, it falls back to a local backup bundled with the package. Before this incident, the fallback was completely silent -- no warning was logged.
A contributor PR introduced an extra { bracket, producing invalid JSON. The remote fetch failed with JSONDecodeError, triggering the silent fallback. Users on older package versions had backup files missing newer models.
Timeline:
- Malformed JSON merged to
main - LiteLLM installations fall back to local backup on next import
- Users report
"This model isn't mapped yet"for newer models - Bad commit identified and reverted (~20 minutes)
Remediation​
| # | Action | Status | Code |
|---|---|---|---|
| 1 | CI validation on model_prices_and_context_window.json | ✅ Done | test-model-map.yaml |
| 2 | Warning log on fallback to local backup | ✅ Done | get_model_cost_map.py#L57-L68 |
| 3 | GetModelCostMap class with integrity validation helpers | ✅ Done | get_model_cost_map.py#L24-L149 |
| 4 | Resilience test suite (bad hosted map, fallback, completion) | ✅ Done | test_model_cost_map_resilience.py#L150-L291 |
| 5 | Test that backup model cost map always exists and contains common models | ✅ Done | test_model_cost_map_resilience.py#L213-L228 |
Enterprises that require zero external dependencies at import time can set LITELLM_LOCAL_MODEL_COST_MAP=True to skip the GitHub fetch entirely.
Other dependencies on external resources​
| Dependency | Impact if unavailable | Fallback |
|---|---|---|
| Model cost map (GitHub) | Cost tracking for newer models | Local backup (now with warning) |
| JWT public keys (IDP/SSO) | Auth fails | None |
| OIDC UserInfo (IDP/SSO) | Auth fails | None |
| HuggingFace model API | HF provider calls fail | None |
| Ollama tags (localhost) | Ollama model list stale | Static list |