Introducing LiteLLM’s Unified Cost Accounting Engine

Accurate, reliable AI spend tracking at scale.

A spend total should be a number you can act on. You use it to set budgets and decide whether a workload is worth scaling. That confidence depends on the accounting behind it, especially when requests retry or a reporting update gets interrupted.

We’re rebuilding that accounting in LiteLLM. The Unified Cost Accounting Engine gives each finalized charge a durable record that budget enforcement and reporting can share. It also changes how we can scale spend processing, and how you can use that data in your own analytics stack.

Give budgets and reports the same source of truth​

Budget tracking and reporting need to agree on what a request costs. That sounds straightforward. In practice, usage arrives through different call paths, and a single caller request can involve multiple provider attempts. Updating a budget and writing a report are separate operations; either can be interrupted.

The new engine makes the charge itself the shared reference. UsageMeter records usage while preserving the distinction between a request and its provider attempts. CostCompiler then calculates the charge and saves the pricing context used to produce it.

Once that charge is saved, downstream systems use the same result. BudgetEnforcer applies it to budget settlement. CostAggregator uses it for reporting.

Suppose settlement succeeds, but the reporting write fails. The pending reporting work can resume from the durable record, with duplicate protection at each consumer. Committed usage survives a restart. Reports may briefly lag while they catch up, but recovery preserves the original charge. Active budget reservations remain separate from settled spend.

Saving the pricing context matters here, too. When rates change, you still have the inputs needed to explain an earlier charge.

Accounting should scale on its own​

As traffic grows, the proxy has more requests to serve. Spend processing grows with it, but its capacity needs can be different. A heavier reporting workload should be something you can address by adding aggregation capacity.

That is why this refactor separates the accounting components. With clear boundaries around compilation and downstream consumers, we can move that work into separate services and scale them independently of the proxy fleet.

Redis Streams or another message queue is a planned next step. A relay can deliver pending work to dedicated workers, while PostgreSQL remains the authoritative record. Compilation workers persist the finalized charges; settlement and aggregation workers consume them with the same recovery and duplicate protection.

Request-time budget admission and reservations stay in the proxy. The opportunity is to move downstream accounting off the request-serving fleet, so its processing capacity can grow where it is needed.

Make spend data useful to your team​

A total tells you how much you spent. A breakdown helps you understand why it changed. You might compare spend across teams, then investigate the users or models behind an increase. Another organization may need to attribute costs by application or business unit.

CostAggregator gives us a place to support those different views without coupling accounting to a particular analytics database. Your reporting infrastructure can change while the finalized charges stay consistent.

PostgreSQL is the initial reporting destination. We’re planning ClickHouse and export adapters so you can bring the same accounting data into the infrastructure you already use. Each destination can build its own aggregates from saved charges, without implementing pricing again.

The sample dashboard illustrates team trends alongside user-level spend. The adapter design also creates a path to custom dimensions, provided those dimensions are captured in the accounting data. That opens up reporting shaped around your organization, rather than a fixed set of breakdowns.