Traceloop (OpenLLMetry) - Tracing LLMs with OpenTelemetry
Traceloop is a platform for monitoring and debugging the quality of your LLM outputs. It provides you with a way to track the performance of your LLM application; rollout changes with confidence; and debug issues in production. It is based on OpenTelemetry, so it can provide full visibility to your LLM requests, as well vector DB usage, and other infra in your stack.
Install the Traceloop SDK:
pip install traceloop-sdk
Use just 2 lines of code, to instantly log your LLM responses with OpenTelemetry:
Traceloop.init(app_name=<YOUR APP NAME>, disable_batch=True)
litellm.success_callback = ["traceloop"]
To get better visualizations on how your code behaves, you may want to annotate specific parts of your LLM chain. See Traceloop docs on decorators for more information.
Exporting traces to other systems (e.g. Datadog, New Relic, and others)
Since Traceloop SDK uses OpenTelemetry to send data, you can easily export your traces to other systems, such as Datadog, New Relic, and others. See Traceloop docs on exporters for more information.