truera/trulens

Health v2 · maintenance only

Adoption not part of the score

3525 stars · 331 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

TruLens is a Python library for evaluating and tracking LLM-based applications and AI agents. It provides OpenTelemetry-native tracing of every step (latency, tokens, cost) plus LLM-as-a-judge feedback metrics like the RAG triad, with version comparison and leaderboards.

Use cases

• evaluate rag pipeline quality with groundedness metrics
• trace and debug why my ai agent gave a bad answer
• compare llm app versions on cost latency and quality
• run llm as a judge evals on agent traces
• monitor token usage and cost per step of llm calls
• catch agent errors automatically with eval metrics
• benchmark rag evaluation metrics against human annotations

When to choose

• you need systematic evaluation and observability for LLM apps or agents in Python
• you want OpenTelemetry-native traces portable to any OTLP backend
• you need benchmarked RAG triad or agent error-detection metrics out of the box

When to avoid

• you need a fully managed hosted LLM observability platform rather than an open-source library
• your stack is not Python-based
• you only need simple logging without evaluation metrics

Facets

library · maturity active

monitoring tracing testing benchmarking machine-learning llm-inference rag agent-framework analytics large-language-models machine-learning developer-tools monitoring data-science python cross-platform llm-evaluation llmops agentops ai-observability opentelemetry llm-as-a-judge rag-triad evals tracing ai-agents retrieval-augmented-generation

Member repositories

For agents

markdown · JSON · MCP: product_card(name="truera/trulens")