evidentlyai/evidently

Health v2 · maintenance only

Adoption not part of the score

7849 stars · 901 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Evidently is an open-source Python framework for evaluating, testing, and monitoring ML and LLM-powered systems, with 100+ built-in metrics from data drift detection to LLM judges. It supports one-off reports, pass/fail test suites for CI/CD, and a self-hostable monitoring dashboard service.

Use cases

• detect data drift in production ML models
• evaluate LLM and RAG pipeline quality
• run data validation checks in CI/CD
• monitor model performance over time with a dashboard
• generate HTML reports for ML experiments
• test tabular and text data quality
• set up regression tests for ML models

When to choose

• you need to evaluate or monitor ML models or LLM apps in Python
• you want drift detection, data quality, and LLM evals in one framework
• you need pass/fail test suites for ML pipelines in CI/CD
• you want a self-hostable monitoring UI for model metrics

When to avoid

• you need a fully managed no-code monitoring platform (consider Evidently Cloud or alternatives)
• you only need generic application observability like logs and traces rather than ML-specific evals
• you work outside the Python/pandas ecosystem

Facets

library · maturity active

monitoring testing data-science machine-learning llm-inference rag data-visualization analytics machine-learning data-science large-language-models developer-tools python cli self-hosted cross-platform mlops llmops data-drift model-monitoring data-quality data-validation llm-evaluation jupyter-notebook html-reports test-suites data-engineering

Member repositories

For agents

markdown · JSON · MCP: product_card(name="evidentlyai/evidently")