uptrain-ai/uptrain

Health v2 · maintenance only

Adoption not part of the score

2362 stars · 205 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

UpTrain is an open-source Python platform for evaluating and improving LLM applications, offering 20+ preconfigured evaluation checks covering language, code, and embedding use cases. It performs root cause analysis on failure cases, supports automated regression testing and prompt versioning, and includes locally run dashboards for visualizing results.

Use cases

• evaluate llm responses for factual accuracy and hallucinations
• test rag pipeline retrieval quality
• run regression tests on prompt changes
• find root causes of llm application failures
• score chatbot response quality and tonality
• detect jailbreak attempts in llm outputs
• build custom llm evaluation metrics

When to choose

• you need automated evaluation of LLM application outputs with prebuilt metrics
• you want root cause analysis and failure pattern detection for RAG or chatbot pipelines
• you need regression testing and prompt versioning for LLM apps
• you prefer an open-source, self-hostable LLMOps evaluation tool

When to avoid

• you need full observability/tracing of production LLM traffic rather than evaluation
• you want a managed SaaS-only evaluation service
• your stack is not Python-based
• you need evaluation of non-LLM machine learning models

Facets

library · maturity active

testing monitoring benchmarking llm-inference prompt-engineering rag analytics large-language-models machine-learning developer-tools artificial-intelligence testing python self-hosted llm-evaluation llmops hallucination-detection root-cause-analysis rag-evaluation regression-testing observability docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="uptrain-ai/uptrain")