# mattpocock/evalite

Evaluate your LLM-powered apps with TypeScript

Repository: https://github.com/mattpocock/evalite
Canonical: https://ross.abutalabs.com/products/evalite
Homepage: https://www.evalite.dev/
Language: TypeScript
License: MIT
License Family: permissive
Topics: ai, evals, typescript
Last push: 2026-04-28T18:31:29+00:00

## Health v2 (maintenance only)
Score: 60/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 79, release rhythm 43, longevity 47
- inputs: {"age_days": 659, "days_push": 127, "days_rel": 300, "gap_med": 0, "n_releases_24m": 72}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1669, forks 102 (observed 2026-08-28T04:05:19.734375+00:00)

## What it is
Evalite is a TypeScript-native eval runner for testing LLM-powered applications, built on Vitest with a local dev server UI for exploring outputs, traces, and scores. It supports any LLM provider, scoring libraries like autoevals, and CI-friendly static HTML export with score thresholds.

## Use cases
- evaluate llm app outputs in typescript
- run evals on my ai-powered app
- test chatbot responses against expected answers
- score llm outputs with levenshtein or llm-based scorers
- view llm eval traces in a local ui
- fail ci builds when eval scores drop below threshold
- compare outputs across different llm providers

## When to choose
- you build LLM-powered apps in TypeScript and want a Vitest-familiar eval workflow
- you want a local-first eval UI without vendor lock-in
- you need evals that run in CI with score thresholds

## When to avoid
- you need Python-based eval frameworks like promptfoo alternatives in Python
- you need large-scale distributed eval pipelines or hosted eval dashboards
- your project is not in the Node/TypeScript ecosystem

## Facets
- artifact type: library
- maturity: active
- function: testing, e2e-testing, benchmarking, llm-inference, developer-tools
- domain: large-language-models, machine-learning, developer-tools, testing, artificial-intelligence
- platform: cli
- tags: llm-evals, vitest, eval-runner, local-first, ai-testing, scoring, nodejs, typescript, web-server

## Member repositories
- mattpocock/evalite (main) score 60

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:19.734375+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:42:47.015154+00:00, confidence not recorded.
  - readme: https://github.com/mattpocock/evalite (fetched 2026-08-28T04:05:19.734375+00:00, sha 779c410fd6e1)
  - homepage: https://www.evalite.dev/ (fetched 2026-08-29T11:16:26.016580+00:00, sha 02516939e10c)
  - site_page: https://www.evalite.dev/quickstart (fetched 2026-08-29T11:16:26.027074+00:00, sha 7694e8905032)
  - registry_npm: https://registry.npmjs.org/evalite (fetched 2026-08-29T11:16:26.028735+00:00, sha c544c4c3a9a1)
  - site_page: https://v1.evalite.dev (fetched 2026-08-29T11:16:26.025274+00:00, sha 695cf08f4943)
- Data as of 2026-08-30T08:39:29.467469+00:00.