langwatch/langwatch

Health v2 · maintenance only

Adoption not part of the score

3511 stars · 355 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

LangWatch is an open-source LLMOps platform for LLM evaluation, AI agent testing, tracing, and production observability. It combines agent simulations, dataset-driven experiments, online evaluation monitors, prompt management, and an OpenAI-compatible AI gateway, deployable as a cloud service or self-hosted via Helm.

Use cases

• evaluate llm outputs for quality and safety
• run agent simulations before release
• monitor production llm traffic for regressions
• trace and debug ai agent behavior
• catch prompt regressions in ci/cd
• red team agents for jailbreaks and unsafe tool calls
• manage prompts and optimize models across providers
• self-host an llm observability platform

When to choose

• you need end-to-end evaluation, tracing, and prompt management in one platform
• you want simulation-based agent testing with simulated users and LLM judges
• you need OpenTelemetry-native, provider-agnostic instrumentation
• you require self-hosting for GDPR/SOC 2 or air-gapped environments

When to avoid

• you only need simple token/cost logging without evaluation workflows
• you want a fully free managed service with no usage-based pricing
• you need a lightweight library rather than a full platform deployment

Facets

service · maturity active

monitoring tracing testing benchmarking analytics prompt-engineering rag llm-inference api-gateway sdk large-language-models machine-learning developer-tools monitoring testing self-hosted self-hosted python cloud llm-evaluation llmops agent-testing agent-simulation opentelemetry red-teaming guardrails prompt-optimization dspy open-core ai-agents docker kubernetes web-server nodejs

Member repositories

For agents

markdown · JSON · MCP: product_card(name="langwatch/langwatch")