ianarawjo/ChainForge

Health v2 · maintenance only

Adoption not part of the score

3027 stars · 254 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

ChainForge is an open-source visual programming environment for battle-testing prompts to LLMs, built on ReactFlow and Flask. It lets users query multiple LLMs at once, compare responses across prompt permutations and models, and evaluate and visualize results with little to no coding.

Use cases

• compare prompt variations across multiple LLMs
• evaluate LLM response quality with scoring metrics
• visualize prompt evaluation results across models
• test prompt robustness before deploying to production
• run ground-truth evaluations against benchmark datasets
• query OpenAI, Anthropic, Gemini, and Ollama models side by side
• build LLM evaluation flows without writing code

When to choose

• you need to rapidly compare prompts, models, and settings across multiple LLM providers
• you want a visual, low-code interface for LLM evaluation and experimentation
• you need to score and plot LLM responses with custom Python/JavaScript evaluators or LLM-based scorers
• you want to test prompts against local models via Ollama or cloud APIs in one tool

When to avoid

• you need a production LLM pipeline or deployment orchestration rather than experimentation
• you require Safari or other non-Chromium/Firefox browser support
• you want fully automated CI-based eval pipelines with no interactive UI
• you need to avoid executing untrusted code, since evaluator nodes run locally without sandboxing

Facets

application · maturity active

prompt-engineering llm-inference data-visualization machine-learning developer-tools large-language-models artificial-intelligence developer-tools data-visualization python cross-platform browser self-hosted llm-evaluation visual-programming prompt-testing llmops reactflow a-b-testing-prompts model-comparison web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ianarawjo/ChainForge")