hegelai/prompttools

Health v2 · maintenance only

Adoption not part of the score

3056 stars · 255 forks observed · 2026-10-16

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

PromptTools is an open-source Python library for testing, experimenting with, and evaluating prompts, LLMs (OpenAI, Anthropic, LLaMA, etc.), and vector databases (Chroma, Weaviate, LanceDB). It supports evaluation via code, Jupyter notebooks, and a local Streamlit playground.

Use cases

• compare prompt outputs across multiple LLM models
• evaluate retrieval accuracy of a vector database
• A/B test prompt templates and model parameters like temperature
• run LLM experiments in a Jupyter notebook
• self-host a playground for prompt experimentation
• evaluate responses from OpenAI, Anthropic, or LLaMA models

When to choose

• you need to systematically test and compare prompts or LLM parameters
• you want to benchmark vector database retrieval quality
• you prefer notebook-based or self-hosted evaluation workflows
• you want a free, open-source alternative to hosted prompt evaluation tools

When to avoid

• you need production LLM observability and tracing in live applications
• you want a fully managed cloud evaluation service
• you need CI-integrated LLM testing pipelines with team dashboards

Facets

library · maturity active

prompt-engineering testing benchmarking machine-learning llm-inference vector-database data-visualization large-language-models machine-learning deep-learning developer-tools databases python cross-platform self-hosted llm-evaluation prompt-testing vector-search experimentation notebooks playground search

Member repositories

For agents

markdown · JSON · MCP: product_card(name="hegelai/prompttools")