utkusen/promptmap

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1254 stars · 133 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

promptmap2 is an automated prompt injection scanner for custom LLM applications, supporting white-box testing of system prompts and black-box testing of HTTP endpoints. It uses a dual-LLM architecture where a controller LLM evaluates whether attack prompts against the target LLM succeeded, with 50+ customizable YAML-based test rules.

Use cases

• test my chatbot for prompt injection vulnerabilities
• scan LLM app system prompts for jailbreak weaknesses
• run automated security tests against an OpenAI or Claude-based app
• black-box pentest an external LLM HTTP endpoint
• check if my LLM leaks its system prompt
• evaluate LLM app for harmful content and bias risks
• test local models via Ollama for prompt attacks

When to choose

• you need automated prompt injection and jailbreak testing for custom LLM apps
• you want white-box testing of system prompts or black-box testing of HTTP LLM endpoints
• you need multi-provider support including OpenAI, Claude, Gemini, Grok, and Ollama local models
• you want customizable YAML-based test rules with pass/fail conditions

When to avoid

• you need general-purpose application security scanning beyond LLM prompts
• you require a GUI or SaaS platform rather than a Python CLI
• your target is not an LLM-based application
• you need continuous runtime monitoring rather than point-in-time scanning

Facets

cli-tool · maturity active

security penetration-testing vulnerability-scanning llm-inference prompt-engineering cli security artificial-intelligence large-language-models developer-tools penetration-testing python cli cross-platform prompt-injection llm-security jailbreak-testing white-box-testing black-box-testing ai-red-teaming ollama yaml-rules

Member repositories

For agents

markdown · JSON · MCP: product_card(name="utkusen/promptmap")