NVIDIA/garak

Health v2 · maintenance only

Adoption not part of the score

9033 stars · 1224 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

garak is a command-line LLM vulnerability scanner that probes large language models for failures like hallucination, data leakage, prompt injection, toxicity, and jailbreaks. It supports many model backends including Hugging Face, OpenAI, AWS Bedrock, LiteLLM, and generic REST endpoints.

Use cases

• scan an LLM for prompt injection vulnerabilities
• red-team a chatbot before deployment
• test if a model leaks training data
• check an LLM for jailbreak susceptibility
• evaluate hallucination and toxicity in a generative model
• security-assess models served via OpenAI-compatible APIs

When to choose

• you need automated adversarial probing of an LLM or dialog system
• you want a broad, extensible suite of LLM vulnerability probes from the CLI
• you need to test models across Hugging Face, OpenAI, Bedrock, or REST endpoints

When to avoid

• you need static code security scanning rather than LLM behavior testing
• you want a GUI-based evaluation dashboard
• you need formal ML benchmarking rather than security-focused failure probing

Facets

cli-tool · maturity active

security vulnerability-scanning penetration-testing llm-inference testing cli security large-language-models artificial-intelligence developer-tools windows python cli llm-security red-teaming prompt-injection jailbreaks llm-evaluation nvidia command-line linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="NVIDIA/garak")