Giskard-AI/giskard-oss

Health v2 · maintenance only

Adoption not part of the score

5773 stars · 523 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Giskard is an open-source Python library for testing, evaluating, and red-teaming LLM-based agents and RAG systems. Its v3 rewrite provides modular packages for checks, vulnerability scanning, and dynamic multi-turn agent testing.

Use cases

• evaluate llm agents automatically
• red team my chatbot for vulnerabilities
• test rag pipeline quality
• scan llm app for security issues
• generate test cases for ai agents
• benchmark llm outputs for hallucinations

When to choose

• you need programmatic, Python-based evaluation of LLM agents or RAG systems
• you want automated AI vulnerability scanning and red teaming in CI
• you prefer a lightweight, modular, async-first testing library

When to avoid

• you need a GUI-based enterprise testing platform with team collaboration (consider Giskard Hub)
• you need to test tabular/ML models - that legacy scan is v2-only and unmaintained
• your environment is below Python 3.12

Facets

library · maturity active

testing llm-inference rag agent-framework security benchmarking artificial-intelligence large-language-models machine-learning testing security python cli llm-evaluation red-teaming ai-safety llmops vulnerability-scanning rag-evaluation agent-testing responsible-ai ai-agents retrieval-augmented-generation

Member repositories

For agents

markdown · JSON · MCP: product_card(name="Giskard-AI/giskard-oss")