YiVal/YiVal

Health v2 · maintenance only

Adoption not part of the score

2133 stars · 329 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

YiVal is an open-source Python framework that automatically tunes prompts, RAG configurations, and model parameters for generative AI applications using a data-driven, evaluation-centric approach. It replaces manual prompt iteration with automated experiments that optimize for quality, latency, and inference cost.

Use cases

• automatically improve my LLM prompts instead of guessing
• evaluate and compare different prompt variants for my GenAI app
• tune RAG configuration parameters with real metrics
• reduce GPT-4 inference costs by finding cheaper model configs with similar quality
• run A/B experiments on prompt engineering for my chatbot
• find the right metrics and evaluators for my LLM use case
• fine-tune model parameters without deep ML expertise

When to choose

• you are building LLM or GenAI applications and want data-driven prompt optimization
• you need systematic evaluation of prompts, RAG settings, or model configs
• you want to cut inference costs while maintaining output quality
• you prefer a Python framework with notebook tutorials and Docker deployment

When to avoid

• you need a production-hardened, actively maintained tool - the project appears to have stalled after early 2024
• you only need simple one-off prompt testing without evaluation pipelines
• your stack is not Python-based
• you need a managed commercial platform rather than a self-run framework

Facets

framework · maturity experimental

prompt-engineering machine-learning llm-inference rag benchmarking testing llm-training large-language-models artificial-intelligence machine-learning developer-tools python cross-platform cli prompt-tuning auto-prompting genai evaluation llm-ops experiment-framework aigc retrieval-augmented-generation docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="YiVal/YiVal")