hexo-ai/sia

Health v2 · maintenance only

Flags: no_releases young

Adoption not part of the score

2123 stars · 254 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

SIA is a Python framework implementing a self-improving AI loop in which a Meta-Agent generates a task-specific Target Agent and a Feedback Agent iteratively updates both the agent's harness and model weights to improve benchmark performance. It is the official open-source implementation of the SIA paper (Hebbar et al., 2026) and reports state-of-the-art results on benchmarks like MLE-Bench, LawBench, and CUDA kernel optimization.

Use cases

• automatically improve an AI agent's performance on a benchmark task
• run a self-improving loop that updates agent harness and weights
• optimize LLM agents for Kaggle-style ML competitions
• improve model performance on scientific tasks like RNA denoising
• generate and refine task-specific agents autonomously
• benchmark self-improving agent systems

When to choose

• you want an agent system that autonomously improves itself over successive generations
• you need to optimize a model or agent against a measurable benchmark task
• you are researching self-improving or meta-learning agent architectures
• you want to reproduce or build on the SIA paper's results

When to avoid

• you need a simple, static agent pipeline without iterative self-modification
• you lack the compute budget for repeated training and evaluation loops
• you need a production-ready, battle-tested framework rather than a research system
• your task has no clear automated evaluation signal for the feedback loop

Facets

framework · maturity active

agent-framework llm-training machine-learning benchmarking artificial-intelligence machine-learning large-language-models python self-improving-ai meta-agent feedback-loop agent-optimization llm-agents autonomous-agents research ai-agents linux macos gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="hexo-ai/sia")