argilla-io/argilla

Health v2 · maintenance only

Adoption not part of the score

5085 stars · 503 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Argilla is an open-source collaboration tool for AI engineers and domain experts to build, annotate, and curate high-quality datasets for NLP, LLM, and multimodal projects. It combines a Python SDK, a FastAPI server backed by Elasticsearch/OpenSearch, and a Vue.js annotation UI, deployable via Docker or Hugging Face Spaces.

Use cases

• annotate text classification and NER datasets with domain experts
• collect human preference data for RLHF and LLM fine-tuning
• review and evaluate RAG system outputs
• build training datasets with human-in-the-loop workflows
• run active learning and weak supervision pipelines
• log model predictions for continuous evaluation
• self-host a data annotation platform for an ML team

When to choose

• you need a free, self-hosted annotation tool with full data ownership
• you want programmatic dataset workflows via a Python SDK rather than manual labeling
• you're collecting human feedback for LLM tuning, RAG evaluation, or classic NLP tasks
• you want one-click deployment on Hugging Face Spaces

When to avoid

• you need active feature development or new capabilities - the original team has moved on and only bug fixes are published
• you need synthetic data generation - use the companion distilabel library instead
• you need model training - Argilla only manages data, not training
• you need a fully managed commercial annotation workforce platform

Facets

application · maturity maintenance

data-science machine-learning nlp rag llm-training self-hosted sdk machine-learning data-science large-language-models developer-tools self-hosted python self-hosted cloud data-annotation human-in-the-loop text-labeling rlhf active-learning weak-supervision dataset-curation hugging-face vuejs-ui fastapi annotation natural-language-processing retrieval-augmented-generation docker web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="argilla-io/argilla")