datadreamer-dev/DataDreamer

Health v2 · maintenance only

Adoption not part of the score

1117 stars · 58 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

DataDreamer is an open-source Python library for prompting LLMs, generating synthetic datasets, and training or aligning models in reproducible workflows. It provides research-grade building blocks for data generation, distillation, instruction-tuning, and preference alignment.

Use cases

• generate synthetic training data with an llm
• distill gpt-4 capabilities into a smaller cheaper model
• instruction-tune a large language model
• align a model with human preferences
• augment an existing dataset using llms
• clean or filter a dataset with llm prompts
• bootstrap few-shot examples for prompting
• train a self-improving llm with self-rewarding

When to choose

• you need reproducible, research-grade synthetic data generation pipelines
• you want to distill a large LLM into a smaller model
• you are doing NLP/ML research involving instruction-tuning or alignment
• you want caching and saved outputs for LLM workflows out of the box

When to avoid

• you only need a simple OpenAI API wrapper without training features
• you need a production serving/inference platform rather than data and training workflows
• you work outside Python or the PyTorch/Transformers ecosystem

Facets

library · maturity active

machine-learning llm-training prompt-engineering rag data-generation etl machine-learning large-language-models data-science artificial-intelligence python cross-platform synthetic-data llm-workflows instruction-tuning alignment distillation pytorch transformers openai research-tools llmops natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="datadreamer-dev/DataDreamer")