# datadreamer-dev/DataDreamer

DataDreamer: Prompt. Generate Synthetic Data. Train & Align Models.    🤖💤

Repository: https://github.com/datadreamer-dev/DataDreamer
Canonical: https://ross.abutalabs.com/products/datadreamer
Homepage: https://datadreamer.dev
Language: Python
License: MIT
License Family: permissive
Topics: deep-learning, machine-learning, natural-language-processing, nlp, nlp-library, python, pytorch, transformers, alignment, fine-tuning, gpt, instruction-tuning, llm, llmops, llms, openai, synthetic-data, synthetic-dataset-generation
Last push: 2025-02-02T21:23:50+00:00

## Health v2 (maintenance only)
Score: 33/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 4, release rhythm 40, longevity 84
- inputs: {"age_days": 1188, "days_push": 577, "days_rel": 577, "gap_med": 0, "n_releases_24m": 8}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1117, forks 58 (observed 2026-08-28T04:03:39.069712+00:00)

## What it is
DataDreamer is an open-source Python library for prompting LLMs, generating synthetic datasets, and training or aligning models in reproducible workflows. It provides research-grade building blocks for data generation, distillation, instruction-tuning, and preference alignment.

## Use cases
- generate synthetic training data with an llm
- distill gpt-4 capabilities into a smaller cheaper model
- instruction-tune a large language model
- align a model with human preferences
- augment an existing dataset using llms
- clean or filter a dataset with llm prompts
- bootstrap few-shot examples for prompting
- train a self-improving llm with self-rewarding

## When to choose
- you need reproducible, research-grade synthetic data generation pipelines
- you want to distill a large LLM into a smaller model
- you are doing NLP/ML research involving instruction-tuning or alignment
- you want caching and saved outputs for LLM workflows out of the box

## When to avoid
- you only need a simple OpenAI API wrapper without training features
- you need a production serving/inference platform rather than data and training workflows
- you work outside Python or the PyTorch/Transformers ecosystem

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, llm-training, prompt-engineering, rag, data-generation, etl
- domain: machine-learning, large-language-models, data-science, artificial-intelligence
- platform: python, cross-platform
- tags: synthetic-data, llm-workflows, instruction-tuning, alignment, distillation, pytorch, transformers, openai, research-tools, llmops, natural-language-processing, gpu

## Member repositories
- datadreamer-dev/DataDreamer (main) score 33

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:03:39.069712+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T06:41:53.201547+00:00, confidence not recorded.
  - readme: https://github.com/datadreamer-dev/DataDreamer (fetched 2026-08-28T04:03:39.069712+00:00, sha 248c7326918f)
  - homepage: https://datadreamer.dev (fetched 2026-08-29T12:45:59.739980+00:00, sha f9eb22f1969b)
  - site_page: https://datadreamer.dev/docs/latest/pages/get_started/installation.html (fetched 2026-08-29T12:45:59.749071+00:00, sha 44136fa355b3)
  - site_page: https://datadreamer.dev/docs/latest/pages/get_started/quick_tour/index.html (fetched 2026-08-29T12:45:59.750801+00:00, sha a5da9539f76f)
  - site_page: https://datadreamer.dev/docs/latest/pages/get_started/motivation_and_design.html (fetched 2026-08-29T12:45:59.752934+00:00, sha 35a2f166823a)
  - site_page: https://datadreamer.dev/docs/latest/pages/get_started/quick_tour/abstract_to_tweet.html (fetched 2026-08-29T12:45:59.755490+00:00, sha aa515dd9acfd)
  - site_page: https://datadreamer.dev/docs/latest/pages/get_started/quick_tour/attributed_prompts.html (fetched 2026-08-29T12:45:59.757556+00:00, sha 528c62dec0b1)
  - site_page: https://datadreamer.dev/docs/latest/pages/get_started/quick_tour/openai_distillation.html (fetched 2026-08-29T12:45:59.759216+00:00, sha f455d2b07ca0)
  - site_page: https://datadreamer.dev/docs/latest/pages/get_started/quick_tour/dataset_augmentation.html (fetched 2026-08-29T12:45:59.760744+00:00, sha f95b33c9b937)
  - site_page: https://datadreamer.dev/docs/latest/pages/get_started/quick_tour/dataset_cleaning.html (fetched 2026-08-29T12:45:59.762278+00:00, sha fc5a838745a8)
- Data as of 2026-08-30T08:39:29.467469+00:00.