# YangLing0818/RPG-DiffusionMaster

[ICML 2024] Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (RPG)

Repository: https://github.com/YangLing0818/RPG-DiffusionMaster
Canonical: https://ross.abutalabs.com/products/rpg-diffusionmaster
Homepage: https://proceedings.mlr.press/v235/yang24ai.html
Language: Jupyter Notebook
License: MIT
License Family: permissive
Topics: large-language-models, multimodal-large-language-models, image-editting, text-to-image
Last push: 2025-02-01T13:28:48+00:00

## Health v2 (maintenance only)
Score: 28/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 4, release rhythm 35, longevity 68
- inputs: {"age_days": 955, "days_push": 578, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1842, forks 100 (observed 2026-08-28T04:05:43.215607+00:00)

## What it is
Official implementation of RPG (Recaption, Plan, Generate), a training-free framework that uses multimodal LLMs as prompt recaptioners and region planners to improve compositional text-to-image generation and editing with diffusion models. It supports proprietary and open-source MLLMs and arbitrary diffusion backbones, enabling high-resolution, multi-object image generation.

## Use cases
- generate complex multi-object images from long text prompts
- improve compositional text-to-image generation with diffusion models
- edit images with text-guided closed-loop generation
- generate super high resolution images with regional diffusion
- use GPT-4 or local MLLMs to plan image subregions
- research training-free prompt planning for diffusion models

## When to choose
- you need better multi-object composition and text-image alignment than vanilla Stable Diffusion or SDXL
- you want a training-free framework compatible with various MLLMs and diffusion backbones
- you need high-resolution or region-wise controlled image generation and editing

## When to avoid
- you need a production-ready end-user image generation app rather than research code
- you cannot access an MLLM API or run a local multimodal model
- you need fast single-prompt generation without LLM planning overhead

## Facets
- artifact type: library
- maturity: active
- function: llm-inference, image-processing, prompt-engineering, machine-learning
- domain: artificial-intelligence, image-processing, large-language-models, deep-learning
- platform: python, cross-platform
- tags: text-to-image, diffusion-models, multimodal-llm, stable-diffusion, image-editing, research-code, icml-2024, training-free, gpu

## Member repositories
- YangLing0818/RPG-DiffusionMaster (main) score 28

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:43.215607+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:17:58.956067+00:00, confidence not recorded.
  - readme: https://github.com/YangLing0818/RPG-DiffusionMaster (fetched 2026-08-28T04:05:43.215607+00:00, sha 16ff95d54381)
  - homepage: https://proceedings.mlr.press/v235/yang24ai.html (fetched 2026-08-29T10:57:17.289319+00:00, sha aec34a95f56b)
  - site_page: https://proceedings.mlr.press/faq.html (fetched 2026-08-29T10:57:17.292469+00:00, sha 8708020cd335)
- Data as of 2026-08-30T08:39:29.467469+00:00.