HVision-NKU/StoryDiffusion

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

6452 stars · 641 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

StoryDiffusion is the official implementation of a NeurIPS 2024 Spotlight paper introducing Consistent Self-Attention for character-consistent long-range image generation and a motion predictor for long video generation. It plugs into SD1.5 and SDXL-based diffusion models to generate consistent image sequences, comics, and videos from text prompts.

Use cases

• generate consistent comic strips from text prompts
• create character-consistent image sequences with stable diffusion
• generate long videos from a sequence of images
• make a visual story with the same character across many images
• add consistent self-attention to my SDXL pipeline
• turn text prompts into an illustrated storybook

When to choose

• you need the same character to appear consistently across many generated images
• you want to build comics or visual stories from text prompts
• you want to extend consistent images into long videos
• you already use SD1.5 or SDXL pipelines and want a hot-pluggable consistency module

When to avoid

• you need single unrelated images with no character consistency
• you lack a GPU or the compute for diffusion model inference
• you need production-grade video generation rather than a research implementation
• you need fine-grained control over video motion beyond the motion predictor

Facets

library · maturity active

image-processing video-processing machine-learning deep-learning artificial-intelligence image-processing deep-learning python diffusion-models stable-diffusion text-to-image image-to-video character-consistency story-generation comics neurips-2024 video gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="HVision-NKU/StoryDiffusion")