lucidrains/video-diffusion-pytorch

Health v2 · maintenance only

Adoption not part of the score

1383 stars · 139 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Pytorch implementation of the Video Diffusion Models paper by Jonathan Ho, extending denoising diffusion probabilistic models (DDPMs) to video generation using a space-time factored 3D U-net. It supports unconditional and text-conditioned (BERT embeddings) video synthesis for research experimentation.

Use cases

• generate short videos from text descriptions
• train a diffusion model on video datasets
• reproduce the Video Diffusion Models paper
• experiment with space-time factored U-nets for video synthesis
• generate moving MNIST-style video samples
• build a research prototype for text-to-video generation

When to choose

• you want a lightweight, readable Pytorch implementation of video diffusion for research
• you need a starting codebase to extend with newer text-to-video techniques
• you want to train small-scale video diffusion models on custom datasets

When to avoid

• you need a production-ready text-to-video system with high-quality output
• you want a pretrained model that generates realistic videos out of the box
• you need large-scale, well-supported training infrastructure or long video generation

Facets

library · maturity experimental

deep-learning machine-learning video-processing deep-learning artificial-intelligence python diffusion-models ddpm text-to-video video-generation pytorch research-code unet video gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lucidrains/video-diffusion-pytorch")