ali-vilab/dreamtalk

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1787 stars · 216 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

DreamTalk is the official implementation of a diffusion-based framework for generating expressive, audio-driven talking head videos from a single portrait image. It combines a denoising network, a style-aware lip expert, and a style predictor to produce high-quality talking faces across diverse speaking styles, languages, and out-of-domain portraits.

Use cases

• generate a talking head video from a portrait and audio clip
• animate a face photo to match speech audio
• create lip-synced avatar videos with different speaking styles
• make a portrait sing along to a song
• research expressive talking head generation with diffusion models
• transfer head pose and speaking style from a reference video to a new face

When to choose

• you need audio-driven expressive face animation from a single portrait
• you want research-grade talking head generation with style control via 3DMM parameters
• your inputs include songs, multilingual speech, or noisy audio
• you are doing academic research on diffusion-based face generation

When to avoid

• you need production deployment - checkpoints require an email request restricted to academic research
• you want a plug-and-play tool without GPU setup and 3DMM preprocessing pipelines
• you need real-time or low-latency avatar generation
• you require commercial usage rights

Facets

library · maturity maintenance

deep-learning video-processing audio-processing machine-learning computer-vision deep-learning artificial-intelligence media python talking-head-generation diffusion-models face-animation audio-driven lip-sync research-code video-generation linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ali-vilab/dreamtalk")