ZiqiaoPeng/SyncTalk

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1626 stars · 196 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

SyncTalk is the official PyTorch implementation of a CVPR 2024 paper that synthesizes speech-driven, synchronized talking head videos using NeRF-based tri-plane hash representations. It generates lip-synced speech, facial expressions, stable head poses, and high-resolution hair and torso details from a portrait image and audio input.

Use cases

• generate a talking head video from a photo and audio clip
• synthesize lip-synced avatar videos from speech
• research code for audio-driven talking face generation
• create high-fidelity digital human videos
• reproduce CVPR 2024 talking head synthesis results
• animate a portrait with realistic head poses and expressions

When to choose

• you need research-grade, synchronized talking head synthesis with identity preservation
• you want NeRF-based 3D-aware portrait animation with expressions and head pose
• you are reproducing or building on the SyncTalk paper

When to avoid

• you need a production-ready, easy-to-deploy avatar API rather than research code
• you lack a CUDA GPU or cannot handle a heavy PyTorch/CUDA setup
• you need real-time 2D generation - the author's SyncTalk_2D may fit better

Facets

library · maturity active

machine-learning deep-learning video-processing audio-processing speech-recognition computer-vision computer-vision deep-learning artificial-intelligence windows python talking-head-synthesis nerf audio-driven face-generation cvpr-2024 research-code neural-rendering audio video linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ZiqiaoPeng/SyncTalk")