TMElyralab/MuseTalk

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

6459 stars · 931 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

MuseTalk is a real-time, high-fidelity lip-sync model that modifies a face region in video according to input audio via latent space inpainting. It runs at 30fps+ on an NVIDIA V100 and supports multiple languages, with inference, training code, and model weights available.

Use cases

• synchronize a person's lips in a video to new audio
• dub videos into other languages with matching lip movement
• build a talking virtual human avatar from a video
• real-time lip-sync for live streaming avatars
• retime an actor's mouth to a different voice track
• generate lip-synced video from text-to-speech audio

When to choose

• you need real-time (30fps+) lip-sync inference on a GPU
• you want to dub video into Chinese, English, or Japanese with accurate mouth movement
• you need both inference and training code for a lip-sync model
• you're building a virtual human pipeline alongside video generation tools

When to avoid

• you need full-face reenactment or head pose control, not just the mouth region
• you have no GPU available for inference
• you need a permissive license - the license is custom/non-standard
• you need 4K or full-frame generation rather than a 256x256 face region

Facets

library · maturity active

video-processing machine-learning deep-learning speech-recognition image-processing artificial-intelligence deep-learning media python cli lip-sync talking-head video-dubbing virtual-human audio-driven latent-space-inpainting real-time-inference video gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="TMElyralab/MuseTalk")