hkchengrex/MMAudio

Health v2 · maintenance only

Adoption not part of the score

2264 stars · 266 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

MMAudio is a PyTorch-based model for generating synchronized audio from video and/or text inputs, using multimodal joint training across audio-visual and audio-text datasets. It includes a synchronization module that aligns generated audio with video frames, published as a CVPR 2025 paper.

Use cases

• generate sound effects for silent videos
• add synchronized audio to AI-generated video from Sora or Veo
• synthesize audio from a text prompt
• dub foley sounds onto video footage automatically
• research video-to-audio synthesis models

When to choose

• you need to generate audio tracks synced to video content
• you want state-of-the-art video-to-audio or text-to-audio synthesis
• you have a CUDA GPU and want to run inference locally or via demo services

When to avoid

• you need real-time audio generation on CPU-only hardware
• you need production audio editing rather than generative synthesis
• you require Windows or macOS support, since only Ubuntu is tested

Facets

library · maturity active

audio-processing machine-learning deep-learning video-processing artificial-intelligence deep-learning machine-learning python video-to-audio text-to-audio audio-synthesis generative-ai cvpr-2025 pytorch audio video linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="hkchengrex/MMAudio")