microsoft/VibeVoice

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

53225 stars · 6004 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

VibeVoice is Microsoft's open-source frontier voice AI framework combining a next-token diffusion text-to-speech model for expressive, long-form, multi-speaker conversational audio (up to 90 minutes, 4 speakers) with VibeVoice-ASR, a unified speech-to-text model for hour-long transcriptions with speaker and timestamp annotations. It is distributed as Python models on Hugging Face and PyPI, with finetuning code and an edge CPU inference engine.

Use cases

• generate a multi-speaker podcast from a text script
• synthesize long-form conversational speech with consistent voices
• transcribe hour-long audio with speaker labels and timestamps
• convert text to expressive speech with emotion and singing
• cross-lingual speech generation between Mandarin and English
• run speech recognition on CPU at the edge
• finetune a TTS or ASR model on custom voice data

When to choose

• you need long-form multi-speaker speech synthesis beyond typical 1-2 speaker TTS limits
• you want open-source, MIT-licensed state-of-the-art TTS or ASR models
• you need structured transcriptions with speaker and timestamp metadata
• you want to finetune speech models on your own data

When to avoid

• you need a lightweight production TTS API with minimal compute
• you require strict guarantees against voice cloning misuse
• you need real-time low-latency streaming synthesis on modest hardware
• you only need simple single-phrase text-to-speech

Facets

library · maturity active

tts speech-recognition machine-learning llm-inference speech-processing artificial-intelligence deep-learning python cross-platform text-to-speech speech-synthesis asr multi-speaker podcast-generation diffusion voice-ai long-form-audio audio gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/VibeVoice")