kyutai-labs/delayed-streams-modeling

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

3017 stars · 313 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Kyutai's repository of Speech-To-Text and Text-To-Speech models built on the Delayed Streams Modeling framework, with implementations in PyTorch, Rust, and MLX. The models support streaming, real-time inference with word-level timestamps and semantic voice activity detection, suitable for voice agents.

Use cases

• transcribe live audio streams in real time
• build a voice agent with speech-to-text and text-to-speech
• run speech recognition on-device on a Mac or iPhone
• serve many concurrent transcription streams over websockets
• convert text to natural-sounding speech
• get word-level timestamps from spoken audio

When to choose

• you need low-latency streaming STT or TTS rather than batch transcription
• you want to build interactive voice AI applications or agents
• you need efficient batching of many simultaneous audio streams on GPU
• you want on-device inference on Apple silicon via MLX

When to avoid

• you only need offline batch transcription of recorded files with no latency constraints
• you need STT for languages beyond English and French
• you cannot run GPU inference and need a lightweight CPU-only solution
• you need a fully managed cloud speech API rather than self-hosted models

Facets

library · maturity active

speech-recognition tts machine-learning llm-inference speech-processing artificial-intelligence python rust cli streaming-inference voice-agents word-level-timestamps voice-activity-detection mlx pytorch websocket-server real-time-transcription audio macos ios gpu web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="kyutai-labs/delayed-streams-modeling")