huggingface/speech-to-speech

Health v2 · maintenance only

Adoption not part of the score

12882 stars · 1595 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A low-latency, fully modular voice-agent pipeline (VAD -> STT -> LLM -> TTS) built from open-source models, exposed via the OpenAI Realtime event set over WebSocket and WebRTC. Every component is swappable, supporting hosted providers or fully local stacks with vLLM or llama.cpp.

Use cases

• build a voice assistant with open-source models
• run a fully local speech-to-speech pipeline
• serve an OpenAI Realtime-compatible voice agent endpoint
• swap STT, LLM, and TTS components in a voice pipeline
• add conversational voice backend to a robot
• translate speech to speech in real time

When to choose

• you want a modular, swappable voice-agent pipeline with open-source models
• you need OpenAI Realtime API compatibility without OpenAI
• you want a fully local, self-hosted voice stack
• you need low-latency voice conversation for robots or apps

When to avoid

• you only need simple text-based chatbots without audio
• you want a turnkey closed-source commercial voice product
• you need non-Python tooling or managed-only deployment

Facets

framework · maturity active

speech-recognition tts llm-inference websocket agent-framework chatbot speech-processing artificial-intelligence large-language-models python cross-platform self-hosted voice-agents speech-to-speech vad openai-realtime-api webrtc low-latency modular-pipeline ai-agents voice web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="huggingface/speech-to-speech")