ictnlp/StreamSpeech

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1287 stars · 105 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

StreamSpeech is an 'All in One' seamless model for offline and simultaneous speech recognition, speech translation, and speech synthesis, presented in an ACL 2024 paper. It performs streaming ASR, simultaneous speech-to-text and speech-to-speech translation with a single multi-task model, showing intermediate results in real time.

Use cases

• translate speech to speech in real time
• streaming speech recognition while audio is being spoken
• simultaneous speech-to-text translation
• build a low-latency voice interpreter
• research on simultaneous speech-to-speech translation models
• show live ASR and translation transcripts during a call

When to choose

• you need state-of-the-art simultaneous (streaming) speech-to-speech translation
• you want one model handling ASR, translation, and synthesis together
• you are doing research on low-latency speech translation

When to avoid

• you need a production-ready hosted translation API
• you only need simple offline batch transcription with mature tooling
• you lack GPU resources for large speech models

Facets

library · maturity active

speech-recognition tts nlp machine-learning audio-processing speech-processing machine-learning artificial-intelligence python simultaneous-translation speech-to-speech-translation streaming asr tts multi-task-learning research-code acl-2024 natural-language-processing linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ictnlp/StreamSpeech")