lucidrains/soundstorm-pytorch

Health v2 · maintenance only

Adoption not part of the score

1546 stars · 94 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch implementation of SoundStorm, Google DeepMind's efficient parallel audio generation model that applies MaskGiT-style masked generation to residual vector-quantized audio codes using a Conformer transformer. It is a research library for training and generating audio non-autoregressively in a small number of steps.

Use cases

• generate audio in parallel with a non-autoregressive transformer
• train a SoundStorm model on residual VQ audio codes
• implement MaskGiT-style iterative demasking for audio tokens
• build a text-to-speech pipeline with SoundStream and semantic TTS models
• experiment with Conformer architectures for audio token generation

When to choose

• you want an open PyTorch implementation of the SoundStorm paper
• you need fast parallel audio generation instead of slow autoregressive decoding
• you already have SoundStream/EnCodec codes or want to train end-to-end on raw audio

When to avoid

• you need a production-ready, polished TTS service with pretrained voices
• you want a simple inference-only text-to-speech tool without training
• you need non-Python environments or non-PyTorch frameworks

Facets

library · maturity active

machine-learning deep-learning audio-processing tts artificial-intelligence deep-learning speech-processing python audio-generation soundstorm maskgit conformer non-autoregressive pytorch research-implementation audio

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lucidrains/soundstorm-pytorch")