# BayLing-Models/BayLing-Speech

LLaMA-Omni is a low-latency and high-quality end-to-end speech interaction model built upon Llama-3.1-8B-Instruct, aiming to achieve speech capabilities at the GPT-4o level.

Repository: https://github.com/BayLing-Models/BayLing-Speech
Canonical: https://ross.abutalabs.com/products/bayling-speech
Homepage: https://arxiv.org/abs/2409.06666
Language: Python
License: Apache-2.0
License Family: permissive
Topics: large-language-models, multimodal-large-language-models, speech-language-model, speech-to-speech, speech-to-text, speech-interaction
Last push: 2025-05-19T02:24:42+00:00

## Health v2 (maintenance only)
Score: 32/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 22, release rhythm 35, longevity 51
- inputs: {"age_days": 722, "days_push": 472, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3146, forks 225 (observed 2026-08-28T04:07:46.014306+00:00)

## What it is
LLaMA-Omni is an end-to-end speech interaction model built on Llama-3.1-8B-Instruct that generates simultaneous text and speech responses from speech instructions with latency as low as 226ms. It combines a pretrained speech encoder, speech adaptor, LLM, and streaming speech decoder, trained on the InstructS2S-200K dataset.

## Use cases
- build a voice assistant that talks to an LLM in real time
- generate speech and text responses from spoken instructions
- run a low-latency speech-to-speech conversation model locally
- fine-tune an open-source LLM with speech capabilities
- research speech-language model architectures
- create a hands-free voice chatbot without transcription

## When to choose
- you need open-source real-time voice interaction with an LLM
- you want simultaneous text and speech output from speech input
- you need low-latency speech responses without ASR transcription
- you're researching speech-language models on Llama backbones

## When to avoid
- you only need text-based chat without audio
- you need a production-ready hosted voice API rather than a research model
- you lack GPU resources for 8B-parameter inference
- you need non-English speech interaction

## Facets
- artifact type: library
- maturity: active
- function: speech-recognition, tts, llm-inference, machine-learning
- domain: speech-processing, large-language-models, artificial-intelligence
- platform: python, cli
- tags: speech-language-model, speech-to-speech, multimodal, llama, voice-assistant, streaming-speech-decoder, low-latency, natural-language-processing, gpu, linux

## Member repositories
- BayLing-Models/BayLing-Speech (main) score 32

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:46.014306+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:25:56.958355+00:00, confidence not recorded.
  - readme: https://github.com/BayLing-Models/BayLing-Speech (fetched 2026-08-28T04:07:46.014306+00:00, sha 8e0d161302a7)
  - homepage: https://arxiv.org/abs/2409.06666 (fetched 2026-08-29T09:40:35.626375+00:00, sha fbb5aed1e97b)
  - site_page: https://info.arxiv.org/about/donate.html (fetched 2026-08-29T09:40:35.629074+00:00, sha cca9c3a11c56)
  - site_page: https://info.arxiv.org/about/ourmembers.html (fetched 2026-08-29T09:40:35.632679+00:00, sha 47cbc55ff1de)
  - site_page: https://info.arxiv.org/about (fetched 2026-08-29T09:40:35.634500+00:00, sha a1f16f915a9a)
  - site_page: https://info.arxiv.org/labs/index.html (fetched 2026-08-29T09:40:35.630994+00:00, sha b14a8d05a0ec)
- Data as of 2026-08-30T08:39:29.467469+00:00.