# dphnAI/sonar

Large-scale LLM inference engine

Repository: https://github.com/dphnAI/sonar
Canonical: https://ross.abutalabs.com/products/sonar
Homepage: https://sonar.dphn.ai
Language: C++
License: AGPL-3.0
License Family: copyleft
Topics: api-rest, inference-engine, machine-learning, cuda, inferentia, rocm, intel, lora, speculative-decoding, tpu
Last push: 2026-08-13T03:29:23+00:00

## Health v2 (maintenance only)
Score: 94/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 97, release rhythm 95, longevity 83
- inputs: {"age_days": 1168, "days_push": 20, "days_rel": 33, "gap_med": 17, "n_releases_24m": 20}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1843, forks 207 (observed 2026-08-28T04:05:43.282700+00:00)

## What it is
Sonar is a large-scale LLM inference engine for Hugging Face-compatible language and multimodal models, based on vLLM (formerly the Aphrodite engine). It provides continuous batching, paged KV-cache, quantization, speculative decoding, LoRA serving, and distributed serving behind OpenAI-compatible APIs.

## Use cases
- serve llm models with an openai-compatible api
- run high-throughput batch inference on gpus
- deploy quantized models on limited vram
- speed up generation with speculative decoding
- serve lora adapters alongside a base model
- run multimodal image audio video model inference
- scale inference across multiple gpus or nodes

## When to choose
- you need production-grade LLM serving with OpenAI-compatible endpoints
- you want vLLM features plus extra quantization formats, sampling methods, and hardware support (ROCm, Intel XPU, TPU, Metal)
- you need multi-node serving, prefix caching, or prefill/decode disaggregation
- you serve LoRA adapters or multimodal models at scale

## When to avoid
- you need native Windows support (only WSL 2 is supported)
- you want a lightweight single-user desktop inference tool rather than a server engine
- you need a permissive license - Sonar is AGPL-3.0
- your use case is training or fine-tuning rather than inference

## Facets
- artifact type: library
- maturity: active
- function: llm-inference, http-server, gpu-computing, rag, chatbot
- domain: large-language-models, machine-learning, artificial-intelligence, developer-tools
- platform: python, cpp, cli
- tags: vllm-fork, openai-compatible-api, speculative-decoding, quantization, continuous-batching, paged-kv-cache, lora, multimodal, aphrodite-engine, distributed-serving, linux, macos, gpu, docker

## Member repositories
- dphnAI/sonar (main) score 94

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:43.282700+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:17:57.600986+00:00, confidence not recorded.
  - readme: https://github.com/dphnAI/sonar (fetched 2026-08-28T04:05:43.282700+00:00, sha 54daa652d5df)
  - homepage: https://sonar.dphn.ai (fetched 2026-08-29T10:56:55.958991+00:00, sha 7097760ee3c7)
  - site_page: https://sonar.dphn.ai/getting-started/installation (fetched 2026-08-29T10:56:55.961646+00:00, sha 21d2d76e959c)
  - site_page: https://sonar.dphn.ai/getting-started/choose-a-path (fetched 2026-08-29T10:56:55.964072+00:00, sha 58eb2fc5aeba)
- Data as of 2026-08-30T08:39:29.467469+00:00.