ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework observed · 2026-08-28
Health v2 · maintenance only
87/100
- Activity 99
- Release rhythm 99
- Longevity 39
Flags: no_license
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: 14.5
- age_days: 548
- days_rel: 11
- days_push: 7
- n_releases_24m: 31
Adoption not part of the score
7872 stars · 1496 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded
NVIDIA Dynamo is an open-source, datacenter-scale distributed inference serving framework that orchestrates inference engines like vLLM, SGLang, and TensorRT-LLM into coordinated multi-node systems. It provides disaggregated serving, KV-aware routing, multi-tier KV caching, and autoscaling, deployable on Kubernetes, Slurm, or locally.
Use cases
- serve LLMs at datacenter scale across multiple nodes
- run disaggregated prefill and decode serving for lower latency
- route requests with KV-cache-aware routing to maximize cache reuse
- offload and manage KV cache across memory tiers
- autoscale inference workers on Kubernetes
- deploy models with vLLM, SGLang, or TensorRT-LLM backends
- serve multimodal and video generation workloads
- benchmark inference performance with AIPerf
When to choose
- you need multi-node, datacenter-scale LLM inference with high throughput
- you want disaggregated prefill/decode serving with intelligent routing
- you deploy inference on Kubernetes or Slurm with NVIDIA, AMD, or Intel accelerators
- you want an orchestration layer on top of existing inference engines rather than replacing them
When to avoid
- you just need to serve a single model on one GPU without distributed orchestration
- you need a simple inference server with minimal operational complexity
- your stack doesn't use supported engines like vLLM, SGLang, or TensorRT-LLM
Facets
framework · maturity active
llm-inference routing caching deployment microservices api-gateway monitoring benchmarking large-language-models artificial-intelligence gpu-computing microservices cloud-computing python rust cli self-hosted inference-serving disaggregated-serving kv-cache vllm sglang tensorrt-llm nvidia autoscaling kv-aware-routing prefill-decode-disaggregation containers devops kubernetes docker linux gpu
9 sources
- readme: https://github.com/ai-dynamo/dynamo · fetched 2026-08-28 · e8da334fcb86
- homepage: https://docs.nvidia.com/dynamo/latest · fetched 2026-08-29 · 23d31e034815
- site_page: https://docs.nvidia.com/dynamo/kubernetes/getting-started/introduction · fetched 2026-08-29 · df7a8511c8ad
- site_page: https://docs.nvidia.com/dynamo/cli/getting-started/introduction · fetched 2026-08-29 · bb981bcd1d9c
- site_page: https://www.nvidia.com/en-us/about-nvidia/privacy-policy · fetched 2026-08-29 · 5362c58d0750
- site_page: https://www.nvidia.com/en-us/about-nvidia/privacy-center · fetched 2026-08-29 · b098377da9cc
- site_page: https://www.nvidia.com/en-us/about-nvidia/terms-of-service · fetched 2026-08-29 · 85469b6ff0a1
- site_page: https://www.nvidia.com/en-us/about-nvidia/accessibility · fetched 2026-08-29 · 8f61ce4143ec
- site_page: https://www.nvidia.com/en-us/about-nvidia/company-policies · fetched 2026-08-29 · b169611fd7d6
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| ai-dynamo/dynamo | main | 87 |
For agents
markdown · JSON · MCP: product_card(name="ai-dynamo/dynamo")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem