vLLM
A high-throughput and memory-efficient inference and serving engine for LLMs observed · 2026-10-18
Health v2 · maintenance only
98/100
- Activity 100
- Release rhythm 98
- Longevity 96
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-10-21. Adoption (stars, forks) is never an input.
- gap_med: 11.5
- age_days: 1349
- days_rel: 15
- days_push: 3
- n_releases_24m: 57
Adoption not part of the score
94014 stars · 23509 forks observed · 2026-10-18
What it is AI-extracted, prompt v1, taxonomy v1, 2026-09-21, confidence 0.99
vLLM is a Python library that provides fast, high‑throughput inference and serving for large language models. It supports advanced features such as paged attention, speculative decoding, and multi‑GPU/CPU deployment.
Use cases
- Serve LLMs with low latency for chat applications
- Run batch inference on GPUs for research experiments
- Deploy an OpenAI‑compatible API for custom models
- Perform quantized inference to reduce memory usage
- Stream generated text in real time to end users
- Scale LLM inference across multiple nodes in a cluster
When to choose
- You need state‑of‑the‑art throughput and low latency for LLM serving
- You want seamless integration with Hugging Face models and OpenAI‑compatible APIs
- Your workload benefits from advanced features like speculative decoding or prefix caching
- You require support for multiple hardware backends (NVIDIA, AMD, Intel, CPU)
When to avoid
- You need a pure JavaScript or browser‑only solution
- Your inference workload is tiny and does not justify the overhead of a high‑performance engine
- You require a fully managed SaaS offering rather than self‑hosted software
Facets
library · maturity active
llm-inference api-framework http-server serialization streaming large-language-models artificial-intelligence machine-learning python windows cloud high-throughput memory-efficient quantization openai-compatible distributed-inference gpu-accelerated prefill-caching linux macos docker
1 source
- readme: https://github.com/vllm-project/vllm · fetched 2026-10-18 · 292534695eff
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| vllm-project/vllm | main | 98 |
| vllm-project/vllm-omni | plugin | 84 |
| vllm-project/vllm-ascend | plugin | 83 |
| vllm-project/router | plugin | 62 |
| vllm-project/compressed-tensors | infra | 90 |
| vllm-project/agentic-api | plugin | 81 |
| vllm-project/afd-plugin | plugin | 78 |
| vllm-project/vllm-skills | plugin | 46 |
| vllm-project/vllm-xpu-kernels | plugin | 84 |
| vllm-project/vllm-gaudi | plugin | 82 |
| vllm-project/ci-infra | infra | 70 |
| vllm-project/vllm-bench | infra | 10 |
For agents
markdown · JSON · MCP: product_card(name="vllm-project/vllm")
Data as of 2026-10-19T08:05:51.212177+00:00 · Report a problem