# vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

Repository: https://github.com/vllm-project/vllm-ascend
Canonical: https://ross.abutalabs.com/products/vllm-ascend
Homepage: https://docs.vllm.ai/projects/ascend
Language: C++
License: Apache-2.0
License Family: permissive
Topics: ascend, inference, llm, llm-serving, llmops, mlops, model-serving, transformer, vllm
Last push: 2026-08-26T14:58:16+00:00

## Health v2 (maintenance only)
Score: 83/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 86, longevity 41
- inputs: {"age_days": 581, "days_push": 7, "days_rel": 17, "gap_med": 90.0, "n_releases_24m": 7}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2711, forks 2121 (observed 2026-08-28T04:07:11.785242+00:00)

## What it is
vllm-ascend is a community-maintained hardware plugin that enables vLLM to run large language model inference on Huawei Ascend NPUs. It implements the vLLM hardware-pluggable interface, supporting Transformer, MoE, embedding, and multi-modal models on Atlas hardware.

## Use cases
- serve llm inference on huawei ascend npu
- run vllm on atlas 800 hardware
- deploy deepseek or qwen models on ascend npus
- set up prefill-decode disaggregation with mooncake
- run multi-node distributed inference with ray on npus
- accelerate decoding with suffix speculative decoding
- serve multimodal llms on non-gpu accelerators

## When to choose
- you have Ascend NPU hardware (Atlas 800 A2/A3, Atlas 300I DUO) and want vLLM serving
- you need PD disaggregation, expert parallelism, or multi-node inference on Ascend
- you want the officially recommended Ascend backend for vLLM

## When to avoid
- you are running on NVIDIA/AMD GPUs - plain vLLM already supports those natively
- you cannot install the required CANN/TorchNPU software stack
- you need Triton-based features, which are not supported on this backend

## Facets
- artifact type: plugin
- maturity: active
- function: llm-inference, plugin-system, machine-learning
- domain: large-language-models, artificial-intelligence, gpu-computing, developer-tools
- platform: python, cpp
- tags: vllm, ascend-npu, huawei, hardware-plugin, model-serving, torch-npu, pd-disaggregation, speculative-decoding, linux, docker, gpu

## Member repositories
- vllm-project/vllm-ascend (main) score 83

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:11.785242+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:15:35.392827+00:00, confidence not recorded.
  - readme: https://github.com/vllm-project/vllm-ascend (fetched 2026-08-28T04:07:11.785242+00:00, sha 25387d25474b)
  - homepage: https://docs.vllm.ai/projects/ascend (fetched 2026-08-29T09:58:45.608621+00:00, sha 0c3bd92af16b)
  - site_page: https://docs.vllm.ai/projects/ascend/en/latest/installation.html (fetched 2026-08-29T09:58:45.617827+00:00, sha fad6dd84f837)
  - site_page: https://docs.vllm.ai/projects/ascend/en/latest/tutorials/features/index.html (fetched 2026-08-29T09:58:45.620473+00:00, sha ab4ab23e427d)
  - site_page: https://docs.vllm.ai/projects/ascend/en/latest/tutorials/features/pd_colocated_mooncake_multi_instance.html (fetched 2026-08-29T09:58:45.622186+00:00, sha 085f81c41e0f)
  - site_page: https://docs.vllm.ai/projects/ascend/en/latest/tutorials/features/pd_disaggregation_mooncake_single_node.html (fetched 2026-08-29T09:58:45.623998+00:00, sha 8be6c39d6639)
  - site_page: https://docs.vllm.ai/projects/ascend/en/latest/tutorials/features/pd_disaggregation_mooncake_multi_node.html (fetched 2026-08-29T09:58:45.625805+00:00, sha a55e81d79f27)
  - site_page: https://docs.vllm.ai/projects/ascend/en/latest/tutorials/features/dynamic_chunked_pipeline_parallel.html (fetched 2026-08-29T09:58:45.627858+00:00, sha 0610f65f988f)
  - site_page: https://docs.vllm.ai/projects/ascend/en/latest/tutorials/features/suffix_speculative_decoding.html (fetched 2026-08-29T09:58:45.629760+00:00, sha 8e16d7466199)
  - site_page: https://docs.vllm.ai/projects/ascend/en/latest/tutorials/features/ray.html (fetched 2026-08-29T09:58:45.631568+00:00, sha e13acd3e2316)
- Data as of 2026-08-30T08:39:29.467469+00:00.