# gpustack/gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

Repository: https://github.com/gpustack/gpustack
Canonical: https://ross.abutalabs.com/products/gpustack
Homepage: https://gpustack.ai
Language: Python
License: Apache-2.0
License Family: permissive
Topics: ascend, cuda, deepseek, distributed-inference, genai, inference, llama, llm, maas, openai, qwen, rocm, vllm, mindie, llm-inference, llm-serving, sglang, high-performance-inference
Last push: 2026-08-26T02:09:56+00:00

## Health v2 (maintenance only)
Score: 90/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 95, longevity 60
- inputs: {"age_days": 844, "days_push": 8, "days_rel": 33, "gap_med": 23.0, "n_releases_24m": 25}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 5560, forks 628 (observed 2026-08-28T04:09:22.365071+00:00)

## What it is
GPUStack is an open-source GPU cluster manager for AI model serving that orchestrates inference engines like vLLM, SGLang, and TensorRT-LLM across heterogeneous GPU hardware (NVIDIA, AMD, Ascend, and more) on-premises, in Kubernetes, or in the cloud. It also provisions on-demand SSH-accessible GPU instances and provides enterprise features such as monitoring, authentication, load balancing, and token metering for delivering Model-as-a-Service.

## Use cases
- manage a gpu cluster for llm inference
- serve openai-compatible llm apis on my own gpus
- deploy vllm or sglang across multiple gpu servers
- run large models with distributed inference on heterogeneous gpus
- provision ssh-accessible gpu instances for fine-tuning
- build a model-as-a-service platform for my team
- schedule llm workloads across nvidia and amd gpus
- monitor gpu utilization and token usage for model serving

## When to choose
- you need to serve LLMs across a fleet of heterogeneous GPUs (NVIDIA, AMD, Ascend, etc.) with automatic engine selection
- you want an OpenAI-compatible API gateway with auth, rate limiting, and metering out of the box
- you need multi-cluster GPU management spanning on-premises servers, Kubernetes, and cloud providers
- you want day-0 support for newly released models via pluggable inference engines
- you need SSH-accessible GPU instances for development and fine-tuning workloads

## When to avoid
- you only need to run a single model on one local machine - simpler tools like Ollama or llama.cpp suffice
- your infrastructure is already standardized on Kubernetes and you prefer a native Kubernetes operator or KServe
- you need training or fine-tuning orchestration as the primary workload rather than inference serving
- you require Windows or macOS worker nodes - GPU workers are Linux-only

## Facets
- artifact type: service
- maturity: active
- function: llm-inference, gpu-computing, container-orchestration, monitoring, auth, rate-limiting, scheduling, ssh, api-gateway, self-hosted
- domain: large-language-models, gpu-computing, infrastructure-as-code, cloud-computing, self-hosted, artificial-intelligence, microservices
- platform: python, self-hosted, cloud
- tags: gpu-cluster-manager, model-as-a-service, vllm, sglang, tensorrt-llm, heterogeneous-gpu, distributed-inference, gpu-instances, openai-compatible-api, nvidia, amd, ascend, rocm, cuda, speculative-decoding, kv-cache, multi-cluster, metering, rbac, devops, linux, docker, kubernetes

## Member repositories
- gpustack/gpustack (main) score 90

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:09:22.365071+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:55:38.199714+00:00, confidence not recorded.
  - readme: https://github.com/gpustack/gpustack (fetched 2026-08-28T04:09:22.365071+00:00, sha 393c4d308f87)
  - homepage: https://gpustack.ai (fetched 2026-08-29T08:50:56.021693+00:00, sha e86ac456e465)
  - site_page: https://docs.gpustack.ai/latest/quickstart (fetched 2026-08-29T08:50:56.033417+00:00, sha fedeae8b98b8)
  - site_page: https://docs.gpustack.ai/ (fetched 2026-08-29T08:50:56.031578+00:00, sha 36c6c3c2e4f9)
  - site_page: https://gpustack.ai/enterprise (fetched 2026-08-29T08:50:56.035210+00:00, sha 03370540b851)
- Data as of 2026-08-30T08:39:29.467469+00:00.