vllm-project/aibrix

Health v2 · maintenance only

Adoption not part of the score

5038 stars · 663 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

AIBrix is an open-source, cloud-native framework from the vLLM project providing essential building blocks for scalable, cost-efficient GenAI/LLM inference infrastructure on Kubernetes. It offers a pluggable control plane with components for model serving, autoscaling, routing, and gateway management tailored to enterprise LLM deployments.

Use cases

• deploy and scale vLLM LLM inference on Kubernetes
• build a cost-efficient LLM serving control plane
• autoscale large language model inference workloads
• route traffic across multiple LLM model replicas
• run DeepSeek-R1 or other large models in production
• manage GenAI inference infrastructure for enterprise

When to choose

• you serve LLMs at scale on Kubernetes and need autoscaling, routing, and gateway features
• you want a Kubernetes-native control plane purpose-built for vLLM inference
• you need cost-efficient multi-model LLM serving infrastructure

When to avoid

• you only need to run a single LLM locally without Kubernetes
• you use a managed LLM API and don't self-host models
• your stack is not container/Kubernetes based

Facets

framework · maturity active

llm-inference container-orchestration api-gateway deployment monitoring routing large-language-models infrastructure-as-code cloud-computing microservices go cloud self-hosted genai-inference vllm llm-serving control-plane autoscaling model-serving kubernetes-operators devops kubernetes docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="vllm-project/aibrix")