eugr/spark-vllm-docker resource

Health v2 · maintenance only

Adoption not part of the score

2189 stars · 370 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Docker configuration and set of shell scripts for running vLLM inference on NVIDIA DGX Spark hardware, from single nodes to multi-node clusters using Ray or PyTorch distributed. It includes support for InfiniBand/RDMA networking, fast model loading, and tested nightly Docker images.

Use cases

• run vllm on dgx spark
• set up multi-node llm inference cluster
• serve large language models across two dgx sparks
• dockerize vllm with infiniband support
• deploy vllm with ray distributed backend
• benchmark llm inference on spark hardware
• download and load models fast with fastsafetensors

When to choose

• you own one or more DGX Spark machines and want optimized vLLM inference
• you need multi-node LLM serving with InfiniBand/RDMA or QSFP networking
• you want pre-tested nightly Docker images instead of hand-building vLLM
• you need cluster orchestration via Ray or native PyTorch distributed mode

When to avoid

• you run inference on non-DGX-Spark GPUs or cloud instances
• you need a general-purpose vLLM deployment on Kubernetes
• you want a managed inference service rather than self-hosted infrastructure
• you need training or fine-tuning rather than inference

Facets

infra-config · maturity active

llm-inference container-runtime deployment gpu-computing developer-tools large-language-models gpu-computing infrastructure-as-code self-hosted self-hosted vllm dgx-spark multi-node infiniband nccl ray docker-compose nvidia containers docker linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="eugr/spark-vllm-docker")