antirez/ds4

Health v2 · maintenance only

Flags: no_releases young

Adoption not part of the score

23620 stars · 2274 forks observed · 2026-10-07

What it is AI-extracted, prompt v1, taxonomy v1, 2026-10-12, confidence 0.97

DwarfStar (ds4) is a native inference engine for running large language models like DeepSeek V4 Flash and GLM 5.x on consumer hardware. It provides a fast local server with support for Metal, CUDA, and ROCm, plus advanced features such as SSD streaming and multi‑GPU tensor parallelism.

Use cases

• Run high‑performance LLM inference on a MacBook using Metal.
• Deploy a multi‑GPU LLM server on NVIDIA CUDA hardware for concurrent sessions.
• Stream large GGUF models from SSD to run on machines with limited RAM.
• Combine multiple Macs with RDMA for tensor‑parallel inference of 4‑bit models.
• Leverage pipeline parallelism across heterogeneous systems to run very large models.

When to choose

• You have a Mac with Apple Silicon or a Linux machine with NVIDIA/AMD GPUs and need fast local LLM inference.
• You want to serve multiple LLM sessions simultaneously on multi‑GPU hardware.
• You need SSD streaming to handle models larger than available RAM.
• You prefer a self‑contained engine optimized for specific models rather than a generic GGUF runner.

When to avoid

• You require a Windows‑only solution or lack Metal/CUDA/ROCm support.
• You need a fully generic GGUF runner that supports any model out of the box.
• Your deployment environment cannot accommodate native binaries or the required GPU drivers.

Facets

application · maturity active

llm-inference http-server api-framework large-language-models metal cuda rocm multi-gpu ssd-streaming tensor-parallelism gguf ai-agents devops macos linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="antirez/ds4")