thu-pacman/chitu

Health v2 · maintenance only

Adoption not part of the score

2995 stars · 256 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Chitu (赤兔) is a high-performance inference framework for large language models, positioned as a production-grade LLM serving engine. It supports diverse hardware including NVIDIA GPUs, Huawei Ascend, and other domestic chips, with scalable deployment from CPU-only to large clusters.

Use cases

• serve deepseek-r1 671b on a single gpu
• deploy llm inference on huawei ascend 910b
• run quantized fp8 or fp4 large language models
• set up multi-node llm serving cluster with pd disaggregation
• serve qwen or glm models in production
• run llm inference on domestic chinese gpu hardware

When to choose

• you need production-grade LLM serving on non-NVIDIA or domestic Chinese accelerators
• you want to run very large MoE models like DeepSeek-R1 with limited GPU memory via CPU+GPU hybrid inference
• you need scalable deployment from single GPU to multi-node clusters

When to avoid

• you need the broadest ecosystem and community plugins, where vLLM or SGLang may be safer
• you only need lightweight local inference on consumer hardware, where llama.cpp fits better
• you require guaranteed commercial support without a service agreement

Facets

library · maturity active

llm-inference gpu-computing machine-learning large-language-models deep-learning artificial-intelligence python cloud llm-serving model-serving deepseek quantization moe inference-engine domestic-gpu-support cpu-gpu-hybrid linux gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="thu-pacman/chitu")