lyogavin/airllm

Health v2 · maintenance only

Adoption not part of the score

39932 stars · 4069 forks observed · 2026-10-21

What it is AI-extracted, prompt v1, taxonomy v1, 2026-09-28, confidence 0.97

AirLLM is a Python library that enables running extremely large language models on a single low‑VRAM GPU. It achieves this without quantization, pruning, or distillation by streaming weights and using efficient memory tricks.

Use cases

• run a 70B LLM on a 4GB GPU for inference
• run massive models like Kimi K3 (2.8T) with under 4GB VRAM
• train Qwen 125B model on a consumer‑grade GPU with limited memory
• deploy large LLMs in resource‑constrained environments
• experiment with huge open‑source models without expensive hardware

When to choose

• You have only a low‑VRAM GPU (4‑12GB) but need to run or fine‑tune very large LLMs.
• You want to avoid quantization or model distillation and keep full‑precision inference.
• You need a Python‑based solution that integrates with Hugging Face Transformers.

When to avoid

• You require maximum throughput and can afford high‑end multi‑GPU setups.
• You need support for non‑Python environments or other programming languages.
• Your workload demands advanced distributed training beyond single‑GPU streaming.

Facets

library · maturity active

llm-inference llm-training gpu-computing large-language-models artificial-intelligence machine-learning python windows low-vram-inference memory-efficient transformer open-source no-quantization linux macos gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lyogavin/airllm")