Nunchaku

Health v2 · maintenance only

Adoption not part of the score

3953 stars · 279 forks observed · 2026-09-23

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Nunchaku is a high-performance inference engine for 4-bit quantized diffusion models (and LLMs) based on the SVDQuant technique from an ICLR 2025 paper. It provides a Diffusers-compatible Python API and a ComfyUI plugin for running models like FLUX.1, Qwen-Image, and SDXL with low VRAM usage.

Use cases

• run FLUX.1 image generation in 4-bit on consumer GPUs
• generate images with Qwen-Image using less VRAM
• use LoRAs with quantized diffusion models without requantization
• run SDXL text-to-image on low-VRAM GPUs
• use 4-bit diffusion models in ComfyUI workflows
• run image editing models like FLUX.1-Kontext or Qwen-Image-Edit efficiently
• offload diffusion models to CPU to fit in 3-4GB VRAM

When to choose

• you want to run large diffusion models (FLUX, Qwen-Image, SDXL) on consumer NVIDIA GPUs with limited VRAM
• you need fast 4-bit inference with minimal quality loss
• you use ComfyUI or Diffusers and want drop-in quantized model support
• you want to use off-the-shelf LoRAs with quantized models

When to avoid

• you need CPU-only or AMD/Intel GPU inference
• you need to quantize your own custom models (use DeepCompressor instead)
• you need maximum quality at full precision
• you work with non-supported model architectures

Facets

library · maturity active

llm-inference machine-learning image-processing gpu-computing deep-learning image-processing machine-learning gpu-computing python windows diffusion-models quantization svdquant 4-bit flux lora comfyui inference-engine text-to-image linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="nunchux-ai/nunchaku")