bitsandbytes-foundation/bitsandbytes

Health v2 · maintenance only

Adoption not part of the score

8439 stars · 916 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

bitsandbytes is a Python library providing k-bit quantization primitives for PyTorch, enabling 8-bit (LLM.int8()) and 4-bit (QLoRA) quantization plus 8-bit optimizers to dramatically reduce memory usage for LLM inference and training. It integrates tightly with the Hugging Face ecosystem and supports NVIDIA, AMD, Intel, and Apple Silicon accelerators.

Use cases

• run large language models on GPUs with limited VRAM via 8-bit quantization
• finetune LLMs with QLoRA 4-bit quantization on consumer GPUs
• reduce optimizer memory usage during model training
• load Hugging Face Transformers models in 4-bit or 8-bit
• train 70B parameter models with FSDP-QLoRA on dual 24GB GPUs

When to choose

• you need to fit large LLMs into memory-constrained GPUs for inference or training
• you use Hugging Face Transformers/PEFT and want drop-in quantization via BitsAndBytesConfig
• you want memory-efficient 8-bit optimizers without accuracy loss

When to avoid

• you need maximum inference speed rather than memory savings, as quantization adds overhead
• your workload is CPU-only or uses small models where quantization gains are negligible
• you require non-PyTorch frameworks like JAX or TensorFlow

Facets

library · maturity active

machine-learning llm-training llm-inference gpu-computing large-language-models machine-learning deep-learning python windows cross-platform quantization qlora 8-bit-optimizers pytorch memory-optimization hugging-face gpu linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="bitsandbytes-foundation/bitsandbytes")