AutoGPTQ/AutoGPTQ

Health v2 · maintenance only

Flags: archived

Adoption not part of the score

5070 stars · 543 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

AutoGPTQ is a Python library for quantizing large language models using the GPTQ weight-only quantization algorithm, with user-friendly APIs integrated into Hugging Face Transformers, Optimum, and PEFT. The project is now unmaintained, with users directed to its successor GPTQModel.

Use cases

• quantize an LLM to 4-bit with GPTQ
• run GPTQ-quantized models on consumer GPUs
• reduce VRAM usage for llama inference
• load int4 quantized transformers models
• benchmark quantized vs fp16 inference speed

When to choose

• you need to quantize or run GPTQ models on Linux or Windows with CUDA
• you want Hugging Face Transformers integration for GPTQ models

When to avoid

• starting a new project - use the maintained GPTQModel fork instead
• you need macOS support or support for the latest model architectures
• you need bug fixes or ongoing maintenance

Facets

library · maturity abandoned

llm-inference machine-learning deep-learning large-language-models deep-learning machine-learning python windows quantization gptq transformers pytorch model-compression unmaintained natural-language-processing linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="AutoGPTQ/AutoGPTQ")