qwopqwop200/GPTQ-for-LLaMa

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

3073 stars · 453 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Python library that applies GPTQ 4-bit weight quantization to LLaMA large language models, drastically reducing memory usage and checkpoint size. It is now superseded by AutoGPTQ, which the author recommends instead.

Use cases

• quantize llama models to 4 bits
• run llama 7b on a smaller gpu
• compress large language model checkpoints
• reduce vram usage for llama inference
• convert llama weights with gptq

When to choose

• you specifically need the original GPTQ-for-LLaMA implementation or its benchmarks
• you are on Linux and want a simple standalone quantization script for LLaMA

When to avoid

• you want an actively maintained or feature-rich quantization toolkit - use AutoGPTQ instead
• you are on Windows without WSL2, since Triton support is Linux-only
• you need the fastest possible quantized inference kernels

Facets

library · maturity maintenance

llm-inference machine-learning deep-learning large-language-models machine-learning gpu-computing python quantization gptq llama 4-bit model-compression triton linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="qwopqwop200/GPTQ-for-LLaMa")