intel/neural-compressor

Health v2 · maintenance only

Adoption not part of the score

2704 stars · 320 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Intel Neural Compressor is an open-source Python library providing state-of-the-art low-bit quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4), sparsity, and other model compression techniques for deep learning frameworks including PyTorch, TensorFlow, JAX, and ONNX Runtime. It supports advanced quantization of large language models and vision-language models with extensive optimization for Intel hardware (Gaudi, Xeon, Core Ultra, Data Center GPUs) and limited support for AMD, ARM, and NVIDIA hardware.

Use cases

• quantize an LLM to INT4 or FP8 for faster inference
• apply SmoothQuant or GPTQ to compress a large language model
• run post-training quantization on a PyTorch model
• prune or sparsify a deep learning model
• optimize models for Intel Xeon CPUs or Gaudi accelerators
• quantize models to MXFP4 or NVFP4 low-precision formats
• compress a vision-language model like LLaMA or Qwen
• perform quantization-aware training on TensorFlow models

When to choose

• you need low-bit quantization of LLMs or VLMs across multiple data types
• you target Intel hardware (Gaudi, Xeon, Core Ultra, Data Center GPUs) for inference optimization
• you work across PyTorch, TensorFlow, JAX, or ONNX Runtime and want a unified compression API
• you need sparsity, pruning, or knowledge distillation alongside quantization

When to avoid

• you only target NVIDIA GPUs and prefer ecosystem-native tools like TensorRT-LLM
• you need a simple one-line quantization without tuning and prefer lighter-weight tools like AutoRound directly
• your framework is outside PyTorch, TensorFlow, JAX, or ONNX Runtime

Facets

library · maturity active

machine-learning deep-learning llm-inference llm-training machine-learning deep-learning large-language-models developer-tools python cross-platform quantization model-compression sparsity pruning knowledge-distillation post-training-quantization quantization-aware-training smoothquant gptq awq int8 int4 fp8 mxfp4 pytorch tensorflow onnx-runtime intel-hardware linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="intel/neural-compressor")