openvinotoolkit/nncf

Health v2 · maintenance only

Adoption not part of the score

1197 stars · 302 forks observed · 2026-09-03

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

NNCF is Intel's Neural Network Compression Framework, a Python library providing post-training and training-time compression algorithms (quantization, pruning, sparsity) for optimizing neural network inference with OpenVINO. It supports models from PyTorch, TorchFX, ONNX, and OpenVINO, including LLM weight compression.

Use cases

• quantize a model for faster OpenVINO inference
• compress LLM weights to run on CPU
• apply post-training quantization with minimal accuracy drop
• prune or sparsify a PyTorch model
• optimize object detection or segmentation models for deployment
• quantize ONNX models

When to choose

• you deploy models with OpenVINO and want smaller/faster inference
• you need quantization-aware training or post-training quantization for PyTorch/ONNX models
• you want to compress LLMs for Intel hardware

When to avoid

• you target GPU inference with TensorRT or other non-OpenVINO runtimes
• you need compression for frameworks like JAX or Keras
• you want a general-purpose training framework rather than compression

Facets

library · maturity active

machine-learning deep-learning llm-inference deep-learning machine-learning large-language-models developer-tools python windows quantization pruning sparsity model-compression openvino pytorch onnx post-training-quantization inference-optimization linux macos gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="openvinotoolkit/nncf")