# NVIDIA/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

Repository: https://github.com/NVIDIA/Model-Optimizer
Canonical: https://ross.abutalabs.com/products/model-optimizer
Homepage: https://nvidia.github.io/Model-Optimizer/
Language: Python
License: Apache-2.0
License Family: permissive
Last push: 2026-08-26T22:43:18+00:00

## Health v2 (maintenance only)
Score: 91/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 98, longevity 61
- inputs: {"age_days": 862, "days_push": 7, "days_rel": 15, "gap_med": 28.0, "n_releases_24m": 21}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3488, forks 552 (observed 2026-08-28T04:08:07.081631+00:00)

## What it is
NVIDIA Model Optimizer (ModelOpt) is a Python library of state-of-the-art model optimization techniques including quantization, pruning, distillation, NAS, speculative decoding, and sparsity. It takes Hugging Face, PyTorch, or ONNX models as input and exports optimized quantized checkpoints ready for deployment in inference frameworks like TensorRT-LLM, TensorRT, vLLM, and SGLang.

## Use cases
- quantize an LLM to FP8 or INT4 for faster inference
- compress a Hugging Face model for TensorRT-LLM deployment
- apply quantization-aware training to recover accuracy after quantization
- prune a deep learning model to reduce its size
- distill a large model into a smaller student model
- export a quantized checkpoint for vLLM or SGLang
- quantize ONNX models for onnxruntime deployment
- speed up LLM inference with speculative decoding

## When to choose
- you need to compress or quantize PyTorch, Hugging Face, or ONNX models for NVIDIA GPU inference
- you deploy models with TensorRT-LLM, TensorRT, vLLM, or SGLang and want optimized checkpoints
- you want QAT, distillation, pruning, or NAS in a single unified API
- you need FP8/INT4/NVFP4 quantization for large language models

## When to avoid
- you only target non-NVIDIA hardware without NVIDIA deployment frameworks
- you need a one-click inference server rather than a model optimization library
- your models are small enough that compression overhead is not worthwhile

## Facets
- artifact type: library
- maturity: active
- function: machine-learning, deep-learning, llm-inference, llm-training, gpu-computing, sdk
- domain: machine-learning, deep-learning, large-language-models, gpu-computing, developer-tools
- platform: python, windows
- tags: quantization, model-compression, pruning, distillation, neural-architecture-search, speculative-decoding, sparsity, tensorrt-llm, vllm, onnx, pytorch, huggingface, nvidia, linux, gpu

## Member repositories
- NVIDIA/Model-Optimizer (main) score 91

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:08:07.081631+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:36:05.731930+00:00, confidence not recorded.
  - readme: https://github.com/NVIDIA/Model-Optimizer (fetched 2026-08-28T04:08:07.081631+00:00, sha b09d0c013b60)
  - homepage: https://nvidia.github.io/Model-Optimizer/ (fetched 2026-08-29T09:30:08.015832+00:00, sha 908d3062393a)
- Data as of 2026-08-30T08:39:29.467469+00:00.