NVIDIA/Model-Optimizer

A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed. observed · 2026-08-28

github.com/NVIDIA/Model-Optimizer · homepage · Python · Apache-2.0 (permissive) observed · 2026-08-28

Health v2 · maintenance only

91/100

Activity 99
Release rhythm 98
Longevity 61

How is this computed?

round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.

gap_med: 28.0
age_days: 862
days_rel: 15
days_push: 7
n_releases_24m: 21

Full methodology

Adoption not part of the score

3488 stars · 552 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

NVIDIA Model Optimizer (ModelOpt) is a Python library of state-of-the-art model optimization techniques including quantization, pruning, distillation, NAS, speculative decoding, and sparsity. It takes Hugging Face, PyTorch, or ONNX models as input and exports optimized quantized checkpoints ready for deployment in inference frameworks like TensorRT-LLM, TensorRT, vLLM, and SGLang.

Use cases

quantize an LLM to FP8 or INT4 for faster inference
compress a Hugging Face model for TensorRT-LLM deployment
apply quantization-aware training to recover accuracy after quantization
prune a deep learning model to reduce its size
distill a large model into a smaller student model
export a quantized checkpoint for vLLM or SGLang
quantize ONNX models for onnxruntime deployment
speed up LLM inference with speculative decoding

When to choose

you need to compress or quantize PyTorch, Hugging Face, or ONNX models for NVIDIA GPU inference
you deploy models with TensorRT-LLM, TensorRT, vLLM, or SGLang and want optimized checkpoints
you want QAT, distillation, pruning, or NAS in a single unified API
you need FP8/INT4/NVFP4 quantization for large language models

When to avoid

you only target non-NVIDIA hardware without NVIDIA deployment frameworks
you need a one-click inference server rather than a model optimization library
your models are small enough that compression overhead is not worthwhile

Facets

library · maturity active

machine-learning deep-learning llm-inference llm-training gpu-computing sdk machine-learning deep-learning large-language-models gpu-computing developer-tools python windows quantization model-compression pruning distillation neural-architecture-search speculative-decoding sparsity tensorrt-llm vllm onnx pytorch huggingface nvidia linux gpu

2 sources

readme: https://github.com/NVIDIA/Model-Optimizer · fetched 2026-08-28 · b09d0c013b60
homepage: https://nvidia.github.io/Model-Optimizer/ · fetched 2026-08-29 · 908d3062393a

Member repositories

Repository	Role	Health v2
NVIDIA/Model-Optimizer	main	91

For agents

markdown · JSON · MCP: product_card(name="NVIDIA/Model-Optimizer")

Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem