NVIDIA/TensorRT

Health v2 · maintenance only

Adoption not part of the score

13293 stars · 2396 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

NVIDIA TensorRT is an SDK for high-performance deep learning inference on NVIDIA GPUs, comprising an inference compiler that converts trained models (ONNX, PyTorch, TensorFlow) into optimized GPU engines and a runtime for executing them. This repository contains the open-source components, including plugins, an ONNX parser, and sample applications.

Use cases

• deploy trained deep learning models on NVIDIA GPUs with low latency
• convert ONNX models to optimized TensorRT inference engines
• quantize models to FP8/INT8/INT4 for faster inference
• run LLM inference with transformer-specific optimizations
• benchmark inference performance with trtexec
• write custom layers and plugins for unsupported ops

When to choose

• you need maximum inference throughput or minimum latency on NVIDIA GPUs
• you are deploying models to production in data centers or edge devices
• you want quantization (FP8/FP4/INT8/INT4) to reduce memory and speed up inference
• you need a mature, vendor-supported inference stack with C++ and Python APIs

When to avoid

• you are targeting non-NVIDIA hardware (CPUs, AMD/Intel GPUs, Apple Silicon)
• you only need training rather than inference
• you want a simple portable runtime without GPU-specific engine compilation
• your models change frequently and recompiling engines per GPU is too costly

Facets

library · maturity stable

machine-learning llm-inference compiler gpu-computing sdk parser deep-learning machine-learning gpu-computing artificial-intelligence large-language-models windows cpp python cross-platform inference-optimization onnx quantization model-deployment nvidia-cuda trtexec engine-builder linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="NVIDIA/TensorRT")