# NVIDIA/TensorRT

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

Repository: https://github.com/NVIDIA/TensorRT
Canonical: https://ross.abutalabs.com/products/tensorrt
Homepage: https://developer.nvidia.com/tensorrt
Language: C++
License: Apache-2.0
License Family: permissive
Topics: tensorrt, nvidia, deep-learning, inference, gpu-acceleration
Last push: 2026-08-25T21:23:36+00:00

## Health v2 (maintenance only)
Score: 94/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 84, longevity 100
- inputs: {"age_days": 2680, "days_push": 8, "days_rel": 29, "gap_med": 36, "n_releases_24m": 18}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 13293, forks 2396 (observed 2026-08-28T04:11:03.070690+00:00)

## What it is
NVIDIA TensorRT is an SDK for high-performance deep learning inference on NVIDIA GPUs, comprising an inference compiler that converts trained models (ONNX, PyTorch, TensorFlow) into optimized GPU engines and a runtime for executing them. This repository contains the open-source components, including plugins, an ONNX parser, and sample applications.

## Use cases
- deploy trained deep learning models on NVIDIA GPUs with low latency
- convert ONNX models to optimized TensorRT inference engines
- quantize models to FP8/INT8/INT4 for faster inference
- run LLM inference with transformer-specific optimizations
- benchmark inference performance with trtexec
- write custom layers and plugins for unsupported ops

## When to choose
- you need maximum inference throughput or minimum latency on NVIDIA GPUs
- you are deploying models to production in data centers or edge devices
- you want quantization (FP8/FP4/INT8/INT4) to reduce memory and speed up inference
- you need a mature, vendor-supported inference stack with C++ and Python APIs

## When to avoid
- you are targeting non-NVIDIA hardware (CPUs, AMD/Intel GPUs, Apple Silicon)
- you only need training rather than inference
- you want a simple portable runtime without GPU-specific engine compilation
- your models change frequently and recompiling engines per GPU is too costly

## Facets
- artifact type: library
- maturity: stable
- function: machine-learning, llm-inference, compiler, gpu-computing, sdk, parser
- domain: deep-learning, machine-learning, gpu-computing, artificial-intelligence, large-language-models
- platform: windows, cpp, python, cross-platform
- tags: inference-optimization, onnx, quantization, model-deployment, nvidia-cuda, trtexec, engine-builder, linux, gpu

## Member repositories
- NVIDIA/TensorRT (main) score 94

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:11:03.070690+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:13:22.549753+00:00, confidence not recorded.
  - readme: https://github.com/NVIDIA/TensorRT (fetched 2026-08-28T04:11:03.070690+00:00, sha 983ff90f14a2)
  - homepage: https://developer.nvidia.com/tensorrt (fetched 2026-08-29T08:08:31.809832+00:00, sha f964e64e2727)
  - site_page: https://docs.nvidia.com/deeplearning/tensorrt (fetched 2026-08-29T08:08:31.819207+00:00, sha 429832b9bac4)
  - site_page: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/index.html (fetched 2026-08-29T08:08:31.821295+00:00, sha 40e6655bdc6e)
  - site_page: https://developer.nvidia.com/tensorrt-getting-started (fetched 2026-08-29T08:08:31.823259+00:00, sha 6856989f516c)
  - site_page: https://docs.nvidia.com/deeplearning/tensorrt/latest/getting-started/quick-start-guide.html (fetched 2026-08-29T08:08:31.825038+00:00, sha 02488d10fa3f)
  - site_page: https://docs.omniverse.nvidia.com/extensions/latest/ext_product-configurator.html (fetched 2026-08-29T08:08:31.826786+00:00, sha 206706f14512)
  - site_page: https://docs.nvidia.com/deeplearning/tensorrt-rtx/latest/installing-tensorrt-rtx/installing.html (fetched 2026-08-29T08:08:31.828239+00:00, sha 72fd66bbefea)
- Data as of 2026-08-30T08:39:29.467469+00:00.