pytorch/TensorRT

Health v2 · maintenance only

Adoption not part of the score

2986 stars · 409 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Torch-TensorRT is a compiler library that accelerates PyTorch model inference on NVIDIA GPUs using TensorRT. It supports just-in-time compilation via torch.compile and ahead-of-time export for deployment in Python or C++ (libtorch) environments.

Use cases

• speed up pytorch model inference on nvidia gpus
• compile pytorch models with tensorrt backend
• deploy pytorch models in c++ without python dependency
• optimize inference latency for deep learning models
• export optimized torch models for production serving
• accelerate model inference on jetson devices

When to choose

• you run PyTorch models on NVIDIA GPUs and need lower inference latency
• you want a one-line torch.compile backend for GPU acceleration
• you need to serialize optimized models for C++/libtorch deployment
• you target Jetson or other NVIDIA embedded platforms

When to avoid

• you run inference on CPUs or non-NVIDIA GPUs
• you need training acceleration rather than inference
• your stack is TensorFlow, JAX, or ONNX Runtime based
• you cannot install CUDA and TensorRT dependencies

Facets

library · maturity active

machine-learning llm-inference compiler gpu-computing deep-learning machine-learning gpu-computing developer-tools windows python cpp tensorrt pytorch nvidia cuda inference-optimization torch-compile model-deployment jetson linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="pytorch/TensorRT")