ELS-RD/transformer-deploy

Health v2 · maintenance only

Adoption not part of the score

1689 stars · 153 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Python tool that optimizes Hugging Face transformer models for production inference using ONNX Runtime and NVIDIA TensorRT, deploying them on NVIDIA Triton inference servers. It claims 5-10X speedups over vanilla PyTorch with a single command line workflow.

Use cases

• deploy hugging face transformer models to production
• speed up transformer inference on gpu
• convert pytorch models to onnx or tensorrt
• serve sentence embeddings with low latency
• quantize transformer models for cpu and gpu
• set up a triton inference server for nlp models

When to avoid

• you only need occasional or offline inference without strict latency requirements
• you do not use NVIDIA GPUs or Triton-compatible infrastructure
• your model cannot be exported to ONNX
• you need a fully managed inference service rather than self-hosted deployment

Facets

library · maturity maintenance

llm-inference machine-learning deployment cli machine-learning deep-learning python hugging-face onnx tensorrt triton-inference-server quantization model-optimization natural-language-processing devops linux docker gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ELS-RD/transformer-deploy")