OpenNMT/CTranslate2

Health v2 · maintenance only

Adoption not part of the score

4644 stars · 520 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

CTranslate2 is a C++ and Python library for fast, memory-efficient inference of Transformer models on CPU and GPU. It uses quantization, layer fusion, and other optimizations to accelerate encoder-decoder, decoder-only, and encoder-only models like NLLB, Llama, T5, and Whisper.

Use cases

• run machine translation models fast in production
• serve whisper speech recognition with low memory
• run llama or mistral inference on cpu
• quantize transformer models to int8 or fp16
• embed transformer inference in a c++ application
• convert huggingface transformers models to an optimized format
• speed up text generation compared to pytorch

When to choose

• you need fast transformer inference in production, especially on cpu
• you want to reduce model memory footprint via quantization
• you need to embed models in a c++ application with minimal dependencies
• you require fine control over threading and memory usage

When to avoid

• you want to train or fine-tune custom architectures
• your model architecture is not among the supported types
• you just need a quick prototype where pytorch suffices

Facets

library · maturity stable

llm-inference machine-learning deep-learning speech-recognition nlp sdk machine-learning deep-learning large-language-models speech-processing performance windows python cpp cross-platform transformer-inference quantization model-conversion cpu-optimization int8 beam-search opennmt whisper llama natural-language-processing linux macos gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="OpenNMT/CTranslate2")