marella/ctransformers

Health v2 · maintenance only

Adoption not part of the score

1885 stars · 143 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

CTransformers provides Python bindings for transformer language models implemented in C/C++ using the GGML library. It offers a unified interface to run quantized models like LLaMA, Falcon, and GPT-2 locally with optional CUDA and Metal acceleration.

Use cases

• run quantized llm models locally in python
• load ggml or gguf model files for text generation
• use llama models without a gpu
• integrate local llms with langchain
• stream llm output token by token
• run inference on cpu with c/c++ speed

When to choose

• you want fast CPU inference of quantized transformer models in Python
• you need LangChain or Hugging Face Transformers compatibility for local models
• you want optional CUDA or Metal acceleration for supported models

When to avoid

• you need the latest model architectures or active updates, since the project is in maintenance
• you require full GPU training or fine-tuning rather than inference
• you prefer llama.cpp's native Python bindings with broader model support

Facets

library · maturity maintenance

llm-inference machine-learning sdk large-language-models artificial-intelligence python cross-platform ggml gguf quantized-models langchain huggingface cpu-inference text-generation natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="marella/ctransformers")