ggml-org/llama.cpp

Health v2 · maintenance only

Adoption not part of the score

131193 stars · 24413 forks observed · 2026-10-15

What it is AI-extracted, prompt v1, taxonomy v1, 2026-09-21, confidence 0.97

llama.cpp is a high-performance C/C++ library for running large language models locally. It provides both a library API and ready-to-use command‑line tools for inference on CPU and GPU.

Use cases

• run a large language model on a personal laptop without internet
• serve an LLM via a local REST endpoint for private applications
• integrate LLM inference into a C++ or Python project
• experiment with quantized models on low‑resource hardware
• use a command‑line interface to generate text from a model

When to choose

• You need full privacy and no external API calls
• You want a single‑binary solution that works on Windows, macOS, and Linux
• You have limited hardware and need efficient CPU/GPU quantization
• You prefer an open‑source, MIT‑licensed implementation

When to avoid

• You require distributed, high‑throughput serving across many nodes
• You need advanced training or fine‑tuning capabilities
• Your workload depends on proprietary GPU kernels not supported by ggml

Facets

library · maturity active

llm-inference cli http-server large-language-models artificial-intelligence windows cli c++ ggml quantization local-llm offline-inference cpu-gpu cross-platform self-hosted devops linux macos docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ggml-org/llama.cpp")