google/gemma.cpp

Health v2 · maintenance only

Adoption not part of the score

7031 stars · 659 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A lightweight, standalone C++ inference engine for Google's Gemma foundation models (Gemma 2/3, PaliGemma 2), with a small ~2K LoC core and portable SIMD via Google Highway. It is designed for experimentation and research rather than production deployment.

Use cases

• run gemma models locally on cpu
• embed an llm inference engine in a c++ project
• experiment with llm model internals and low-level computation
• research on quantized weight compression and mixed-precision gemm
• run a small language model without python dependencies
• modify an llm runtime for algorithm research

When to choose

• you want a minimal, hackable C++ codebase for LLM research
• you need CPU-only inference of Gemma models with minimal dependencies
• you want to embed Gemma inference directly into another C++ application
• you want to experiment with SIMD, quantization, or backward passes on transformer models

When to avoid

• you need production-grade edge or server LLM deployment - use JAX, PyTorch, or Transformers instead
• you need GPU acceleration or broad multi-model support - use llama.cpp or vLLM
• you want a turnkey chat application rather than an inference library

Facets

library · maturity active

llm-inference machine-learning deep-learning large-language-models artificial-intelligence machine-learning cpp windows gemma cpu-inference simd on-device-llm minimalist research linux macos cpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="google/gemma.cpp")