RWKV/rwkv.cpp

Health v2 · maintenance only

Adoption not part of the score

1579 stars · 129 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A C++ port of the RWKV language model to the ggml tensor library, providing FP32, FP16, and quantized INT4/INT5/INT8 inference focused on CPU with optional cuBLAS GPU support. It ships a C library (rwkv.h) and a Python wrapper, supporting RWKV v4 through v7 architectures and LoRA checkpoint merging.

Use cases

• run RWKV language models locally on CPU
• run quantized INT4 or INT8 LLM inference on a laptop
• generate text with RWKV models from Python
• measure perplexity and latency of quantized model formats
• merge LoRA adapters into a quantized RWKV model
• run an LLM without a GPU
• embed RWKV inference into a C or C++ application

When to choose

• you want CPU-friendly LLM inference with low memory footprint via quantization
• you specifically want to run RWKV v4-v7 models
• you need a C API or Python wrapper for embedding RWKV inference
• you want to benchmark quantization trade-offs (perplexity vs latency vs size)

When to avoid

• you need to run Transformer-based models like LLaMA or GPT
• you need heavy GPU-accelerated training or fine-tuning
• you want a high-level chat application with a UI out of the box

Facets

library · maturity active

llm-inference machine-learning deep-learning sdk large-language-models deep-learning machine-learning developer-tools cpp python windows cli rwkv ggml quantization cpu-inference int4 int8 fp16 lora linux macos gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="RWKV/rwkv.cpp")