li-plus/chatglm.cpp

Health v2 · maintenance only

Adoption not part of the score

2962 stars · 326 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A pure C++ implementation of ChatGLM-6B, ChatGLM2-6B, ChatGLM3, and GLM-4(V) based on ggml, enabling memory-efficient real-time chatting on consumer hardware. It provides int4/int8 quantization, streaming generation, and Python bindings with web demo and API server options.

Use cases

• run chatglm locally on a macbook
• quantize chatglm-6b to int4 for cpu inference
• serve a chatglm model behind an api
• chat with glm-4 offline without a gpu
• embed chatglm inference in a python app
• run codegeex2 for code completion locally

When to choose

• you want fast, memory-efficient local inference of ChatGLM/GLM-4 models on CPU or Apple Silicon
• you need llama.cpp-style quantized GGML models with streaming output
• you want Python bindings plus CLI and API server options in one project

When to avoid

• you need models outside the ChatGLM/GLM/CodeGeeX family
• you require heavy GPU multi-node inference or production-scale serving
• you need fine-grained training or finetuning rather than inference

Facets

library · maturity active

llm-inference machine-learning chatbot cli http-server large-language-models machine-learning developer-tools windows python cpp cli ggml quantization chatglm glm4 cpu-inference llama.cpp-style local-llm python-bindings natural-language-processing linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="li-plus/chatglm.cpp")