wangzhaode/mnn-llm

Health v2 · maintenance only

Adoption not part of the score

1616 stars · 177 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

mnn-llm is a C++ library for deploying large language models on-device using Alibaba's MNN inference engine, with demos for CLI, web, Android, iOS, and Python. The project has been merged into the main MNN repository and is no longer developed independently.

Use cases

• run llm inference locally on android
• deploy chatglm or qwen on ios
• run a small language model on cpu with cuda or opencl backend
• embed an llm chatbot in a mobile app
• run llm inference from python
• run llm inference from the command line

When to choose

• you need on-device llm inference on mobile platforms like android or ios
• you want a lightweight C++ inference runtime with cpu, cuda, opencl, or metal backends
• you are already using the MNN ecosystem

When to avoid

• starting new projects - use the merged MNN transformers/llm code instead
• you need server-side multi-GPU serving of large models
• you need broad model support beyond the ChatGLM/Qwen/Baichuan family

Facets

library · maturity maintenance

llm-inference machine-learning chatbot sdk large-language-models machine-learning mobile-development developer-tools windows python cpp cross-platform mnn on-device-inference chatglm qwen mobile-llm merged-into-mnn linux macos android ios

Member repositories

For agents

markdown · JSON · MCP: product_card(name="wangzhaode/mnn-llm")