# wangzhaode/mnn-llm

llm deploy project based mnn. This project has merged into MNN.

Repository: https://github.com/wangzhaode/mnn-llm
Canonical: https://ross.abutalabs.com/products/mnn-llm
Language: C++
License: Apache-2.0
License Family: permissive
Topics: chatglm-6b, cpp, cuda, mnn, opencl, chatglm2-6b, qwen-7b, baichuan2-7b, codegeex2-6b
Last push: 2025-01-20T12:22:23+00:00

## Health v2 (maintenance only)
Score: 22/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 2, release rhythm 8, longevity 90
- inputs: {"age_days": 1265, "days_push": 590, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1616, forks 177 (observed 2026-08-28T04:05:11.905928+00:00)

## What it is
mnn-llm is a C++ library for deploying large language models on-device using Alibaba's MNN inference engine, with demos for CLI, web, Android, iOS, and Python. The project has been merged into the main MNN repository and is no longer developed independently.

## Use cases
- run llm inference locally on android
- deploy chatglm or qwen on ios
- run a small language model on cpu with cuda or opencl backend
- embed an llm chatbot in a mobile app
- run llm inference from python
- run llm inference from the command line

## When to choose
- you need on-device llm inference on mobile platforms like android or ios
- you want a lightweight C++ inference runtime with cpu, cuda, opencl, or metal backends
- you are already using the MNN ecosystem

## When to avoid
- starting new projects - use the merged MNN transformers/llm code instead
- you need server-side multi-GPU serving of large models
- you need broad model support beyond the ChatGLM/Qwen/Baichuan family

## Facets
- artifact type: library
- maturity: maintenance
- function: llm-inference, machine-learning, chatbot, sdk
- domain: large-language-models, machine-learning, mobile-development, developer-tools
- platform: windows, python, cpp, cross-platform
- tags: mnn, on-device-inference, chatglm, qwen, mobile-llm, merged-into-mnn, linux, macos, android, ios

## Member repositories
- wangzhaode/mnn-llm (main) score 22

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:11.905928+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:50:04.805296+00:00, confidence not recorded.
  - readme: https://github.com/wangzhaode/mnn-llm (fetched 2026-08-28T04:05:11.905928+00:00, sha 381c7efbe801)
- Data as of 2026-08-30T08:39:29.467469+00:00.