UbiquitousLearning/mllm

Health v2 · maintenance only

Adoption not part of the score

1617 stars · 218 forks observed · 2026-10-02

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

MLLM is a fast, lightweight multimodal LLM inference engine written in C++ for mobile and edge devices, with backends for ARM CPU, Qualcomm QNN NPU, OpenCL, CUDA (Jetson), and Ascend NPU. It includes a Python runtime (pymllm) supporting quantized serving of models like Qwen3 and Qwen3-VL.

Use cases

• run multimodal LLMs on Android phones
• on-device LLM inference on mobile devices
• run Qwen3-VL on Jetson Orin with quantization
• deploy LLM inference on NPU with QNN
• serve quantized LLMs on edge devices
• run vision-language models offline on mobile

When to choose

• you need fast multimodal LLM inference on mobile or edge hardware
• you target Qualcomm NPU, Ascend, or Jetson devices
• you want C++ on-device inference with quantization support

When to avoid

• you need a mature general-purpose server LLM stack for datacenter GPUs
• you only need standard CPU inference on desktops where llama.cpp suffices
• you need broad model coverage beyond supported families like Qwen

Facets

library · maturity active

llm-inference machine-learning deep-learning sdk cli large-language-models artificial-intelligence mobile-development embedded-systems gpu-computing cross-platform cpp python embedded multimodal-llm on-device-inference edge-ai npu qnn quantization jetson ascend qwen arm-cpu android ios gpu mobile

Member repositories

For agents

markdown · JSON · MCP: product_card(name="UbiquitousLearning/mllm")