vitoplantamura/OnnxStream

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

2086 stars · 99 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A lightweight C++ inference library for ONNX models that streams weights to run large models in very little memory, accelerated by XNNPACK. It can run Stable Diffusion XL on a Raspberry Pi Zero 2 (298MB RAM), LLMs like Mistral 7B on desktops, and models like YOLOv8 and Whisper in the browser via WASM.

Use cases

• run stable diffusion on a raspberry pi with 512MB ram
• run mistral 7B locally with minimal memory
• run whisper speech recognition in the browser
• run yolov8 object detection in the browser with wasm
• run onnx models on embedded arm or risc-v devices
• generate images from text on low-memory hardware
• run llm inference on a tiny sbc

When to choose

• you need to run large ONNX models (diffusion, LLMs) in constrained memory environments
• you target ARM, RISC-V, x86, or WASM without GPU dependencies
• you want a lightweight C++ inference engine with Python/C#/JS bindings

When to avoid

• you need maximum throughput on GPU servers with mature CUDA stacks
• you need broad ONNX operator coverage or production support like ONNX Runtime
• you need a permissively documented license (license is non-standard)

Facets

library · maturity active

llm-inference machine-learning stable-diffusion speech-recognition computer-vision sdk machine-learning deep-learning artificial-intelligence image-processing speech-processing embedded-systems gpu-computing windows wasm embedded cpp python cross-platform browser onnx raspberry-pi xnnpack low-memory tinyml edge-inference llm yolov8 whisper bindings linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="vitoplantamura/OnnxStream")