PrismML-Eng/Bonsai-demo

Health v2 · maintenance only

Flags: no_releases young

Adoption not part of the score

2252 stars · 229 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A demo repository for running PrismML's Bonsai family of 1-bit and ternary-quantized language models locally via llama.cpp and MLX. It provides setup scripts and a local chat server (with vision, tool calling, MCP, and long-context support) for models ranging from 1.7B to 27B parameters.

Use cases

• run a 27B LLM locally on a laptop or iPhone
• chat with a local vision-language model about screenshots and PDFs
• self-host an OpenAI-compatible LLM server with tool calling
• run an LLM offline with full data privacy
• test ultra-low-bit quantized models on Metal, CUDA, Vulkan, or ROCm
• build agentic workflows with a small local model and MCP servers

When to choose

• you want a capable LLM running entirely on-device with a tiny memory footprint
• you need local inference on Apple Silicon, consumer GPUs, or CPU
• you want vision, reasoning, and tool calling without cloud APIs
• energy efficiency and token throughput on edge hardware matter

When to avoid

• you need maximum output quality from frontier-scale models
• you require a managed cloud inference service
• you need fine-tuning or training rather than inference
• you depend on an ecosystem beyond the provided llama.cpp/MLX demo scripts

Facets

application · maturity active

llm-inference chatbot rag mcp machine-learning large-language-models artificial-intelligence machine-learning self-hosted developer-tools windows cross-platform cli self-hosted llamacpp mlx 1-bit-quantization ternary-weights local-llm on-device-ai vision-language-model small-models bonsai prism-ml macos linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="PrismML-Eng/Bonsai-demo")