RightNow-AI/picolm

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1921 stars · 240 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

PicoLM is a pure C11 LLM inference engine that runs 1-billion parameter models in GGUF format on extremely constrained hardware like $10 boards with 256MB RAM. It compiles to a single ~80KB binary with zero dependencies, uses ~45MB of runtime RAM, and supports JSON grammar mode for reliable structured output.

Use cases

• run an LLM locally on a raspberry pi or cheap SBC
• offline chatbot inference without cloud APIs
• pipe prompts to an LLM from a shell script or agent
• serve as the local brain for an embedded AI assistant
• generate valid JSON tool calls from a small 1B model
• run LLM inference on 256MB RAM devices
• build a fully offline AI agent with no API keys

When to choose

• you need LLM inference on extremely constrained embedded hardware
• you want a zero-dependency, single-binary, no-Python solution
• privacy and offline operation are requirements
• you're building an agent like PicoClaw that spawns an LLM subprocess

When to avoid

• you need large models (7B+) or GPU acceleration
• you want a full-featured inference server with OpenAI-compatible APIs
• you need broad model architecture support or frequent cutting-edge features
• you prefer Python tooling and ecosystem integration

Facets

library · maturity active

llm-inference cli machine-learning large-language-models embedded-systems artificial-intelligence embedded cli cross-platform gguf quantization edge-ai offline-inference zero-dependencies risc-v raspberry-pi single-binary low-resource command-line linux arm

Member repositories

For agents

markdown · JSON · MCP: product_card(name="RightNow-AI/picolm")