cactus-compute/needle

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

9311 stars · 601 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Needle 2 is a 45M-parameter open language model packaged as a single 14MB binary for tool calling, device use, and structured extraction on tiny devices like phones, wearables, and robots. The repository provides a Python package for inference, LoRA fine-tuning, and export, with grammar-constrained JSON output and confidence-gated responses.

Use cases

• run tool-calling LLM on a phone or wearable
• extract structured JSON from text on-device
• add offline AI to a smart home device or robot
• fine-tune a tiny model with LoRA for edge deployment
• constrained function calling with bounded memory
• on-device agent with cloud fallback escalation

When to choose

• you need LLM inference on memory-constrained hardware with ~28MB RAM
• you want deterministic structured tool-call output without network access
• you need a tiny model for agents on phones, wearables, or microcontrollers

When to avoid

• you need general-purpose chat or long-context reasoning from a large model
• you want a full training framework for large models
• your target devices have ample compute and memory for bigger models

Facets

library · maturity active

llm-inference agent-framework machine-learning sdk artificial-intelligence large-language-models embedded-systems iot python embedded iot cross-platform on-device-ai tiny-language-model tool-calling structured-extraction edge-inference lora-fine-tuning quantization wearables robotics ai-agents android ios

Member repositories

For agents

markdown · JSON · MCP: product_card(name="cactus-compute/needle")