samuel-vitorino/lm.rs

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1033 stars · 43 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A minimal Rust implementation for running LLM inference locally on CPU without ML libraries, supporting Gemma 2, Llama 3.2, and Phi 3.5 (including vision) models with quantization. Inspired by Karpathy's llama2.c and llm.c.

Use cases

• run LLM inference locally on CPU
• run language models in Rust without ML frameworks
• run quantized Llama or Gemma models on a local machine
• run multimodal vision-language models on CPU
• learn how LLM inference works from minimal code

When to choose

• you want dependency-free CPU-only LLM inference in Rust
• you want a small, readable codebase to study inference internals
• you need to run small quantized models (1B-9B) on modest hardware

When to avoid

• you need GPU acceleration or maximum throughput
• you need broad model support or production-grade features
• you need a battle-tested inference server for production workloads

Facets

library · maturity active

llm-inference machine-learning large-language-models machine-learning developer-tools rust windows cli cpu-inference quantization minimal-implementation multimodal gemma llama phi-3 linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="samuel-vitorino/lm.rs")