jncraton/languagemodels

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1192 stars · 80 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Python library providing simple building blocks for running large language models locally with as little as 512MB of RAM. It offers instruction following, text completion, retrieval, and other NLP tasks with all inference performed on-device for privacy.

Use cases

• run an llm locally in python with minimal ram
• ask questions to a local language model from a script
• translate text offline with a small model
• classify text into fixed choices using an llm
• complete text with a local model
• retrieve passages for rag without external services
• keep llm inference private on-device

When to choose

• you need local, private LLM inference on constrained hardware like 512MB RAM
• you want a dead-simple Python API for small language models
• you're teaching or prototyping LLM concepts without GPUs or cloud APIs

When to avoid

• you need state-of-the-art model quality or long-context reasoning
• you require high-throughput production serving
• you need fine-tuning or training capabilities

Facets

library · maturity active

llm-inference nlp rag sdk large-language-models machine-learning python cross-platform local-inference low-resource on-device-llm privacy small-models natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="jncraton/languagemodels")