pytorch/torchchat

Health v2 · maintenance only

Flags: no_releases archived

Adoption not part of the score

3612 stars · 248 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

torchchat is a compact PyTorch-native codebase for running large language models locally via Python, a C/C++ runner on desktop/server, or on iOS and Android devices. It supports popular models like Llama 3, Mistral, and DeepSeek R1 Distill with multiple quantization and execution modes.

Use cases

• run llama 3 locally on my mac
• chat with an llm from the command line
• deploy a small language model on android or ios
• run llm inference in c++ without python
• quantize and evaluate llms locally with pytorch
• run multimodal llama 3.2 vision model locally

When to choose

• you want PyTorch-native local LLM inference across server, desktop, and mobile
• you need to run LLMs in C/C++ applications without a Python dependency
• you want on-device LLM deployment on iOS or Android via ExecuTorch

When to avoid

• you need actively developed tooling - the project is no longer under active development
• you want GPU-cluster-scale inference or serving many concurrent users
• you need broad model support beyond the Llama-family and a few supported models

Facets

cli-tool · maturity maintenance

llm-inference cli chatbot large-language-models machine-learning developer-tools mobile-development python cli cpp pytorch local-inference llm on-device-ai aot-inductor executorch quantization multimodal linux macos android ios

Member repositories

For agents

markdown · JSON · MCP: product_card(name="pytorch/torchchat")