jingyaogong/minimind resource

Health v2 · maintenance only

Adoption not part of the score

55036 stars · 7180 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

MiniMind is an open-source project that trains tiny (64M-parameter) large language models completely from scratch in pure PyTorch, covering the full pipeline from tokenizer to pretraining, SFT, LoRA, RLHF/DPO, RLAIF (PPO/GRPO/CISPO), tool use, and distillation. It doubles as a hands-on tutorial for understanding LLM internals, with models deployable via vLLM, ollama, llama.cpp, or an OpenAI-compatible API.

Use cases

• train a small LLM from scratch on a single consumer GPU
• learn how LLM pretraining and SFT work internally
• experiment with RLHF, DPO, PPO, and GRPO implementations
• understand MoE architectures with readable code
• build a tiny chatbot with tool calling and reasoning
• teach an LLM course with a reproducible codebase
• fine-tune and distill a lightweight language model
• serve a custom-trained model with an OpenAI-compatible API

When to choose

• you want to understand every step of LLM training without black-box abstractions
• you have limited GPU budget and want to train a model from zero cheaply
• you need an educational, reproducible reference implementation of modern LLM training techniques
• you want a tiny model compatible with vLLM, ollama, or llama.cpp

When to avoid

• you need a production-grade, high-capability LLM for real applications
• you only want to fine-tune an existing large model with LoRA
• you need multimodal, speech, or vision capabilities out of the box (see MiniMind-V/O instead)
• you require enterprise support, scaling, or extensive evaluation benchmarks

Facets

learning-resource · maturity active

llm-training machine-learning deep-learning llm-inference chatbot rag agent-framework large-language-models artificial-intelligence deep-learning education python cross-platform train-from-scratch pytorch moe rlhf dpo grpo lora sft pretraining model-distillation tool-calling tutorial small-language-model openai-api-compatible natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="jingyaogong/minimind")