resource: llm-training
250 resources, primary matches first, then adoption-weighted; health v2 shown.
| Resource | Health v2 | Stars | Maturity |
|---|---|---|---|
| LianjiaTech/BELLE BELLE is an open-source project providing Chinese-optimized instruction-tuned large language models, training data, and fine-tuning code bu… | 21 | 8275 | maintenance |
| OpenCoder-llm/OpenCoder-llm OpenCoder is a fully open and reproducible family of code large language models (1.5B and 8B base and chat variants) trained on 2.5 trillio… | 22 | 2111 | active |
| ise-uiuc/magicoder Magicoder is a family of fully open-source code LLMs (under 7B parameters) trained with OSS-Instruct, a method that seeds LLMs with open-so… | 27 | 2094 | stable |
| ZJU4HealthCare/Foundations-of-Medical-LLMs An open-source Chinese-language textbook, 'Foundations of Medical Large Language Models', systematically covering medical AI and LLM fundam… | 53 | 2078 | active |
| horseee/Awesome-Efficient-LLM A curated awesome-list of research papers and resources on efficient large language models, covering pruning, quantization, knowledge disti… | 41 | 2036 | active |
| GAIR-NLP/O1-Journey A research project and report series from GAIR at Shanghai Jiao Tong University documenting the journey of replicating OpenAI's O1 reasonin… | 22 | 2002 | active |
| NVIDIA-NeMo/Nemotron NVIDIA's developer asset hub for the Nemotron family of open LLMs, providing reproducible training recipes (pretraining, SFT, RL), usage co… | 64 | 1983 | active |
| ymcui/Chinese-LLaMA-Alpaca-3 Chinese-LLaMA-Alpaca-3 is a project releasing Chinese-adapted Llama-3 base and instruct large language models, trained via incremental pret… | 52 | 1979 | active |
| ornith-ai/Ornith-1 Ornith is a family of open-weight large language models (397B scale) designed for agentic tasks, released under MIT license with weights ho… | 58 | 1958 | active |
| HHHHHejia/Awesome-AgenticLLM-RL-Papers A curated paper list accompanying the survey 'The Landscape of Agentic Reinforcement Learning for LLMs', cataloguing RL algorithms (PPO, DP… | 57 | 1874 | active |
| HuangOwen/Awesome-LLM-Compression A curated awesome-list of research papers and tools on large language model compression, covering quantization, pruning and sparsity, disti… | 74 | 1865 | active |
| peremartra/Large-Language-Model-Notebooks-Course A free, hands-on Jupyter Notebook course on building applications with Large Language Models, covering OpenAI and Hugging Face models, Lang… | 67 | 1821 | active |
| brevdev/launchables A collection of Jupyter notebook guides from the Brev.dev (NVIDIA Brev) team covering LLM fine-tuning, multi-modal models, image segmentati… | 70 | 1816 | active |
| thinkwee/AgentsMeetRL A curated awesome list of open-source repositories for training LLM agents with reinforcement learning, organized by taxonomy categories li… | 62 | 1799 | active |
| DGoettlich/history-llms An information hub for an academic research project at the University of Zurich training large language models from scratch on historical, … | 42 | 1782 | active |
| mlcommons/training Reference implementations of the MLPerf Training benchmark suite maintained by MLCommons, covering models from LLMs to recommendation and v… | 67 | 1771 | active |
| Toyhom/Chinese-medical-dialogue-data A Chinese medical dialogue dataset containing 792,099 patient-doctor question-answer pairs organized into six departments: internal medicin… | 32 | 1756 | stable |
| R6410418/Jackrong-llm-finetuning-guide An open-source educational knowledge base covering LLM fine-tuning, dataset distillation, reinforcement learning workflows (SFT, GRPO, GSPO… | 56 | 1665 | active |
| Tongyun1/from-minimind-to-more A detailed Chinese-language study guide and annotated walkthrough of the Minimind project for training a large language model from scratch,… | 53 | 1652 | active |
| tencent-ailab/persona-hub PersonaHub is Tencent AI Lab's collection of 1 billion diverse personas plus code for persona-driven synthetic data creation with LLMs. It … | 27 | 1642 | active |
| deepseek-ai/DeepSeek-Math-V2 DeepSeekMath-V2 is a research release from DeepSeek AI presenting an open large language model trained for self-verifiable mathematical rea… | 41 | 1603 | active |
| karpathy/LLM101n LLM101n is a planned course by Andrej Karpathy and Eureka Labs that teaches building a Storyteller AI large language model end-to-end, from… | 10 | 37505 | experimental |
| adam-maj/deep-learning An educational repository tracing the full history of deep learning, from feed-forward networks to GPT-4o, organized around seven key const… | 24 | 1575 | stable |
| Osilly/Vision-R1 Vision-R1 is the official repository for a research paper on training reasoning-capable multimodal large language models using R1-like rein… | 53 | 1571 | active |
| google-research/FLAN Google Research's repository for generating the FLAN instruction tuning dataset collections, including the original Flan 2021 and the expan… | 73 | 1567 | stable |
| WeiboAI/VibeThinker VibeThinker is a family of small dense reasoning language models (1.5B and 3B parameters) from WeiboAI, trained with a Spectrum-to-Signal p… | 60 | 1561 | active |
| wasiahmad/Awesome-LLM-Synthetic-Data A curated awesome-list of papers, tools, and blogs about synthetic data generation with large language models. It organizes resources by su… | 34 | 1551 | active |
| SkyworkAI/Skywork The Skywork series of open-source large language models (13B Base, Chat, Math, and multimodal MM variants) pre-trained on 3.2TB of multilin… | 32 | 1498 | active |
| modelscope/modelscope-classroom ModelScope Classroom is a collection of deep learning and large language model tutorials from the ModelScope community, delivered as Jupyte… | 61 | 1482 | active |
| mlfoundations/dclm DataComp-LM (DCLM) is a benchmark and dataset suite for curating training data for language models, including a leaderboard and evaluation … | 42 | 1467 | active |
| elicit/machine-learning-list A curated, tiered reading list and curriculum for learning machine learning with a focus on language models and foundation models, from fun… | 49 | 1464 | active |
| Sun-Haoyuan23/Awesome-RL-based-Reasoning-MLLMs A curated awesome-list repository collecting papers and resources on reinforcement learning-based reasoning for multimodal large language m… | 63 | 1441 | active |
| MiuLab/Taiwan-LLM A collection of Traditional Mandarin large language models (Taiwan-LLM / TAME) built on the Llama-3 architecture, fine-tuned on Traditional… | 26 | 1426 | active |
| no-magic-ai/no-magic A curated collection of 48 single-file, zero-dependency Python implementations of core AI algorithms, from GPT and LSTM to LoRA, quantizati… | 67 | 1410 | active |
| philschmid/deep-learning-pytorch-huggingface A collection of Jupyter notebook tutorials and examples for deep learning with PyTorch and Hugging Face libraries like Transformers, Datase… | 35 | 1392 | active |
| SCIR-HI/Huatuo-Llama-Med-Chinese BenTsao (formerly HuaTuo) is a collection of large language models instruction-tuned on Chinese medical knowledge, built from LLaMA, Chines… | 71 | 4986 | maintenance |
| Langboat/Mengzi3 Mengzi3 is a series of open-source Chinese-focused large language models (8B and 13B parameters) released by Langboat in Base and Chat vari… | 25 | 1369 | active |
| XiaomiMiMo/MiMo-V2-Flash MiMo-V2-Flash is a 309B-parameter Mixture-of-Experts language model (15B active) from Xiaomi designed for efficient reasoning, coding, and … | 43 | 1368 | active |
| zzli2022/Awesome-System2-Reasoning-LLM A curated awesome-list tracking the latest advances in System-2 reasoning for large language models, accompanying a survey paper on reasoni… | 32 | 1353 | active |
| tanishqkumar/beyond-nanogpt An educational repository of minimal, annotated, from-scratch implementations of ~100 modern deep learning techniques, bridging nanoGPT and… | 48 | 1340 | active |
| Open-Source-O1/Open-O1 Open O1 is an open-source effort to replicate the reasoning capabilities of OpenAI's proprietary O1 model by curating chain-of-thought SFT … | 22 | 1340 | active |
| VikParuchuri/zero_to_gpt A free course of Jupyter notebooks that takes learners from no deep learning knowledge to implementing a GPT model from scratch in Python a… | 32 | 1309 | active |
| Tebmer/Awesome-Knowledge-Distillation-of-LLMs A curated awesome-list of research papers accompanying the survey 'A Survey on Knowledge Distillation of Large Language Models'. It organiz… | 30 | 1305 | active |
| CharlesQ9/Self-Evolving-Agents A curated survey repository cataloging research papers on self-evolving AI agents, organized by what, when, how, and where agents evolve. I… | 39 | 1302 | active |
| NVIDIA/dgx-spark-playbooks A collection of step-by-step playbooks (Jupyter Notebook-based guides) for setting up AI/ML workloads on NVIDIA DGX Spark devices with Blac… | 60 | 1299 | active |
| XueFuzhao/awesome-mixture-of-experts A curated awesome-list collecting papers, open models, and libraries about Mixture-of-Experts (MoE) architectures in deep learning. It serv… | 32 | 1287 | active |
| datawhalechina/diy-llm A Chinese-language, code-driven course (adapted from Stanford CS336) for systematically building large language models from scratch. It cov… | 79 | 1278 | active |
| sail-sg/understand-r1-zero A research codebase and paper reproduction for critically analyzing R1-Zero-like LLM training, examining the roles of base models and reinf… | 37 | 1273 | active |
| callummcdougall/ARENA_3.0 ARENA is a hands-on curriculum of exercises and Streamlit pages covering deep learning fundamentals, transformer interpretability, RL, and … | 72 | 1257 | active |
| whwangovo/pyre-code A self-hosted coding practice platform with ~76 ML implementation problems covering attention, training, RLHF, diffusion, and GNNs, graded … | 51 | 1253 | active |
| Instruction-Tuning-with-GPT-4/GPT-4-LLM A research dataset release of GPT-4-generated instruction-following data for fine-tuning large language models, including English and Chine… | 30 | 4334 | maintenance |
| ScalingIntelligence/KernelBench KernelBench is a benchmark and toolkit from Stanford's Scaling Intelligence Lab that evaluates whether LLMs can generate correct and effici… | 55 | 1214 | active |
| thu-coai/Safety-Prompts A dataset of 100k Chinese safety prompts with ChatGPT responses covering typical unsafe scenarios and instruction attacks, for evaluating a… | 30 | 1214 | stable |
| karpathy/makemore A single-file, hackable PyTorch implementation of an autoregressive character-level language model, ranging from bigrams to a full Transfor… | 32 | 4215 | maintenance |
| Profluent-AI/OpenCRISPR OpenCRISPR is a collection of AI-designed gene editing systems released by Profluent Bio, including OpenCRISPR-1, a Cas9-like protein and g… | 44 | 1204 | active |
| srush/LLM-Training-Puzzles A collection of 8 challenging Jupyter notebook puzzles about training large language models on many GPUs, teaching memory efficiency and co… | 29 | 1190 | stable |
| CrazyBoyM/llama3-Chinese-chat A community repository releasing Chinese post-trained (SFT and DPO) versions of Llama3/Llama3.1, including open model weights, quantized GG… | 55 | 4146 | maintenance |
| helblazer811/Diffusion-Explorer An interactive web-based educational tool that visualizes the geometric intuition behind diffusion and flow-based generative models. It let… | 68 | 1164 | active |
| hans0809/MiniMind-in-Depth A tutorial series providing line-by-line source code analysis of the MiniMind lightweight LLM project, covering tokenizer training, RoPE, M… | 31 | 1152 | active |
| xid32/SoundMind SoundMind is an Audio Logical Reasoning (ALR) dataset of 6,446 audio-text annotated samples with chain-of-thought reasoning, paired with a … | 43 | 1113 | active |
| rasbt/LLM-workshop-2024 A 4-hour hands-on coding workshop by Sebastian Raschka that teaches how large language models are implemented and used, based on his 'Build… | 24 | 1113 | active |
| vivekkalyanarangan30/llm_from_scratch A hands-on Python/PyTorch curriculum that builds large language models from scratch, covering transformer architecture, training, modern im… | 39 | 1102 | active |
| weiruihhh/cs336_note_and_hw A collection of personal course notes and completed homework solutions for Stanford CS336: Building Large Language Models from scratch. It … | 55 | 1088 | active |
| wdndev/tiny-llm-zh An educational project that implements a small-parameter Chinese large language model from scratch, covering the full pipeline: tokenizer t… | 25 | 1078 | active |
| Kent0n-Li/ChatDoctor ChatDoctor is a medical chat model fine-tuned on LLaMA using real patient-doctor conversations, with training code, datasets (HealthCareMag… | 30 | 3631 | maintenance |
| DjangoPeng/LLM-quickstart A quickstart learning resource for large language models combining theoretical study with hands-on fine-tuning practice, delivered as Jupyt… | 38 | 1056 | active |
| LC1332/Luotuo-Chinese-LLM Luotuo (Camel) is an umbrella open-source project for Chinese large language models, releasing a series of fine-tuned LLMs, datasets, train… | 30 | 3588 | maintenance |
| WecoAI/awesome-autoresearch A curated awesome-list of AutoResearch use cases, where a coding agent iteratively optimizes a file against an evaluation metric in a keep/… | 57 | 1038 | active |
| AlphaFin-proj/AlphaFin AlphaFin is a financial analysis benchmark dataset plus StockGPT chat models and the Stock-Chain retrieval-augmented framework, targeting s… | 59 | 1029 | active |
| mryab/efficient-dl-systems Course materials for the Efficient Deep Learning Systems course taught at HSE University and Yandex School of Data Analysis. It covers GPU/… | 70 | 1028 | active |
| iusztinpaul/hands-on-llms A free open-source course teaching LLMs, LLMOps, and vector databases by building a real-time financial advisor LLM system with training, s… | 10 | 3422 | maintenance |
| fufankeji/LLMs-Technology-Community-Beyondata A Chinese-language LLM technology community repository offering full-pipeline tutorials for large language models, covering environment set… | 42 | 1009 | active |
| CVI-SZU/Linly Linly is a project releasing Chinese-adapted open large language models, including Chinese-LLaMA 1&2, Chinese-Falcon, Linly-OpenLLaMA base … | 21 | 3044 | maintenance |
| baichuan-inc/Baichuan-13B Baichuan-13B is an open-source, commercially usable 13-billion-parameter large language model from Baichuan Intelligence, released as both … | 29 | 2926 | maintenance |
| EdjeElectronics/TensorFlow-Object-Detection-API-Tutorial-Train-Multiple-Objects-Windows-10 A step-by-step tutorial repository teaching how to train a TensorFlow Object Detection classifier for multiple custom objects on Windows 10… | 32 | 2919 | maintenance |
| thunlp/UltraChat UltraChat is a large-scale dataset of 1.57M informative, diverse multi-round dialogue instructions generated with LLMs, plus UltraLM, a ser… | 30 | 2890 | maintenance |
| wenge-research/YAYI2 YAYI 2 is a family of open-source multilingual large language models (30B Base and Chat variants) developed by Wenge Research, pretrained o… | 10 | 2796 | maintenance |
| HarderThenHarder/transformers_tasks A collection of Jupyter Notebook-based NLP algorithm implementations built on the Hugging Face transformers library, covering text classifi… | 32 | 2420 | maintenance |
| LinkSoul-AI/Chinese-Llama-2-7b An open-source, commercially usable Chinese-adapted LLaMA 2 7B chat model with a bilingual Chinese-English SFT dataset, following the llama… | 28 | 2203 | maintenance |
| km1994/LLMsNineStoryDemonTower A curated Chinese-language tutorial collection ('Nine-Story Demon Tower') covering hands-on practice with open-source LLMs such as ChatGLM,… | 30 | 2168 | maintenance |
| openai/prm800k PRM800K is OpenAI's process supervision dataset containing 800,000 step-level correctness labels for model-generated solutions to MATH data… | 10 | 2150 | maintenance |
| chenking2020/FindTheChatGPTer A curated Chinese-language list of open-source alternatives to ChatGPT and GPT-4, covering text LLMs and multimodal models such as ChatGLM,… | 30 | 2005 | maintenance |
| thu-coai/CDial-GPT CDial-GPT provides LCCC, a large-scale cleaned Chinese short-text conversation dataset, along with Chinese GPT-2-based dialogue models pre-… | 32 | 1962 | maintenance |
| Linaqruf/kohya-trainer A collection of Jupyter/Colab notebooks adapting kohya-ss's sd-scripts for training Stable Diffusion models, including LoRA and Dreambooth … | 10 | 1899 | maintenance |
| tobegit3hub/tensorflow_template_application A template application demonstrating end-to-end deep learning with TensorFlow, covering data formats (CSV, LIBSVM, TFRecords), multiple net… | 32 | 1874 | maintenance |
| fastai/lm-hackers A Jupyter notebook and accompanying video guide from fast.ai explaining how language models work, including tokenization, base models, inst… | 27 | 1869 | maintenance |
| Arturus/kaggle-web-traffic The 1st place solution code for the Kaggle Web Traffic Time Series Forecasting competition, implementing an RNN encoder-decoder (seq2seq) m… | 32 | 1849 | maintenance |
| VHellendoorn/Code-LMs A guide to using pre-trained large language models of source code, centered on the PolyCoder models with instructions for Hugging Face infe… | 32 | 1842 | maintenance |
| AberHu/Knowledge-Distillation-Zoo A PyTorch reference collection implementing many classic knowledge distillation methods (logits, soft target, attention transfer, FitNet, P… | 32 | 1759 | maintenance |
| charent/ChatLM-mini-Chinese ChatLM-Chinese-0.2B is a small (0.2B parameter) Chinese conversational language model with fully open-sourced training pipeline code coveri… | 28 | 1728 | maintenance |
| XueFuzhao/OpenMoE OpenMoE is a family of open-sourced Mixture-of-Experts (MoE) large language models, including base and chat variants at 8B scale, released … | 28 | 1695 | maintenance |
| jacobhilton/deep_learning_curriculum An advanced deep learning curriculum focused on large language model alignment, written by a researcher as of July 2022. It is a curated re… | 32 | 1686 | maintenance |
| teknium1/GPTeacher GPTeacher is a collection of modular instruction-tuning datasets generated by GPT-4, including General-Instruct, Roleplay-Instruct, Code-In… | 30 | 1667 | maintenance |
| tczhangzhi/pytorch-distributed A collection of PyTorch example scripts demonstrating different distributed/multi-GPU training approaches (DataParallel, torch.distributed,… | 32 | 1655 | maintenance |
| phodal/aigc An open-source ebook (in Chinese) on building real-world applications with large language models, covering prompt writing and management, L… | 19 | 1650 | maintenance |
| Beomi/KoAlpaca KoAlpaca is an open-source Korean instruction-following language model project with training code, datasets, and fine-tuned model weights b… | 30 | 1573 | maintenance |
| sahil280114/codealpaca Code Alpaca is a 20K instruction-following dataset and training code for fine-tuning LLaMA models on code generation tasks, based on Stanfo… | 30 | 1514 | maintenance |
| THUDM/AgentTuning AgentTuning is a research project from Tsinghua University that instruction-tunes LLMs on multi-task agent interaction trajectories to impr… | 27 | 1504 | maintenance |
| openai/following-instructions-human-feedback The official companion repository for OpenAI's InstructGPT paper on aligning language models with human intent via reinforcement learning f… | 10 | 1259 | maintenance |
| hikariming/chat-dataset-baseline A curated Chinese conversational dataset plus fine-tuning scripts for training chat models like ChatGLM, built on top of LLaMA-Factory. It … | 39 | 1191 | maintenance |