Open-source llm-training software
812 open-source libraries, tools and applications tagged llm-training, ranked by how closely llm-training is their primary purpose, then by adoption and maintenance health. Each entry links to a page with sources, use cases and when to choose or avoid it.
812 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| PrimeIntellect-ai/prime-rl prime-rl is a Python framework for large-scale, fully asynchronous reinforcement learning training of language models, built on FSDP2 for t… | 88 | 2164 | active |
| hexo-ai/sia SIA is a Python framework implementing a self-improving AI loop in which a Meta-Agent generates a task-specific Target Agent and a Feedback… | 56 | 2161 | active |
| google/trax Trax is an end-to-end deep learning library built on JAX and TensorFlow that focuses on clear code and speed, developed and maintained by t… | 10 | 8306 | maintenance |
| kohya-ss/musubi-tuner Musubi Tuner is a set of Python scripts for training LoRA (Low-Rank Adaptation) adapters for video and image generation model architectures… | 88 | 2103 | active |
| Open-Reasoner-Zero/Open-Reasoner-Zero Open-Reasoner-Zero is an open-source implementation of large-scale reinforcement learning training for reasoning-oriented language models, … | 28 | 2098 | active |
| bigcode-project/starcoder2 StarCoder2 is a family of open code generation language models (3B, 7B, 15B) trained on 600+ programming languages from The Stack v2, with … | 27 | 2095 | active |
| HRM-Text HRM-Text is a 1B-parameter text generation model based on the hierarchical recurrent HRM architecture, released with a complete pretraining… | 58 | 2093 | active |
| NVIDIA-NeMo/RL NeMo RL is NVIDIA's open-source post-training library for scaling reinforcement learning methods (GRPO, PPO, DPO, SFT, distillation) on LLM… | 80 | 2073 | active |
| bytedance/Protenix Protenix is an open-source PyTorch reproduction of AlphaFold 3 for high-accuracy biomolecular structure prediction, covering proteins, nucl… | 78 | 2070 | active |
| lupantech/AgentFlow AgentFlow is a trainable, tool-integrated agentic framework that coordinates planner, executor, verifier, and generator modules through an … | 44 | 2064 | active |
| DeePMD-kit DeePMD-kit is a deep learning package for building many-body potential energy representations and running molecular dynamics simulations. I… | 92 | 2055 | stable |
| Yi Yi is a family of open-source large language models trained from scratch by 01.AI, including base and chat models in multiple sizes, with b… | 28 | 7830 | maintenance |
| tdrussell/diffusion-pipe A Python training script for fine-tuning diffusion models (image and video generation) using DeepSpeed pipeline parallelism across multiple… | 67 | 2028 | active |
| XavierXiao/Dreambooth-Stable-Diffusion An implementation of Google's Dreambooth fine-tuning method applied to Stable Diffusion, enabling personalization of a text-to-image diffus… | 32 | 7730 | maintenance |
| cambrian-mllm/cambrian Cambrian-1 is a fully open family of vision-centric multimodal large language models (MLLMs) from NYU's VISIONx group, with training and ev… | 43 | 2011 | active |
| lyhue1991/torchkeras torchkeras is a lightweight PyTorch model training template library that brings Keras-style compile/fit/evaluate APIs to PyTorch. Its core … | 52 | 2010 | active |
| meta-recsys/generative-recommenders Meta's research library implementing HSTU and M-FALCON from the ICML'24 paper 'Actions Speak Louder than Words: Trillion-Parameter Sequenti… | 70 | 2004 | active |
| zai-org/GLM-130B GLM-130B is an open bilingual (English and Chinese) 130-billion-parameter dense language model pre-trained with the General Language Model … | 32 | 7650 | maintenance |
| lucidrains/titans-pytorch An unofficial PyTorch implementation of the Titans architecture, a neural long-term memory module for transformers that learns to memorize … | 64 | 1996 | active |
| haykgrigo3/TimeCapsuleLLM TimeCapsuleLLM is a research project training language models from scratch (and fine-tuning small base models) exclusively on text from spe… | 70 | 1989 | active |
| GPT2-Chinese A Python library providing GPT-2 training and text generation code tailored for Chinese, built on HuggingFace Transformers with BERT or BPE… | 32 | 7594 | maintenance |
| cloneofsimo/lora A Python library for applying Low-Rank Adaptation (LoRA) to quickly fine-tune text-to-image diffusion models like Stable Diffusion. It prod… | 23 | 7557 | maintenance |
| adobe-research/custom-diffusion Custom Diffusion is a research codebase for efficiently fine-tuning text-to-image diffusion models like Stable Diffusion on a few example i… | 66 | 1977 | stable |
| 2U1/Qwen-VL-Series-Finetune An open-source Python repository providing training scripts for fine-tuning Alibaba's Qwen-VL series of vision-language models (Qwen2-VL, Q… | 66 | 1977 | active |
| openlm-research/open_llama OpenLLaMA is a permissively licensed (Apache-2.0) open reproduction of Meta AI's LLaMA, releasing 3B, 7B, and 13B pretrained model weights … | 30 | 7527 | maintenance |
| StarCoder StarCoder is a 15B-parameter code language model trained on 80+ programming languages, and this repository hosts the fine-tuning and infere… | 30 | 7501 | maintenance |
| deepseek-ai/TileKernels TileKernels is a Python library of optimized GPU kernels for LLM operations, written in the TileLang DSL. It provides kernels for MoE routi… | 58 | 1949 | active |
| kyegomez/BitNet A PyTorch implementation of the BitNet architecture from the paper 'BitNet: Scaling 1-bit Transformers for Large Language Models', providin… | 73 | 1947 | active |
| BytedTsinghua-SIA/DAPO DAPO is an open-source reinforcement learning system for large-scale LLM training, released by ByteDance Seed and Tsinghua AIR. It implemen… | 26 | 1946 | active |
| llvm/torch-mlir Torch-MLIR is a compiler project providing first-class translation of PyTorch programs into the MLIR compiler ecosystem. It lets hardware v… | 99 | 1935 | active |
| LeapLabTHU/Absolute-Zero-Reasoner Official implementation of Absolute Zero Reasoner (AZR), a system that trains LLM reasoning via reinforced self-play with zero external dat… | 33 | 1916 | active |
| deepseek-ai/DeepSeek-LLM DeepSeek LLM is a family of open-source large language models (7B and 67B, Base and Chat variants) trained from scratch on 2 trillion token… | 27 | 7251 | maintenance |
| flexflow/flexflow-train FlexFlow Train is a deep learning framework that accelerates distributed DNN training by automatically searching for efficient parallelizat… | 67 | 1903 | active |
| policy-gradient/GRPO-Zero A minimal from-scratch Python implementation of DeepSeek's GRPO (Group Relative Policy Optimization) algorithm for reinforcement learning t… | 24 | 1899 | active |
| d8ahazard/sd_dreambooth_extension A Stable Diffusion WebUI extension that adds DreamBooth fine-tuning capabilities, ported from Shivam Shrirao's optimized Diffusers implemen… | 38 | 1884 | active |
| THUDM/LongWriter LongWriter is a research project from Tsinghua's THUDM group providing fine-tuned LLMs (based on GLM-4-9B and Llama-3.1-8B) plus training a… | 33 | 1874 | active |
| e-p-armstrong/augmentoolkit Augmentoolkit is a Python application that generates domain-expert fine-tuning datasets from uploaded documents and trains custom LLMs on t… | 65 | 1872 | active |
| PRIME-RL/PRIME PRIME (Process Reinforcement through Implicit Rewards) is an open-source Python framework for online reinforcement learning with process re… | 23 | 1872 | active |
| AkaliKong/MiniOneRec MiniOneRec is an open-source framework for generative recommendation built on large language models, covering the full pipeline of semantic… | 59 | 1868 | active |
| PRIME-RL/SimpleVLA-RL SimpleVLA-RL is an open-source reinforcement learning framework for training Vision-Language-Action (VLA) models for robotic manipulation, … | 43 | 1864 | active |
| laekov/fastmoe FastMoE is a PyTorch library providing efficient Mixture of Experts (MoE) layers with custom C/CUDA operators. It supports distributed expe… | 23 | 1860 | active |
| OpenNMT OpenNMT is an open-source ecosystem for neural machine translation and sequence learning, with PyTorch (OpenNMT-py) and TensorFlow (OpenNMT… | 40 | 7018 | maintenance |
| openreasoner/openr OpenR is an open-source Python framework that integrates search, reinforcement learning, and process supervision to improve chain-of-though… | 23 | 1859 | active |
| Tele-AI/Telechat TeleChat is a family of open-source bilingual (Chinese-English) large language models (1B, 7B, 12B) developed by China Telecom's AI team, r… | 64 | 1852 | active |
| trevin-creator/autoresearch-mlx An Apple Silicon (MLX) port of Karpathy's autoresearch that runs autonomous AI research loops on a Mac without PyTorch or CUDA. A coding ag… | 52 | 1852 | active |
| Robbyant/lingbot-vla LingBot-VLA is a Vision-Language-Action foundation model for robot manipulation, pretrained on 20,000 hours of real-world dual-arm robot da… | 52 | 1837 | active |
| ytongbai/LVM LVM is a large vision model trained with sequential next-token prediction over 'visual sentences', using no linguistic data. It builds on O… | 31 | 1836 | active |
| microsoft/mattergen MatterGen is Microsoft's official implementation of a generative diffusion model for designing inorganic crystalline materials across the p… | 65 | 1830 | active |
| Angel Angel is a high-performance distributed parameter server for large-scale machine learning and graph computing, developed by Tencent and Pek… | 66 | 6794 | maintenance |
| thu-ml/RoboticsDiffusionTransformer RDT-1B is a 1B-parameter diffusion foundation model for robot bimanual manipulation, pre-trained on 1M+ multi-robot episodes to predict rob… | 48 | 1806 | active |
| SmartFlowAI/EmoLLM EmoLLM is a series of open-source large language models fine-tuned for mental health understanding and support, built on models like Intern… | 56 | 1786 | active |
| DataArcTech/DataArc-SynData-Toolkit DataArc SynData Toolkit is a Python-based synthetic data generation platform for creating customized LLM training data from local corpora, … | 59 | 1783 | active |
| Simple-Efficient/RL-Factory RLFactory is a reinforcement learning post-training framework for training agentic LLM models, decoupling the environment from RL training … | 31 | 1783 | active |
| Emu Series Emu3 is a suite of state-of-the-art multimodal models from BAAI trained solely with next-token prediction, tokenizing images, text, and vid… | 54 | 1777 | active |
| AnswerDotAI/ModernBERT ModernBERT is the research repository for a modernized BERT-family bidirectional encoder trained on 2 trillion tokens with an 8192-token co… | 53 | 1758 | active |
| Maximal Update Parametrization (μP) The `mup` Python package implements Maximal Update Parametrization (μP) for PyTorch models, enabling optimal hyperparameters to remain stab… | 23 | 1757 | stable |
| T5 The official T5 library from Google Research, implementing the text-to-text transfer transformer for NLP tasks like summarization, question… | 67 | 6562 | maintenance |
| SamsungSAILMontreal/TinyRecursiveModels Official codebase for the Tiny Recursive Model (TRM) paper, a recursive reasoning approach where a tiny 7M-parameter neural network iterati… | 10 | 6561 | maintenance |
| facebookresearch/metaseq Metaseq is a PyTorch codebase from Meta AI for training and working with large-scale Open Pre-trained Transformers (OPT), forked from fairs… | 10 | 6544 | maintenance |
| sunny-glow/Auto-BenchMax Auto-BenchMax is a Python pipeline that automatically synthesizes benchmark-targeted training data for LLMs, claiming to more than double a… | 52 | 1745 | active |
| facebookresearch/multimodal TorchMultimodal is a PyTorch library from Meta for training state-of-the-art multimodal multi-task models at scale, covering both content u… | 77 | 1740 | active |
| AvaLovelace1/BrickGPT BrickGPT is the official implementation of an ICCV 2025 Best Paper approach that generates physically stable, buildable toy brick (LEGO) mo… | 54 | 1726 | active |
| meta-pytorch/tnt TNT (torchtnt) is a lightweight library from Meta providing tools and utilities for PyTorch training workflows. It offers building blocks f… | 76 | 1724 | active |
| facebookresearch/coconut Official PyTorch implementation of Coconut, a method for training large language models to reason in a continuous latent space instead of e… | 58 | 1723 | active |
| lightly-ai/lightly-train LightlyTrain is a Python framework for training computer vision models, covering pretraining of vision foundation models (DINOv2/v3) on unl… | 83 | 1716 | active |
| McGill-NLP/llm2vec LLM2Vec is a Python library that converts decoder-only large language models into powerful text encoders via bidirectional attention, maske… | 46 | 1716 | active |
| kingoflolz/mesh-transformer-jax A JAX/Haiku library implementing model-parallel training and inference of transformer models using xmap/pjit operators, similar to Megatron… | 32 | 6384 | maintenance |
| lsdefine/simple_GRPO A minimal (~200 lines) Python implementation of GRPO reinforcement learning for training LLMs to develop r1-like reasoning, built on PyTorc… | 41 | 1714 | active |
| jiaweizzhao/GaLore GaLore is a PyTorch library implementing Gradient Low-Rank Projection for memory-efficient full-parameter training of large language models… | 26 | 1708 | active |
| xingyaoww/code-act CodeAct is a research framework and agent system that uses executable Python code as a unified action space for LLM agents, enabling multi-… | 27 | 1705 | active |
| lucidrains/alphafold3-pytorch A PyTorch implementation of Google DeepMind's Alphafold 3 model for protein and molecular structure prediction. It is a research-oriented r… | 67 | 1699 | active |
| TensorPack Tensorpack is a high-level neural network training interface built on graph-mode TensorFlow, focused on training speed and flexibility for … | 23 | 6284 | maintenance |
| elixir-nx/axon Axon is a neural network library for Elixir built on top of Nx numerical definitions. It provides a functional API, a high-level model crea… | 89 | 1692 | active |
| intelligent-machine-learning/dlrover DLRover is an automatic distributed deep learning system that manages training of large AI models on Kubernetes and Ray clusters. It provid… | 80 | 1689 | active |
| Anemll/Anemll ANEMLL is an open-source Python library and toolchain for converting and running large language models on the Apple Neural Engine (ANE) via… | 70 | 1686 | active |
| AgentR1/Agent-R1 Agent-R1 is a modular Python framework for training LLM agents with end-to-end reinforcement learning. It models each interaction turn as a… | 65 | 1682 | active |
| gensyn-ai/rl-swarm RL Swarm is an open-source, permissionless peer-to-peer framework for running reinforcement learning training swarms over the internet, bui… | 49 | 1676 | active |
| Gen-Verse/MMaDA MMaDA is an open-source family of multimodal large diffusion language models that unify textual reasoning, multimodal understanding, and te… | 46 | 1673 | active |
| Tencent/AngelSlim AngelSlim is a Python toolkit from Tencent for compressing large language models and related architectures (VLMs, diffusion, audio models) … | 70 | 1669 | active |
| JiuhaiChen/BLIP3o Official implementation of the BLIP3o-Series, a unified autoregressive-plus-diffusion model for text-to-image generation and editing. It co… | 41 | 1667 | active |
| ZJU4HealthCare/HealthGPT HealthGPT is a medical multimodal large language model family unifying medical image comprehension and generation via heterogeneous knowled… | 60 | 1662 | active |
| OpenScholar OpenScholar is a retrieval-augmented language model system from the Allen Institute for AI that answers scientific questions by searching t… | 35 | 1659 | active |
| XueZeyue/DanceGRPO Official implementation of DanceGRPO, a framework applying Group Relative Policy Optimization (GRPO) to fine-tune visual generation models … | 37 | 1654 | active |
| GoMLX GoMLX is an accelerated machine learning and math framework for Go, comparable to PyTorch/JAX/TensorFlow. It offers differentiable operator… | 96 | 1651 | active |
| bowang-lab/scGPT scGPT is a foundation model for single-cell multi-omics built with a generative transformer, providing pretrained checkpoints and a Python … | 64 | 1646 | active |
| pengxiao-song/LaWGPT LaWGPT is a series of open-source large language models tuned with Chinese legal knowledge, built on Chinese-LLaMA/ChatGLM base models with… | 30 | 6061 | maintenance |
| meta-llama/synthetic-data-kit A Python CLI tool from Meta for generating high-quality synthetic datasets to fine-tune LLMs. It follows a four-stage pipeline (ingest, cre… | 39 | 1639 | active |
| Higgsfield Higgsfield is an open-source GPU orchestration and machine learning framework for fault-tolerant, distributed training of very large models… | 67 | 6047 | maintenance |
| yoshitomo-matsubara/torchdistill torchdistill is a modular, configuration-driven PyTorch framework for knowledge distillation and general deep learning experiments, requiri… | 84 | 1633 | active |
| aiwaves-cn/agents Agents 2.0 is a Python framework for data-centric, self-evolving autonomous language agents based on symbolic learning. It adapts neural-ne… | 29 | 5964 | maintenance |
| PKU-Alignment/safe-rlhf Beaver is a modular open-source framework from Peking University for training large language models with SFT, RLHF, and Safe RLHF (constrai… | 50 | 1617 | active |
| PaddlePaddle/PaddleSlim PaddleSlim is an open-source library built on PaddlePaddle for deep learning model compression and architecture search. It provides low-bit… | 46 | 1612 | active |
| OpenGVLab/LLaMA-Adapter Official implementation of LLaMA-Adapter and its V2 successor, a parameter-efficient fine-tuning method that adapts LLaMA models to follow … | 21 | 5913 | maintenance |
| SalesforceAIResearch/uni2ts Uni2TS is a PyTorch library for unified pre-training, fine-tuning, inference, and evaluation of universal time series forecasting transform… | 54 | 1599 | active |
| alibaba/Pai-Megatron-Patch Pai-Megatron-Patch is an open-source deep learning training toolkit from Alibaba Cloud for large-scale training and inference of LLMs and V… | 50 | 1599 | active |
| databricks/megablocks MegaBlocks is a lightweight Python library for efficient training of mixture-of-experts (MoE) models, built around its dropless-MoE (dMoE) … | 59 | 1596 | active |
| stanfordnlp/pyreft pyreft is Stanford NLP's Python library for Representation Finetuning (ReFT), which adapts frozen language models by learning task-specific… | 52 | 1587 | active |
| meta-pytorch/torchtune Torchtune is a PyTorch-native library for authoring, post-training, and experimenting with large language models. It provides hackable trai… | 68 | 5812 | maintenance |
| Xilinx/brevitas Brevitas is a PyTorch library for neural network quantization supporting both post-training quantization (PTQ) and quantization-aware train… | 93 | 1582 | active |
| tensorflow/model-optimization The TensorFlow Model Optimization Toolkit (tfmot) is a Python library providing tools to optimize machine learning models for deployment, i… | 79 | 1579 | stable |