Open-source llm-training software
812 open-source libraries, tools and applications tagged llm-training, ranked by how closely llm-training is their primary purpose, then by adoption and maintenance health. Each entry links to a page with sources, use cases and when to choose or avoid it.
812 products, primary matches first, then adoption-weighted; health v2 shown.
| Product | Health v2 | Stars | Maturity |
|---|---|---|---|
| tensorflow/tensorflow TensorFlow is an open‑source end‑to‑end platform for building and deploying machine‑learning models. It provides stable Python and C++ APIs… | 83 | 200740 | active |
| deepseek-ai/DeepSeek-V3 Official repository for DeepSeek-V3, a 671B-parameter Mixture-of-Experts language model (37B activated per token) with open weights, infere… | 26 | 104498 | active |
| karpathy/autoresearch A framework by Andrej Karpathy where AI agents autonomously run LLM research experiments on a single-GPU nanochat training setup. The agent… | 45 | 98112 | active |
| tensorflow/models The TensorFlow Model Garden is a repository of official and community implementations of state-of-the-art machine learning models built wit… | 82 | 77653 | active |
| hiyouga/LlamaFactory LlamaFactory is an open‑source Python platform that streamlines fine‑tuning, quantization, and inference of over 100 LLM and VLM models. It… | 81 | 75500 | active |
| karpathy/nanochat nanochat is a minimal, hackable full-stack LLM training harness that covers tokenization, pretraining, finetuning, evaluation, and inferenc… | 59 | 58750 | active |
| DeepSpeed DeepSpeed is an open‑source deep learning optimization library that enables efficient distributed training and inference of large models. I… | 98 | 43216 | active |
| lyogavin/airllm AirLLM is a Python library that enables running extremely large language models on a single low‑VRAM GPU. It achieves this without quantiza… | 95 | 39932 | active |
| Lightning-AI/pytorch-lightning PyTorch Lightning is a deep learning framework that organizes PyTorch code to eliminate boilerplate for training, finetuning, and scaling A… | 93 | 31417 | stable |
| karpathy/llm.c llm.c is a pure C/CUDA implementation for training large language models like GPT-2 and GPT-3, with no dependency on PyTorch or Python. It … | 34 | 31176 | active |
| huggingface/open-r1 Open R1 is Hugging Face's fully open reproduction of the DeepSeek-R1 training pipeline, providing scripts for supervised fine-tuning, GRPO … | 65 | 26481 | active |
| haotian-liu/LLaVA LLaVA (Large Language and Vision Assistant) is an open-source multimodal large language model framework implementing visual instruction tun… | 21 | 25067 | active |
| deepseek-ai/DeepSeek-Coder DeepSeek Coder is a series of open code language models (1B to 33B) trained from scratch on 2T tokens of code and natural language, with a … | 47 | 24295 | active |
| PaddlePaddle PaddlePaddle is an industrial-grade deep learning framework written in C++ with Python APIs, supporting high-performance single-machine and… | 92 | 24105 | active |
| verl-project/verl Verl is a flexible, efficient, and production‑ready reinforcement learning library for large language models. It provides a modular API to … | 84 | 23828 | active |
| huggingface/peft 🤗 PEFT is a Python library that provides state‑of‑the‑art parameter‑efficient fine‑tuning methods for large pretrained models. It integrate… | 95 | 21795 | active |
| AI4Finance-Foundation/FinGPT FinGPT is an open-source financial large language model project from the AI4Finance Foundation, providing domain-tuned models, fine-tuning … | 76 | 21352 | active |
| kvcache-ai/ktransformers KTransformers is a Python library that provides a high‑performance framework for heterogeneous CPU‑GPU inference and fine‑tuning of large l… | 91 | 19618 | active |
| state-spaces/mamba Mamba is a PyTorch library implementing the selective state space model (SSM) architecture for linear-time sequence modeling, with hardware… | 80 | 18874 | active |
| meta-llama/llama-cookbook The Llama Cookbook is a collection of Jupyter notebooks and scripts that demonstrate how to use Meta's Llama models for inference, fine‑tun… | 54 | 18559 | active |
| microsoft/agent-lightning Agent Lightning is a lightweight (~3,500-line) agentic reinforcement learning framework from Microsoft for training LLM-powered agents with… | 82 | 18410 | active |
| xming521/WeClone WeClone is an end-to-end Python framework for creating a personal AI digital twin by fine-tuning large language models on your exported cha… | 78 | 18278 | active |
| NVIDIA/Megatron-LM NVIDIA's GPU-optimized library for training large transformer models at scale, comprising Megatron-LM (reference training scripts) and Mega… | 100 | 18178 | active |
| microsoft/SkillOpt SkillOpt is a Python library from Microsoft that trains reusable natural-language skills for frozen LLM agents using trajectory-driven edit… | 73 | 17295 | active |
| tracel-ai/burn Burn is a Rust-based tensor library and deep learning framework supporting training and inference through a unified API. It JIT-compiles te… | 86 | 16008 | active |
| kubeflow/kubeflow Kubeflow is a CNCF-graduated, Kubernetes-native toolkit for building AI/ML platforms, comprising subprojects like Pipelines, Notebooks, Tra… | 78 | 15915 | stable |
| modelscope/ms-swift ms-swift is a ModelScope community framework for fine-tuning, inference, evaluation, quantization, and deployment of 600+ text LLMs and 400… | 90 | 15809 | active |
| mindverse/Second-Me Second Me is an open-source platform for training a personal AI self from your own memories using hierarchical memory modeling and alignmen… | 38 | 15682 | active |
| Lightning-AI/litgpt LitGPT is a Python library providing from-scratch, hackable implementations of 20+ open-source large language models with recipes for pretr… | 87 | 13718 | active |
| ShishirPatil/gorilla Gorilla is a UC Berkeley research project for training and evaluating LLMs to make function/tool calls, including the OpenFunctions fine-tu… | 52 | 13044 | active |
| Orchestra-Research/AI-Research-SKILLs An open-source library of ~98 packaged 'skills' (knowledge modules) that turn AI coding agents like Claude Code, Codex, or Gemini into auto… | 69 | 12991 | active |
| PaddlePaddle/PaddleFormers PaddleFormers is a Transformers-style library built on PaddlePaddle providing a model zoo of 100+ large language models and vision-language… | 88 | 12983 | active |
| PaddlePaddle/PaddleNLP PaddleNLP is an easy-to-use NLP and large language model development kit built on the PaddlePaddle deep learning framework, with a large pr… | 59 | 12977 | active |
| sapientinc/HRM Official PyTorch implementation of the Hierarchical Reasoning Model (HRM), a 27M-parameter recurrent architecture with high-level and low-l… | 49 | 12642 | active |
| bmaltais/kohya_ss A Gradio-based GUI and CLI wrapper around Kohya's Stable Diffusion training scripts for fine-tuning diffusion image generation models. It s… | 89 | 12616 | active |
| axolotl-ai-cloud/axolotl Axolotl is a free, open-source, config-driven framework for fine-tuning large language models, supporting SFT, preference learning (DPO/KTO… | 98 | 12591 | active |
| ostris/ai-toolkit An all-in-one open-source training toolkit for finetuning diffusion models (image and video) on consumer-grade hardware. It supports many r… | 72 | 12107 | active |
| ludwig-ai/ludwig Ludwig is a declarative, low-code deep learning framework for training, fine-tuning, and deploying AI models — from LLMs to tabular, image,… | 96 | 11775 | active |
| MiniCPM MiniCPM is a family of small, state-of-the-art on-device language models from OpenBMB, with MiniCPM5-1B being a dense 1B Transformer for lo… | 74 | 11232 | active |
| OpenPipe/ART ART (Agent Reinforcement Trainer) is an open-source Python framework for training multi-step LLM agents using reinforcement learning techni… | 85 | 10813 | active |
| SkyPilot SkyPilot is an open-source AI compute platform that unifies fragmented infrastructure (Kubernetes, Slurm, VMs, 20+ clouds) into a single po… | 100 | 10724 | active |
| bigscience-workshop/petals Petals is a Python library that lets you run and fine-tune large language models (Llama 3.1, Mixtral, Falcon, BLOOM) on a BitTorrent-style … | 23 | 10614 | active |
| facebookresearch/xformers xFormers is a PyTorch-based library of hackable, optimized Transformer building blocks with custom CUDA kernels for fast, memory-efficient … | 87 | 10565 | active |
| deepseek-ai/DeepEP DeepEP is a high-performance GPU communication library for expert parallelism (EP) in MoE training and inference, providing high-throughput… | 55 | 10263 | active |
| OpenRLHF/OpenRLHF OpenRLHF is a high-performance, production-ready open-source RLHF framework built on Ray + vLLM + DeepSpeed for scalable reinforcement lear… | 89 | 10040 | active |
| huggingface/accelerate Hugging Face Accelerate is a Python library that lets you run the same PyTorch training and inference code on any device or distributed con… | 94 | 9916 | stable |
| MakazhanAlpamys/Soup Soup is a Python CLI that fine-tunes and post-trains LLMs from a single YAML config, supporting 23 methods (SFT, DPO, ORPO, SimPO, KTO, etc… | 79 | 9486 | active |
| oumi-ai/oumi Oumi is an open-source Python framework and platform for the end-to-end lifecycle of open-weight LLMs: data synthesis, fine-tuning (SFT, Lo… | 91 | 9407 | active |
| keras-team/autokeras AutoKeras is an AutoML library for deep learning built on Keras, developed by DATA Lab at Texas A&M University. It automates model architec… | 46 | 9336 | active |
| bytedance/monolith Monolith is a deep learning framework built on TensorFlow for large-scale recommendation modeling. It provides collisionless embedding tabl… | 10 | 9286 | active |
| modelscope/modelscope ModelScope is a Python library and ecosystem built on the 'Model-as-a-Service' concept, providing unified APIs to download, run inference o… | 96 | 9149 | active |
| NVIDIA/apex NVIDIA-maintained PyTorch extension providing utilities for easy mixed precision and distributed training. It offers up-to-date CUDA and C+… | 75 | 9000 | active |
| elder-plinius/OBLITERATUS OBLITERATUS is a Python toolkit for abliteration — identifying and removing refusal behaviors from large language models by locating and in… | 79 | 8861 | active |
| DeepDive slime is an open-source LLM post-training framework for reinforcement learning scaling, connecting Megatron-based training with SGLang-base… | 79 | 8606 | active |
| bitsandbytes-foundation/bitsandbytes bitsandbytes is a Python library providing k-bit quantization primitives for PyTorch, enabling 8-bit (LLM.int8()) and 4-bit (QLoRA) quantiz… | 94 | 8514 | active |
| OptimalScale/LMFlow LMFlow is an extensible Python toolkit for finetuning and inference of large foundation models such as LLaMA, GPT-2, and Galactica. It prov… | 61 | 8487 | active |
| lm-sys/FastChat FastChat is an open platform for training, serving, and evaluating large language model based chatbots, originally releasing the Vicuna mod… | 54 | 39548 | maintenance |
| TheLastBen/fast-stable-diffusion A collection of Google Colab notebooks for quickly running Stable Diffusion UIs (AUTOMATIC1111, ComfyUI) and training DreamBooth models for… | 53 | 7911 | active |
| ERNIE Official repository for Baidu's ERNIE 4.5 family of large multimodal models and ERNIEKit, an industrial-grade training toolkit built on Pad… | 62 | 7737 | active |
| EleutherAI/gpt-neox GPT-NeoX is EleutherAI's library for training large-scale autoregressive transformer language models on GPUs, built on NVIDIA's Megatron an… | 67 | 7471 | active |
| LargeWorldModel/LWM Large World Model (LWM) is a family of open-source 7B-parameter multimodal autoregressive transformer models trained on long videos and boo… | 26 | 7425 | active |
| arcee-ai/mergekit mergekit is a Python toolkit for merging pre-trained large language models directly in weight space, supporting many merge methods (SLERP, … | 64 | 7389 | active |
| google/flax Flax is a neural network library and ecosystem for JAX designed for flexibility, featuring the newer NNX API with first-class Python refere… | 99 | 7348 | active |
| InternLM/InternLM Official repository for the InternLM series of open-source large language models (InternLM, InternLM2, InternLM2.5, InternLM3), including p… | 38 | 7276 | active |
| ml-explore/mlx-lm A Python package for running and fine-tuning large language models on Apple silicon using the MLX framework. It integrates with the Hugging… | 78 | 7259 | active |
| kohya-ss/sd-scripts A collection of Python training, generation, and utility scripts for Stable Diffusion and other image generation models, most widely used f… | 96 | 7242 | active |
| deepseek-ai/DeepSpec DeepSpec is a full-stack Python codebase from DeepSeek for training and evaluating draft models used in speculative decoding of large langu… | 51 | 7207 | active |
| flwrlabs/flower Flower (flwr) is an open-source Python framework for building federated and collaborative AI systems, supporting any ML framework such as P… | 99 | 7162 | active |
| ymcui/Chinese-LLaMA-Alpaca-2 A project releasing Chinese-adapted LLaMA-2 base models and Alpaca-2 instruction-tuned chat models with an extended Chinese vocabulary, inc… | 51 | 7112 | stable |
| zai-org/GLM-4 Official repository for the GLM-4 series of open-weight, multilingual (primarily Chinese/English) multimodal chat language models from Z.ai… | 64 | 7069 | active |
| apple/corenet CoreNet is Apple's deep neural network training toolkit for training standard and novel small and large-scale models, including foundation … | 68 | 7008 | active |
| gepa-ai/gepa GEPA is a Python framework that optimizes textual system parameters such as prompts, code, and agent configurations using LLM-based reflect… | 81 | 6996 | active |
| facebookresearch/fairseq Fairseq is a PyTorch-based sequence modeling toolkit from Facebook AI Research for training custom models for translation, summarization, l… | 10 | 32211 | maintenance |
| simplescaling/s1 s1 is an open-source research project implementing simple test-time scaling for large language models, including the s1K dataset of 1,000 c… | 30 | 6675 | active |
| linkedin/Liger-Kernel Liger Kernel is a collection of Triton kernels designed to make LLM training faster and more memory-efficient, offering drop-in replacement… | 91 | 6664 | active |
| yangjianxin1/Firefly Firefly is an open-source one-stop training tool for large language models, supporting pretraining, instruction fine-tuning (SFT), and DPO … | 21 | 6653 | active |
| Harbor Harbor is a Python framework from the creators of Terminal-Bench for evaluating and optimizing AI agents and language models in containeriz… | 86 | 6309 | active |
| Akegarasu/lora-scripts SD-Trainer is a GUI application and set of scripts for training LoRA and Dreambooth fine-tunes of Stable Diffusion diffusion models, wrappi… | 66 | 6166 | active |
| FATE FATE (Federated AI Technology Enabler) is an industrial-grade open-source federated learning framework hosted by the Linux Foundation. It e… | 23 | 6107 | active |
| om-ai-lab/VLM-R1 VLM-R1 is a framework for training R1-style large vision-language models using reinforcement learning (GRPO) on top of Qwen2.5-VL. It provi… | 60 | 6029 | active |
| lucidrains/x-transformers A concise PyTorch library implementing full-attention transformer architectures (encoder, decoder, encoder-decoder, and vision transformers… | 83 | 5960 | active |
| fla-org/flash-linear-attention A PyTorch library providing hardware-efficient implementations of emerging sequence model architectures, including linear attention, sparse… | 86 | 5940 | active |
| rllm-org/rllm rLLM is an open-source Python framework for training language agents with reinforcement learning, supporting any agent harness (Claude Code… | 75 | 5862 | active |
| shibing624/MedicalGPT MedicalGPT is a Python training framework for building medical-domain large language models using the full ChatGPT-style training pipeline:… | 86 | 5843 | active |
| areal-project/AReaL AReaL is a large-scale asynchronous reinforcement learning system that bridges foundation model training with agent-based applications, sup… | 86 | 5842 | active |
| pytorch/torchtitan torchtitan is a PyTorch-native platform for large-scale training of generative AI models, offering a clean-room implementation of PyTorch's… | 81 | 5830 | active |
| google-deepmind/gemma The official JAX-based Python library from Google DeepMind for running, sampling from, and fine-tuning the Gemma family of open-weight larg… | 85 | 5796 | active |
| Gen-Verse/OpenClaw-RL OpenClaw-RL is a framework for training personalized AI agents through reinforcement learning using natural conversation as feedback. It us… | 49 | 5722 | active |
| huggingface/alignment-handbook A collection of robust training recipes and scripts from Hugging Face for aligning large language models with human and AI preferences, cov… | 72 | 5696 | active |
| Parler‑TTS Parler-TTS is a lightweight text-to-speech library from Hugging Face that generates high-quality, natural-sounding speech controllable via … | 26 | 5587 | active |
| LaurentMazare/tch-rs tch-rs is a Rust crate providing thin bindings to the C++ API of PyTorch (libtorch), staying close to the original API. It enables tensor o… | 64 | 5503 | active |
| mosaicml/composer Composer is an open-source PyTorch-based deep learning training library by MosaicML (now Databricks) for training neural networks faster an… | 66 | 5499 | active |
| karpathy/minGPT A minimal, clean PyTorch re-implementation of OpenAI's GPT covering both training and inference in roughly 300 lines of code. It is designe… | 32 | 24956 | maintenance |
| RLinf/RLinf RLinf is an open-source, flexible and scalable reinforcement learning training infrastructure for embodied AI (vision-language-action model… | 72 | 5453 | active |
| PeterGriffinJin/Search-R1 Search-R1 is an open-source reinforcement learning framework for training LLMs that interleave reasoning with search engine calls, built on… | 40 | 5446 | active |
| InternLM/xtuner XTuner is an open-source LLM training engine from InternLM designed for fine-tuning ultra-large-scale Mixture-of-Experts (MoE) models, with… | 68 | 5211 | active |
| lightvector/KataGo KataGo is an open-source Go (baduk) engine trained via AlphaZero-like self-play, one of the strongest Go bots available. It runs as a GTP e… | 96 | 5204 | active |
| Transformer Lab Transformer Lab is an open-source desktop application (built with Electron and Python) that provides a unified GUI for training, fine-tunin… | 82 | 5195 | active |
| hiyouga/EasyR1 EasyR1 is an efficient, scalable reinforcement learning training framework for large language models and vision-language models, built as a… | 62 | 5192 | active |
| h2oai/h2o-llmstudio H2O LLM Studio is a framework and no-code GUI for fine-tuning state-of-the-art large language models, built by H2O.ai. It supports LoRA and… | 96 | 5184 | active |
page 1 / 9 next →