pytorch/torchtitan

Health v2 · maintenance only

Adoption not part of the score

5830 stars · 1044 forks observed · 2026-10-21

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

torchtitan is a PyTorch-native platform for large-scale training of generative AI models, offering a clean-room implementation of PyTorch's distributed scaling techniques such as multi-dimensional parallelism. It supports pretraining LLMs like Llama 3.1 and includes an experimental RL training stack (TitanRL) that integrates with vLLM.

Use cases

• pretrain large language models like Llama 3.1 on GPU clusters
• experiment with multi-dimensional parallelism in PyTorch
• train generative AI models at scale with minimal code changes
• run reinforcement learning training with vLLM generation
• benchmark PyTorch distributed training features
• extend a minimal training codebase with custom model architectures

When to choose

• you want a clean, hackable PyTorch-native codebase for LLM pretraining
• you need to apply FSDP, tensor parallelism, or pipeline parallelism with minimal model code changes
• you are researching new model architectures or distributed training techniques
• you want to stay on the latest PyTorch features and nightlies

When to avoid

• you need a turnkey fine-tuning product with a high-level API like Axolotl or LLaMA-Factory
• you require stable long-term support on older PyTorch releases
• you are not training on GPUs or lack multi-GPU infrastructure
• you need production serving or inference rather than training

Facets

framework · maturity active

llm-training machine-learning deep-learning gpu-computing benchmarking large-language-models deep-learning machine-learning gpu-computing developer-tools python cloud pytorch distributed-training llm-pretraining parallelism reinforcement-learning generative-ai gpu linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="pytorch/torchtitan")