huggingface/nanotron

Health v2 · maintenance only

Adoption not part of the score

2800 stars · 328 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Nanotron is a minimalistic Python library from Hugging Face for pretraining large language models with 3D parallelism (data, tensor, and pipeline parallelism). It offers a simple, flexible API for training transformer models on custom datasets at scale.

Use cases

• pretrain a Llama-style transformer from scratch
• train LLMs with data, tensor, and pipeline parallelism
• scale model training across multiple GPUs and nodes
• learn how large-scale LLM training works
• run distributed pretraining on custom datasets
• benchmark distributed training performance

When to choose

• you need a lightweight, hackable codebase for LLM pretraining research
• you want to combine data, tensor, and pipeline parallelism in one tool
• you are learning distributed training techniques via the Ultrascale Playbook
• you want PyTorch-native training without heavyweight framework abstractions

When to avoid

• you only need to fine-tune existing models rather than pretrain from scratch
• you want a turnkey training framework with broad model-type support
• your project requires features outside transformer pretraining
• you need long-term commercial support or extensive documentation

Facets

library · maturity active

llm-training deep-learning machine-learning large-language-models deep-learning machine-learning gpu-computing python 3d-parallelism pretraining transformers distributed-training huggingface gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="huggingface/nanotron")