# huggingface/picotron

Minimalistic 4D-parallelism distributed training framework for education purpose

Repository: https://github.com/huggingface/picotron
Canonical: https://ross.abutalabs.com/products/picotron
Language: Python
License: Apache-2.0
License Family: permissive
Last push: 2025-08-26T13:47:24+00:00

## Health v2 (maintenance only)
Score: 40/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 38, release rhythm 35, longevity 51
- inputs: {"age_days": 714, "days_push": 372, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2289, forks 202 (observed 2026-08-28T04:06:34.800574+00:00)

## What it is
Picotron is a minimalist, hackable distributed training framework for pre-training Llama-like large language models using 4D parallelism (data, tensor, pipeline, and context parallel). Built by Hugging Face in the spirit of NanoGPT, it prioritizes readability and education over performance, with each core file kept under 300 lines of code.

## Use cases
- learn how distributed LLM training works
- pre-train a Llama-style model with data, tensor, and pipeline parallelism
- understand 4D parallelism with a small readable codebase
- experiment with distributed training techniques on a multi-GPU cluster
- follow a tutorial to build a distributed training framework from scratch
- run 3D parallelism training on Slurm

## When to choose
- you want to learn or teach distributed LLM training concepts
- you need a small, hackable codebase to modify for research experiments
- you want a simpler alternative to Megatron-LM or Nanotron for study
- you are pre-training small-to-medium Llama-like models on a few GPUs

## When to avoid
- you need maximum training throughput or production-grade performance
- you want a battle-tested framework for large-scale production pre-training
- you need broad model architecture support beyond Llama-like models
- you require extensive documentation and enterprise support

## Facets
- artifact type: framework
- maturity: active
- function: llm-training, machine-learning, deep-learning
- domain: large-language-models, deep-learning, machine-learning, education
- platform: python
- tags: distributed-training, 4d-parallelism, llm-pretraining, educational, pytorch, tensor-parallelism, pipeline-parallelism, data-parallelism, context-parallelism, nanogpt-inspired, gpu, linux, docker

## Member repositories
- huggingface/picotron (main) score 40

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:34.800574+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:40:49.504832+00:00, confidence not recorded.
  - readme: https://github.com/huggingface/picotron (fetched 2026-08-28T04:06:34.800574+00:00, sha 45cece186da9)
- Data as of 2026-08-30T08:39:29.467469+00:00.