PiotrNawrot/nanoT5 resource

Health v2 · maintenance only

Adoption not part of the score

1021 stars · 78 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

nanoT5 is a PyTorch codebase for pre-training and fine-tuning T5-style encoder-decoder language models on a limited budget, e.g. a single A100 GPU in under 24 hours. It serves as an accessible, reproducible research template and educational baseline, complementing nanoGPT and Cramming.

Use cases

• pre-train a T5 model from scratch on a single GPU
• reproduce T5 v1.1 pre-training in PyTorch
• fine-tune an encoder-decoder model on Super-Natural Instructions
• learn how LLM pre-training pipelines work
• get a starting template for NLP research experiments
• compare against nanoGPT for encoder-decoder architectures

When to choose

• you want to pre-train or fine-tune a T5-style model with limited compute
• you need a reproducible, simple PyTorch baseline for encoder-decoder LLM research
• you want to learn the end-to-end pre-training pipeline

When to avoid

• you need production-scale distributed training on TPU pods
• you want a decoder-only (GPT-style) model - use nanoGPT instead
• you just need to use an off-the-shelf pretrained T5 from HuggingFace

Facets

learning-resource · maturity maintenance

llm-training machine-learning nlp large-language-models deep-learning tutorials python t5 encoder-decoder pre-training pytorch research-template nano-models fine-tuning natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="PiotrNawrot/nanoT5")