pbloem/former

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1097 stars · 172 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A minimal, from-scratch implementation of the transformer architecture in PyTorch, accompanying a tutorial blog post on how modern transformers work. The GitHub repository is archived and no longer maintained; the latest version lives on Codeberg.

Use cases

• learn how transformer self-attention works
• read a simple transformer implementation in pytorch
• understand transformers without historical baggage
• teach a deep learning class about attention
• experiment with a minimal transformer codebase
• study self-attention sequence-to-sequence operations

When to choose

• you want a small, readable transformer implementation for learning
• you are following the accompanying 'Transformers from scratch' tutorial
• you need educational code rather than production performance

When to avoid

• you need a production-grade or feature-complete transformer library
• you want maintained software with active support
• you need optimized training of large language models

Facets

library · maturity abandoned

machine-learning deep-learning machine-learning deep-learning large-language-models tutorials python transformer pytorch self-attention educational from-scratch archived

Member repositories

For agents

markdown · JSON · MCP: product_card(name="pbloem/former")