Kyubyong/transformer resource

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

4474 stars · 1303 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A readable TensorFlow 1.x implementation of the Transformer architecture from the paper 'Attention Is All You Need', targeted at machine translation (IWSLT 2016 de-en). It serves primarily as an educational reference with well-commented, modular code.

Use cases

• learn how the transformer architecture works by reading code
• understand positional encoding and masking implementations
• train a small machine translation model on IWSLT de-en
• study a minimal transformer implementation before using production frameworks
• reproduce the Attention Is All You Need paper results on a small dataset

When to choose

• you want to understand the transformer architecture through readable, commented code
• you need a small, quick-to-run transformer for experimentation or teaching
• you are learning TensorFlow 1.x and sequence-to-sequence modeling

When to avoid

• you need production-grade training on large corpora like WMT - use tensor2tensor or PyTorch alternatives
• you need TensorFlow 2.x or modern framework support
• you need beam search decoding or the latest transformer variants

Facets

learning-resource · maturity maintenance

machine-learning deep-learning nlp machine-learning education python transformer attention-mechanism machine-translation tensorflow paper-implementation educational natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="Kyubyong/transformer")