lucidrains/x-transformers

Health v2 · maintenance only

Adoption not part of the score

5942 stars · 517 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A concise PyTorch library implementing full-attention transformer architectures (encoder, decoder, encoder-decoder, and vision transformers) with many experimental attention features drawn from recent research papers. It lets researchers and practitioners quickly assemble GPT-like, BERT-like, or ViT models with a few lines of code.

Use cases

• build a gpt-style decoder-only transformer in pytorch
• implement an encoder-decoder transformer for seq2seq tasks
• experiment with new attention mechanisms from papers
• train a vision transformer for image classification
• prototype transformer architectures for research
• build an image captioning model

When to choose

• you want a lightweight, hackable transformer implementation for research or experimentation
• you need quick access to many experimental attention variants without writing them yourself
• you are prototyping custom architectures rather than serving production LLMs

When to avoid

• you need a production-grade, optimized inference engine for large language models
• you want a full training framework with data pipelines, distributed training, and fine-tuning tooling
• you prefer batteries-included frameworks like Hugging Face Transformers with pretrained model hubs

Facets

library · maturity active

machine-learning deep-learning llm-training deep-learning large-language-models artificial-intelligence python transformers attention-mechanism pytorch research encoder-decoder gpt bert vision-transformer natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="lucidrains/x-transformers")