kimiyoung/transformer-xl

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

3714 stars · 765 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Official implementation of Transformer-XL, an attention-based language model architecture that extends context beyond a fixed length via segment-level recurrence. Includes PyTorch and TensorFlow code, training scripts, and pretrained models achieving state-of-the-art language modeling results.

Use cases

• train a language model with long context beyond fixed length
• download and fine-tune pretrained Transformer-XL models
• reproduce state-of-the-art results on enwiki8 text8 and WikiText-103
• run char-level language modeling on PTB
• study segment-level recurrence attention implementation
• train language models on multiple GPUs or TPUs

When to choose

• you need the original Transformer-XL architecture code for research or reproduction
• you want pretrained models for language modeling benchmarks
• you need both PyTorch and TensorFlow implementations

When to avoid

• you want a maintained library integrated into modern frameworks - use Hugging Face Transformers instead
• you need production LLM inference or serving
• you need recent model architectures or active community support

Facets

library · maturity maintenance

machine-learning deep-learning llm-training transformers deep-learning large-language-models python transformer-xl language-modeling pytorch tensorflow pretrained-models research-code natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="kimiyoung/transformer-xl")