test-time-training/ttt-lm-pytorch

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1388 stars · 81 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Official PyTorch implementation of Test-Time Training (TTT) layers, a sequence modeling layer with linear complexity whose hidden state is itself a small ML model updated via self-supervised learning. It provides TTT-Linear and TTT-MLP causal language models integrated with Huggingface Transformers.

Use cases

• run a language model with linear-complexity TTT layers instead of self-attention
• experiment with TTT-Linear and TTT-MLP architectures in PyTorch
• load and generate text with TTTForCausalLM via Huggingface Transformers
• study the implementation of learning-at-test-time sequence layers
• prototype long-context sequence models without quadratic attention cost

When to choose

• you want a readable, tutorial-style PyTorch reference implementation of TTT layers
• you need to integrate TTT models into a Huggingface Transformers pipeline
• you are doing research or education on expressive hidden states for RNNs

When to avoid

• you need fast training - the authors recommend the JAX codebase instead
• you need optimized inference speed - use the ttt-lm-kernels repository
• you need a production-ready, systems-optimized language model implementation

Facets

library · maturity experimental

machine-learning deep-learning llm-inference deep-learning large-language-models python test-time-training rnn sequence-modeling pytorch huggingface-transformers research-code natural-language-processing gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="test-time-training/ttt-lm-pytorch")