SakanaAI/self-adaptive-llms

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1225 stars · 142 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Transformer² is a research framework from SakanaAI that adapts large language models to unseen tasks in real-time by selectively adjusting singular components of weight matrices. It uses a two-pass inference mechanism with a task dispatch system and reinforcement-learning-trained expert vectors that are dynamically mixed per prompt.

Use cases

• adapt an LLM to unseen tasks in real-time without full fine-tuning
• train task-specific expert vectors with reinforcement learning
• evaluate self-adaptive LLM behavior on benchmarks
• research lightweight parameter-efficient LLM adaptation
• dynamically mix expert vectors based on incoming prompt task type

When to choose

• you are doing research on self-adaptive or parameter-efficient LLM adaptation
• you want to avoid computationally expensive static fine-tuning for diverse tasks
• you want to reproduce the Transformer² paper results

When to avoid

• you need a production-ready, well-supported LLM serving framework
• you need simple standard fine-tuning rather than adaptive expert-vector mixing
• you lack GPU resources or are not working in a research context

Facets

library · maturity experimental

machine-learning llm-training reinforcement-learning llm-inference large-language-models machine-learning artificial-intelligence python self-adaptation research transformer-squared fine-tuning expert-vectors svd linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="SakanaAI/self-adaptive-llms")