jquesnelle/yarn

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1777 stars · 132 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Reference implementation of YaRN, an efficient method for extending the context window of large language models, published as an ICLR 2024 paper. It includes training code, data, and fine-tuned Llama 2, Mistral, and SOLAR model variants with 32K-128K context windows on Hugging Face.

Use cases

• extend the context window of a llama model to 128k tokens
• fine-tune mistral 7b for long context
• reproduce the yarn paper results
• train a model with rope scaling for long sequences
• download long-context llama 2 checkpoints
• apply yarn context extension to my own llm

When to choose

• you need to extend a transformer LLM's context window efficiently
• you want ready-made long-context Llama 2 or Mistral checkpoints
• you want to reproduce or build on the YaRN research

When to avoid

• you need a maintained production training framework rather than a research repo
• you use models other than Llama/Mistral/SOLAR without porting the method
• you need closed-context or non-transformer architectures

Facets

library · maturity maintenance

llm-training machine-learning deep-learning large-language-models deep-learning machine-learning python context-window-extension yarn llama mistral positional-encoding fine-tuning research-paper long-context gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="jquesnelle/yarn")