# jquesnelle/yarn

YaRN: Efficient Context Window Extension of Large Language Models

Repository: https://github.com/jquesnelle/yarn
Canonical: https://ross.abutalabs.com/products/jquesnelle-yarn
Language: Python
License: MIT
License Family: permissive
Last push: 2024-04-17T18:29:36+00:00

## Health v2 (maintenance only)
Score: 29/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 35, longevity 83
- inputs: {"age_days": 1164, "days_push": 868, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1777, forks 132 (observed 2026-08-28T04:05:34.672238+00:00)

## What it is
Reference implementation of YaRN, an efficient method for extending the context window of large language models, published as an ICLR 2024 paper. It includes training code, data, and fine-tuned Llama 2, Mistral, and SOLAR model variants with 32K-128K context windows on Hugging Face.

## Use cases
- extend the context window of a llama model to 128k tokens
- fine-tune mistral 7b for long context
- reproduce the yarn paper results
- train a model with rope scaling for long sequences
- download long-context llama 2 checkpoints
- apply yarn context extension to my own llm

## When to choose
- you need to extend a transformer LLM's context window efficiently
- you want ready-made long-context Llama 2 or Mistral checkpoints
- you want to reproduce or build on the YaRN research

## When to avoid
- you need a maintained production training framework rather than a research repo
- you use models other than Llama/Mistral/SOLAR without porting the method
- you need closed-context or non-transformer architectures

## Facets
- artifact type: library
- maturity: maintenance
- function: llm-training, machine-learning, deep-learning
- domain: large-language-models, deep-learning, machine-learning
- platform: python
- tags: context-window-extension, yarn, llama, mistral, positional-encoding, fine-tuning, research-paper, long-context, gpu, linux

## Member repositories
- jquesnelle/yarn (main) score 29

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:34.672238+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:25:13.172998+00:00, confidence not recorded.
  - readme: https://github.com/jquesnelle/yarn (fetched 2026-08-28T04:05:34.672238+00:00, sha 08b886bc66ad)
- Data as of 2026-08-30T08:39:29.467469+00:00.