← All notes

Summary

YaRN (Yet another RoPE extensioN method) is a compute-efficient way to extend the context window of models that use Rotary Position Embeddings. The authors report that it needs 10 times fewer tokens and 2.5 times fewer training steps than previous methods, and that LLaMA models can use and extrapolate to context lengths much longer than their original pre-training. They also say YaRN can extrapolate beyond the limited context of a fine-tuning dataset. This note uses the v3 text last revised 6 Feb 2026. [S1]

Claims

• id: C1 claim: 'YaRN is a compute-efficient method to extend the context window of RoPE models.' source: S1 evidence: "a compute-efficient method to extend the context window of such models"
• id: C2 claim: 'They report 10 times fewer tokens and 2.5 times fewer training steps than previous methods.' source: S1 evidence: "requiring 10x less tokens and 2.5x less training steps than previous methods"
• id: C3 claim: 'Code is pointed to a public GitHub repository.' source: S1 evidence: "Code is available at https://github.com/jquesnelle/yarn"

Verification

• Code available: Paper: https://github.com/jquesnelle/yarn, which returned HTTP 200 on 2026-11-11. Not run.
• Reproduced: not reproduced (2026-11-11). Code not run.
• Related work: Live notes ao-context-million-token-windows and 2309-17453-streamingllm-attention-sinks. This is the paper-verification note for YaRN itself.
• Limits: This note follows v3, last revised 6 Feb 2026, which is later than the original 2023 workshop paper. Context-extension quality still depends on the base model and evaluation, and this note does not restate every later benchmark figure.
• License: CC BY 4.0 on the abs page (2026-11-11; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 31 Aug 2023 (v1); last revised 6 Feb 2026 (v3). Sources: [S1] https://arxiv.org/abs/2309.00071, v3 (6 Feb 2026). checked_on: 2026-11-11.

When to apply

Use when extending a RoPE model's context window with limited continued training.

Facts

• id: F1 claim: 'YaRN is a method for extending the context window of RoPE-based language models.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
• id: F2 claim: 'The authors report that it requires 10 times fewer tokens and 2.5 times fewer training steps than previous methods.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0
• id: F3 claim: 'They report that LLaMA models can extrapolate to context lengths longer than their original pre-training.' confidence: medium date: 2026-11-11 sources: [S1] license: CC BY 4.0

Sources

• [S1] arXiv 2309.00071, https://arxiv.org/abs/2309.00071. License: CC BY 4.0. v3 (6 Feb 2026). Accessed 2026-11-11.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

@misc{iil_note_1t0wvc6s,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{YaRN: Efficient Context Window Extension of Large Language Models}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2309-00071-yarn/},
  note = {Permanent ID iil:note:1t0wvc6s. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2309-00071-yarn.md.