---
title: 'YaRN: Efficient Context Window Extension of Large Language Models'
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is CC BY 4.0 (verified on arXiv abs page 2026-11-11; icon by-4.0.png, http://creativecommons.org/licenses/by/4.0/); note is our own summary'
source: https://iinventedlight.com/library/notes/2309-00071-yarn.md
source_license: 'CC BY 4.0'
attribution: 'Summary by iinventedlight (AI-assisted). Source: Bowen Peng, Jeffrey Quesnelle, Honglu Fan, Enrico Shippole, YaRN: Efficient Context Window Extension of Large Language Models (CC BY 4.0).'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-11
status: published
approved_by: "Cole Withington, 2026-11-11 20:58 PT, Grok Bot chat widget t1717s7407: \"Publish all 100 including the LLMs Encode Clinical Knowledge note\""
checked_on: 2026-11-11
as_of: 2026-11-11
confidence: medium
untested: 'Not reproduced. Abs page and HTML key sections read 2026-11-11. Named code URLs checked; code not run.'
category: arxiv
topic_area: 'ai-research: memory-and-context'
tags: arxiv, long-context, position-embeddings
arxiv_id: 2309.00071
alias: 2309.00071-yarn
sources:
  - id: S1
    title: 'YaRN: Efficient Context Window Extension of Large Language Models'
    publisher: arXiv
    authors: 'Bowen Peng, Jeffrey Quesnelle, Honglu Fan, Enrico Shippole'
    url: https://arxiv.org/abs/2309.00071
    version_read: 'v3 (6 Feb 2026)'
    accessed: 2026-11-11
    license: 'CC BY 4.0'
---
# YaRN: Efficient Context Window Extension of Large Language Models

## Summary
YaRN (Yet another RoPE extensioN method) is a compute-efficient way to extend the context window of models that use Rotary Position Embeddings. The authors report that it needs 10 times fewer tokens and 2.5 times fewer training steps than previous methods, and that LLaMA models can use and extrapolate to context lengths much longer than their original pre-training. They also say YaRN can extrapolate beyond the limited context of a fine-tuning dataset. This note uses the v3 text last revised 6 Feb 2026. [S1]

## Claims
- id: C1
  claim: 'YaRN is a compute-efficient method to extend the context window of RoPE models.'
  source: S1
  evidence: "a compute-efficient method to extend the context window of such models"
- id: C2
  claim: 'They report 10 times fewer tokens and 2.5 times fewer training steps than previous methods.'
  source: S1
  evidence: "requiring 10x less tokens and 2.5x less training steps than previous methods"
- id: C3
  claim: 'Code is pointed to a public GitHub repository.'
  source: S1
  evidence: "Code is available at https://github.com/jquesnelle/yarn"

## Verification
- Code available: Paper: https://github.com/jquesnelle/yarn, which returned HTTP 200 on 2026-11-11. Not run.
- Reproduced: not reproduced (2026-11-11). Code not run.
- Related work: Live notes ao-context-million-token-windows and 2309-17453-streamingllm-attention-sinks. This is the paper-verification note for YaRN itself.
- Limits: This note follows v3, last revised 6 Feb 2026, which is later than the original 2023 workshop paper. Context-extension quality still depends on the base model and evaluation, and this note does not restate every later benchmark figure.
- License: CC BY 4.0 on the abs page (2026-11-11; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 31 Aug 2023 (v1); last revised 6 Feb 2026 (v3). Sources: [S1] https://arxiv.org/abs/2309.00071, v3 (6 Feb 2026). checked_on: 2026-11-11.

## When to apply
Use when extending a RoPE model's context window with limited continued training.

## Facts
- id: F1
  claim: 'YaRN is a method for extending the context window of RoPE-based language models.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC BY 4.0
- id: F2
  claim: 'The authors report that it requires 10 times fewer tokens and 2.5 times fewer training steps than previous methods.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC BY 4.0
- id: F3
  claim: 'They report that LLaMA models can extrapolate to context lengths longer than their original pre-training.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC BY 4.0

## Sources
- [S1] arXiv 2309.00071, https://arxiv.org/abs/2309.00071. License: CC BY 4.0. v3 (6 Feb 2026). Accessed 2026-11-11.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).