Lost in the Middle: How Language Models Use Long Contexts
Summary
Lost in the Middle: How Language Models Use Long Contexts (2023) documented a failure mode that still shows up in long-context systems: models often use the beginning and the end of a long prompt well, and miss evidence placed in the middle.
What it introduced: a controlled study of how position in the prompt affects the chance that a model will actually use a given piece of evidence, across several models and context lengths, with a clear lost-in-the-middle pattern.
What it led to: later long-context benchmarks and compaction research that treat usable context as shorter than the advertised window. Live notes: ao-context-long-context-limits, ao-context-longbench-v2, and 2404-06654-ruler. Batch D also has a pointer to LongBench.
Where to read it: https://arxiv.org/abs/2307.03172 (v3, last revised 20 Nov 2023). Accepted in TACL, 2023.
The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.
License: CC-BY-4.0 · Pointer by iinventedlight (AI-assisted). Source: Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, Percy Liang, Lost in the Middle: How Language Models Use Long Contexts (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.
Cite this
'iinventedlight (AI-assisted)'. 2026. "Lost in the Middle: How Language Models Use Long Contexts." iinventedlight Library, iil:note:p8wv7x8z, version of 2026-11-11. https://iinventedlight.com/library/notes/2307-03172-lost-in-the-middle/. License: CC-BY-4.0.
@misc{iil_note_p8wv7x8z,
author = {{'iinventedlight (AI-assisted)'}},
title = {{Lost in the Middle: How Language Models Use Long Contexts}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-11},
url = {https://iinventedlight.com/library/notes/2307-03172-lost-in-the-middle/},
note = {Permanent ID iil:note:p8wv7x8z. License: CC-BY-4.0.},
}Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2307-03172-lost-in-the-middle.md.