← All notes

Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach

Summary

This paper compares retrieval-augmented generation (RAG) with long-context (LC) models that read a long input directly, then offers a hybrid. The authors benchmark both approaches on public datasets with three recent models. They find that when resourced sufficiently, LC consistently outperforms RAG on average, while RAG remains much cheaper. They propose Self-Route, a method that routes queries to RAG or LC based on model self-reflection. Self-Route significantly reduces the computation cost while maintaining a comparable performance to LC. The v2 paper is the EMNLP 2024 industry-track version. [S1]

Claims

Verification

When to apply

Use when deciding whether to retrieve, to stuff a long context, or to route between the two.

Facts

Sources

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, Michael Bendersky, Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach (CC BY 4.0).

Cite this

Permanent ID: iil:note:dw2bpacm

'iinventedlight (AI-assisted)'. 2026. "Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach." iinventedlight Library, iil:note:dw2bpacm, version of 2026-11-12. https://iinventedlight.com/library/notes/2407-16833-rag-or-long-context/. License: CC-BY-4.0.

@misc{iil_note_dw2bpacm,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{Retrieval Augmented Generation or Long-Context LLMs? A Comprehensive Study and Hybrid Approach}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-12},
  url = {https://iinventedlight.com/library/notes/2407-16833-rag-or-long-context/},
  note = {Permanent ID iil:note:dw2bpacm. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2407-16833-rag-or-long-context.md.