← All notes

Summary

Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention (2024), by Tsendsuren Munkhdalai, Manaal Faruqui, and Siddharth Gopal, is at https://arxiv.org/abs/2404.07143. Infini-attention places a compressive memory inside ordinary attention and, in one Transformer block, combines masked local attention with a long-term linear attention mechanism. The authors report results on long-context language modeling, on passkey retrieval at 1M sequence length, and on book summarization at 500K length, using 1B and 8B models. The approach introduces minimal bounded memory parameters and enables fast streaming inference. They demonstrate the approach on long-context language modeling benchmarks, 1M sequence length passkey context block retrieval and 500K length book summarization tasks with 1B and 8B LLMs. A key component is a new attention technique dubbed Infini-attention, which incorporates a compressive memory into the vanilla attention mechanism and builds in both masked local attention and long-term linear attention mechanisms in a single Transformer block. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v2 (9 Aug 2024), submitted 10 Apr 2024.

Claims

Pointer notes do not quote the paper. See Verification and Sources for the link and license.

Verification

• Format: pointer note only. The paper is not reproduced here.
• Why it matters: It matters as a compact proposal for running Transformers over very long streams with bounded memory, by compressing older context rather than keeping a full key-value cache forever.
• Reproduced: not reproduced (2026-11-11).
• Related work: Related notes already in the library include Million-token context windows and StreamingLLM.
• Limits: This is a pointer note only. The 1M passkey and 500K book-summarization figures are the authors' reported results from the v2 paper (9 Aug 2024).
• License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 10 Apr 2024 (v1); last revised 9 Aug 2024 (v2). Sources: [S1] https://arxiv.org/abs/2404.07143, v2 (9 Aug 2024). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

When to apply

Use when you need the original Infini-attention paper on compressive memory inside a Transformer block for very long streams.

Facts

• id: F1 claim: 'Infini-attention incorporates a compressive memory into vanilla attention and combines masked local attention with long-term linear attention in a single Transformer block.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F2 claim: 'The authors demonstrate the approach on long-context language modeling, 1M sequence length passkey context block retrieval, and 500K length book summarization with 1B and 8B LLMs.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F3 claim: 'They say the method introduces minimal bounded memory parameters and enables fast streaming inference.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0

Sources

• [S1] arXiv 2404.07143, https://arxiv.org/abs/2404.07143. Paper license: arXiv default (non-exclusive distribution license). v2 (9 Aug 2024). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

@misc{iil_note_g080r6fq,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2404-07143-infini-attention/},
  note = {Permanent ID iil:note:g080r6fq. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2404-07143-infini-attention.md.