Fast Transformer Decoding: One Write-Head is All You Need
Summary
Fast Transformer Decoding: One Write-Head is All You Need (2019) introduced multi-query attention, in which many query heads share a single key and value head. The point is faster incremental decoding: inference is often limited by repeatedly loading a large key-value cache, and shrinking that cache reduces the memory-bandwidth cost.
What it introduced: the idea that you do not need a separate key and value projection for every attention head at decode time. One shared write-head for keys and values can preserve most of the quality of multi-head attention while making incremental inference much cheaper.
What it led to: grouped-query attention, now a default in many open language models, and the broader focus on KV-cache cost as a first-class inference constraint. Live note: 2305-13245-gqa. This short paper is the ancestor of that design.
Use it when you want the original multi-query attention note. Where to read it: https://arxiv.org/abs/1911.02150 (v1 (6 Nov 2019)).
The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.
License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.
License: CC-BY-4.0 · Pointer by iinventedlight (AI-assisted). Source: Noam Shazeer, Fast Transformer Decoding: One Write-Head is All You Need (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.
Cite this
'iinventedlight (AI-assisted)'. 2026. "Fast Transformer Decoding: One Write-Head is All You Need." iinventedlight Library, iil:note:759rfm83, version of 2026-11-12. https://iinventedlight.com/library/notes/1911-02150-multi-query-attention/. License: CC-BY-4.0.
@misc{iil_note_759rfm83,
author = {{'iinventedlight (AI-assisted)'}},
title = {{Fast Transformer Decoding: One Write-Head is All You Need}},
howpublished = {iinventedlight Library},
year = {2026},
date = {2026-11-12},
url = {https://iinventedlight.com/library/notes/1911-02150-multi-query-attention/},
note = {Permanent ID iil:note:759rfm83. License: CC-BY-4.0.},
}
Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.
For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/1911-02150-multi-query-attention.md.