← Reference notes

Efficient Memory Management for Large Language Model Serving with PagedAttention

Summary

The KV cache for each request is large and changes size over time. When it is managed poorly, fragmentation and duplication waste memory and limit batch size. PagedAttention borrows OS virtual-memory paging to store the KV cache in blocks. vLLM, built on it, gets near-zero KV-cache waste, shares cache within and across requests, and delivers 2-4x the throughput of FasterTransformer and Orca at the same latency.

Draft. License: CC BY 4.0 (this summary). Sources: [S1] arXiv, "Efficient Memory Management for Large Language Model Serving with PagedAttention" (CC BY 4.0).

When to apply

• Choosing or understanding an LLM inference server; background for why vLLM batches well.

Paper

• Title: Efficient Memory Management for Large Language Model Serving with PagedAttention
• Authors: Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, et al.
• arXiv: https://arxiv.org/abs/2309.06180 (first posted 2023-09-12)
• License: CC BY 4.0 (checked on the arXiv abstract page 2026-11-02)
• Read for this note: arXiv abstract page; figures restate the abstract.

Facts

• id: F1 claim: High-throughput LLM serving needs large batches, but each request's KV cache is large and grows and shrinks dynamically. confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02
• id: F2 claim: Inefficient KV-cache management wastes memory through fragmentation and redundant duplication, limiting batch size. confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02
• id: F3 claim: PagedAttention is an attention algorithm inspired by virtual memory and paging in operating systems. confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02
• id: F4 claim: vLLM achieves near-zero KV-cache waste and flexible KV-cache sharing within and across requests. confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02
• id: F5 claim: vLLM improved throughput 2-4x at the same latency versus FasterTransformer and Orca, more so for longer sequences, larger models and complex decoding (author-reported). confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02

Caveats

• Comparisons are to 2023 baselines; serving systems have moved on.

Pointers

• id: P1 answers: Full method, experiments and appendices where: https://arxiv.org/abs/2309.06180 access: free, public web page

@misc{iil_note_2xd6xte4,
  author = {{iinventedlight Library}},
  title = {{Efficient Memory Management for Large Language Model Serving with PagedAttention}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-02},
  url = {https://iinventedlight.com/library/reference/arxiv/2309-06180-pagedattention-vllm/},
  note = {Permanent ID iil:note:2xd6xte4. License: CC-BY-4.0. Summary of https://arxiv.org/abs/2309.06180.},
}

For language models and agents: start at /llms.txt. This page in Markdown: /library/reference/arxiv/2309-06180-pagedattention-vllm.md.