---
title: "Efficient Memory Management for Large Language Model Serving with PagedAttention"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source: https://arxiv.org/abs/2309.06180
source_license: "CC BY 4.0"
attribution: "Summary by iinventedlight (AI-assisted). Source: arXiv, \"Efficient Memory Management for Large Language Model Serving with PagedAttention\" (CC BY 4.0)."
condensed_by: "iinventedlight (Grok Bot, AI-assisted)"
fact_checked: 2026-11-02
date: 2026-11-02
status: reviewed
reviewed_on: 2026-11-02
category: arxiv
arxiv_id: 2309-06180-pagedattention-vllm
alias: 2309-06180-pagedattention-vllm
checked_on: 2026-11-02
license_url: https://creativecommons.org/licenses/by/4.0/
source: https://arxiv.org/abs/2309.06180
source_license: "CC BY 4.0 (arXiv paper)"
arxiv_license_url: https://creativecommons.org/licenses/by/4.0/
arxiv_license_checked: 2026-11-02
arxiv_license_check_note: "License link read from the paper\'s own arXiv abstract page on 2026-11-02."
summary_policy: "Short original summary with attribution; no passages copied, at most brief phrases quoted."
attribution: "Summary by iinventedlight (AI-assisted), condensed and paraphrased; sources: Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, et al., \"Efficient Memory Management for Large Language Model Serving with PagedAttention\", arXiv:2309.06180 (CC BY 4.0)"
condensed_by: "iinventedlight (Grok Bot, AI-assisted)"
fact_checked: 2026-11-02
date: 2026-11-02
category: arxiv
tags: arxiv, llm, paper-summary
checked_on: 2026-11-02
sources:
  - id: S1
    title: "Efficient Memory Management for Large Language Model Serving with PagedAttention"
    authors: "Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, et al."
    publisher: "arXiv"
    url: https://arxiv.org/abs/2309.06180
    accessed: 2026-11-02
    license: "CC BY 4.0"
    attribution: "Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, et al., \"Efficient Memory Management for Large Language Model Serving with PagedAttention\", arXiv:2309.06180 (CC BY 4.0)"
---
# Efficient Memory Management for Large Language Model Serving with PagedAttention

# Efficient Memory Management for Large Language Model Serving with PagedAttention

## Summary
The KV cache for each request is large and changes size over time. When it is managed poorly, fragmentation and duplication waste memory and limit batch size. PagedAttention borrows OS virtual-memory paging to store the KV cache in blocks. vLLM, built on it, gets near-zero KV-cache waste, shares cache within and across requests, and delivers 2-4x the throughput of FasterTransformer and Orca at the same latency.

*Draft. License: CC BY 4.0 (this summary). Sources: [S1] arXiv, "Efficient Memory Management for Large Language Model Serving with PagedAttention" (CC BY 4.0).*

## When to apply
- Choosing or understanding an LLM inference server; background for why vLLM batches well.

## Paper
- Title: Efficient Memory Management for Large Language Model Serving with PagedAttention
- Authors: Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, et al.
- arXiv: https://arxiv.org/abs/2309.06180 (first posted 2023-09-12)
- License: CC BY 4.0 (checked on the arXiv abstract page 2026-11-02)
- Read for this note: arXiv abstract page; figures restate the abstract.

## Facts
- id: F1
  claim: High-throughput LLM serving needs large batches, but each request's KV cache is large and grows and shrinks dynamically.
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02
- id: F2
  claim: Inefficient KV-cache management wastes memory through fragmentation and redundant duplication, limiting batch size.
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02
- id: F3
  claim: PagedAttention is an attention algorithm inspired by virtual memory and paging in operating systems.
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02
- id: F4
  claim: vLLM achieves near-zero KV-cache waste and flexible KV-cache sharing within and across requests.
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02
- id: F5
  claim: vLLM improved throughput 2-4x at the same latency versus FasterTransformer and Orca, more so for longer sequences, larger models and complex decoding (author-reported).
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02

## Caveats
- Comparisons are to 2023 baselines; serving systems have moved on.

## Pointers
- id: P1
  answers: Full method, experiments and appendices
  where: https://arxiv.org/abs/2309.06180
  access: free, public web page