---
source: https://iinventedlight.com/library/notes/hf-transformers-attention-backends.md
title: "Attention backends: sdpa, FlashAttention, flex, paged"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source_license: "Apache-2.0 (huggingface/transformers, huggingface_hub, diffusers docs; LICENSE verified in each repo); prose restates cited source lines in own words"
license_family: hf-apache
attribution: "Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs, Attention backends, Apache-2.0 (huggingface docs repos); Hugging Face docs, How caching works, Apache-2.0 (huggingface docs repos)."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-09
status: published
checked_on: 2026-11-09
as_of: 2026-11-09
confidence: medium
untested: "No code was run for this note; only the fetched documentation pages were read."
category: coding-references
topic_area: "code-ai-eng: Hugging Face"
tags: [hugging-face, transformers, attention, flash-attention]
client: Librarian
sources:
  - id: S1
    title: "Attention backends"
    publisher: Hugging Face docs
    url: https://huggingface.co/docs/transformers/attention_interface
    accessed: 2026-11-09
    license: "Apache-2.0 (huggingface docs repos)"
  - id: S2
    title: "How caching works"
    publisher: Hugging Face docs
    url: https://huggingface.co/docs/transformers/cache_explanation
    accessed: 2026-11-09
    license: "Apache-2.0 (huggingface docs repos)"
---
# Attention backends: sdpa, FlashAttention, flex, paged

**Answer:** All attention implementations perform the same computation; backends differ in how they do it. Use the `attn_implementation` argument in `from_pretrained` to set an attention function.

## Why backends matter

Basic attention materializes the full attention matrix in memory, which slows down inference. Optimized implementations rearrange the math to reduce memory traffic.

## Options

- `sdpa`: the built-in PyTorch implementation of scaled dot product attention; the default when no backend is named.
- `flash_attention_2`: tiles computations into smaller blocks and uses fast on-chip memory.
- `flash_attention_3`: improves FlashAttention-2 by overlapping operations.
- `flex_attention`: custom patterns (sparse, block-local, sliding window) without writing low-level kernels.
- `paged|sdpa`, `paged|eager` and paged FlashAttention: paged versions of SDPA, eager, and FlashAttention.

## Switching and kernels

Switch backends at runtime without reloading the model using `set_attn_implementation`. The Kernels library downloads compiled kernels from the Hub at runtime, avoiding mismatched PyTorch or CUDA versions.

Multimodal models can map each backbone to a different attention function with a dict keyed by sub-config names.

## Artifacts

- [Hugging Face model load choices (table)](https://iinventedlight.com/library/artifacts/hf-load-choices.table.json)
- [Hugging Face model load choices (markdown table)](https://iinventedlight.com/library/artifacts/hf-load-choices.table.md)
- [Topic packet: huggingface](https://iinventedlight.com/library/packets/huggingface.json)

## See also

- [hf-transformers-kv-cache](https://iinventedlight.com/library/notes/hf-transformers-kv-cache/)

## Claims

- id: C1
  claim: All attention implementations compute the same thing.
  source: S1
  evidence: "All attention implementations perform the same computation."

- id: C2
  claim: Backends can be switched at runtime without reloading.
  source: S1
  evidence: "Switch between attention backends at runtime without reloading the model"

- id: C3
  claim: Kernels avoid mismatched PyTorch or CUDA versions.
  source: S1
  evidence: "This avoids packaging issues from mismatched PyTorch or CUDA versions."

- id: C4
  claim: Attention is computed per layer.
  source: S2
  evidence: "Attention is calculated independently in each layer of the model"

## Sources

- [S1] Attention backends. Hugging Face docs. https://huggingface.co/docs/transformers/attention_interface . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.

- [S2] How caching works. Hugging Face docs. https://huggingface.co/docs/transformers/cache_explanation . License: Apache-2.0 (huggingface docs repos). Accessed 2026-11-09.


License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Sources: Hugging Face docs (huggingface/transformers, huggingface/huggingface_hub, huggingface/diffusers on GitHub), Apache License 2.0 (https://www.apache.org/licenses/LICENSE-2.0). Text was condensed and changed from the originals.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).