← All notes

EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty

Summary

EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty (2024) is a speculative-decoding method that drafts from second-to-top-layer features rather than from tokens alone, and then accounts for the uncertainty in that feature-level prediction.

What it introduced: the Extrapolation Algorithm for Greater Language-model Efficiency (EAGLE). The authors argue that autoregression is easier at the feature level than at the token level, but that leftover uncertainty at that level limits quality unless it is handled. Their fix is to feed a token sequence advanced by one step so the feature predictor can be accurate with little extra cost. On LLaMA2-Chat 70B they report a 2.7x to 3.5x latency speedup and doubled throughput, without changing the output distribution. Tests also cover Vicuna, Mixtral, and tasks in dialogue, code, math, and instruction following.

What it led to: a widely deployed speculative-decoding method in serving stacks. Live notes: 2211-17192-speculative-decoding and ao-cost-speculative-decoding.

Where to read it: https://arxiv.org/abs/2401.15077 (v3, last revised 4 Mar 2025).

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

License: CC-BY-4.0 · Pointer by iinventedlight (AI-assisted). Source: Yuhui Li, Fangyun Wei, Chao Zhang, Hongyang Zhang, EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.

Cite this

Permanent ID: iil:note:pfv01494

'iinventedlight (AI-assisted)'. 2026. "EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty." iinventedlight Library, iil:note:pfv01494, version of 2026-11-12. https://iinventedlight.com/library/notes/2401-15077-eagle/. License: CC-BY-4.0.

@misc{iil_note_pfv01494,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-12},
  url = {https://iinventedlight.com/library/notes/2401-15077-eagle/},
  note = {Permanent ID iil:note:pfv01494. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2401-15077-eagle.md.