---
title: 'LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale'
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is CC BY 4.0 (verified on arXiv abs page 2026-11-11; icon by-4.0.png, http://creativecommons.org/licenses/by/4.0/); note is our own summary'
source: https://iinventedlight.com/library/notes/2208-07339-llm-int8.md
source_license: 'CC BY 4.0'
attribution: 'Summary by iinventedlight (AI-assisted). Source: Tim Dettmers, Mike Lewis, Younes Belkada, Luke Zettlemoyer, LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale (CC BY 4.0).'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-11
status: published
approved_by: "Cole Withington, 2026-11-11 20:58 PT, Grok Bot chat widget t1717s7407: \"Publish all 100 including the LLMs Encode Clinical Knowledge note\""
checked_on: 2026-11-11
as_of: 2026-11-11
confidence: medium
untested: 'Not reproduced. Abs page and HTML key sections read 2026-11-11. Named code URLs checked; code not run.'
category: arxiv
topic_area: 'ai-research: efficiency'
tags: arxiv, efficiency, quantization
arxiv_id: 2208.07339
alias: 2208.07339-llm-int8
sources:
  - id: S1
    title: 'LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale'
    publisher: arXiv
    authors: 'Tim Dettmers, Mike Lewis, Younes Belkada, Luke Zettlemoyer'
    url: https://arxiv.org/abs/2208.07339
    version_read: 'v2 (10 Nov 2022)'
    accessed: 2026-11-11
    license: 'CC BY 4.0'
---
# LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale

## Summary
LLM.int8() is an 8-bit matrix multiplication procedure for transformer feed-forward and attention projection layers. The authors say it cuts the memory needed for inference by half while retaining full precision performance, including for checkpoints with up to 175B parameters. The method uses vector-wise quantization plus a mixed-precision decomposition that isolates outlier feature dimensions in 16-bit, while still more than 99.9% of values are multiplied in 8-bit. They open source the software. [S1]

## Claims
- id: C1
  claim: 'The procedure cuts the memory needed for inference by half while retaining full precision performance.'
  source: S1
  evidence: "cut the memory needed for inference by half while retaining full precision performance"
- id: C2
  claim: 'They say more than 99.9% of values are multiplied in 8-bit, with outliers in 16-bit.'
  source: S1
  evidence: "still more than 99.9% of values are multiplied in 8-bit"
- id: C3
  claim: 'They open-source the software at a public GitHub repository.'
  source: S1
  evidence: "https://github.com/TimDettmers/bitsandbytes"

## Verification
- Code available: Paper: https://github.com/TimDettmers/bitsandbytes, which returned HTTP 200 and redirected to https://github.com/bitsandbytes-foundation/bitsandbytes on 2026-11-11. Not run.
- Reproduced: not reproduced (2026-11-11). Code not run.
- Related work: Live note 2305-14314-qlora. This is the paper-verification note for LLM.int8() itself.
- Limits: The 'no performance degradation' claim is the authors' reported result for this inference-time method on the models they tested. Later quantization methods may trade a small quality loss for higher compression.
- License: CC BY 4.0 on the abs page (2026-11-11; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 15 Aug 2022 (v1); last revised 10 Nov 2022 (v2). Sources: [S1] https://arxiv.org/abs/2208.07339, v2 (10 Nov 2022). checked_on: 2026-11-11.

## When to apply
Use when the goal is 8-bit inference that preserves quality by treating outlier features separately, including on very large checkpoints.

## Facts
- id: F1
  claim: 'LLM.int8() is an 8-bit matrix multiplication method for transformer feed-forward and attention projection layers.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC BY 4.0
- id: F2
  claim: 'The authors report cutting inference memory by half while retaining full precision performance for models up to 175B parameters.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC BY 4.0
- id: F3
  claim: 'They report that more than 99.9% of values are multiplied in 8-bit, with outlier dimensions kept in 16-bit.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC BY 4.0

## Sources
- [S1] arXiv 2208.07339, https://arxiv.org/abs/2208.07339. License: CC BY 4.0. v2 (10 Nov 2022). Accessed 2026-11-11.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).