---
title: "QLoRA: Efficient Finetuning of Quantized LLMs"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source: https://arxiv.org/abs/2305.14314
source_license: "CC BY 4.0"
attribution: "Summary by iinventedlight (AI-assisted). Source: arXiv, \"QLoRA: Efficient Finetuning of Quantized LLMs\" (CC BY 4.0)."
condensed_by: "iinventedlight (Grok Bot, AI-assisted)"
fact_checked: 2026-11-02
date: 2026-11-02
status: reviewed
reviewed_on: 2026-11-02
category: arxiv
arxiv_id: 2305-14314-qlora
alias: 2305-14314-qlora
checked_on: 2026-11-02
license_url: https://creativecommons.org/licenses/by/4.0/
source: https://arxiv.org/abs/2305.14314
source_license: "CC BY 4.0 (arXiv paper)"
arxiv_license_url: https://creativecommons.org/licenses/by/4.0/
arxiv_license_checked: 2026-11-02
arxiv_license_check_note: "License link read from the paper\'s own arXiv abstract page on 2026-11-02."
summary_policy: "Short original summary with attribution; no passages copied, at most brief phrases quoted."
attribution: "Summary by iinventedlight (AI-assisted), condensed and paraphrased; sources: Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer, \"QLoRA: Efficient Finetuning of Quantized LLMs\", arXiv:2305.14314 (CC BY 4.0)"
condensed_by: "iinventedlight (Grok Bot, AI-assisted)"
fact_checked: 2026-11-02
date: 2026-11-02
category: arxiv
tags: arxiv, llm, paper-summary
checked_on: 2026-11-02
sources:
  - id: S1
    title: "QLoRA: Efficient Finetuning of Quantized LLMs"
    authors: "Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer"
    publisher: "arXiv"
    url: https://arxiv.org/abs/2305.14314
    accessed: 2026-11-02
    license: "CC BY 4.0"
    attribution: "Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer, \"QLoRA: Efficient Finetuning of Quantized LLMs\", arXiv:2305.14314 (CC BY 4.0)"
---
# QLoRA: Efficient Finetuning of Quantized LLMs

# QLoRA: Efficient Finetuning of Quantized LLMs

## Summary
QLoRA fine-tunes a frozen, 4-bit-quantized model by training Low-Rank Adapters (LoRA) on top of it. This cut memory enough to fine-tune a 65B model on a single 48GB GPU while keeping 16-bit fine-tuning performance. It adds three ideas: the NF4 data type, double quantization and paged optimizers. The authors' Guanaco models reached 99.3% of ChatGPT's level on the Vicuna benchmark.

*Draft. License: CC BY 4.0 (this summary). Sources: [S1] arXiv, "QLoRA: Efficient Finetuning of Quantized LLMs" (CC BY 4.0).*

## When to apply
- Fine-tuning large open-weight models on limited GPU memory.

## Paper
- Title: QLoRA: Efficient Finetuning of Quantized LLMs
- Authors: Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer
- arXiv: https://arxiv.org/abs/2305.14314 (first posted 2023-05-23)
- License: CC BY 4.0 (checked on the arXiv abstract page 2026-11-02)
- Read for this note: arXiv abstract page; figures restate the abstract.

## Facts
- id: F1
  claim: QLoRA backpropagates through a frozen 4-bit quantized model into Low-Rank Adapters (LoRA).
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02
- id: F2
  claim: It allowed fine-tuning a 65B-parameter model on a single 48GB GPU while preserving full 16-bit fine-tuning performance (author-reported).
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02
- id: F3
  claim: Its innovations: 4-bit NormalFloat (NF4), double quantization of the quantization constants, and paged optimizers for memory spikes.
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02
- id: F4
  claim: The Guanaco model family reached 99.3% of ChatGPT's performance on the Vicuna benchmark after 24 hours of fine-tuning on one GPU (author-reported).
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02
- id: F5
  claim: The authors fine-tuned over 1,000 models and found GPT-4 evaluations a cheap, reasonable alternative to human evaluation, while warning that chatbot benchmarks were not trustworthy.
  confidence: medium
  date: 2026-11-02
  sources: [S1]
  license: CC BY 4.0
  checked_on: 2026-11-02

## Caveats
- The authors themselves caution about the chatbot benchmarks used for the headline comparison.

## Pointers
- id: P1
  answers: Full method, experiments and appendices
  where: https://arxiv.org/abs/2305.14314
  access: free, public web page