---
title: "LIMA: Less Is More for Alignment"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is CC BY 4.0 (verified on arXiv abs page 2026-11-12; icon by-4.0.png, http://creativecommons.org/licenses/by/4.0/); note is our own summary'
source: https://iinventedlight.com/library/notes/2305-11206-lima.md
source_license: 'CC BY 4.0'
attribution: 'Summary by iinventedlight (AI-assisted). Source: Chunting Zhou, Pengfei Liu, Puxin Xu, and 12 coauthors, LIMA: Less Is More for Alignment (CC BY 4.0).'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: "Not reproduced. Abs page and HTML key sections read 2026-11-12. Code not run."
category: arxiv
topic_area: 'ai-research: alignment-and-safety'
tags: arxiv, alignment, instruction-tuning, lima, llama
arxiv_id: 2305.11206
alias: 2305-11206-lima
sources:
  - id: S1
    title: "LIMA: Less Is More for Alignment"
    publisher: arXiv
    authors: "Chunting Zhou and 14 coauthors (listed on the arXiv abs page)"
    url: https://arxiv.org/abs/2305.11206
    version_read: "v1 (18 May 2023)"
    accessed: 2026-11-12
    license: 'CC BY 4.0'
---
# LIMA: Less Is More for Alignment

## Summary
LIMA measures the relative importance of pretraining versus a small, carefully curated fine-tuning set. The authors fine-tune a 65B LLaMa model with the standard supervised loss on only 1,000 curated prompts and responses, without reinforcement learning or preference modeling. They report that the model picks up response formats from a handful of examples and generalizes to tasks not in the training set. In a controlled human study, LIMA responses were equivalent or strictly preferred to GPT-4 in 43% of cases, to Bard in 58%, and to DaVinci003 in 65%. The authors argue that most knowledge is learned in pretraining, and that only limited instruction data is needed to teach a high-quality output style. [S1]

## Claims
- id: C1
  claim: "LIMA is a 65B LLaMa model fine-tuned on only 1,000 curated prompts and responses, without RL or preference modeling."
  source: S1
  evidence: "fine-tuned with the standard supervised loss on only 1,000 carefully curated prompts and responses"
- id: C2
  claim: "In a human study, LIMA was equivalent or strictly preferred to GPT-4 in 43% of cases."
  source: S1
  evidence: "equivalent or strictly preferred to GPT-4 in 43% of cases"
- id: C3
  claim: "The authors argue that almost all knowledge is learned in pretraining and that only limited instruction data is needed."
  source: S1
  evidence: "almost all knowledge in large language models is learned during pretraining"

## Verification
- Code available: No official code or data URL is named on the abs page. Not run.
- Reproduced: not reproduced (2026-11-12). Code not run.
- Related work: See [2203-02155-instructgpt](https://iinventedlight.com/library/notes/2203-02155-instructgpt/), [2212-10560-self-instruct](https://iinventedlight.com/library/notes/2212-10560-self-instruct/), and [2305-18290-direct-preference-optimization](https://iinventedlight.com/library/notes/2305-18290-direct-preference-optimization/).
- Limits: The 43% figure is a human preference rate against GPT-4, not a claim that LIMA matches GPT-4 overall.
- License: CC BY 4.0 on the abs page (2026-11-12; icon by-4.0.png; http://creativecommons.org/licenses/by/4.0/). Submitted 18 May 2023 (v1). Sources: [S1] https://arxiv.org/abs/2305.11206, v1 (18 May 2023). checked_on: 2026-11-12.

## When to apply
Use when asking how much curated supervised data is enough for instruction alignment.

## Facts
- id: F1
  claim: "LIMA fine-tunes 65B LLaMa on 1,000 curated examples with supervised loss only."
  confidence: medium
  date: 2026-11-12
  sources: [S1]
  license: CC BY 4.0
- id: F2
  claim: "Human raters preferred or tied LIMA with GPT-4 in 43% of cases, Bard in 58%, and DaVinci003 in 65%."
  confidence: medium
  date: 2026-11-12
  sources: [S1]
  license: CC BY 4.0
- id: F3
  claim: "The paper claims limited instruction data can teach output style after pretraining."
  confidence: medium
  date: 2026-11-12
  sources: [S1]
  license: CC BY 4.0

## Sources
- [S1] arXiv 2305.11206, https://arxiv.org/abs/2305.11206. License: CC BY 4.0. v1 (18 May 2023). Accessed 2026-11-12.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
