← All notes

LIMA: Less Is More for Alignment

Summary

LIMA measures the relative importance of pretraining versus a small, carefully curated fine-tuning set. The authors fine-tune a 65B LLaMa model with the standard supervised loss on only 1,000 curated prompts and responses, without reinforcement learning or preference modeling. They report that the model picks up response formats from a handful of examples and generalizes to tasks not in the training set. In a controlled human study, LIMA responses were equivalent or strictly preferred to GPT-4 in 43% of cases, to Bard in 58%, and to DaVinci003 in 65%. The authors argue that most knowledge is learned in pretraining, and that only limited instruction data is needed to teach a high-quality output style. [S1]

Claims

Verification

When to apply

Use when asking how much curated supervised data is enough for instruction alignment.

Facts

Sources

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted). Paper is CC BY 4.0.

License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted). Source: Chunting Zhou, Pengfei Liu, Puxin Xu, and 12 coauthors, LIMA: Less Is More for Alignment (CC BY 4.0).

Cite this

Permanent ID: iil:note:87z8xj14

'iinventedlight (AI-assisted)'. 2026. "LIMA: Less Is More for Alignment." iinventedlight Library, iil:note:87z8xj14, version of 2026-11-12. https://iinventedlight.com/library/notes/2305-11206-lima/. License: CC-BY-4.0.

@misc{iil_note_87z8xj14,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{LIMA: Less Is More for Alignment}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-12},
  url = {https://iinventedlight.com/library/notes/2305-11206-lima/},
  note = {Permanent ID iil:note:87z8xj14. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2305-11206-lima.md.