← All notes

Summary

Large Language Models are Zero-Shot Reasoners (2022), by Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa, is at https://arxiv.org/abs/2205.11916. The paper shows that LLMs are decent zero-shot reasoners by adding Let's think step by step before each answer. The authors evaluate a single template, which they call Zero-shot-CoT, on arithmetic tasks such as MultiArith, GSM8K, AQUA-RAT, and SVAMP, on symbolic tasks such as Last Letter and Coin Flip, and on other logical tasks such as Date Understanding and Tracking Shuffled Objects. With the large InstructGPT model text-davinci-002, they report MultiArith accuracy moving from 17.7% to 78.7% and GSM8K from 10.4% to 40.7%, and they report similar gains with a 540B parameter PaLM model. Using the same single prompt template, they report that Zero-shot-CoT significantly outperforms zero-shot LLM performances on diverse benchmark reasoning tasks including arithmetics and symbolic reasoning, without any hand-crafted few-shot examples. They report increasing the accuracy on MultiArith from 17.7% to 78.7% and GSM8K from 10.4% to 40.7% with a large InstructGPT model. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v4 (29 Jan 2023), submitted 24 May 2022.

Claims

Pointer notes do not quote the paper. See Verification and Sources for the link and license.

Verification

• Format: pointer note only. The paper is not reproduced here.
• Why it matters: It matters because it showed that a short, generic cue can elicit multi-step reasoning without hand-written few-shot examples.
• Reproduced: not reproduced (2026-11-11).
• Related work: Related notes already in the library include Chain-of-Thought Prompting and Plan-and-Solve Prompting.
• Limits: This is a pointer note only. The MultiArith and GSM8K figures are the authors' reported results with text-davinci-002 in the v4 paper (29 Jan 2023).
• License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 24 May 2022 (v1); last revised 29 Jan 2023 (v4). Sources: [S1] https://arxiv.org/abs/2205.11916, v4 (29 Jan 2023). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

When to apply

Use when you need the original zero-shot chain-of-thought paper that studies the Let's think step by step cue.

Facts

• id: F1 claim: 'The paper shows that adding a Let''s think step by step cue before the answer can elicit zero-shot chain-of-thought reasoning.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F2 claim: 'With text-davinci-002, the authors report MultiArith accuracy moving from 17.7% to 78.7% and GSM8K from 10.4% to 40.7%.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F3 claim: 'They report similar magnitudes of improvement with a 540B parameter PaLM model.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0

Sources

• [S1] arXiv 2205.11916, https://arxiv.org/abs/2205.11916. Paper license: arXiv default (non-exclusive distribution license). v4 (29 Jan 2023). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

@misc{iil_note_89jy3m8v,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{Large Language Models are Zero-Shot Reasoners}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2205-11916-zero-shot-reasoners/},
  note = {Permanent ID iil:note:89jy3m8v. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2205-11916-zero-shot-reasoners.md.