← All notes

Short answer

Step-back prompting first asks the model a more general question: what concept or principle applies here? Only then does it answer the specific one. Grounding the reasoning in that abstraction improved accuracy on science, knowledge and multi-hop questions. [S1]

How to use it

1. Given a detailed question, prompt the model to state the high-level concept or first principle behind it (the step-back question). [S1]
2. Have the model answer that general question.
3. Feed the principle back in and ask the original question, reasoning from it. [S1]

Reported results

• Tested on PaLM-2L, GPT-4 and Llama2-70B. [S1]
• PaLM-2L gained 7% on MMLU Physics, 11% on MMLU Chemistry, 27% on TimeQA and 7% on MuSiQue. [S1]

When to try it

Use it on questions buried in specifics, where the right path depends on recognizing the underlying rule. Compare it with chain-of-thought on your own tasks before adopting it.

Claims

• id: C1 claim: Step-Back Prompting has LLMs derive high-level concepts and first principles from detailed instances. source: S1 evidence: "derive high-level concepts and first principles from instances"
• id: C2 claim: Experiments used PaLM-2L, GPT-4 and Llama2-70B. source: S1 evidence: "PaLM-2L, GPT-4 and Llama2-70B models"
• id: C3 claim: Step-Back Prompting improved PaLM-2L on TimeQA by 27%. source: S1 evidence: "TimeQA by 27%"

Sources

• [S1] Take a Step Back: Evoking Reasoning via Abstraction in Large Language Models. arXiv. https://arxiv.org/abs/2310.06117 . License: CC BY 4.0. Accessed 2026-11-05.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_8yfy1wr9,
  author = {{iinventedlight}},
  title = {{Take a Step Back: Step-Back Prompting}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/2310-06117-step-back-prompting/},
  note = {Permanent ID iil:note:8yfy1wr9. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/2310-06117-step-back-prompting.md.