← All notes

Answer: One controlled study found that letting LLM agents reflect on their wrong answers and write themselves guidance significantly improved re-answer accuracy (p < 0.001), and compared eight kinds of reflection.

The experiment

The authors gave nine popular LLMs a set of multiple-choice questions to set a baseline. For every question a model got wrong, eight types of self-reflecting agents were told to reflect on the mistake and write guidance. Each agent then re-answered the same questions using that guidance.

Findings

• Self-reflection significantly improved problem-solving performance.
• The authors compared reflection types to see how much each contributed.
• Code and data were released on GitHub.

Where reflection sits in agent design

A survey of LLM-based autonomous agents (S2) proposes a unified framework for building such agents and reviews how they are evaluated. Reflection is typically one stage in that loop, after acting and observing.

Cautions for operators

• The study re-answers the same questions the model already saw, so gains may partly reflect knowing which answers were wrong.
• Reflection adds extra model calls and tokens per task.
• Treat reflection as something to measure on your own tasks, not a guaranteed win.

Claims

• id: C1 claim: The study instructed nine popular LLMs to answer multiple-choice questions as a baseline. source: S1 evidence: "We instructed nine popular LLMs to answer a series of multiple-choice questions"
• id: C2 claim: Eight types of self-reflecting agents reflected on their mistakes. source: S1 evidence: "we instructed eight types of self-reflecting LLM agents to reflect on their mistakes"
• id: C3 claim: Self-reflection significantly improved problem-solving performance. source: S1 evidence: "LLM agents are able to significantly improve their problem-solving performance through self-reflection"
• id: C4 claim: The survey proposes a unified framework for constructing LLM-based autonomous agents. source: S2 evidence: "we propose a unified framework that encompasses a majority of the previous work"

Sources

• [S1] [2405.06682] Self-Reflection in LLM Agents: Effects on Problem-Solving Performance. arXiv. https://arxiv.org/abs/2405.06682 . License: CC BY 4.0. Accessed 2026-11-04.
• [S2] [2308.11432] A Survey on Large Language Model based Autonomous Agents. arXiv. https://arxiv.org/abs/2308.11432 . License: CC BY 4.0. Accessed 2026-11-04.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Summary by iinventedlight (AI-assisted).

@misc{iil_note_6vtsdqq9,
  author = {{iinventedlight}},
  title = {{Self-reflection in LLM agents: does it help?}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-04},
  url = {https://iinventedlight.com/library/notes/ao-reflection-self-reflection/},
  note = {Permanent ID iil:note:6vtsdqq9. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/ao-reflection-self-reflection.md.