Teaching Large Language Models to Self-Debug

Summary by iinventedlight (AI-assisted), condensed and paraphrased; sources: Xinyun Chen, Maxwell Lin, Nathanael Schärli, Denny Zhou, "Teaching Large Language Models to Self-Debug", arXiv:2304.05128 (CC BY 4.0)

Teaching Large Language Models to Self-Debug

Summary

Self-Debugging teaches an LLM, through few-shot examples, to debug the code it generated. The model looks at execution results and explains its code in plain language, a form of 'rubber duck debugging'. This works even without human feedback on whether the code is correct. It improved results on Spider (text-to-SQL), TransCoder (C++ to Python) and MBPP (text to Python), by up to 12% where unit tests were available.

Draft. License: CC BY 4.0 (this summary). Sources: [S1] arXiv, "Teaching Large Language Models to Self-Debug" (CC BY 4.0).

When to apply

Code-generation agents that can run code: feed execution results back in and ask for an explanation before the fix.

Paper

Title: Teaching Large Language Models to Self-Debug
Authors: Xinyun Chen, Maxwell Lin, Nathanael Schärli, Denny Zhou
arXiv: https://arxiv.org/abs/2304.05128 (first posted 2023-04-11)
License: CC BY 4.0 (checked on the arXiv abstract page 2026-11-02)
Read for this note: arXiv abstract page; figures restate the abstract.

Facts

id: F1 claim: Self-Debugging teaches an LLM via few-shot demonstrations to debug its own predicted programs. confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02
id: F2 claim: The model identifies mistakes by examining execution results and explaining its code in natural language, without human feedback on correctness. confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02
id: F3 claim: On Spider (no unit tests), code explanation improved the baseline by 2-3%, and by 9% on the hardest problems (author-reported). confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02
id: F4 claim: On TransCoder and MBPP, where unit tests are available, accuracy improved by up to 12% (author-reported). confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02
id: F5 claim: By reusing failed predictions and feedback, it can match or beat baselines that generate more than 10x as many candidate programs (author-reported). confidence: medium date: 2026-11-02 sources: [S1] license: CC BY 4.0 checked_on: 2026-11-02

Caveats

Contrast with 2310.01798: the gains here come with an external signal (execution results).

Pointers

id: P1 answers: Full method, experiments and appendices where: https://arxiv.org/abs/2304.05128 access: free, public web page

Source: https://arxiv.org/abs/2304.05128 · License: CC-BY-4.0 · Summary by iinventedlight (AI-assisted), condensed and paraphrased; sources: Xinyun Chen, Maxwell Lin, Nathanael Schärli, Denny Zhou, \"Teaching Large Language Models to Self-Debug\", arXiv:2304.05128 (CC BY 4.0)

Cite this

Permanent ID: iil:note:sbk84hqs

iinventedlight Library. 2026. "Teaching Large Language Models to Self-Debug." iinventedlight Library, iil:note:sbk84hqs, version of 2026-11-02. https://iinventedlight.com/library/reference/arxiv/2304-05128-self-debugging/. License: CC-BY-4.0. Summary of https://arxiv.org/abs/2304.05128.

@misc{iil_note_sbk84hqs,
  author = {{iinventedlight Library}},
  title = {{Teaching Large Language Models to Self-Debug}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-02},
  url = {https://iinventedlight.com/library/reference/arxiv/2304-05128-self-debugging/},
  note = {Permanent ID iil:note:sbk84hqs. License: CC-BY-4.0. Summary of https://arxiv.org/abs/2304.05128.},
}

For language models and agents: start at /llms.txt. This page in Markdown: /library/reference/arxiv/2304-05128-self-debugging.md.