← All notes

Summary

Chain-of-thought prompting writes steps before an answer, but one greedy decode can still pick a weak path. Three inference methods (described here only from their abstracts) change that: sample traces and pick a consistent answer, search a tree of thoughts, or have the same model critique and rewrite its output. None of the abstracts say you must retrain the base model.

Sample paths

Self-consistency replaces greedy decoding in chain-of-thought. Sample many reasoning traces, then take the most consistent answer after those traces are marginalized out. A hard problem often has several valid routes that should meet at one answer. The abstract reports gains on arithmetic and commonsense sets (GSM8K, SVAMP, AQuA, StrategyQA, ARC-challenge). Use it when extra samples are affordable and the final answer can be checked.

Search thoughts

Left-to-right token sampling fails when a task needs exploration, lookahead, or a careful first move. Tree of Thoughts (ToT) treats coherent text spans as intermediate "thoughts" you can branch on. The model considers several paths, scores them, and can look ahead or backtrack. The abstract tests Game of 24, Creative Writing, and Mini Crosswords; on Game of 24 it reports chain-of-thought at 4% and ToT at 74%. Use a tree when a linear trace cannot recover from an early mistake.

Refine with the same model

Models often miss their best output on the first try. Self-Refine generates, has the same model give feedback, revises, and repeats — one model as generator, critic, and editor, with no extra training or reinforcement learning. Across seven tasks (dialogue to math) with GPT-3.5, ChatGPT, and GPT-4, the abstract reports about 20% absolute average gain versus one-step generation, and that GPT-4 still improved at test time.

Choose sample-and-vote for a short checkable answer, a thought tree for planning or search, and self-feedback for a better draft. Prompt details live in the papers, which were not fetched.

Claims

• id: C1 claim: Self-consistency is a decode method that replaces greedy decoding in chain-of-thought prompting. source: S1 evidence: "we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting."
• id: C2 claim: It samples many reasoning paths and then picks the most consistent answer from those samples. source: S1 evidence: "It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths."
• id: C3 claim: Tree of Thoughts is an inference framework that generalizes chain-of-thought by searching over intermediate text thoughts. source: S2 evidence: "we introduce a new framework for language model inference, Tree of Thoughts (ToT)"
• id: C4 claim: ToT lets a model consider several reasoning paths, score choices, and look ahead or backtrack. source: S2 evidence: "ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action"
• id: C5 claim: Self-Refine improves a first draft by iterating feedback and revision from the same model. source: S3 evidence: "we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement."
• id: C6 claim: Self-Refine uses one model as generator, refiner, and feedback source and does not need extra training or reinforcement learning. source: S3 evidence: "Self-Refine does not require any supervised training data, additional training, or reinforcement learning"

@misc{iil_note_9kgbky6d,
  author = {{iinventedlight}},
  title = {{LLM reasoning: sample, search, and refine}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-03},
  url = {https://iinventedlight.com/library/notes/llm-reasoning-sampling-search-refine/},
  note = {Permanent ID iil:note:9kgbky6d. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt. This page in Markdown: /library/notes/llm-reasoning-sampling-search-refine.md.