---
title: "LLM reasoning: sample, search, and refine"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
source: https://iinventedlight.com/library/notes/llm-reasoning-sampling-search-refine.md
source_license: "CC BY 4.0; restated in our own words"
attribution: "Summary by iinventedlight (AI-assisted). Sources: arXiv, Self-Consistency Improves Chain of Thought Reasoning in Language Models, CC BY 4.0; arXiv, Tree of Thoughts: Deliberate Problem Solving with Large Language Models, CC BY 4.0; arXiv, Self-Refine: Iterative Refinement with Self-Feedback, CC BY 4.0."
author: "iinventedlight (synthesis of cited sources); text drafted with AI assistance"
date: 2026-11-03
status: published
checked_on: 2026-11-03
as_of: 2026-11-03
confidence: low
untested: "Only the arXiv abstract pages were fetched, not the PDFs or experiments; reported scores were not reproduced; no prompts were run."
category: coverage-gaps
topic_area: "ai-research: self-consistency, tree of thoughts, self-refine"
tags: llm, reasoning, prompting, self-consistency, tree-of-thoughts, self-refine
sources:
  - id: S1
    title: "Self-Consistency Improves Chain of Thought Reasoning in Language Models"
    publisher: arXiv
    url: https://arxiv.org/abs/2203.11171
    accessed: 2026-11-03
    license: "CC BY 4.0"
  - id: S2
    title: "Tree of Thoughts: Deliberate Problem Solving with Large Language Models"
    publisher: arXiv
    url: https://arxiv.org/abs/2305.10601
    accessed: 2026-11-03
    license: "CC BY 4.0"
  - id: S3
    title: "Self-Refine: Iterative Refinement with Self-Feedback"
    publisher: arXiv
    url: https://arxiv.org/abs/2303.17651
    accessed: 2026-11-03
    license: "CC BY 4.0"
---
# LLM reasoning: sample, search, and refine

## Summary
Chain-of-thought prompting writes steps before an answer, but one greedy decode can still pick a weak path. Three inference methods (described here only from their abstracts) change that: sample traces and pick a consistent answer, search a tree of thoughts, or have the same model critique and rewrite its output. None of the abstracts say you must retrain the base model.

## Sample paths
Self-consistency replaces greedy decoding in chain-of-thought. Sample many reasoning traces, then take the most consistent answer after those traces are marginalized out. A hard problem often has several valid routes that should meet at one answer. The abstract reports gains on arithmetic and commonsense sets (GSM8K, SVAMP, AQuA, StrategyQA, ARC-challenge). Use it when extra samples are affordable and the final answer can be checked.

## Search thoughts
Left-to-right token sampling fails when a task needs exploration, lookahead, or a careful first move. Tree of Thoughts (ToT) treats coherent text spans as intermediate "thoughts" you can branch on. The model considers several paths, scores them, and can look ahead or backtrack. The abstract tests Game of 24, Creative Writing, and Mini Crosswords; on Game of 24 it reports chain-of-thought at 4% and ToT at 74%. Use a tree when a linear trace cannot recover from an early mistake.

## Refine with the same model
Models often miss their best output on the first try. Self-Refine generates, has the same model give feedback, revises, and repeats — one model as generator, critic, and editor, with no extra training or reinforcement learning. Across seven tasks (dialogue to math) with GPT-3.5, ChatGPT, and GPT-4, the abstract reports about 20% absolute average gain versus one-step generation, and that GPT-4 still improved at test time.

Choose sample-and-vote for a short checkable answer, a thought tree for planning or search, and self-feedback for a better draft. Prompt details live in the papers, which were not fetched.

## Claims
- id: C1
  claim: Self-consistency is a decode method that replaces greedy decoding in chain-of-thought prompting.
  source: S1
  evidence: "we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting."
- id: C2
  claim: It samples many reasoning paths and then picks the most consistent answer from those samples.
  source: S1
  evidence: "It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths."
- id: C3
  claim: Tree of Thoughts is an inference framework that generalizes chain-of-thought by searching over intermediate text thoughts.
  source: S2
  evidence: "we introduce a new framework for language model inference, Tree of Thoughts (ToT)"
- id: C4
  claim: ToT lets a model consider several reasoning paths, score choices, and look ahead or backtrack.
  source: S2
  evidence: "ToT allows LMs to perform deliberate decision making by considering multiple different reasoning paths and self-evaluating choices to decide the next course of action"
- id: C5
  claim: Self-Refine improves a first draft by iterating feedback and revision from the same model.
  source: S3
  evidence: "we introduce Self-Refine, an approach for improving initial outputs from LLMs through iterative feedback and refinement."
- id: C6
  claim: Self-Refine uses one model as generator, refiner, and feedback source and does not need extra training or reinforcement learning.
  source: S3
  evidence: "Self-Refine does not require any supervised training data, additional training, or reinforcement learning"