---
title: "Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is arXiv non-exclusive distribution license (verified on arXiv abs page 2026-11-12); this pointer note is our own words and is CC BY 4.0; we link to the paper rather than reproduce it'
source: https://iinventedlight.com/library/notes/2403-09629-quiet-star.md
source_license: "arXiv non-exclusive distribution license"
attribution: 'Pointer by iinventedlight (AI-assisted). Source: Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman, Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: 'Pointer note only. Paper not reproduced. Abs page and license checked 2026-11-12.'
category: arxiv
topic_area: 'ai-research: reasoning'
tags: arxiv, reasoning, quiet-star, rationales, pretraining
arxiv_id: 2403.09629
alias: 2403-09629-quiet-star
type: pointer
format: pointer
sources:
  - id: S1
    title: "Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking"
    publisher: arXiv
    authors: "Eric Zelikman, Georges Harik, Yijia Shao, Varuna Jayasiri, Nick Haber, Noah D. Goodman"
    url: https://arxiv.org/abs/2403.09629
    version_read: "v2 (18 Mar 2024)"
    accessed: 2026-11-12
    license: "arXiv non-exclusive distribution license"
---
# Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking

## Summary
[Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking](https://arxiv.org/abs/2403.09629) (2024) generalizes the earlier Self-Taught Reasoner idea. Instead of learning rationales only on question-answering items that have a correct answer, the model learns to generate internal thoughts at each token to help predict later text, including on ordinary internet text.

What it introduced: a tokenwise parallel sampling method, learnable start and end thought tokens, and an extended teacher-forcing setup so the model can practice those internal thoughts during continued pretraining. After that training, the authors report zero-shot gains on GSM8K and CommonsenseQA with no task-specific fine-tuning, and a perplexity improvement on harder tokens in natural text.

What it led to: later work on learned thinking tokens and on reasoning that is trained into the model rather than only prompted at inference. Live notes: [2201-11903-chain-of-thought-prompting](https://iinventedlight.com/library/notes/2201-11903-chain-of-thought-prompting/) and [2110-14168-gsm8k](https://iinventedlight.com/library/notes/2110-14168-gsm8k/).

Where to read it: https://arxiv.org/abs/2403.09629 (v2, last revised 18 Mar 2024).

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
