← All notes

Summary

WebGPT: Browser-assisted question-answering with human feedback (2021), by Reiichiro Nakano, Jacob Hilton, Suchir Balaji, and coauthors, is at https://arxiv.org/abs/2112.09332. The authors fine-tune GPT-3 in a text-based browsing environment so the model can search and navigate the web. They set the task up so people can also perform it, first training by imitation and then optimizing answer quality with human feedback. To make factual evaluation easier, models must collect references while browsing. Training and evaluation use ELI5, a set of questions asked by Reddit users. The best model, obtained by behavior cloning and then rejection sampling against a reward model of human preferences, is reported to have answers preferred by humans 56% of the time over those of human demonstrators, and 69% of the time over the highest-voted Reddit answer. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v3 (1 Jun 2022), submitted 17 Dec 2021.

Claims

Pointer notes do not quote the paper. See Verification and Sources for the link and license.

Verification

• Format: pointer note only. The paper is not reproduced here.
• Why it matters: It matters as an early system that trained a language model to answer questions with a web browser and human feedback, and to attach references to those answers.
• Reproduced: not reproduced (2026-11-11).
• Related work: Related notes already in the library include RAG basics and Gorilla.
• Limits: This is a pointer note only. Preference rates are the authors' reported human evaluations from the v3 paper (1 Jun 2022).
• License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 17 Dec 2021 (v1); last revised 1 Jun 2022 (v3). Sources: [S1] https://arxiv.org/abs/2112.09332, v3 (1 Jun 2022). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

When to apply

Use when tracing early browser-assisted question answering and the use of collected references plus human feedback.

Facts

• id: F1 claim: 'WebGPT fine-tunes GPT-3 to answer long-form questions in a text-based web-browsing environment, and models must collect references while browsing.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F2 claim: 'The authors train and evaluate on ELI5, a dataset of questions asked by Reddit users.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0
• id: F3 claim: 'They report that their best model''s answers are preferred by humans 56% of the time to those of human demonstrators, and 69% of the time to the highest-voted Reddit answer.' confidence: medium date: 2026-11-11 sources: [S1] license: CC-BY-4.0

Sources

• [S1] arXiv 2112.09332, https://arxiv.org/abs/2112.09332. Paper license: arXiv default (non-exclusive distribution license). v3 (1 Jun 2022). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

@misc{iil_note_78w52110,
  author = {{'iinventedlight (AI-assisted)'}},
  title = {{WebGPT: Browser-assisted question-answering with human feedback}},
  howpublished = {iinventedlight Library},
  year = {2026},
  date = {2026-11-11},
  url = {https://iinventedlight.com/library/notes/2112-09332-webgpt/},
  note = {Permanent ID iil:note:78w52110. License: CC-BY-4.0.},
}

Markdown

Spotted a mistake? Use the MCP feedback tool or email hello@iinventedlight.com.

For language models and agents: start at /llms.txt | /agents.md | /.well-known/agent-library.json. This page in Markdown: /library/notes/2112-09332-webgpt.md.