---
title: "BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents"
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'paper is arXiv non-exclusive distribution license (verified on arXiv abs page 2026-11-12); this pointer note is our own words and is CC BY 4.0; we link to the paper rather than reproduce it'
source: https://iinventedlight.com/library/notes/2504-12516-browsecomp.md
source_license: "arXiv non-exclusive distribution license"
attribution: 'Pointer by iinventedlight (AI-assisted). Source: Jason Wei, Zhiqing Sun, Spencer Papay, and 7 coauthors, BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents (arXiv non-exclusive distribution license). This note links to the paper rather than reproducing it.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-12
status: published
approved_by: "Cole Withington, 2026-11-12 14:44 PT, Librarian chat widget t51s316: \"Yes, publish all 100 second-list arXiv notes\""
checked_on: 2026-11-12
as_of: 2026-11-12
confidence: medium
untested: 'Pointer note only. Paper not reproduced. Abs page and license checked 2026-11-12.'
category: arxiv
topic_area: 'ai-research: evals'
tags: arxiv, evals, browsing-agents, web-agents, browsecomp
arxiv_id: 2504.12516
alias: 2504-12516-browsecomp
type: pointer
format: pointer
sources:
  - id: S1
    title: "BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents"
    publisher: arXiv
    authors: "Jason Wei and 9 coauthors (listed on the arXiv abs page)"
    url: https://arxiv.org/abs/2504.12516
    version_read: "v1 (16 Apr 2025)"
    accessed: 2026-11-12
    license: "arXiv non-exclusive distribution license"
---
# BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents

## Summary
[BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agents](https://arxiv.org/abs/2504.12516) (2025) is a short, hard test of whether an agent can persist on the live web to find facts that are difficult to locate. It has 1,266 questions whose answers are short and easy to check, even though getting to those answers may take a long, entangled search.

What it introduced: a browsing analogue of a programming contest: incomplete as a picture of real user queries, but useful as a core test of persistence and creativity in finding information. The benchmark deliberately skips long-form answers and ambiguous questions in order to keep grading simple.

What it led to: a compact eval used alongside other web-agent suites when people ask whether a browsing agent can actually find buried facts. Live notes include [2307-13854-webarena](https://iinventedlight.com/library/notes/2307-13854-webarena/). Related pointer notes in this batch cover Mind2Web, VisualWebArena, and WebVoyager. SimpleQA, a short-form factuality set from the same group, is also in this batch.

Where to read it: https://arxiv.org/abs/2504.12516 (v1, 16 Apr 2025). The authors point to https://github.com/openai/simple-evals.

The paper is posted on arXiv under the arXiv non-exclusive distribution license. This note links to it rather than reproducing it. The note itself is CC BY 4.0.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).
