---
title: 'GAIA: a benchmark for General AI Assistants'
license: CC-BY-4.0
license_url: https://creativecommons.org/licenses/by/4.0/
license_note: 'This note is CC BY 4.0. It links to the paper and does not reproduce it. Paper license: arXiv non-exclusive distribution (verified on arXiv abs page 2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/).'
source: https://iinventedlight.com/library/notes/2311-12983-gaia.md
source_license: 'arXiv default (non-exclusive distribution license)'
attribution: 'Pointer note by iinventedlight (AI-assisted). Links to: Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, Thomas Scialom, GAIA: a benchmark for General AI Assistants (arXiv). This note does not reproduce the paper.'
author: 'iinventedlight (AI-assisted)'
date: 2026-11-11
status: published
approved_by: "Cole Withington, 2026-11-11 20:58 PT, Grok Bot chat widget t1717s7407: \"Publish all 100 including the LLMs Encode Clinical Knowledge note\""
checked_on: 2026-11-11
as_of: 2026-11-11
confidence: medium
untested: 'Pointer note only. Abs page read 2026-11-11. The paper is not reproduced here.'
category: arxiv
topic_area: 'ai-research: evals'
tags: arxiv, pointer, evals, agents
arxiv_id: 2311.12983
alias: 2311.12983-gaia
format: pointer
type: pointer
sources:
  - id: S1
    title: 'GAIA: a benchmark for General AI Assistants'
    publisher: arXiv
    authors: 'Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, Thomas Scialom'
    url: https://arxiv.org/abs/2311.12983
    version_read: 'v1 (21 Nov 2023)'
    accessed: 2026-11-11
    license: 'arXiv default (non-exclusive distribution license)'
---
# GAIA: a benchmark for General AI Assistants

## Summary
GAIA: a benchmark for General AI Assistants (2023), by Gregoire Mialon, Clementine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom, is at https://arxiv.org/abs/2311.12983. The authors present questions that are conceptually simple for people but hard for advanced models, requiring reasoning, multi-modality handling, web browsing, and tool use. They report that human respondents obtain 92% versus 15% for GPT-4 equipped with plugins. They devise 466 questions and their answers, and they posit that a system should exhibit similar robustness as the average human does on such questions. They release the questions while holding back answers to 300 of them to power a leaderboard at https://huggingface.co/gaia-benchmark. They show that human respondents obtain 92% versus 15% for GPT 4 equipped with plugins. They release their questions while retaining answers to 300 of them to power a leader-board available at https://huggingface.co/gaia-benchmark. Title, authors, abstract, and comments were taken from the arXiv abs page. The version is v1 (21 Nov 2023), submitted 21 Nov 2023.

## Claims
Pointer notes do not quote the paper. See Verification and Sources for the link and license.

## Verification
- Format: pointer note only. The paper is not reproduced here.
- Why it matters: It matters because it became a standard test of general assistants that must use tools, browse, and handle realistic multi-step questions, rather than only professional exams.
- Reproduced: not reproduced (2026-11-11).
- Related work: A related note already in the library is [Evaluating multimodal agents](https://iinventedlight.com/library/notes/ao-evals-multimodal-agents/).
- Limits: This is a pointer note only. The 92% versus 15% comparison is the authors' reported human versus GPT-4 with plugins result from the 21 Nov 2023 version.
- License: arXiv default (non-exclusive distribution license) on the abs page (2026-11-11; http://arxiv.org/licenses/nonexclusive-distrib/1.0/). Submitted 21 Nov 2023 (v1). Sources: [S1] https://arxiv.org/abs/2311.12983, v1 (21 Nov 2023). checked_on: 2026-11-11. Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it. The note itself is CC BY 4.0.

## When to apply
Use when you need the original GAIA paper for general assistant evaluation involving tools, browsing, and multi-step questions.

## Facts
- id: F1
  claim: 'GAIA is a benchmark of 466 real-world questions that require reasoning, multi-modality handling, web browsing, and tool use.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC-BY-4.0
- id: F2
  claim: 'The authors report that human respondents obtain 92% versus 15% for GPT-4 equipped with plugins.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC-BY-4.0
- id: F3
  claim: 'They release questions while retaining answers to 300 of them for a leaderboard at https://huggingface.co/gaia-benchmark.'
  confidence: medium
  date: 2026-11-11
  sources: [S1]
  license: CC-BY-4.0

## Sources
- [S1] arXiv 2311.12983, https://arxiv.org/abs/2311.12983. Paper license: arXiv default (non-exclusive distribution license). v1 (21 Nov 2023). Accessed 2026-11-11. This note links to the paper and does not reproduce it.

License: this note is CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). Pointer by iinventedlight (AI-assisted). Paper license: arXiv non-exclusive distribution; this note links to it and does not reproduce it.

---

Agents: [/agents.md](/agents.md) | [/.well-known/agent-library.json](/.well-known/agent-library.json).